Compare commits

..

9 Commits

Author SHA1 Message Date
admin 861d32a4b4 CHANGELOG: unreleased — R-349 agent_sha256, R-25 agent half (burn-down night)
gates / gates (push) Successful in 45s
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
2026-10-05 21:16:51 +02:00
admin 769c4c3cf2 R-25 (agent half): the format answer carries the NEW filesystem's UUID, bound to the durable id
After mkfs the agent re-resolves the bound durable id, requires it to name the
device it just formatted, reads the superblock back (blkid -p, requested fstype)
and returns fs_uuid in POST /disks/format and GET /disks/format/status. Anything
unverified returns "" — never a path-resolved guess. The controller half
(mount fs_uuid instead of re-resolving the UUID from the /dev path) is owed
in felhom-controller.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
2026-10-05 21:16:11 +02:00
admin 64f704d0f7 R-349: the host report carries the sha256 of the RUNNING agent binary
A hand-built proof binary and the published artifact share a version string
but not their bytes, so no version check could see the divergence. The agent
now reports agent_sha256 (hash of /proc/self/exe, once per process; empty =
unknown) beside agent_version, the same mechanism as host.wrapper_sha256.
The hub half (compare against the vouched agent_sha256, surface drift) is
owed in the hub repo.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
2026-10-05 21:16:11 +02:00
admin 208fac8027 CHANGELOG/REPORT: v0.147.0 released (shas)
gates / gates (push) Successful in 56s
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
2026-10-05 18:58:37 +02:00
admin f1b9b41214 R-124 recipe root namespace as PBS spells it; R-118 no root size for an absent drive; R-269 rotated-out token rejected at once; R-317 dnsmasq install probed by its unit (burn-down round 2)
gates / gates (push) Successful in 47s
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
2026-10-05 18:56:13 +02:00
admin d83316326e R-291 retention record source, R-348 restart comment (no binary change; burn-down)
gates / gates (push) Successful in 22s
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
2026-10-05 16:56:18 +02:00
admin e06ed97fa8 agent v0.146.1 REPORT
gates / gates (push) Successful in 22s
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
2026-10-05 13:00:22 +02:00
admin e4b5cf9693 R-880: build-step-bundle.py — the transition bundle for a release whose bundle adds paths (an installed felhom-os-apply refuses unknown paths, R16)
gates / gates (push) Successful in 21s
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
2026-10-05 12:23:55 +02:00
admin faa3cad92e agent v0.146.1 CHANGELOG (released)
gates / gates (push) Successful in 19s
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
2026-10-05 12:14:07 +02:00
24 changed files with 809 additions and 68 deletions
+74
View File
@@ -1,3 +1,77 @@
## unreleased
- **R-349:** the host report carries `agent_sha256`, the sha256 of the running agent binary (read once from
`/proc/self/exe`; empty = unknown), so a hand-built binary under the vouched version name becomes visible. The hub
comparison is a separate hub change. Test `TestCollect_AgentSHA256IsTheRunningBinary`; red-proved.
- **R-25 (agent half):** `POST /disks/format` and `GET /disks/format/status` return `fs_uuid`, the new filesystem's UUID
read back after mkfs only when the bound durable id still resolves to the formatted device and the superblock is the
requested type (empty = not verified); `DeviceProbe` gains `FSUUID` from blkid. Tests `TestFormat_*FSUUID*`; three
red-proofs. The controller half (mount that UUID) is a controller change.
## v0.147.0 — the recovery recipe spells the root namespace the way PBS does; a removed drive no longer shows the root disk's size; a rotated-out token stops at once; the dnsmasq check looks at the right package (burn-down round 2: R-124, R-118, R-269, R-317) (2026-10-05)
Released by `scripts/release-agent.sh`: binary sha256 `642c4d196c48671c14ff653118303c5903af1670b7abb550edeaf73e701cd5b8`,
config bundle sha256 `326527d0993c9a62df2f790c7700ca645cedbf0673dcfb6dc1768d8610b8007d` (tag `v0.147.0` = `f1b9b41`).
Delivery order as for v0.146.1: signed `agent_update`, then signed `agent_config_update`.
MinAgent impact: none (the controller needs nothing new from this agent). Config bundle content unchanged from v0.146.1.
- **R-124 (operator ruling 2026-10-05: fix it):** the DR recipe's `pbs.namespace` for a box in PBS's ROOT namespace is
now `""` — PBS's own spelling — beside `namespace_state: resolved`; it used to be the word `root`, which no namespace
is named, so `--ns root` failed in a recovery. `hub.PBSRootNamespace`; `TestR124_RootNamespaceOnTheWireIsPBSSpelling`
(red-proof: back to "root" → FAIL). Runbook: `felhom.eu runbooks/ep0-datastore-copy.md` step 2 says how to read it
(and to treat a recorded `root` from older agents as empty). The hub stores the recipe raw; its fixture follows.
- **R-118:** the local API's drive list reads a drive's capacity only while its DEVICE is present — with the device gone
the bare mountpoint is a directory on the root filesystem, whose size was reported as the drive's.
`TestDisks_UnionPath_AbsentDeviceReportsNoRootCapacity` (red-proof convicts).
- **R-269:** the token store re-reads its shared file whenever it has grown, BEFORE answering — so a token rotated out by
another process stops authorizing on its next use (it used to keep working until an unrelated miss). One `stat` per
call. `TestTokenStore_RotatedOutTokenRejectedFirst` (red-proof convicts).
- **R-317:** the LAN resolver decides whether to install `dnsmasq` by its service UNIT, not by `/usr/sbin/dnsmasq` (which
the `dnsmasq-base` package also ships). Same `apt-get install` command; no sudoers change. `TestEnsureDnsmasq_*`
(red-proof convicts). Red-proofs: `felhom.eu/documentation/audits/burndown2-2026-10-05/agent-red-proofs.txt`,
`r124-red-proof.txt`.
Also in this release (no binary effect; from burn-down round 1):
- **R-291:** `scripts/retention-policy.json` names where its 10 comes from — the R-267 newest-10 prune of generic
packages, established 2026-08-10 (R-287) — instead of „observed, no located ruling"; the non-existent
`registry-retention.md` reader is dropped. `check-published-versions.py` still reads 10 (checked).
- **R-348:** `internal/backup/store.go` no longer says backups are „unaffected" by a restart: the reported backup list
reads 0 until the next backup runs; only the hub's verdict (7-day look-back) is unaffected.
## v0.146.1 — R-861 review fixes: the signed update flips a root-owned copy; no Wants=/continuations in mount units; the escrow read follows no symlink anywhere (2026-10-05)
Released by `scripts/release-agent.sh`: binary sha256 `badd6c9a2e40c8bfe856d2d1a203443b21b7eb92ecc35d6090ab44518d4d082a`,
config bundle sha256 `42333e969028867ad8142335e6c1bc4040eec231de0d8d330c2d4b2cf7bc3442`. **Supersedes v0.146.0, which
was released but never vouched or delivered to any box.** The same order applies: signed `agent_update` first, then the
signed `agent_config_update`.
**Delivery needs a STEP bundle (R-880, found while delivering).** An installed `felhom-os-apply` checks an incoming
bundle's paths against its OWN table (R16), so every box on the v0.145.0 bundle REFUSES the v0.146.1 bundle (it adds 4
paths). `scripts/build-step-bundle.py` builds the transition: the box's current bundle with ONLY `felhom-os-apply`
replaced (same paths — the old wrapper accepts it), published as bundle version `0.146.1-step1`; then the release's own
bundle. Order on a box: `agent_update` 0.146.1 → `agent_config_update` 0.146.1-step1 → `agent_config_update` 0.146.1.
Tests `StepBundle` (the R16 refusal reproduced; the step accepted; exactly one file changed). Tooling only — not in the
binary or the bundle.
A background security review of the v0.146.0 commit found three holes in the new code; each is fixed and red-proved
(`felhom.eu/documentation/audits/hub-safety-2026-10-05/partF/red-proof.txt`, S1–S3):
- **S1 — a race in the signed update.** `felhom-os-apply` hashed the agent's staged file and then let the A/B wrapper copy
it BY PATH; the agent owns that directory and could swap the file in between. Now the root step reads the file ONCE
(`read_staged_once`: O_NOFOLLOW, fstat, owner, size), hashes those bytes, writes them to a root-owned directory
(`/var/lib/felhom-os-apply/agent-update/`) and hands ONLY that copy to `felhom-selfupdate-guarded apply`, which now
refuses any other directory, a symlink, or a file not owned by root. Tests: `AgentUpdate` (+1),
`SelfupdateWrapperConfinement`.
- **S2 — an allowlist escape in `felhom-priv-apply`.** `[Unit]` accepted `Wants=`/`Requires=`/`Before=` naming any unit, so
a mount unit could start e.g. `reboot.target`. `[Unit]` now holds only `Description` and `After=local-fs-pre.target`
(what the renderers write), and any line ending in a backslash (a systemd continuation this parser would read
differently) is refused. Tests `test_U2_wants_starts_another_unit`, `test_U2_continuation_line`.
- **S3 — a path traversal in the escrow read.** `O_NOFOLLOW` guards only the last component; a symlinked DIRECTORY in the
agent's own state dir still redirected the root read. `readStagedNoFollow` now walks the path from `/` with
`openat(O_NOFOLLOW)` per component. Test `TestAttach_RefusesASymlinkedDirectory`.
## v0.146.0 — the agent's root grants narrowed: exact sudo patterns, a root content checker, fixed files from the bundle, the signed update checked as root (R-861) (2026-10-05) ## v0.146.0 — the agent's root grants narrowed: exact sudo patterns, a root content checker, fixed files from the bundle, the signed update checked as root (R-861) (2026-10-05)
Released by `scripts/release-agent.sh`: binary sha256 `b860af465076041e07f35fed1b12d64ae2b2985d8995f0ce167418d39c2b00d5`, Released by `scripts/release-agent.sh`: binary sha256 `b860af465076041e07f35fed1b12d64ae2b2985d8995f0ce167418d39c2b00d5`,
+7 -11
View File
@@ -1,14 +1,10 @@
# REPORT — agent v0.145.0 (2026-10-05, afternoon): the OS update repairs itself after a power cut # REPORT — agent v0.147.0 (2026-10-05, burn-down round 2)
Brief: the 2026-10-05 catch-up brief (operator), Parts C (R-874, R-875) and D (R-876). Full session report: Full session report: `felhom.eu/REPORT-burndown2-2026-10-05.md`. Baseline `d833163` (v0.146.1). Code commit `f1b9b41`
`felhom.eu/REPORT-catchup-2026-10-05.md`. Architecture: `11-os-updates.md` §5.4.1, §8.4–8.5; `09` decisions 117–118. (CI job 1365 success), tag `v0.147.0`, binary sha256 `642c4d19…`, bundle sha256 `326527d0…` (verified by download).
| Row | Fix | Proof | Rows: R-124 (recipe root namespace = ""), R-118 (no root size for an absent drive), R-269 (rotated-out token rejected at
|---|---|---| once), R-317 (dnsmasq install probed by its unit). Tests + red-proofs: `felhom.eu/documentation/audits/burndown2-2026-10-05/`.
| R-876 | the wrapper reads `dpkg --audit` AND the update journal in ONE call, repairs on either; belt: repair + retry once when apt says "interrupted" | 3 tests + 3 red-proofs; **live (operator's go): crash mid-unpack on demo-hp → the next pass `REPAIR … journal=1` → `DONE rc=0 upgraded=12`, nobody touched the box** | `go build/vet/test ./...` green; `agent_gates.py --fast` green after the release (release-complete needs the tag).
| R-874 | the restore-test's first due-check 30 min after start | 2 tests + red-proof; live on demo-felhom: passed restore-test at start + 30 min |
| R-875 | a kept report's reason is neutral ("sent late …") | test + red-proof |
Released by `scripts/release-agent.sh`: v0.145.0 (`894da35c…`, bundle `78c00adc…`), verified by download; signed Delivery: see the session report (vouch, signed jobs per box, the hub System page afterwards).
`agent_update` + `agent_config_update` to demo-hp, demo-felhom, tester-1 (71/71 after each bundle); vouched with
golden 0.295.0, min_agent 0.131.0. `go test ./...` rc 0, Python suites OK, `agent_gates.py` OK.
+3 -2
View File
@@ -66,8 +66,8 @@
|---|---|---|---|---| |---|---|---|---|---|
| `IntentStore` (`Get/SetEnrolled/SetEjected/SetDecommissioned/OnAbsent`) | internal/storage/intent.go | `OpenIntentStore(path)` | drive intent (4-state self-heal) | Keyed by durable-id only; `OnAbsent` is the ONLY ejected→enrolled path; refuses empty ids | | `IntentStore` (`Get/SetEnrolled/SetEjected/SetDecommissioned/OnAbsent`) | internal/storage/intent.go | `OpenIntentStore(path)` | drive intent (4-state self-heal) | Keyed by durable-id only; `OnAbsent` is the ONLY ejected→enrolled path; refuses empty ids |
| `GuestBindStore` (`Record/Remove/Guests`) | internal/localapi/guestbindstore.go | `OpenGuestBindStore(path)` | per-guest enrolled binds (F9 re-assert) | Same tmp+rename 0600 pattern as IntentStore | | `GuestBindStore` (`Record/Remove/Guests`) | internal/localapi/guestbindstore.go | `OpenGuestBindStore(path)` | per-guest enrolled binds (F9 re-assert) | Same tmp+rename 0600 pattern as IntentStore |
| `FormatJobStore` + `startFormatDetached` + `RecoverFormatJob` | internal/localapi/formatjob.go | `startFormatDetached(device, durableID, fstype, blank) <-chan error` | detached, restart-surviving mkfs (F20-BUG3) | Runs off `s.baseCtx` (60-min bound) so a request deadline can't SIGKILL mkfs; recovery re-resolves by durable id; blank jobs re-check STILL-blank | | `FormatJobStore` + `startFormatDetached` + `RecoverFormatJob` | internal/localapi/formatjob.go | `startFormatDetached(device, durableID, fstype, blank) (*formatJob, <-chan error)` | detached, restart-surviving mkfs (F20-BUG3) | Runs off `s.baseCtx` (60-min bound) so a request deadline can't SIGKILL mkfs; recovery re-resolves by durable id; blank jobs re-check STILL-blank; on success `job.FSUUID` = the new filesystem's UUID, read back only when the durable id still resolves to the formatted device (R-25) — read it only after `done` delivers |
| `TokenStore.Mint` / `Lookup` | internal/localapi/tokenstore.go | `Mint(vmid) (plaintext, error)` | per-guest local-API tokens | Only the SHA-256 hash persists (fsync'd append log); constant-time compare on lookup; plaintext returned exactly once. Lookup RELOADS the file once on a miss (v0.63.0, B3): the one-shot provisioner mints into the same file the daemon indexes — cross-process coherence without a restart; append-only size check bounds the re-read | | `TokenStore.Mint` / `Lookup` | internal/localapi/tokenstore.go | `Mint(vmid) (plaintext, error)` | per-guest local-API tokens | Only the SHA-256 hash persists (fsync'd append log); constant-time compare on lookup; plaintext returned exactly once. Lookup stats the file on EVERY call and reloads BEFORE answering when the append-only log grew (R-269; was reload-on-miss only, v0.63.0 B3, which let a token rotated out by another process keep authorizing as a map hit): the one-shot provisioner mints into the same file the daemon indexes — cross-process coherence both ways without a restart; unchanged size = no re-read. Pinned by `TestTokenStore_RotatedOutTokenRejectedFirst` |
| `FileNonceStore.SeenOrRecord` | internal/authz/noncestore.go | `SeenOrRecord(nonce, exp) bool` | durable anti-replay | fsync'd before returning false; prune only after exp | | `FileNonceStore.SeenOrRecord` | internal/authz/noncestore.go | `SeenOrRecord(nonce, exp) bool` | durable anti-replay | fsync'd before returning false; prune only after exp |
| `Journal` (`Append/Latest/InFlight/AlreadyApplied`) | internal/reconcile/journal.go | `OpenJournal(path)` | op journal + idempotency + crash recovery | `Recover` consumes `InFlight()`; scratch entries special-cased | | `Journal` (`Append/Latest/InFlight/AlreadyApplied`) | internal/reconcile/journal.go | `OpenJournal(path)` | op journal + idempotency + crash recovery | `Recover` consumes `InFlight()`; scratch entries special-cased |
@@ -157,6 +157,7 @@
| `storage.HostOps` | internal/storage/hostops.go | `*SudoHostOps` (prod), `NoopHostOps` (degraded) | fakes in internal/storage/observe_test.go, watchdog_test.go | | `storage.HostOps` | internal/storage/hostops.go | `*SudoHostOps` (prod), `NoopHostOps` (degraded) | fakes in internal/storage/observe_test.go, watchdog_test.go |
| `storage.HostReader` | internal/storage/hostread.go | `*ProcHostReader` | `fakeHostReader` internal/localapi/disks_test.go; internal/storage/role_test.go. v0.87.0: `BlockSlaves(name)` lists `/sys/block/<name>/slaves` (root-free) — backs the `SystemDisks` dm/md walk (`physicalDisksOf`/`walkSlaves`, role.go); per-branch conservatism: an unresolvable slave fails the WHOLE walk → all-system fail-safe. NEVER weaken the signature test `TestSystemDisks_WalkTopologies` (root-backing disk always in the system set). | | `storage.HostReader` | internal/storage/hostread.go | `*ProcHostReader` | `fakeHostReader` internal/localapi/disks_test.go; internal/storage/role_test.go. v0.87.0: `BlockSlaves(name)` lists `/sys/block/<name>/slaves` (root-free) — backs the `SystemDisks` dm/md walk (`physicalDisksOf`/`walkSlaves`, role.go); per-branch conservatism: an unresolvable slave fails the WHOLE walk → all-system fail-safe. NEVER weaken the signature test `TestSystemDisks_WalkTopologies` (root-backing disk always in the system set). |
| `localapi.DiskOps` / `StorageGate` / `GuestAttacher` / `GuestLister` | internal/localapi/disks.go | `*storage.SudoHostOps`; `storageGateAdapter` (cmd/felhom-agent/main.go); `*GuestBinder`; `*proxmox.Client` | `fakeDiskOps`/`fakeGate`/`fakeGuestAttacher`/`fakeGuestList` internal/localapi/disks_test.go | | `localapi.DiskOps` / `StorageGate` / `GuestAttacher` / `GuestLister` | internal/localapi/disks.go | `*storage.SudoHostOps`; `storageGateAdapter` (cmd/felhom-agent/main.go); `*GuestBinder`; `*proxmox.Client` | `fakeDiskOps`/`fakeGate`/`fakeGuestAttacher`/`fakeGuestList` internal/localapi/disks_test.go |
| `lanresolver.hostRoot` + `dnsmasqUnitPaths` (data seam, R-317) | internal/lanresolver/lanresolver.go | prod `hostRoot = "/"`; probe = the `dnsmasq` package's systemd UNIT, never `/usr/sbin/dnsmasq` (owned by `dnsmasq-base`) | internal/lanresolver/ensure_dnsmasq_test.go — fixture root tree + recording `proxmox.Runner`; the REAL `os.Stat` probe and `EnsureDnsmasq` run. `TestEnsureDnsmasq_ProductionProbeIsTheUnit` pins the production wiring |
| `localapi.GuestAPI` / `BackupService` / `BackupStore` / `TokenAuthority` | internal/localapi/server.go | `*proxmox.Client`, `*backup.BackupRunner`, `*backup.Store`, `*TokenStore` | `fakeGuests`/`fakeBackups`/`fakeStore` internal/localapi/server_test.go | | `localapi.GuestAPI` / `BackupService` / `BackupStore` / `TokenAuthority` | internal/localapi/server.go | `*proxmox.Client`, `*backup.BackupRunner`, `*backup.Store`, `*TokenStore` | `fakeGuests`/`fakeBackups`/`fakeStore` internal/localapi/server_test.go |
| `backup.InFlight` | internal/backup/inflight.go | `TryAcquire(what) (release, busy, ok)` / `Busy()` | THE host-wide "one heavy guest operation at a time" gate — shared by the local-API backup path and the restore-test scheduler (R-85) | A **LINK** guard, not a lock one: the scratch VMID never touches the live guest's vzdump lock, but an offsite restore PULLS multi-GB over the tunnel a backup PUSHES one. Callers **DEFER, never cancel** — a deferred restore-test costs coverage, a cancelled backup costs the backup. A nil gate is ungated (pre-R-85 callers). | | `backup.InFlight` | internal/backup/inflight.go | `TryAcquire(what) (release, busy, ok)` / `Busy()` | THE host-wide "one heavy guest operation at a time" gate — shared by the local-API backup path and the restore-test scheduler (R-85) | A **LINK** guard, not a lock one: the scratch VMID never touches the live guest's vzdump lock, but an offsite restore PULLS multi-GB over the tunnel a backup PUSHES one. Callers **DEFER, never cancel** — a deferred restore-test costs coverage, a cancelled backup costs the backup. A nil gate is ungated (pre-R-85 callers). |
| `capability` store-grant probe (`storeGrantStatuses` / `storeGrantVerdict` / `Client.Permissions`) | cmd/felhom-agent/main.go, internal/proxmox/query.go | *"may the agent READ this backup tier?"*, one `capability.Status` per configured tier | R-185. **Never infer permission from an empty content listing** — `{"data":[]}` is what a FORBIDDEN tier and a NEWBORN tier both return, and that ambiguity hid an unreadable host tier on both demo boxes. Ask `/access/permissions` **as the agent's own token** (root always says yes). **The ungranted answer is not empty and not a 403** — it carries the privileges inherited from the box-wide `/` grant, so test for **`Datastore.AllocateSpace`** specifically; path-presence or `Datastore.Audit` reports a blinded storage healthy. Probed set comes from `BackupTiers()`, never a fixed list. Critical except the `local` fallback. Composes AROUND the sudo prober (the `poolReadStatus` precedent); `Status`'s wire shape is untouched so the hub alert is free. Unreachable PVE ⇒ degraded, never ok. | | `capability` store-grant probe (`storeGrantStatuses` / `storeGrantVerdict` / `Client.Permissions`) | cmd/felhom-agent/main.go, internal/proxmox/query.go | *"may the agent READ this backup tier?"*, one `capability.Status` per configured tier | R-185. **Never infer permission from an empty content listing** — `{"data":[]}` is what a FORBIDDEN tier and a NEWBORN tier both return, and that ambiguity hid an unreadable host tier on both demo boxes. Ask `/access/permissions` **as the agent's own token** (root always says yes). **The ungranted answer is not empty and not a 403** — it carries the privileges inherited from the box-wide `/` grant, so test for **`Datastore.AllocateSpace`** specifically; path-presence or `Datastore.Audit` reports a blinded storage healthy. Probed set comes from `BackupTiers()`, never a fixed list. Critical except the `local` fallback. Composes AROUND the sudo prober (the `poolReadStatus` precedent); `Status`'s wire shape is untouched so the hub alert is free. Unreachable PVE ⇒ degraded, never ok. |
+62
View File
@@ -667,5 +667,67 @@ class SelfupdateWrapperConfinement(unittest.TestCase):
self.assertEqual(p.returncode, 1, p.stderr) self.assertEqual(p.returncode, 1, p.stderr)
self.assertIn("outside /var/lib/felhom-os-apply/agent-update", p.stderr) self.assertIn("outside /var/lib/felhom-os-apply/agent-update", p.stderr)
_sb = importlib.machinery.SourceFileLoader("stepbuild", str(REPO / "scripts" / "build-step-bundle.py"))
_ss = importlib.util.spec_from_loader("stepbuild", _sb)
stepbuild = importlib.util.module_from_spec(_ss)
_sb.exec_module(stepbuild)
NEW_IN_0146 = {"/usr/local/sbin/felhom-priv-apply", "/var/lib/vz/snippets/felhom-guest-hook.sh",
"/usr/local/sbin/felhom-shared-parent.sh", "/etc/systemd/system/felhom-shared-parent.service"}
class StepBundle(unittest.TestCase):
"""R-880 (agent v0.146.1): an INSTALLED wrapper checks an incoming bundle's paths against its OWN table (R16), so a
release that adds paths needs a step bundle: the boxes' current bundle with only felhom-os-apply replaced.
RED-PROOF: deliver the full bundle to the old table → R16 (test_the_full_bundle_is_refused_by_an_old_table)."""
def old_world(self):
"""The base bundle an older wrapper (no R-861 paths) installed, and that wrapper's table."""
full = json.loads(builder.build("0.145.0"))
full["files"] = [e for e in full["files"] if e["path"] not in NEW_IN_0146]
old_wrapper = b'# the v0.145.0 wrapper stands in here\nBUNDLE_OP = "agent_config_update"\n'
for e in full["files"]:
if e["path"] == "/usr/local/sbin/felhom-os-apply":
e["content_b64"], e["sha256"] = base64.b64encode(old_wrapper).decode(), hashlib.sha256(old_wrapper).hexdigest()
base = (json.dumps(full, indent=1, sort_keys=True) + "\n").encode()
old_dests = {k: v for k, v in osapply.BUNDLE_DESTS.items() if k not in NEW_IN_0146}
return base, old_dests
def parse_with_table(self, data, dests, version):
saved = osapply.BUNDLE_DESTS
osapply.BUNDLE_DESTS = dests
try:
return osapply.Bundle(osapply.Apply(Box(b"{}", None), "")).parse(data, hashlib.sha256(data).hexdigest(), version)
finally:
osapply.BUNDLE_DESTS = saved
def test_the_full_bundle_is_refused_by_an_old_table(self):
_, old_dests = self.old_world()
full = builder.build("0.146.1")
with self.assertRaises(osapply.Refused) as cm:
self.parse_with_table(full, old_dests, "0.146.1")
self.assertEqual(cm.exception.code, "R16")
def test_the_step_bundle_is_accepted_by_the_old_table_and_changes_only_the_wrapper(self):
base, old_dests = self.old_world()
new_wrapper = (HERE / "felhom-os-apply").read_bytes()
step = stepbuild.build_step(base, "0.146.1-step1", new_wrapper)
ver, files = self.parse_with_table(step, old_dests, "0.146.1-step1")
self.assertEqual(ver, "0.146.1-step1")
b, s_ = json.loads(base), json.loads(step)
self.assertEqual(sorted(e["path"] for e in b["files"]), sorted(e["path"] for e in s_["files"]), "the paths must not change")
changed = [e["path"] for e, f in zip(sorted(b["files"], key=lambda x: x["path"]), sorted(s_["files"], key=lambda x: x["path"]))
if e != f]
self.assertEqual(changed, ["/usr/local/sbin/felhom-os-apply"], "exactly the wrapper changes")
installed = dict((d, c) for d, c, *_ in files)
self.assertEqual(installed["/usr/local/sbin/felhom-os-apply"], new_wrapper)
# and the NEW wrapper (now installed) knows every path the release's full bundle names
self.assertTrue({e["path"] for e in json.loads(builder.build("0.146.1"))["files"]} <= set(osapply.BUNDLE_DESTS))
def test_a_step_version_must_carry_a_suffix(self):
base, _ = self.old_world()
with self.assertRaises(SystemExit):
stepbuild.build_step(base, "0.146.1", b"x")
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()
+5 -1
View File
@@ -25,7 +25,11 @@ import (
// merges the two — see hub.ProvenRestoreTestReporter. This store remains the ONLY place a FAILURE is // merges the two — see hub.ProvenRestoreTestReporter. This store remains the ONLY place a FAILURE is
// recorded, and that asymmetry is deliberate: a failing tier stays due and is retried, so a lost // recorded, and that asymmetry is deliberate: a failing tier stays due and is retried, so a lost
// failure heals itself, while a lost success leaves the system quietly less tested than it believes. // failure heals itself, while a lost success leaves the system quietly less tested than it believes.
// Backups are unaffected — their freshness has a ground truth on the storage (R-84). // Backups are NOT unaffected (corrected 2026-10-05, R-348): byTarget is in memory too, so after a restart the
// reported backup LIST reads 0 until the next backup of each tier runs (daily local, weekly offsite) — measured
// 2026-08-20, two consecutive host-reports with `0 backups` while `pvesm list` showed archives on both tiers. What
// is unaffected is the hub's VERDICT: it looks back 7 days over stored reports (felhom.eu hub/internal/monitor/
// deadline.go backupEvidenceLookback) and the storage stays the ground truth (R-84).
type Store struct { type Store struct {
mu sync.Mutex mu sync.Mutex
byTarget map[string]hub.Backup // latest backup per target id byTarget map[string]hub.Backup // latest backup per target id
+26 -2
View File
@@ -10,6 +10,7 @@ import (
"log/slog" "log/slog"
"os" "os"
"strings" "strings"
"sync"
"time" "time"
"gitea.dooplex.hu/admin/felhom-agent/internal/capability" "gitea.dooplex.hu/admin/felhom-agent/internal/capability"
@@ -115,6 +116,7 @@ type Collector struct {
backupTarget func() ConfiguredBackupTarget // R-109: primary backup tier id (nil → recipe records unknown) backupTarget func() ConfiguredBackupTarget // R-109: primary backup tier id (nil → recipe records unknown)
hostID string hostID string
agentVersion string agentVersion string
selfSHA func() string // R-349: sha256 of the running binary; default runningBinarySHA256
logger *slog.Logger logger *slog.Logger
now func() time.Time now func() time.Time
} }
@@ -135,6 +137,7 @@ func NewCollector(px proxmoxReader, cf CloudflaredProber, storage StorageObserve
temp: SysfsTempReader{}, // slice 9: real sysfs reader by default; tests inject a fake temp: SysfsTempReader{}, // slice 9: real sysfs reader by default; tests inject a fake
hostID: hostID, hostID: hostID,
agentVersion: agentVersion, agentVersion: agentVersion,
selfSHA: runningBinarySHA256,
logger: logger, logger: logger,
now: func() time.Time { return time.Now().UTC() }, now: func() time.Time { return time.Now().UTC() },
} }
@@ -337,6 +340,7 @@ func (c *Collector) Collect(ctx context.Context) (*HostReport, error) {
HostID: c.hostID, HostID: c.hostID,
ReportedAt: c.now().Format(time.RFC3339), ReportedAt: c.now().Format(time.RFC3339),
AgentVersion: c.agentVersion, AgentVersion: c.agentVersion,
AgentSHA256: c.agentSHA256(),
Host: host, Host: host,
Guests: c.collectGuests(ctx), Guests: c.collectGuests(ctx),
// storage_targets populated this slice (slice 5) via the observer; the rest stay // storage_targets populated this slice (slice 5) via the observer; the rest stay
@@ -431,8 +435,28 @@ const pbsWrapperPath = "/usr/local/sbin/felhom-pbs-apply"
// unreadable file yields "", which the hub reads as UNKNOWN rather than as drift — a host that // unreadable file yields "", which the hub reads as UNKNOWN rather than as drift — a host that
// legitimately has no DR wrapper must not light up amber. The file is 0755, so no privilege is // legitimately has no DR wrapper must not light up amber. The file is 0755, so no privilege is
// needed to read it. // needed to read it.
func pbsWrapperSHA256() string { func pbsWrapperSHA256() string { return fileSHA256(pbsWrapperPath) }
f, err := os.Open(pbsWrapperPath)
// selfExePath is the running binary as the kernel holds it. /proc/self/exe, not the installed path:
// after an A/B flip the file at /usr/local/bin/felhom-agent may already be the NEXT binary while this
// process still runs the old one, and the report must describe what runs (R-349). Test seam.
var selfExePath = "/proc/self/exe"
// runningBinarySHA256 hashes the running binary ONCE per process — the bytes cannot change under a
// running process, and re-hashing ~20 MB every report cycle buys nothing. A failed read is cached as
// "" (UNKNOWN); it never fails the report.
var runningBinarySHA256 = sync.OnceValue(func() string { return fileSHA256(selfExePath) })
func (c *Collector) agentSHA256() string {
if c.selfSHA == nil {
return ""
}
return c.selfSHA()
}
// fileSHA256 is the hex sha256 of a file's bytes, or "" when it cannot be read.
func fileSHA256(path string) string {
f, err := os.Open(path)
if err != nil { if err != nil {
return "" return ""
} }
+64
View File
@@ -0,0 +1,64 @@
package hub
import (
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"os"
"path/filepath"
"strings"
"testing"
)
// R-349: the report carries the sha256 of the binary that is RUNNING, so the hub can tell a
// hand-built proof binary from the vouched artifact of the same version string. The consequence
// asserted: the wire field equals the hash of this very test binary's bytes (read independently via
// os.Executable, a different channel from /proc/self/exe), and it is on the wire as agent_sha256.
func TestCollect_AgentSHA256IsTheRunningBinary(t *testing.T) {
exe, err := os.Executable()
if err != nil {
t.Skipf("os.Executable: %v", err)
}
raw, err := os.ReadFile(exe)
if err != nil {
t.Fatalf("read own binary: %v", err)
}
sum := sha256.Sum256(raw)
want := hex.EncodeToString(sum[:])
px := &fakePx{node: "n", ns: newTestNodeStatus()}
c := NewCollector(px, fakeProber{status: "running"}, nil, nil, nil, nil, "h", "0.3.0", quietLogger())
r, err := c.Collect(context.Background())
if err != nil {
t.Fatalf("Collect: %v", err)
}
if r.AgentSHA256 != want {
t.Fatalf("agent_sha256 = %q, want the running binary's %q", r.AgentSHA256, want)
}
b, err := json.Marshal(r)
if err != nil {
t.Fatal(err)
}
if !strings.Contains(string(b), `"agent_sha256":"`+want+`"`) {
t.Fatalf("agent_sha256 not on the wire: %s", b)
}
}
// An unreadable binary is UNKNOWN (empty, omitted) — never a made-up hash, never a failed report.
func TestFileSHA256_UnreadableIsEmpty(t *testing.T) {
if got := fileSHA256(filepath.Join(t.TempDir(), "absent")); got != "" {
t.Fatalf("absent file hashed to %q, want empty", got)
}
px := &fakePx{node: "n", ns: newTestNodeStatus()}
c := NewCollector(px, fakeProber{status: "running"}, nil, nil, nil, nil, "h", "0.3.0", quietLogger())
c.selfSHA = func() string { return "" }
r, err := c.Collect(context.Background())
if err != nil {
t.Fatalf("Collect must not fail on an unreadable binary: %v", err)
}
b, _ := json.Marshal(r)
if strings.Contains(string(b), "agent_sha256") {
t.Fatalf("empty agent_sha256 must be omitted: %s", b)
}
}
+9 -7
View File
@@ -54,11 +54,13 @@ const (
DRReasonNoPBSStorage = "no_pbs_storage_observed" DRReasonNoPBSStorage = "no_pbs_storage_observed"
) )
// PBSRootNamespace is how the recipe spells PBS's root namespace. The PBS API spells it as the EMPTY // PBSRootNamespace is how the recipe spells PBS's root namespace: the EMPTY string, PBS's own spelling (R-124,
// string (and `pct restore --ns root` would name a namespace that does not exist) — "root" is a display // agent v0.147.0). It used to be the display word "root", which no PBS namespace is named — an operator pasting it
// convention this wire has always used, kept here so the field's meaning did not change under R-106. // into `proxmox-backup-client … --ns root` during a real recovery got a failure. An empty namespace is ambiguous on
// Only a box with no `namespace` line in its pbs storage.cfg stanza ever emits it. // its own, so READ IT WITH namespace_state: resolved + "" = the root namespace (pass no --ns, or --ns ""); unknown +
const PBSRootNamespace = "root" // "" = the agent could not tell. Only a box with no `namespace` line in its pbs storage.cfg stanza emits it.
// Pinned by TestDRRecipe_PBSNamespaceRootIsResolvedNotUnknown and TestR124_RootNamespaceOnTheWireIsPBSSpelling.
const PBSRootNamespace = ""
// DRRecipeHostHalf is the agent-emitted half (guest/drive/storage/PBS scaffolding). Derived entirely // DRRecipeHostHalf is the agent-emitted half (guest/drive/storage/PBS scaffolding). Derived entirely
// from facts the report already collects — no new privileged reads. // from facts the report already collects — no new privileged reads.
@@ -123,8 +125,8 @@ type DRPBSCoord struct {
RepoID string `json:"repo_id"` // the PVE pbs storage id (e.g. "felhom-pbs") — not a token RepoID string `json:"repo_id"` // the PVE pbs storage id (e.g. "felhom-pbs") — not a token
// Namespace is the PBS namespace the restore targets, resolved from the pbs storage's storage.cfg // Namespace is the PBS namespace the restore targets, resolved from the pbs storage's storage.cfg
// stanza — the same field `vzdump --storage <pbs>` makes PVE read, so the recipe cannot disagree // stanza — the same field `vzdump --storage <pbs>` makes PVE read, so the recipe cannot disagree
// with the backup that produced the snapshot. PBSRootNamespace when the box has no namespace // with the backup that produced the snapshot. PBSRootNamespace ("", PBS's spelling, R-124) when the box has no
// configured; "" when NamespaceState is unknown. // namespace configured; also "" when NamespaceState is unknown — consult NamespaceState.
// //
// R-106: this used to come from the listed snapshot's own `ns`, which PBS does not echo per item once // R-106: this used to come from the listed snapshot's own `ns`, which PBS does not echo per item once
// the request is already namespace-scoped via `?ns=` (internal/pbs/client.go). The field was // the request is already namespace-scoped via `?ns=` (internal/pbs/client.go). The field was
+39 -3
View File
@@ -63,8 +63,8 @@ func TestBuildDRRecipeHostHalf(t *testing.T) {
t.Error("felhom-flash (local-dir user-data drive) missing from drives") t.Error("felhom-flash (local-dir user-data drive) missing from drives")
} }
// pbs: latest snapshot's coords + the pbs storage id as repo_id. // pbs: latest snapshot's coords + the pbs storage id as repo_id.
if h.PBS == nil || h.PBS.RepoID != "felhom-pbs" || h.PBS.Namespace != "root" || h.PBS.LatestSnapshotID != "9201" { if h.PBS == nil || h.PBS.RepoID != "felhom-pbs" || h.PBS.Namespace != PBSRootNamespace || h.PBS.LatestSnapshotID != "9201" {
t.Errorf("pbs coord = %+v, want repo felhom-pbs/root/9201", h.PBS) t.Errorf("pbs coord = %+v, want repo felhom-pbs, the root namespace (\"\", R-124), snapshot 9201", h.PBS)
} }
} }
@@ -252,7 +252,7 @@ func TestDRRecipe_PBSNamespaceIsThePerCustomerOne(t *testing.T) {
} }
// TestDRRecipe_PBSNamespaceRootIsResolvedNotUnknown: a box with a pbs storage and NO namespace line is // TestDRRecipe_PBSNamespaceRootIsResolvedNotUnknown: a box with a pbs storage and NO namespace line is
// genuinely in the root namespace. That is an answer, not a gap — it must read resolved/"root", so the // genuinely in the root namespace. That is an answer, not a gap — it must read resolved/"" (PBS's spelling, R-124), so the
// honest root case is never confused with "I could not tell". // honest root case is never confused with "I could not tell".
func TestDRRecipe_PBSNamespaceRootIsResolvedNotUnknown(t *testing.T) { func TestDRRecipe_PBSNamespaceRootIsResolvedNotUnknown(t *testing.T) {
h := BuildDRRecipeHostHalf(nil, h := BuildDRRecipeHostHalf(nil,
@@ -453,3 +453,39 @@ func assertNoSecretKeys(t *testing.T, jsonBytes []byte) {
} }
walk("<root>", v) walk("<root>", v)
} }
// R-124: on the WIRE the root namespace is PBS's own spelling — an empty string, present (not omitted), beside
// namespace_state "resolved". The display word "root" names no PBS namespace, and `--ns root` fails in a recovery.
// RED-PROOF: set PBSRootNamespace back to "root" → this test fails.
func TestR124_RootNamespaceOnTheWireIsPBSSpelling(t *testing.T) {
h := BuildDRRecipeHostHalf(nil,
[]StorageTarget{{Name: "felhom-pbs", Type: StorageTypePBS, Content: "backup", PBSNamespace: ""}},
capturedDemoFelhomSnapshots(),
ConfiguredBackupTarget{StorageID: "felhom-pbs", Known: true})
b, err := json.Marshal(h.PBS)
if err != nil {
t.Fatal(err)
}
var m map[string]any
if err := json.Unmarshal(b, &m); err != nil {
t.Fatal(err)
}
ns, present := m["namespace"]
if !present {
t.Fatalf("namespace key missing from %s — an omitted key reads as 'unknown', not 'root'", b)
}
if ns != "" {
t.Fatalf("root namespace on the wire = %q, want \"\" (PBS's spelling; no namespace is named %q)", ns, ns)
}
if m["namespace_state"] != DRStateResolved {
t.Fatalf("namespace_state = %v, want %q beside the empty root namespace", m["namespace_state"], DRStateResolved)
}
// A configured namespace still passes through unchanged.
h2 := BuildDRRecipeHostHalf(nil,
[]StorageTarget{{Name: "felhom-pbs", Type: StorageTypePBS, Content: "backup", PBSNamespace: "demo-felhom"}},
capturedDemoFelhomSnapshots(),
ConfiguredBackupTarget{StorageID: "felhom-pbs", Known: true})
if h2.PBS.Namespace != "demo-felhom" {
t.Fatalf("configured namespace = %q, want demo-felhom", h2.PBS.Namespace)
}
}
+10
View File
@@ -18,6 +18,16 @@ type HostReport struct {
HostID string `json:"host_id"` // echoes config.Hub.HostID HostID string `json:"host_id"` // echoes config.Hub.HostID
ReportedAt string `json:"reported_at"` // RFC3339, agent clock ReportedAt string `json:"reported_at"` // RFC3339, agent clock
AgentVersion string `json:"agent_version"` AgentVersion string `json:"agent_version"`
// AgentSHA256 is the sha256 of the binary this process is RUNNING (read through /proc/self/exe,
// once per process), R-349. The version string cannot tell a hand-built proof binary from the
// published, vouched artifact of the same version — same source, different bytes (`-trimpath
// -buildvcs=false` in release-agent.sh) — so self-update sees "already installed" and never
// corrects it. Reporting the bytes lets the hub compare against the vouched agent_sha256, the
// same mechanism host.wrapper_sha256 is for the PBS wrapper (R-50b(a)).
//
// Empty = unreadable, which the hub must treat as UNKNOWN, never as drift. Pinned by
// TestCollect_AgentSHA256IsTheRunningBinary.
AgentSHA256 string `json:"agent_sha256,omitempty"`
Host HostMetrics `json:"host"` Host HostMetrics `json:"host"`
Guests []Guest `json:"guests"` Guests []Guest `json:"guests"`
+122
View File
@@ -0,0 +1,122 @@
package lanresolver
import (
"context"
"io"
"log/slog"
"os"
"path/filepath"
"strings"
"sync"
"testing"
)
// recRunner records every privileged command EnsureDnsmasq would run and succeeds — nothing reaches
// apt, systemctl or the root checker.
type recRunner struct {
mu sync.Mutex
calls []string
}
func (r *recRunner) Run(_ context.Context, name string, args ...string) ([]byte, []byte, error) {
r.mu.Lock()
defer r.mu.Unlock()
r.calls = append(r.calls, strings.Join(append([]string{name}, args...), " "))
return nil, nil, nil
}
func (r *recRunner) RunStdin(ctx context.Context, _ io.Reader, name string, args ...string) ([]byte, []byte, error) {
return r.Run(ctx, name, args...)
}
func (r *recRunner) installed() bool {
for _, c := range r.calls {
if strings.HasPrefix(c, "apt-get install") && strings.HasSuffix(c, " dnsmasq") {
return true
}
}
return false
}
// fixtureRoot builds a fake host root holding exactly the given relative files and points the REAL
// probe at it for the test's duration.
func fixtureRoot(t *testing.T, files ...string) {
t.Helper()
root := t.TempDir()
for _, f := range files {
p := filepath.Join(root, f)
if err := os.MkdirAll(filepath.Dir(p), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(p, nil, 0o644); err != nil {
t.Fatal(err)
}
}
prev := hostRoot
hostRoot = root
t.Cleanup(func() { hostRoot = prev })
}
func ensure(t *testing.T) *recRunner {
t.Helper()
r := &recRunner{}
m := NewManager(r, "192.0.2.10", []string{"1.1.1.1"}, slog.New(slog.NewTextHandler(io.Discard, nil)))
if err := m.EnsureDnsmasq(context.Background()); err != nil {
t.Fatalf("EnsureDnsmasq: %v", err)
}
return r
}
// R-317: a host with `dnsmasq-base` (the /usr/sbin/dnsmasq binary) but WITHOUT the `dnsmasq` package
// (the service unit) must get the package installed — else the following `systemctl enable --now
// dnsmasq` hits a unit that does not exist and LAN name resolution silently never comes up.
//
// RED-PROOF: probe "usr/sbin/dnsmasq" instead of the unit paths in dnsmasqUnitInstalled → this fails
// with "install was skipped".
func TestEnsureDnsmasq_BinaryWithoutUnitInstalls(t *testing.T) {
fixtureRoot(t, "usr/sbin/dnsmasq")
r := ensure(t)
if !r.installed() {
t.Fatalf("install was skipped on a dnsmasq-base-only host (binary present, unit absent) — "+
"the enable that follows targets a missing unit (R-317). calls: %q", r.calls)
}
}
func TestEnsureDnsmasq_UnitPresentSkipsInstall(t *testing.T) {
for _, unit := range []string{"usr/lib/systemd/system/dnsmasq.service", "lib/systemd/system/dnsmasq.service"} {
t.Run(unit, func(t *testing.T) {
fixtureRoot(t, "usr/sbin/dnsmasq", unit)
if r := ensure(t); r.installed() {
t.Fatalf("apt-get install ran although the dnsmasq unit is present at %s: %q", unit, r.calls)
}
})
}
}
func TestEnsureDnsmasq_NothingPresentInstalls(t *testing.T) {
fixtureRoot(t)
if r := ensure(t); !r.installed() {
t.Fatalf("install skipped on a host with no dnsmasq at all: %q", r.calls)
}
}
// Production wiring for the hostRoot seam: the shipped probe resolves against the real root and asks
// about the unit the `dnsmasq` package owns — never the dnsmasq-base binary.
func TestEnsureDnsmasq_ProductionProbeIsTheUnit(t *testing.T) {
if hostRoot != "/" {
t.Fatalf("hostRoot default = %q, want \"/\" — the production probe would look in the wrong tree", hostRoot)
}
var sawUsrLib bool
for _, p := range dnsmasqUnitPaths {
full := filepath.Join(hostRoot, p)
if strings.HasSuffix(full, "/sbin/dnsmasq") || strings.HasSuffix(full, "/bin/dnsmasq") {
t.Errorf("probe path %s is the dnsmasq-base binary, not the dnsmasq unit (R-317)", full)
}
if full == "/usr/lib/systemd/system/dnsmasq.service" {
sawUsrLib = true
}
}
if !sawUsrLib {
t.Errorf("probe paths %q miss /usr/lib/systemd/system/dnsmasq.service (dpkg -S: owned by dnsmasq)", dnsmasqUnitPaths)
}
}
+26 -2
View File
@@ -101,11 +101,35 @@ func NewManager(runner proxmox.Runner, hostIP string, upstreams []string, logger
} }
} }
// hostRoot is the filesystem root the install probe resolves against: "/" in production; a test
// points it at a fixture tree so the REAL probe runs against files it controls.
var hostRoot = "/"
// dnsmasqUnitPaths are where the `dnsmasq` package ships its systemd unit (Debian; /lib is the
// pre-usrmerge spelling). R-317: probe the UNIT, never /usr/sbin/dnsmasq — that binary belongs to
// `dnsmasq-base`, so a host carrying dnsmasq-base without dnsmasq used to skip the install and then
// `systemctl enable --now dnsmasq` failed against a unit that is not there (resolver never up).
// Pinned by TestEnsureDnsmasq_BinaryWithoutUnitInstalls.
var dnsmasqUnitPaths = []string{
"usr/lib/systemd/system/dnsmasq.service",
"lib/systemd/system/dnsmasq.service",
}
// dnsmasqUnitInstalled reports whether the dnsmasq service unit (the `dnsmasq` package) is present.
func dnsmasqUnitInstalled() bool {
for _, p := range dnsmasqUnitPaths {
if _, err := os.Stat(filepath.Join(hostRoot, p)); err == nil {
return true
}
}
return false
}
// EnsureDnsmasq makes dnsmasq present + enabled and writes the host base config. Idempotent: it // EnsureDnsmasq makes dnsmasq present + enabled and writes the host base config. Idempotent: it
// installs the package only when absent, and writes the base drop-in only when its content changes. // installs the package only when absent, and writes the base drop-in only when its content changes.
func (m *Manager) EnsureDnsmasq(ctx context.Context) error { func (m *Manager) EnsureDnsmasq(ctx context.Context) error {
if _, err := os.Stat("/usr/sbin/dnsmasq"); err != nil { // metadata read, no privilege needed if !dnsmasqUnitInstalled() { // metadata read, no privilege needed
m.logger.Info("lanresolver: dnsmasq absent — installing") m.logger.Info("lanresolver: dnsmasq service unit absent — installing")
if out, errOut, ierr := m.runner.Run(ctx, "apt-get", "install", "-y", "-q", "dnsmasq"); ierr != nil { if out, errOut, ierr := m.runner.Run(ctx, "apt-get", "install", "-y", "-q", "dnsmasq"); ierr != nil {
return fmt.Errorf("install dnsmasq: %s: %w", strings.TrimSpace(string(errOut))+string(out), ierr) return fmt.Errorf("install dnsmasq: %s: %w", strings.TrimSpace(string(errOut))+string(out), ierr)
} }
+20 -8
View File
@@ -398,9 +398,16 @@ func (s *Server) handleDisks(w http.ResponseWriter, r *http.Request, vmid int) {
di.Smart = &sm di.Smart = &sm
} }
} }
if total, used, okc := statfsCapacity(d.MountPath); okc { // R-118: statfs ONLY while the drive's device is present. With the device gone the raw
di.TotalBytes, di.UsedBytes = total, used // mountpoint reverts to a bare directory on the ROOT filesystem, and statfs would report
di.UsedFraction = float64(used) / float64(total) // pve-root's size as this drive's (measured: a 4 GB drive advertising 46 GiB). Same trap
// observe.go guards on the Observe path. Absent → capacity left zero (unknown), never root's.
// Pinned by TestDisks_UnionPath_AbsentDeviceReportsNoRootCapacity.
if s.devicePresent(d.MountPath) {
if total, used, okc := statfsCapacity(d.MountPath); okc {
di.TotalBytes, di.UsedBytes = total, used
di.UsedFraction = float64(used) / float64(total)
}
} }
out = append(out, di) out = append(out, di)
} }
@@ -761,6 +768,11 @@ type FormatResponse struct {
// signature — the customer authorizes the wipe of their own data drive. // signature — the customer authorizes the wipe of their own data drive.
NeedsConfirmation bool `json:"needs_confirmation,omitempty"` NeedsConfirmation bool `json:"needs_confirmation,omitempty"`
DurableID string `json:"durable_id,omitempty"` // the durable id to confirm against (user-data) DurableID string `json:"durable_id,omitempty"` // the durable id to confirm against (user-data)
// FSUUID (on Formatted) is the UUID of the filesystem the agent just made, verified against the
// bound durable id after mkfs (R-25). The caller mounts THIS — re-resolving a UUID from the /dev
// path later can name another disk if /dev re-enumerated. "" = not verified: the caller must not
// substitute a path-resolved guess silently.
FSUUID string `json:"fs_uuid,omitempty"`
// PendingOp is set on a SYSTEM/BACKUP data-bearing refusal — the exact op the operator must sign. // PendingOp is set on a SYSTEM/BACKUP data-bearing refusal — the exact op the operator must sign.
PendingOp *PendingOp `json:"pending_op,omitempty"` PendingOp *PendingOp `json:"pending_op,omitempty"`
} }
@@ -808,7 +820,7 @@ func (s *Server) handleDiskFormatStatus(w http.ResponseWriter, r *http.Request,
writeOK(w, map[string]any{ writeOK(w, map[string]any{
"vmid": vmid, "phase": job.Phase, "device": job.Device, "fstype": job.FSType, "vmid": vmid, "phase": job.Phase, "device": job.Device, "fstype": job.FSType,
"durable_id": job.DurableID, "error": job.Error, "started_at": job.StartedAt, "updated_at": job.UpdatedAt, "durable_id": job.DurableID, "error": job.Error, "started_at": job.StartedAt, "updated_at": job.UpdatedAt,
"job_id": job.JobID, "job_id": job.JobID, "fs_uuid": job.FSUUID, // R-25: "" until done + verified
}) })
} }
@@ -871,7 +883,7 @@ func (s *Server) handleDiskFormat(w http.ResponseWriter, r *http.Request, vmid i
"format refused (device may have changed since inspection): "+rerr.Error()) "format refused (device may have changed since inspection): "+rerr.Error())
return return
} }
done := s.startFormatDetached(device, blankDurable, req.FSType, true) job, done := s.startFormatDetached(device, blankDurable, req.FSType, true)
if err := s.awaitFormat(r.Context(), done, vmid, device); err != nil { if err := s.awaitFormat(r.Context(), done, vmid, device); err != nil {
if err == errFormatClientGone { if err == errFormatClientGone {
return // client gone; mkfs continues detached + the job record records the outcome return // client gone; mkfs continues detached + the job record records the outcome
@@ -880,7 +892,7 @@ func (s *Server) handleDiskFormat(w http.ResponseWriter, r *http.Request, vmid i
writeErr(w, http.StatusBadGateway, "format failed: "+err.Error()) writeErr(w, http.StatusBadGateway, "format failed: "+err.Error())
return return
} }
writeOK(w, FormatResponse{VMID: vmid, Device: device, Formatted: true, DataBearing: false, DurableID: blankDurable, Reason: "blank device formatted " + req.FSType}) writeOK(w, FormatResponse{VMID: vmid, Device: device, Formatted: true, DataBearing: false, DurableID: blankDurable, FSUUID: job.FSUUID, Reason: "blank device formatted " + req.FSType})
return return
} }
@@ -917,7 +929,7 @@ func (s *Server) handleDiskFormat(w http.ResponseWriter, r *http.Request, vmid i
// F20-BUG3: run the destructive mkfs DETACHED off s.baseCtx (bound durable id recorded for // F20-BUG3: run the destructive mkfs DETACHED off s.baseCtx (bound durable id recorded for
// restart-recovery), so a request/client deadline can never SIGKILL it mid-write and corrupt the // restart-recovery), so a request/client deadline can never SIGKILL it mid-write and corrupt the
// disk. We still wait to return the synchronous result (backward-compatible with the controller). // disk. We still wait to return the synchronous result (backward-compatible with the controller).
done := s.startFormatDetached(device, deviceDurable, req.FSType, false) job, done := s.startFormatDetached(device, deviceDurable, req.FSType, false)
if err := s.awaitFormat(r.Context(), done, vmid, device); err != nil { if err := s.awaitFormat(r.Context(), done, vmid, device); err != nil {
if err == errFormatClientGone { if err == errFormatClientGone {
return // client gone; the wipe continues detached + survives a restart via the job record return // client gone; the wipe continues detached + survives a restart via the job record
@@ -929,7 +941,7 @@ func (s *Server) handleDiskFormat(w http.ResponseWriter, r *http.Request, vmid i
s.logger.Warn("local-api: USER-DATA data-bearing format — CUSTOMER CONFIRMED (no operator signature)", s.logger.Warn("local-api: USER-DATA data-bearing format — CUSTOMER CONFIRMED (no operator signature)",
"vmid", vmid, "device", device, "durable_id", deviceDurable, "fstype", req.FSType, "why", probe.Reason()) "vmid", vmid, "device", device, "durable_id", deviceDurable, "fstype", req.FSType, "why", probe.Reason())
writeOK(w, FormatResponse{VMID: vmid, Device: device, Formatted: true, DataBearing: true, writeOK(w, FormatResponse{VMID: vmid, Device: device, Formatted: true, DataBearing: true,
Role: string(role), DurableID: deviceDurable, Reason: "customer-confirmed wipe (" + probe.Reason() + ")"}) Role: string(role), DurableID: deviceDurable, FSUUID: job.FSUUID, Reason: "customer-confirmed wipe (" + probe.Reason() + ")"})
return return
} }
@@ -5,6 +5,7 @@ import (
"encoding/json" "encoding/json"
"io" "io"
"log/slog" "log/slog"
"runtime"
"strings" "strings"
"testing" "testing"
@@ -184,3 +185,39 @@ func TestDisks_DevicePresence_WireFieldIsFalseOnDeviceLoss(t *testing.T) {
t.Fatalf("the drive never reached the wire: %s", body) t.Fatalf("the drive never reached the wire: %s", body)
} }
} }
// ── R-118 — an absent drive must not advertise the ROOT filesystem's capacity ───────────────────
// TestDisks_UnionPath_AbsentDeviceReportsNoRootCapacity drives the REAL statfsCapacity (no capacity
// seam): the registry drive's mount path is a real, bare temp directory — exactly what /mnt/<name>
// becomes once its device is gone (a plain directory on the host's filesystem). With the device absent
// the row must carry NO capacity; before R-118 the union path statfs'd that bare directory and reported
// the host filesystem's size and usage as the drive's (46 GiB at 9.2 % for a 4 GB drive, measured).
// The present half proves the test is not hollow: the same directory DOES yield capacity when the
// device is there, so a zero on the absent half is the guard's doing, not a statfs failure.
//
// RED-PROOF: drop the `if s.devicePresent(d.MountPath)` guard around statfsCapacity in disks.go → the
// absent subtest fails with "advertises ... bytes".
func TestDisks_UnionPath_AbsentDeviceReportsNoRootCapacity(t *testing.T) {
if runtime.GOOS != "linux" {
t.Skip("statfsCapacity is linux-only; production target is linux")
}
bare := t.TempDir()
known := []storage.KnownTarget{
{Name: "cel", Type: hub.StorageTypeUSB, MountPath: bare, DurableID: "uuid:4242", UUID: "4242"},
}
t.Run("absent", func(t *testing.T) {
di := diskByMount(t, presenceServer(t, nil, known, true, false), bare)
if di.TotalBytes != 0 || di.UsedBytes != 0 || di.UsedFraction != 0 {
t.Errorf("absent drive advertises total=%d used=%d frac=%.3f — that is the filesystem UNDER "+
"the bare mountpoint, not the drive (R-118)", di.TotalBytes, di.UsedBytes, di.UsedFraction)
}
})
t.Run("present", func(t *testing.T) {
di := diskByMount(t, presenceServer(t, nil, known, true, true), bare)
if di.TotalBytes <= 0 {
t.Errorf("present drive reports no capacity (total=%d) — the guard over-corrected and the "+
"size bar is gone for every healthy registry drive", di.TotalBytes)
}
})
}
+6
View File
@@ -28,6 +28,7 @@ type fakeDiskOps struct {
unmountCalls []string unmountCalls []string
candidates []storage.CandidateDisk // returned by ListCandidateDisks candidates []storage.CandidateDisk // returned by ListCandidateDisks
candErr error candErr error
afterFormat *storage.DeviceProbe // R-25: when set, InspectDevice returns it once a format ran
} }
func (f *fakeDiskOps) ListCandidateDisks(_ context.Context) ([]storage.CandidateDisk, error) { func (f *fakeDiskOps) ListCandidateDisks(_ context.Context) ([]storage.CandidateDisk, error) {
@@ -35,7 +36,12 @@ func (f *fakeDiskOps) ListCandidateDisks(_ context.Context) ([]storage.Candidate
} }
func (f *fakeDiskOps) InspectDevice(_ context.Context, device string) (storage.DeviceProbe, error) { func (f *fakeDiskOps) InspectDevice(_ context.Context, device string) (storage.DeviceProbe, error) {
f.mu.Lock()
p := f.probe p := f.probe
if f.afterFormat != nil && len(f.formatCalls) > 0 {
p = *f.afterFormat
}
f.mu.Unlock()
p.Device = device p.Device = device
return p, f.inspectErr return p, f.inspectErr
} }
+96
View File
@@ -0,0 +1,96 @@
package localapi
import (
"context"
"encoding/json"
"net/http"
"sync"
"testing"
"gitea.dooplex.hu/admin/felhom-agent/internal/storage"
)
// R-25: the format answer carries the UUID of the filesystem the agent JUST made, verified against the
// bound durable id after mkfs, so the controller mounts that filesystem rather than whatever the /dev
// path resolves to a few requests later. The consequence asserted: the UUID on the wire (and in the
// polled job record) is the new superblock's — and is EMPTY whenever the binding cannot be re-proved.
const newFSUUID = "0fc63daf-8483-4772-8e79-3d69d8477de4"
func confirmedFormat(t *testing.T, d *fakeDiskOps, srv *Server, fj *FormatJobStore) (string, *formatJob) {
t.Helper()
w := do(t, srv.Handler(), "POST", "/disks/format", "A", `{"device":"/dev/sdb1","fstype":"ext4","confirmed":true,"durable_id":"byid:wwn-/dev/sdb1"}`)
if w.Code != http.StatusOK {
t.Fatalf("confirmed format: %d (%s)", w.Code, w.Body.String())
}
var resp struct {
Data FormatResponse `json:"data"`
}
if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
t.Fatalf("decode: %v (%s)", err, w.Body.String())
}
if !resp.Data.Formatted {
t.Fatalf("not formatted: %s", w.Body.String())
}
return resp.Data.FSUUID, waitFormatPhase(t, fj, formatPhaseDone)
}
func confirmedGate() *fakeGate {
return &fakeGate{decision: WipeDecision{Allowed: true, Tier: "customer_confirmable", Reason: "customer_confirmed"}}
}
func TestFormat_ReportsNewFilesystemUUID(t *testing.T) {
d := &fakeDiskOps{probe: deviceProbeDataBearing(),
afterFormat: &storage.DeviceProbe{Probed: true, HasFilesystem: true, FSType: "ext4", FSUUID: newFSUUID}}
fj := tempFormatStore(t)
srv := formatServer(t, d, confirmedGate(), fj)
got, job := confirmedFormat(t, d, srv, fj)
if got != newFSUUID {
t.Fatalf("response fs_uuid = %q, want the new filesystem's %q", got, newFSUUID)
}
if job.FSUUID != newFSUUID {
t.Fatalf("job record fs_uuid = %q, want %q (the polled status path)", job.FSUUID, newFSUUID)
}
}
// The node moved between mkfs and the read-back: the bound durable id now resolves elsewhere. The
// UUID must NOT be reported — reading it would name the other disk's filesystem.
func TestFormat_FSUUIDWithheldWhenDurableIDMoved(t *testing.T) {
d := &fakeDiskOps{probe: deviceProbeDataBearing(),
afterFormat: &storage.DeviceProbe{Probed: true, HasFilesystem: true, FSType: "ext4", FSUUID: newFSUUID}}
fj := tempFormatStore(t)
srv := formatServer(t, d, confirmedGate(), fj)
var mu sync.Mutex
calls := 0
srv.reresolveWipe = func(_ context.Context, _ string) (string, error) {
mu.Lock()
defer mu.Unlock()
calls++
if calls == 1 {
return "/dev/sdb", nil // the pre-mkfs anti-retarget re-resolve
}
return "/dev/sdc", nil // after mkfs: the durable id now names another node
}
got, job := confirmedFormat(t, d, srv, fj)
if got != "" || job.FSUUID != "" {
t.Fatalf("fs_uuid reported after the durable id moved (response %q, job %q) — must be empty", got, job.FSUUID)
}
if calls < 2 {
t.Fatalf("the post-mkfs re-resolve never ran (calls=%d)", calls)
}
}
// The superblock did not read back as the requested filesystem → not verified → empty.
func TestFormat_FSUUIDWithheldOnFSTypeMismatch(t *testing.T) {
d := &fakeDiskOps{probe: deviceProbeDataBearing(),
afterFormat: &storage.DeviceProbe{Probed: true, HasFilesystem: true, FSType: "xfs", FSUUID: newFSUUID}}
fj := tempFormatStore(t)
srv := formatServer(t, d, confirmedGate(), fj)
got, job := confirmedFormat(t, d, srv, fj)
if got != "" || job.FSUUID != "" {
t.Fatalf("fs_uuid reported for a superblock of the wrong type (response %q, job %q)", got, job.FSUUID)
}
}
+39 -3
View File
@@ -22,6 +22,10 @@ type formatJob struct {
Blank bool `json:"blank,omitempty"` // audit D3: blank (benign) format — recovery re-checks STILL-blank, not data-bearing Blank bool `json:"blank,omitempty"` // audit D3: blank (benign) format — recovery re-checks STILL-blank, not data-bearing
Phase string `json:"phase"` // running | done | failed Phase string `json:"phase"` // running | done | failed
Error string `json:"error,omitempty"` Error string `json:"error,omitempty"`
// FSUUID is the filesystem UUID of the NEW filesystem, read by the agent right after mkfs on the
// device the bound durable id still resolves to (R-25). "" = not verified (the controller must not
// read that as a UUID). Set only on phase done.
FSUUID string `json:"fs_uuid,omitempty"`
StartedAt string `json:"started_at"` StartedAt string `json:"started_at"`
UpdatedAt string `json:"updated_at"` UpdatedAt string `json:"updated_at"`
} }
@@ -96,7 +100,10 @@ func (s *FormatJobStore) save(j *formatJob) error {
// runs to completion and records the outcome. device is the ALREADY anti-retarget-resolved device; the // runs to completion and records the outcome. device is the ALREADY anti-retarget-resolved device; the
// record carries durableID so a restart can re-resolve + re-run. blank marks a benign (blank-device) // record carries durableID so a restart can re-resolve + re-run. blank marks a benign (blank-device)
// format, so restart recovery re-checks STILL-blank rather than data-bearing (audit D3). // format, so restart recovery re-checks STILL-blank rather than data-bearing (audit D3).
func (s *Server) startFormatDetached(device, durableID, fstype string, blank bool) <-chan error { //
// The returned job may be read (FSUUID) only AFTER a value arrives on done — the goroutine writes it
// before the send, which is the happens-before edge.
func (s *Server) startFormatDetached(device, durableID, fstype string, blank bool) (*formatJob, <-chan error) {
base := s.baseCtx base := s.baseCtx
if base == nil { if base == nil {
base = context.Background() base = context.Background()
@@ -116,10 +123,39 @@ func (s *Server) startFormatDetached(device, durableID, fstype string, blank boo
ctx, cancel := context.WithTimeout(base, 60*time.Minute) ctx, cancel := context.WithTimeout(base, 60*time.Minute)
defer cancel() defer cancel()
err := s.disks.Format(ctx, device, fstype) err := s.disks.Format(ctx, device, fstype)
if err == nil {
job.FSUUID = s.formattedFSUUID(ctx, device, durableID, fstype)
}
s.finishFormatJob(job, err) s.finishFormatJob(job, err)
done <- err done <- err
}() }()
return done return job, done
}
// formattedFSUUID reads the UUID of the filesystem the agent has JUST made (R-25). The caller used to
// re-resolve the UUID from the mutable /dev path afterwards, over separate requests — a re-enumeration
// in that window could hand it ANOTHER disk's filesystem to mount. Here the bound durable id must still
// resolve to the very device that was formatted (and re-derive to the same id), and the superblock must
// carry the fstype that was asked for; anything else returns "" (not verified), never a guess.
func (s *Server) formattedFSUUID(ctx context.Context, device, durableID, fstype string) string {
if durableID == "" || s.reresolveWipe == nil {
return ""
}
// The device now holds a filesystem, so the data-bearing anti-retarget re-resolve is the right one.
now, err := s.reresolveWipe(ctx, durableID)
if err != nil || now != device {
s.logger.Warn("format: new filesystem UUID NOT reported — bound durable id no longer resolves to the formatted device",
"device", device, "durable_id", durableID, "resolves_to", now, "err", err)
return ""
}
probe, err := s.disks.InspectDevice(ctx, device)
if err != nil || !probe.Probed || probe.FSType != fstype || probe.FSUUID == "" {
s.logger.Warn("format: new filesystem UUID NOT reported — superblock did not read back as the requested filesystem",
"device", device, "want_fstype", fstype, "got_fstype", probe.FSType, "has_uuid", probe.FSUUID != "", "err", err)
return ""
}
s.logger.Info("format: new filesystem bound to its durable id", "device", device, "durable_id", durableID, "fs_uuid", probe.FSUUID)
return probe.FSUUID
} }
// finishFormatJob updates the persisted record to done/failed. // finishFormatJob updates the persisted record to done/failed.
@@ -172,7 +208,7 @@ func (s *Server) RecoverFormatJob(ctx context.Context) {
return return
} }
s.logger.Warn("format-job recover: re-running interrupted format detached", "durable_id", job.DurableID, "device", device, "fstype", job.FSType, "blank", job.Blank) s.logger.Warn("format-job recover: re-running interrupted format detached", "durable_id", job.DurableID, "device", device, "fstype", job.FSType, "blank", job.Blank)
_ = s.startFormatDetached(device, job.DurableID, job.FSType, job.Blank) // detached; updates the record on completion _, _ = s.startFormatDetached(device, job.DurableID, job.FSType, job.Blank) // detached; updates the record on completion
} }
// nowFn returns the server clock (testable), defaulting to time.Now. // nowFn returns the server clock (testable), defaulting to time.Now.
+31 -17
View File
@@ -154,12 +154,15 @@ func (s *TokenStore) Mint(vmid int) (string, error) {
// looks it up; the per-candidate comparison is constant-time to avoid a timing oracle on the // looks it up; the per-candidate comparison is constant-time to avoid a timing oracle on the
// stored hash. ok is false for an unknown/empty token. // stored hash. ok is false for an unknown/empty token.
// //
// Reload-on-miss (B3): the store FILE is shared across processes — the one-shot provisioner // Reload-on-change (B3, R-269): the store FILE is shared across processes — the one-shot
// (`--selftest=provision`) Mints into it while the long-lived daemon serves Lookup from an index // provisioner (`--selftest=provision`) Mints into it while the long-lived daemon serves Lookup from
// built at open. On a miss, re-read the file ONCE and re-check, so a token minted after this // an index built at open. Every Lookup stats the file first and re-reads it when the append-only
// process started authorizes without a daemon restart (the drill's fresh-install 401). The // log has grown, BEFORE answering — so a token minted elsewhere authorizes without a restart AND a
// append-only log makes an unchanged file size proof of no new records, so a genuinely unknown // token rotated out elsewhere stops authorizing on its very next presentation. (Before R-269 the
// token costs at most one stat once the index is current — never a reload loop. // re-read ran only on a MISS, so a superseded token was a direct map hit and kept authorizing until
// some unrelated miss forced the reload.) An unchanged size is proof of no new records, so the
// steady state costs one stat per call and never a reload loop. Pinned by
// TestTokenStore_RotatedOutTokenRejectedFirst.
func (s *TokenStore) Lookup(token string) (int, bool) { func (s *TokenStore) Lookup(token string) (int, bool) {
if token == "" { if token == "" {
return 0, false return 0, false
@@ -167,23 +170,34 @@ func (s *TokenStore) Lookup(token string) (int, bool) {
want := hashToken(token) want := hashToken(token)
s.mu.Lock() s.mu.Lock()
defer s.mu.Unlock() defer s.mu.Unlock()
// Direct map hit is the common path; the constant-time compare guards against a timing st, statErr := os.Stat(s.path)
// side-channel by re-checking the matched key (map lookup itself is not the secret-bearing if statErr == nil && st.Size() != s.loadedSize {
// comparison — the hash of a random 256-bit token is not feasibly guessable regardless). // The log changed under us (another process minted/rotated): converge first, then answer.
if vmid, ok := s.byHash[want]; ok { s.reloads++
if subtle.ConstantTimeCompare([]byte(want), []byte(s.byVMID[vmid])) == 1 { if err := s.reloadLocked(); err != nil {
return vmid, true return 0, false // unreadable store: fail closed, never crash the auth path
} }
return s.matchLocked(want)
} }
// Miss: skip the re-read when the append-only log has not grown (nothing new to see). if vmid, ok := s.matchLocked(want); ok {
// A stat error falls through to the reload, which handles a missing file as empty. return vmid, true
if st, err := os.Stat(s.path); err == nil && st.Size() == s.loadedSize {
return 0, false
} }
if statErr == nil {
return 0, false // file unchanged since the last (re)load: genuinely unknown
}
// Stat failed (e.g. the file vanished): reload, which treats a missing file as empty.
s.reloads++ s.reloads++
if err := s.reloadLocked(); err != nil { if err := s.reloadLocked(); err != nil {
return 0, false // unreadable store: fail closed, never crash the auth path return 0, false
} }
return s.matchLocked(want)
}
// matchLocked answers from the in-memory index. Direct map hit is the common path; the
// constant-time compare re-checks the matched key against the guest's CURRENT hash (map lookup
// itself is not the secret-bearing comparison — the hash of a random 256-bit token is not
// feasibly guessable regardless). Caller holds the mutex.
func (s *TokenStore) matchLocked(want string) (int, bool) {
if vmid, ok := s.byHash[want]; ok { if vmid, ok := s.byHash[want]; ok {
if subtle.ConstantTimeCompare([]byte(want), []byte(s.byVMID[vmid])) == 1 { if subtle.ConstantTimeCompare([]byte(want), []byte(s.byVMID[vmid])) == 1 {
return vmid, true return vmid, true
+45
View File
@@ -210,6 +210,51 @@ func TestTokenStore_ReloadOnMiss_RemintCoherence(t *testing.T) {
} }
} }
// R-269: a token rotated out by ANOTHER process must stop authorizing on its very next
// presentation — with NO intervening lookup of the new token. This is the order the operator hits
// after rotating a leaked token: the leaked one is presented first. RemintCoherence above looks the
// NEW token up first, and that miss is what used to evict the old hash, so it passed while the leaked
// token kept returning HTTP 200 on hardware (2026-08-09) until something unrelated forced a reload.
//
// RED-PROOF: restore the reload-on-MISS-only Lookup (answer a map hit before stat-ing the file) and
// this fails with "rotated-out token still authorizes".
func TestTokenStore_RotatedOutTokenRejectedFirst(t *testing.T) {
path := filepath.Join(t.TempDir(), "tokens.log")
daemon, err := OpenTokenStore(path)
if err != nil {
t.Fatalf("open daemon store: %v", err)
}
defer daemon.Close()
minter, err := OpenTokenStore(path)
if err != nil {
t.Fatalf("open minter store: %v", err)
}
defer minter.Close()
old, err := minter.Mint(130)
if err != nil {
t.Fatalf("mint old: %v", err)
}
if vmid, ok := daemon.Lookup(old); !ok || vmid != 130 { // the daemon has learned the old token
t.Fatalf("old token before rotation: (%d,%v), want (130,true)", vmid, ok)
}
fresh, err := minter.Mint(130) // rotation, written by another process
if err != nil {
t.Fatalf("mint fresh: %v", err)
}
if vmid, ok := daemon.Lookup(old); ok { // the leaked token FIRST
t.Fatalf("rotated-out token still authorizes vmid %d on its first presentation after rotation — "+
"Mint's 'any previous token for this guest is revoked' is false across processes (R-269)", vmid)
}
if vmid, ok := daemon.Lookup(fresh); !ok || vmid != 130 {
t.Fatalf("fresh token after rotation: (%d,%v), want (130,true)", vmid, ok)
}
if vmid, ok := daemon.Lookup(old); ok {
t.Fatalf("rotated-out token authorizes vmid %d after the fresh one was seen", vmid)
}
}
// §8 edge: the store file deleted between open and a miss — reload treats it as empty; Lookup // §8 edge: the store file deleted between open and a miss — reload treats it as empty; Lookup
// fails closed, no crash. // fails closed, no crash.
func TestTokenStore_ReloadOnMiss_MissingFile(t *testing.T) { func TestTokenStore_ReloadOnMiss_MissingFile(t *testing.T) {
+2 -2
View File
@@ -60,8 +60,8 @@ func TestLiveReporter_CoordPresentWithoutPriorVerify(t *testing.T) {
if h.PBS == nil { if h.PBS == nil {
t.Fatal("pbs coord absent despite a reachable PBS — the gap this fixes") t.Fatal("pbs coord absent despite a reachable PBS — the gap this fixes")
} }
if h.PBS.RepoID != "felhom-pbs" || h.PBS.Namespace != "root" || h.PBS.LatestSnapshotID != "9201" { if h.PBS.RepoID != "felhom-pbs" || h.PBS.Namespace != hub.PBSRootNamespace || h.PBS.LatestSnapshotID != "9201" {
t.Errorf("pbs coord = %+v, want felhom-pbs/root/9201", h.PBS) t.Errorf("pbs coord = %+v, want felhom-pbs, the root namespace (\"\", R-124), 9201", h.PBS)
} }
// COMPANION (pre-fix): the bare SnapshotStore (no live read) with an empty store omits pbs. // COMPANION (pre-fix): the bare SnapshotStore (no live read) with an empty store omits pbs.
+6
View File
@@ -57,6 +57,10 @@ type DeviceProbe struct {
HasPartitions bool `json:"has_partitions"` // child partitions present (lsblk) HasPartitions bool `json:"has_partitions"` // child partitions present (lsblk)
Mounted bool `json:"mounted"` // currently mounted somewhere Mounted bool `json:"mounted"` // currently mounted somewhere
FSType string `json:"fstype,omitempty"` FSType string `json:"fstype,omitempty"`
// FSUUID is the filesystem UUID blkid read from the on-disk superblock (`blkid -p`, no cache),
// "" when there is none. R-25: the format path reports it so the caller mounts the filesystem the
// agent just made, not whatever a /dev path resolves to later.
FSUUID string `json:"fs_uuid,omitempty"`
} }
// DataBearing is the conservative verdict: any signature / partition table / partition / mount — // DataBearing is the conservative verdict: any signature / partition table / partition / mount —
@@ -423,6 +427,8 @@ func (h *SudoHostOps) InspectDevice(ctx context.Context, device string) (DeviceP
probe.FSType = v probe.FSType = v
case "PTTYPE": case "PTTYPE":
probe.HasPartitionTable = true probe.HasPartitionTable = true
case "UUID":
probe.FSUUID = v
case "USAGE": case "USAGE":
if v != "" { if v != "" {
probe.HasFilesystem = true // filesystem/raid/crypto member = data-bearing probe.HasFilesystem = true // filesystem/raid/crypto member = data-bearing
+14
View File
@@ -208,3 +208,17 @@ func TestFormat_RejectsBadArgs(t *testing.T) {
t.Fatalf("mkfs ran despite invalid input: %v", r.calls) t.Fatalf("mkfs ran despite invalid input: %v", r.calls)
} }
} }
// R-25: the probe carries the superblock's filesystem UUID so the format path can report the new one.
func TestInspect_ReadsFilesystemUUID(t *testing.T) {
r := &scriptedRunner{
outputs: map[string][]byte{
"blkid": []byte("DEVNAME=/dev/sdb\nUUID=0fc63daf-8483-4772-8e79-3d69d8477de4\nTYPE=ext4\nUSAGE=filesystem\n"),
"lsblk": []byte(`{"blockdevices":[{"name":"sdb","fstype":"ext4","pttype":null,"mountpoint":null}]}`),
},
}
p, _ := newSudo(r).InspectDevice(context.Background(), "/dev/sdb")
if p.FSUUID != "0fc63daf-8483-4772-8e79-3d69d8477de4" {
t.Fatalf("FSUUID = %q, want the blkid UUID", p.FSUUID)
}
}
+59
View File
@@ -0,0 +1,59 @@
#!/usr/bin/env python3
"""build-step-bundle.py — the TRANSITION bundle for a release whose bundle ADDS a path (R-880, `11` §5.4.2).
Usage: python3 scripts/build-step-bundle.py <base-bundle.json> <step-version> <out.json> (prints the sha256)
WHY. A box's INSTALLED felhom-os-apply checks every path of an incoming bundle against ITS OWN table (rule R16) — so a
bundle that adds a path (v0.146.1 adds felhom-priv-apply, the guest hook and the shared-parent files, R-861) is refused
by every box still running an older wrapper. The fix is a step: first a bundle the old wrapper accepts that brings ONLY
the new felhom-os-apply (the new table), then the release's own bundle, which the new wrapper accepts.
WHAT IT BUILDS. <base-bundle.json> is the bundle the boxes run now (download it from the package registry, e.g.
felhom-agent/0.145.0/felhom-config-bundle.json, and check its sha against the hub's record). The step bundle is that
bundle with EXACTLY ONE change: the /usr/local/sbin/felhom-os-apply entry's content is replaced by configs/felhom-os-apply
(this tree). Same paths, same modes, same checks, every other byte identical; agent_version is <step-version> (e.g.
0.146.1-step1). The old wrapper verifies it like any bundle (signature, sha, R16, content checks, self-check of the new
wrapper) — nothing about the trust route changes.
Pinned by configs/test_felhom_config_bundle.py (StepBundle): same paths as the base, only the wrapper differs, the
new wrapper's table is a superset of the base's paths.
"""
import base64
import hashlib
import json
import pathlib
import re
import sys
REPO = pathlib.Path(__file__).resolve().parent.parent
OSAPPLY_DEST = "/usr/local/sbin/felhom-os-apply"
def build_step(base_bytes, version, new_osapply_bytes):
if not re.match(r"^[0-9]+\.[0-9]+\.[0-9]+-[0-9A-Za-z.]+$", version):
raise SystemExit(f"build-step-bundle: {version!r} must be a semver with a step suffix, e.g. 0.146.1-step1")
base = json.loads(base_bytes)
files = base.get("files")
if base.get("format") != 1 or not isinstance(files, list):
raise SystemExit("build-step-bundle: the base is not a format-1 bundle")
hit = [e for e in files if e.get("path") == OSAPPLY_DEST]
if len(hit) != 1:
raise SystemExit(f"build-step-bundle: the base has {len(hit)} {OSAPPLY_DEST} entries, want exactly 1")
hit[0]["content_b64"] = base64.b64encode(new_osapply_bytes).decode()
hit[0]["sha256"] = hashlib.sha256(new_osapply_bytes).hexdigest()
base["agent_version"] = version
return (json.dumps(base, indent=1, sort_keys=True) + "\n").encode()
def main(argv):
if len(argv) != 4:
print(__doc__, file=sys.stderr)
return 2
data = build_step(pathlib.Path(argv[1]).read_bytes(), argv[2], (REPO / "configs" / "felhom-os-apply").read_bytes())
pathlib.Path(argv[3]).write_bytes(data)
print(hashlib.sha256(data).hexdigest())
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv))
+7 -10
View File
@@ -10,13 +10,11 @@
"CI went red at a commit whose own run had been green the day before, on a true finding that", "CI went red at a commit whose own run had been green the day before, on a true finding that",
"no one could act on. The red will return at the next publish unless the two read one number.", "no one could act on. The red will return at the next publish unless the two read one number.",
"", "",
"HOW THE NUMBER WAS ARRIVED AT — stated honestly, because it is weaker than it looks.", "HOW THE NUMBER WAS ARRIVED AT. generic_versions_kept is 10 because that is what the registry",
"generic_versions_kept is 10 because that is what the registry demonstrably holds today", "keeps: the deleter was ESTABLISHED on 2026-08-10 (R-287) — the newest-10 prune of generic packages",
"(felhom-agent 0.121.0..0.128.0 = 10 versions, queried 2026-08-09). It is an OBSERVED state,", "run under R-267 (felhom-agent and felhom-golden generic held exactly 10 afterwards). Until then this",
"NOT a ruling anyone has been able to locate: no register row records a package prune, R-210", "file called the 10 an observed state with no located ruling; that is superseded (corrected",
"is WAITING-ON-OPERATOR and says 'Nothing was deleted; this is a list, not an action', and it", "2026-10-05, R-291). Container packages are not pruned by that rule.",
"concerns local Docker images rather than this registry. Container packages currently hold 19",
"each, so there is no uniform ten-per-package cap visible either. See R-287.",
"", "",
"SO THIS FILE IS A FLOOR, NOT A LICENCE. It says: CI may assume nothing older than the newest", "SO THIS FILE IS A FLOOR, NOT A LICENCE. It says: CI may assume nothing older than the newest",
"N generic versions is still downloadable. It does NOT authorise deleting anything, and the", "N generic versions is still downloadable. It does NOT authorise deleting anything, and the",
@@ -36,9 +34,8 @@
], ],
"generic_versions_kept": 10, "generic_versions_kept": 10,
"readers": [ "readers": [
"scripts/check-published-versions.py — bounds its assertion to the newest N versions", "scripts/check-published-versions.py — bounds its assertion to the newest N versions"
"documentation/runbooks/registry-retention.md (felhom.eu) — the prune procedure"
], ],
"recorded": "2026-08-09", "recorded": "2026-08-09",
"recorded_by": "CC, from the registry's observed state; NOT from a located operator ruling" "recorded_by": "CC 2026-08-09; the number's source (the R-267 newest-10 prune, established 2026-08-10 by R-287) recorded 2026-10-05 (R-291)"
} }