slice 10B: operator-signed destructive completion (offline key + signing CLI) (v0.16.0)

A destructive op runs ONLY on a pinned-key-verified, nonce-fresh, in-window,
host-bound, durable-id-bound operator signature. New cmd/felhom-opsign signs
canonical OpBlobs offline via ssh-keygen -Y sign (hardware-ready); the signing
key is never in the hub or agent. New internal/signedjobs runner verifies each
queued blob through the gate and only on all-pass runs the WipeExecutor, which
re-resolves the DURABLE device id + re-inspects (8C) before mkfs — closing the
8C data-bearing-wipe pending_signature gap. New storage durable-device
resolution; authz.CanonicalBlob promoted to production. Real-crypto tests assert
valid executes and forged/replay/expired/retarget/non-pinned are rejected
(executor never called).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-10 20:14:16 +02:00
parent 8ecf8929fb
commit 588fed2aa9
19 changed files with 1523 additions and 68 deletions
+47
View File
@@ -3,6 +3,53 @@
All notable changes to **felhom-agent** are recorded here. Update on every code
change that gets pushed.
## v0.16.0 — slice 10B: operator-signed destructive completion (offline key + signing CLI) (2026-06-10)
The security centerpiece: a destructive op runs ONLY on a verified, operator-signed authorization
— signature valid against a **pinned** operator pubkey (never the hub's or the blob's), nonce
unseen + durably burned, in-window, host-bound, and **resource-bound to a DURABLE device id** that
execution re-resolves + re-inspects. Decision (a): **offline operator key + signing CLI**,
hardware-key-ready (`sk-`/YubiKey via ssh-keygen). The key floor holds: the signing key is NOT in
the hub and NOT in the agent. Concrete consumer: this **closes the 8C data-bearing-wipe
`pending_signature` gap**. Pairs with hub v0.10.0.
### Added
- **`cmd/felhom-opsign`** — the operator's offline signing CLI. Builds the canonical `OpBlob` by
**reusing `authz.CanonicalBlob`** (the exact production path the verifier authenticates over — so
signer + verifier can never drift) and signs it with **`ssh-keygen -Y sign -n felhom-op-v1`**
(hardware-ready). Output: a `{op_blob_b64, sig_armored}` envelope to hand to the hub jobs queue
(optional `--upload`). Touches ONLY the operator's signing key.
- **`authz.CanonicalBlob`** — promoted to production (was test-only) so the CLI + verifier share one
canonical-bytes source; params canonicalized (sorted keys, compact).
- **`internal/storage` durable device identity** (`durable_device.go`): `DeviceDurableID` (derive a
stable `byid:`(wwn/serial)/`byuuid:` id from the world-readable udev symlinks — no privilege, no
subprocess) + `ResolveDurableDevice` (re-resolve to the current `/dev` path; a path-only/unknown
scheme is REFUSED). The resource-level anti-retarget.
- **`internal/signedjobs`** (new): the queue consumer. `Runner` fetches each opaque job → runs it
through the **gate** (the LOCKED authz pipeline) → on all-pass hands the verified op to an
`Executor`; the order is **verify → nonce-burn (durable, in Verify) → execute → clear job**. The
**`WipeExecutor`** is the 8C consumer: resolve the signed durable id → **re-derive + match**
(anti-retarget) → **re-inspect (8C classifier)** the device is still the data-bearing target →
`mkfs`. A vanished/changed/non-data-bearing device or a path-only binding is refused **even with a
valid signature**. Wired as a second `EnvelopeObserver` (runs on `HasSignedOps`).
- **`hub.Client.Jobs` / `CompleteJob`** + `hub.MultiObserver`; the 8C format refusal now **surfaces
the bound op** (op + durable id + host) in its 403 `pending_op` + a `felhom-opsign …` hint.
### Pinning / rotation
- Operator pubkeys are pinned via `authz.signers` (config, trusted path — provision/agent config,
NEVER hub-alone), **multiple** keys (KeyID selects; role-scoped), so a backup/rotation key exists
without a flag-day. Unchanged from the slice-4 verifier wiring; 10B activates the execute path.
### Tests (real crypto, non-hollow)
- `signedjobs` runner over the **real** gate+verifier (in-Go minted SSHSIGs): valid → executor runs
once + job cleared; **replay** (nonce burned) / **non-pinned signer** / **expired** / **retarget**
(other host) / **forged sig** / **no pinned signer** → all rejected, **executor never called**;
malformed envelope cleared.
- `WipeExecutor`: valid → `mkfs` runs; **path-only**, **durable-id mismatch**, **device gone**,
**re-inspect non-data-bearing**, **not-probed** → all refused, `Format` not called.
- `storage` durable: wwn-preference, uuid-fallback, path-only/traversal refusal, round-trip,
missing-device error (symlink tests gated to Linux — the agent's OS).
## v0.15.0 — slice 10A: hub desired-state serving — the "Down" channel (2026-06-10)
The agent half of slice 10A. The control envelope (`hub.ControlEnvelope`) stops being "reserved — ignored" and becomes the live **Down channel**: a cheap change-notification on every heartbeat. The agent caches the hub's desired-state + its generation; only when **`DesiredGeneration` advances** does it fetch the full state (the heartbeat stays light, the heavy state moves on change). The engine then reconciles **benign** deltas and the gate marks an explicit **destructive** delta `pending_signature` (no signer in 10A → never executed; signed execution is 10B). Pairs with hub v0.9.0.
+47 -45
View File
@@ -1,61 +1,63 @@
# REPORT — slice 10A (agent half): hub desired-state serving — the "Down" channel (v0.15.0) (2026-06-10)
# REPORT — slice 10B (agent half): operator-signed destructive completion (v0.16.0) (2026-06-10)
> Overwrite-latest report. Cumulative history: [CHANGELOG.md](CHANGELOG.md).
## What was implemented
The agent half of **slice 10A**: activate the control envelope as the live **Down channel** and feed
a hub-backed desired-state into the reconcile engine. Pairs with hub v0.9.0.
The security centerpiece: a destructive op runs ONLY on a verified, operator-signed authorization.
Decision (a): **offline operator key + signing CLI**, hardware-key-ready. The signing key is NOT in
the hub and NOT in the agent. Concrete consumer: this **closes the 8C data-bearing-wipe
`pending_signature` gap**. Pairs with hub v0.10.0.
### The control loop (now live)
report (heartbeat) → control envelope → (DesiredGeneration advanced past cache? fetch desired-state)
→ reconcile benign / gate destructive → report. The heartbeat stays light; the heavy desired-state is
fetched **only on a generation advance**.
### The flow (end-to-end)
8C format of a data-bearing device → agent refuses `pending_signature` + **surfaces the bound op**
(durable id + host) → operator **signs offline** (`felhom-opsign`) → uploads to the hub jobs queue →
agent's signed-jobs runner **verifies** + **executes** the wipe (re-resolve durable id + re-inspect
8C → `mkfs`) → clears the job.
### `internal/reconcile`
- **`DesiredGuest.Decommission`** — the canonical **destructive** desired-state delta (an EXPLICIT
flag, never "absent from the list", so a partial hub list can't mass-destroy). Planner emits
`ActionDecommission``ClassDecommission` → Destructive → the gate refuses `pending_signature`.
- **`Reconcile`** now counts a `pending_signature` gate refusal as **`Result.Pending`** (expected,
INFO-logged), not a failure; any other refusal stays a real failure. `ActionDecommission` has **no
executor** (10B) — a defensive guard refuses to run it.
- **`CachingProvider`** — thread-safe DesiredState + generation cache (`Desired`/`Update`/
`Generation`); the production provider, replacing `EmptyProvider` in the daemon engine. Empty until
the hub serves intent → cold-start is a live no-op (unchanged behaviour).
### `cmd/felhom-opsign` (new) — the offline signing CLI
- Builds the canonical `OpBlob` by **reusing `authz.CanonicalBlob`** (the exact bytes the verifier
authenticates over — signer/verifier can't drift) and signs with **`ssh-keygen -Y sign -n
felhom-op-v1`** (hardware-ready: `sk-`/YubiKey work unchanged). Output: `{op_blob_b64,
sig_armored}`; optional `--upload`. Touches only the operator's key.
### `internal/hub`
- `ControlEnvelope` fields are now active. New wire types **`DesiredStateResponse`** +
**`WireDesiredState`** (guests + forward-compat `restore_directive` (10D) / `pbs_namespace` / opaque
`storage_manifest`+`backup_policy`) + **`WireDesiredGuest`**. New **`Client.FetchDesiredState`**
(GET `/api/v1/hosts/{host_id}/desired-state`, self-scoped to the client's own host). New
**`EnvelopeObserver`** loop seam + `SetEnvelopeObserver` (hub does not import reconcile/desired).
### The verify-and-execute machinery
- **`internal/signedjobs.Runner`** — fetches each opaque job → runs the **gate** (the LOCKED authz
pipeline: pinned-key SSHSIG → namespace → allow-list by key MATERIAL → crypto over raw bytes →
host target → time window → **durable nonce-burn LAST**) → on all-pass hands the verified op to an
`Executor`. Order: **verify → burn nonce (durable) → execute → clear**. Rejects (forged/replayed/
expired/retargeted/non-pinned) never reach the executor.
- **`WipeExecutor`** (the 8C consumer) — resolve the signed **durable** id → re-derive + **match**
(anti-retarget) → **re-inspect (8C classifier)** still-data-bearing → `mkfs`. A path-only binding,
a vanished/changed device, or a non-data-bearing target is refused **even with a valid signature**.
- **`internal/storage` durable identity** — `DeviceDurableID` / `ResolveDurableDevice` over the
world-readable udev symlinks (`byid:` wwn/serial, `byuuid:` fallback) — no privilege, no subprocess.
- **`authz.CanonicalBlob`** promoted to production. `hub.Client.Jobs`/`CompleteJob` + `MultiObserver`.
The 8C 403 now carries a `pending_op` (op + durable id + host) + a `felhom-opsign` hint.
### `internal/desired` (new) + wiring
- **`Syncer`** — implements `hub.EnvelopeObserver`; fetches on a generation advance, maps wire→domain,
updates the `CachingProvider`. Caches the **fetched** generation (race-robust); a fetch failure keeps
the last-known state. `restore_directive` carried + logged, not acted on (10D). Wired in
`cmd/felhom-agent`: provider → engine, syncer → loop.
### Pinning / rotation
Operator pubkeys pinned via `authz.signers` (config, trusted path — NEVER hub-alone), **multi-key**
(KeyID selects, role-scoped) for backup/rotation without a flag-day. Unchanged verifier wiring; 10B
activates the execute path (the runner is the second `EnvelopeObserver`, runs on `HasSignedOps`).
## Tests (all green)
- reconcile: **benign applied + destructive decommission gated pending (not executed)**; Plan
decommission-only + classifies Destructive; CachingProvider update/isolation.
- desired: **fetch-once-on-advance** / no-refetch-on-unchanged / fetch-failure-keeps-cache /
caches-the-fetched-generation.
- hub: `FetchDesiredState` path+auth+decode (incl. `restore_directive`) + typed 403; loop notifies the
observer + adopts `PollIntervalSeconds`, skips the observer on a report error.
- cross-repo golden (`desired-state` + `control-envelope`) decode + key-set guard, byte-identical with
felhom.eu/hub. `go test ./...` green.
## Tests (real crypto, non-hollow — assert the op did/did NOT run)
- `signedjobs` over the **real** gate+verifier (in-Go minted SSHSIGs): valid → executor runs once +
job cleared; **replay / non-pinned / expired / retarget / forged / no-signer** → rejected, executor
never called; malformed cleared.
- `WipeExecutor`: valid → `mkfs`; path-only / durable-mismatch / device-gone / re-inspect-non-data-
bearing / not-probed → refused, `Format` not called.
- `storage` durable: wwn-preference, uuid-fallback, path-only+traversal refusal, round-trip, missing
(symlink tests gated to Linux). `go test ./...` green.
## Versioning / docs
- Version `0.14.0 → 0.15.0`; `CHANGELOG.md` updated. Doc 03 §4 (control loop live) + §9 (slice table:
10A done, 10B/10C/10D pending) updated.
- Version `0.15.0 → 0.16.0`; `CHANGELOG.md`. Doc 03 §4 (signed path live) + §6 (8C wipe completes) +
§9 (10B done) updated.
## Out of scope (per the task)
- Signed-op **execution** (verify + run the gated destructive op) → 10B (10A marks it pending only).
- **Restore-mode / re-enroll** consumption (a new box's first directive) → 10D; 10A serves
already-authenticated hosts only.
- Other destructive executors (guest_destroy, decommission, **restore-overwrite → 10D**) reuse the
same gate+runner; their executors plug in per-slice. 10B ships the machinery + the storage-wipe.
## Pending
- **Live validation** on the demo: build+deploy agent v0.15.0 + hub v0.9.0; admin-set a desired-state
with a benign + a decommission delta → generation bumps → agent fetches → reconciles benign + gates
the decommission; change `poll_interval_seconds`; confirm a host can't fetch another host's state.
- **Live validation** on the demo: data-bearing wipe → `pending_signature` → sign offline with a real
operator key → hub queue → agent verifies + wipes; confirm replay + a non-pinned-key signature are
rejected. (Also validates the Linux-only durable-device tests + ssh-keygen interop.)
+18 -2
View File
@@ -34,6 +34,7 @@ import (
applog "gitea.dooplex.hu/admin/felhom-agent/internal/log"
"gitea.dooplex.hu/admin/felhom-agent/internal/pbs"
"gitea.dooplex.hu/admin/felhom-agent/internal/provision"
"gitea.dooplex.hu/admin/felhom-agent/internal/signedjobs"
"gitea.dooplex.hu/admin/felhom-agent/internal/proxmox"
"gitea.dooplex.hu/admin/felhom-agent/internal/reconcile"
"gitea.dooplex.hu/admin/felhom-agent/internal/storage"
@@ -41,7 +42,7 @@ import (
// version is the agent version. Overridable at build time with
// -ldflags "-X main.version=<v>"; defaults to the in-repo CHANGELOG version.
var version = "0.15.0"
var version = "0.16.0"
func main() {
var (
@@ -245,7 +246,9 @@ func runDaemon(cfg config.Config, logger *slog.Logger) int {
desiredProvider := reconcile.NewCachingProvider()
// The "Down" channel sync hook: on each heartbeat, fetch desired-state when the generation
// advances. The loop calls it via the EnvelopeObserver seam (hub does not import desired).
loop.SetEnvelopeObserver(desired.NewSyncer(client, desiredProvider, logger))
desiredSyncer := desired.NewSyncer(client, desiredProvider, logger)
// The signed-jobs runner (slice 10B) is wired as a SECOND envelope observer below (after the
// gate is built) — when the heartbeat flags pending signed ops, it fetches + verifies + executes.
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
@@ -287,6 +290,18 @@ func runDaemon(cfg config.Config, logger *slog.Logger) int {
}
gate := reconcile.NewGate(verifier, cfg.Hub.HostID, reconcile.SlogAudit{Logger: logger}, logger)
// Signed-jobs runner (slice 10B): the consumer of the hub's signed-jobs queue. On a heartbeat
// that flags pending signed ops, it fetches each opaque blob, runs it through the gate (the
// LOCKED authz pipeline: pinned-key SSHSIG → namespace → allow-list → crypto → host → time →
// durable nonce-burn) and, only on all-pass, hands the verified op to the storage-WIPE executor
// — which re-resolves the DURABLE device id + re-inspects (8C) before mkfs. This closes the 8C
// data-bearing `pending_signature` gap. With no signers pinned the gate refuses every job
// (pending_signature) and nothing executes — correct. Wired as a second envelope observer
// alongside the desired-state syncer.
wipeExec := signedjobs.NewWipeExecutor(hostOps, logger)
jobsRunner := signedjobs.NewRunner(client, gate, wipeExec, cfg.Hub.HostID, logger)
loop.SetEnvelopeObserver(hub.MultiObserver(desiredSyncer, jobsRunner))
// Storage watchdog (slice 5): the third daemon goroutine. Fast-polls the known target
// set for attached↔disconnected transitions → debounced out-of-band report; and, on a
// known mount-backed target's device returning unmounted, dispatches a benign re-mount
@@ -510,6 +525,7 @@ func buildLocalAPIServer(cfg config.Config, px *proxmox.Client, store *backup.St
// Host metrics (slice 9): the shared collector serves GET /host/metrics — a fresh host +
// per-storage view to the customer's monitoring page (reuses the slice-4 collector).
HostMetrics: collector,
HostID: cfg.Hub.HostID, // slice 10B: anti-retarget host in the data-bearing-format pending-op
Logger: logger,
})
if err != nil {
+186
View File
@@ -0,0 +1,186 @@
// Command felhom-opsign is the OPERATOR's offline signing CLI for destructive ops (slice 10B,
// decision (a): offline operator key + signing CLI, hardware-key-ready).
//
// It constructs the canonical OpBlob bytes by REUSING internal/authz.CanonicalBlob — the exact
// production path the agent's verifier authenticates over — so signer and verifier can never drift.
// It signs that canonical message with the operator's key via `ssh-keygen -Y sign`, which makes it
// hardware-ready: an `sk-`/YubiKey key works through ssh-keygen unchanged. The output is a signed-op
// envelope { op_blob_b64, sig_armored } to hand to the hub's jobs queue (optionally uploaded with
// --upload). This CLI touches ONLY the operator's signing key — never the hub's or agent's keys.
//
// Example — sign a data-bearing wipe (closing the 8C pending_signature gap):
//
// felhom-opsign -op storage_wipe -host demo-felhom-01 \
// -durable-id byid:wwn-0x5000c500abcd1234 -fstype ext4 \
// -key-id ops-key-1 -key ~/.ssh/felhom_op_ed25519 -ttl 30m
package main
import (
"bytes"
"crypto/rand"
"encoding/base64"
"encoding/hex"
"encoding/json"
"flag"
"fmt"
"io"
"net/http"
"os"
"os/exec"
"path/filepath"
"strings"
"time"
"gitea.dooplex.hu/admin/felhom-agent/internal/authz"
)
func main() {
if err := run(); err != nil {
fmt.Fprintln(os.Stderr, "felhom-opsign:", err)
os.Exit(1)
}
}
func run() error {
var (
op = flag.String("op", "", "op class to sign, e.g. storage_wipe | guest_destroy | decommission")
host = flag.String("host", "", "target host_id (anti-retarget — the op runs ONLY on this host)")
guest = flag.String("guest", "", "target guest_id (\"\" = host-scoped op)")
keyID = flag.String("key-id", "", "key id of the signing key (must match a pinned agent signer)")
paramsRaw = flag.String("params", "", "op params as JSON (overrides -durable-id/-fstype)")
durableID = flag.String("durable-id", "", "for storage_wipe: the DURABLE device id (byid:…|byuuid:…)")
fstype = flag.String("fstype", "ext4", "for storage_wipe: the filesystem to mkfs after wipe")
keyFile = flag.String("key", "", "operator signing key (ssh private key / sk- key handle) for ssh-keygen -Y sign")
ttl = flag.Duration("ttl", 30*time.Minute, "validity window from now (issued_at..expires_at)")
nonce = flag.String("nonce", "", "explicit nonce (default: a fresh 128-bit random nonce)")
uploadURL = flag.String("upload", "", "optional hub base URL to POST the signed op to the jobs queue")
hubKey = flag.String("hub-key", "", "hub bearer key for --upload (operator's hub key — NOT a signing key)")
out = flag.String("o", "", "write the envelope JSON to this file (default: stdout)")
)
flag.Parse()
if *op == "" || *host == "" || *keyID == "" || *keyFile == "" {
return fmt.Errorf("-op, -host, -key-id and -key are required")
}
// Params: explicit JSON, or built from the wipe convenience flags.
params := strings.TrimSpace(*paramsRaw)
if params == "" {
if *op == "storage_wipe" {
if *durableID == "" {
return fmt.Errorf("storage_wipe needs -durable-id (byid:…|byuuid:…) — a path-only binding is refused by the agent")
}
pj, _ := json.Marshal(map[string]string{"durable_id": *durableID, "fstype": *fstype})
params = string(pj)
} else {
params = "{}"
}
}
n := *nonce
if n == "" {
var b [16]byte
if _, err := rand.Read(b[:]); err != nil {
return fmt.Errorf("generating nonce: %w", err)
}
n = hex.EncodeToString(b[:])
}
now := time.Now().UTC()
issued := now
expires := now.Add(*ttl)
// Canonical OpBlob bytes — the EXACT bytes the agent verifier authenticates over.
blob, err := authz.CanonicalBlob(*op, *host, *guest, *keyID, n, params, issued, expires)
if err != nil {
return err
}
sigArmored, err := signWithSSHKeygen(blob, *keyFile)
if err != nil {
return err
}
env := map[string]string{
"op_blob_b64": base64.StdEncoding.EncodeToString(blob),
"sig_armored": sigArmored,
}
envJSON, _ := json.Marshal(env)
// Emit the envelope (stdout or file). Also print the human summary to stderr (never the key).
fmt.Fprintf(os.Stderr, "signed: op=%s host=%s guest=%q key_id=%s nonce=%s expires=%s\n",
*op, *host, *guest, *keyID, n, expires.Format(time.RFC3339))
if *out != "" {
if err := os.WriteFile(*out, envJSON, 0o600); err != nil {
return err
}
fmt.Fprintln(os.Stderr, "wrote envelope to", *out)
} else {
fmt.Println(string(envJSON))
}
if *uploadURL != "" {
if *hubKey == "" {
return fmt.Errorf("--upload needs --hub-key (the operator's hub bearer key)")
}
if err := upload(*uploadURL, *hubKey, *host, envJSON); err != nil {
return fmt.Errorf("upload to hub: %w", err)
}
fmt.Fprintln(os.Stderr, "uploaded signed op to the hub jobs queue")
}
return nil
}
// signWithSSHKeygen signs `message` with `ssh-keygen -Y sign -n <namespace>`, the hardware-ready
// path (sk-/YubiKey keys work unchanged). It writes the message to a temp file, runs ssh-keygen,
// and reads the armored SSHSIG it produces. The namespace is the agent's FIXED domain separator.
func signWithSSHKeygen(message []byte, keyFile string) (string, error) {
dir, err := os.MkdirTemp("", "felhom-opsign-")
if err != nil {
return "", err
}
defer os.RemoveAll(dir)
msgPath := filepath.Join(dir, "op.blob")
if err := os.WriteFile(msgPath, message, 0o600); err != nil {
return "", err
}
cmd := exec.Command("ssh-keygen", "-Y", "sign", "-n", authz.Namespace, "-f", keyFile, msgPath)
var stderr bytes.Buffer
cmd.Stderr = &stderr
if err := cmd.Run(); err != nil {
return "", fmt.Errorf("ssh-keygen -Y sign: %w: %s", err, strings.TrimSpace(stderr.String()))
}
sig, err := os.ReadFile(msgPath + ".sig")
if err != nil {
return "", fmt.Errorf("reading produced signature: %w", err)
}
return string(sig), nil
}
// upload POSTs the signed-op envelope to the hub's jobs queue (POST /api/v1/admin/hosts/{id}/jobs).
// The queued blob is base64(envelope JSON); the hub stores it opaquely (it cannot forge or open it).
func upload(baseURL, hubKey, hostID string, envJSON []byte) error {
body, _ := json.Marshal(map[string]string{
"blob_b64": base64.StdEncoding.EncodeToString(envJSON),
})
// 10A's enqueue lives under /admin/hosts/{id}/jobs (operator/global key). The queued blob is
// base64(envelope JSON); the hub stores it opaquely.
url := strings.TrimRight(baseURL, "/") + "/api/v1/admin/hosts/" + hostID + "/jobs"
req, err := http.NewRequest(http.MethodPost, url, bytes.NewReader(body))
if err != nil {
return err
}
req.Header.Set("Authorization", "Bearer "+hubKey)
req.Header.Set("Content-Type", "application/json")
hc := &http.Client{Timeout: 15 * time.Second}
resp, err := hc.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
raw, _ := io.ReadAll(io.LimitReader(resp.Body, 4<<10))
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return fmt.Errorf("HTTP %d: %s", resp.StatusCode, strings.TrimSpace(string(raw)))
}
return nil
}
BIN
View File
Binary file not shown.
+30
View File
@@ -2,9 +2,39 @@ package authz
import (
"encoding/json"
"fmt"
"strings"
"time"
)
// CanonicalBlob builds the canonical OpBlob bytes (phase4 §2 field order: keys sorted at
// every level, no insignificant whitespace, no trailing newline, UTF-8). This is the SINGLE
// production source of the signed bytes — the operator signing CLI (cmd/felhom-opsign) and the
// in-Go test minting both call it, so the signer can NEVER drift from what the verifier expects
// (the verifier authenticates over the RAW received bytes, so these bytes ARE the contract).
//
// params is canonicalized internally (parsed + re-marshaled → object keys sorted, whitespace
// stripped) so the same op+params always yields identical bytes; "" → "{}". Returns an error
// only when params is not valid JSON.
func CanonicalBlob(op, hostID, guestID, keyID, nonce, paramsJSON string, issued, expires time.Time) ([]byte, error) {
params := strings.TrimSpace(paramsJSON)
if params == "" {
params = "{}"
}
var pv interface{}
if err := json.Unmarshal([]byte(params), &pv); err != nil {
return nil, fmt.Errorf("authz: params is not valid JSON: %w", err)
}
pc, err := json.Marshal(pv) // Go marshals object keys sorted, compact
if err != nil {
return nil, fmt.Errorf("authz: canonicalizing params: %w", err)
}
return []byte(fmt.Sprintf(
`{"expires_at":%q,"issued_at":%q,"key_id":%q,"nonce":%q,"op":%q,"params":%s,"target":{"guest_id":%q,"host_id":%q}}`,
expires.UTC().Format(time.RFC3339), issued.UTC().Format(time.RFC3339),
keyID, nonce, op, pc, guestID, hostID)), nil
}
// Target binds an op to a specific box (and optionally a guest) — the anti-retarget
// field. The §7 reference omitted the json tags; production needs them so the
// signed canonical bytes decode correctly.
+6 -10
View File
@@ -6,7 +6,6 @@ import (
"crypto/sha256"
"encoding/binary"
"encoding/pem"
"fmt"
"testing"
"time"
@@ -17,17 +16,14 @@ import (
// production framing. They reuse the production signedData()/sshsigBlob so a test
// can never drift from the verifier's notion of the signed bytes.
// canonicalBlob builds an op blob in the §2 canonical field order. (Self-consistent
// for the in-Go path: we sign exactly these bytes and verify the same bytes. The
// committed ssh-keygen fixture exercises real OpenSSH canonical interop.)
// canonicalBlob delegates to the production CanonicalBlob (so the in-Go test minting can never
// drift from the real signed-bytes path). Panics on a params error — tests pass valid JSON.
func canonicalBlob(op, hostID, guestID, keyID, nonce, paramsJSON string, issued, expires time.Time) []byte {
if paramsJSON == "" {
paramsJSON = "{}"
b, err := CanonicalBlob(op, hostID, guestID, keyID, nonce, paramsJSON, issued, expires)
if err != nil {
panic(err)
}
return []byte(fmt.Sprintf(
`{"expires_at":%q,"issued_at":%q,"key_id":%q,"nonce":%q,"op":%q,"params":%s,"target":{"guest_id":%q,"host_id":%q}}`,
expires.UTC().Format(time.RFC3339), issued.UTC().Format(time.RFC3339),
keyID, nonce, op, paramsJSON, guestID, hostID))
return b
}
// mintArmor builds an armored SSHSIG over message, using sign to produce the inner
+63
View File
@@ -144,6 +144,69 @@ func (c *Client) FetchDesiredState(ctx context.Context) (*DesiredStateResponse,
return &out, nil
}
// JobWire is one queued signed-op job as served by GET /hosts/{id}/jobs (slice 10A). The blob is
// OPAQUE to the hub — for slice 10B it is a base64 `SignedJobEnvelope` (op-blob + armored SSHSIG)
// the agent verifies before executing.
type JobWire struct {
JobID string `json:"job_id"`
BlobB64 string `json:"blob_b64"`
CreatedAt string `json:"created_at"`
}
// Jobs fetches this host's pending signed-op jobs (slice 10B). Self-scoped server-side (the
// per-host key only reads its own host). The agent verifies each before executing.
func (c *Client) Jobs(ctx context.Context) ([]JobWire, error) {
if c.hostID == "" {
return nil, fmt.Errorf("hub: Jobs requires a configured host_id")
}
req, err := http.NewRequestWithContext(ctx, http.MethodGet, c.baseURL+"/api/v1/hosts/"+c.hostID+"/jobs", nil)
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+c.apiKey)
req.Header.Set("Accept", "application/json")
resp, err := c.hc.Do(req)
if err != nil {
return nil, &TransportError{Err: err}
}
defer resp.Body.Close()
raw, _ := io.ReadAll(io.LimitReader(resp.Body, 4<<20))
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return nil, &HTTPError{StatusCode: resp.StatusCode, BodyTail: tail(raw, 256)}
}
var out struct {
Jobs []JobWire `json:"jobs"`
}
if err := json.Unmarshal(raw, &out); err != nil {
return nil, fmt.Errorf("hub: decoding jobs: %w", err)
}
return out.Jobs, nil
}
// CompleteJob clears a processed job from the host's queue (slice 10B): DELETE
// /hosts/{id}/jobs/{job_id}, self-scoped. Called after a job is executed OR permanently rejected
// (the nonce is already durably burned on a passing verify, so re-processing is replay-safe).
func (c *Client) CompleteJob(ctx context.Context, jobID string) error {
if c.hostID == "" || jobID == "" {
return fmt.Errorf("hub: CompleteJob requires host_id + job_id")
}
req, err := http.NewRequestWithContext(ctx, http.MethodDelete, c.baseURL+"/api/v1/hosts/"+c.hostID+"/jobs/"+jobID, nil)
if err != nil {
return err
}
req.Header.Set("Authorization", "Bearer "+c.apiKey)
resp, err := c.hc.Do(req)
if err != nil {
return &TransportError{Err: err}
}
defer resp.Body.Close()
raw, _ := io.ReadAll(io.LimitReader(resp.Body, 4<<10))
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return &HTTPError{StatusCode: resp.StatusCode, BodyTail: tail(raw, 256)}
}
return nil
}
func tail(b []byte, max int) string {
s := strings.TrimSpace(string(b))
if len(s) > max {
+16
View File
@@ -29,6 +29,22 @@ type EnvelopeObserver interface {
OnEnvelope(ctx context.Context, env *ControlEnvelope)
}
// MultiObserver fans one envelope out to several observers in order (e.g. the desired-state
// syncer + the signed-jobs runner). nil entries are skipped.
func MultiObserver(observers ...EnvelopeObserver) EnvelopeObserver {
return multiObserver(observers)
}
type multiObserver []EnvelopeObserver
func (m multiObserver) OnEnvelope(ctx context.Context, env *ControlEnvelope) {
for _, o := range m {
if o != nil {
o.OnEnvelope(ctx, env)
}
}
}
// Loop is the agent's first daemon run loop: collect a host-report, POST it, adopt
// the hub's cadence, repeat. It is resilient — a collect or report error is logged
// and the loop continues (the data plane is independent of the agent; a hub outage
+38 -8
View File
@@ -158,13 +158,28 @@ type formatRequest struct {
// inspects the device itself (8C invariant).
}
// FormatResponse is POST /disks/format.
// FormatResponse is POST /disks/format. On a data-bearing refusal (slice 10B) it SURFACES the
// bound op the operator must sign: the op class + the DURABLE device id (not the mutable path) +
// the fstype — so the operator can `felhom-opsign -op storage_wipe -durable-id <…>` offline, the
// hub queues it, and the agent's signed-jobs runner verifies + executes the wipe (re-resolving the
// durable id). This is the "records/reports the bound op intent so the operator sees what to sign".
type FormatResponse struct {
VMID int `json:"vmid"`
Device string `json:"device"`
Formatted bool `json:"formatted"`
DataBearing bool `json:"data_bearing"`
Reason string `json:"reason"`
// PendingOp is set on a data-bearing refusal — the exact op to sign (slice 10B).
PendingOp *PendingOp `json:"pending_op,omitempty"`
}
// PendingOp is the bound destructive intent the operator must sign offline (slice 10B). Params bind
// to the DURABLE device id so the signed authorization can't be retargeted to another disk.
type PendingOp struct {
Op string `json:"op"` // e.g. "storage_wipe"
HostScope string `json:"host_scope"` // the agent's host id (anti-retarget target)
DurableID string `json:"durable_id"` // byid:…|byuuid:… — the device's stable identity
FSType string `json:"fstype"` // the filesystem to mkfs after the wipe
}
// handleDiskFormat is the security centerpiece. The agent INSPECTS the device; if it is
@@ -198,18 +213,33 @@ func (s *Server) handleDiskFormat(w http.ResponseWriter, r *http.Request, vmid i
// inspect error → fail-safe data-bearing (probe.DataBearing() is true on !Probed)
}
if probe.DataBearing() {
// Destructive: route through the gate. With no operator signature (8C) → pending_signature.
// Destructive: route through the gate. With no operator signature → pending_signature.
allowed, reason := s.diskGate.AuthorizeWipe(req.Device)
s.logger.Warn("local-api: refusing format of a data-bearing device",
"vmid", vmid, "device", req.Device, "why", probe.Reason(), "gate", reason)
if !allowed {
// Surface the bound op the operator must sign (slice 10B): derive the DURABLE device id
// so the signed wipe binds to this exact physical disk (not the mutable path), and the
// runner can re-resolve it at execution. A durable-id derivation failure is non-fatal —
// the refusal still stands; we just can't pre-fill the durable id.
var pending *PendingOp
if durableID, derr := storage.DeviceDurableID(req.Device); derr == nil {
pending = &PendingOp{Op: "storage_wipe", HostScope: s.hostID, DurableID: durableID, FSType: req.FSType}
s.logger.Warn("local-api: data-bearing format refused — PENDING OPERATOR SIGNATURE",
"vmid", vmid, "device", req.Device, "durable_id", durableID, "fstype", req.FSType,
"why", probe.Reason(), "to_authorize", "felhom-opsign -op storage_wipe -host "+s.hostID+" -durable-id "+durableID)
} else {
s.logger.Warn("local-api: data-bearing format refused (no durable id)",
"vmid", vmid, "device", req.Device, "why", probe.Reason(), "derive_err", derr)
}
writeStatus(w, http.StatusForbidden, false,
FormatResponse{VMID: vmid, Device: req.Device, Formatted: false, DataBearing: true, Reason: probe.Reason()},
"device is data-bearing — format requires operator authorization ("+reason+")")
FormatResponse{VMID: vmid, Device: req.Device, Formatted: false, DataBearing: true, Reason: probe.Reason(), PendingOp: pending},
"device is data-bearing — format requires an operator signature ("+reason+")")
return
}
// A signed completion would land here in slice 10; 8C never reaches it (gate refuses unsigned).
writeErr(w, http.StatusForbidden, "data-bearing format is not supported in this slice")
// A signed wipe is executed by the signed-jobs runner (queue → verify gate → durable
// re-resolve + re-inspect → mkfs), NOT this synchronous path. This branch (gate ALLOWED a
// data-bearing format inline) is unreachable: the inline path passes signed=nil → always
// pending. Fail safe.
writeErr(w, http.StatusForbidden, "data-bearing format must be completed via a signed job (felhom-opsign → hub queue)")
return
}
+2 -2
View File
@@ -129,8 +129,8 @@ func TestFormat_DataBearingDevice_RefusedNoMkfs(t *testing.T) {
if len(g.calls) != 1 || g.calls[0] != "/dev/sdb" {
t.Fatalf("gate not consulted for the destructive format: %v", g.calls)
}
if !strings.Contains(w.Body.String(), "operator authorization") {
t.Fatalf("response did not signal operator-authorization needed: %s", w.Body.String())
if !strings.Contains(w.Body.String(), "operator signature") {
t.Fatalf("response did not signal an operator signature is needed: %s", w.Body.String())
}
}
+6 -1
View File
@@ -84,7 +84,10 @@ type Options struct {
// cpu-temp) + per-storage capacity, host-wide and token-authed (one-customer-per-host). When
// nil the endpoint reports "not configured" (host still reports/reconciles).
HostMetrics HostMetricsProvider
Logger *slog.Logger
// HostID is this agent's host id — surfaced in a data-bearing-format pending-op so the operator
// signs an op bound to THIS host (slice 10B anti-retarget). Optional (only used for the hint).
HostID string
Logger *slog.Logger
}
// defaultBackupCadence is the fallback /backup/due window when none is configured.
@@ -129,6 +132,7 @@ type Server struct {
guestList GuestLister // slice 8C (optional)
hostMetrics HostMetricsProvider // slice 9 (optional)
hostID string // slice 10B: for the data-bearing-format pending-op hint
jobsMu sync.Mutex
jobs map[int]*backupJob // per-guest backup job state (slice 8B)
@@ -166,6 +170,7 @@ func NewServer(o Options) (*Server, error) {
diskGate: o.DiskGate,
guestList: o.Guests2,
hostMetrics: o.HostMetrics,
hostID: o.HostID,
jobs: map[int]*backupJob{},
}, nil
}
+115
View File
@@ -0,0 +1,115 @@
package signedjobs
import (
"crypto/ed25519"
"crypto/rand"
"crypto/sha256"
"encoding/base64"
"encoding/json"
"encoding/pem"
"testing"
"time"
"gitea.dooplex.hu/admin/felhom-agent/internal/authz"
"gitea.dooplex.hu/admin/felhom-agent/internal/hub"
"golang.org/x/crypto/ssh"
)
// In-test SSHSIG minter (mirrors internal/reconcile/mint_test.go's framing) so the runner tests
// exercise the REAL authz.Verifier + reconcile.Gate over genuinely-signed blobs — a positive case
// verifying proves the framing, and the adversarial cases (non-pinned/replay/expired/retarget/
// forged) exercise the real rejection path end-to-end. Production authz stays verify-only.
const sshsigMagic = "SSHSIG"
type sshsigBlob struct {
Version uint32
PublicKey string
Namespace string
Reserved string
HashAlgo string
Signature string
}
func signedDataForTest(ns string, msg []byte) []byte {
h := sha256Sum(msg)
body := ssh.Marshal(struct {
Namespace string
Reserved string
HashAlgo string
Hash []byte
}{ns, "", "sha256", h})
return append([]byte(sshsigMagic), body...)
}
func mintArmor(pubMarshaled []byte, namespace string, message []byte, sign func([]byte) ssh.Signature) []byte {
sb := &sshsigBlob{Version: 1, PublicKey: string(pubMarshaled), Namespace: namespace, Reserved: "", HashAlgo: "sha256"}
sig := sign(signedDataForTest(namespace, message))
sb.Signature = string(ssh.Marshal(&sig))
raw := append([]byte(sshsigMagic), ssh.Marshal(sb)...)
return pem.EncodeToMemory(&pem.Block{Type: "SSH SIGNATURE", Bytes: raw})
}
type testSigner struct {
pub ssh.PublicKey
line string
sign func([]byte) ssh.Signature
}
func newTestSigner(t *testing.T) testSigner {
t.Helper()
pub, priv, err := ed25519.GenerateKey(rand.Reader)
if err != nil {
t.Fatal(err)
}
sshPub, err := ssh.NewPublicKey(pub)
if err != nil {
t.Fatal(err)
}
return testSigner{
pub: sshPub,
line: string(ssh.MarshalAuthorizedKey(sshPub)),
sign: func(d []byte) ssh.Signature {
return ssh.Signature{Format: ssh.KeyAlgoED25519, Blob: ed25519.Sign(priv, d)}
},
}
}
func (s testSigner) allowed(t *testing.T, keyID string, role authz.KeyRole) authz.AllowedSigner {
t.Helper()
as, err := authz.NewAllowedSigner(keyID, role, s.line)
if err != nil {
t.Fatalf("NewAllowedSigner: %v", err)
}
return as
}
// mintJob builds a hub.JobWire carrying a signed storage_wipe envelope from the given signer.
func mintJob(t *testing.T, s testSigner, jobID, host, guest, keyID, paramsJSON string, issued, expires time.Time) hub.JobWire {
t.Helper()
blob, err := authz.CanonicalBlob("storage_wipe", host, guest, keyID, randNonce(), paramsJSON, issued, expires)
if err != nil {
t.Fatalf("CanonicalBlob: %v", err)
}
sig := mintArmor(s.pub.Marshal(), authz.Namespace, blob, s.sign)
env := Envelope{OpBlobB64: base64.StdEncoding.EncodeToString(blob), SigArmored: string(sig)}
envJSON, _ := json.Marshal(env)
return hub.JobWire{JobID: jobID, BlobB64: base64.StdEncoding.EncodeToString(envJSON)}
}
func randNonce() string {
var b [16]byte
rand.Read(b[:])
const hexd = "0123456789abcdef"
out := make([]byte, 32)
for i, x := range b {
out[i*2] = hexd[x>>4]
out[i*2+1] = hexd[x&0x0f]
}
return string(out)
}
func sha256Sum(b []byte) []byte {
h := sha256.Sum256(b)
return h[:]
}
+216
View File
@@ -0,0 +1,216 @@
// Package signedjobs is the slice-10B consumer of the hub's signed-jobs queue: it fetches each
// opaque queued blob, runs it through the reversibility gate (the LOCKED authz pipeline: pinned-key
// SSHSIG → namespace → allow-list-by-key-material → crypto → host target → time window → durable
// nonce-burn), and ONLY on all-pass hands the verified+bound op to an Executor. The order is
// verify → burn nonce (durable, inside Verify) → execute → report-completion, so no destructive op
// runs on an unsigned, non-pinned-signer, replayed, expired, retargeted, or path-only blob, and an
// interrupted op can't be re-authorized by replaying the same blob after a crash.
package signedjobs
import (
"context"
"encoding/base64"
"encoding/json"
"fmt"
"log/slog"
"sync"
"gitea.dooplex.hu/admin/felhom-agent/internal/authz"
"gitea.dooplex.hu/admin/felhom-agent/internal/hub"
"gitea.dooplex.hu/admin/felhom-agent/internal/reconcile"
)
// Envelope is the queued signed-op job payload (the opaque bytes the hub stores per job). The
// operator's signing CLI (cmd/felhom-opsign) produces it; this runner consumes it. The hub never
// forges one (it holds no signing key) and never opens it.
type Envelope struct {
OpBlobB64 string `json:"op_blob_b64"` // base64 of the canonical OpBlob JSON
SigArmored string `json:"sig_armored"` // armored SSHSIG over the op-blob (namespace felhom-op-v1)
}
// JobSource fetches + clears jobs. Satisfied by *hub.Client.
type JobSource interface {
Jobs(ctx context.Context) ([]hub.JobWire, error)
CompleteJob(ctx context.Context, jobID string) error
}
// Authorizer is the reversibility gate. Satisfied by *reconcile.Gate.
type Authorizer interface {
Authorize(intent reconcile.Intent, signed *reconcile.SignedOp) reconcile.Decision
}
// Executor runs a verified+bound destructive op. 10B wires the storage-wipe executor; other op
// classes (guest_destroy, decommission, restore_overwrite — 10D) plug in per-slice. The executor
// owns the RESOURCE-level anti-retarget (durable-id resolution + re-inspection at execution).
type Executor interface {
// Execute runs the op named by `op` with the VERIFIED canonical params. ErrNoExecutor means
// "this op has no executor in this build" (left in the queue for a later slice).
Execute(ctx context.Context, op string, params json.RawMessage) error
}
// ErrNoExecutor signals an op class with no executor wired in this build (don't clear the job).
var ErrNoExecutor = fmt.Errorf("signedjobs: no executor for this op class in this build")
// Runner polls the jobs queue and drives verify→execute for each job. Single-flight: a RunOnce
// already in progress short-circuits a concurrent trigger.
type Runner struct {
source JobSource
gate Authorizer
exec Executor
hostID string
logger *slog.Logger
mu sync.Mutex
running bool
}
// NewRunner builds a runner. All deps are required.
func NewRunner(source JobSource, gate Authorizer, exec Executor, hostID string, logger *slog.Logger) *Runner {
if logger == nil {
logger = slog.Default()
}
return &Runner{source: source, gate: gate, exec: exec, hostID: hostID, logger: logger}
}
// OnEnvelope implements hub.EnvelopeObserver: when the heartbeat flags pending signed ops, run a
// pass (off the heartbeat path so a long mkfs never blocks the loop). Single-flight.
func (r *Runner) OnEnvelope(ctx context.Context, env *hub.ControlEnvelope) {
if env == nil || !env.HasSignedOps {
return
}
go func() {
if _, err := r.RunOnce(context.Background()); err != nil {
r.logger.Warn("signedjobs: pass failed", "err", err)
}
}()
}
// RunOnce fetches the pending jobs and processes each. Returns the number of jobs PROCESSED
// (executed or terminally rejected/cleared). A fetch error returns early; per-job errors are
// logged and do not abort the pass.
func (r *Runner) RunOnce(ctx context.Context) (int, error) {
r.mu.Lock()
if r.running {
r.mu.Unlock()
return 0, nil // a pass is already in progress
}
r.running = true
r.mu.Unlock()
defer func() { r.mu.Lock(); r.running = false; r.mu.Unlock() }()
jobs, err := r.source.Jobs(ctx)
if err != nil {
return 0, fmt.Errorf("signedjobs: fetch jobs: %w", err)
}
processed := 0
for _, j := range jobs {
if r.processJob(ctx, j) {
processed++
}
}
return processed, nil
}
// processJob handles one job; returns true if it was terminally handled (cleared from the queue).
func (r *Runner) processJob(ctx context.Context, j hub.JobWire) bool {
signed, ob, ok := r.decode(j)
if !ok {
// A malformed/forged envelope is permanently bad — clear it loudly (a queued garbage job
// must not wedge the queue). No nonce was consumed (we never reached Verify).
r.logger.Error("signedjobs: malformed job envelope — clearing", "job", j.JobID)
r.complete(ctx, j.JobID)
return true
}
intent := reconcile.Intent{
Class: reconcile.OpClass(ob.Op),
HostID: ob.Target.HostID,
GuestID: ob.Target.GuestID,
ParamsJSON: ob.Params,
Source: reconcile.SourceOneShotJob,
// Provenance is the zero value (no agent-internal evidence) — a hub-sourced op is
// classified by its op class alone; a destructive class MUST carry a valid signature.
}
dec := r.gate.Authorize(intent, signed)
if !dec.Allowed {
// The signature/binding failed (forged, replayed, expired, retargeted, role-denied, …).
// The executor is NEVER called. This blob can never become valid, so clear it — but LOG
// it as a security event (a compromised hub queuing a forged op lands here).
r.logger.Warn("signedjobs: REJECTED signed op — executor not called",
"job", j.JobID, "op", ob.Op, "reason", dec.Reason, "err", errStr(dec.Err))
r.complete(ctx, j.JobID)
return true
}
// Allowed: the nonce is already durably burned (Verify, before this point). Execute.
r.logger.Warn("signedjobs: AUTHORIZED signed op — executing",
"job", j.JobID, "op", ob.Op, "key_id", dec.Verified.KeyID, "nonce", dec.Verified.Nonce)
err := r.exec.Execute(ctx, ob.Op, ob.Params)
switch {
case err == nil:
r.logger.Warn("signedjobs: signed op COMPLETED", "job", j.JobID, "op", ob.Op)
r.complete(ctx, j.JobID)
return true
case errorsIs(err, ErrNoExecutor):
// No executor for this op class in this build — leave it queued for the owning slice.
r.logger.Info("signedjobs: no executor for op (left queued)", "job", j.JobID, "op", ob.Op)
return false
default:
// Execution failed AFTER a passing verify — the nonce is spent, so re-running the same
// blob would be rejected as a replay anyway. Clear it (the operator re-signs with a fresh
// nonce to retry) and log the failure loudly.
r.logger.Error("signedjobs: signed op execution FAILED (nonce spent — clearing)",
"job", j.JobID, "op", ob.Op, "err", err)
r.complete(ctx, j.JobID)
return true
}
}
// decode unpacks a JobWire → SignedOp{op-blob, armored sig} + the parsed OpBlob (for routing).
func (r *Runner) decode(j hub.JobWire) (*reconcile.SignedOp, authz.OpBlob, bool) {
raw, err := base64.StdEncoding.DecodeString(j.BlobB64)
if err != nil {
return nil, authz.OpBlob{}, false
}
var env Envelope
if err := json.Unmarshal(raw, &env); err != nil || env.OpBlobB64 == "" || env.SigArmored == "" {
return nil, authz.OpBlob{}, false
}
opBlob, err := base64.StdEncoding.DecodeString(env.OpBlobB64)
if err != nil {
return nil, authz.OpBlob{}, false
}
var ob authz.OpBlob
if err := json.Unmarshal(opBlob, &ob); err != nil || ob.Op == "" {
return nil, authz.OpBlob{}, false
}
return &reconcile.SignedOp{Blob: opBlob, Sig: []byte(env.SigArmored)}, ob, true
}
func (r *Runner) complete(ctx context.Context, jobID string) {
if err := r.source.CompleteJob(ctx, jobID); err != nil {
r.logger.Warn("signedjobs: clearing job failed (will retry next pass)", "job", jobID, "err", err)
}
}
func errStr(e error) string {
if e == nil {
return ""
}
return e.Error()
}
func errorsIs(err, target error) bool {
for err != nil {
if err == target {
return true
}
type w interface{ Unwrap() error }
if u, ok := err.(w); ok {
err = u.Unwrap()
} else {
return false
}
}
return false
}
+240
View File
@@ -0,0 +1,240 @@
package signedjobs
import (
"context"
"encoding/base64"
"encoding/json"
"io"
"log/slog"
"sync"
"testing"
"time"
"gitea.dooplex.hu/admin/felhom-agent/internal/authz"
"gitea.dooplex.hu/admin/felhom-agent/internal/hub"
"gitea.dooplex.hu/admin/felhom-agent/internal/reconcile"
)
func quiet() *slog.Logger { return slog.New(slog.NewTextHandler(io.Discard, nil)) }
const testHost = "demo-felhom-01"
// jobsQueue is an in-memory JobSource for the runner tests.
type jobsQueue struct {
mu sync.Mutex
jobs []hub.JobWire
completed []string
}
func (q *jobsQueue) add(j hub.JobWire) { q.mu.Lock(); q.jobs = append(q.jobs, j); q.mu.Unlock() }
func (q *jobsQueue) Jobs(context.Context) ([]hub.JobWire, error) {
q.mu.Lock()
defer q.mu.Unlock()
out := append([]hub.JobWire(nil), q.jobs...)
return out, nil
}
func (q *jobsQueue) CompleteJob(_ context.Context, jobID string) error {
q.mu.Lock()
defer q.mu.Unlock()
q.completed = append(q.completed, jobID)
// also remove from the pending list so a re-run doesn't re-process it
kept := q.jobs[:0]
for _, j := range q.jobs {
if j.JobID != jobID {
kept = append(kept, j)
}
}
q.jobs = kept
return nil
}
func (q *jobsQueue) wasCompleted(jobID string) bool {
q.mu.Lock()
defer q.mu.Unlock()
for _, id := range q.completed {
if id == jobID {
return true
}
}
return false
}
// corruptSig flips bytes in the envelope's armored signature (a hub forging/altering a blob).
func corruptSig(t *testing.T, j hub.JobWire) hub.JobWire {
t.Helper()
raw, _ := base64.StdEncoding.DecodeString(j.BlobB64)
var env Envelope
json.Unmarshal(raw, &env)
// Replace the armored signature with a structurally-valid-but-wrong one: re-arm random bytes.
env.SigArmored = env.SigArmored[:len(env.SigArmored)/2] + "AAAA" + env.SigArmored[len(env.SigArmored)/2:]
out, _ := json.Marshal(env)
j.BlobB64 = base64.StdEncoding.EncodeToString(out)
return j
}
// fakeExecutor records Execute calls and returns a configurable error.
type fakeExecutor struct {
mu sync.Mutex
calls []string // ops executed
err error
}
func (f *fakeExecutor) Execute(_ context.Context, op string, _ json.RawMessage) error {
f.mu.Lock()
f.calls = append(f.calls, op)
f.mu.Unlock()
return f.err
}
func (f *fakeExecutor) count() int { f.mu.Lock(); defer f.mu.Unlock(); return len(f.calls) }
// newRealGateRunner builds a runner with the REAL gate+verifier pinned to `signer`, an in-memory
// nonce store, plus a fake jobs source + fake executor. wipeParams is the params to sign.
func newRealGateRunner(t *testing.T, signer testSigner) (*Runner, *jobsQueue, *fakeExecutor) {
t.Helper()
store := authz.NewMemoryNonceStore()
verifier := authz.New([]authz.AllowedSigner{signer.allowed(t, "ops-1", authz.RoleOperational)}, store, testHost)
gate := reconcile.NewGate(verifier, testHost, nil, quiet())
src := &jobsQueue{}
exec := &fakeExecutor{}
return NewRunner(src, gate, exec, testHost, quiet()), src, exec
}
const wipeParamsJSON = `{"durable_id":"byid:wwn-0xtest","fstype":"ext4"}`
// VALID: a correctly-signed wipe → executor runs once + the job is cleared.
func TestRunner_ValidSignedWipeExecutes(t *testing.T) {
s := newTestSigner(t)
r, src, exec := newRealGateRunner(t, s)
now := time.Now().UTC()
src.add(mintJob(t, s, "j1", testHost, "", "ops-1", wipeParamsJSON, now, now.Add(time.Hour)))
n, err := r.RunOnce(context.Background())
if err != nil {
t.Fatalf("RunOnce: %v", err)
}
if n != 1 || exec.count() != 1 || exec.calls[0] != "storage_wipe" {
t.Fatalf("executor calls = %v (n=%d), want one storage_wipe", exec.calls, n)
}
if !src.wasCompleted("j1") {
t.Error("valid job was not cleared after execution")
}
}
// REPLAY: the same blob resubmitted (a second job) → rejected, executor NOT called again.
func TestRunner_ReplayRejected(t *testing.T) {
s := newTestSigner(t)
r, src, exec := newRealGateRunner(t, s)
now := time.Now().UTC()
job := mintJob(t, s, "j1", testHost, "", "ops-1", wipeParamsJSON, now, now.Add(time.Hour))
src.add(job)
if _, err := r.RunOnce(context.Background()); err != nil {
t.Fatal(err)
}
if exec.count() != 1 {
t.Fatalf("first run executed %d times, want 1", exec.count())
}
// Resubmit the IDENTICAL signed blob (same nonce) as a new job.
replay := job
replay.JobID = "j1-replay"
src.add(replay)
if _, err := r.RunOnce(context.Background()); err != nil {
t.Fatal(err)
}
if exec.count() != 1 {
t.Errorf("replay caused a second execution (count=%d) — nonce-burn failed", exec.count())
}
if !src.wasCompleted("j1-replay") {
t.Error("replayed job should be cleared (rejected)")
}
}
// NON-PINNED signer → rejected, executor NOT called.
func TestRunner_NonPinnedSignerRejected(t *testing.T) {
pinned := newTestSigner(t)
attacker := newTestSigner(t) // a DIFFERENT key, not pinned
r, src, exec := newRealGateRunner(t, pinned)
now := time.Now().UTC()
src.add(mintJob(t, attacker, "j1", testHost, "", "ops-1", wipeParamsJSON, now, now.Add(time.Hour)))
r.RunOnce(context.Background())
if exec.count() != 0 {
t.Errorf("a non-pinned signature was EXECUTED (count=%d) — must be rejected", exec.count())
}
if !src.wasCompleted("j1") {
t.Error("rejected job should be cleared")
}
}
// EXPIRED → rejected, executor NOT called.
func TestRunner_ExpiredRejected(t *testing.T) {
s := newTestSigner(t)
r, src, exec := newRealGateRunner(t, s)
past := time.Now().UTC().Add(-2 * time.Hour)
src.add(mintJob(t, s, "j1", testHost, "", "ops-1", wipeParamsJSON, past, past.Add(time.Hour))) // expired an hour ago
r.RunOnce(context.Background())
if exec.count() != 0 {
t.Errorf("an EXPIRED op was executed (count=%d)", exec.count())
}
}
// RETARGETED (Target.HostID = another host) → rejected, executor NOT called.
func TestRunner_RetargetRejected(t *testing.T) {
s := newTestSigner(t)
r, src, exec := newRealGateRunner(t, s)
now := time.Now().UTC()
src.add(mintJob(t, s, "j1", "some-other-host", "", "ops-1", wipeParamsJSON, now, now.Add(time.Hour)))
r.RunOnce(context.Background())
if exec.count() != 0 {
t.Errorf("an op targeting another host was executed on this host (count=%d) — anti-retarget failed", exec.count())
}
}
// FORGED (a compromised hub queues a blob with a garbage signature) → rejected, executor NOT called.
func TestRunner_ForgedSignatureRejected(t *testing.T) {
s := newTestSigner(t)
r, src, exec := newRealGateRunner(t, s)
now := time.Now().UTC()
job := mintJob(t, s, "j1", testHost, "", "ops-1", wipeParamsJSON, now, now.Add(time.Hour))
// Corrupt the envelope's signature (simulate a hub forging/altering the blob).
job = corruptSig(t, job)
src.add(job)
r.RunOnce(context.Background())
if exec.count() != 0 {
t.Errorf("a forged/altered signature was executed (count=%d) — the compromised-hub case must be rejected", exec.count())
}
if !src.wasCompleted("j1") {
t.Error("forged job should be cleared")
}
}
// No verifier pinned (no signers) → every signed op is refused pending_signature, executor not called.
func TestRunner_NoSignersAllRejected(t *testing.T) {
s := newTestSigner(t)
gate := reconcile.NewGate(nil, testHost, nil, quiet()) // nil verifier
src := &jobsQueue{}
exec := &fakeExecutor{}
r := NewRunner(src, gate, exec, testHost, quiet())
now := time.Now().UTC()
src.add(mintJob(t, s, "j1", testHost, "", "ops-1", wipeParamsJSON, now, now.Add(time.Hour)))
r.RunOnce(context.Background())
if exec.count() != 0 {
t.Errorf("with no pinned signer a signed op executed (count=%d) — must be pending_signature", exec.count())
}
}
// A malformed envelope → cleared without ever calling the executor.
func TestRunner_MalformedEnvelopeCleared(t *testing.T) {
s := newTestSigner(t)
r, src, exec := newRealGateRunner(t, s)
src.add(hub.JobWire{JobID: "bad", BlobB64: "!!!not base64!!!"})
r.RunOnce(context.Background())
if exec.count() != 0 {
t.Errorf("a malformed envelope reached the executor (count=%d)", exec.count())
}
if !src.wasCompleted("bad") {
t.Error("malformed job should be cleared")
}
}
+111
View File
@@ -0,0 +1,111 @@
package signedjobs
import (
"context"
"encoding/json"
"fmt"
"log/slog"
"gitea.dooplex.hu/admin/felhom-agent/internal/storage"
)
// opStorageWipe is the op class this executor serves (mirrors reconcile.ClassStorageWipe; the
// literal avoids importing reconcile here just for the string).
const opStorageWipe = "storage_wipe"
// WipeOps is the privileged storage surface the wipe executor needs (a subset of storage.HostOps).
type WipeOps interface {
InspectDevice(ctx context.Context, device string) (storage.DeviceProbe, error)
Format(ctx context.Context, device, fstype string) error
}
// wipeParams is the verified params of a storage_wipe op. The device is named by a DURABLE id
// (never a mutable /dev path), so execution re-resolves it to the exact physical device.
type wipeParams struct {
DurableID string `json:"durable_id"`
FSType string `json:"fstype"`
}
// WipeExecutor is the slice-10B storage-wipe consumer — it CLOSES the 8C data-bearing-format
// `pending_signature` gap. Given a gate-VERIFIED+bound storage_wipe op, it:
// 1. resolves the op's DURABLE device id → the current /dev path (vanished/replaced → refuse);
// 2. re-derives that device's durable id and requires it to MATCH the signed id (resource-level
// anti-retarget — "wipe device X" wipes exactly X, not whatever is at /dev/sdb now);
// 3. re-inspects (8C classifier) to confirm it is STILL the data-bearing target;
// 4. mkfs.
//
// A path-only binding, a device that vanished/changed, or a target that is no longer data-bearing
// is refused EVEN WITH a valid signature. The gate has already burned the nonce durably before we
// run, so an interrupted wipe can't be re-authorized by replaying the blob.
type WipeExecutor struct {
ops WipeOps
resolve func(durableID string) (string, error)
derive func(device string) (string, error)
logger *slog.Logger
}
// NewWipeExecutor wires the production durable resolver/deriver (storage package).
func NewWipeExecutor(ops WipeOps, logger *slog.Logger) *WipeExecutor {
if logger == nil {
logger = slog.Default()
}
return &WipeExecutor{
ops: ops,
resolve: storage.ResolveDurableDevice,
derive: storage.DeviceDurableID,
logger: logger,
}
}
// Execute implements signedjobs.Executor for the storage_wipe op class.
func (w *WipeExecutor) Execute(ctx context.Context, op string, params json.RawMessage) error {
if op != opStorageWipe {
return ErrNoExecutor // not ours — the runner leaves it queued for the owning slice
}
var p wipeParams
if err := json.Unmarshal(params, &p); err != nil {
return fmt.Errorf("wipe: bad params: %w", err)
}
if p.DurableID == "" {
// The whole point of the durable binding: a path-only op is refused (anti-retarget).
return fmt.Errorf("wipe: op has no durable_id — refusing a path-only wipe binding")
}
// 1. Resolve the durable id → current device (gone/replaced → refuse).
device, err := w.resolve(p.DurableID)
if err != nil {
return fmt.Errorf("wipe: durable id %q no longer resolves (device removed/replaced?) — refusing: %w", p.DurableID, err)
}
// 2. Anti-retarget: re-derive the resolved device's durable id; it MUST equal the signed id.
got, err := w.derive(device)
if err != nil {
return fmt.Errorf("wipe: cannot re-derive durable id for %s — refusing: %w", device, err)
}
if got != p.DurableID {
return fmt.Errorf("wipe: durable-id mismatch — %s now has id %q, signed id was %q — refusing", device, got, p.DurableID)
}
// 3. Re-inspect (8C classifier) — confirm the resolved device is STILL the data-bearing target.
probe, err := w.ops.InspectDevice(ctx, device)
if err != nil {
return fmt.Errorf("wipe: re-inspect %s failed — refusing: %w", device, err)
}
if !probe.Probed {
return fmt.Errorf("wipe: %s did not probe cleanly at execution — refusing", device)
}
if !probe.DataBearing() {
// The signed op authorized wiping a DATA-BEARING device; if it is now blank, the target
// changed since signing — refuse rather than wipe the wrong (or an unexpected) device.
return fmt.Errorf("wipe: %s is no longer data-bearing (target changed since signing) — refusing", device)
}
// 4. Execute. The nonce was durably burned by the gate's Verify BEFORE this point (crash-safety:
// a replay after an interrupted wipe is rejected). This log line is the audit trail of a
// destructive, operator-authorized wipe actually running.
w.logger.Warn("wipe: executing operator-signed data-bearing wipe",
"device", device, "durable_id", p.DurableID, "fstype", p.FSType, "data_reason", probe.Reason())
if err := w.ops.Format(ctx, device, p.FSType); err != nil {
return fmt.Errorf("wipe: mkfs %s %s: %w", p.FSType, device, err)
}
w.logger.Warn("wipe: operator-signed wipe complete", "device", device, "durable_id", p.DurableID)
return nil
}
+137
View File
@@ -0,0 +1,137 @@
package signedjobs
import (
"context"
"encoding/json"
"errors"
"testing"
"gitea.dooplex.hu/admin/felhom-agent/internal/storage"
)
// fakeWipeOps records Format calls and returns a configurable InspectDevice probe.
type fakeWipeOps struct {
probe storage.DeviceProbe
inspectErr error
formatCalls []string
}
func (f *fakeWipeOps) InspectDevice(_ context.Context, device string) (storage.DeviceProbe, error) {
p := f.probe
p.Device = device
return p, f.inspectErr
}
func (f *fakeWipeOps) Format(_ context.Context, device, _ string) error {
f.formatCalls = append(f.formatCalls, device)
return nil
}
// newWipeExec builds a WipeExecutor with injected resolve/derive (so durable resolution is
// deterministic in a unit test, no real /dev needed).
func newWipeExec(ops WipeOps, resolve func(string) (string, error), derive func(string) (string, error)) *WipeExecutor {
w := NewWipeExecutor(ops, quiet())
w.resolve = resolve
w.derive = derive
return w
}
const durable = "byid:wwn-0xtest"
func params(t *testing.T, durableID, fstype string) json.RawMessage {
t.Helper()
b, _ := json.Marshal(map[string]string{"durable_id": durableID, "fstype": fstype})
return b
}
// VALID: durable resolves, re-derives to the same id, device is data-bearing → Format runs.
func TestWipe_ValidExecutes(t *testing.T) {
ops := &fakeWipeOps{probe: storage.DeviceProbe{Probed: true, HasFilesystem: true, FSType: "ext4"}}
w := newWipeExec(ops,
func(id string) (string, error) { return "/dev/sdb", nil },
func(dev string) (string, error) { return durable, nil })
if err := w.Execute(context.Background(), "storage_wipe", params(t, durable, "ext4")); err != nil {
t.Fatalf("valid wipe: %v", err)
}
if len(ops.formatCalls) != 1 || ops.formatCalls[0] != "/dev/sdb" {
t.Fatalf("Format calls = %v, want one on /dev/sdb", ops.formatCalls)
}
}
// PATH-ONLY (no durable_id) → refused, Format NOT called.
func TestWipe_PathOnlyRefused(t *testing.T) {
ops := &fakeWipeOps{probe: storage.DeviceProbe{Probed: true, HasFilesystem: true}}
w := newWipeExec(ops, func(string) (string, error) { return "/dev/sdb", nil }, func(string) (string, error) { return durable, nil })
// params carry a raw device path, NOT a durable id.
p, _ := json.Marshal(map[string]string{"device": "/dev/sdb", "fstype": "ext4"})
if err := w.Execute(context.Background(), "storage_wipe", p); err == nil {
t.Fatal("path-only wipe must be refused")
}
if len(ops.formatCalls) != 0 {
t.Errorf("Format was called on a path-only binding: %v", ops.formatCalls)
}
}
// DURABLE MISMATCH: the resolved device re-derives to a DIFFERENT id → refused (anti-retarget).
func TestWipe_DurableMismatchRefused(t *testing.T) {
ops := &fakeWipeOps{probe: storage.DeviceProbe{Probed: true, HasFilesystem: true}}
w := newWipeExec(ops,
func(string) (string, error) { return "/dev/sdb", nil },
func(string) (string, error) { return "byid:wwn-0xDIFFERENT", nil }) // re-derive disagrees
if err := w.Execute(context.Background(), "storage_wipe", params(t, durable, "ext4")); err == nil {
t.Fatal("a durable-id mismatch must be refused")
}
if len(ops.formatCalls) != 0 {
t.Errorf("Format was called despite a durable-id mismatch: %v", ops.formatCalls)
}
}
// DEVICE GONE: the durable id no longer resolves → refused (device removed/replaced).
func TestWipe_ResolveFailureRefused(t *testing.T) {
ops := &fakeWipeOps{probe: storage.DeviceProbe{Probed: true, HasFilesystem: true}}
w := newWipeExec(ops,
func(string) (string, error) { return "", errors.New("no such by-id link") },
func(string) (string, error) { return durable, nil })
if err := w.Execute(context.Background(), "storage_wipe", params(t, durable, "ext4")); err == nil {
t.Fatal("an unresolvable durable id must be refused")
}
if len(ops.formatCalls) != 0 {
t.Errorf("Format was called on an unresolvable device: %v", ops.formatCalls)
}
}
// RE-INSPECT: the resolved device is no longer data-bearing (target changed) → refused.
func TestWipe_ReinspectNonDataBearingRefused(t *testing.T) {
ops := &fakeWipeOps{probe: storage.DeviceProbe{Probed: true /* blank: no fs/parts */}}
w := newWipeExec(ops,
func(string) (string, error) { return "/dev/sdb", nil },
func(string) (string, error) { return durable, nil })
if err := w.Execute(context.Background(), "storage_wipe", params(t, durable, "ext4")); err == nil {
t.Fatal("a device that is no longer data-bearing must be refused (target changed)")
}
if len(ops.formatCalls) != 0 {
t.Errorf("Format was called on a non-target device: %v", ops.formatCalls)
}
}
// RE-INSPECT: the device did not probe cleanly → refused (fail-safe).
func TestWipe_NotProbedRefused(t *testing.T) {
ops := &fakeWipeOps{probe: storage.DeviceProbe{Probed: false}}
w := newWipeExec(ops,
func(string) (string, error) { return "/dev/sdb", nil },
func(string) (string, error) { return durable, nil })
if err := w.Execute(context.Background(), "storage_wipe", params(t, durable, "ext4")); err == nil {
t.Fatal("a device that did not probe cleanly must be refused")
}
if len(ops.formatCalls) != 0 {
t.Errorf("Format was called on an unprobed device: %v", ops.formatCalls)
}
}
// A non-wipe op → ErrNoExecutor (left for the owning slice; Format not called).
func TestWipe_OtherOpIsNoExecutor(t *testing.T) {
ops := &fakeWipeOps{}
w := newWipeExec(ops, func(string) (string, error) { return "/dev/sdb", nil }, func(string) (string, error) { return durable, nil })
if err := w.Execute(context.Background(), "guest_destroy", params(t, durable, "ext4")); !errors.Is(err, ErrNoExecutor) {
t.Fatalf("guest_destroy err = %v, want ErrNoExecutor", err)
}
}
+128
View File
@@ -0,0 +1,128 @@
package storage
import (
"fmt"
"os"
"path/filepath"
"sort"
"strings"
)
// Durable BLOCK-DEVICE identity for the slice-10B operator-signed wipe (anti-retarget). The
// signed op binds a DURABLE id (a WWN / hardware serial, falling back to a filesystem UUID), and
// execution re-resolves it to the CURRENT /dev path — so "wipe device X" wipes that exact physical
// device, never whatever happens to be at /dev/sdb now. These reads are world-readable udev
// symlinks under /dev/disk/by-id and /dev/disk/by-uuid — NO privilege and NO subprocess (the
// root-CLI fence is untouched).
//
// devDiskRoot is overridable for tests (a fake /dev/disk layout).
var devDiskRoot = "/dev/disk"
// durable-id scheme prefixes. byid: a /dev/disk/by-id/<name> entry (preferred — wwn/serial,
// survives reformat + re-cabling); byuuid: a filesystem UUID (fallback — survives re-cabling but
// not a reformat, acceptable for a one-shot wipe resolved immediately before wiping).
const (
durableByID = "byid:"
durableByUUID = "byuuid:"
)
// byIDPriority ranks /dev/disk/by-id link prefixes most-stable-first: wwn (hardware world-wide
// name) > nvme-eui (NVMe EUI) > nvme-/ata-/scsi-/usb- (model+serial). dm-/lvm-/md- names are
// excluded (they are mapper constructs, not the physical disk we want to bind a wipe to).
var byIDPriority = []string{"wwn-", "nvme-eui.", "nvme-", "ata-", "scsi-", "usb-"}
// DeviceDurableID derives a stable durable identifier for a block device by scanning the
// world-readable udev symlinks. Prefers a by-id (wwn/serial) link, then a filesystem UUID. Errors
// when neither is resolvable (a device with no durable identity cannot be safely wipe-bound).
func DeviceDurableID(device string) (string, error) {
target, err := filepath.EvalSymlinks(device)
if err != nil {
return "", fmt.Errorf("storage: resolve %s: %w", device, err)
}
// 1. best by-id link pointing at this device.
if name := bestByIDLink(target); name != "" {
return durableByID + name, nil
}
// 2. fallback: a filesystem UUID symlink.
if uuid := byUUIDFor(target); uuid != "" {
return durableByUUID + uuid, nil
}
return "", fmt.Errorf("storage: no durable id (wwn/serial/uuid) for device %s", device)
}
// ResolveDurableDevice resolves a durable id back to the CURRENT canonical /dev path, or errors if
// it no longer resolves (the device was physically removed/replaced — the wipe must then refuse).
func ResolveDurableDevice(durableID string) (string, error) {
switch {
case strings.HasPrefix(durableID, durableByID):
name := strings.TrimPrefix(durableID, durableByID)
if !safeLinkName(name) {
return "", fmt.Errorf("storage: unsafe durable id %q", durableID)
}
return filepath.EvalSymlinks(filepath.Join(devDiskRoot, "by-id", name))
case strings.HasPrefix(durableID, durableByUUID):
uuid := strings.TrimPrefix(durableID, durableByUUID)
if !safeLinkName(uuid) {
return "", fmt.Errorf("storage: unsafe durable id %q", durableID)
}
return filepath.EvalSymlinks(filepath.Join(devDiskRoot, "by-uuid", uuid))
default:
// A bare path or unknown scheme is REFUSED — a wipe must bind to a durable id, never a
// mutable /dev path (the anti-retarget invariant).
return "", fmt.Errorf("storage: durable id %q is not a durable scheme (byid:/byuuid:) — refusing path-only binding", durableID)
}
}
// bestByIDLink returns the highest-priority /dev/disk/by-id link name whose target is `device`
// (already symlink-resolved), or "" if none.
func bestByIDLink(device string) string {
dir := filepath.Join(devDiskRoot, "by-id")
entries, err := os.ReadDir(dir)
if err != nil {
return ""
}
// collect matching names, then pick by priority (then lexically for determinism).
var matches []string
for _, e := range entries {
name := e.Name()
if tgt, err := filepath.EvalSymlinks(filepath.Join(dir, name)); err == nil && tgt == device {
matches = append(matches, name)
}
}
if len(matches) == 0 {
return ""
}
sort.Strings(matches)
for _, pfx := range byIDPriority {
for _, name := range matches {
if strings.HasPrefix(name, pfx) {
return name
}
}
}
return matches[0] // some by-id link exists but not a recognized prefix — still durable enough
}
// byUUIDFor returns the filesystem UUID whose by-uuid link targets `device`, or "".
func byUUIDFor(device string) string {
dir := filepath.Join(devDiskRoot, "by-uuid")
entries, err := os.ReadDir(dir)
if err != nil {
return ""
}
for _, e := range entries {
if tgt, err := filepath.EvalSymlinks(filepath.Join(dir, e.Name())); err == nil && tgt == device {
return e.Name()
}
}
return ""
}
// safeLinkName rejects a durable-id component that could escape the by-id/by-uuid dir (path
// traversal / separators) — defense even though the value is operator-signed.
func safeLinkName(s string) bool {
if s == "" || strings.ContainsAny(s, "/\x00") || s == "." || s == ".." {
return false
}
return true
}
+117
View File
@@ -0,0 +1,117 @@
package storage
import (
"os"
"path/filepath"
"runtime"
"testing"
)
// fakeDevDisk builds a temp /dev/disk tree: a real "device" file + by-id/by-uuid symlinks to it.
// Returns the disk root + the device path. (filepath.EvalSymlinks needs real targets, so the
// "device" is a regular file standing in for a block device.) Skips on Windows where creating a
// symlink needs a privilege — the durable-device code is Linux-only (the agent runs on the PVE
// host), so these run on the build server / demo host.
func fakeDevDisk(t *testing.T, links map[string]string, uuids map[string]string) (root, device string) {
t.Helper()
if runtime.GOOS == "windows" {
t.Skip("durable-device symlink tests run on Linux (the agent's OS); symlink creation needs privilege on Windows")
}
base := t.TempDir()
device = filepath.Join(base, "sdb")
if err := os.WriteFile(device, []byte("x"), 0o600); err != nil {
t.Fatal(err)
}
root = filepath.Join(base, "disk")
mk := func(sub, name, target string) {
dir := filepath.Join(root, sub)
os.MkdirAll(dir, 0o755)
if err := os.Symlink(target, filepath.Join(dir, name)); err != nil {
t.Fatal(err)
}
}
for name := range links {
mk("by-id", name, device)
}
for uuid := range uuids {
mk("by-uuid", uuid, device)
}
return root, device
}
func withDevDiskRoot(t *testing.T, root string) {
t.Helper()
old := devDiskRoot
devDiskRoot = root
t.Cleanup(func() { devDiskRoot = old })
}
// DeviceDurableID prefers a wwn- by-id link over ata-/serial links and over a uuid.
func TestDeviceDurableID_PrefersWWN(t *testing.T) {
root, device := fakeDevDisk(t,
map[string]string{"wwn-0x5000c500abcd": "", "ata-Samsung_SSD_850_S1": "", "scsi-35000c500abcd": ""},
map[string]string{"1111-2222": ""})
withDevDiskRoot(t, root)
id, err := DeviceDurableID(device)
if err != nil {
t.Fatalf("DeviceDurableID: %v", err)
}
if id != "byid:wwn-0x5000c500abcd" {
t.Errorf("durable id = %q, want the wwn link", id)
}
}
// With no by-id link, it falls back to a filesystem UUID.
func TestDeviceDurableID_FallsBackToUUID(t *testing.T) {
root, device := fakeDevDisk(t, map[string]string{}, map[string]string{"abcd-ef01": ""})
withDevDiskRoot(t, root)
id, err := DeviceDurableID(device)
if err != nil {
t.Fatalf("DeviceDurableID: %v", err)
}
if id != "byuuid:abcd-ef01" {
t.Errorf("durable id = %q, want byuuid:abcd-ef01", id)
}
}
// A device with no durable identity at all → error (cannot be wipe-bound).
func TestDeviceDurableID_NoneErrors(t *testing.T) {
root, device := fakeDevDisk(t, map[string]string{}, map[string]string{})
withDevDiskRoot(t, root)
if _, err := DeviceDurableID(device); err == nil {
t.Fatal("a device with no wwn/serial/uuid must error (no durable id)")
}
}
// ResolveDurableDevice round-trips a by-id id back to the canonical device path.
func TestResolveDurableDevice_RoundTrip(t *testing.T) {
root, device := fakeDevDisk(t, map[string]string{"wwn-0xabc": ""}, map[string]string{})
withDevDiskRoot(t, root)
got, err := ResolveDurableDevice("byid:wwn-0xabc")
if err != nil {
t.Fatalf("ResolveDurableDevice: %v", err)
}
want, _ := filepath.EvalSymlinks(device)
if got != want {
t.Errorf("resolved %q, want %q", got, want)
}
}
// A path-only / unknown-scheme id is REFUSED (the anti-retarget invariant).
func TestResolveDurableDevice_RefusesPathOnly(t *testing.T) {
withDevDiskRoot(t, t.TempDir())
for _, bad := range []string{"/dev/sdb", "sdb", "uuid-no-scheme", "byid:../../etc/passwd", "byuuid:../x"} {
if _, err := ResolveDurableDevice(bad); err == nil {
t.Errorf("ResolveDurableDevice(%q) succeeded, want refusal", bad)
}
}
}
// A durable id whose link is gone → error (device removed/replaced).
func TestResolveDurableDevice_MissingErrors(t *testing.T) {
withDevDiskRoot(t, t.TempDir()) // empty: no by-id dir
if _, err := ResolveDurableDevice("byid:wwn-0xgone"); err == nil {
t.Fatal("an absent durable id must error")
}
}