admin
b2ca63ee9f
Closes the agent half of R-39's fleet fix. Requires hub >=0.68.0 for the re-arm signal; that hub is safe for 0.90.0 agents (unknown key dropped), so it deploys first. Three compounding defects let a box report `applied` while every PBS request 401'd: 1. The re-key was INVISIBLE. An ep0 re-issue rotates the secret of an existing token, so token_id/fingerprint/datastore/namespace come back byte-identical and the descriptor content hash never moved — the converged agent short-circuited and never consumed the fresh secret. WirePBSDR.SecretGeneration (field-exact with the hub) is what moves the hash now, because descriptorHash marshals this struct. 2. The agent could not READ its own credential. It writes /etc/pve/priv/storage/<id>.pw through the root wrapper, but that dir is 0700 root:www-data and the wrapper had no read verb — so the target resolver got "permission denied" every cycle, warned, and skipped. The one loop that could have caught the 401 was blind BY CONSTRUCTION. Adds a narrow `read` verb (+ exactly one sudoers line, + a pbsdr-read capability row): one secret to stdout, no network, no mutation, never in argv (sudo logs argv), traversal refused by the id grammar, the dir allowlist AND a resolved-path prefix assertion. 3. Nothing probed AUTHENTICATION. pbs.ProbeAuth (GET /version + an ErrUnauthorized sentinel) runs on the 15-minute collect path and its verdict becomes a loud `auth_failed` the hub escalates to a fresh mint. /version needs no datastore, namespace or privilege, so a 401 means the CREDENTIAL is bad; 403 is deliberately NOT treated as unauthorized, since re-keying a too-narrow token would mint forever without fixing anything. A transport error is UNKNOWN, never a rejection — otherwise every network blip burns a credential. Recovery self-clears. R-50b(a): the report now carries the installed wrapper's sha256 so drift against the vouched manifest value is answerable. Empty = unknown, never drift. Three red-proofs, all at the assertion level. Removing SecretGeneration fails the re-arm test with "consume calls=1, want 2". Swallowing the probe result leaves State:applied AuthFailed:false — the July-18 shape exactly. Notably, deleting the wrapper's id charset guard alone does NOT open a traversal hole (readlink + the prefix assertion still catch it), so the isolating red-proof removes BOTH and shows the out-of-tree secret printed — the layering is real, and a single-guard red-proof would have passed vacuously.
160 lines
8.9 KiB
Bash
160 lines
8.9 KiB
Bash
#!/bin/bash
|
|
#===============================================================================
|
|
# felhom-pbs-apply — the ONLY storage-entry path the felhom-agent sudoers permits for the PBS DR
|
|
# tier (slice 2; SPIKE-pbs-tier-provisioning-2026-07-10 §2b). The guarded-mkfs shape: the agent's
|
|
# in-process validation is the PRIMARY gate (descriptor-field charset + equality checks BEFORE
|
|
# exec); this wrapper is the minimal, auditable second gate as root.
|
|
#
|
|
# THE SET-ONLY LAW (spike §4, data-loss class): `pvesm` entry deletion DESTROYS the client
|
|
# encryption key file (<id>.enc = K) — un-decryptable backups. This wrapper therefore contains
|
|
# NO deletion path of any kind, and a re-apply is `pvesm set`-only. Grep-assertable; do not add
|
|
# a "cleanup" verb here, ever. Deprovision is a deliberate future operator op, not this tool.
|
|
#
|
|
# SECRET DISCIPLINE (spike §2b): sudo logs its full argv to auth.log → the PBS token secret
|
|
# arrives on STDIN, never as an argument to this wrapper. Inside, it is passed to pvesm's
|
|
# --password (root-local, transient ps exposure — the accepted spike posture); it is never
|
|
# echoed, never written anywhere except by pvesm itself (the 0600 .pw store).
|
|
#
|
|
# Ops (non-secret args on argv):
|
|
# create <id> <server> <datastore> <namespace> <token-id> <fingerprint> <secret-dir>
|
|
# secret on stdin (required). Creates the pbs entry with --encryption-key autogen
|
|
# (K born at /etc/pve/priv/storage/<id>.enc), then places .pw/.enc copies in
|
|
# <secret-dir> when it differs (the §4b WARN-fix dir; escrow-create's PBSEncKeyPath
|
|
# must find K there — spike §4 flag).
|
|
# reconcile <id> <server> <namespace> <token-id> <fingerprint> <secret-dir>
|
|
# secret on stdin (optional; empty = no credential change). `pvesm set` ONLY:
|
|
# server/fingerprint (+ --password when a secret is fed, e.g. after a hub re-issue).
|
|
# NOTE datastore is deliberately NOT settable, and namespace/token-id are accepted
|
|
# for validation parity but NOT applied — tenancy identity is adopt-only (the
|
|
# demo's live entry must never be repointed at a different namespace).
|
|
# read <id> <secret-dir>
|
|
# R-39 leg (b): print the storage's token secret to STDOUT and nothing else.
|
|
# The non-root agent WRITES this file through this wrapper but could never read it
|
|
# back (/etc/pve/priv is 0700 root:www-data and there is no read verb), so its
|
|
# 15-minute PBS verify loop was permanently blind to the one failure it exists to
|
|
# catch — an `applied` tier authenticating 401. This verb is that missing read.
|
|
# It is deliberately the narrowest thing that works: no network, no mutation, no
|
|
# logging of the value, one file, prefix-asserted under the given secret dir.
|
|
#
|
|
# grant <id>
|
|
# The Part-0-evidenced dual-grant: FelhomAgentStore on /storage/<id> to the agent
|
|
# user AND token (privsep intersection). Datastore.Audit reads ride the base role.
|
|
#===============================================================================
|
|
set -euo pipefail
|
|
|
|
die() { echo "felhom-pbs-apply: REFUSED: $*" >&2; exit 1; }
|
|
|
|
op="${1:-}"; id="${2:-}"
|
|
[[ -n "$op" && -n "$id" ]] || die "usage: felhom-pbs-apply <create|reconcile|grant|read> <storage-id> ..."
|
|
|
|
# Storage id: PVE grammar, conservative. Also the ACL path component — no slashes possible.
|
|
[[ "$id" =~ ^[A-Za-z][A-Za-z0-9_.-]{0,27}$ ]] || die "bad storage id ($id)"
|
|
|
|
STORECFG=/etc/pve/storage.cfg
|
|
PRIVDIR=/etc/pve/priv/storage
|
|
|
|
entry_exists() { grep -Eq "^pbs: ${id}\$" "$STORECFG"; }
|
|
|
|
val_server() { [[ "$1" =~ ^[A-Za-z0-9][A-Za-z0-9.-]{0,62}$ ]] || die "bad server ($1)"; }
|
|
val_datastore() { [[ "$1" =~ ^[A-Za-z0-9][A-Za-z0-9_.-]{0,31}$ ]] || die "bad datastore ($1)"; }
|
|
val_ns() { [[ "$1" =~ ^[A-Za-z0-9][A-Za-z0-9_.-]{0,31}$ ]] || die "bad namespace ($1)"; }
|
|
val_tok() { [[ "$1" =~ ^[A-Za-z0-9_.-]+@[A-Za-z0-9]+![A-Za-z0-9_.-]+$ ]] || die "bad token id ($1)"; }
|
|
val_fp() { [[ "$1" =~ ^([A-Fa-f0-9]{2}:){31}[A-Fa-f0-9]{2}$ ]] || die "bad fingerprint"; }
|
|
val_sdir() {
|
|
case "$1" in
|
|
/etc/pve/priv/storage) : ;;
|
|
/var/lib/felhom-agent/*) [[ "$1" != *..* ]] || die "secret-dir traversal" ;;
|
|
*) die "secret-dir must be $PRIVDIR or under /var/lib/felhom-agent ($1)" ;;
|
|
esac
|
|
}
|
|
|
|
read_secret() { # → SECRET (may be empty when optional)
|
|
local s
|
|
s=$(head -c 256 || true)
|
|
s="${s%$'\n'}"; s="${s%$'\r'}"
|
|
printf '%s' "$s"
|
|
}
|
|
|
|
place_copies() { # secret-dir — the §4b WARN-fix placement (non-root agent can't read /etc/pve/priv)
|
|
local sdir="$1"
|
|
[[ "$sdir" == "$PRIVDIR" ]] && return 0
|
|
install -d -o felhom-agent -g felhom-agent -m 0700 "$sdir"
|
|
[[ -f "$PRIVDIR/$id.pw" ]] && install -o felhom-agent -g felhom-agent -m 0600 "$PRIVDIR/$id.pw" "$sdir/$id.pw"
|
|
# K's copy: escrow-create stats PBSEncKeyPath(<secret-dir>/<id>.enc) — the spike §4 flag.
|
|
[[ -f "$PRIVDIR/$id.enc" ]] && install -o root -g felhom-agent -m 0640 "$PRIVDIR/$id.enc" "$sdir/$id.enc"
|
|
return 0
|
|
}
|
|
|
|
case "$op" in
|
|
create)
|
|
[[ $# -eq 8 ]] || die "create needs 7 args: <id> <server> <datastore> <namespace> <token-id> <fingerprint> <secret-dir>"
|
|
server="$3"; datastore="$4"; ns="$5"; tok="$6"; fp="$7"; sdir="$8"
|
|
val_server "$server"; val_datastore "$datastore"; val_ns "$ns"; val_tok "$tok"; val_fp "$fp"; val_sdir "$sdir"
|
|
entry_exists && die "entry $id already exists (reconcile is the re-apply path — set-only law)"
|
|
SECRET=$(read_secret)
|
|
[[ -n "$SECRET" ]] || die "create requires the token secret on stdin"
|
|
pvesm add pbs "$id" \
|
|
--server "$server" --datastore "$datastore" --namespace "$ns" \
|
|
--username "$tok" --password "$SECRET" --fingerprint "$fp" \
|
|
--content backup --encryption-key autogen >&2
|
|
[[ -f "$PRIVDIR/$id.enc" ]] || die "pvesm add succeeded but K ($PRIVDIR/$id.enc) was not born"
|
|
place_copies "$sdir"
|
|
echo "felhom-pbs-apply: created $id (K born; encryption-key autogen)" >&2
|
|
;;
|
|
reconcile)
|
|
[[ $# -eq 7 ]] || die "reconcile needs 6 args: <id> <server> <namespace> <token-id> <fingerprint> <secret-dir>"
|
|
server="$3"; ns="$4"; tok="$5"; fp="$6"; sdir="$7"
|
|
val_server "$server"; val_ns "$ns"; val_tok "$tok"; val_fp "$fp"; val_sdir "$sdir"
|
|
entry_exists || die "entry $id does not exist (create is the fresh path)"
|
|
SECRET=$(read_secret)
|
|
# R-39 (2026-07-18): NEVER pass --server to `pvesm set`. PVE treats `server` as a CREATE-ONLY
|
|
# parameter and rejects the ENTIRE call — "can't change value of fixed parameter 'server'" —
|
|
# even when the value is byte-identical to the stored one. That made every reconcile exit 255,
|
|
# so each hub-re-issued one-time secret was consumed-then-burned and the tier stayed pinned to
|
|
# a revoked credential (401 forever). Proven live on the N100 demo host: `pvesm set <id>
|
|
# --server <same> --fingerprint <same>` -> rejected; the same call without --server -> rc 0.
|
|
# The server address is immutable by construction (relocating a PBS endpoint needs a fresh
|
|
# create), so there is nothing here to reconcile. Guarded by
|
|
# TestReconcileNeverPassesServerToPvesmSet.
|
|
args=(--fingerprint "$fp")
|
|
[[ -n "$SECRET" ]] && args+=(--password "$SECRET")
|
|
pvesm set "$id" "${args[@]}" >&2
|
|
place_copies "$sdir"
|
|
echo "felhom-pbs-apply: reconciled $id (set-only; tenancy identity untouched)" >&2
|
|
;;
|
|
read)
|
|
# R-39(b): the missing read path. Prints the secret to STDOUT, nothing else — no stderr note (it
|
|
# would be the only verb whose success line could be confused with the value), no mutation.
|
|
#
|
|
# Traversal is refused three times over, because this is the one verb that EXFILTRATES a file and
|
|
# its argv is attacker-shaped if the agent is ever compromised:
|
|
# 1. `id` already matched ^[A-Za-z][A-Za-z0-9_.-]{0,27}$ at the top — it cannot start with a dot
|
|
# and cannot contain a slash, so "../../etc/shadow" never reaches here;
|
|
# 2. val_sdir pins the directory to PRIVDIR or under /var/lib/felhom-agent, rejecting "..";
|
|
# 3. the RESOLVED path is prefix-asserted under that directory below, so even a future change to
|
|
# either grammar cannot walk out.
|
|
[[ $# -eq 3 ]] || die "read needs 2 args: <id> <secret-dir>"
|
|
sdir="$3"
|
|
val_sdir "$sdir"
|
|
target="$sdir/$id.pw"
|
|
# Belt: resolve and re-check the prefix (guards a symlinked <id>.pw pointing outside the dir).
|
|
resolved=$(readlink -f -- "$target" 2>/dev/null || true)
|
|
[[ -n "$resolved" ]] || die "secret file not found ($target)"
|
|
case "$resolved" in
|
|
"$sdir"/*) : ;;
|
|
*) die "resolved secret path escapes the secret dir" ;;
|
|
esac
|
|
[[ -f "$resolved" ]] || die "secret file not found ($target)"
|
|
cat -- "$resolved"
|
|
;;
|
|
grant)
|
|
[[ $# -eq 2 ]] || die "grant takes only <id>"
|
|
pveum acl modify "/storage/$id" --users felhom-agent@pve --roles FelhomAgentStore >&2
|
|
pveum acl modify "/storage/$id" --tokens 'felhom-agent@pve!agent' --roles FelhomAgentStore >&2
|
|
echo "felhom-pbs-apply: granted FelhomAgentStore on /storage/$id (user + token)" >&2
|
|
;;
|
|
*)
|
|
die "unknown op ($op)"
|
|
;;
|
|
esac
|