installer (not tagged): a new box trusts Proxmox by a copy of its own root CA; the pinned certificate is the fallback (R-1380)
This commit is contained in:
1 parent
327a7668e6
commit
fe8247665d
3 files changed
+182
-7
No files matched your search
@@ -1,3 +1,15 @@
|
||||
## Unreleased (2026-11-14 night) — the installer: a new box trusts Proxmox by a copy of its own root CA (R-1380, D80) — on `main`, NOT tagged, so NOT published
|
||||
|
||||
- `felhom-host-install.sh` `resolve_pve_trust` (new, called by `step_agent_config`): copy `/etc/pve/pve-root-ca.pem` to
|
||||
`/etc/felhom-agent/pve-root-ca.pem` when it is one unexpired CA certificate; prove a TLS handshake to
|
||||
`127.0.0.1:8006` with the copy alone; then write `proxmox.tls.ca_file` and NO fingerprint. Anything else: the served
|
||||
certificate is pinned as before (never neither). `SCRIPT_VERSION` is unchanged: the version moves with the tag.
|
||||
- `test_hostinstall.py`: five tests against a real CA, a real leaf and a real `openssl s_server` (skipped where openssl
|
||||
is missing — the CI runner); six mutations seen red (`documentation/audits/night-build-2026-11-14/A/redproof-installer.txt`).
|
||||
- Design: `documentation/architecture/03-host-agent.md` §12a. Needs agent ≥ the release that carries wrapper mode
|
||||
`pve-ca-copy` only for the REFRESH of the copy; the copy itself and `ca_file` work with any agent (the key exists
|
||||
since the first agent).
|
||||
|
||||
## 2026-11-07 — the installer pin moves to `INSTALLER-TAG`; DooPlex's copy of the website retired (R-1228, operator yes)
|
||||
|
||||
- **Why.** Since 2026-10-30 Cloudflare Pages serves felhom.eu; DooPlex's git-sync + nginx copy was the way back until
|
||||
|
||||
@@ -3173,16 +3173,67 @@ EOF
|
||||
#-------------------------------------------------------------------------------
|
||||
# STEP 6 — write agent config + ensure service healthy
|
||||
#-------------------------------------------------------------------------------
|
||||
# resolve_pve_trust — R-1380 (D80, `09` §3 decision 303; `03-host-agent.md` §12a): a NEW box trusts Proxmox by a copy of
|
||||
# its OWN root CA, not by one pinned certificate (Proxmox makes a new node certificate by itself, at most every two
|
||||
# years, and a pinned agent is then blind). The agent's user cannot read Proxmox's CA file, so root copies that ONE
|
||||
# public certificate to where `proxmox.tls.ca_file` points; the agent's root step keeps it fresh afterwards
|
||||
# (felhom-os-apply, mode pve-ca-copy — the same source, the same place, the same checks).
|
||||
#
|
||||
# NEVER NEITHER: the copy is used only after a TLS handshake to the address the agent calls was verified with the copy
|
||||
# ALONE. If the source is not one CA certificate, or the copy cannot be written, or that handshake does not verify, the
|
||||
# box gets the pinned certificate as before (and the agent moves to the copy by itself later).
|
||||
# Sets PVE_CA_FILE (the copy, or "") and PVE_TLS_FP (the pin: EMPTY when the copy is used).
|
||||
resolve_pve_trust() {
|
||||
local src="${PVE_CA_SOURCE:-/etc/pve/pve-root-ca.pem}" dst="${PVE_CA_COPY:-/etc/felhom-agent/pve-root-ca.pem}"
|
||||
local addr="${PVE_API_ADDR:-127.0.0.1:8006}" fp why=""
|
||||
PVE_CA_FILE=""; PVE_TLS_FP=""
|
||||
# the SERVED leaf cert fingerprint (not pvesh node info — may differ): the fallback
|
||||
fp=$(echo | openssl s_client -connect "$addr" 2>/dev/null | openssl x509 -noout -fingerprint -sha256 2>/dev/null | sed 's/.*=//')
|
||||
if [[ ! -r "$src" ]]; then
|
||||
why="$src is not readable"
|
||||
elif [[ "$(grep -c 'BEGIN CERTIFICATE' "$src")" != "1" ]] || grep -q 'PRIVATE KEY' "$src"; then
|
||||
why="$src is not exactly one certificate"
|
||||
elif ! openssl x509 -in "$src" -noout -ext basicConstraints 2>/dev/null | grep -q 'CA:TRUE'; then
|
||||
why="$src is not a CA certificate"
|
||||
elif ! openssl x509 -in "$src" -noout -checkend 0 >/dev/null 2>&1; then
|
||||
why="$src has ended"
|
||||
fi
|
||||
if [[ -z "$why" ]]; then
|
||||
if $DRY_RUN; then
|
||||
log_dry "copy $src -> $dst (0644) and prove a TLS handshake to $addr with the copy alone; then proxmox.tls.ca_file=$dst, no fingerprint"
|
||||
PVE_CA_FILE="$dst"; return 0
|
||||
fi
|
||||
mkdir -p "$(dirname "$dst")"
|
||||
if install -m 0644 "$src" "$dst.felhom-new" && mv -f "$dst.felhom-new" "$dst"; then
|
||||
# THE PROOF, before the config names the copy: verified by the copy ALONE (no system roots), for the address
|
||||
# the agent calls (the IP must be in the certificate, as the agent's own check demands)
|
||||
if echo | openssl s_client -connect "$addr" -CAfile "$dst" -no-CApath -verify_return_error \
|
||||
-verify_ip "${addr%%:*}" 2>&1 | grep -q 'Verify return code: 0 (ok)'; then
|
||||
PVE_CA_FILE="$dst"
|
||||
log_info " Proxmox trust: the box's own root CA ($dst; a handshake to $addr verified with it alone) — no pinned certificate"
|
||||
return 0
|
||||
fi
|
||||
why="a TLS handshake to $addr did not verify with the copy alone"
|
||||
else
|
||||
why="the copy could not be written to $dst"
|
||||
fi
|
||||
rm -f "$dst.felhom-new"
|
||||
fi
|
||||
PVE_TLS_FP="$fp"
|
||||
log_warn " Proxmox trust: the root CA copy is NOT used ($why) — the served certificate is pinned instead, as before; the agent moves to the CA copy by itself when it can (R-1380)"
|
||||
[[ -n "$fp" ]] || log_warn " could not compute TLS fingerprint either (leaving both empty — agent will use system trust)"
|
||||
}
|
||||
|
||||
step_agent_config() {
|
||||
log_step "6/8 agent config + service"
|
||||
# E-2 Part 2: resolve the whole-guest backup target BEFORE the config is written — the resolved
|
||||
# value feeds local_backup_target below. Scenario F safe: it returns early, changing nothing, when
|
||||
# a target already exists, and `setdefault` preserves an existing box's whole backup block anyway.
|
||||
configure_backup_target
|
||||
# TLS pin: the SERVED leaf cert fingerprint (not pvesh node info — may differ)
|
||||
# Trust in Proxmox (R-1380): the box's own root CA copy; the pinned certificate only as the fallback.
|
||||
local fp
|
||||
fp=$(echo | openssl s_client -connect 127.0.0.1:8006 2>/dev/null | openssl x509 -noout -fingerprint -sha256 2>/dev/null | sed 's/.*=//')
|
||||
[[ -n "$fp" ]] || log_warn " could not compute TLS fingerprint (leaving empty — agent will use system trust)"
|
||||
resolve_pve_trust
|
||||
fp="$PVE_TLS_FP"
|
||||
# bridge / local-api addr. LAN_IP (vmbr0) is always resolved — it is the lan_resolver anchor even
|
||||
# on an island install (Finding-1), and the historical bind on a non-island install.
|
||||
LAN_IP=$(ip -4 -o addr show vmbr0 2>/dev/null | awk '{print $4}' | cut -d/ -f1 | head -1)
|
||||
@@ -3200,10 +3251,10 @@ step_agent_config() {
|
||||
elif [[ "$BRIDGE_ADDR" != *:* ]]; then
|
||||
BRIDGE_ADDR="${BRIDGE_ADDR}:8443"
|
||||
fi
|
||||
log_info " node=$NODE local_api=$BRIDGE_ADDR tls_fp=${fp:0:17}…"
|
||||
log_info " node=$NODE local_api=$BRIDGE_ADDR tls_ca=${PVE_CA_FILE:-<none>} tls_fp=${fp:0:17}${fp:+…}"
|
||||
|
||||
if $DRY_RUN; then
|
||||
log_dry "write $AGENT_CONFIG (0600): proxmox{endpoint,node=$NODE,token=<secret>,tls.fingerprint=$fp} hub{url=$HUB_URL,host_id=$HOST_ID,api_key=<secret>} local_api{$BRIDGE_ADDR}"
|
||||
log_dry "write $AGENT_CONFIG (0600): proxmox{endpoint,node=$NODE,token=<secret>,tls.ca_file=$PVE_CA_FILE,tls.fingerprint=$fp} hub{url=$HUB_URL,host_id=$HOST_ID,api_key=<secret>} local_api{$BRIDGE_ADDR}"
|
||||
if [[ -n "$RESOLVED_OP_LINE$RESOLVED_REC_LINE" ]]; then
|
||||
log_dry "write authz.signers: operational=${RESOLVED_OP_ID:-<none>} recovery=${RESOLVED_REC_ID:-<none>} (operator-signed self-update authority)"
|
||||
else
|
||||
@@ -3222,7 +3273,7 @@ step_agent_config() {
|
||||
# Build config: optional preserve base + fresh-host defaults + Day-0 overrides.
|
||||
# Secrets passed via env (NOT argv) to avoid ps exposure.
|
||||
PVE_TOKEN="$PVE_TOKEN" HOST_API_KEY="$HOST_API_KEY" \
|
||||
NODE="$NODE" FP="$fp" HUB_URL="$HUB_URL" HOST_ID="$HOST_ID" BRIDGE_ADDR="$BRIDGE_ADDR" \
|
||||
NODE="$NODE" FP="$fp" PVE_CA_FILE="$PVE_CA_FILE" HUB_URL="$HUB_URL" HOST_ID="$HOST_ID" BRIDGE_ADDR="$BRIDGE_ADDR" \
|
||||
ISLAND_ON="$ISLAND_ON" ISLAND_BRIDGE="$ISLAND_BRIDGE" ISLAND_GUEST_CIDR="$ISLAND_GUEST_CIDR" LAN_IP="$LAN_IP" \
|
||||
OP_KEY_ID="$RESOLVED_OP_ID" OP_KEY_LINE="$RESOLVED_OP_LINE" \
|
||||
REC_KEY_ID="$RESOLVED_REC_ID" REC_KEY_LINE="$RESOLVED_REC_LINE" \
|
||||
@@ -3332,7 +3383,9 @@ base['proxmox'] = {
|
||||
"endpoint":"https://127.0.0.1:8006",
|
||||
"node": os.environ['NODE'],
|
||||
"token": os.environ['PVE_TOKEN'],
|
||||
"tls": {"fingerprint": os.environ['FP'], "insecure_skip_verify": False},
|
||||
# R-1380: ca_file (the box's own root CA copy) OR the pinned fingerprint — never both, and the fingerprint only
|
||||
# when the copy could not be proven. A set fingerprint wins over ca_file in the agent.
|
||||
"tls": {"ca_file": os.environ.get('PVE_CA_FILE', ''), "fingerprint": os.environ['FP'], "insecure_skip_verify": False},
|
||||
}
|
||||
base['hub'] = {
|
||||
"url": os.environ['HUB_URL'],
|
||||
|
||||
@@ -22,6 +22,7 @@ import stat
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
SCRIPT = os.path.join(HERE, "felhom-host-install.sh")
|
||||
@@ -726,6 +727,115 @@ def test_http_code_prints_the_status_on_success():
|
||||
assert _http_code_with("200").endswith("[200]")
|
||||
|
||||
|
||||
|
||||
# ── R-1380 (D80): a new box trusts Proxmox by a copy of its own root CA; the pinned certificate is the fallback ──────
|
||||
# NOT stubs: a real CA, a real leaf for 127.0.0.1 and a real `openssl s_server`, because the proof IS a TLS handshake.
|
||||
# Skipped (and it says so) where openssl is missing — the CI runner has none; the pre-push run on DooPlex has.
|
||||
class _PVE:
|
||||
def __init__(self, sb):
|
||||
import socket
|
||||
self.sb, self.proc = sb, None
|
||||
d = sb.path("pki")
|
||||
os.mkdir(d)
|
||||
def sh(*a):
|
||||
subprocess.run(a, check=True, capture_output=True, cwd=d)
|
||||
for ca in ("ca", "other"):
|
||||
sh("openssl", "req", "-x509", "-newkey", "rsa:2048", "-nodes", "-keyout", ca + ".key", "-out", ca + ".pem", "-days", "30",
|
||||
"-subj", "/CN=" + ca, "-addext", "basicConstraints=critical,CA:TRUE")
|
||||
sh("openssl", "req", "-newkey", "rsa:2048", "-nodes", "-keyout", "leaf.key", "-out", "leaf.csr", "-subj", "/CN=node")
|
||||
open(os.path.join(d, "ext"), "w").write("subjectAltName=IP:127.0.0.1\nbasicConstraints=CA:FALSE\n")
|
||||
sh("openssl", "x509", "-req", "-in", "leaf.csr", "-CA", "ca.pem", "-CAkey", "ca.key", "-CAcreateserial", "-out", "leaf.pem",
|
||||
"-days", "30", "-extfile", "ext")
|
||||
s = socket.socket(); s.bind(("127.0.0.1", 0)); self.port = s.getsockname()[1]; s.close()
|
||||
self.proc = subprocess.Popen(["openssl", "s_server", "-accept", "127.0.0.1:%d" % self.port, "-cert", "leaf.pem", "-key", "leaf.key",
|
||||
"-www", "-quiet"], cwd=d, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
|
||||
for _ in range(50):
|
||||
try:
|
||||
socket.create_connection(("127.0.0.1", self.port), timeout=0.2).close(); break
|
||||
except OSError:
|
||||
time.sleep(0.1)
|
||||
self.dir = d
|
||||
self.leaf_fp = subprocess.run(["openssl", "x509", "-in", "leaf.pem", "-noout", "-fingerprint", "-sha256"], capture_output=True,
|
||||
text=True, cwd=d).stdout.strip().split("=", 1)[1]
|
||||
|
||||
def close(self):
|
||||
if self.proc:
|
||||
self.proc.kill(); self.proc.wait()
|
||||
|
||||
|
||||
def _trust_run(source, dry=False):
|
||||
"""Run resolve_pve_trust against a real TLS server. Returns (ca_file, fp, copy bytes or None, source bytes, output, leaf fp, dst)."""
|
||||
if not shutil.which("openssl"):
|
||||
return None
|
||||
sb = Sandbox()
|
||||
pve = _PVE(sb)
|
||||
try:
|
||||
dst = sb.path("etc-felhom-agent", "pve-root-ca.pem")
|
||||
src = os.path.join(pve.dir, source)
|
||||
rc, out = sb.run(prelude(dry) + func("resolve_pve_trust") +
|
||||
'PVE_CA_SOURCE="%s" PVE_CA_COPY="%s" PVE_API_ADDR=127.0.0.1:%d\nresolve_pve_trust\n'
|
||||
'echo "RESULT ca=[$PVE_CA_FILE] fp=[$PVE_TLS_FP]"\n' % (src, dst, pve.port))
|
||||
assert rc == 0, out
|
||||
m = re.search(r"RESULT ca=\[(.*?)\] fp=\[(.*?)\]", out)
|
||||
assert m, out
|
||||
copy = open(dst, "rb").read() if os.path.exists(dst) else None
|
||||
return m.group(1), m.group(2), copy, open(src, "rb").read() if os.path.exists(src) else None, out, pve.leaf_fp, dst
|
||||
finally:
|
||||
pve.close()
|
||||
sb.close()
|
||||
|
||||
|
||||
def test_pve_trust_a_new_box_gets_the_ca_copy_and_no_fingerprint():
|
||||
r = _trust_run("ca.pem")
|
||||
if r is None:
|
||||
print(" (skipped: no openssl here)"); return
|
||||
ca, fp, copy, src, out, _, dst = r
|
||||
assert ca == dst and fp == "", "the box's own CA verified the handshake, so: ca_file set, NO fingerprint. Got ca=%r fp=%r\n%s" % (ca, fp, out)
|
||||
assert copy == src, "the copy is not the source, byte for byte"
|
||||
|
||||
|
||||
def test_pve_trust_a_copy_that_does_not_verify_falls_back_to_the_pin():
|
||||
r = _trust_run("other.pem") # a CA certificate, but not the one that signed the served certificate
|
||||
if r is None:
|
||||
print(" (skipped: no openssl here)"); return
|
||||
ca, fp, _, _, out, leaf_fp, _ = r
|
||||
assert ca == "" and fp == leaf_fp, "never neither: no proven copy → the served certificate is pinned. Got ca=%r fp=%r\n%s" % (ca, fp, out)
|
||||
assert "NOT used" in out and "did not verify" in out, out
|
||||
|
||||
|
||||
def test_pve_trust_a_source_that_is_not_a_ca_is_never_copied():
|
||||
for source, why in (("leaf.pem", "not a CA certificate"), ("leaf.key", "not exactly one certificate"), ("missing.pem", "not readable")):
|
||||
r = _trust_run(source)
|
||||
if r is None:
|
||||
print(" (skipped: no openssl here)"); return
|
||||
ca, fp, copy, _, out, leaf_fp, _ = r
|
||||
assert ca == "" and copy is None, "%s: copied although it is %s" % (source, why)
|
||||
assert fp == leaf_fp, "%s: the fallback pin is missing: %r" % (source, fp)
|
||||
assert why in out, "%s: the reason is not named:\n%s" % (source, out)
|
||||
|
||||
|
||||
def test_pve_trust_dry_run_writes_nothing():
|
||||
r = _trust_run("ca.pem", dry=True)
|
||||
if r is None:
|
||||
print(" (skipped: no openssl here)"); return
|
||||
_, _, copy, _, out, _, _ = r
|
||||
assert copy is None and "[DRY-RUN]" in out, out
|
||||
|
||||
|
||||
def test_pve_trust_is_wired_into_the_agent_config():
|
||||
body = func("step_agent_config")
|
||||
assert re.search(r"^\s+resolve_pve_trust\s*$", body, re.M), "step_agent_config no longer calls resolve_pve_trust"
|
||||
assert 'fp="$PVE_TLS_FP"' in body and 'PVE_CA_FILE="$PVE_CA_FILE"' in body, "the result does not reach the config writer"
|
||||
assert "\"ca_file\": os.environ.get('PVE_CA_FILE', '')" in body and "\"fingerprint\": os.environ['FP']" in body, \
|
||||
"the config writer does not write both keys"
|
||||
# the agent's root step and the installer name the same two files
|
||||
wrapper = open(os.path.join(os.path.dirname(SCRIPT), "..", "..", "felhom-agent", "configs", "felhom-os-apply"), encoding="utf-8").read() \
|
||||
if os.path.exists(os.path.join(os.path.dirname(SCRIPT), "..", "..", "felhom-agent", "configs", "felhom-os-apply")) else None
|
||||
if wrapper is not None:
|
||||
for name, path in (("PVE_CA_SOURCE", "/etc/pve/pve-root-ca.pem"), ("PVE_CA_COPY", "/etc/felhom-agent/pve-root-ca.pem")):
|
||||
assert '%s = "%s"' % (name, path) in wrapper and path in func("resolve_pve_trust"), "the installer and the agent's root step disagree on %s" % name
|
||||
|
||||
|
||||
def main():
|
||||
tests = [(n, f) for n, f in sorted(globals().items()) if n.startswith("test_") and callable(f)]
|
||||
fails = 0
|
||||
|
||||
Reference in new issue
Block a user