2b (unreleased): the agent moves from a pinned Proxmox certificate to the box's root CA copy — copy, proof, then the config; the cause of a trust failure is named (R-1380)
gates / gates (push) Successful in 38s
gates / gates (push) Successful in 38s
This commit is contained in:
1 parent
a9ab47ec71
commit
386a0c4cf4
13 files changed
+1191
-6
No files matched your search
@@ -9,6 +9,27 @@ session's. `configs/` changes, so the release needs BOTH signed jobs.
|
||||
name nothing (T1); no source (T2); not one CA certificate (T3). No appliance check: the copy is the agent's own trust
|
||||
file and changes nothing of a BYO owner's host. Tests `PVECACopy` (9, one with the real openssl through the real
|
||||
runner); 7 mutations seen red (felhom.eu `audits/night-build-2026-11-14/A/redproof-wrapper.txt`).
|
||||
- **The move of a box that exists, in a safe order** (`internal/pvetrust`, new; wired in `cmd/felhom-agent/main.go`
|
||||
at the agent's start, after a config bundle install, on a timer — hourly until moved, daily after — and on every
|
||||
call to Proxmox that fails on trust). (1) the root step makes the copy; (2) ONE real call to Proxmox with the copy
|
||||
ALONE, while the pinned fingerprint is still the configured trust; (3) only then ONE write of `agent.json` sets
|
||||
`proxmox.tls.ca_file` and clears `fingerprint` (the agent's own file, rewritten in place after a recovery copy
|
||||
`agent.json.pre-pve-trust` was parked, read back and checked); (4) the RUNNING client takes the new trust
|
||||
(`proxmox.Client.SetTLS`, new) — no restart. A failed copy, proof or write changes nothing: the fingerprint stays.
|
||||
**`proxmox.tls` keeps its order (a set fingerprint wins over a CA file), so the fingerprint is cleared LAST.** An
|
||||
operator's own CA file, a box with checks switched off and a config that is not a file are never moved.
|
||||
- **A pinned box whose certificate changed heals by itself:** the failed call starts the same order, the box moves to
|
||||
the CA copy and the next call works. A refreshed copy (a renewed root CA) is proven and reloaded by the running
|
||||
client.
|
||||
- **The cause is named** (`proxmox.TrustProblem`, new): the pinned certificate changed; the CA copy did not sign the
|
||||
served certificate; a certificate ended; the copy is unreadable; the copy ends within 30 days; the root step gave no
|
||||
copy; the proof failed. An ERROR line in the journal, and the new `proxmox_trust` stanza of the host report
|
||||
(`mode`, `problem`, `detail`, `ca_not_after`, `ca_days_left`, `moved_at`). **The hub does not read the stanza yet**
|
||||
(felhom.eu R-1396), and a box that is fully blind sends no host report at all — that case still shows only as
|
||||
„host stale" on the hub.
|
||||
- Tests: `internal/pvetrust` (11, against a real TLS server that changes its certificate and renews its CA), the AST
|
||||
wiring test `TestMainWiresPVETrust`; 12 + 3 mutations seen red (felhom.eu `audits/night-build-2026-11-14/A/redproof-agent.txt`;
|
||||
one mutation is not a rule and says so).
|
||||
|
||||
## v0.174.0 (2026-11-14) — the box removes its old kernels (R-1368); security profiles are refreshed after a kernel change (R-1388, R-1301); Go 1.27.4 (R-1391) — RELEASED 2026-11-14 (release-agent.sh from the clean working clone at `cc63dd5`, CI run id 2641; agent sha256 6efbb97809135483bc446d71d1321f9e2f7fc3268fe9b20e7fe742a2eb13db92, bundle sha256 7d87d0584dee79eee766f01c27b9e89273d39c89a14b0725a6e5564a55b3ec5f) and VOUCHED 2026-11-14 15:21:45 CET
|
||||
|
||||
|
||||
@@ -100,6 +100,7 @@
|
||||
| `osupdate.KernelVerdict` / `Leg.KernelAfterBoot` | internal/osupdate/kernel.go | `KernelVerdict(before, after, tunnel, hubReached) (ok, why)` | R-836: THE one-shot-boot rule — the host health rule (`HostHealthVerdict`) AND the box reached the hub since the boot | The only judge of a new kernel. Never reboot the host from Go: every reboot is the wrapper's (`kernel-reboot`, `kernel-revert`), and only for a staged step. A new kernel-lane state lives in the wrapper's `/var/lib/felhom-kernel/state.json`, never in the agent's own files (the agent can write those). |
|
||||
| `Leg.runKernelCleanup` / `kernelCleanupSkip` / `Leg.KernelCleanupDryRun` + the wrapper's `Kernel.cleanup` | internal/osupdate/kernel.go; configs/felhom-os-apply (mode `kernel-cleanup`) | `kernelCleanupSkip(appliance, trigger, host, pve, kernel, blk, okStep) string` ("" = call it) | R-1368 (`11` §5.11.1): old kernel images are removed by the WRAPPER's rule; the agent decides only WHEN (the night leg, after a healthy host step, never in a night with a kernel step) | The plan names NO package (C1). Never `apt-get autoremove`. Whether a box may do it is the root-owned ring-0 mark, never the hub's ring. The hub report carries NO kernel view and no kernel version (the hub would read it as a kernel step). `--selftest=kernel-cleanup-dry-run` changes nothing. |
|
||||
| `Leg.ProfileRefresh` + the wrapper's `Apply.profile_refresh` | internal/osupdate/profilerefresh.go; configs/felhom-os-apply (mode `profile-refresh`) | `ProfileRefresh(ctx, why, gate, tries, wait) string` (the outcome) | R-1388 (`11` §5.12): load every AppArmor profile from a fresh compile once per kernel the box starts on; restart only a FAILED service whose own program's cached profile was stale | The plan names nothing (F1). A stale copy is told by the CACHED FILE changing at the rewrite — never by the kernel's loaded hash (it differs between a cache load and a fresh load for multi-profile files; `lxc-containers` never compiles the same twice). Called at agent start and after a bundle install, under the heavy-op gate. |
|
||||
| `pvetrust.Manager` (`Ensure`, `OnCallError`, `Status`) + `proxmox.Client.SetTLS` / `proxmox.TrustProblem` + `Leg.PVECACopy` + the wrapper's `pve_ca_copy` | internal/pvetrust/pvetrust.go; internal/proxmox/{client,trust}.go; internal/osupdate/pvecacopy.go; configs/felhom-os-apply (mode `pve-ca-copy`) | `Ensure(ctx, why) (changed bool)`; `TrustProblem(err) string` | R-1380 (felhom.eu `03` §12a): trust in Proxmox by a copy of the box's own root CA; the move from a pinned certificate; the cause of a trust failure | THE ORDER: the copy, a real call with the copy ALONE, then ONE config write that clears the fingerprint LAST (a set fingerprint wins over `ca_file` in `TLSConfig.build`). Never clear the fingerprint anywhere else. The wrapper's plan names nothing (T1). `agent.json` is written by the agent in place, with a parked recovery copy — the same way as the backup-target move. |
|
||||
| `Client.WaitTask` | internal/proxmox/task.go | `WaitTask(ctx, upid, opts) (TaskStatus, error)` | asserting EVERY mutating op | POST 200 ≠ success; authz can fail at task exec; `AllowWarnings` opt-in |
|
||||
| `Client.Pool` | internal/proxmox/query.go | `Pool(ctx, name) (PoolInfo, error)` | felhom-pool membership (the ownership registry, A1) | Needs `Pool.Audit` at `/pool/<name>` (host-install v1.9.0+); `Pool.Allocate` does NOT satisfy the read; members can be storages (type `storage`, vmid 0) — filter them |
|
||||
| `Client` mutate wrappers (`RestoreLXC/Vzdump/DestroyLXC/Snapshot/Rollback/SetConfig/ResizeLXC/Start/Stop`) | internal/proxmox/mutate.go | return `(upid, error)` | all API mutations | Async → always pair with WaitTask; route via gate/queue, not ad-hoc |
|
||||
|
||||
@@ -54,6 +54,7 @@ import (
|
||||
"gitea.dooplex.hu/admin/felhom-agent/internal/poweroff"
|
||||
"gitea.dooplex.hu/admin/felhom-agent/internal/provision"
|
||||
"gitea.dooplex.hu/admin/felhom-agent/internal/proxmox"
|
||||
"gitea.dooplex.hu/admin/felhom-agent/internal/pvetrust"
|
||||
"gitea.dooplex.hu/admin/felhom-agent/internal/reconcile"
|
||||
"gitea.dooplex.hu/admin/felhom-agent/internal/restorespace"
|
||||
"gitea.dooplex.hu/admin/felhom-agent/internal/selfheal"
|
||||
@@ -988,6 +989,13 @@ func runDaemon(cfg config.Config, logger *slog.Logger, logRing *applog.Ring) int
|
||||
osLeg := newOSLeg(cfg, client, px, logger)
|
||||
collector.SetSystemReporter(&factsReporter{leg: osLeg, guest: firstGuest(px)}) // R-852: the versions
|
||||
desiredSyncer.AddConsumer(osLeg)
|
||||
// R-1380 (D80, felhom.eu `03` §12a): trust in Proxmox by a copy of the box's own root CA. At this start the root
|
||||
// step makes the copy, a REAL call proves it while the pinned fingerprint is still the trust, and only then ONE
|
||||
// write of the config sets ca_file and clears the fingerprint. A call that fails on trust later names the cause and
|
||||
// runs the same order again (a pinned box whose certificate changed moves right then, with no restart).
|
||||
pveTrust := newPVETrust(cfg, px, osLeg, logger)
|
||||
collector.SetProxmoxTrustSource(func() any { return pveTrust.Status() })
|
||||
loop.SetCollectErrorHook(pveTrust.OnCallError)
|
||||
// S5: consume a host_loss restore_directive into an inspectable restore PLAN (derive + surface,
|
||||
// execute nothing). The recipe is fetched on-demand (rare directive) via a fresh Collect.
|
||||
desiredSyncer.AddConsumer(dr.NewConsumer(func(ctx context.Context) *hub.DRRecipeHostHalf {
|
||||
@@ -1018,6 +1026,10 @@ func runDaemon(cfg config.Config, logger *slog.Logger, logRing *applog.Ring) int
|
||||
// network and stays under the hub's 45-minute host_stale (its
|
||||
// alerting.stale_threshold; host_down at 90). On a box without the kernel lane (an older wrapper, a BYO host) the check is refused and logged.
|
||||
go osLeg.KernelAfterBoot(ctx, 0, osupdate.KernelJudge{Wait: osupdate.DefaultKernelJudgeWait})
|
||||
// R-1380: the trust order at this start (the copy, the proof, then the config) — before the first report cycle, so
|
||||
// a pinned box whose certificate changed while the agent was down is not blind for a cycle. Then on a timer.
|
||||
pveTrust.Ensure(ctx, "start")
|
||||
go pveTrust.Run(ctx, time.Hour, 24*time.Hour)
|
||||
// R-1192: every guest may swap as much as its RAM — raised once at start (a live change, no restart). A minute
|
||||
// after start so the guest has settled; a failure is logged and the next start retries.
|
||||
go func() {
|
||||
@@ -1301,6 +1313,8 @@ func runDaemon(cfg config.Config, logger *slog.Logger, logRing *applog.Ring) int
|
||||
// R-1388: a new wrapper may know the profile refresh the old one refused — ask again (its own goroutine:
|
||||
// it may wait for the heavy-op gate, and the signed job must not wait with it).
|
||||
go osLeg.ProfileRefresh(daemonCtx, "config bundle installed", profileGate, 12, 5*time.Minute)
|
||||
// R-1380: a new wrapper may know the CA-copy step the old one refused — run the trust order again.
|
||||
go pveTrust.Ensure(daemonCtx, "config bundle installed")
|
||||
}}
|
||||
// R-1388 / R-1301 (`11` §5.12): after a start, load the security profiles from a fresh compile when the box runs
|
||||
// another kernel than at the last refresh (the wrapper keeps that record and answers "not needed" otherwise).
|
||||
@@ -3948,6 +3962,29 @@ func (f *selftestFlag) Set(v string) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// newPVETrust wires internal/pvetrust (R-1380): the root step through the OS wrapper, the proof as a throwaway client
|
||||
// with EXACTLY the trust under test, and the running client's SetTLS.
|
||||
func newPVETrust(cfg config.Config, px *proxmox.Client, leg *osupdate.Leg, logger *slog.Logger) *pvetrust.Manager {
|
||||
m := &pvetrust.Manager{ConfigPath: cfg.SourcePath, StateDir: agentStateDir(cfg), Logger: logger,
|
||||
EnsureCopy: func(ctx context.Context) (pvetrust.Copy, error) {
|
||||
r, err := leg.PVECACopy(ctx)
|
||||
return pvetrust.Copy{Path: r.Path, Outcome: r.Outcome, NotAfter: r.NotAfter, DaysLeft: r.DaysLeft}, err
|
||||
},
|
||||
Probe: func(ctx context.Context, t proxmox.TLSConfig) error {
|
||||
p, err := proxmox.NewClient(proxmox.Config{Endpoint: cfg.Proxmox.Endpoint, Node: cfg.Proxmox.Node, Token: cfg.Proxmox.Token, TLS: t})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
_, err = p.Version(ctx)
|
||||
return err
|
||||
},
|
||||
Apply: px.SetTLS,
|
||||
}
|
||||
m.Start(proxmox.TLSConfig{CAFile: cfg.Proxmox.TLS.CAFile, Fingerprint: cfg.Proxmox.TLS.Fingerprint,
|
||||
InsecureSkipVerify: cfg.Proxmox.TLS.InsecureSkipVerify})
|
||||
return m
|
||||
}
|
||||
|
||||
// newOSLeg builds the OS-update leg (agent v0.140.0). The wrapper runs through sudo (FELHOM_OSAPPLY); the plan and
|
||||
// the once-per-night marker live in the agent's own os/ dir.
|
||||
func newOSLeg(cfg config.Config, client *hub.Client, px *proxmox.Client, logger *slog.Logger) *osupdate.Leg {
|
||||
|
||||
@@ -0,0 +1,62 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"go/ast"
|
||||
"go/parser"
|
||||
"go/token"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// R-1380: the trust move is a component with seams (the root step, the proof, the running client, the report stanza,
|
||||
// the failed-call hook). A unit-green pvetrust that main.go never starts is the inert-seam defect, so the wiring is
|
||||
// asserted from the AST: built once, run at start and after a bundle install, on a timer, fed by a failed collect, and
|
||||
// reported. The same for the profile refresh (R-1388), which had no such test.
|
||||
func TestMainWiresPVETrust(t *testing.T) {
|
||||
fset := token.NewFileSet()
|
||||
f, err := parser.ParseFile(fset, "main.go", nil, 0)
|
||||
if err != nil {
|
||||
t.Fatalf("parse main.go: %v", err)
|
||||
}
|
||||
seen := map[string]int{}
|
||||
ast.Inspect(f, func(n ast.Node) bool {
|
||||
switch node := n.(type) {
|
||||
case *ast.CallExpr:
|
||||
sel, ok := node.Fun.(*ast.SelectorExpr)
|
||||
if !ok {
|
||||
return true
|
||||
}
|
||||
x, _ := sel.X.(*ast.Ident)
|
||||
switch {
|
||||
case x != nil && x.Name == "pveTrust" && sel.Sel.Name == "Ensure":
|
||||
seen["ensure"]++
|
||||
case sel.Sel.Name == "SetProxmoxTrustSource":
|
||||
seen["stanza"]++
|
||||
case sel.Sel.Name == "SetCollectErrorHook" && len(node.Args) == 1:
|
||||
if a, ok := node.Args[0].(*ast.SelectorExpr); ok && a.Sel.Name == "OnCallError" {
|
||||
seen["hook"]++
|
||||
}
|
||||
case x != nil && x.Name == "osLeg" && sel.Sel.Name == "ProfileRefresh":
|
||||
seen["profile-refresh"]++
|
||||
}
|
||||
case *ast.GoStmt:
|
||||
if sel, ok := node.Call.Fun.(*ast.SelectorExpr); ok && sel.Sel.Name == "Run" {
|
||||
if id, ok := sel.X.(*ast.Ident); ok && id.Name == "pveTrust" {
|
||||
seen["timer"]++
|
||||
}
|
||||
}
|
||||
case *ast.KeyValueExpr:
|
||||
if k, ok := node.Key.(*ast.Ident); ok && k.Name == "Apply" {
|
||||
if v, ok := node.Value.(*ast.SelectorExpr); ok && v.Sel.Name == "SetTLS" {
|
||||
seen["apply=SetTLS"]++
|
||||
}
|
||||
}
|
||||
}
|
||||
return true
|
||||
})
|
||||
want := map[string]int{"ensure": 2, "stanza": 1, "hook": 1, "timer": 1, "apply=SetTLS": 1, "profile-refresh": 2}
|
||||
for k, n := range want {
|
||||
if seen[k] < n {
|
||||
t.Errorf("main.go wires %q %d time(s), want at least %d (the two-call ones: at start and after a bundle install)", k, seen[k], n)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -124,6 +124,7 @@ type Collector struct {
|
||||
temp TempReader // slice 9: host CPU/chassis temp (nil-safe → nil temp)
|
||||
capProbe func(ctx context.Context) []capability.Status // v0.44.0: privileged-capability self-check (nil → empty)
|
||||
leafFP string // v0.48.0: served local-API leaf fp (static per process; "" when local API disabled)
|
||||
pveTrust func() any // R-1380: the proxmox_trust stanza (nil = omitted)
|
||||
addrEnum AddressEnumerator // v0.119.0: host interface enumeration; nil => the REAL one (see collectAddresses)
|
||||
wg WireguardReporter // S3: offsite-tunnel status (nil → stanza omitted)
|
||||
pbsdr PBSDRReporter // slice 2: PBS DR tier bridge state (nil → stanza omitted)
|
||||
@@ -209,6 +210,12 @@ func (c *Collector) SetCapabilityProber(probe func(ctx context.Context) []capabi
|
||||
return c
|
||||
}
|
||||
|
||||
// SetProxmoxTrustSource wires the `proxmox_trust` stanza (R-1380). Optional; nil = the stanza is omitted.
|
||||
func (c *Collector) SetProxmoxTrustSource(f func() any) *Collector {
|
||||
c.pveTrust = f
|
||||
return c
|
||||
}
|
||||
|
||||
// SetLeafFingerprint records the served local-API leaf fp (v0.48.0) to ride every host report (the hub
|
||||
// watches it for a re-key). Static per process — set once at startup. "" when the local API is
|
||||
// disabled. Returns the collector for chaining.
|
||||
@@ -391,6 +398,13 @@ func (c *Collector) SetOOBReporter(o OOBReporter) *Collector {
|
||||
// error (no useful report — the cycle skips the POST); a failed per-guest
|
||||
// GuestConfig degrades that guest to status="unknown" without spec but still sends;
|
||||
// a cloudflared probe failure yields status="unknown" and is never fatal.
|
||||
func (c *Collector) proxmoxTrust() any {
|
||||
if c.pveTrust == nil {
|
||||
return nil
|
||||
}
|
||||
return c.pveTrust()
|
||||
}
|
||||
|
||||
func (c *Collector) Collect(ctx context.Context) (*HostReport, error) {
|
||||
ns, err := c.px.NodeStatus(ctx)
|
||||
if err != nil {
|
||||
@@ -420,6 +434,7 @@ func (c *Collector) Collect(ctx context.Context) (*HostReport, error) {
|
||||
Cloudflared: c.cloudflared(ctx),
|
||||
Capabilities: c.capabilities(ctx),
|
||||
LeafFingerprint: c.leafFP,
|
||||
ProxmoxTrust: c.proxmoxTrust(),
|
||||
Addresses: c.collectAddresses(),
|
||||
System: c.systemInfo(ctx, ns),
|
||||
}
|
||||
|
||||
@@ -70,8 +70,14 @@ type Loop struct {
|
||||
// reported (R-998) is called after every report the hub ACCEPTED, with that report. The power-off
|
||||
// path waits on it so the hub has the deliberate-off record before the box goes down.
|
||||
reported func(r *HostReport)
|
||||
// collectErr (R-1380) sees every failed collect.
|
||||
collectErr func(ctx context.Context, err error)
|
||||
}
|
||||
|
||||
// SetCollectErrorHook wires an observer of a failed collect (R-1380: a call to Proxmox that fails on TRUST starts the
|
||||
// move to the CA copy, or a reload of a refreshed copy). Optional; it must not block for long.
|
||||
func (l *Loop) SetCollectErrorHook(f func(ctx context.Context, err error)) { l.collectErr = f }
|
||||
|
||||
// SetReportedHook wires the R-998 after-report observer. Optional.
|
||||
func (l *Loop) SetReportedHook(f func(r *HostReport)) { l.reported = f }
|
||||
|
||||
@@ -144,6 +150,9 @@ func (l *Loop) cycle(ctx context.Context, current time.Duration) time.Duration {
|
||||
report, err := l.collector.Collect(ctx)
|
||||
if err != nil {
|
||||
l.logger.Warn("hub: collect failed; skipping this cycle's report", "err", err)
|
||||
if l.collectErr != nil {
|
||||
l.collectErr(ctx, err)
|
||||
}
|
||||
return current
|
||||
}
|
||||
// Fulfill a pending log-pull: attach the ring tail to THIS report and clear the
|
||||
|
||||
@@ -53,6 +53,11 @@ type HostReport struct {
|
||||
// alert. Not a secret (the fp is public; the token is never reported).
|
||||
LeafFingerprint string `json:"leaf_fingerprint"`
|
||||
|
||||
// ProxmoxTrust (R-1380, felhom.eu `03` §12a): how the agent trusts Proxmox's certificate (`mode`: ca | fingerprint |
|
||||
// …), and the CAUSE when something about it is wrong (`problem`, `detail`): the proof with the CA copy failed, the
|
||||
// copy ends soon, the root step gave no copy. Omitted when not wired. No secret: a mode, a cause, a date.
|
||||
ProxmoxTrust any `json:"proxmox_trust,omitempty"`
|
||||
|
||||
// Addresses are the host's routable addresses, one entry per (interface, address) — the LAN
|
||||
// bridge, the WG tunnel, a tailnet. Added v0.119.0 because the hub could not show a managed
|
||||
// box's IP anywhere: nothing in this report carried one. Non-nil so it marshals as [];
|
||||
|
||||
@@ -139,7 +139,9 @@ type WrapperReport struct {
|
||||
KernelCleanup json.RawMessage `json:"kernel_cleanup"`
|
||||
// ProfileRefresh (R-1388, `11` §5.12): mode profile-refresh's result {outcome, kernel, rewritten, restarted, seconds}.
|
||||
ProfileRefresh json.RawMessage `json:"profile_refresh"`
|
||||
OutcomeHint string `json:"outcome_hint"`
|
||||
// PVECACopy (R-1380, felhom.eu `03` §12a): mode pve-ca-copy's result {outcome, path, sha256, not_after, days_left}.
|
||||
PVECACopy json.RawMessage `json:"pve_ca_copy"`
|
||||
OutcomeHint string `json:"outcome_hint"`
|
||||
// R-868 (v0.144.0): the agent's ids, echoed from the plan, so a report kept on disk can be sent without the
|
||||
// agent process that started the pass. ReleaseID / VMID were always in the report.
|
||||
RunID string `json:"run_id"`
|
||||
|
||||
@@ -0,0 +1,45 @@
|
||||
package osupdate
|
||||
|
||||
// The root step that keeps a readable copy of Proxmox's root CA for the agent (R-1380, D80, felhom.eu `03` §12a).
|
||||
// The rule lives in the wrapper (mode pve-ca-copy: one fixed source, one fixed place, a CA certificate only); the plan
|
||||
// names nothing. internal/pvetrust decides when to ask and what to do with the answer.
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
)
|
||||
|
||||
// PVECACopyPath is where the wrapper puts the copy (felhom-os-apply PVE_CA_COPY) — the agent's `proxmox.tls.ca_file`.
|
||||
const PVECACopyPath = "/etc/felhom-agent/pve-root-ca.pem"
|
||||
|
||||
// PVECACopyResult is the wrapper's pve_ca_copy object.
|
||||
type PVECACopyResult struct {
|
||||
Outcome string `json:"outcome"` // written | same
|
||||
Path string `json:"path"`
|
||||
SHA256 string `json:"sha256"`
|
||||
NotAfter string `json:"not_after"`
|
||||
DaysLeft int `json:"days_left"`
|
||||
Replaced bool `json:"replaced"`
|
||||
}
|
||||
|
||||
// PVECACopy asks the wrapper to make or refresh the copy. An error names a refusal (T1–T3, or R11 from a wrapper that
|
||||
// does not know the mode yet) or a wrapper that gave no answer.
|
||||
func (l *Leg) PVECACopy(ctx context.Context) (PVECACopyResult, error) {
|
||||
wr, err := l.call(ctx, "pveca"+l.now().UTC().Format("150405"), map[string]any{"release_id": "pve-ca-copy",
|
||||
"layer": LayerHost, "mode": "pve-ca-copy"})
|
||||
if err != nil {
|
||||
return PVECACopyResult{}, err
|
||||
}
|
||||
if wr.refused() {
|
||||
return PVECACopyResult{}, fmt.Errorf("pve-ca-copy refused: %s", wr.Refused)
|
||||
}
|
||||
if wr.failed() {
|
||||
return PVECACopyResult{}, fmt.Errorf("pve-ca-copy failed: %s", wr.Failed)
|
||||
}
|
||||
var res PVECACopyResult
|
||||
if jerr := json.Unmarshal(wr.PVECACopy, &res); jerr != nil || res.Outcome == "" || res.Path == "" {
|
||||
return PVECACopyResult{}, fmt.Errorf("pve-ca-copy: the wrapper gave no result (%v)", jerr)
|
||||
}
|
||||
return res, nil
|
||||
}
|
||||
@@ -10,6 +10,7 @@ import (
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"gitea.dooplex.hu/admin/felhom-agent/internal/httpx"
|
||||
@@ -45,6 +46,39 @@ type Client struct {
|
||||
node string
|
||||
token string
|
||||
http doer
|
||||
|
||||
// tlsMu guards http: SetTLS swaps the transport while calls run (R-1380 — the move from a pinned certificate to the
|
||||
// box's own root CA, and a refreshed CA copy, take effect with no agent restart).
|
||||
tlsMu sync.RWMutex
|
||||
timeout time.Duration
|
||||
}
|
||||
|
||||
func (c *Client) httpDoer() doer {
|
||||
c.tlsMu.RLock()
|
||||
defer c.tlsMu.RUnlock()
|
||||
return c.http
|
||||
}
|
||||
|
||||
// SetTLS replaces how this client trusts the Proxmox certificate, for every call that starts after it returns. On an
|
||||
// error (an unreadable CA file, a bad fingerprint) nothing changes.
|
||||
func (c *Client) SetTLS(t TLSConfig) error {
|
||||
tlsCfg, err := t.build()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
timeout := c.timeout
|
||||
if timeout == 0 {
|
||||
timeout = 30 * time.Second
|
||||
}
|
||||
hc := &http.Client{Timeout: timeout, Transport: httpx.NewTransport(tlsCfg, 0)}
|
||||
c.tlsMu.Lock()
|
||||
old := c.http
|
||||
c.http = hc
|
||||
c.tlsMu.Unlock()
|
||||
if o, ok := old.(*http.Client); ok && o != nil {
|
||||
o.CloseIdleConnections() // a kept-alive connection made under the old trust must not serve the next call
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// NewClient builds an API client. It validates required config and constructs the
|
||||
@@ -74,10 +108,11 @@ func NewClient(cfg Config) (*Client, error) {
|
||||
Transport: httpx.NewTransport(tlsCfg, 0),
|
||||
}
|
||||
return &Client{
|
||||
base: strings.TrimRight(cfg.Endpoint, "/") + "/api2/json",
|
||||
node: cfg.Node,
|
||||
token: cfg.Token,
|
||||
http: hc,
|
||||
base: strings.TrimRight(cfg.Endpoint, "/") + "/api2/json",
|
||||
node: cfg.Node,
|
||||
token: cfg.Token,
|
||||
http: hc,
|
||||
timeout: timeout,
|
||||
}, nil
|
||||
}
|
||||
|
||||
@@ -125,7 +160,7 @@ func (c *Client) doBody(ctx context.Context, method, path string, body io.Reader
|
||||
req.Header.Set("Content-Type", contentType)
|
||||
}
|
||||
|
||||
resp, err := c.http.Do(req)
|
||||
resp, err := c.httpDoer().Do(req)
|
||||
if err != nil {
|
||||
return fmt.Errorf("proxmox: %s %s: %w", method, path, err)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,43 @@
|
||||
package proxmox
|
||||
|
||||
import (
|
||||
"crypto/x509"
|
||||
"errors"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// The causes a call to Proxmox can fail on TRUST (R-1380, felhom.eu `03` §12a). A symptom alarm („the agent channel is
|
||||
// down") exists; these name the cause.
|
||||
const (
|
||||
TrustPinnedCertChanged = "pinned_cert_changed" // the pinned certificate is not the one Proxmox serves any more
|
||||
TrustCAMismatch = "ca_mismatch" // the CA file did not sign the served certificate
|
||||
TrustCertExpired = "cert_expired" // the served certificate (or the CA) has ended
|
||||
TrustCAFileUnreadable = "ca_file_unreadable" // the CA file is missing or holds no certificate
|
||||
TrustNameMismatch = "name_mismatch" // the certificate does not name the address the agent calls
|
||||
)
|
||||
|
||||
// TrustProblem names why err is a TRUST failure, or "" when it is none (a refused connection, a 401, a timeout).
|
||||
func TrustProblem(err error) string {
|
||||
if err == nil {
|
||||
return ""
|
||||
}
|
||||
var ua x509.UnknownAuthorityError
|
||||
var inv x509.CertificateInvalidError
|
||||
var hn x509.HostnameError
|
||||
switch {
|
||||
case strings.Contains(err.Error(), "TLS pin mismatch"):
|
||||
return TrustPinnedCertChanged
|
||||
case errors.As(err, &ua):
|
||||
return TrustCAMismatch
|
||||
case errors.As(err, &inv):
|
||||
if inv.Reason == x509.Expired {
|
||||
return TrustCertExpired
|
||||
}
|
||||
return TrustCAMismatch
|
||||
case errors.As(err, &hn):
|
||||
return TrustNameMismatch
|
||||
case strings.Contains(err.Error(), "reading TLS CA file"), strings.Contains(err.Error(), "contained no usable certificates"):
|
||||
return TrustCAFileUnreadable
|
||||
}
|
||||
return ""
|
||||
}
|
||||
@@ -0,0 +1,421 @@
|
||||
// Package pvetrust moves the agent's trust in Proxmox from ONE pinned certificate to the box's own root CA, and names
|
||||
// the cause when a call fails on trust (R-1380, D80, felhom.eu `09` §3 decision 303; design `03-host-agent.md` §12a).
|
||||
//
|
||||
// WHY. Proxmox makes a new node certificate by itself, at most every two years. An agent that pins the old one is then
|
||||
// blind: every call fails, no host report reaches the hub, and the only alarm names a symptom. A copy of the box's own
|
||||
// root CA verifies every certificate that CA signs.
|
||||
//
|
||||
// THE ORDER, and it is the whole point (a box must never be left with neither way working):
|
||||
//
|
||||
// 1. the root step makes or refreshes the copy (it refuses anything that is not one CA certificate);
|
||||
// 2. a REAL call to Proxmox is made with the copy ALONE — while the fingerprint still is the configured trust;
|
||||
// 3. only after that call worked, ONE write of the agent's config sets `ca_file` and clears the fingerprint;
|
||||
// 4. the running client takes the new trust (no restart).
|
||||
//
|
||||
// If 1 or 2 fails nothing is written and the fingerprint stays. If 3 fails the fingerprint stays, in the file and in
|
||||
// the running client. `proxmox.tls` keeps its order in code — a set fingerprint wins over a CA file — so the
|
||||
// fingerprint is cleared LAST, in the same write that sets `ca_file`.
|
||||
//
|
||||
// WHO WRITES THE CONFIG. The agent itself: `agent.json` is the agent's own file (0600) in a root-owned folder, so it is
|
||||
// rewritten in place after a recovery copy was parked in the agent's state folder — the same way the backup-target move
|
||||
// and the PBS seed write it. No root step touches it.
|
||||
package pvetrust
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"gitea.dooplex.hu/admin/felhom-agent/internal/proxmox"
|
||||
)
|
||||
|
||||
// EndsSoonDays: the copy's certificate ends within this many days → the problem „ca_copy_ends_soon". Proxmox renews its
|
||||
// root CA when less than two years are left, so a healthy box never comes near it.
|
||||
const EndsSoonDays = 30
|
||||
|
||||
// Problems this package names beyond proxmox.Trust*.
|
||||
const (
|
||||
ProblemCopyUnavailable = "ca_copy_unavailable" // the root step refused or did not answer
|
||||
ProblemCopyNoProof = "ca_copy_does_not_verify" // a call with the copy alone failed — the fingerprint stays
|
||||
ProblemCopyEndsSoon = "ca_copy_ends_soon"
|
||||
ProblemConfigNotSaved = "config_not_written" // the proof worked but the config could not be written — the fingerprint stays
|
||||
)
|
||||
|
||||
// Copy is what the root step answered.
|
||||
type Copy struct {
|
||||
Path string
|
||||
Outcome string // written | same
|
||||
NotAfter string
|
||||
DaysLeft int
|
||||
}
|
||||
|
||||
// Status is the stanza that rides the host report (`proxmox_trust`). No secret: a mode, a cause, a date.
|
||||
type Status struct {
|
||||
Mode string `json:"mode"` // ca | fingerprint | ca_file_own | insecure | system
|
||||
Problem string `json:"problem,omitempty"` // "" = none
|
||||
Detail string `json:"detail,omitempty"`
|
||||
CANotAfter string `json:"ca_not_after,omitempty"`
|
||||
CADaysLeft *int `json:"ca_days_left,omitempty"`
|
||||
MovedAt string `json:"moved_at,omitempty"` // when this box moved from the pinned certificate to the CA copy
|
||||
CheckedAt string `json:"checked_at,omitempty"`
|
||||
}
|
||||
|
||||
// Manager holds the trust state of one agent process.
|
||||
type Manager struct {
|
||||
// EnsureCopy runs the root step (osupdate.Leg.PVECACopy).
|
||||
EnsureCopy func(ctx context.Context) (Copy, error)
|
||||
// Probe makes one real call to Proxmox with EXACTLY this trust (a throwaway client).
|
||||
Probe func(ctx context.Context, t proxmox.TLSConfig) error
|
||||
// Apply gives the running client the new trust (proxmox.Client.SetTLS).
|
||||
Apply func(t proxmox.TLSConfig) error
|
||||
// ConfigPath is agent.json ("" = the config came from the environment: nothing can be saved, so nothing is moved).
|
||||
ConfigPath string
|
||||
StateDir string
|
||||
Logger *slog.Logger
|
||||
Now func() time.Time
|
||||
// MinGap between two runs that a failed call asked for (default 10 minutes).
|
||||
MinGap time.Duration
|
||||
|
||||
mu sync.Mutex
|
||||
cur proxmox.TLSConfig
|
||||
status Status
|
||||
lastRun time.Time
|
||||
}
|
||||
|
||||
func (m *Manager) now() time.Time {
|
||||
if m.Now != nil {
|
||||
return m.Now()
|
||||
}
|
||||
return time.Now()
|
||||
}
|
||||
|
||||
func (m *Manager) log() *slog.Logger {
|
||||
if m.Logger != nil {
|
||||
return m.Logger
|
||||
}
|
||||
return slog.Default()
|
||||
}
|
||||
|
||||
// Start records the trust the process started with. Call once, before Ensure.
|
||||
func (m *Manager) Start(t proxmox.TLSConfig) {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
m.cur = t
|
||||
m.status = Status{Mode: modeOf(t)}
|
||||
}
|
||||
|
||||
func modeOf(t proxmox.TLSConfig) string {
|
||||
switch {
|
||||
case t.InsecureSkipVerify:
|
||||
return "insecure"
|
||||
case t.Fingerprint != "":
|
||||
return "fingerprint"
|
||||
case t.CAFile != "":
|
||||
return "ca"
|
||||
}
|
||||
return "system"
|
||||
}
|
||||
|
||||
// Status is the current stanza (nil-safe).
|
||||
func (m *Manager) Status() *Status {
|
||||
if m == nil {
|
||||
return nil
|
||||
}
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
s := m.status
|
||||
return &s
|
||||
}
|
||||
|
||||
// Moved reports whether the box trusts Proxmox by the CA copy (nothing is left to move).
|
||||
func (m *Manager) Moved() bool {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
return m.cur.Fingerprint == "" && m.cur.CAFile != ""
|
||||
}
|
||||
|
||||
// Run asks again on a timer until ctx ends: every `pending` while the box is not moved yet (a wrapper that did not
|
||||
// know the mode, a Proxmox that did not answer), every `settled` after (a renewed CA refreshes the copy).
|
||||
func (m *Manager) Run(ctx context.Context, pending, settled time.Duration) {
|
||||
for {
|
||||
d := settled
|
||||
if !m.Moved() {
|
||||
d = pending
|
||||
}
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-time.After(d):
|
||||
m.Ensure(ctx, "timer")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Current is the trust the running client uses.
|
||||
func (m *Manager) Current() proxmox.TLSConfig {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
return m.cur
|
||||
}
|
||||
|
||||
// OnCallError is the hook for a failed Proxmox call: when the failure is one of TRUST, it names the cause and runs
|
||||
// Ensure (at most once per MinGap) — a pinned box whose certificate changed moves to the CA copy right then, and a box
|
||||
// whose CA copy was refreshed reloads it. A failure of another kind does nothing.
|
||||
func (m *Manager) OnCallError(ctx context.Context, err error) {
|
||||
cause := proxmox.TrustProblem(err)
|
||||
if cause == "" {
|
||||
return
|
||||
}
|
||||
m.mu.Lock()
|
||||
gap := m.MinGap
|
||||
if gap <= 0 {
|
||||
gap = 10 * time.Minute
|
||||
}
|
||||
tooSoon := !m.lastRun.IsZero() && m.now().Sub(m.lastRun) < gap
|
||||
if m.status.Problem == "" {
|
||||
m.status.Problem, m.status.Detail = cause, causeText(cause)
|
||||
}
|
||||
m.mu.Unlock()
|
||||
m.log().Error("pvetrust: a call to Proxmox failed on TRUST — "+causeText(cause), "cause", cause, "err", err)
|
||||
if tooSoon {
|
||||
return
|
||||
}
|
||||
m.Ensure(ctx, cause)
|
||||
}
|
||||
|
||||
func causeText(cause string) string {
|
||||
switch cause {
|
||||
case proxmox.TrustPinnedCertChanged:
|
||||
return "the pinned certificate is not the one Proxmox serves any more (Proxmox made a new certificate)"
|
||||
case proxmox.TrustCAMismatch:
|
||||
return "the CA copy did not sign the certificate Proxmox serves (the box's root CA changed, or the copy is not this box's)"
|
||||
case proxmox.TrustCertExpired:
|
||||
return "the certificate Proxmox serves, or its CA, has ended"
|
||||
case proxmox.TrustCAFileUnreadable:
|
||||
return "the CA copy is missing or holds no certificate"
|
||||
case proxmox.TrustNameMismatch:
|
||||
return "the certificate Proxmox serves does not name the address the agent calls"
|
||||
}
|
||||
return cause
|
||||
}
|
||||
|
||||
// Ensure runs the order of the package comment once. why is a log word ("start", "daily", a trust cause). It returns
|
||||
// true when the running trust changed.
|
||||
func (m *Manager) Ensure(ctx context.Context, why string) bool {
|
||||
m.mu.Lock()
|
||||
cur := m.cur
|
||||
m.lastRun = m.now()
|
||||
m.mu.Unlock()
|
||||
lg := m.log().With("why", why, "mode", modeOf(cur))
|
||||
set := func(mode, problem, detail string, c *Copy) {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
moved := m.status.MovedAt
|
||||
m.status = Status{Mode: mode, Problem: problem, Detail: detail, MovedAt: moved, CheckedAt: m.now().UTC().Format(time.RFC3339)}
|
||||
if c != nil {
|
||||
d := c.DaysLeft
|
||||
m.status.CANotAfter, m.status.CADaysLeft = c.NotAfter, &d
|
||||
}
|
||||
}
|
||||
if cur.InsecureSkipVerify {
|
||||
lg.Warn("pvetrust: certificate checks are switched off in the config (insecure_skip_verify) — nothing is moved")
|
||||
set("insecure", "", "", nil)
|
||||
return false
|
||||
}
|
||||
if m.EnsureCopy == nil || m.Probe == nil {
|
||||
return false
|
||||
}
|
||||
// 1. the copy
|
||||
c, err := m.EnsureCopy(ctx)
|
||||
if err != nil {
|
||||
lg.Warn("pvetrust: the root step gave no CA copy — the trust stays as it is", "err", err)
|
||||
set(modeOf(cur), ProblemCopyUnavailable, truncate(err.Error(), 240), nil)
|
||||
return false
|
||||
}
|
||||
ca := proxmox.TLSConfig{CAFile: c.Path}
|
||||
onCopy := cur.Fingerprint == "" && cur.CAFile == c.Path
|
||||
if cur.Fingerprint == "" && cur.CAFile != "" && cur.CAFile != c.Path {
|
||||
// an operator's own CA file: never replaced
|
||||
lg.Info("pvetrust: the config names its own CA file — left as it is", "ca_file", cur.CAFile)
|
||||
set("ca_file_own", "", "", &c)
|
||||
return false
|
||||
}
|
||||
if onCopy && c.Outcome != "written" && why != proxmox.TrustCAMismatch && why != proxmox.TrustCAFileUnreadable && why != proxmox.TrustCertExpired {
|
||||
// the box already trusts by the copy and the copy did not change: nothing to prove again
|
||||
problem, detail := "", ""
|
||||
if c.DaysLeft < EndsSoonDays {
|
||||
problem, detail = ProblemCopyEndsSoon, fmt.Sprintf("the CA copy ends in %d day(s), on %s", c.DaysLeft, c.NotAfter)
|
||||
lg.Error("pvetrust: "+detail, "days_left", c.DaysLeft)
|
||||
}
|
||||
set("ca", problem, detail, &c)
|
||||
return false
|
||||
}
|
||||
// 2. the proof: a real call with the copy ALONE — the fingerprint is still the configured trust
|
||||
pctx, cancel := context.WithTimeout(ctx, 30*time.Second)
|
||||
perr := m.Probe(pctx, ca)
|
||||
cancel()
|
||||
if perr != nil {
|
||||
cause := proxmox.TrustProblem(perr)
|
||||
detail := "a call to Proxmox with the CA copy alone failed"
|
||||
if cause != "" {
|
||||
detail += ": " + causeText(cause)
|
||||
} else {
|
||||
detail += " (not a trust failure: " + truncate(perr.Error(), 160) + ")"
|
||||
}
|
||||
if onCopy {
|
||||
lg.Error("pvetrust: the box trusts Proxmox by the CA copy and the copy does not verify the served certificate", "err", perr)
|
||||
set("ca", firstNonEmpty(cause, ProblemCopyNoProof), detail, &c)
|
||||
return false
|
||||
}
|
||||
if cause == "" {
|
||||
// Proxmox did not answer at all (it restarts, the box just booted): nothing is known about the copy, so no
|
||||
// cause is named; the next run asks again.
|
||||
lg.Warn("pvetrust: the proof could not be made now (not a trust failure) — the trust stays as it is; asked again later", "err", perr)
|
||||
set(modeOf(cur), "", "", &c)
|
||||
return false
|
||||
}
|
||||
lg.Error("pvetrust: the proof with the CA copy FAILED — the fingerprint STAYS, nothing was written", "err", perr)
|
||||
set(modeOf(cur), ProblemCopyNoProof, detail+" — the fingerprint stays", &c)
|
||||
return false
|
||||
}
|
||||
if onCopy {
|
||||
// the copy was refreshed (or a trust failure asked): the running client reloads it
|
||||
if m.Apply != nil {
|
||||
if aerr := m.Apply(ca); aerr != nil {
|
||||
set("ca", proxmox.TrustCAFileUnreadable, truncate(aerr.Error(), 200), &c)
|
||||
return false
|
||||
}
|
||||
}
|
||||
lg.Warn("pvetrust: the CA copy was refreshed and proven — the running client uses it now", "outcome", c.Outcome)
|
||||
set("ca", "", "", &c)
|
||||
return true
|
||||
}
|
||||
// 3. the config: ONE write sets ca_file and clears the fingerprint — only now, after the proof
|
||||
if m.ConfigPath == "" {
|
||||
lg.Warn("pvetrust: the proof worked, but the config did not come from a file — nothing can be saved, so the trust stays as it is")
|
||||
set(modeOf(cur), ProblemConfigNotSaved, "the config is not a file; the move was not made", &c)
|
||||
return false
|
||||
}
|
||||
if werr := m.writeConfig(c.Path); werr != nil {
|
||||
lg.Error("pvetrust: the proof worked but the config could NOT be written — the fingerprint stays", "err", werr)
|
||||
set(modeOf(cur), ProblemConfigNotSaved, truncate(werr.Error(), 200), &c)
|
||||
return false
|
||||
}
|
||||
// 4. the running client
|
||||
if m.Apply != nil {
|
||||
if aerr := m.Apply(ca); aerr != nil {
|
||||
lg.Error("pvetrust: the config is written but the running client could not take the CA copy — it keeps the old trust until the next start", "err", aerr)
|
||||
}
|
||||
}
|
||||
m.mu.Lock()
|
||||
m.cur = ca
|
||||
m.mu.Unlock()
|
||||
lg.Warn("pvetrust: MOVED — the agent trusts Proxmox by the box's own root CA now; the pinned fingerprint was cleared AFTER a working call with the copy",
|
||||
"ca_file", c.Path, "ca_not_after", c.NotAfter)
|
||||
set("ca", "", "", &c)
|
||||
m.mu.Lock()
|
||||
m.status.MovedAt = m.now().UTC().Format(time.RFC3339)
|
||||
m.mu.Unlock()
|
||||
return true
|
||||
}
|
||||
|
||||
func firstNonEmpty(a, b string) string {
|
||||
if a != "" {
|
||||
return a
|
||||
}
|
||||
return b
|
||||
}
|
||||
|
||||
func truncate(s string, n int) string {
|
||||
if len(s) <= n {
|
||||
return s
|
||||
}
|
||||
return s[:n]
|
||||
}
|
||||
|
||||
// writeConfig sets proxmox.tls.ca_file and clears proxmox.tls.fingerprint in ONE write of agent.json. Every other key is
|
||||
// kept byte for byte. A recovery copy is parked in the state folder first; the result is read back and checked.
|
||||
func (m *Manager) writeConfig(caPath string) error {
|
||||
raw, err := os.ReadFile(m.ConfigPath)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var doc map[string]json.RawMessage
|
||||
if err := json.Unmarshal(raw, &doc); err != nil {
|
||||
return fmt.Errorf("parse %s: %w", m.ConfigPath, err)
|
||||
}
|
||||
px := map[string]json.RawMessage{}
|
||||
if cur, ok := doc["proxmox"]; ok {
|
||||
if err := json.Unmarshal(cur, &px); err != nil {
|
||||
return fmt.Errorf("parse the proxmox section: %w", err)
|
||||
}
|
||||
}
|
||||
tlsSec := map[string]json.RawMessage{}
|
||||
if cur, ok := px["tls"]; ok {
|
||||
if err := json.Unmarshal(cur, &tlsSec); err != nil {
|
||||
return fmt.Errorf("parse the proxmox.tls section: %w", err)
|
||||
}
|
||||
}
|
||||
caJSON, _ := json.Marshal(caPath)
|
||||
tlsSec["ca_file"] = caJSON
|
||||
tlsSec["fingerprint"] = json.RawMessage(`""`)
|
||||
b, _ := json.Marshal(tlsSec)
|
||||
px["tls"] = b
|
||||
b, _ = json.Marshal(px)
|
||||
doc["proxmox"] = b
|
||||
out, err := json.MarshalIndent(doc, "", " ")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
out = append(out, '\n')
|
||||
st, err := os.Stat(m.ConfigPath)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if m.StateDir != "" {
|
||||
if err := os.MkdirAll(m.StateDir, 0o700); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(m.StateDir, "agent.json.pre-pve-trust"), raw, 0o600); err != nil {
|
||||
return fmt.Errorf("parking the recovery copy: %w", err)
|
||||
}
|
||||
}
|
||||
// In place (O_TRUNC), not tmp+rename: the folder is root-owned and the file is the agent's (the same reason as the
|
||||
// backup-target move and the PBS seed). One small write, then fsync.
|
||||
f, err := os.OpenFile(m.ConfigPath, os.O_WRONLY|os.O_TRUNC, st.Mode().Perm())
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := f.Write(out); err != nil {
|
||||
f.Close()
|
||||
_ = os.WriteFile(m.ConfigPath, raw, st.Mode().Perm()) // put the old bytes back: the fingerprint must not be lost
|
||||
return err
|
||||
}
|
||||
if err := f.Sync(); err != nil {
|
||||
f.Close()
|
||||
return err
|
||||
}
|
||||
if err := f.Close(); err != nil {
|
||||
return err
|
||||
}
|
||||
// read back: the file must say exactly what was meant
|
||||
var check struct {
|
||||
Proxmox struct {
|
||||
TLS struct {
|
||||
CAFile string `json:"ca_file"`
|
||||
Fingerprint string `json:"fingerprint"`
|
||||
} `json:"tls"`
|
||||
} `json:"proxmox"`
|
||||
}
|
||||
back, err := os.ReadFile(m.ConfigPath)
|
||||
if err != nil || json.Unmarshal(back, &check) != nil || check.Proxmox.TLS.CAFile != caPath || check.Proxmox.TLS.Fingerprint != "" {
|
||||
_ = os.WriteFile(m.ConfigPath, raw, st.Mode().Perm())
|
||||
return fmt.Errorf("the config did not read back as written (%v) — the old config was put back", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,489 @@
|
||||
package pvetrust
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/ecdsa"
|
||||
"crypto/elliptic"
|
||||
"crypto/rand"
|
||||
"crypto/sha256"
|
||||
"crypto/tls"
|
||||
"crypto/x509"
|
||||
"crypto/x509/pkix"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"encoding/pem"
|
||||
"errors"
|
||||
"fmt"
|
||||
"math/big"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"gitea.dooplex.hu/admin/felhom-agent/internal/proxmox"
|
||||
)
|
||||
|
||||
// ---- a small Proxmox: a root CA that signs node certificates, and a server that can change its certificate ----
|
||||
|
||||
type pki struct {
|
||||
t *testing.T
|
||||
caKey *ecdsa.PrivateKey
|
||||
caCert *x509.Certificate
|
||||
caPEM []byte
|
||||
}
|
||||
|
||||
func newPKI(t *testing.T, cn string) *pki {
|
||||
t.Helper()
|
||||
k, _ := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
|
||||
return (&pki{t: t, caKey: k}).renewCA(cn)
|
||||
}
|
||||
|
||||
// renewCA makes a new CA certificate with the SAME key and name (what Proxmox does when its root CA is renewed).
|
||||
func (p *pki) renewCA(cn string) *pki {
|
||||
tpl := &x509.Certificate{SerialNumber: big.NewInt(time.Now().UnixNano()), Subject: pkix.Name{CommonName: cn},
|
||||
NotBefore: time.Now().Add(-time.Hour), NotAfter: time.Now().Add(24 * time.Hour), IsCA: true, BasicConstraintsValid: true,
|
||||
KeyUsage: x509.KeyUsageCertSign | x509.KeyUsageCRLSign}
|
||||
der, err := x509.CreateCertificate(rand.Reader, tpl, tpl, &p.caKey.PublicKey, p.caKey)
|
||||
if err != nil {
|
||||
p.t.Fatal(err)
|
||||
}
|
||||
p.caCert, _ = x509.ParseCertificate(der)
|
||||
p.caPEM = pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: der})
|
||||
return p
|
||||
}
|
||||
|
||||
func (p *pki) leaf() tls.Certificate {
|
||||
k, _ := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
|
||||
tpl := &x509.Certificate{SerialNumber: big.NewInt(time.Now().UnixNano()), Subject: pkix.Name{CommonName: "node"},
|
||||
NotBefore: time.Now().Add(-time.Hour), NotAfter: time.Now().Add(12 * time.Hour), IPAddresses: []net.IP{net.ParseIP("127.0.0.1")},
|
||||
KeyUsage: x509.KeyUsageDigitalSignature, ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth}}
|
||||
der, err := x509.CreateCertificate(rand.Reader, tpl, p.caCert, &k.PublicKey, p.caKey)
|
||||
if err != nil {
|
||||
p.t.Fatal(err)
|
||||
}
|
||||
return tls.Certificate{Certificate: [][]byte{der}, PrivateKey: k}
|
||||
}
|
||||
|
||||
func fp(c tls.Certificate) string {
|
||||
s := sha256.Sum256(c.Certificate[0])
|
||||
return hex.EncodeToString(s[:])
|
||||
}
|
||||
|
||||
type pve struct {
|
||||
srv *httptest.Server
|
||||
mu sync.Mutex
|
||||
cert tls.Certificate
|
||||
}
|
||||
|
||||
func newPVE(t *testing.T, c tls.Certificate) *pve {
|
||||
p := &pve{cert: c}
|
||||
p.srv = httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
||||
fmt.Fprint(w, `{"data":{"version":"9.2.2","release":"9.2","repoid":"x"}}`)
|
||||
}))
|
||||
// GetConfigForClient, not GetCertificate: httptest adds its own certificate, and for a call by IP address (no
|
||||
// server name) that one would win over GetCertificate.
|
||||
p.srv.TLS = &tls.Config{GetConfigForClient: func(*tls.ClientHelloInfo) (*tls.Config, error) {
|
||||
p.mu.Lock()
|
||||
defer p.mu.Unlock()
|
||||
return &tls.Config{Certificates: []tls.Certificate{p.cert}}, nil
|
||||
}}
|
||||
p.srv.StartTLS()
|
||||
t.Cleanup(p.srv.Close)
|
||||
return p
|
||||
}
|
||||
|
||||
// newCert is `pveupdate` making a new node certificate: served from the next connection on.
|
||||
func (p *pve) newCert(c tls.Certificate) {
|
||||
p.mu.Lock()
|
||||
p.cert = c
|
||||
p.mu.Unlock()
|
||||
p.srv.CloseClientConnections()
|
||||
}
|
||||
|
||||
// ---- the agent's side ----
|
||||
|
||||
type box struct {
|
||||
t *testing.T
|
||||
m *Manager
|
||||
px *proxmox.Client
|
||||
cfgPath string
|
||||
copyPath string
|
||||
source []byte // what the root step would copy (nil = the root step refuses)
|
||||
events []string
|
||||
applyErr error
|
||||
}
|
||||
|
||||
func cfgJSON(fingerprint, caFile string) string {
|
||||
return fmt.Sprintf(`{
|
||||
"hub": {"url": "https://hub.example", "host_id": "box-1", "api_key": "KEEP-ME"},
|
||||
"proxmox": {"endpoint": "X", "node": "n", "token": "u@pve!t=s", "tls": {"fingerprint": %q, "ca_file": %q, "insecure_skip_verify": false}, "custom": [1, 2, 3]},
|
||||
"unknown_key": {"stays": true}
|
||||
}`, fingerprint, caFile)
|
||||
}
|
||||
|
||||
func newBox(t *testing.T, srv *pve, start proxmox.TLSConfig, source []byte) *box {
|
||||
t.Helper()
|
||||
dir := t.TempDir()
|
||||
b := &box{t: t, cfgPath: filepath.Join(dir, "agent.json"), copyPath: filepath.Join(dir, "pve-root-ca.pem"), source: source}
|
||||
if start.CAFile == "COPY" {
|
||||
start.CAFile = b.copyPath
|
||||
if err := os.WriteFile(b.copyPath, source, 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if err := os.WriteFile(b.cfgPath, []byte(cfgJSON(start.Fingerprint, start.CAFile)), 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
pcfg := proxmox.Config{Endpoint: srv.srv.URL, Node: "n", Token: "u@pve!t=s", TLS: start, HTTPTimeout: 5 * time.Second}
|
||||
px, err := proxmox.NewClient(pcfg)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
b.px = px
|
||||
b.m = &Manager{ConfigPath: b.cfgPath, StateDir: filepath.Join(dir, "state"),
|
||||
EnsureCopy: func(context.Context) (Copy, error) {
|
||||
b.events = append(b.events, "copy")
|
||||
if b.source == nil {
|
||||
return Copy{}, errors.New(`pve-ca-copy refused: {"code": "T3", "reason": "not a CA certificate"}`)
|
||||
}
|
||||
outcome := "same"
|
||||
if cur, _ := os.ReadFile(b.copyPath); string(cur) != string(b.source) {
|
||||
outcome = "written"
|
||||
if err := os.WriteFile(b.copyPath, b.source, 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
return Copy{Path: b.copyPath, Outcome: outcome, NotAfter: "2036-08-17T15:44:42Z", DaysLeft: 3560}, nil
|
||||
},
|
||||
Probe: func(ctx context.Context, tl proxmox.TLSConfig) error {
|
||||
b.events = append(b.events, fmt.Sprintf("probe(fp=%q,ca=%v) while the file says fp=%q", tl.Fingerprint, tl.CAFile != "", b.fileTLS().Fingerprint))
|
||||
c := pcfg
|
||||
c.TLS = tl
|
||||
p, err := proxmox.NewClient(c)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
_, err = p.Version(ctx)
|
||||
return err
|
||||
},
|
||||
Apply: func(tl proxmox.TLSConfig) error {
|
||||
b.events = append(b.events, fmt.Sprintf("apply(fp=%q,ca=%v)", tl.Fingerprint, tl.CAFile != ""))
|
||||
if b.applyErr != nil {
|
||||
return b.applyErr
|
||||
}
|
||||
return b.px.SetTLS(tl)
|
||||
},
|
||||
}
|
||||
b.m.Start(start)
|
||||
return b
|
||||
}
|
||||
|
||||
type fileTLS struct {
|
||||
Fingerprint string `json:"fingerprint"`
|
||||
CAFile string `json:"ca_file"`
|
||||
}
|
||||
|
||||
func (b *box) fileTLS() fileTLS {
|
||||
var d struct {
|
||||
Proxmox struct {
|
||||
TLS fileTLS `json:"tls"`
|
||||
} `json:"proxmox"`
|
||||
}
|
||||
raw, _ := os.ReadFile(b.cfgPath)
|
||||
if err := json.Unmarshal(raw, &d); err != nil {
|
||||
b.t.Fatalf("agent.json is not JSON any more: %v\n%s", err, raw)
|
||||
}
|
||||
return d.Proxmox.TLS
|
||||
}
|
||||
|
||||
func (b *box) call() error {
|
||||
_, err := b.px.Version(context.Background())
|
||||
return err
|
||||
}
|
||||
|
||||
func (b *box) has(prefix string) bool {
|
||||
for _, e := range b.events {
|
||||
if strings.HasPrefix(e, prefix) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// Scenario: a NEW box — the installer wrote the copy and ca_file, no fingerprint. Nothing is proven again, nothing written.
|
||||
func TestNewBox_CAOnly_NothingToMove(t *testing.T) {
|
||||
p := newPKI(t, "PVE CA")
|
||||
srv := newPVE(t, p.leaf())
|
||||
b := newBox(t, srv, proxmox.TLSConfig{CAFile: "COPY"}, p.caPEM)
|
||||
before, _ := os.ReadFile(b.cfgPath)
|
||||
if b.m.Ensure(context.Background(), "start") {
|
||||
t.Fatal("a box that already trusts by the copy changed its trust")
|
||||
}
|
||||
after, _ := os.ReadFile(b.cfgPath)
|
||||
if string(before) != string(after) || b.has("probe") || b.has("apply") {
|
||||
t.Fatalf("the config was touched or a proof ran: %v", b.events)
|
||||
}
|
||||
if st := b.m.Status(); st.Mode != "ca" || st.Problem != "" || st.CADaysLeft == nil || *st.CADaysLeft != 3560 {
|
||||
t.Fatalf("status = %+v", st)
|
||||
}
|
||||
if err := b.call(); err != nil {
|
||||
t.Fatalf("the call fails: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// Scenario: an OLD box moves. The proof runs with the CA copy ALONE while the file still holds the fingerprint; only
|
||||
// then ONE write sets ca_file and clears the fingerprint; every other key stays; a recovery copy is parked.
|
||||
// COMPANION RED-PROOFS (observed, felhom.eu audits/night-build-2026-11-14/A/redproof-agent.txt): write the config
|
||||
// BEFORE the probe → the order check fails; skip the probe → TestProofFails_TheFingerprintStays fails.
|
||||
func TestOldBox_MovesAndTheFingerprintIsClearedOnlyAfterTheProof(t *testing.T) {
|
||||
p := newPKI(t, "PVE CA")
|
||||
leaf := p.leaf()
|
||||
srv := newPVE(t, leaf)
|
||||
b := newBox(t, srv, proxmox.TLSConfig{Fingerprint: fp(leaf)}, p.caPEM)
|
||||
old, _ := os.ReadFile(b.cfgPath)
|
||||
if !b.m.Ensure(context.Background(), "start") {
|
||||
t.Fatalf("the box did not move: %+v %v", b.m.Status(), b.events)
|
||||
}
|
||||
want := []string{"copy", fmt.Sprintf("probe(fp=\"\",ca=true) while the file says fp=%q", fp(leaf)), "apply(fp=\"\",ca=true)"}
|
||||
if strings.Join(b.events, " | ") != strings.Join(want, " | ") {
|
||||
t.Fatalf("order = %v\nwant %v", b.events, want)
|
||||
}
|
||||
if got := b.fileTLS(); got.Fingerprint != "" || got.CAFile != b.copyPath {
|
||||
t.Fatalf("the config's tls = %+v", got)
|
||||
}
|
||||
raw, _ := os.ReadFile(b.cfgPath)
|
||||
for _, keep := range []string{`"KEEP-ME"`, `"unknown_key"`, `"stays"`, `"custom"`, `"u@pve!t=s"`, `"insecure_skip_verify"`} {
|
||||
if !strings.Contains(string(raw), keep) {
|
||||
t.Fatalf("the write lost %s:\n%s", keep, raw)
|
||||
}
|
||||
}
|
||||
if parked, _ := os.ReadFile(filepath.Join(b.m.StateDir, "agent.json.pre-pve-trust")); string(parked) != string(old) {
|
||||
t.Fatal("no recovery copy of the old config was parked")
|
||||
}
|
||||
if st := b.m.Status(); st.Mode != "ca" || st.MovedAt == "" || st.Problem != "" {
|
||||
t.Fatalf("status = %+v", st)
|
||||
}
|
||||
// and now Proxmox makes a new certificate: the agent goes on, with no restart and no new client
|
||||
srv.newCert(p.leaf())
|
||||
if err := b.call(); err != nil {
|
||||
t.Fatalf("after a new Proxmox certificate the moved box fails: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// Scenario: the proof fails (the copy is another box's CA). The fingerprint stays — in the file, byte for byte, and in
|
||||
// the running client — and the cause is named.
|
||||
func TestProofFails_TheFingerprintStays(t *testing.T) {
|
||||
p := newPKI(t, "PVE CA")
|
||||
leaf := p.leaf()
|
||||
srv := newPVE(t, leaf)
|
||||
other := newPKI(t, "another box")
|
||||
b := newBox(t, srv, proxmox.TLSConfig{Fingerprint: fp(leaf)}, other.caPEM)
|
||||
before, _ := os.ReadFile(b.cfgPath)
|
||||
if b.m.Ensure(context.Background(), "start") {
|
||||
t.Fatal("the trust changed although the proof failed")
|
||||
}
|
||||
after, _ := os.ReadFile(b.cfgPath)
|
||||
if string(before) != string(after) {
|
||||
t.Fatal("the config was written although the proof failed")
|
||||
}
|
||||
if b.has("apply") {
|
||||
t.Fatalf("the running client was changed: %v", b.events)
|
||||
}
|
||||
if st := b.m.Status(); st.Mode != "fingerprint" || st.Problem != ProblemCopyNoProof || !strings.Contains(st.Detail, "did not sign") {
|
||||
t.Fatalf("status = %+v", st)
|
||||
}
|
||||
if err := b.call(); err != nil {
|
||||
t.Fatalf("the fingerprint no longer works: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// Scenario: Proxmox makes a new certificate on a box that still PINS the old one. The failed call names the cause, the
|
||||
// box moves to the CA copy right then, and the next call works — no restart.
|
||||
func TestPinnedBox_NewCertificate_HealsWithNoRestart(t *testing.T) {
|
||||
p := newPKI(t, "PVE CA")
|
||||
leaf := p.leaf()
|
||||
srv := newPVE(t, leaf)
|
||||
b := newBox(t, srv, proxmox.TLSConfig{Fingerprint: fp(leaf)}, p.caPEM)
|
||||
srv.newCert(p.leaf())
|
||||
err := b.call()
|
||||
if proxmox.TrustProblem(err) != proxmox.TrustPinnedCertChanged {
|
||||
t.Fatalf("fixture: want a pin mismatch, got %v", err)
|
||||
}
|
||||
b.m.OnCallError(context.Background(), err)
|
||||
if err := b.call(); err != nil {
|
||||
t.Fatalf("the box is still blind after the move: %v (%v)", err, b.events)
|
||||
}
|
||||
if got := b.fileTLS(); got.Fingerprint != "" || got.CAFile != b.copyPath {
|
||||
t.Fatalf("the config's tls = %+v", got)
|
||||
}
|
||||
// a failure that is NOT one of trust does nothing
|
||||
n := len(b.events)
|
||||
b.m.OnCallError(context.Background(), errors.New("proxmox: GET /version: dial tcp 127.0.0.1:8006: connect: connection refused"))
|
||||
if len(b.events) != n {
|
||||
t.Fatalf("a refused connection started a trust run: %v", b.events[n:])
|
||||
}
|
||||
}
|
||||
|
||||
// Scenario: the root CA itself is renewed with the same key. The copy is refreshed, proven, and the running client
|
||||
// reloads it; the config is not written again.
|
||||
func TestCARenewedWithTheSameKey_TheCopyIsRefreshedAndReloaded(t *testing.T) {
|
||||
p := newPKI(t, "PVE CA")
|
||||
srv := newPVE(t, p.leaf())
|
||||
b := newBox(t, srv, proxmox.TLSConfig{CAFile: "COPY"}, p.caPEM)
|
||||
oldCA := string(p.caPEM)
|
||||
p.renewCA("PVE CA")
|
||||
srv.newCert(p.leaf())
|
||||
b.source = p.caPEM
|
||||
before, _ := os.ReadFile(b.cfgPath)
|
||||
if !b.m.Ensure(context.Background(), "daily") {
|
||||
t.Fatalf("a refreshed copy was not taken: %+v %v", b.m.Status(), b.events)
|
||||
}
|
||||
if cur, _ := os.ReadFile(b.copyPath); string(cur) == oldCA || string(cur) != string(p.caPEM) {
|
||||
t.Fatal("the copy was not refreshed")
|
||||
}
|
||||
if !b.has("probe") || !b.has("apply") {
|
||||
t.Fatalf("a refreshed copy must be proven and reloaded: %v", b.events)
|
||||
}
|
||||
if after, _ := os.ReadFile(b.cfgPath); string(after) != string(before) {
|
||||
t.Fatal("the config was written again")
|
||||
}
|
||||
if err := b.call(); err != nil {
|
||||
t.Fatalf("the call fails after the CA's renewal: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// The root step refuses (the source is not a CA certificate): nothing changes, the cause is named.
|
||||
func TestRootStepRefuses_NothingChanges(t *testing.T) {
|
||||
p := newPKI(t, "PVE CA")
|
||||
leaf := p.leaf()
|
||||
srv := newPVE(t, leaf)
|
||||
b := newBox(t, srv, proxmox.TLSConfig{Fingerprint: fp(leaf)}, nil)
|
||||
before, _ := os.ReadFile(b.cfgPath)
|
||||
if b.m.Ensure(context.Background(), "start") || b.has("probe") || b.has("apply") {
|
||||
t.Fatalf("something ran without a copy: %v", b.events)
|
||||
}
|
||||
if after, _ := os.ReadFile(b.cfgPath); string(after) != string(before) {
|
||||
t.Fatal("the config was written")
|
||||
}
|
||||
if st := b.m.Status(); st.Problem != ProblemCopyUnavailable || !strings.Contains(st.Detail, "T3") {
|
||||
t.Fatalf("status = %+v", st)
|
||||
}
|
||||
}
|
||||
|
||||
// The proof works but the config cannot be written: the fingerprint stays, in the file and in the running client.
|
||||
func TestConfigCannotBeWritten_TheFingerprintStays(t *testing.T) {
|
||||
if os.Geteuid() == 0 {
|
||||
t.Skip("root writes a read-only file")
|
||||
}
|
||||
p := newPKI(t, "PVE CA")
|
||||
leaf := p.leaf()
|
||||
srv := newPVE(t, leaf)
|
||||
b := newBox(t, srv, proxmox.TLSConfig{Fingerprint: fp(leaf)}, p.caPEM)
|
||||
if err := os.Chmod(b.cfgPath, 0o400); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if b.m.Ensure(context.Background(), "start") || b.has("apply") {
|
||||
t.Fatalf("the trust changed without a saved config: %v", b.events)
|
||||
}
|
||||
if got := b.fileTLS(); got.Fingerprint != fp(leaf) {
|
||||
t.Fatalf("the fingerprint is gone from the file: %+v", got)
|
||||
}
|
||||
if st := b.m.Status(); st.Mode != "fingerprint" || st.Problem != ProblemConfigNotSaved {
|
||||
t.Fatalf("status = %+v", st)
|
||||
}
|
||||
}
|
||||
|
||||
// What is never moved: an operator's own CA file, a box with checks switched off, a config that is not a file.
|
||||
func TestNeverMoved(t *testing.T) {
|
||||
p := newPKI(t, "PVE CA")
|
||||
leaf := p.leaf()
|
||||
srv := newPVE(t, leaf)
|
||||
own := filepath.Join(t.TempDir(), "own-ca.pem")
|
||||
os.WriteFile(own, p.caPEM, 0o644)
|
||||
for name, start := range map[string]proxmox.TLSConfig{"own CA file": {CAFile: own}, "insecure": {InsecureSkipVerify: true}} {
|
||||
b := newBox(t, srv, start, p.caPEM)
|
||||
before, _ := os.ReadFile(b.cfgPath)
|
||||
if b.m.Ensure(context.Background(), "start") || b.has("apply") {
|
||||
t.Fatalf("%s: moved (%v)", name, b.events)
|
||||
}
|
||||
if after, _ := os.ReadFile(b.cfgPath); string(after) != string(before) {
|
||||
t.Fatalf("%s: the config was written", name)
|
||||
}
|
||||
}
|
||||
b := newBox(t, srv, proxmox.TLSConfig{Fingerprint: fp(leaf)}, p.caPEM)
|
||||
b.m.ConfigPath = ""
|
||||
if b.m.Ensure(context.Background(), "start") || b.has("apply") || b.m.Status().Problem != ProblemConfigNotSaved {
|
||||
t.Fatalf("a config that is not a file was 'moved': %v %+v", b.events, b.m.Status())
|
||||
}
|
||||
}
|
||||
|
||||
// The copy ends soon: the cause is named (the box still works).
|
||||
func TestCopyEndsSoon(t *testing.T) {
|
||||
p := newPKI(t, "PVE CA")
|
||||
srv := newPVE(t, p.leaf())
|
||||
b := newBox(t, srv, proxmox.TLSConfig{CAFile: "COPY"}, p.caPEM)
|
||||
inner := b.m.EnsureCopy
|
||||
b.m.EnsureCopy = func(ctx context.Context) (Copy, error) {
|
||||
c, err := inner(ctx)
|
||||
c.DaysLeft, c.NotAfter = 12, "2026-11-26T00:00:00Z"
|
||||
return c, err
|
||||
}
|
||||
b.m.Ensure(context.Background(), "daily")
|
||||
if st := b.m.Status(); st.Problem != ProblemCopyEndsSoon || !strings.Contains(st.Detail, "12 day") {
|
||||
t.Fatalf("status = %+v", st)
|
||||
}
|
||||
}
|
||||
|
||||
// A trust failure asks at most once per MinGap.
|
||||
func TestOnCallError_Throttled(t *testing.T) {
|
||||
p := newPKI(t, "PVE CA")
|
||||
leaf := p.leaf()
|
||||
srv := newPVE(t, leaf)
|
||||
other := newPKI(t, "another box")
|
||||
b := newBox(t, srv, proxmox.TLSConfig{Fingerprint: fp(leaf)}, other.caPEM)
|
||||
srv.newCert(p.leaf())
|
||||
err := b.call()
|
||||
now := time.Date(2026, 11, 15, 3, 0, 0, 0, time.UTC)
|
||||
b.m.Now = func() time.Time { return now }
|
||||
b.m.OnCallError(context.Background(), err)
|
||||
n := len(b.events)
|
||||
b.m.OnCallError(context.Background(), err)
|
||||
if len(b.events) != n {
|
||||
t.Fatalf("a second failure inside the gap ran again: %v", b.events[n:])
|
||||
}
|
||||
now = now.Add(11 * time.Minute)
|
||||
b.m.OnCallError(context.Background(), err)
|
||||
if len(b.events) == n {
|
||||
t.Fatal("no new run after the gap")
|
||||
}
|
||||
if st := b.m.Status(); st.Problem == "" {
|
||||
t.Fatalf("a blind box names no cause: %+v", st)
|
||||
}
|
||||
}
|
||||
|
||||
// Proxmox does not answer at all (it restarts): nothing is known about the copy — no cause is named, nothing is written,
|
||||
// and the box counts as not moved, so the timer asks again soon.
|
||||
func TestProofCannotBeMadeNow_NoCauseNamed(t *testing.T) {
|
||||
p := newPKI(t, "PVE CA")
|
||||
leaf := p.leaf()
|
||||
srv := newPVE(t, leaf)
|
||||
b := newBox(t, srv, proxmox.TLSConfig{Fingerprint: fp(leaf)}, p.caPEM)
|
||||
b.m.Probe = func(context.Context, proxmox.TLSConfig) error {
|
||||
return errors.New("proxmox: GET /version: dial tcp 127.0.0.1:8006: connect: connection refused")
|
||||
}
|
||||
before, _ := os.ReadFile(b.cfgPath)
|
||||
if b.m.Ensure(context.Background(), "start") {
|
||||
t.Fatal("moved without a proof")
|
||||
}
|
||||
if after, _ := os.ReadFile(b.cfgPath); string(after) != string(before) {
|
||||
t.Fatal("the config was written")
|
||||
}
|
||||
if st := b.m.Status(); st.Problem != "" || st.Mode != "fingerprint" || b.m.Moved() {
|
||||
t.Fatalf("status = %+v moved=%v", st, b.m.Moved())
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user