hub (unreleased): a give-up mark keeps its slot for a week after the box stops listing the guest (the commit's security review); docs for the CA copy's changeover
gates / gates (push) Successful in 2m10s
gates / gates (push) Successful in 2m10s
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
This commit is contained in:
1 parent
78c08dc909
commit
d2683b28d3
7 files changed
+82
-12
No files matched your search
@@ -1007,7 +1007,11 @@ a config that holds both is a pinned box — the agent treats it as „not moved
|
||||
certificates.** The root step writes the new root FIRST and keeps the copy's previous first certificate behind it:
|
||||
the copy is replaced before anything can be proven with the new root, and Proxmox serves its old certificate until
|
||||
pveproxy is reloaded — without the old root, an agent that restarted in that window had no root that verified the
|
||||
served certificate and no fingerprint. Never more than two; an ended one verifies nothing. (b) **One run at a time**
|
||||
served certificate and no fingerprint. **Only for the changeover:** at every later run the step makes one handshake to
|
||||
the local Proxmox (`127.0.0.1:8006`) with the new root alone and drops the previous root as soon as that verifies, or
|
||||
when the previous root is no longer a CA that has not ended. The copy is compared exactly, never by its start. By
|
||||
hand, to drop an old root at once (a root that was changed because it leaked): delete the copy; the next run writes
|
||||
the new root alone. Checked with the real openssl and a real TLS server in the wrapper's tests. (b) **One run at a time**
|
||||
(`Ensure` is called from the start, the timer, a failed collect and a bundle install), and the root step has a
|
||||
90-second deadline — it runs on the daemon's start path and inside the report loop. (c) **One lock for every writer
|
||||
of `agent.json`** in the process (`config.FileMu`: the trust move, the PBS seed, the backup-target move): each reads
|
||||
|
||||
@@ -731,7 +731,8 @@ the same report lists (not running), 16 entries at most.
|
||||
**The mails (after review, 2026-11-14):** each of these events tells one thing once and keeps a mark, so its mail
|
||||
cooldown is keyed per thing (`notify.perSubjectCooldownEvents`) — with the plain per-type key the second job, guest or
|
||||
skipped check inside the hour was suppressed and never told. The lost-mail scan tells only mails that failed before
|
||||
this hub process started (one that failed since still has its retry). One box holds at most 8 give-up marks.
|
||||
this hub process started (one that failed since still has its retry). One box holds at most 8 give-up marks, and the
|
||||
mark of a guest the box stops listing keeps its slot for a week.
|
||||
|
||||
**Not proven on a box or a deployed hub:** all five. The tests use fakes for the mail service and the real gate
|
||||
and verifier for the signed jobs (`audits/night-build-2026-11-14/B/`, 43 mutations seen red).
|
||||
|
||||
@@ -67,3 +67,42 @@
|
||||
|
||||
the real file: 0 failing line(s)
|
||||
3 mutations, 0 not a red-proof
|
||||
---- the CA copy, second pass (after the commit's security review: the previous root only for the changeover; exact compare) ----
|
||||
== MUTATION: the previous root is never dropped (it stays a trust anchor for ever)
|
||||
RED on the mutant: 3 failing line(s)
|
||||
FAIL: test_the_previous_root_stays_only_for_the_changeover (__main__.PVECACopy.test_the_previous_root_stays_only_for_the_changeover)
|
||||
AssertionError: b'---[86 chars]---\n-----BEGIN CERTIFICATE-----\nQUJDREVGR0hJ[49 chars]--\n' != b'---[86 chars]---\n' : after the changeover the copy is the new root alone
|
||||
FAILED (failures=1)
|
||||
== MUTATION: the previous root is dropped while Proxmox still serves the old certificate
|
||||
RED on the mutant: 3 failing line(s)
|
||||
FAIL: test_the_previous_root_stays_only_for_the_changeover (__main__.PVECACopy.test_the_previous_root_stays_only_for_the_changeover)
|
||||
AssertionError: Tuples differ: (0, 'written', [('/etc/felhom-agent/pve-root-c[108 chars]20)]) != (0, 'same', [])
|
||||
FAILED (failures=1)
|
||||
== MUTATION: a copy that only STARTS with the source is same (unchecked content stays)
|
||||
RED on the mutant: 4 failing line(s)
|
||||
KeyError: 'kept_previous'
|
||||
FAIL: test_a_copy_with_anything_unchecked_in_it_is_rewritten (__main__.PVECACopy.test_a_copy_with_anything_unchecked_in_it_is_rewritten)
|
||||
AssertionError: b'anything' unexpectedly found in b'-----BEGIN CERTIFICATE-----\nWllYV1ZVVFNSUVBPTk1MS0pJSEdGRURDQkE=\n-----END CERTIFICATE-----\nanything at all\n' : text after the source
|
||||
== MUTATION: an ended previous root is kept at a change
|
||||
RED on the mutant: 3 failing line(s)
|
||||
FAIL: test_the_previous_root_stays_only_for_the_changeover (__main__.PVECACopy.test_the_previous_root_stays_only_for_the_changeover)
|
||||
AssertionError: b'---[86 chars]---\n-----BEGIN CERTIFICATE-----\nQUJDREVGR0hJ[49 chars]--\n' != b'---[86 chars]---\n'
|
||||
FAILED (failures=1)
|
||||
== MUTATION: the handshake is verified with the system's roots too
|
||||
RED on the mutant: 3 failing line(s)
|
||||
FAIL: test_the_previous_root_stays_only_for_the_changeover (__main__.PVECACopy.test_the_previous_root_stays_only_for_the_changeover)
|
||||
AssertionError: '-no-CApath' not found in ['openssl', 's_client', '-connect', '127.0.0.1:8006', '-CAfile', '/etc/pve/pve-root-ca.pem', '-verify_return_error']
|
||||
FAILED (failures=1)
|
||||
== MUTATION: any answer of the handshake counts as verified
|
||||
RED on the mutant: 5 failing line(s)
|
||||
FAIL: test_the_previous_root_stays_only_for_the_changeover (__main__.PVECACopy.test_the_previous_root_stays_only_for_the_changeover)
|
||||
AssertionError: Tuples differ: (0, 'written', [('/etc/felhom-agent/pve-root-c[108 chars]20)]) != (0, 'same', [])
|
||||
FAIL: test_with_the_real_openssl_and_the_real_runner (__main__.PVECACopy.test_with_the_real_openssl_and_the_real_runner)
|
||||
== MUTATION: a leaf counts as a live CA
|
||||
RED on the mutant: 3 failing line(s)
|
||||
FAIL: test_with_the_real_openssl_and_the_real_runner (__main__.PVECACopy.test_with_the_real_openssl_and_the_real_runner)
|
||||
AssertionError: True is not false : a leaf is not a CA
|
||||
FAILED (failures=1)
|
||||
|
||||
the real file: 0 failing line(s)
|
||||
7 mutations, 0 not a red-proof
|
||||
@@ -110,3 +110,13 @@
|
||||
system_b3_test.go:63: last ask no_answer: {Text:7.0.14-28-pve · 0 rewritten · 18 h ago · last ask NO_ANSWER 1 h ago Class:warn Title:last ask: no_answer}, want amber, the failed ask AND the older record
|
||||
|
||||
1 mutations, 0 not a red-proof
|
||||
== MUTATION: a mark is dropped the moment the box stops listing the guest (a box mints 8 alarms per report)
|
||||
RED on the mutant, GREEN on the real file
|
||||
--- FAIL: TestGuestPowerGaveUp_ABoxCannotFillTheSettings (0.05s)
|
||||
review_fixes_test.go:69: 40 alarms for five reports of invented guests, want at most 8
|
||||
== MUTATION: a mark of a guest that is gone is never dropped
|
||||
RED on the mutant, GREEN on the real file
|
||||
--- FAIL: TestGuestPowerGaveUp_ABoxCannotFillTheSettings (0.05s)
|
||||
review_fixes_test.go:82: marks for guests the box has not listed for a week: 8 (rows 8), want 0
|
||||
|
||||
2 mutations, 0 not a red-proof
|
||||
+2
-1
@@ -8,7 +8,8 @@
|
||||
(`notify.perSubjectCooldownEvents`, details `subject`): the plain `customer:type` key suppressed the second token,
|
||||
certificate, job, guest or cause inside the hour, and its mark was already set, so it was never mailed. (2) What a
|
||||
BOX sends is bounded and checked: the trust event takes only a closed mode word and a real date; one box holds at
|
||||
most 8 give-up marks and a mark for a guest it no longer lists is dropped; web-certificate marks are per (household,
|
||||
most 8 give-up marks, and the mark of a guest it stops listing keeps its slot for a week (so new invented guests
|
||||
give 8 alarms a week, not 8 per report); web-certificate marks are per (household,
|
||||
name, stage), hold the end date told, are capped at 24 per household, and a name is told again only for an end
|
||||
date a day later. (3) `SetOSAlarmRaised` with no time DELETES the row (a cleared mark left an empty row for ever).
|
||||
(4) The first run of the end-date check marks what has already ended and does not mail it. (5) The off-site record:
|
||||
|
||||
@@ -3313,6 +3313,9 @@ const guestPowerMaxEntries = 16
|
||||
// guestPowerMaxMarks bounds how many give-up marks ONE box can hold in the hub's settings at a time.
|
||||
const guestPowerMaxMarks = 8
|
||||
|
||||
// guestPowerMarkKeep: how long the mark of a guest the box stopped listing keeps its slot.
|
||||
const guestPowerMarkKeep = 7 * 24 * time.Hour
|
||||
|
||||
func (h *Handler) noteSignedJobOutcome(hostID, jobID, outcome, reason, op string) {
|
||||
switch outcome {
|
||||
case "rejected", "failed", "malformed":
|
||||
@@ -3380,20 +3383,22 @@ func (h *Handler) checkGuestPowerGaveUp(hostID, custID string, gave []hostGuestP
|
||||
prefix := fmt.Sprintf("guest_power:%s:", hostID)
|
||||
key := func(vmid int) string { return prefix + strconv.Itoa(vmid) }
|
||||
// The marks this box has now. The report's own guest list is the BOX's word too, so „the report lists the guest”
|
||||
// bounds nothing by itself (review, 2026-11-14): the number of marks per box is capped, and a mark for a guest the
|
||||
// box no longer lists is dropped — a destroyed guest must not hold a mark, or a slot, for ever.
|
||||
// bounds nothing by itself (review, 2026-11-14): one box holds at most guestPowerMaxMarks marks, and a mark for a
|
||||
// guest the box no longer lists keeps its slot for guestPowerMarkKeep before it is dropped. So a box that invents
|
||||
// new guests at every report gets guestPowerMaxMarks alarms in that time, not that many per report — each alarm
|
||||
// is mailed (the mail cooldown is per guest). A destroyed guest's mark still goes, a week later.
|
||||
raised := map[int]bool{}
|
||||
for _, k := range h.store.OSAlarmKeys(prefix) {
|
||||
vmid, err := strconv.Atoi(strings.TrimPrefix(k, prefix))
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
if !known[vmid] {
|
||||
if !known[vmid] && time.Since(h.store.OSAlarmRaised(k)) > guestPowerMarkKeep {
|
||||
_ = h.store.SetOSAlarmRaised(k, time.Time{})
|
||||
h.logger.Printf("[INFO] host %s: guest %d is no longer reported — its give-up mark is dropped", hostID, vmid)
|
||||
h.logger.Printf("[INFO] host %s: guest %d has not been reported for %s — its give-up mark is dropped", hostID, vmid, guestPowerMarkKeep)
|
||||
continue
|
||||
}
|
||||
raised[vmid] = true
|
||||
raised[vmid] = true // also a guest the box stopped listing, for guestPowerMarkKeep: its mark keeps its slot
|
||||
}
|
||||
for _, g := range gave {
|
||||
if g.VMID <= 0 || !known[g.VMID] || running[g.VMID] || raised[g.VMID] {
|
||||
|
||||
@@ -63,13 +63,23 @@ func TestGuestPowerGaveUp_ABoxCannotFillTheSettings(t *testing.T) {
|
||||
t.Fatalf("round %d: %d give-up marks for one box, want at most %d", round, m, guestPowerMaxMarks)
|
||||
}
|
||||
}
|
||||
if n := len(r1383Events(t, st, eventGuestPowerGaveUp)); n > 5*guestPowerMaxMarks {
|
||||
t.Fatalf("%d events for five reports", n)
|
||||
// five reports with 80 invented guests: at most guestPowerMaxMarks ALARMS in all — not that many per report. A box
|
||||
// that drops a guest from its list does not get the slot back at once.
|
||||
if n := len(r1383Events(t, st, eventGuestPowerGaveUp)); n > guestPowerMaxMarks {
|
||||
t.Fatalf("%d alarms for five reports of invented guests, want at most %d", n, guestPowerMaxMarks)
|
||||
}
|
||||
// the box now reports only its real guest: the invented marks keep their slots for a week …
|
||||
h.checkGuestPowerGaveUp("h1", "c1", nil, map[int]bool{100: true}, map[int]bool{100: true})
|
||||
if marks() != guestPowerMaxMarks {
|
||||
t.Fatalf("marks right after the box stopped listing the guests: %d, want %d (kept for %s)", marks(), guestPowerMaxMarks, guestPowerMarkKeep)
|
||||
}
|
||||
// … and are gone after it, rows and all
|
||||
for _, k := range st.OSAlarmKeys("guest_power:h1:") {
|
||||
_ = st.SetOSAlarmRaised(k, time.Now().Add(-guestPowerMarkKeep-time.Hour))
|
||||
}
|
||||
// the box now reports only its real guest: every invented mark is gone, and so is its row
|
||||
h.checkGuestPowerGaveUp("h1", "c1", nil, map[int]bool{100: true}, map[int]bool{100: true})
|
||||
if marks() != 0 || rows() != 0 {
|
||||
t.Fatalf("marks for guests the box no longer lists: %d (rows %d), want 0", marks(), rows())
|
||||
t.Fatalf("marks for guests the box has not listed for a week: %d (rows %d), want 0", marks(), rows())
|
||||
}
|
||||
// a cleared mark leaves no row at all
|
||||
_ = st.SetOSAlarmRaised("probe:x", time.Now())
|
||||
|
||||
Reference in new issue
Block a user