Compare commits

..

4 Commits

Author SHA1 Message Date
admin c9013bb47d CHANGELOG/REPORT: v0.153.0 released and delivered
gates / gates (push) Successful in 1m33s
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
2026-10-07 19:06:04 +02:00
admin 2d1e5d0774 R-898: ring 0 stages exactly the told kernel (select listed, KernelSet(kver))
gates / gates (push) Successful in 1m6s
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
2026-10-07 18:45:28 +02:00
admin f09c53efc1 REPORT/CONTEXT: v0.152.0 the kernel lane
gates / gates (push) Successful in 1m9s
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
2026-10-07 18:20:11 +02:00
admin 92d647a6f6 CHANGELOG: v0.152.0 released (shas, step bundle); host_stale is 45 min (comment fix)
gates / gates (push) Successful in 1m0s
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
2026-10-07 15:42:26 +02:00
7 changed files with 100 additions and 15 deletions
+24 -2
View File
@@ -1,4 +1,26 @@
## Unreleased — part of v0.152.0: the kernel lane (R-836; `09` §3 decisions 164, 172; `11` §5.11) (2026-10-07) ## v0.153.0 — ring 0 stages exactly the told kernel (R-898; `09` §3 decision 176) (2026-10-07)
Released by `scripts/release-agent.sh`: binary sha256 `b204ebe6d65944f43b70dcfa4ae0dfb90da38c92e2ba6a38a1826e488eeb6630`, bundle
`6db216275eb0dd39188d93481a2045998a69e8cb7838ad908a58d65d9a6a9b57` (tag `v0.153.0` = `2d1e5d0`). Delivered (binary only — the
bundle files are unchanged from 0.152.0) to demo-hp, demo-felhom, Tester 1 on 2026-10-07 19:03.
**Delivery: the agent binary only** — no root file changed (the wrapper is unchanged; its tests gained two cases).
- `internal/osupdate/kernel.go`: ring 0's night kernel step stages EXACTLY the kernel the household was told about
(select `listed`, `KernelSet(kver)` = the series meta-package and the signed image at the kernel's own version) instead
of "whatever is pending tonight". Seen 2026-10-07: demo-felhom was told about 7.0.14-20 while its sources offered
7.0.14-22 by night — the old code staged `pending-kernel` and the wrapper refused it (R23), losing the night. A told
version that is no longer installable is refused by the wrapper before any change (R7) and the hub tells the household
again for the newer kernel (hub v0.143.1). Tests `TestKernel_Ring0ToldNightStagesThenReboots` (red-proved against the
old select), `TestKernelSet`; wrapper `test_ring0_listed_installs_the_told_kernel_not_the_newest`,
`test_ring0_told_kernel_gone_is_refused_before_any_change`.
## v0.152.0 — the kernel lane (R-836; `09` §3 decisions 164, 172; `11` §5.11) (2026-10-07)
Released by `scripts/release-agent.sh`: binary sha256 `95ff42208e36ba49b6e2b09a97e81a6fa11562ecc8042f1ed18d378b8b1f88b1`,
config bundle sha256 `f0c2cec374b711b3c131c955012b33fd0ad495337d049b7a743c3eef9e85c20b` (tag `v0.152.0` = `d03ab7f`).
Step bundle `0.152.0-step1` sha256 `0b71d32b054cf3b7ade0234ffcbb0df159901f542cde540adaee411db466f48e` (the 0.151.0 bundle
with only `felhom-os-apply` replaced; `scripts/build-step-bundle.py`), published as package version `0.152.0-step1`.
**Delivery: agent binary, then the STEP bundle `0.152.0-step1`, then the bundle `0.152.0`** — the bundle ADDS two paths **Delivery: agent binary, then the STEP bundle `0.152.0-step1`, then the bundle `0.152.0`** — the bundle ADDS two paths
(the GRUB generators), and an installed `felhom-os-apply` refuses a path its own table lacks (R16, R-880). (the GRUB generators), and an installed `felhom-os-apply` refuses a path its own table lacks (R16, R-880).
@@ -34,7 +56,7 @@ candidate 2 (`audits/kernel-spike-2026-10-07/`), with option C on the one-shot e
signed `os_kernel_step` staged (`KernelStepExecutor`: stage only, under the heavy-op gate). The hub hears `staged` signed `os_kernel_step` staged (`KernelStepExecutor`: stage only, under the heavy-op gate). The hub hears `staged`
BEFORE the reboot. At every start `KernelAfterBoot`: on the new kernel it JUDGES the boot — `KernelVerdict` = the BEFORE the reboot. At every start `KernelAfterBoot`: on the new kernel it JUDGES the boot — `KernelVerdict` = the
host health rule (`11` §8.2) AND the box reached the hub (the `judging` report itself) — for 20 minutes (measured: host health rule (`11` §8.2) AND the box reached the hub (the `judging` report itself) — for 20 minutes (measured:
everything healthy 68 s after the reboot on demo-felhom, 272 s on demo-hp; under the hub's 30-minute `host_stale`). everything healthy 68 s after the reboot on demo-felhom, 272 s on demo-hp; under the hub's 45-minute `host_stale` (`alerting.stale_threshold`)).
Healthy → `kernel-good`, outcome `applied`; not healthy → outcome `health_failed`, then ONE `kernel-revert`. Healthy → `kernel-good`, outcome `applied`; not healthy → outcome `health_failed`, then ONE `kernel-revert`.
Tests: `TestKernel*` (13); red-proofs in the same file. Tests: `TestKernel*` (13); red-proofs in the same file.
- `internal/hub`: `WireOSUpdate.Kernel` {kver, tonight, notified_at}. `internal/reconcile`: `os_kernel_step` is - `internal/hub`: `WireOSUpdate.Kernel` {kver, tonight, notified_at}. `internal/reconcile`: `os_kernel_step` is
+2
View File
@@ -1,5 +1,7 @@
# CONTEXT — felhom-agent working state # CONTEXT — felhom-agent working state
> **2026-10-07 (evening) — v0.152.0, the kernel lane (R-836, decision 172, `11` §5.11).** Wrapper layer `kernel` + two GRUB generators in the bundle (delivered with step bundle `0.152.0-step1` — the bundle adds paths, R-880); `osupdate/kernel.go`: night step on told nights only, after-boot judge (host rule + hub reached, 20 min), ONE self-revert, `os_kernel_step` stages only. Proven on Tester 1 (panic → fell_back, held guest → self_reverted, healthy → 7.0.14-22 default). Open: R-898, R-897; the ring-0 night run.
> **2026-10-04 night — v0.143.0 RELEASED + vouched (R-840, decision 96): the config bundle.** `felhom-os-apply` mode > **2026-10-04 night — v0.143.0 RELEASED + vouched (R-840, decision 96): the config bundle.** `felhom-os-apply` mode
> `bundle` (signed `agent_config_update`, verified by the wrapper itself; trust files never bundle paths) + > `bundle` (signed `agent_config_update`, verified by the wrapper itself; trust files never bundle paths) +
> `--install-bundle` (installer 1.31.0); `BUNDLE_FILES` is the one table; `scripts/build-config-bundle.py`; > `--install-bundle` (installer 1.31.0); `BUNDLE_FILES` is the one table; `scripts/build-config-bundle.py`;
+8 -8
View File
@@ -1,9 +1,9 @@
# REPORT — v0.151.0 released and delivered (2026-10-07 day) # REPORT — agent v0.153.0: ring 0 stages exactly the told kernel (2026-10-07, late evening)
On the operator's word (`09` §3 decisions 163, 165, 168, 169). sha `0464354f…`, bundle `bacd1d17…`, tag `v0.151.0` = `dd7cdc0`. - R-898 (closed): the night kernel step on ring 0 stages the kernel the household was told about (`select listed`,
Delivered binary first, then the bundle (its sudoers drops the `tee` grant 0.150.0 used), to demo-hp, demo-felhom and `osupdate.KernelSet(kver)`), never "whatever is pending tonight". A told version no longer installable → the wrapper
Tester 1 — probe 68/68 on each. **Vouch refused by the hub** (`golden_behind_fleet`): new installs keep 0.150.0 until the refuses before any change (R7); the hub re-tells the household for the newer kernel.
weekly golden. Carries: the Proxmox package lane + `/etc/pve` write gate (R-812 A — proven on demo-felhom: 65 packages, - Tests: `TestKernel_Ring0ToldNightStagesThenReboots` (red-proved against the old select), `TestKernelSet`; wrapper
70 s, healthy), the `controller-image` root verb (R-861 a — on demo-hp an `alpine` ref is refused; a managed swap not yet cases `test_ring0_listed_installs_the_told_kernel_not_the_newest`, `test_ring0_told_kernel_gone_is_refused_before_any_change`.
seen), anchored felhom-op lines (B2), the other-key archive ledger (R-366), the `-directive` flag removed (R-105). Evidence - Released `v0.153.0`, binary `b204ebe6…`; delivered (binary only) to demo-hp, demo-felhom, Tester 1 at 19:03.
`felhom.eu/documentation/audits/day-2026-10-07/`. Shared rule file: decision 162 line added. - Tonight (7→8): demo-felhom stages 7.0.14-20, demo-hp 7.0.14-22 — both households told. Read back 2026-10-08.
+2 -1
View File
@@ -881,7 +881,8 @@ func runDaemon(cfg config.Config, logger *slog.Logger, logRing *applog.Ring) int
// judge it (the host health rule + the hub reached) for KernelJudgeWait, then make it the default or revert ONCE. // judge it (the host health rule + the hub reached) for KernelJudgeWait, then make it the default or revert ONCE.
// The wait: measured 2026-10-07 (`audits/kernel-lane-2026-10-07/B/`) — every container healthy 68 s after the // The wait: measured 2026-10-07 (`audits/kernel-lane-2026-10-07/B/`) — every container healthy 68 s after the
// reboot on demo-felhom and 272 s on demo-hp (the hub reached at 63 s / 189 s); 20 minutes leaves room for a slow // reboot on demo-felhom and 272 s on demo-hp (the hub reached at 63 s / 189 s); 20 minutes leaves room for a slow
// network and stays under the hub's 30-minute host_stale. On a box without the kernel lane (an older wrapper, a BYO host) the check is refused and logged. // network and stays under the hub's 45-minute host_stale (its
// alerting.stale_threshold; host_down at 90). On a box without the kernel lane (an older wrapper, a BYO host) the check is refused and logged.
go osLeg.KernelAfterBoot(ctx, 0, osupdate.KernelJudge{Wait: osupdate.DefaultKernelJudgeWait}) go osLeg.KernelAfterBoot(ctx, 0, osupdate.KernelJudge{Wait: osupdate.DefaultKernelJudgeWait})
// Reconcile (slice 4) runs alongside the hub loop, sharing the per-guest queue // Reconcile (slice 4) runs alongside the hub loop, sharing the per-guest queue
+27
View File
@@ -1838,6 +1838,33 @@ class KernelLane(unittest.TestCase):
m.origins = {"proxmox-kernel-7.0": DEB} m.origins = {"proxmox-kernel-7.0": DEB}
self.refused(m, "R2") self.refused(m, "R2")
# R-898: ring 0 stages EXACTLY the told kernel (select listed, the set derived from it) — even when the sources
# offer a newer one by night; a told version that can no longer be installed is refused BEFORE any change (R7).
def test_ring0_listed_installs_the_told_kernel_not_the_newest(self):
f = kfake()
f.live["proxmox-kernel-7.0"] = {"7.0.14-20", "7.0.14-22", "7.0.2-6"}
f.live["proxmox-kernel-7.0.14-20-pve-signed"] = {"7.0.14-20"}
told = [{"name": "proxmox-kernel-7.0", "version": "7.0.14-20", "origin": "Proxmox Debian Repository"},
{"name": "proxmox-kernel-7.0.14-20-pve-signed", "version": "7.0.14-20", "origin": "Proxmox Debian Repository"}]
f.plan.update(select="listed", packages=told, expect_kver="7.0.14-20-pve")
rc, rep = run(f)
self.assertEqual(rc, 0, rep)
self.assertEqual(rep["authority"], "ring0")
self.assertEqual(f.env, {"felhom_next": "7.0.14-20-pve"})
self.assertEqual(f.installed["proxmox-kernel-7.0"], "7.0.14-20")
self.assertNotIn("7.0.14-22-pve", f.boot)
def test_ring0_told_kernel_gone_is_refused_before_any_change(self):
f = kfake()
f.live["proxmox-kernel-7.0"] = {"7.0.14-22"} # 7.0.14-20 is no longer in the archive
told = [{"name": "proxmox-kernel-7.0", "version": "7.0.14-20", "origin": "Proxmox Debian Repository"},
{"name": "proxmox-kernel-7.0.14-20-pve-signed", "version": "7.0.14-20", "origin": "Proxmox Debian Repository"}]
f.plan.update(select="listed", packages=told, expect_kver="7.0.14-20-pve")
rep = self.refused(f, "R7")
self.assertIsNone(f.env)
self.assertNotIn(osapply.KERNEL_DEFAULT_CFG, f.tree, "refused before the default was even pinned")
self.assertEqual(f.installed["proxmox-kernel-7.0"], "7.0.2-6")
def test_nothing_pending_changes_nothing(self): def test_nothing_pending_changes_nothing(self):
f = kfake(kernel_pending=[]) f = kfake(kernel_pending=[])
rc, rep = run(f) rc, rep = run(f)
+21 -2
View File
@@ -33,7 +33,7 @@ const OpKernelStep = "os_kernel_step"
// DefaultKernelJudgeWait is how long a one-shot boot may take to come back healthy before the agent reverts it ONCE. // DefaultKernelJudgeWait is how long a one-shot boot may take to come back healthy before the agent reverts it ONCE.
// Measured 2026-10-07 (`audits/kernel-lane-2026-10-07/B/`): every container healthy 68 s after a reboot on demo-felhom, // Measured 2026-10-07 (`audits/kernel-lane-2026-10-07/B/`): every container healthy 68 s after a reboot on demo-felhom,
// 272 s on demo-hp; 20 minutes stays under the hub's 30-minute host_stale (a box that never comes back alarms after it). // 272 s on demo-hp; 20 minutes stays under the hub's 45-minute host_stale (a box that never comes back alarms after it).
const DefaultKernelJudgeWait = 20 * time.Minute const DefaultKernelJudgeWait = 20 * time.Minute
var kverRE = regexp.MustCompile(`^[0-9]+\.[0-9]+\.[0-9]+-[0-9]+-pve$`) var kverRE = regexp.MustCompile(`^[0-9]+\.[0-9]+\.[0-9]+-[0-9]+-pve$`)
@@ -119,8 +119,12 @@ func (l *Leg) runKernel(ctx context.Context, runID string, vmid int, trigger str
lg.Info("osupdate: kernel step skipped — ring 1 boots only a kernel a signed os_kernel_step staged", "phase", st.Phase, "staged", st.To, "want", want) lg.Info("osupdate: kernel step skipped — ring 1 boots only a kernel a signed os_kernel_step staged", "phase", st.Phase, "staged", st.To, "want", want)
return Report{} return Report{}
default: default:
// R-898: EXACTLY the kernel the household was told about — never "whatever is pending tonight" (the sources can
// offer a newer one by night; the step then refused, R23, and the night was lost). A version no longer
// installable is refused by the wrapper before any change (R7) and the hub tells the household again.
wr, cerr := l.call(ctx, runID, kernelPlan("apply", vmid, map[string]any{"release_id": "ring0-" + runID, wr, cerr := l.call(ctx, runID, kernelPlan("apply", vmid, map[string]any{"release_id": "ring0-" + runID,
"select": "pending-kernel", "expect_kver": want, "run_id": runID, "trigger": trigger, "ring": blk.Ring})) "select": "listed", "packages": KernelSet(want), "expect_kver": want, "run_id": runID, "trigger": trigger,
"ring": blk.Ring}))
rep.unsent = reportFile(l.planDir(), runID, LayerKernel, "apply") rep.unsent = reportFile(l.planDir(), runID, LayerKernel, "apply")
rep.Kernel = rawOrNil(wr.Kernel) rep.Kernel = rawOrNil(wr.Kernel)
switch { switch {
@@ -322,6 +326,21 @@ func truncate(s string, n int) string {
return s[:n] return s[:n]
} }
// KernelSet is the package set that installs exactly kver (R-898; the hub's kernelSet, field-exact): the series
// meta-package and the signed image, both at the kernel's own version. Proxmox keeps old kernel versions in its archive.
// nil for a string that is not a kernel version.
func KernelSet(kver string) []Package {
m := kverSeriesRE.FindStringSubmatch(kver)
if m == nil {
return nil
}
v := kver[:len(kver)-len("-pve")]
return []Package{{Name: "proxmox-kernel-" + m[1], Version: v, Origin: PVEOrigin},
{Name: "proxmox-kernel-" + kver + "-signed", Version: v, Origin: PVEOrigin}}
}
var kverSeriesRE = regexp.MustCompile(`^([0-9]+\.[0-9]+)\.[0-9]+-[0-9]+-pve$`)
// KernelStepParams are a signed os_kernel_step's params: the exact kernel set (the wrapper compares it with the plan). // KernelStepParams are a signed os_kernel_step's params: the exact kernel set (the wrapper compares it with the plan).
type KernelStepParams struct { type KernelStepParams struct {
ReleaseID string `json:"release_id"` ReleaseID string `json:"release_id"`
+16 -2
View File
@@ -54,8 +54,12 @@ func TestKernel_Ring0ToldNightStagesThenReboots(t *testing.T) {
ap = x ap = x
} }
} }
if ap["select"] != "pending-kernel" || ap["expect_kver"] != kNew || ap["lane"] != "slow" { // R-898: EXACTLY the told kernel — the listed set derived from it, never "pending" (red before the fix: select was
t.Fatalf("stage plan = %v", ap) // pending-kernel, so a newer kernel in the sources by night was refused R23 and the night was lost)
pk, _ := json.Marshal(ap["packages"])
if ap["select"] != "listed" || ap["expect_kver"] != kNew || ap["lane"] != "slow" ||
string(pk) != `[{"name":"proxmox-kernel-7.0","origin":"Proxmox Debian Repository","version":"7.0.14-22"},{"name":"proxmox-kernel-7.0.14-22-pve-signed","origin":"Proxmox Debian Repository","version":"7.0.14-22"}]` {
t.Fatalf("stage plan = %v (packages %s)", ap, pk)
} }
if p.Kernel.Outcome != "staged" || !p.Kernel.Healthy { if p.Kernel.Outcome != "staged" || !p.Kernel.Healthy {
t.Fatalf("kernel report = %+v", p.Kernel) t.Fatalf("kernel report = %+v", p.Kernel)
@@ -312,3 +316,13 @@ func TestKernel_KeptStageReportIsStaged(t *testing.T) {
t.Fatalf("kept = %+v", rep) t.Fatalf("kept = %+v", rep)
} }
} }
func TestKernelSet(t *testing.T) {
if got := KernelSet("7.0.14-20-pve"); len(got) != 2 || got[0].Name != "proxmox-kernel-7.0" || got[0].Version != "7.0.14-20" ||
got[1].Name != "proxmox-kernel-7.0.14-20-pve-signed" {
t.Fatalf("%+v", got)
}
if KernelSet("7.0; reboot") != nil || KernelSet("") != nil {
t.Fatal("a non-kernel string must give no set")
}
}