Compare commits
4 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| c9013bb47d | |||
| 2d1e5d0774 | |||
| f09c53efc1 | |||
| 92d647a6f6 |
+24
-2
@@ -1,4 +1,26 @@
|
|||||||
## Unreleased — part of v0.152.0: the kernel lane (R-836; `09` §3 decisions 164, 172; `11` §5.11) (2026-10-07)
|
## v0.153.0 — ring 0 stages exactly the told kernel (R-898; `09` §3 decision 176) (2026-10-07)
|
||||||
|
|
||||||
|
Released by `scripts/release-agent.sh`: binary sha256 `b204ebe6d65944f43b70dcfa4ae0dfb90da38c92e2ba6a38a1826e488eeb6630`, bundle
|
||||||
|
`6db216275eb0dd39188d93481a2045998a69e8cb7838ad908a58d65d9a6a9b57` (tag `v0.153.0` = `2d1e5d0`). Delivered (binary only — the
|
||||||
|
bundle files are unchanged from 0.152.0) to demo-hp, demo-felhom, Tester 1 on 2026-10-07 19:03.
|
||||||
|
|
||||||
|
**Delivery: the agent binary only** — no root file changed (the wrapper is unchanged; its tests gained two cases).
|
||||||
|
|
||||||
|
- `internal/osupdate/kernel.go`: ring 0's night kernel step stages EXACTLY the kernel the household was told about
|
||||||
|
(select `listed`, `KernelSet(kver)` = the series meta-package and the signed image at the kernel's own version) instead
|
||||||
|
of "whatever is pending tonight". Seen 2026-10-07: demo-felhom was told about 7.0.14-20 while its sources offered
|
||||||
|
7.0.14-22 by night — the old code staged `pending-kernel` and the wrapper refused it (R23), losing the night. A told
|
||||||
|
version that is no longer installable is refused by the wrapper before any change (R7) and the hub tells the household
|
||||||
|
again for the newer kernel (hub v0.143.1). Tests `TestKernel_Ring0ToldNightStagesThenReboots` (red-proved against the
|
||||||
|
old select), `TestKernelSet`; wrapper `test_ring0_listed_installs_the_told_kernel_not_the_newest`,
|
||||||
|
`test_ring0_told_kernel_gone_is_refused_before_any_change`.
|
||||||
|
|
||||||
|
## v0.152.0 — the kernel lane (R-836; `09` §3 decisions 164, 172; `11` §5.11) (2026-10-07)
|
||||||
|
|
||||||
|
Released by `scripts/release-agent.sh`: binary sha256 `95ff42208e36ba49b6e2b09a97e81a6fa11562ecc8042f1ed18d378b8b1f88b1`,
|
||||||
|
config bundle sha256 `f0c2cec374b711b3c131c955012b33fd0ad495337d049b7a743c3eef9e85c20b` (tag `v0.152.0` = `d03ab7f`).
|
||||||
|
Step bundle `0.152.0-step1` sha256 `0b71d32b054cf3b7ade0234ffcbb0df159901f542cde540adaee411db466f48e` (the 0.151.0 bundle
|
||||||
|
with only `felhom-os-apply` replaced; `scripts/build-step-bundle.py`), published as package version `0.152.0-step1`.
|
||||||
|
|
||||||
**Delivery: agent binary, then the STEP bundle `0.152.0-step1`, then the bundle `0.152.0`** — the bundle ADDS two paths
|
**Delivery: agent binary, then the STEP bundle `0.152.0-step1`, then the bundle `0.152.0`** — the bundle ADDS two paths
|
||||||
(the GRUB generators), and an installed `felhom-os-apply` refuses a path its own table lacks (R16, R-880).
|
(the GRUB generators), and an installed `felhom-os-apply` refuses a path its own table lacks (R16, R-880).
|
||||||
@@ -34,7 +56,7 @@ candidate 2 (`audits/kernel-spike-2026-10-07/`), with option C on the one-shot e
|
|||||||
signed `os_kernel_step` staged (`KernelStepExecutor`: stage only, under the heavy-op gate). The hub hears `staged`
|
signed `os_kernel_step` staged (`KernelStepExecutor`: stage only, under the heavy-op gate). The hub hears `staged`
|
||||||
BEFORE the reboot. At every start `KernelAfterBoot`: on the new kernel it JUDGES the boot — `KernelVerdict` = the
|
BEFORE the reboot. At every start `KernelAfterBoot`: on the new kernel it JUDGES the boot — `KernelVerdict` = the
|
||||||
host health rule (`11` §8.2) AND the box reached the hub (the `judging` report itself) — for 20 minutes (measured:
|
host health rule (`11` §8.2) AND the box reached the hub (the `judging` report itself) — for 20 minutes (measured:
|
||||||
everything healthy 68 s after the reboot on demo-felhom, 272 s on demo-hp; under the hub's 30-minute `host_stale`).
|
everything healthy 68 s after the reboot on demo-felhom, 272 s on demo-hp; under the hub's 45-minute `host_stale` (`alerting.stale_threshold`)).
|
||||||
Healthy → `kernel-good`, outcome `applied`; not healthy → outcome `health_failed`, then ONE `kernel-revert`.
|
Healthy → `kernel-good`, outcome `applied`; not healthy → outcome `health_failed`, then ONE `kernel-revert`.
|
||||||
Tests: `TestKernel*` (13); red-proofs in the same file.
|
Tests: `TestKernel*` (13); red-proofs in the same file.
|
||||||
- `internal/hub`: `WireOSUpdate.Kernel` {kver, tonight, notified_at}. `internal/reconcile`: `os_kernel_step` is
|
- `internal/hub`: `WireOSUpdate.Kernel` {kver, tonight, notified_at}. `internal/reconcile`: `os_kernel_step` is
|
||||||
|
|||||||
@@ -1,5 +1,7 @@
|
|||||||
# CONTEXT — felhom-agent working state
|
# CONTEXT — felhom-agent working state
|
||||||
|
|
||||||
|
> **2026-10-07 (evening) — v0.152.0, the kernel lane (R-836, decision 172, `11` §5.11).** Wrapper layer `kernel` + two GRUB generators in the bundle (delivered with step bundle `0.152.0-step1` — the bundle adds paths, R-880); `osupdate/kernel.go`: night step on told nights only, after-boot judge (host rule + hub reached, 20 min), ONE self-revert, `os_kernel_step` stages only. Proven on Tester 1 (panic → fell_back, held guest → self_reverted, healthy → 7.0.14-22 default). Open: R-898, R-897; the ring-0 night run.
|
||||||
|
|
||||||
> **2026-10-04 night — v0.143.0 RELEASED + vouched (R-840, decision 96): the config bundle.** `felhom-os-apply` mode
|
> **2026-10-04 night — v0.143.0 RELEASED + vouched (R-840, decision 96): the config bundle.** `felhom-os-apply` mode
|
||||||
> `bundle` (signed `agent_config_update`, verified by the wrapper itself; trust files never bundle paths) +
|
> `bundle` (signed `agent_config_update`, verified by the wrapper itself; trust files never bundle paths) +
|
||||||
> `--install-bundle` (installer 1.31.0); `BUNDLE_FILES` is the one table; `scripts/build-config-bundle.py`;
|
> `--install-bundle` (installer 1.31.0); `BUNDLE_FILES` is the one table; `scripts/build-config-bundle.py`;
|
||||||
|
|||||||
@@ -1,9 +1,9 @@
|
|||||||
# REPORT — v0.151.0 released and delivered (2026-10-07 day)
|
# REPORT — agent v0.153.0: ring 0 stages exactly the told kernel (2026-10-07, late evening)
|
||||||
|
|
||||||
On the operator's word (`09` §3 decisions 163, 165, 168, 169). sha `0464354f…`, bundle `bacd1d17…`, tag `v0.151.0` = `dd7cdc0`.
|
- R-898 (closed): the night kernel step on ring 0 stages the kernel the household was told about (`select listed`,
|
||||||
Delivered binary first, then the bundle (its sudoers drops the `tee` grant 0.150.0 used), to demo-hp, demo-felhom and
|
`osupdate.KernelSet(kver)`), never "whatever is pending tonight". A told version no longer installable → the wrapper
|
||||||
Tester 1 — probe 68/68 on each. **Vouch refused by the hub** (`golden_behind_fleet`): new installs keep 0.150.0 until the
|
refuses before any change (R7); the hub re-tells the household for the newer kernel.
|
||||||
weekly golden. Carries: the Proxmox package lane + `/etc/pve` write gate (R-812 A — proven on demo-felhom: 65 packages,
|
- Tests: `TestKernel_Ring0ToldNightStagesThenReboots` (red-proved against the old select), `TestKernelSet`; wrapper
|
||||||
70 s, healthy), the `controller-image` root verb (R-861 a — on demo-hp an `alpine` ref is refused; a managed swap not yet
|
cases `test_ring0_listed_installs_the_told_kernel_not_the_newest`, `test_ring0_told_kernel_gone_is_refused_before_any_change`.
|
||||||
seen), anchored felhom-op lines (B2), the other-key archive ledger (R-366), the `-directive` flag removed (R-105). Evidence
|
- Released `v0.153.0`, binary `b204ebe6…`; delivered (binary only) to demo-hp, demo-felhom, Tester 1 at 19:03.
|
||||||
`felhom.eu/documentation/audits/day-2026-10-07/`. Shared rule file: decision 162 line added.
|
- Tonight (7→8): demo-felhom stages 7.0.14-20, demo-hp 7.0.14-22 — both households told. Read back 2026-10-08.
|
||||||
|
|||||||
@@ -881,7 +881,8 @@ func runDaemon(cfg config.Config, logger *slog.Logger, logRing *applog.Ring) int
|
|||||||
// judge it (the host health rule + the hub reached) for KernelJudgeWait, then make it the default or revert ONCE.
|
// judge it (the host health rule + the hub reached) for KernelJudgeWait, then make it the default or revert ONCE.
|
||||||
// The wait: measured 2026-10-07 (`audits/kernel-lane-2026-10-07/B/`) — every container healthy 68 s after the
|
// The wait: measured 2026-10-07 (`audits/kernel-lane-2026-10-07/B/`) — every container healthy 68 s after the
|
||||||
// reboot on demo-felhom and 272 s on demo-hp (the hub reached at 63 s / 189 s); 20 minutes leaves room for a slow
|
// reboot on demo-felhom and 272 s on demo-hp (the hub reached at 63 s / 189 s); 20 minutes leaves room for a slow
|
||||||
// network and stays under the hub's 30-minute host_stale. On a box without the kernel lane (an older wrapper, a BYO host) the check is refused and logged.
|
// network and stays under the hub's 45-minute host_stale (its
|
||||||
|
// alerting.stale_threshold; host_down at 90). On a box without the kernel lane (an older wrapper, a BYO host) the check is refused and logged.
|
||||||
go osLeg.KernelAfterBoot(ctx, 0, osupdate.KernelJudge{Wait: osupdate.DefaultKernelJudgeWait})
|
go osLeg.KernelAfterBoot(ctx, 0, osupdate.KernelJudge{Wait: osupdate.DefaultKernelJudgeWait})
|
||||||
|
|
||||||
// Reconcile (slice 4) runs alongside the hub loop, sharing the per-guest queue
|
// Reconcile (slice 4) runs alongside the hub loop, sharing the per-guest queue
|
||||||
|
|||||||
@@ -1838,6 +1838,33 @@ class KernelLane(unittest.TestCase):
|
|||||||
m.origins = {"proxmox-kernel-7.0": DEB}
|
m.origins = {"proxmox-kernel-7.0": DEB}
|
||||||
self.refused(m, "R2")
|
self.refused(m, "R2")
|
||||||
|
|
||||||
|
# R-898: ring 0 stages EXACTLY the told kernel (select listed, the set derived from it) — even when the sources
|
||||||
|
# offer a newer one by night; a told version that can no longer be installed is refused BEFORE any change (R7).
|
||||||
|
def test_ring0_listed_installs_the_told_kernel_not_the_newest(self):
|
||||||
|
f = kfake()
|
||||||
|
f.live["proxmox-kernel-7.0"] = {"7.0.14-20", "7.0.14-22", "7.0.2-6"}
|
||||||
|
f.live["proxmox-kernel-7.0.14-20-pve-signed"] = {"7.0.14-20"}
|
||||||
|
told = [{"name": "proxmox-kernel-7.0", "version": "7.0.14-20", "origin": "Proxmox Debian Repository"},
|
||||||
|
{"name": "proxmox-kernel-7.0.14-20-pve-signed", "version": "7.0.14-20", "origin": "Proxmox Debian Repository"}]
|
||||||
|
f.plan.update(select="listed", packages=told, expect_kver="7.0.14-20-pve")
|
||||||
|
rc, rep = run(f)
|
||||||
|
self.assertEqual(rc, 0, rep)
|
||||||
|
self.assertEqual(rep["authority"], "ring0")
|
||||||
|
self.assertEqual(f.env, {"felhom_next": "7.0.14-20-pve"})
|
||||||
|
self.assertEqual(f.installed["proxmox-kernel-7.0"], "7.0.14-20")
|
||||||
|
self.assertNotIn("7.0.14-22-pve", f.boot)
|
||||||
|
|
||||||
|
def test_ring0_told_kernel_gone_is_refused_before_any_change(self):
|
||||||
|
f = kfake()
|
||||||
|
f.live["proxmox-kernel-7.0"] = {"7.0.14-22"} # 7.0.14-20 is no longer in the archive
|
||||||
|
told = [{"name": "proxmox-kernel-7.0", "version": "7.0.14-20", "origin": "Proxmox Debian Repository"},
|
||||||
|
{"name": "proxmox-kernel-7.0.14-20-pve-signed", "version": "7.0.14-20", "origin": "Proxmox Debian Repository"}]
|
||||||
|
f.plan.update(select="listed", packages=told, expect_kver="7.0.14-20-pve")
|
||||||
|
rep = self.refused(f, "R7")
|
||||||
|
self.assertIsNone(f.env)
|
||||||
|
self.assertNotIn(osapply.KERNEL_DEFAULT_CFG, f.tree, "refused before the default was even pinned")
|
||||||
|
self.assertEqual(f.installed["proxmox-kernel-7.0"], "7.0.2-6")
|
||||||
|
|
||||||
def test_nothing_pending_changes_nothing(self):
|
def test_nothing_pending_changes_nothing(self):
|
||||||
f = kfake(kernel_pending=[])
|
f = kfake(kernel_pending=[])
|
||||||
rc, rep = run(f)
|
rc, rep = run(f)
|
||||||
|
|||||||
@@ -33,7 +33,7 @@ const OpKernelStep = "os_kernel_step"
|
|||||||
|
|
||||||
// DefaultKernelJudgeWait is how long a one-shot boot may take to come back healthy before the agent reverts it ONCE.
|
// DefaultKernelJudgeWait is how long a one-shot boot may take to come back healthy before the agent reverts it ONCE.
|
||||||
// Measured 2026-10-07 (`audits/kernel-lane-2026-10-07/B/`): every container healthy 68 s after a reboot on demo-felhom,
|
// Measured 2026-10-07 (`audits/kernel-lane-2026-10-07/B/`): every container healthy 68 s after a reboot on demo-felhom,
|
||||||
// 272 s on demo-hp; 20 minutes stays under the hub's 30-minute host_stale (a box that never comes back alarms after it).
|
// 272 s on demo-hp; 20 minutes stays under the hub's 45-minute host_stale (a box that never comes back alarms after it).
|
||||||
const DefaultKernelJudgeWait = 20 * time.Minute
|
const DefaultKernelJudgeWait = 20 * time.Minute
|
||||||
|
|
||||||
var kverRE = regexp.MustCompile(`^[0-9]+\.[0-9]+\.[0-9]+-[0-9]+-pve$`)
|
var kverRE = regexp.MustCompile(`^[0-9]+\.[0-9]+\.[0-9]+-[0-9]+-pve$`)
|
||||||
@@ -119,8 +119,12 @@ func (l *Leg) runKernel(ctx context.Context, runID string, vmid int, trigger str
|
|||||||
lg.Info("osupdate: kernel step skipped — ring 1 boots only a kernel a signed os_kernel_step staged", "phase", st.Phase, "staged", st.To, "want", want)
|
lg.Info("osupdate: kernel step skipped — ring 1 boots only a kernel a signed os_kernel_step staged", "phase", st.Phase, "staged", st.To, "want", want)
|
||||||
return Report{}
|
return Report{}
|
||||||
default:
|
default:
|
||||||
|
// R-898: EXACTLY the kernel the household was told about — never "whatever is pending tonight" (the sources can
|
||||||
|
// offer a newer one by night; the step then refused, R23, and the night was lost). A version no longer
|
||||||
|
// installable is refused by the wrapper before any change (R7) and the hub tells the household again.
|
||||||
wr, cerr := l.call(ctx, runID, kernelPlan("apply", vmid, map[string]any{"release_id": "ring0-" + runID,
|
wr, cerr := l.call(ctx, runID, kernelPlan("apply", vmid, map[string]any{"release_id": "ring0-" + runID,
|
||||||
"select": "pending-kernel", "expect_kver": want, "run_id": runID, "trigger": trigger, "ring": blk.Ring}))
|
"select": "listed", "packages": KernelSet(want), "expect_kver": want, "run_id": runID, "trigger": trigger,
|
||||||
|
"ring": blk.Ring}))
|
||||||
rep.unsent = reportFile(l.planDir(), runID, LayerKernel, "apply")
|
rep.unsent = reportFile(l.planDir(), runID, LayerKernel, "apply")
|
||||||
rep.Kernel = rawOrNil(wr.Kernel)
|
rep.Kernel = rawOrNil(wr.Kernel)
|
||||||
switch {
|
switch {
|
||||||
@@ -322,6 +326,21 @@ func truncate(s string, n int) string {
|
|||||||
return s[:n]
|
return s[:n]
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// KernelSet is the package set that installs exactly kver (R-898; the hub's kernelSet, field-exact): the series
|
||||||
|
// meta-package and the signed image, both at the kernel's own version. Proxmox keeps old kernel versions in its archive.
|
||||||
|
// nil for a string that is not a kernel version.
|
||||||
|
func KernelSet(kver string) []Package {
|
||||||
|
m := kverSeriesRE.FindStringSubmatch(kver)
|
||||||
|
if m == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
v := kver[:len(kver)-len("-pve")]
|
||||||
|
return []Package{{Name: "proxmox-kernel-" + m[1], Version: v, Origin: PVEOrigin},
|
||||||
|
{Name: "proxmox-kernel-" + kver + "-signed", Version: v, Origin: PVEOrigin}}
|
||||||
|
}
|
||||||
|
|
||||||
|
var kverSeriesRE = regexp.MustCompile(`^([0-9]+\.[0-9]+)\.[0-9]+-[0-9]+-pve$`)
|
||||||
|
|
||||||
// KernelStepParams are a signed os_kernel_step's params: the exact kernel set (the wrapper compares it with the plan).
|
// KernelStepParams are a signed os_kernel_step's params: the exact kernel set (the wrapper compares it with the plan).
|
||||||
type KernelStepParams struct {
|
type KernelStepParams struct {
|
||||||
ReleaseID string `json:"release_id"`
|
ReleaseID string `json:"release_id"`
|
||||||
|
|||||||
@@ -54,8 +54,12 @@ func TestKernel_Ring0ToldNightStagesThenReboots(t *testing.T) {
|
|||||||
ap = x
|
ap = x
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if ap["select"] != "pending-kernel" || ap["expect_kver"] != kNew || ap["lane"] != "slow" {
|
// R-898: EXACTLY the told kernel — the listed set derived from it, never "pending" (red before the fix: select was
|
||||||
t.Fatalf("stage plan = %v", ap)
|
// pending-kernel, so a newer kernel in the sources by night was refused R23 and the night was lost)
|
||||||
|
pk, _ := json.Marshal(ap["packages"])
|
||||||
|
if ap["select"] != "listed" || ap["expect_kver"] != kNew || ap["lane"] != "slow" ||
|
||||||
|
string(pk) != `[{"name":"proxmox-kernel-7.0","origin":"Proxmox Debian Repository","version":"7.0.14-22"},{"name":"proxmox-kernel-7.0.14-22-pve-signed","origin":"Proxmox Debian Repository","version":"7.0.14-22"}]` {
|
||||||
|
t.Fatalf("stage plan = %v (packages %s)", ap, pk)
|
||||||
}
|
}
|
||||||
if p.Kernel.Outcome != "staged" || !p.Kernel.Healthy {
|
if p.Kernel.Outcome != "staged" || !p.Kernel.Healthy {
|
||||||
t.Fatalf("kernel report = %+v", p.Kernel)
|
t.Fatalf("kernel report = %+v", p.Kernel)
|
||||||
@@ -312,3 +316,13 @@ func TestKernel_KeptStageReportIsStaged(t *testing.T) {
|
|||||||
t.Fatalf("kept = %+v", rep)
|
t.Fatalf("kept = %+v", rep)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestKernelSet(t *testing.T) {
|
||||||
|
if got := KernelSet("7.0.14-20-pve"); len(got) != 2 || got[0].Name != "proxmox-kernel-7.0" || got[0].Version != "7.0.14-20" ||
|
||||||
|
got[1].Name != "proxmox-kernel-7.0.14-20-pve-signed" {
|
||||||
|
t.Fatalf("%+v", got)
|
||||||
|
}
|
||||||
|
if KernelSet("7.0; reboot") != nil || KernelSet("") != nil {
|
||||||
|
t.Fatal("a non-kernel string must give no set")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user