Compare commits
3 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 2d1e5d0774 | |||
| f09c53efc1 | |||
| 92d647a6f6 |
+20
-2
@@ -1,4 +1,22 @@
|
||||
## Unreleased — part of v0.152.0: the kernel lane (R-836; `09` §3 decisions 164, 172; `11` §5.11) (2026-10-07)
|
||||
## Unreleased — part of v0.153.0: ring 0 stages exactly the told kernel (R-898; `09` §3 decision 176) (2026-10-07)
|
||||
|
||||
**Delivery: the agent binary only** — no root file changed (the wrapper is unchanged; its tests gained two cases).
|
||||
|
||||
- `internal/osupdate/kernel.go`: ring 0's night kernel step stages EXACTLY the kernel the household was told about
|
||||
(select `listed`, `KernelSet(kver)` = the series meta-package and the signed image at the kernel's own version) instead
|
||||
of "whatever is pending tonight". Seen 2026-10-07: demo-felhom was told about 7.0.14-20 while its sources offered
|
||||
7.0.14-22 by night — the old code staged `pending-kernel` and the wrapper refused it (R23), losing the night. A told
|
||||
version that is no longer installable is refused by the wrapper before any change (R7) and the hub tells the household
|
||||
again for the newer kernel (hub v0.143.1). Tests `TestKernel_Ring0ToldNightStagesThenReboots` (red-proved against the
|
||||
old select), `TestKernelSet`; wrapper `test_ring0_listed_installs_the_told_kernel_not_the_newest`,
|
||||
`test_ring0_told_kernel_gone_is_refused_before_any_change`.
|
||||
|
||||
## v0.152.0 — the kernel lane (R-836; `09` §3 decisions 164, 172; `11` §5.11) (2026-10-07)
|
||||
|
||||
Released by `scripts/release-agent.sh`: binary sha256 `95ff42208e36ba49b6e2b09a97e81a6fa11562ecc8042f1ed18d378b8b1f88b1`,
|
||||
config bundle sha256 `f0c2cec374b711b3c131c955012b33fd0ad495337d049b7a743c3eef9e85c20b` (tag `v0.152.0` = `d03ab7f`).
|
||||
Step bundle `0.152.0-step1` sha256 `0b71d32b054cf3b7ade0234ffcbb0df159901f542cde540adaee411db466f48e` (the 0.151.0 bundle
|
||||
with only `felhom-os-apply` replaced; `scripts/build-step-bundle.py`), published as package version `0.152.0-step1`.
|
||||
|
||||
**Delivery: agent binary, then the STEP bundle `0.152.0-step1`, then the bundle `0.152.0`** — the bundle ADDS two paths
|
||||
(the GRUB generators), and an installed `felhom-os-apply` refuses a path its own table lacks (R16, R-880).
|
||||
@@ -34,7 +52,7 @@ candidate 2 (`audits/kernel-spike-2026-10-07/`), with option C on the one-shot e
|
||||
signed `os_kernel_step` staged (`KernelStepExecutor`: stage only, under the heavy-op gate). The hub hears `staged`
|
||||
BEFORE the reboot. At every start `KernelAfterBoot`: on the new kernel it JUDGES the boot — `KernelVerdict` = the
|
||||
host health rule (`11` §8.2) AND the box reached the hub (the `judging` report itself) — for 20 minutes (measured:
|
||||
everything healthy 68 s after the reboot on demo-felhom, 272 s on demo-hp; under the hub's 30-minute `host_stale`).
|
||||
everything healthy 68 s after the reboot on demo-felhom, 272 s on demo-hp; under the hub's 45-minute `host_stale` (`alerting.stale_threshold`)).
|
||||
Healthy → `kernel-good`, outcome `applied`; not healthy → outcome `health_failed`, then ONE `kernel-revert`.
|
||||
Tests: `TestKernel*` (13); red-proofs in the same file.
|
||||
- `internal/hub`: `WireOSUpdate.Kernel` {kver, tonight, notified_at}. `internal/reconcile`: `os_kernel_step` is
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# CONTEXT — felhom-agent working state
|
||||
|
||||
> **2026-10-07 (evening) — v0.152.0, the kernel lane (R-836, decision 172, `11` §5.11).** Wrapper layer `kernel` + two GRUB generators in the bundle (delivered with step bundle `0.152.0-step1` — the bundle adds paths, R-880); `osupdate/kernel.go`: night step on told nights only, after-boot judge (host rule + hub reached, 20 min), ONE self-revert, `os_kernel_step` stages only. Proven on Tester 1 (panic → fell_back, held guest → self_reverted, healthy → 7.0.14-22 default). Open: R-898, R-897; the ring-0 night run.
|
||||
|
||||
> **2026-10-04 night — v0.143.0 RELEASED + vouched (R-840, decision 96): the config bundle.** `felhom-os-apply` mode
|
||||
> `bundle` (signed `agent_config_update`, verified by the wrapper itself; trust files never bundle paths) +
|
||||
> `--install-bundle` (installer 1.31.0); `BUNDLE_FILES` is the one table; `scripts/build-config-bundle.py`;
|
||||
|
||||
@@ -1,9 +1,18 @@
|
||||
# REPORT — v0.151.0 released and delivered (2026-10-07 day)
|
||||
# REPORT — agent v0.152.0: the kernel lane (2026-10-07)
|
||||
|
||||
On the operator's word (`09` §3 decisions 163, 165, 168, 169). sha `0464354f…`, bundle `bacd1d17…`, tag `v0.151.0` = `dd7cdc0`.
|
||||
Delivered binary first, then the bundle (its sudoers drops the `tee` grant 0.150.0 used), to demo-hp, demo-felhom and
|
||||
Tester 1 — probe 68/68 on each. **Vouch refused by the hub** (`golden_behind_fleet`): new installs keep 0.150.0 until the
|
||||
weekly golden. Carries: the Proxmox package lane + `/etc/pve` write gate (R-812 A — proven on demo-felhom: 65 packages,
|
||||
70 s, healthy), the `controller-image` root verb (R-861 a — on demo-hp an `alpine` ref is refused; a managed swap not yet
|
||||
seen), anchored felhom-op lines (B2), the other-key archive ledger (R-366), the `-directive` flag removed (R-105). Evidence
|
||||
`felhom.eu/documentation/audits/day-2026-10-07/`. Shared rule file: decision 162 line added.
|
||||
**What:** R-836, `09` §3 decision 172 — a new kernel boots ONCE through a flag on the ESP; a crash falls back to the old
|
||||
kernel by itself; a healthy boot (host health rule + the hub reached, 20 min) makes it the default; a booted-but-unhealthy
|
||||
one is reverted ONCE. Design: `felhom.eu/documentation/architecture/11-os-updates.md` §5.11.
|
||||
|
||||
- Wrapper `configs/felhom-os-apply`: layer `kernel` (stage, kernel-reboot, kernel-boot, kernel-good, kernel-revert,
|
||||
kernel-cancel, kernel-status; R20–R23). Bundle: `/etc/grub.d/01_felhom_oneshot`, `/etc/grub.d/42_felhom_oneshot`.
|
||||
- Agent `internal/osupdate/kernel.go`: the night step (told nights only), `KernelAfterBoot`, `KernelStepExecutor`.
|
||||
- Tests: `KernelLane` (27), `KernelStepCannotLeaveTheBoxOff` (3), `TestKernel*` (13); red-proofs
|
||||
`felhom.eu/documentation/audits/kernel-lane-2026-10-07/A/redproof.txt`.
|
||||
- Released `v0.152.0` (`d03ab7f`; binary `95ff4220…`, bundle `f0c2cec3…`) + step bundle `0.152.0-step1` (`0b71d32b…`).
|
||||
Delivered by signed jobs to Tester 1, demo-hp, demo-felhom (binary → step bundle → bundle). Not vouched (the golden is
|
||||
behind; waiver to 2026-10-13).
|
||||
- **Proven on the Tester 1 box** (`felhom.eu/documentation/audits/kernel-lane-2026-10-07/E/RESULT.md`): forced panic →
|
||||
fell_back by itself; held guest → one self-revert after 20 min; healthy → 7.0.14-22 the default.
|
||||
- **Open:** the ring-0 night run (R-836 says where it stopped); R-898 (ring 0 stages the pending kernel, not exactly the
|
||||
told one); R-897 (post-reboot drive re-bind races the first backup).
|
||||
|
||||
@@ -881,7 +881,8 @@ func runDaemon(cfg config.Config, logger *slog.Logger, logRing *applog.Ring) int
|
||||
// judge it (the host health rule + the hub reached) for KernelJudgeWait, then make it the default or revert ONCE.
|
||||
// The wait: measured 2026-10-07 (`audits/kernel-lane-2026-10-07/B/`) — every container healthy 68 s after the
|
||||
// reboot on demo-felhom and 272 s on demo-hp (the hub reached at 63 s / 189 s); 20 minutes leaves room for a slow
|
||||
// network and stays under the hub's 30-minute host_stale. On a box without the kernel lane (an older wrapper, a BYO host) the check is refused and logged.
|
||||
// network and stays under the hub's 45-minute host_stale (its
|
||||
// alerting.stale_threshold; host_down at 90). On a box without the kernel lane (an older wrapper, a BYO host) the check is refused and logged.
|
||||
go osLeg.KernelAfterBoot(ctx, 0, osupdate.KernelJudge{Wait: osupdate.DefaultKernelJudgeWait})
|
||||
|
||||
// Reconcile (slice 4) runs alongside the hub loop, sharing the per-guest queue
|
||||
|
||||
@@ -1838,6 +1838,33 @@ class KernelLane(unittest.TestCase):
|
||||
m.origins = {"proxmox-kernel-7.0": DEB}
|
||||
self.refused(m, "R2")
|
||||
|
||||
# R-898: ring 0 stages EXACTLY the told kernel (select listed, the set derived from it) — even when the sources
|
||||
# offer a newer one by night; a told version that can no longer be installed is refused BEFORE any change (R7).
|
||||
def test_ring0_listed_installs_the_told_kernel_not_the_newest(self):
|
||||
f = kfake()
|
||||
f.live["proxmox-kernel-7.0"] = {"7.0.14-20", "7.0.14-22", "7.0.2-6"}
|
||||
f.live["proxmox-kernel-7.0.14-20-pve-signed"] = {"7.0.14-20"}
|
||||
told = [{"name": "proxmox-kernel-7.0", "version": "7.0.14-20", "origin": "Proxmox Debian Repository"},
|
||||
{"name": "proxmox-kernel-7.0.14-20-pve-signed", "version": "7.0.14-20", "origin": "Proxmox Debian Repository"}]
|
||||
f.plan.update(select="listed", packages=told, expect_kver="7.0.14-20-pve")
|
||||
rc, rep = run(f)
|
||||
self.assertEqual(rc, 0, rep)
|
||||
self.assertEqual(rep["authority"], "ring0")
|
||||
self.assertEqual(f.env, {"felhom_next": "7.0.14-20-pve"})
|
||||
self.assertEqual(f.installed["proxmox-kernel-7.0"], "7.0.14-20")
|
||||
self.assertNotIn("7.0.14-22-pve", f.boot)
|
||||
|
||||
def test_ring0_told_kernel_gone_is_refused_before_any_change(self):
|
||||
f = kfake()
|
||||
f.live["proxmox-kernel-7.0"] = {"7.0.14-22"} # 7.0.14-20 is no longer in the archive
|
||||
told = [{"name": "proxmox-kernel-7.0", "version": "7.0.14-20", "origin": "Proxmox Debian Repository"},
|
||||
{"name": "proxmox-kernel-7.0.14-20-pve-signed", "version": "7.0.14-20", "origin": "Proxmox Debian Repository"}]
|
||||
f.plan.update(select="listed", packages=told, expect_kver="7.0.14-20-pve")
|
||||
rep = self.refused(f, "R7")
|
||||
self.assertIsNone(f.env)
|
||||
self.assertNotIn(osapply.KERNEL_DEFAULT_CFG, f.tree, "refused before the default was even pinned")
|
||||
self.assertEqual(f.installed["proxmox-kernel-7.0"], "7.0.2-6")
|
||||
|
||||
def test_nothing_pending_changes_nothing(self):
|
||||
f = kfake(kernel_pending=[])
|
||||
rc, rep = run(f)
|
||||
|
||||
@@ -33,7 +33,7 @@ const OpKernelStep = "os_kernel_step"
|
||||
|
||||
// DefaultKernelJudgeWait is how long a one-shot boot may take to come back healthy before the agent reverts it ONCE.
|
||||
// Measured 2026-10-07 (`audits/kernel-lane-2026-10-07/B/`): every container healthy 68 s after a reboot on demo-felhom,
|
||||
// 272 s on demo-hp; 20 minutes stays under the hub's 30-minute host_stale (a box that never comes back alarms after it).
|
||||
// 272 s on demo-hp; 20 minutes stays under the hub's 45-minute host_stale (a box that never comes back alarms after it).
|
||||
const DefaultKernelJudgeWait = 20 * time.Minute
|
||||
|
||||
var kverRE = regexp.MustCompile(`^[0-9]+\.[0-9]+\.[0-9]+-[0-9]+-pve$`)
|
||||
@@ -119,8 +119,12 @@ func (l *Leg) runKernel(ctx context.Context, runID string, vmid int, trigger str
|
||||
lg.Info("osupdate: kernel step skipped — ring 1 boots only a kernel a signed os_kernel_step staged", "phase", st.Phase, "staged", st.To, "want", want)
|
||||
return Report{}
|
||||
default:
|
||||
// R-898: EXACTLY the kernel the household was told about — never "whatever is pending tonight" (the sources can
|
||||
// offer a newer one by night; the step then refused, R23, and the night was lost). A version no longer
|
||||
// installable is refused by the wrapper before any change (R7) and the hub tells the household again.
|
||||
wr, cerr := l.call(ctx, runID, kernelPlan("apply", vmid, map[string]any{"release_id": "ring0-" + runID,
|
||||
"select": "pending-kernel", "expect_kver": want, "run_id": runID, "trigger": trigger, "ring": blk.Ring}))
|
||||
"select": "listed", "packages": KernelSet(want), "expect_kver": want, "run_id": runID, "trigger": trigger,
|
||||
"ring": blk.Ring}))
|
||||
rep.unsent = reportFile(l.planDir(), runID, LayerKernel, "apply")
|
||||
rep.Kernel = rawOrNil(wr.Kernel)
|
||||
switch {
|
||||
@@ -322,6 +326,21 @@ func truncate(s string, n int) string {
|
||||
return s[:n]
|
||||
}
|
||||
|
||||
// KernelSet is the package set that installs exactly kver (R-898; the hub's kernelSet, field-exact): the series
|
||||
// meta-package and the signed image, both at the kernel's own version. Proxmox keeps old kernel versions in its archive.
|
||||
// nil for a string that is not a kernel version.
|
||||
func KernelSet(kver string) []Package {
|
||||
m := kverSeriesRE.FindStringSubmatch(kver)
|
||||
if m == nil {
|
||||
return nil
|
||||
}
|
||||
v := kver[:len(kver)-len("-pve")]
|
||||
return []Package{{Name: "proxmox-kernel-" + m[1], Version: v, Origin: PVEOrigin},
|
||||
{Name: "proxmox-kernel-" + kver + "-signed", Version: v, Origin: PVEOrigin}}
|
||||
}
|
||||
|
||||
var kverSeriesRE = regexp.MustCompile(`^([0-9]+\.[0-9]+)\.[0-9]+-[0-9]+-pve$`)
|
||||
|
||||
// KernelStepParams are a signed os_kernel_step's params: the exact kernel set (the wrapper compares it with the plan).
|
||||
type KernelStepParams struct {
|
||||
ReleaseID string `json:"release_id"`
|
||||
|
||||
@@ -54,8 +54,12 @@ func TestKernel_Ring0ToldNightStagesThenReboots(t *testing.T) {
|
||||
ap = x
|
||||
}
|
||||
}
|
||||
if ap["select"] != "pending-kernel" || ap["expect_kver"] != kNew || ap["lane"] != "slow" {
|
||||
t.Fatalf("stage plan = %v", ap)
|
||||
// R-898: EXACTLY the told kernel — the listed set derived from it, never "pending" (red before the fix: select was
|
||||
// pending-kernel, so a newer kernel in the sources by night was refused R23 and the night was lost)
|
||||
pk, _ := json.Marshal(ap["packages"])
|
||||
if ap["select"] != "listed" || ap["expect_kver"] != kNew || ap["lane"] != "slow" ||
|
||||
string(pk) != `[{"name":"proxmox-kernel-7.0","origin":"Proxmox Debian Repository","version":"7.0.14-22"},{"name":"proxmox-kernel-7.0.14-22-pve-signed","origin":"Proxmox Debian Repository","version":"7.0.14-22"}]` {
|
||||
t.Fatalf("stage plan = %v (packages %s)", ap, pk)
|
||||
}
|
||||
if p.Kernel.Outcome != "staged" || !p.Kernel.Healthy {
|
||||
t.Fatalf("kernel report = %+v", p.Kernel)
|
||||
@@ -312,3 +316,13 @@ func TestKernel_KeptStageReportIsStaged(t *testing.T) {
|
||||
t.Fatalf("kept = %+v", rep)
|
||||
}
|
||||
}
|
||||
|
||||
func TestKernelSet(t *testing.T) {
|
||||
if got := KernelSet("7.0.14-20-pve"); len(got) != 2 || got[0].Name != "proxmox-kernel-7.0" || got[0].Version != "7.0.14-20" ||
|
||||
got[1].Name != "proxmox-kernel-7.0.14-20-pve-signed" {
|
||||
t.Fatalf("%+v", got)
|
||||
}
|
||||
if KernelSet("7.0; reboot") != nil || KernelSet("") != nil {
|
||||
t.Fatal("a non-kernel string must give no set")
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user