hub v0.133.0 (R-859 test approvals end with the test; R-840 bundle on the System page, manifest, alarm); installer 1.31.0 (root files from the config bundle); bundle bootstrap script; golden 0.293.0 evidence; rulings 96–99; drill-r50 removed (evidence)
gates / gates (push) Successful in 29s

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
This commit is contained in:
2026-10-04 20:18:37 +02:00
parent 6b820143f8
commit ff1db11db4
45 changed files with 1707 additions and 42 deletions
+22
View File
@@ -1,3 +1,25 @@
## v0.133.0 — test approvals end with the test (R-859); the root-file bundle on the System page, in the install manifest, and its alarm (R-840) (2026-10-04)
**Agent v0.143.0** reports the bundle; an older agent shows `unknown` in the new column (never a guess).
- **R-859 — test approvals end with the test (`11` §5.3.1).** Every OS approval made while a TEST override
(`OS_APPROVE_AFTER`, `OS_APPROVE_NIGHTS`, `OS_DOCKER_APPROVE_NIGHTS`) is active is stored with a `test` mark. At every
start WITHOUT an override, each test approval no real approval has superseded is CANCELLED: never served again (no
ring-1 box gets a plan from it), ring-1 boxes are bumped, one operator event `os_release_cancelled` each. What boxes
already installed stays. A one-time backfill marks the AUTOMATIC approvals made earlier than 24 h after their set was first
seen (the 2026-10-04 guest and host sets — approved 1.5 h after first seen). The operator's Docker button approval of
that day stays in force (a person approved it; decided by CC unattended — operator may reverse). The System page shows a test
approval in amber and lists the cancellations of the last 7 days. The same set is approved again by the ruled wait.
- **R-840 — the config bundle.** The Configuration vouch resolves the vouched agent's `felhom-config-bundle.json` sha
from the registry by EXACT name (`gitea.ExactFileSHA256` — never the binary's sha as a fallback); the install manifest
(`/api/v1/artifacts/<customer>`) serves it as `bundle`; the installer 1.31.0 installs exactly it. The System page has a
"Root files" column (the box's bundle; amber when behind the vouched one or changed by hand; red from 7 days); the
alarm `os_config_bundle_behind` (operator, warning) fires after `OS_ALARM_BUNDLE_BEHIND_AFTER` (default 7 d; decided by
CC unattended — operator may reverse). A box saying `none` counts as behind; `unknown` never does.
- `GetLatestHostReportJSON` breaks a same-second tie by id (two reports in one second picked an arbitrary one).
- Tests: `testapproval_test.go`, `bundle_alarm_test.go`, `system_bundle_test.go`, `gitea/exact_test.go`. Red-proofs:
`documentation/audits/r840-config-bundle-2026-10-04/partD/d-redproof.txt`, `.../partB/hub-bundle-redproof.txt`.
## v0.132.0 — the System page, the Docker engine release, the crash events (R-852, `09` decisions 87–89) (2026-10-04)
**Needs agent v0.142.0** for the versions, the Docker step and the crash guard; an older agent shows "no versions
+16
View File
@@ -10,6 +10,7 @@ import (
"os/signal"
"path/filepath"
"strconv"
"strings"
"syscall"
"time"
@@ -423,6 +424,20 @@ func main() {
logger.Printf("[ERROR] OS_DOCKER_APPROVE_NIGHTS=%q invalid — keeping 2", v)
}
}
// `11` §5.3.1 (hub v0.133.0): an approval made under ANY of the three TEST overrides carries the test mark, and a start
// without them cancels every test approval no real one superseded (no ring-1 box installs it from then on).
var overrides []string
for _, k := range []string{"OS_APPROVE_AFTER", "OS_APPROVE_NIGHTS", "OS_DOCKER_APPROVE_NIGHTS"} {
if v := os.Getenv(k); v != "" {
overrides = append(overrides, k+"="+v)
}
}
osSvc.TestOverride = strings.Join(overrides, " ")
if cancelled, cerr := osSvc.CancelTestReleases(); cerr != nil {
logger.Printf("[ERROR] osupdates: cancelling test approvals at start: %v (cancelled so far: %v)", cerr, cancelled)
} else if len(cancelled) > 0 {
logger.Printf("[WARN] osupdates: %d TEST approval(s) cancelled at start: %s", len(cancelled), strings.Join(cancelled, ", "))
}
logger.Printf("[INFO] osupdates: approval rule = every ring-0 box healthy for %s and %d night run(s)", osSvc.ApproveAfter, osSvc.NightsRequired)
logger.Printf("[INFO] osupdates: the Docker engine set is approved only by the operator, after %d healthy ring-0 night(s)", osSvc.DockerNights)
apiHandler.SetOSUpdateService(osSvc)
@@ -437,6 +452,7 @@ func main() {
{"OS_ALARM_REBOOT_AFTER", &osSvc.RebootAfter, 14 * 24 * time.Hour},
{"OS_ALARM_RING0_STALL_AFTER", &osSvc.Ring0StallAfter, 7 * 24 * time.Hour},
{"OS_ALARM_NOT_COVERED_AFTER", &osSvc.NotCoveredAfter, 14 * 24 * time.Hour},
{"OS_ALARM_BUNDLE_BEHIND_AFTER", &osSvc.BundleBehindAfter, 7 * 24 * time.Hour},
} {
*a.dst = a.def
if v := os.Getenv(a.env); v != "" {
+5
View File
@@ -2675,6 +2675,8 @@ func (h *Handler) handleConfigRetrieve(w http.ResponseWriter, r *http.Request, c
type artifactManifestResponse struct {
Agent artifactEntry `json:"agent"`
Golden artifactEntry `json:"golden"`
// Bundle is the vouched agent's config bundle (R-840); absent when the vouched agent carries none.
Bundle *artifactEntry `json:"bundle,omitempty"`
}
type artifactEntry struct {
@@ -2720,6 +2722,9 @@ func (h *Handler) handleArtifactManifest(w http.ResponseWriter, r *http.Request,
Agent: artifactEntry{Version: m.AgentVersion, SHA256: m.AgentSHA256},
Golden: artifactEntry{Version: m.GoldenVersion, SHA256: m.GoldenSHA256},
}
if m.BundleSHA256 != "" {
resp.Bundle = &artifactEntry{Version: m.AgentVersion, SHA256: m.BundleSHA256}
}
h.logger.Printf("[INFO] Artifact manifest served for customer %s (agent=%s golden=%s)", customerID, m.AgentVersion, m.GoldenVersion)
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusOK)
+34
View File
@@ -0,0 +1,34 @@
package gitea
import (
"context"
"net/http"
"net/http/httptest"
"testing"
)
// R-840: a version WITHOUT a config bundle must answer "" — never the binary's sha (FileSHA256's fallback would).
func TestExactFileSHA256_NeverFallsBack(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
switch r.URL.Path {
case "/api/v1/packages/admin/generic/felhom-agent/0.142.1/files":
w.Write([]byte(`[{"name":"felhom-agent","sha256":"aaaa"}]`))
case "/api/v1/packages/admin/generic/felhom-agent/0.143.0/files":
w.Write([]byte(`[{"name":"felhom-agent","sha256":"aaaa"},{"name":"felhom-config-bundle.json","sha256":"bbbb"}]`))
default:
http.NotFound(w, r)
}
}))
defer srv.Close()
c := New(srv.URL, "admin", "u", "t")
if got, err := c.ExactFileSHA256(context.Background(), "felhom-agent", "0.142.1", "felhom-config-bundle.json"); err != nil || got != "" {
t.Fatalf("no bundle: got %q %v, want \"\"", got, err)
}
if got, err := c.ExactFileSHA256(context.Background(), "felhom-agent", "0.143.0", "felhom-config-bundle.json"); err != nil || got != "bbbb" {
t.Fatalf("bundle: got %q %v", got, err)
}
if _, err := c.ExactFileSHA256(context.Background(), "felhom-agent", "9.9.9", "felhom-config-bundle.json"); err == nil {
t.Fatal("an unreadable listing must be an error, not \"no bundle\"")
}
}
+20
View File
@@ -107,6 +107,26 @@ func (c *Client) FileSHA256(ctx context.Context, pkgName, version, preferredFile
return files[0].SHA256, nil
}
// ExactFileSHA256 returns the sha256 of exactly `file` in the package version, or ("", nil) when the version has no
// such file. Unlike FileSHA256 it NEVER falls back to another file: the config bundle (R-840) must not inherit the
// binary's sha on a version that carries no bundle.
func (c *Client) ExactFileSHA256(ctx context.Context, pkgName, version, file string) (string, error) {
u := fmt.Sprintf("%s/api/v1/packages/%s/generic/%s/%s/files", c.baseURL, c.owner, pkgName, version)
var files []pkgFile
if err := c.getJSON(ctx, u, &files); err != nil {
return "", err
}
for _, f := range files {
if f.Name == file {
if f.SHA256 == "" {
return "", fmt.Errorf("no sha256 for %s/%s file %q", pkgName, version, file)
}
return f.SHA256, nil
}
}
return "", nil
}
func (c *Client) getJSON(ctx context.Context, url string, out interface{}) error {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
+8 -4
View File
@@ -682,10 +682,14 @@ var operatorOnlyEvents = map[string]bool{
"offsite_window_large_grant": true,
// OS updates (hub v0.130.0, `11` §8 step 2): run failures, rings, switches and approvals are operator facts.
// os_update_applied is deliberately NOT here — it is the household's one line (info: recorded, never mailed).
"os_update_failed": true,
"os_update_health_failed": true,
"os_release_approved": true,
"os_release_approved_now": true,
"os_update_failed": true,
"os_update_health_failed": true,
"os_release_approved": true,
"os_release_approved_now": true,
// hub v0.133.0 (`11` §5.3.1): a TEST approval cancelled at a start without the override.
"os_release_cancelled": true,
// R-840 (hub v0.133.0): a box's root-owned config bundle behind the vouched one for 7 days.
"os_config_bundle_behind": true,
"os_update_settings_changed": true,
// R-841 (hub v0.131.0): the tunnel alarm — a box fact the household can do nothing about from inside.
"tunnel_down": true,
@@ -0,0 +1,90 @@
package osupdates
import (
"testing"
"time"
"gitea.dooplex.hu/admin/felhom-hub/internal/store"
)
func bundleReport(t *testing.T, f *fix, host, version, sha string) {
t.Helper()
body := `{"system":{"pve_version":"pve-manager/9.2.2/x","config_bundle":{"version":"` + version + `","bundle_sha256":"` + sha + `"}}}`
h, err := f.s.Store.GetHost(host)
if err != nil || h == nil {
t.Fatalf("no host %s", host)
}
if err := f.s.Store.SaveHostReport(host, h.CustomerID, []byte(body), store.HostReportDenorm{AgentVersion: "0.143.0"}); err != nil {
t.Fatal(err)
}
}
func vouch(t *testing.T, f *fix, sha string) {
t.Helper()
if err := f.s.Store.SetArtifactManifest(store.ArtifactManifest{AgentVersion: "0.143.0", AgentSHA256: "x", BundleSHA256: sha}); err != nil {
t.Fatal(err)
}
}
func count(sent []string, typ string) int {
n := 0
for _, s := range sent {
if s == typ {
n++
}
}
return n
}
// R-840: a box behind the vouched bundle is told to the operator after 7 days — not before, once, and the clock
// restarts when the box catches up. "none" (no bundle ever) counts; "unknown" never does.
func TestBundleAlarm_AfterSevenDaysBehind(t *testing.T) {
f := newFix(t)
vouch(t, f, "new")
bundleReport(t, f, "cust1", "none", "")
sent, _ := f.s.Alarms()
if count(sent, EventBundleBehind) != 0 {
t.Fatal("alarm on the first sight")
}
f.now = f.now.Add(6 * 24 * time.Hour)
if sent, _ = f.s.Alarms(); count(sent, EventBundleBehind) != 0 {
t.Fatal("alarm before 7 days")
}
f.now = f.now.Add(25 * time.Hour)
if sent, _ = f.s.Alarms(); count(sent, EventBundleBehind) != 1 {
t.Fatalf("no alarm after 7 days: %v", sent)
}
if sent, _ = f.s.Alarms(); count(sent, EventBundleBehind) != 0 {
t.Fatal("the alarm must not repeat at once")
}
bundleReport(t, f, "cust1", "0.143.0", "new")
f.s.Alarms()
if !f.s.Store.BundleBehindSince("cust1").IsZero() {
t.Fatal("caught up: the clock must clear")
}
}
func TestBundleAlarm_UnknownAndNoVouchedBundleSayNothing(t *testing.T) {
f := newFix(t)
vouch(t, f, "new")
bundleReport(t, f, "cust1", "unknown", "")
f.s.Alarms()
f.now = f.now.Add(30 * 24 * time.Hour)
if sent, _ := f.s.Alarms(); count(sent, EventBundleBehind) != 0 || !f.s.Store.BundleBehindSince("cust1").IsZero() {
t.Fatalf("unknown is not a fact: %v", sent)
}
// control: the same box saying "none" IS behind (proves the report above was read at all)
bundleReport(t, f, "cust1", "none", "")
f.s.Alarms()
if f.s.Store.BundleBehindSince("cust1").IsZero() {
t.Fatal("control: a box saying none must start the clock")
}
g := newFix(t)
vouch(t, g, "")
bundleReport(t, g, "cust1", "0.143.0", "some-bundle")
g.s.Alarms()
g.now = g.now.Add(30 * 24 * time.Hour)
if sent, _ := g.s.Alarms(); count(sent, EventBundleBehind) != 0 {
t.Fatalf("nothing vouched, nothing behind: %v", sent)
}
}
+114 -8
View File
@@ -29,6 +29,7 @@ import (
"gitea.dooplex.hu/admin/felhom-hub/internal/semver"
"gitea.dooplex.hu/admin/felhom-hub/internal/store"
"gitea.dooplex.hu/admin/felhom-hub/internal/sysfacts"
)
// Layers.
@@ -68,6 +69,10 @@ const (
EventRebootNeeded = "os_reboot_needed" // warning, operator: reboot needed for RebootAfter
EventRing0Stalled = "os_ring0_stalled" // error, operator: ring 0 approved nothing for Ring0StallAfter
EventNotCovered = "os_not_covered" // warning, operator: fast-lane packages no release covers
// EventCancelled: a TEST approval was cancelled because the hub started without the TEST override (`11` §5.3.1).
EventCancelled = "os_release_cancelled" // warning, operator
// EventBundleBehind: a box's root-owned config bundle has differed from the vouched one for BundleBehindAfter (R-840).
EventBundleBehind = "os_config_bundle_behind" // warning, operator
)
// Package is one name=version with its origin ("Debian" | "Debian-Security").
@@ -165,9 +170,15 @@ type Service struct {
RebootAfter time.Duration // 14 d
Ring0StallAfter time.Duration // 7 d
NotCoveredAfter time.Duration // 14 d
Logger *log.Logger
Now func() time.Time
Bump func(hostID string)
// BundleBehindAfter: a box's config bundle differs from the vouched one this long → an operator alarm (R-840;
// decided by CC unattended — operator may reverse). Zero = 7 d.
BundleBehindAfter time.Duration
Logger *log.Logger
Now func() time.Time
Bump func(hostID string)
// TestOverride names the TEST overrides active at start ("" = none, the ruled waits). Every approval made while it
// is set carries the `test` mark; CancelTestReleases cancels them at a start without it (`11` §5.3.1).
TestOverride string
}
func (s *Service) now() time.Time {
@@ -427,6 +438,8 @@ type ReleaseInfo struct {
ApprovedAt time.Time
ApprovedBy string
Packages int
Test bool // a TEST approval still in force: amber on the System page
Cancelled string // set on a cancelled one (the page lists the last 7 days')
}
// Releases lists the newest release of every layer (guest, host, Docker); a layer with none is absent.
@@ -439,7 +452,20 @@ func (s *Service) Releases() []ReleaseInfo {
}
var list []Package
_ = json.Unmarshal([]byte(rel.PackagesJSON), &list)
out = append(out, ReleaseInfo{Layer: layer, ID: rel.ID, ApprovedAt: rel.ApprovedAt, ApprovedBy: rel.ApprovedBy, Packages: len(list)})
out = append(out, ReleaseInfo{Layer: layer, ID: rel.ID, ApprovedAt: rel.ApprovedAt, ApprovedBy: rel.ApprovedBy, Packages: len(list), Test: rel.Test})
}
return out
}
// CancelledReleases lists the approvals cancelled in the last 7 days (the System page says what stopped being served).
func (s *Service) CancelledReleases() []ReleaseInfo {
rels, _ := s.Store.CancelledOSReleasesSince(s.now().Add(-7 * 24 * time.Hour))
var out []ReleaseInfo
for _, r := range rels {
var list []Package
_ = json.Unmarshal([]byte(r.PackagesJSON), &list)
out = append(out, ReleaseInfo{Layer: r.Layer, ID: r.ID, ApprovedAt: r.ApprovedAt, ApprovedBy: r.ApprovedBy, Packages: len(list),
Test: r.Test, Cancelled: r.CancelledAt})
}
return out
}
@@ -455,6 +481,9 @@ func (s *Service) Candidates() []Status {
return append(out, d)
}
// BundleThreshold is the config-bundle alarm's wait (the System page turns the cell red at it).
func (s *Service) BundleThreshold() time.Duration { return dflt(s.BundleBehindAfter, 7*24*time.Hour) }
// Thresholds are the alarm numbers the System page colours by (the same values the alarms use).
func (s *Service) Thresholds() (stale, reboot, notCovered time.Duration) {
return dflt(s.StaleAfter, 7*24*time.Hour), dflt(s.RebootAfter, 14*24*time.Hour), dflt(s.NotCoveredAfter, 14*24*time.Hour)
@@ -577,12 +606,18 @@ func (s *Service) approve(layer, fp string, list []Package, by string) error {
at := s.now().UTC().Truncate(time.Second)
id := "os-" + layer + "-" + at.Format("20060102-150405")
pj, _ := json.Marshal(list)
if err := s.Store.SaveOSRelease(store.OSRelease{ID: id, Layer: layer, Fingerprint: fp, ApprovedAt: at, ApprovedBy: by, PackagesJSON: string(pj)}); err != nil {
test := s.TestOverride != ""
if err := s.Store.SaveOSRelease(store.OSRelease{ID: id, Layer: layer, Fingerprint: fp, ApprovedAt: at, ApprovedBy: by,
PackagesJSON: string(pj), Test: test}); err != nil {
return err
}
s.logf("[WARN] osupdates: OS release %s (%s) APPROVED by %s (%d packages, fingerprint %s)", id, layer, by, len(list), fp)
s.event("", EventReleaseApprove, "info", fmt.Sprintf("OS release %s (%s) approved (%s, %d packages).", id, layer, by, len(list)),
map[string]any{"release_id": id, "layer": layer, "approved_by": by, "packages": len(list), "fingerprint": fp})
mark := ""
if test {
mark = " — TEST approval (" + s.TestOverride + "); cancelled when the hub starts without the override"
}
s.logf("[WARN] osupdates: OS release %s (%s) APPROVED by %s (%d packages, fingerprint %s)%s", id, layer, by, len(list), fp, mark)
s.event("", EventReleaseApprove, "info", fmt.Sprintf("OS release %s (%s) approved (%s, %d packages)%s.", id, layer, by, len(list), mark),
map[string]any{"release_id": id, "layer": layer, "approved_by": by, "packages": len(list), "fingerprint": fp, "test": test})
if s.Bump != nil && layer != LayerDocker { // a Docker set reaches ring 1 only by a signed job, not the desired state
hosts, _ := s.Store.ListHosts()
for _, h := range hosts {
@@ -594,6 +629,45 @@ func (s *Service) approve(layer, fp string, list []Package, by string) error {
return nil
}
// CancelTestReleases runs at every hub start. Without a TEST override it cancels every test-marked approval that no
// real (non-test) approval has superseded: no ring-1 box installs it from then on; what boxes already installed stays.
// Each cancellation is an operator event; ring-1 boxes are bumped so their next plan has no cancelled release. With the
// override still active it does nothing (the test is still running). Returns the cancelled ids.
func (s *Service) CancelTestReleases() ([]string, error) {
if s.TestOverride != "" {
s.logf("[INFO] osupdates: TEST override active (%s) — test approvals stay in force", s.TestOverride)
return nil, nil
}
var ids []string
for _, layer := range AllLayers {
rels, err := s.Store.UnsupersededTestReleases(layer)
if err != nil {
return ids, err
}
for _, r := range rels {
reason := "approved under a TEST override; the hub started without it"
if err := s.Store.CancelOSRelease(r.ID, reason, s.now()); err != nil {
return ids, err
}
ids = append(ids, r.ID)
s.logf("[WARN] osupdates: TEST approval %s (%s, approved %s by %s) CANCELLED — no ring-1 box installs it from now on",
r.ID, layer, r.ApprovedAt.UTC().Format(time.RFC3339), r.ApprovedBy)
s.event("", EventCancelled, "warning", fmt.Sprintf("OS release %s (%s) was a TEST approval and is cancelled: "+
"no further box installs it. Boxes that already installed it keep it.", r.ID, layer),
map[string]any{"release_id": r.ID, "layer": layer, "approved_at": r.ApprovedAt.UTC().Format(time.RFC3339), "approved_by": r.ApprovedBy})
}
}
if len(ids) > 0 && s.Bump != nil {
hosts, _ := s.Store.ListHosts()
for _, h := range hosts {
if st := s.Store.GetOSHostSettings(h.HostID); st.Ring == 1 && st.Enabled {
s.Bump(h.HostID)
}
}
}
return ids, nil
}
func (s *Service) releaseBlock(layer string) *ReleaseBlock {
rel, err := s.Store.LatestOSRelease(layer)
if err != nil || rel == nil {
@@ -860,6 +934,38 @@ func (s *Service) Alarms() ([]string, error) {
}
}
}
// 5. R-840: the box's ROOT-OWNED config bundle (sudoers, wrappers, units) differs from the vouched agent's for
// BundleBehindAfter. "none" (no bundle ever reached the box) counts as behind; "unknown" (the box could not say)
// counts as nothing — never a guess. Only when the vouched agent carries a bundle at all.
man := s.Store.GetArtifactManifest()
for _, h := range hosts {
if man.BundleSHA256 == "" {
break
}
rj, _ := s.Store.GetLatestHostReportJSON(h.CustomerID)
sys := sysfacts.Parse(rj)
if !sys.Present || sys.Bundle.Version == sysfacts.Unknown {
continue
}
behind := sys.Bundle.BundleSHA256 != man.BundleSHA256
since := s.Store.BundleBehindSince(h.HostID)
switch {
case !behind && !since.IsZero():
_ = s.Store.SetBundleBehindSince(h.HostID, time.Time{})
since = time.Time{}
case behind && since.IsZero():
since = now
_ = s.Store.SetBundleBehindSince(h.HostID, since)
}
after := dflt(s.BundleBehindAfter, 7*24*time.Hour)
if s.raise("bundle:"+h.HostID, behind && now.Sub(since) >= after, h.CustomerID, EventBundleBehind, "warning",
fmt.Sprintf("Root files: %s still runs config bundle %s; the vouched agent %s carries a newer one (since %s). "+
"Send it with a signed agent_config_update (`11` §5.4.2).", h.HostID, sys.Bundle.Version, man.AgentVersion,
since.UTC().Format("2006-01-02")),
map[string]any{"host_id": h.HostID, "box_bundle": sys.Bundle.Version, "vouched_agent": man.AgentVersion, "since": since}) {
sent = append(sent, EventBundleBehind)
}
}
// 3. Ring 0 approved nothing for `stall` while ring 0 has pending fast-lane updates: the whole fleet stopped
// getting fixes.
ring0, _ := s.ring0Hosts()
+173
View File
@@ -0,0 +1,173 @@
package osupdates
import (
"database/sql"
"log"
"os"
"path/filepath"
"testing"
"time"
"gitea.dooplex.hu/admin/felhom-hub/internal/store"
_ "modernc.org/sqlite"
)
// `11` §5.3.1 (hub v0.133.0): test approvals end with the test.
// approveUnderTest makes ring 0 run a set and approves it at once under a TEST override (OS_APPROVE_AFTER=0 shape).
func approveUnderTest(t *testing.T, f *fix, override string, set ...Package) string {
t.Helper()
f.s.TestOverride = override
f.s.ApproveAfter, f.s.NightsRequired = 0, 0
f.report(t, "hp", "debug", true, set...)
f.report(t, "n100", "debug", true, set...)
if _, err := f.s.Evaluate(); err != nil {
t.Fatal(err)
}
rel, _ := f.s.Store.LatestOSRelease(LayerGuest)
if rel == nil {
t.Fatal("not approved")
}
return rel.ID
}
// An approval made under the override carries the mark; one made without it does not.
func TestTestApproval_IsMarked(t *testing.T) {
f := newFix(t)
approveUnderTest(t, f, "OS_APPROVE_AFTER=0s", pk("libc6", "u4"))
rel, _ := f.s.Store.LatestOSRelease(LayerGuest)
if !rel.Test {
t.Fatalf("an approval under a TEST override must be marked: %+v", rel)
}
if info := f.s.Releases(); len(info) != 1 || !info[0].Test {
t.Fatalf("the System page must see the mark: %+v", info)
}
f.s.TestOverride = ""
f.now = f.now.Add(time.Hour)
f.report(t, "hp", "debug", true, pk("libc6", "u5"))
f.report(t, "n100", "debug", true, pk("libc6", "u5"))
f.s.Evaluate()
rel, _ = f.s.Store.LatestOSRelease(LayerGuest)
if rel.Test {
t.Fatalf("an approval without the override must not be marked: %+v", rel)
}
}
// The consequence: after a restart without the override, a ring-1 box gets NO plan from the test approval; the
// cancellation is an operator event and ring-1 boxes are bumped.
func TestTestApproval_CancelledAtAStartWithoutTheOverride(t *testing.T) {
f := newFix(t)
id := approveUnderTest(t, f, "OS_APPROVE_AFTER=0s", pk("libc6", "u4"))
if b := f.s.DesiredBlock("cust1"); b.Release == nil || b.Release.ID != id {
t.Fatalf("before the restart the ring-1 box is served the release: %+v", b)
}
f.events, f.bumps = nil, nil
f.s.TestOverride = "" // the hub restarts without the override
ids, err := f.s.CancelTestReleases()
if err != nil || len(ids) != 1 || ids[0] != id {
t.Fatalf("cancelled %v, %v", ids, err)
}
if b := f.s.DesiredBlock("cust1"); b.Release != nil {
t.Fatalf("a ring-1 box must get no plan from a cancelled test approval: %+v", b.Release)
}
if len(f.events) != 1 || f.events[0] != EventCancelled {
t.Fatalf("events = %v", f.events)
}
if len(f.bumps) != 1 || f.bumps[0] != "cust1" {
t.Fatalf("ring-1 boxes must be bumped: %v", f.bumps)
}
if c := f.s.CancelledReleases(); len(c) != 1 || c[0].ID != id {
t.Fatalf("the page must list the cancellation: %+v", c)
}
// once is enough: a second start cancels nothing more
if again, _ := f.s.CancelTestReleases(); len(again) != 0 {
t.Fatalf("second start cancelled %v", again)
}
}
func TestTestApproval_StaysWhileTheOverrideIsStillOn(t *testing.T) {
f := newFix(t)
approveUnderTest(t, f, "OS_APPROVE_AFTER=0s", pk("libc6", "u4"))
if ids, _ := f.s.CancelTestReleases(); len(ids) != 0 {
t.Fatalf("the test is still running; nothing to cancel: %v", ids)
}
if b := f.s.DesiredBlock("cust1"); b.Release == nil {
t.Fatal("still served while the override is on")
}
}
// A test approval that a REAL approval has superseded is history, not cancelled; the real one stays served.
func TestTestApproval_SupersededIsLeftAlone(t *testing.T) {
f := newFix(t)
old := approveUnderTest(t, f, "OS_APPROVE_AFTER=0s", pk("libc6", "u4"))
f.s.TestOverride = ""
f.now = f.now.Add(time.Hour)
f.report(t, "hp", "debug", true, pk("libc6", "u5"))
f.report(t, "n100", "debug", true, pk("libc6", "u5"))
f.s.Evaluate()
real, _ := f.s.Store.LatestOSRelease(LayerGuest)
if real.ID == old || real.Test {
t.Fatalf("setup: %+v", real)
}
if ids, _ := f.s.CancelTestReleases(); len(ids) != 0 {
t.Fatalf("a superseded test approval must not be cancelled: %v", ids)
}
if b := f.s.DesiredBlock("cust1"); b.Release == nil || b.Release.ID != real.ID {
t.Fatalf("the real release stays served: %+v", b.Release)
}
}
// After a cancellation the SAME set is approved again by the ruled wait (a real release) — the cancel is not a ban.
func TestTestApproval_TheSetIsApprovedAgainByTheRuledWait(t *testing.T) {
f := newFix(t)
set := []Package{pk("libc6", "u4")}
approveUnderTest(t, f, "OS_APPROVE_AFTER=0s", set...)
f.s.TestOverride = ""
f.s.ApproveAfter, f.s.NightsRequired = 24*time.Hour, 1
f.s.CancelTestReleases()
f.now = f.now.Add(25 * time.Hour)
f.report(t, "hp", "night", true, set...)
f.report(t, "n100", "night", true, set...)
f.s.Evaluate()
rel, _ := f.s.Store.LatestOSRelease(LayerGuest)
if rel == nil || rel.Test {
t.Fatalf("the ruled wait must approve the set again, unmarked: %+v", rel)
}
}
// The one-time backfill: on a database from before the mark, an approval earlier than 24 h after its set was first
// seen (the 2026-10-04 shape: 1.5 h) is marked test; one after the ruled wait is not.
func TestTestApproval_BackfillMarksTheEarlyApprovals(t *testing.T) {
path := filepath.Join(t.TempDir(), "hub.db")
db, err := sql.Open("sqlite", path)
if err != nil {
t.Fatal(err)
}
for _, q := range []string{
`CREATE TABLE os_candidates (fingerprint TEXT PRIMARY KEY, first_seen DATETIME NOT NULL, packages_json TEXT NOT NULL)`,
`CREATE TABLE os_releases (id TEXT PRIMARY KEY, fingerprint TEXT NOT NULL, approved_at DATETIME NOT NULL, approved_by TEXT NOT NULL, packages_json TEXT NOT NULL, layer TEXT NOT NULL DEFAULT 'guest')`,
`INSERT INTO os_candidates VALUES ('fpA', '2026-10-04 11:07:00', '[]'), ('fpB', '2026-10-02 08:00:00', '[]')`,
`INSERT INTO os_candidates VALUES ('fpD', '2026-10-04 14:28:00', '[]')`,
`INSERT INTO os_releases VALUES ('os-guest-early', 'fpA', '2026-10-04 12:39:33', 'auto', '[]', 'guest'),
('os-guest-ruled', 'fpB', '2026-10-03 09:00:00', 'auto', '[]', 'guest'),
('os-docker-button', 'fpD', '2026-10-04 14:28:42', 'operator', '[]', 'docker')`,
} {
if _, err := db.Exec(q); err != nil {
t.Fatal(err)
}
}
db.Close()
st, err := store.New(path, log.New(os.Stderr, "", 0))
if err != nil {
t.Fatal(err)
}
defer st.Close()
rels, _ := st.UnsupersededTestReleases(LayerGuest)
if len(rels) != 1 || rels[0].ID != "os-guest-early" {
t.Fatalf("backfill: unsuperseded test releases = %+v", rels)
}
// the operator's own button approval is not backfilled (a person approved it)
if d, _ := st.UnsupersededTestReleases(LayerDocker); len(d) != 0 {
t.Fatalf("backfill marked the operator's Docker approval: %+v", d)
}
}
+103 -9
View File
@@ -53,9 +53,42 @@ func (s *Store) migrateOSUpdates() error {
// Host fast lane (hub v0.131.0): every report and release belongs to a LAYER; old rows are the guest's.
s.db.Exec(`ALTER TABLE os_reports ADD COLUMN layer TEXT NOT NULL DEFAULT 'guest'`)
s.db.Exec(`ALTER TABLE os_releases ADD COLUMN layer TEXT NOT NULL DEFAULT 'guest'`)
// Test approvals end with the test (hub v0.133.0, `11` §5.3.1): an approval made while a TEST override is active
// carries `test`; a start without the override CANCELS every test approval no real one has superseded.
if !s.hasColumn("os_releases", "test") {
if _, err := s.db.Exec(`ALTER TABLE os_releases ADD COLUMN test INTEGER NOT NULL DEFAULT 0`); err != nil {
return err
}
// One-time backfill, from the data: an AUTOMATIC approval earlier than 24 h after its set was first seen
// cannot have passed the ruled wait (24 h + one night) — it was a TEST approval (2026-10-04: the guest and host
// sets, approved 1.5 h after first seen under OS_APPROVE_*). The operator's own Docker button approval of that day
// is NOT backfilled: a person approved it; the override only shortened its precondition (decided by CC
// unattended — operator may reverse). From hub v0.133.0 on, every approval under an override is marked.
s.db.Exec(`UPDATE os_releases SET test = 1 WHERE approved_by = 'auto' AND EXISTS (SELECT 1 FROM os_candidates c
WHERE c.fingerprint = os_releases.fingerprint
AND (julianday(os_releases.approved_at) - julianday(c.first_seen)) * 24 < 24)`)
}
s.db.Exec(`ALTER TABLE os_releases ADD COLUMN cancelled_at TEXT NOT NULL DEFAULT ''`)
s.db.Exec(`ALTER TABLE os_releases ADD COLUMN cancel_reason TEXT NOT NULL DEFAULT ''`)
return nil
}
// hasColumn reports whether table has the column (PRAGMA table_info).
func (s *Store) hasColumn(table, col string) bool {
rows, err := s.db.Query(`SELECT name FROM pragma_table_info(?)`, table)
if err != nil {
return false
}
defer rows.Close()
for rows.Next() {
var n string
if rows.Scan(&n) == nil && n == col {
return true
}
}
return false
}
// OSHostSettings is one box's ring and switch.
type OSHostSettings struct {
HostID string
@@ -191,6 +224,22 @@ type OSRelease struct {
ApprovedAt time.Time
ApprovedBy string
PackagesJSON string
Test bool // approved while a TEST override was active (`11` §5.3.1)
CancelledAt string // "" = in force; a cancelled release is never served (LatestOSRelease skips it)
CancelReason string
}
const osReleaseCols = `id, layer, fingerprint, approved_at, approved_by, packages_json, test, cancelled_at, cancel_reason`
func scanOSRelease(sc interface{ Scan(...any) error }) (*OSRelease, error) {
var r OSRelease
var at string
var test int
if err := sc.Scan(&r.ID, &r.Layer, &r.Fingerprint, &at, &r.ApprovedBy, &r.PackagesJSON, &test, &r.CancelledAt, &r.CancelReason); err != nil {
return nil, err
}
r.ApprovedAt, r.Test = parseSQLiteTime(at), test == 1
return &r, nil
}
// SaveOSRelease stores an approved release.
@@ -199,25 +248,70 @@ func (s *Store) SaveOSRelease(r OSRelease) error {
if layer == "" {
layer = "guest"
}
_, err := s.db.Exec(`INSERT INTO os_releases (id, layer, fingerprint, approved_at, approved_by, packages_json) VALUES (?, ?, ?, ?, ?, ?)`,
r.ID, layer, r.Fingerprint, r.ApprovedAt.UTC().Format("2006-01-02 15:04:05"), r.ApprovedBy, r.PackagesJSON)
test := 0
if r.Test {
test = 1
}
_, err := s.db.Exec(`INSERT INTO os_releases (id, layer, fingerprint, approved_at, approved_by, packages_json, test) VALUES (?, ?, ?, ?, ?, ?, ?)`,
r.ID, layer, r.Fingerprint, r.ApprovedAt.UTC().Format("2006-01-02 15:04:05"), r.ApprovedBy, r.PackagesJSON, test)
return err
}
// LatestOSRelease returns the newest approved release of a layer, or nil.
// LatestOSRelease returns the newest approved release of a layer that is IN FORCE (not cancelled), or nil.
func (s *Store) LatestOSRelease(layer string) (*OSRelease, error) {
var r OSRelease
var at string
err := s.db.QueryRow(`SELECT id, layer, fingerprint, approved_at, approved_by, packages_json FROM os_releases WHERE layer = ? ORDER BY approved_at DESC, id DESC LIMIT 1`, layer).
Scan(&r.ID, &r.Layer, &r.Fingerprint, &at, &r.ApprovedBy, &r.PackagesJSON)
r, err := scanOSRelease(s.db.QueryRow(`SELECT `+osReleaseCols+` FROM os_releases WHERE layer = ? AND cancelled_at = ''
ORDER BY approved_at DESC, id DESC LIMIT 1`, layer))
if err == sql.ErrNoRows {
return nil, nil
}
return r, err
}
// UnsupersededTestReleases returns a layer's test releases still in force and newer than its newest REAL (non-test,
// in-force) release — the ones a start without the TEST override must cancel. Newest first.
func (s *Store) UnsupersededTestReleases(layer string) ([]OSRelease, error) {
rows, err := s.db.Query(`SELECT `+osReleaseCols+` FROM os_releases r WHERE r.layer = ? AND r.test = 1 AND r.cancelled_at = ''
AND r.approved_at >= COALESCE((SELECT MAX(approved_at) FROM os_releases WHERE layer = r.layer AND test = 0 AND cancelled_at = ''), '')
ORDER BY r.approved_at DESC, r.id DESC`, layer)
if err != nil {
return nil, err
}
r.ApprovedAt = parseSQLiteTime(at)
return &r, nil
defer rows.Close()
var out []OSRelease
for rows.Next() {
r, err := scanOSRelease(rows)
if err != nil {
return nil, err
}
out = append(out, *r)
}
return out, rows.Err()
}
// CancelOSRelease marks one release cancelled (it is never served again; its row stays as the record).
func (s *Store) CancelOSRelease(id, reason string, at time.Time) error {
_, err := s.db.Exec(`UPDATE os_releases SET cancelled_at = ?, cancel_reason = ? WHERE id = ? AND cancelled_at = ''`,
at.UTC().Format("2006-01-02 15:04:05"), reason, id)
return err
}
// CancelledOSReleasesSince lists releases cancelled at or after t (the System page shows them), newest first.
func (s *Store) CancelledOSReleasesSince(t time.Time) ([]OSRelease, error) {
rows, err := s.db.Query(`SELECT `+osReleaseCols+` FROM os_releases WHERE cancelled_at != '' AND cancelled_at >= ?
ORDER BY cancelled_at DESC, id DESC`, t.UTC().Format("2006-01-02 15:04:05"))
if err != nil {
return nil, err
}
defer rows.Close()
var out []OSRelease
for rows.Next() {
r, err := scanOSRelease(rows)
if err != nil {
return nil, err
}
out = append(out, *r)
}
return out, rows.Err()
}
// BackdateOSCandidateForTest moves a candidate's first_seen into the past. TEST-ONLY.
+31 -2
View File
@@ -2118,6 +2118,10 @@ type ArtifactManifest struct {
// Recording the hash here does not fix the delivery channel (that is R-50b(b)/(c)) — it makes
// DRIFT VISIBLE, which is the cheap honest first step.
WrapperSHA256 string `json:"wrapper_sha256"`
// BundleSHA256 is the vouched agent version's CONFIG BUNDLE (R-840): every root-owned file the installer writes,
// published beside the binary as felhom-agent/<AgentVersion>/felhom-config-bundle.json. "" = the vouched agent
// carries none (older than v0.143.0). The installer installs exactly this; boxes behind it raise an alarm.
BundleSHA256 string `json:"bundle_sha256"`
}
// hub_settings keys for the artifact manifest (BUNDLE slice). Stored as discrete key/value rows in
@@ -2130,6 +2134,7 @@ const (
settingArtifactGoldenSHA256 = "artifact_golden_sha256"
settingArtifactMinAgent = "artifact_min_agent"
settingArtifactWrapperSHA256 = "artifact_wrapper_sha256" // R-50b(a): the vouched felhom-pbs-apply hash
settingArtifactBundleSHA256 = "artifact_bundle_sha256" // R-840: the vouched agent's config bundle
)
// settingOperatorPasswordHash is the hub_settings key for the operator login password bcrypt hash,
@@ -2178,6 +2183,7 @@ func (s *Store) GetArtifactManifest() ArtifactManifest {
GoldenSHA256: s.getSetting(settingArtifactGoldenSHA256),
MinAgent: s.getSetting(settingArtifactMinAgent),
WrapperSHA256: s.getSetting(settingArtifactWrapperSHA256),
BundleSHA256: s.getSetting(settingArtifactBundleSHA256),
}
}
@@ -2199,7 +2205,30 @@ func (s *Store) SetArtifactManifest(m ArtifactManifest) error {
if err := s.setSetting(settingArtifactMinAgent, m.MinAgent); err != nil {
return err
}
return s.setSetting(settingArtifactWrapperSHA256, m.WrapperSHA256)
if err := s.setSetting(settingArtifactWrapperSHA256, m.WrapperSHA256); err != nil {
return err
}
return s.setSetting(settingArtifactBundleSHA256, m.BundleSHA256)
}
// BundleBehindSince returns since when a box's config bundle has differed from the vouched one (zero = it matches, or
// was never seen behind). R-840's 7-day alarm counts from here.
func (s *Store) BundleBehindSince(hostID string) time.Time {
v := s.getSetting("bundle_behind_since:" + hostID)
if v == "" {
return time.Time{}
}
t, _ := time.Parse(time.RFC3339, v)
return t
}
// SetBundleBehindSince records (or, with a zero time, clears) the first moment a box was seen behind.
func (s *Store) SetBundleBehindSince(hostID string, t time.Time) error {
v := ""
if !t.IsZero() {
v = t.UTC().Format(time.RFC3339)
}
return s.setSetting("bundle_behind_since:"+hostID, v)
}
// EffectiveMinControllerVersion resolves the floor that actually applies to a customer: the
@@ -3487,7 +3516,7 @@ func (s *Store) SaveHostReport(hostID, customerID string, reportJSON []byte, d H
func (s *Store) GetLatestHostReportJSON(customerID string) (string, error) {
var j string
err := s.db.QueryRow(
`SELECT report_json FROM host_reports WHERE customer_id = ? ORDER BY received_at DESC LIMIT 1`,
`SELECT report_json FROM host_reports WHERE customer_id = ? ORDER BY received_at DESC, id DESC LIMIT 1`,
customerID,
).Scan(&j)
if err == sql.ErrNoRows {
+27 -2
View File
@@ -50,6 +50,15 @@ type Guest struct {
UnknownReason string `json:"unknown_reason"`
}
// ConfigBundle is the box's root-owned config bundle (R-840, agent v0.143.0): the agent's own read of the record
// (version "none" = no bundle ever reached the box), with the drift the wrapper's facts add (files changed by hand).
type ConfigBundle struct {
Version string `json:"version"` // agent version of the bundle | none | unknown
BundleSHA256 string `json:"bundle_sha256"`
InstalledAt string `json:"installed_at"`
Drift []string `json:"drift"`
}
// System is the whole stanza. Present is false for a report from an agent older than v0.142.0.
type System struct {
Present bool
@@ -60,6 +69,7 @@ type System struct {
ReadAt string
Host Host
Guest Guest
Bundle ConfigBundle
}
type wire struct {
@@ -70,6 +80,7 @@ type wire struct {
Facts json.RawMessage `json:"facts"`
FactsError string `json:"facts_error"`
ReadAt string `json:"read_at"`
ConfigBundle *ConfigBundle `json:"config_bundle"`
} `json:"system"`
}
@@ -93,11 +104,25 @@ func Parse(reportJSON string) System {
out.PVEVersion, out.KernelVersion = orUnknown(w.System.PVEVersion), orUnknown(w.System.KernelVersion)
out.VMID, out.FactsError, out.ReadAt = w.System.VMID, w.System.FactsError, w.System.ReadAt
var f struct {
Host Host `json:"host"`
Host struct {
Host
ConfigBundle *ConfigBundle `json:"config_bundle"`
} `json:"host"`
Guest Guest `json:"guest"`
}
out.Bundle = ConfigBundle{Version: Unknown}
if w.System.ConfigBundle != nil && w.System.ConfigBundle.Version != "" {
out.Bundle = *w.System.ConfigBundle
}
if len(w.System.Facts) > 0 && json.Unmarshal(w.System.Facts, &f) == nil {
out.Host, out.Guest = f.Host, f.Guest
out.Host, out.Guest = f.Host.Host, f.Guest
// The wrapper's view adds the drift; its record is the same file the agent read.
if fb := f.Host.ConfigBundle; fb != nil && fb.Version != "" && fb.Version != Unknown {
if out.Bundle.Version == Unknown {
out.Bundle = *fb
}
out.Bundle.Drift = fb.Drift
}
}
out.Host.Debian, out.Host.KernelRunning = orUnknown(out.Host.Debian), orUnknown(out.Host.KernelRunning)
out.Host.KernelNextBoot = orUnknown(out.Host.KernelNextBoot)
+14 -1
View File
@@ -1320,6 +1320,18 @@ func (s *Server) handleSetArtifacts(w http.ResponseWriter, r *http.Request) {
http.Redirect(w, r, "/configuration?flash=artifact_sha_invalid", http.StatusSeeOther)
return
}
// R-840: the vouched agent's CONFIG BUNDLE is resolved from the registry by exact name, like the binary. A version
// without one (older than v0.143.0) vouches none — the installer then falls back to its per-file fetches.
bundleSHA := ""
if agentVer != "" && s.gitea != nil {
b, berr := s.gitea.ExactFileSHA256(r.Context(), pkgAgent, agentVer, fileBundle)
if berr != nil {
s.logger.Printf("[WARN] artifact vouch REFUSED: could not read agent %s's config bundle sha: %v", agentVer, berr)
http.Redirect(w, r, "/configuration?flash=artifact_unverifiable", http.StatusSeeOther)
return
}
bundleSHA = b
}
// R-50b(a): the PBS-DR wrapper hash is operator-typed, not resolved from the package registry —
// unlike the agent binary and the golden, this artifact is not published there at all. It is
// installed from raw/branch/main, which is exactly the drift this field makes visible.
@@ -1366,12 +1378,13 @@ func (s *Server) handleSetArtifacts(w http.ResponseWriter, r *http.Request) {
GoldenSHA256: goldenSHA,
MinAgent: minAgent,
WrapperSHA256: wrapperSHA,
BundleSHA256: bundleSHA,
}); err != nil {
s.logger.Printf("[ERROR] Failed to set artifact manifest: %v", err)
http.Error(w, "Internal error", http.StatusInternalServerError)
return
}
s.logger.Printf("[INFO] Artifact manifest set: agent=%s golden=%s min_agent=%q wrapper_sha=%t", agentVer, goldenVer, minAgent, wrapperSHA != "")
s.logger.Printf("[INFO] Artifact manifest set: agent=%s golden=%s min_agent=%q wrapper_sha=%t bundle_sha=%q", agentVer, goldenVer, minAgent, wrapperSHA != "", bundleSHA)
// Agent-plane immediate-sync (Direction-2a, v0.59.0): a MinAgent-floor / vouched-agent change is
// a fleet-wide agent-plane intent shift. Fire-and-forget nudge every box so it re-reports at
// once (the self-update train's signed op / floor re-evaluation lands in seconds, not ≤15 min).
+1
View File
@@ -33,6 +33,7 @@ import (
const (
pkgAgent = "felhom-agent"
fileAgent = "felhom-agent"
fileBundle = "felhom-config-bundle.json" // R-840: the agent's config bundle, beside the binary
pkgGolden = "felhom-golden"
fileGolden = "golden.tar.zst"
)
+42 -4
View File
@@ -31,9 +31,10 @@ type systemRow struct {
FactsNote string
// host
PVE, KernelRunning, KernelNextBoot, HostDebian cell
HostRelease, HostPending, HostNotCovered cell
Held, RebootSince, KernelPanic, Oops cell
CrashRestarts24h, Guard cell
HostRelease, HostPending, HostNotCovered cell
Held, RebootSince, KernelPanic, Oops cell
CrashRestarts24h, Guard cell
Bundle cell // R-840: the root-owned config bundle
// guest
GuestDebian, GuestRelease, GuestPending, GuestRestart cell
// docker
@@ -46,13 +47,43 @@ type systemRow struct {
type OSSystemView interface {
Fleet() ([]osupdates.FleetLine, error)
Releases() []osupdates.ReleaseInfo
CancelledReleases() []osupdates.ReleaseInfo
Candidates() []osupdates.Status
Thresholds() (stale, reboot, notCovered time.Duration)
BundleThreshold() time.Duration
ApproveDocker() (string, error)
}
func plain(s string) cell { return cell{Text: s} }
// bundleCell is the "Root files" cell (R-840): the box's config bundle against the vouched agent's. Amber while behind,
// red from the alarm's wait on, amber when a file was changed by hand (drift); "unknown" is never coloured as a fact.
func bundleCell(f sysfacts.System, vouchedAgent, vouchedSHA string, since time.Time, after time.Duration, now time.Time) cell {
b := f.Bundle
if !f.Present || b.Version == "" || b.Version == sysfacts.Unknown {
return unknownCell("")
}
c := cell{Text: b.Version}
switch {
case vouchedSHA == "":
c.Title = "no vouched bundle to compare with (the vouched agent carries none)"
case b.BundleSHA256 != vouchedSHA:
c.Class, c.Title = "warn", "behind the vouched agent "+vouchedAgent+"'s bundle — send it with a signed agent_config_update"
if !since.IsZero() {
c.Title += " (behind since " + since.UTC().Format("2006-01-02 15:04") + " UTC)"
if now.Sub(since) >= after {
c.Class = "bad"
}
}
}
if len(b.Drift) > 0 {
c.Text += " (changed by hand)"
c.Class = "warn"
c.Title = "files differ from the installed bundle: " + strings.Join(b.Drift, ", ")
}
return c
}
func unknownCell(s string) cell {
if s == "" || s == sysfacts.Unknown {
return cell{Text: "unknown", Class: "warn", Title: "the box could not read it (agent older than v0.142.0, or the guest is down)"}
@@ -207,9 +238,16 @@ func (s *Server) handleSystem(w http.ResponseWriter, r *http.Request) {
}
}
stale, reboot, notCov := view.Thresholds()
rows := buildSystemRows(lines, facts, names, stale, reboot, notCov, time.Now())
man := s.store.GetArtifactManifest()
for i := range rows {
rows[i].Bundle = bundleCell(facts[rows[i].HostID], man.AgentVersion, man.BundleSHA256,
s.store.BundleBehindSince(rows[i].HostID), view.BundleThreshold(), time.Now())
}
data := map[string]interface{}{
"Rows": buildSystemRows(lines, facts, names, stale, reboot, notCov, time.Now()),
"Rows": rows,
"Releases": view.Releases(),
"Cancelled": view.CancelledReleases(),
"Candidates": view.Candidates(),
"Flash": r.URL.Query().Get("flash"),
"FlashErr": r.URL.Query().Get("err"),
+32
View File
@@ -0,0 +1,32 @@
package web
import (
"testing"
"time"
"gitea.dooplex.hu/admin/felhom-hub/internal/sysfacts"
)
// R-840: the "Root files" cell — amber behind, red from the alarm's wait, amber on drift, unknown never coloured as a fact.
func TestBundleCell(t *testing.T) {
now := time.Date(2026, 10, 4, 20, 0, 0, 0, time.UTC)
sys := func(rep string) sysfacts.System { return sysfacts.Parse(rep) }
cur := sys(`{"system":{"config_bundle":{"version":"0.143.0","bundle_sha256":"new"}}}`)
old := sys(`{"system":{"config_bundle":{"version":"none"}}}`)
drift := sys(`{"system":{"config_bundle":{"version":"0.143.0","bundle_sha256":"new"},"facts":{"host":{"config_bundle":{"version":"0.143.0","drift":["/usr/local/sbin/felhom-pbs-apply"]}}}}}`)
if c := bundleCell(cur, "0.143.0", "new", time.Time{}, 7*24*time.Hour, now); c.Class != "" || c.Text != "0.143.0" {
t.Fatalf("current: %+v", c)
}
if c := bundleCell(old, "0.143.0", "new", now.Add(-time.Hour), 7*24*time.Hour, now); c.Class != "warn" || c.Text != "none" {
t.Fatalf("behind 1 h: %+v", c)
}
if c := bundleCell(old, "0.143.0", "new", now.Add(-8*24*time.Hour), 7*24*time.Hour, now); c.Class != "bad" {
t.Fatalf("behind 8 days: %+v", c)
}
if c := bundleCell(drift, "0.143.0", "new", time.Time{}, 7*24*time.Hour, now); c.Class != "warn" || c.Title == "" {
t.Fatalf("drift: %+v", c)
}
if c := bundleCell(sys(`{}`), "0.143.0", "new", time.Time{}, 7*24*time.Hour, now); c.Text != "unknown" {
t.Fatalf("no stanza: %+v", c)
}
}
+15 -4
View File
@@ -41,9 +41,20 @@
<div class="rel-grid">
{{range .Releases}}
<div><strong>{{.Layer}}</strong>: <code>{{.ID}}</code><br>
<span class="text-muted">{{.Packages}} packages · {{.ApprovedAt.UTC.Format "2006-01-02 15:04"}} UTC · by {{.ApprovedBy}}</span></div>
<span class="text-muted">{{.Packages}} packages · {{.ApprovedAt.UTC.Format "2006-01-02 15:04"}} UTC · by {{.ApprovedBy}}</span>
{{if .Test}}<br><span class="c-warn" title="Approved while a TEST wait override was active. It is cancelled when the hub starts without the override (`11` §5.3.1).">TEST approval</span>{{end}}</div>
{{else}}<div class="text-muted">No release approved yet.</div>{{end}}
</div>
{{if .Cancelled}}
<h3>Cancelled approvals (last 7 days)</h3>
<div class="rel-grid">
{{range .Cancelled}}
<div><strong>{{.Layer}}</strong>: <code>{{.ID}}</code><br>
<span class="c-warn">cancelled {{.Cancelled}} UTC{{if .Test}} — a TEST approval{{end}}</span><br>
<span class="text-muted">no further box installs it; boxes that installed it keep it</span></div>
{{end}}
</div>
{{end}}
<h3>What ring 0 runs now</h3>
<div class="rel-grid">
{{range .Candidates}}
@@ -72,12 +83,12 @@
<thead>
<tr>
<th>Box</th><th>Ring / updates</th><th>Tunnel</th>
<th class="grp">Proxmox</th><th>Kernel (running)</th><th>Kernel (next boot)</th><th>Debian</th><th>Felhom release</th><th>Pending</th><th>Not covered</th><th>Held</th><th>Reboot needed</th><th>kernel.panic</th><th>Oops</th><th>Crash restarts 24 h</th><th>Crash guard</th>
<th class="grp">Proxmox</th><th>Kernel (running)</th><th>Kernel (next boot)</th><th>Debian</th><th>Felhom release</th><th>Pending</th><th>Not covered</th><th>Held</th><th>Reboot needed</th><th>kernel.panic</th><th>Oops</th><th>Crash restarts 24 h</th><th>Crash guard</th><th title="The root-owned config bundle: sudoers, wrappers, units (R-840)">Root files</th>
<th class="grp">Guest Debian</th><th>Felhom release</th><th>Pending</th><th>Restart needed</th>
<th class="grp">Docker</th><th>containerd</th><th>live-restore</th><th>Docker release</th>
<th class="grp">Last OS leg</th>
</tr>
<tr class="text-muted"><th></th><th></th><th></th><th class="grp" colspan="13">host</th><th class="grp" colspan="4">guest</th><th class="grp" colspan="4">Docker engine</th><th class="grp"></th></tr>
<tr class="text-muted"><th></th><th></th><th></th><th class="grp" colspan="14">host</th><th class="grp" colspan="4">guest</th><th class="grp" colspan="4">Docker engine</th><th class="grp"></th></tr>
</thead>
<tbody>
{{range .Rows}}
@@ -103,7 +114,7 @@
{{template "sys_cell" .KernelRunning}}{{template "sys_cell" .KernelNextBoot}}{{template "sys_cell" .HostDebian}}
{{template "sys_cell" .HostRelease}}{{template "sys_cell" .HostPending}}{{template "sys_cell" .HostNotCovered}}
{{template "sys_cell" .Held}}{{template "sys_cell" .RebootSince}}{{template "sys_cell" .KernelPanic}}{{template "sys_cell" .Oops}}
{{template "sys_cell" .CrashRestarts24h}}{{template "sys_cell" .Guard}}
{{template "sys_cell" .CrashRestarts24h}}{{template "sys_cell" .Guard}}{{template "sys_cell" .Bundle}}
<td class="grp {{if .GuestDebian.Class}}c-{{.GuestDebian.Class}}{{end}}">{{.GuestDebian.Text}}</td>
{{template "sys_cell" .GuestRelease}}{{template "sys_cell" .GuestPending}}{{template "sys_cell" .GuestRestart}}
<td class="grp {{if .Engine.Class}}c-{{.Engine.Class}}{{end}}">{{.Engine.Text}}</td>