v0.90.0 — guest RAM resize (R-24) + fast-tick-until-convergence (R-28)
MinAgent coupling: felhom-controller v0.143.0 gates its guest-memory-resize UI on this agent (FeatureGuestMemoryResize, MinAgent 0.90.0). R-24 guest RAM resize (internal/localapi/guestmemory.go): self-scoped GET/POST /guest/memory. Agent enforces every bound FRESH per request (min 2048, max host_total-2048, shrink floor max(2048, usage+512)); applies via PVE SetConfig — live cgroup apply, no reboot (Phase-0 proven on the nested demo box). Verify-after-apply re-reads maxmem before claiming success. New narrow MemoryOps seam (GuestAPI untouched); Options.Memory nil -> 503. Memory only. R-28 fast-tick (internal/fasttick): while any desired-state item is unapplied - including the pre-tunnel window a hub poke can't reach - pulse the shared out-of-band trigger every 30s, self-disarm on convergence. Four cached sources (desired-gen==0, reconcile Planned-Pending>0, pbsdr waiting_secret only, wgtunnel desired-not-operational); LOUD pbsdr states + pending_signature excluded. Seams: reconcile.Engine.LastResult() + wgtunnel.Manager.TunnelConvergence() (cached, no per-tick exec). Guests-0/0: hypothesis REFUTED live (9201 IS a pool member; 0/0 was the pre-provision window; PoolAddVMID re-assert already covers restore-over-existing). No code change; the fast-tick mitigates the window. Tests + red-proofs (i floor guard, ii max guard, iii always-pulse) all restored green.
This commit is contained in:
@@ -1,3 +1,48 @@
|
|||||||
|
## v0.90.0 — agent train: guest RAM resize (R-24) + fast-tick-until-convergence (R-28) (2026-07-17)
|
||||||
|
|
||||||
|
**MinAgent coupling:** felhom-controller v0.143.0 gates its guest-memory-resize UI on this agent
|
||||||
|
(`FeatureGuestMemoryResize`, MinAgent 0.90.0). One train, one floor raise (Viktor's ruling).
|
||||||
|
Green: `go build ./... && go vet ./... && go test ./...` all pass.
|
||||||
|
|
||||||
|
- **Item 1 — guest RAM resize (R-24, controller-direct)** (`internal/localapi/guestmemory.go`, new):
|
||||||
|
a new self-scoped local-API surface — `GET /guest/memory` (current allocation, live usage, and the
|
||||||
|
enforced bounds, all agent-computed in MB) and `POST /guest/memory` (bounded resize). The agent is
|
||||||
|
the security boundary: every bound is recomputed FRESH per request from a live read (the UI's numbers
|
||||||
|
are decoration). Ruled bounds — min **2048 MB**; max **host_total − 2048 MB** (host reserve); a SHRINK
|
||||||
|
is refused below **max(2048, usage + 512 MB)** with `below_usage_floor` (stop apps first). Applies via
|
||||||
|
the PVE API `SetConfig` — a **live cgroup apply, no reboot** (Phase-0 PROVEN on the nested demo box:
|
||||||
|
maxmem moves with the guest running, `/proc/meminfo` ripples via lxcfs). Verify-after-apply: the new
|
||||||
|
maxmem is re-read and must equal the target before success is claimed (a pending/reboot outcome is a
|
||||||
|
502, never a false success). Refusals return a machine `code` (below_min/above_max/below_usage_floor)
|
||||||
|
+ fresh bounds at 412; SetConfig is never called on a refusal path. Single-flight per host. Optional
|
||||||
|
`Options.Memory` (nil → 503 "not configured"); a NEW narrow `MemoryOps` interface (does not touch the
|
||||||
|
shared `GuestAPI`). Wired in `main.go` from the existing proxmox client. Memory only; cores stay
|
||||||
|
observation.
|
||||||
|
- **Item 2 — fast-tick-until-first-convergence (R-28)** (`internal/fasttick/`, new): the agent-plane
|
||||||
|
immediacy SECONDARY. While ANY desired-state item is unapplied — most importantly the **pre-tunnel
|
||||||
|
WG-registration window a hub poke cannot reach** — it pulses the SAME out-of-band report trigger the
|
||||||
|
watchdog/poke use, every **30 s**, and **self-disarms emergently** the instant everything converges.
|
||||||
|
State-based by ruling: nothing to journal, no timer to leak. Four cached sources (no exec/network per
|
||||||
|
tick): desired-generation == 0 (never fetched); reconcile `LastResult` actionable drift
|
||||||
|
(`Planned − Pending > 0` — a destructive `pending_signature` is EXPECTED, excluded); pbsdr
|
||||||
|
`waiting_secret` ONLY (the LOUD `consumed_failed`/`verify_failed` are excluded so a stuck box never
|
||||||
|
hammers); wgtunnel block-desired-but-not-operational. Supporting seams: `reconcile.Engine.LastResult()`
|
||||||
|
(mutex-recorded per pass) and `wgtunnel.Manager.TunnelConvergence()` (cached snapshot refreshed at each
|
||||||
|
Apply — the fast-tick never execs `wg`/`systemctl`). A perma-unconverged box fast-ticks at ~2 small
|
||||||
|
reports/min (bounded, documented; timers were deliberately rejected).
|
||||||
|
- **Guests-0/0 (diagnosis passenger):** REFUTED the pool-membership hypothesis on the live nested box —
|
||||||
|
guest 9201 IS a pool member, the agent token sees it (VM.Audit comes from the `/pool/felhom` grant),
|
||||||
|
hub reports 1/1. The observed 0/0 was the legitimate **pre-provision reporting window** (no guest
|
||||||
|
existed yet); the existing `PoolAddVMID` re-assert (bringup.go:498) already covers the known
|
||||||
|
restore-over-existing edge (campaign-2 R2). Item 2's fast-tick is exactly the window's mitigation.
|
||||||
|
No code change.
|
||||||
|
- **Tests + red-proofs (run-fail-restore):** localapi memory (grow/shrink, the three refusals each with
|
||||||
|
a **SetConfig-count == 0** assertion, cross-guest 403, fresh-bounds-per-request, verify-not-reflected
|
||||||
|
502, nil-config 503) — red-proofs (i) floor guard removed → below_usage_floor fails; (ii) max guard
|
||||||
|
removed → above_max fails. fasttick (pulse-while-unconverged, silent-when-converged, the ruled
|
||||||
|
disarm-on-convergence, full-channel non-blocking drop, first-reason) — red-proof (iii) always-pulse →
|
||||||
|
the silent + disarm tests fail. `Engine.LastResult` effect + pre-first-run ok=false. All restored green.
|
||||||
|
|
||||||
## v0.89.0 — agent train: PBS-DR self-grant (R-22) + escrow config live-reload + agent-plane poke listener (2026-07-16)
|
## v0.89.0 — agent train: PBS-DR self-grant (R-22) + escrow config live-reload + agent-plane poke listener (2026-07-16)
|
||||||
|
|
||||||
Three bundled agent-plane items. Green: `go build ./... && go vet ./... && go test ./...` all pass.
|
Three bundled agent-plane items. Green: `go build ./... && go vet ./... && go test ./...` all pass.
|
||||||
|
|||||||
@@ -33,6 +33,7 @@ import (
|
|||||||
"gitea.dooplex.hu/admin/felhom-agent/internal/desired"
|
"gitea.dooplex.hu/admin/felhom-agent/internal/desired"
|
||||||
"gitea.dooplex.hu/admin/felhom-agent/internal/dr"
|
"gitea.dooplex.hu/admin/felhom-agent/internal/dr"
|
||||||
"gitea.dooplex.hu/admin/felhom-agent/internal/escrow"
|
"gitea.dooplex.hu/admin/felhom-agent/internal/escrow"
|
||||||
|
"gitea.dooplex.hu/admin/felhom-agent/internal/fasttick"
|
||||||
"gitea.dooplex.hu/admin/felhom-agent/internal/felhomsshd"
|
"gitea.dooplex.hu/admin/felhom-agent/internal/felhomsshd"
|
||||||
"gitea.dooplex.hu/admin/felhom-agent/internal/guesthook"
|
"gitea.dooplex.hu/admin/felhom-agent/internal/guesthook"
|
||||||
"gitea.dooplex.hu/admin/felhom-agent/internal/hub"
|
"gitea.dooplex.hu/admin/felhom-agent/internal/hub"
|
||||||
@@ -787,6 +788,7 @@ func runDaemon(cfg config.Config, logger *slog.Logger, logRing *applog.Ring) int
|
|||||||
// registration, no report stanza.
|
// registration, no report stanza.
|
||||||
wgServers := 0
|
wgServers := 0
|
||||||
var wgLoop *wgtunnel.Loop
|
var wgLoop *wgtunnel.Loop
|
||||||
|
var wgMgr *wgtunnel.Manager // hoisted for the fast-tick convergence source (nil when tunnel disabled)
|
||||||
var pokeListener *poke.Listener
|
var pokeListener *poke.Listener
|
||||||
{
|
{
|
||||||
wt := cfg.WGTunnel.WithDefaults()
|
wt := cfg.WGTunnel.WithDefaults()
|
||||||
@@ -796,7 +798,7 @@ func runDaemon(cfg config.Config, logger *slog.Logger, logRing *applog.Ring) int
|
|||||||
wtMode = proxmox.RunnerSudo
|
wtMode = proxmox.RunnerSudo
|
||||||
}
|
}
|
||||||
wtRunner := &proxmox.ExecRunner{Mode: wtMode, SudoPath: cfg.Privileged.SudoPath}
|
wtRunner := &proxmox.ExecRunner{Mode: wtMode, SudoPath: cfg.Privileged.SudoPath}
|
||||||
wgMgr := wgtunnel.NewManager(wtRunner, client, wt.StateDir, logger)
|
wgMgr = wgtunnel.NewManager(wtRunner, client, wt.StateDir, logger)
|
||||||
wgMgr.SetStaleAfter(time.Duration(wt.StaleAfterSeconds) * time.Second)
|
wgMgr.SetStaleAfter(time.Duration(wt.StaleAfterSeconds) * time.Second)
|
||||||
wgLoop = wgtunnel.NewLoop(wgMgr, time.Duration(wt.IntervalSeconds)*time.Second, logger)
|
wgLoop = wgtunnel.NewLoop(wgMgr, time.Duration(wt.IntervalSeconds)*time.Second, logger)
|
||||||
desiredSyncer.AddConsumer(wgLoop) // raw desired-state → the wireguard block
|
desiredSyncer.AddConsumer(wgLoop) // raw desired-state → the wireguard block
|
||||||
@@ -883,6 +885,44 @@ func runDaemon(cfg config.Config, logger *slog.Logger, logRing *applog.Ring) int
|
|||||||
if pokeListener != nil {
|
if pokeListener != nil {
|
||||||
go func() { errc <- pokeListener.Run(ctx) }() // agent-plane immediate-sync listener (v0.89.0)
|
go func() { errc <- pokeListener.Run(ctx) }() // agent-plane immediate-sync listener (v0.89.0)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Fast-tick (v0.90.0, R-28): the agent-plane immediacy SECONDARY. While ANY desired-state item is
|
||||||
|
// unapplied — most importantly the pre-tunnel window a hub poke cannot reach — pulse the SAME
|
||||||
|
// out-of-band trigger the watchdog/poke use, every 30 s, and self-disarm the instant everything
|
||||||
|
// converges. Every source is a cached read (no exec/network per tick). The LOUD pbsdr states
|
||||||
|
// (consumed_failed/verify_failed) and the destructive pending_signature drift are deliberately
|
||||||
|
// EXCLUDED (§8) so a stuck-loud box never hammers.
|
||||||
|
fastTick := fasttick.New(storageTrigger, fasttick.DefaultInterval, logger,
|
||||||
|
fasttick.SourceFunc(func() (bool, string) {
|
||||||
|
if desiredProvider.Generation() == 0 {
|
||||||
|
return true, "desired-state not yet fetched"
|
||||||
|
}
|
||||||
|
return false, ""
|
||||||
|
}),
|
||||||
|
fasttick.SourceFunc(func() (bool, string) {
|
||||||
|
if res, ok := engine.LastResult(); ok && res.Planned-res.Pending > 0 {
|
||||||
|
return true, "reconcile drift not yet applied"
|
||||||
|
}
|
||||||
|
return false, ""
|
||||||
|
}),
|
||||||
|
fasttick.SourceFunc(func() (bool, string) {
|
||||||
|
if pbsdrLoop != nil {
|
||||||
|
if st := pbsdrLoop.PBSDRStatus(ctx); st != nil && st.State == "waiting_secret" {
|
||||||
|
return true, "pbsdr waiting for its consume-once secret"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false, ""
|
||||||
|
}),
|
||||||
|
fasttick.SourceFunc(func() (bool, string) {
|
||||||
|
if wgMgr != nil {
|
||||||
|
if desired, operational := wgMgr.TunnelConvergence(); desired && !operational {
|
||||||
|
return true, "wireguard tunnel not yet operational"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false, ""
|
||||||
|
}),
|
||||||
|
)
|
||||||
|
go func() { errc <- fastTick.Run(ctx) }()
|
||||||
if localSrv != nil {
|
if localSrv != nil {
|
||||||
localServers = 1
|
localServers = 1
|
||||||
// Host-reboot remount fix: BEFORE binding into the guest, re-assert enrolled drive MOUNTS on the
|
// Host-reboot remount fix: BEFORE binding into the guest, re-assert enrolled drive MOUNTS on the
|
||||||
@@ -1178,6 +1218,7 @@ func buildLocalAPIServer(cfg config.Config, px *proxmox.Client, store *backup.St
|
|||||||
DiskGate: storageGateAdapter{gate: gate, hostID: cfg.Hub.HostID},
|
DiskGate: storageGateAdapter{gate: gate, hostID: cfg.Hub.HostID},
|
||||||
Guests2: px,
|
Guests2: px,
|
||||||
GuestAttach: guestBinder, // slice 10 P2: bind enrolled data drives into the guest
|
GuestAttach: guestBinder, // slice 10 P2: bind enrolled data drives into the guest
|
||||||
|
Memory: px, // v0.90.0 R-24: guest RAM resize (SetConfig live cgroup apply)
|
||||||
// Network storage (NAS) — Part A1: the privileged host network-mount surface (NFS/SMB automount).
|
// Network storage (NAS) — Part A1: the privileged host network-mount surface (NFS/SMB automount).
|
||||||
NetStorage: hostOps,
|
NetStorage: hostOps,
|
||||||
SmbCredsDir: cfg.Privileged.SmbCredsDir,
|
SmbCredsDir: cfg.Privileged.SmbCredsDir,
|
||||||
|
|||||||
@@ -0,0 +1,105 @@
|
|||||||
|
// Package fasttick is the agent-plane immediacy SECONDARY (v0.90.0, R-28). While ANY desired-state
|
||||||
|
// item is still unapplied — most importantly the pre-tunnel WG-registration window where a hub poke
|
||||||
|
// is undeliverable by construction — it pulses the hub control loop's out-of-band report trigger on
|
||||||
|
// a fast (30 s) cadence, and self-disarms EMERGENTLY the instant everything converges. It is the
|
||||||
|
// state-based complement to the poke: the poke handles hub→box changes once the tunnel exists; the
|
||||||
|
// fast-tick handles the window before that (and any lingering unapplied drift) from the box side.
|
||||||
|
//
|
||||||
|
// By ruling it is STATE-BASED, not a fixed burst and not a timer: there is nothing to journal
|
||||||
|
// (stateless across restarts) and nothing to leak. A perma-unconverged box fast-ticks at ~2 small
|
||||||
|
// reports/min, bounded and visible; the LOUD pbsdr states (consumed_failed/verify_failed) are
|
||||||
|
// deliberately EXCLUDED from the sources so a stuck-loud box does not hammer (§8).
|
||||||
|
//
|
||||||
|
// It pulses the SAME cap-1 channel the storage watchdog and the poke listener use, so a pulse
|
||||||
|
// coalesces with a poke/watchdog nudge for free — no extra debounce here.
|
||||||
|
package fasttick
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"log/slog"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// DefaultInterval is the ruled fast cadence while unconverged.
|
||||||
|
const DefaultInterval = 30 * time.Second
|
||||||
|
|
||||||
|
// Source reports whether one subsystem still has unapplied desired-state. Implementations MUST be a
|
||||||
|
// cheap, CACHED read — no exec, no network per call (the fast-tick calls every source each tick).
|
||||||
|
type Source interface {
|
||||||
|
Unconverged() (unconverged bool, reason string)
|
||||||
|
}
|
||||||
|
|
||||||
|
// SourceFunc adapts a plain func to a Source (main.go closes over each subsystem).
|
||||||
|
type SourceFunc func() (bool, string)
|
||||||
|
|
||||||
|
// Unconverged implements Source.
|
||||||
|
func (f SourceFunc) Unconverged() (bool, string) { return f() }
|
||||||
|
|
||||||
|
// Loop evaluates the sources on a ticker and pulses the out-of-band channel while any is unconverged.
|
||||||
|
type Loop struct {
|
||||||
|
sources []Source
|
||||||
|
out chan<- struct{}
|
||||||
|
interval time.Duration
|
||||||
|
logger *slog.Logger
|
||||||
|
armed bool // for armed↔disarmed transition logging (avoids 30 s reason spam)
|
||||||
|
}
|
||||||
|
|
||||||
|
// New builds a fast-tick loop. out is the hub loop's out-of-band trigger channel (cap-1). A
|
||||||
|
// non-positive interval falls back to DefaultInterval.
|
||||||
|
func New(out chan<- struct{}, interval time.Duration, logger *slog.Logger, sources ...Source) *Loop {
|
||||||
|
if interval <= 0 {
|
||||||
|
interval = DefaultInterval
|
||||||
|
}
|
||||||
|
if logger == nil {
|
||||||
|
logger = slog.Default()
|
||||||
|
}
|
||||||
|
return &Loop{sources: sources, out: out, interval: interval, logger: logger}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Run evaluates the sources every interval until ctx is cancelled. Stateless — nothing to recover.
|
||||||
|
func (l *Loop) Run(ctx context.Context) error {
|
||||||
|
l.logger.Info("fast-tick armed: "+l.interval.String()+" out-of-band cadence while desired-state is unapplied", "interval", l.interval)
|
||||||
|
ticker := time.NewTicker(l.interval)
|
||||||
|
defer ticker.Stop()
|
||||||
|
for {
|
||||||
|
select {
|
||||||
|
case <-ctx.Done():
|
||||||
|
l.logger.Info("fast-tick: shutting down", "reason", ctx.Err())
|
||||||
|
return nil
|
||||||
|
case <-ticker.C:
|
||||||
|
l.step()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// step evaluates the sources once. If any is unconverged it pulses the channel (non-blocking: a full
|
||||||
|
// channel means an out-of-band report is already pending, so the pulse coalesces harmlessly) and, on
|
||||||
|
// the disarmed→armed edge, logs the reason. When all converge it logs the armed→disarmed edge once.
|
||||||
|
// Returns whether this tick found the box unconverged (test hook). No per-tick logging when steady.
|
||||||
|
func (l *Loop) step() bool {
|
||||||
|
unconverged, reason := l.evaluate()
|
||||||
|
if unconverged {
|
||||||
|
select {
|
||||||
|
case l.out <- struct{}{}:
|
||||||
|
default: // an out-of-band report is already queued — coalesce, never block
|
||||||
|
}
|
||||||
|
if !l.armed {
|
||||||
|
l.logger.Info("fast-tick: desired-state unapplied — pulsing out-of-band reports", "reason", reason, "cadence", l.interval)
|
||||||
|
l.armed = true
|
||||||
|
}
|
||||||
|
} else if l.armed {
|
||||||
|
l.logger.Info("fast-tick: desired-state converged — back to the normal cadence")
|
||||||
|
l.armed = false
|
||||||
|
}
|
||||||
|
return unconverged
|
||||||
|
}
|
||||||
|
|
||||||
|
// evaluate returns the first unconverged source's reason (order = priority for the log line).
|
||||||
|
func (l *Loop) evaluate() (bool, string) {
|
||||||
|
for _, s := range l.sources {
|
||||||
|
if u, reason := s.Unconverged(); u {
|
||||||
|
return true, reason
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false, ""
|
||||||
|
}
|
||||||
@@ -0,0 +1,121 @@
|
|||||||
|
package fasttick
|
||||||
|
|
||||||
|
import (
|
||||||
|
"io"
|
||||||
|
"log/slog"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func quiet() *slog.Logger { return slog.New(slog.NewTextHandler(io.Discard, nil)) }
|
||||||
|
|
||||||
|
// flagSource is a fake Source whose convergence the test flips at will.
|
||||||
|
type flagSource struct {
|
||||||
|
unconverged bool
|
||||||
|
reason string
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *flagSource) Unconverged() (bool, string) { return f.unconverged, f.reason }
|
||||||
|
|
||||||
|
// drain reports how many pulses are queued (channel is cap-1 in production; tests may use larger).
|
||||||
|
func drain(ch chan struct{}) int {
|
||||||
|
n := 0
|
||||||
|
for {
|
||||||
|
select {
|
||||||
|
case <-ch:
|
||||||
|
n++
|
||||||
|
default:
|
||||||
|
return n
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// D1 — any source unconverged → one pulse per tick (drained between ticks).
|
||||||
|
func TestFastTick_UnconvergedPulses(t *testing.T) {
|
||||||
|
out := make(chan struct{}, 1)
|
||||||
|
src := &flagSource{unconverged: true, reason: "test drift"}
|
||||||
|
l := New(out, time.Hour, quiet(), src)
|
||||||
|
|
||||||
|
for i := 0; i < 3; i++ {
|
||||||
|
if !l.step() {
|
||||||
|
t.Fatalf("tick %d: step reported converged, want unconverged", i)
|
||||||
|
}
|
||||||
|
if got := drain(out); got != 1 {
|
||||||
|
t.Fatalf("tick %d: pulses = %d, want 1", i, got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// D2 — ALL sources converged → zero pulses across N ticks (the emergent disarm).
|
||||||
|
func TestFastTick_ConvergedSilent(t *testing.T) {
|
||||||
|
out := make(chan struct{}, 4)
|
||||||
|
l := New(out, time.Hour, quiet(), &flagSource{unconverged: false}, &flagSource{unconverged: false})
|
||||||
|
for i := 0; i < 5; i++ {
|
||||||
|
if l.step() {
|
||||||
|
t.Fatalf("tick %d: step reported unconverged with all sources converged", i)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if got := drain(out); got != 0 {
|
||||||
|
t.Fatalf("pulses on a fully-converged box = %d, want 0", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// D3 (the ruled red-proof) — flip unconverged→converged mid-run → pulses STOP from the next tick.
|
||||||
|
func TestFastTick_ConvergenceDisarms(t *testing.T) {
|
||||||
|
out := make(chan struct{}, 8)
|
||||||
|
src := &flagSource{unconverged: true, reason: "drift"}
|
||||||
|
l := New(out, time.Hour, quiet(), src)
|
||||||
|
|
||||||
|
// Two unconverged ticks pulse.
|
||||||
|
l.step()
|
||||||
|
l.step()
|
||||||
|
if got := drain(out); got != 2 {
|
||||||
|
t.Fatalf("pre-convergence pulses = %d, want 2", got)
|
||||||
|
}
|
||||||
|
// Converge.
|
||||||
|
src.unconverged = false
|
||||||
|
// Every subsequent tick is silent — the cadence returns to normal.
|
||||||
|
for i := 0; i < 4; i++ {
|
||||||
|
if l.step() {
|
||||||
|
t.Fatalf("post-convergence tick %d still unconverged", i)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if got := drain(out); got != 0 {
|
||||||
|
t.Fatalf("pulses fired after convergence = %d, want 0 (disarm failed)", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// D4 — the out channel is already full (a poke just landed): the non-blocking send drops, no block,
|
||||||
|
// no goroutine leak, no queue growth.
|
||||||
|
func TestFastTick_ChannelFullDrops(t *testing.T) {
|
||||||
|
out := make(chan struct{}, 1)
|
||||||
|
out <- struct{}{} // pre-fill: an out-of-band report is already pending
|
||||||
|
l := New(out, time.Hour, quiet(), &flagSource{unconverged: true, reason: "drift"})
|
||||||
|
|
||||||
|
done := make(chan bool, 1)
|
||||||
|
go func() {
|
||||||
|
l.step() // must NOT block on the full channel
|
||||||
|
l.step()
|
||||||
|
done <- true
|
||||||
|
}()
|
||||||
|
select {
|
||||||
|
case <-done:
|
||||||
|
case <-time.After(2 * time.Second):
|
||||||
|
t.Fatal("step blocked on a full channel (non-blocking send violated)")
|
||||||
|
}
|
||||||
|
if len(out) != 1 {
|
||||||
|
t.Fatalf("channel depth = %d, want 1 (coalesced, no queue growth)", len(out))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Priority ordering: the first unconverged source supplies the reason.
|
||||||
|
func TestFastTick_FirstReasonWins(t *testing.T) {
|
||||||
|
out := make(chan struct{}, 1)
|
||||||
|
l := New(out, time.Hour, quiet(),
|
||||||
|
&flagSource{unconverged: false},
|
||||||
|
&flagSource{unconverged: true, reason: "second"},
|
||||||
|
&flagSource{unconverged: true, reason: "third"})
|
||||||
|
if u, r := l.evaluate(); !u || r != "second" {
|
||||||
|
t.Fatalf("evaluate = (%v, %q), want (true, second)", u, r)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,259 @@
|
|||||||
|
package localapi
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"net/http"
|
||||||
|
|
||||||
|
"gitea.dooplex.hu/admin/felhom-agent/internal/proxmox"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Guest RAM resize (v0.90.0, R-24 controller-direct). The customer sees the guest's current memory
|
||||||
|
// on the controller's system-settings page and resizes it; the controller calls these self-scoped
|
||||||
|
// endpoints; the AGENT enforces every bound FRESH per request (the UI's numbers are decoration) and
|
||||||
|
// applies via the PVE API's SetConfig — a live cgroup apply, no reboot (Phase-0 proved it: maxmem
|
||||||
|
// moves with the guest running, /proc/meminfo ripples via lxcfs). Memory only; cores stay observation.
|
||||||
|
//
|
||||||
|
// Ruled bounds (Viktor, 2026-07-17):
|
||||||
|
// - min = minGuestMemoryMB (2048)
|
||||||
|
// - max = host_total − hostReserveMB (2048 reserved for the host)
|
||||||
|
// - shrink floor = max(minGuestMemoryMB, current_usage + shrinkUsageMarginMB) — a customer wanting
|
||||||
|
// less must stop applications first (the refusal says so, in the controller's Hungarian).
|
||||||
|
//
|
||||||
|
// The floor gates SHRINK only; a grow is bounded by min/max alone.
|
||||||
|
|
||||||
|
const (
|
||||||
|
// minGuestMemoryMB is the ruled floor for any guest allocation (a Felhom box needs headroom).
|
||||||
|
minGuestMemoryMB int64 = 2048
|
||||||
|
// hostReserveMB is held back from the host total so a resize can never starve the hypervisor.
|
||||||
|
hostReserveMB int64 = 2048
|
||||||
|
// shrinkUsageMarginMB is the safety gap kept above live usage on a SHRINK (the "never below
|
||||||
|
// current usage" ruling, with headroom). One named constant — trivially re-ruled.
|
||||||
|
shrinkUsageMarginMB int64 = 512
|
||||||
|
// mib is one Proxmox "memory" unit (MiB) in bytes. PVE config `memory` is MiB; status/node
|
||||||
|
// memory fields are bytes — convert at this boundary (the §8 units trap).
|
||||||
|
mib int64 = 1 << 20
|
||||||
|
)
|
||||||
|
|
||||||
|
// MemoryOps is the guest-RAM-resize Proxmox surface (v0.90.0). Satisfied by *proxmox.Client. Kept
|
||||||
|
// separate from GuestAPI so adding it breaks no existing fake. Every method is invoked ONLY with the
|
||||||
|
// token-resolved VMID (never a caller-supplied id).
|
||||||
|
type MemoryOps interface {
|
||||||
|
GuestStatus(ctx context.Context, vmid int) (proxmox.Guest, error)
|
||||||
|
GuestConfig(ctx context.Context, vmid int) (proxmox.GuestConfig, error)
|
||||||
|
NodeStatus(ctx context.Context) (proxmox.NodeStatus, error)
|
||||||
|
SetConfig(ctx context.Context, vmid int, params map[string]string) (string, error)
|
||||||
|
WaitTask(ctx context.Context, upid string, opts proxmox.WaitOptions) (proxmox.TaskStatus, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// MemoryInfo is GET /guest/memory — every field in MB, computed agent-side (Scenario C). min/max/floor
|
||||||
|
// are the CURRENT enforced bounds, so a stale UI re-renders honestly on every read/refusal.
|
||||||
|
type MemoryInfo struct {
|
||||||
|
VMID int `json:"vmid"`
|
||||||
|
AllocatedMB int64 `json:"allocated_mb"`
|
||||||
|
UsageMB int64 `json:"usage_mb"`
|
||||||
|
HostTotalMB int64 `json:"host_total_mb"`
|
||||||
|
MinMB int64 `json:"min_mb"`
|
||||||
|
MaxMB int64 `json:"max_mb"`
|
||||||
|
FloorMB int64 `json:"floor_mb"`
|
||||||
|
Running bool `json:"running"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// memoryResizeRequest is the POST /guest/memory body.
|
||||||
|
type memoryResizeRequest struct {
|
||||||
|
VMID int `json:"vmid,omitempty"` // optional; if set must equal the token's guest (self-scope)
|
||||||
|
MemoryMB int64 `json:"memory_mb"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// memoryRefusal is the data field of a 412 refusal: the machine code the controller maps to Hungarian,
|
||||||
|
// plus the fresh bounds so the UI re-renders without a second round-trip.
|
||||||
|
type memoryRefusal struct {
|
||||||
|
Code string `json:"code"` // below_min | above_max | below_usage_floor
|
||||||
|
AllocatedMB int64 `json:"allocated_mb"`
|
||||||
|
UsageMB int64 `json:"usage_mb"`
|
||||||
|
HostTotalMB int64 `json:"host_total_mb"`
|
||||||
|
MinMB int64 `json:"min_mb"`
|
||||||
|
MaxMB int64 `json:"max_mb"`
|
||||||
|
FloorMB int64 `json:"floor_mb"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// memoryResizeResult is the success data field.
|
||||||
|
type memoryResizeResult struct {
|
||||||
|
VMID int `json:"vmid"`
|
||||||
|
OldMB int64 `json:"old_mb"`
|
||||||
|
NewMB int64 `json:"new_mb"`
|
||||||
|
Unchanged bool `json:"unchanged"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// memoryBounds is one fresh snapshot of everything the validation needs.
|
||||||
|
type memoryBounds struct {
|
||||||
|
allocatedMB int64
|
||||||
|
usageMB int64
|
||||||
|
hostTotalMB int64
|
||||||
|
minMB int64
|
||||||
|
maxMB int64
|
||||||
|
floorMB int64
|
||||||
|
running bool
|
||||||
|
}
|
||||||
|
|
||||||
|
func (b memoryBounds) refusal(code string) memoryRefusal {
|
||||||
|
return memoryRefusal{
|
||||||
|
Code: code, AllocatedMB: b.allocatedMB, UsageMB: b.usageMB, HostTotalMB: b.hostTotalMB,
|
||||||
|
MinMB: b.minMB, MaxMB: b.maxMB, FloorMB: b.floorMB,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (b memoryBounds) info(vmid int) MemoryInfo {
|
||||||
|
return MemoryInfo{
|
||||||
|
VMID: vmid, AllocatedMB: b.allocatedMB, UsageMB: b.usageMB, HostTotalMB: b.hostTotalMB,
|
||||||
|
MinMB: b.minMB, MaxMB: b.maxMB, FloorMB: b.floorMB, Running: b.running,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// bytesToMBUp converts bytes → MB rounding UP (usage must never be under-reported for the floor).
|
||||||
|
func bytesToMBUp(b int64) int64 {
|
||||||
|
if b <= 0 {
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
return (b + mib - 1) / mib
|
||||||
|
}
|
||||||
|
|
||||||
|
// readMemoryBounds computes the enforced bounds from a FRESH read of guest config/status + host total.
|
||||||
|
// Called by BOTH the GET and the POST so a stale UI can never smuggle an old max/floor.
|
||||||
|
func (s *Server) readMemoryBounds(ctx context.Context, vmid int) (memoryBounds, error) {
|
||||||
|
cfg, err := s.mem.GuestConfig(ctx, vmid)
|
||||||
|
if err != nil {
|
||||||
|
return memoryBounds{}, fmt.Errorf("guest config: %w", err)
|
||||||
|
}
|
||||||
|
st, err := s.mem.GuestStatus(ctx, vmid)
|
||||||
|
if err != nil {
|
||||||
|
return memoryBounds{}, fmt.Errorf("guest status: %w", err)
|
||||||
|
}
|
||||||
|
node, err := s.mem.NodeStatus(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return memoryBounds{}, fmt.Errorf("node status: %w", err)
|
||||||
|
}
|
||||||
|
b := memoryBounds{
|
||||||
|
allocatedMB: cfg.Memory, // PVE config memory is already MB
|
||||||
|
usageMB: bytesToMBUp(st.Mem), // bytes → MB, rounded up
|
||||||
|
hostTotalMB: node.Memory.Total / mib, // bytes → MB, floor (conservative for max)
|
||||||
|
minMB: minGuestMemoryMB,
|
||||||
|
running: st.Status == "running",
|
||||||
|
}
|
||||||
|
b.maxMB = b.hostTotalMB - hostReserveMB
|
||||||
|
b.floorMB = b.usageMB + shrinkUsageMarginMB
|
||||||
|
if b.floorMB < minGuestMemoryMB {
|
||||||
|
b.floorMB = minGuestMemoryMB
|
||||||
|
}
|
||||||
|
return b, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// handleGuestMemory serves GET /guest/memory — the current allocation, live usage, and the enforced
|
||||||
|
// bounds, all agent-computed (Scenario C).
|
||||||
|
func (s *Server) handleGuestMemory(w http.ResponseWriter, r *http.Request, vmid int) {
|
||||||
|
if s.mem == nil {
|
||||||
|
writeErr(w, http.StatusServiceUnavailable, "guest memory resize not configured on this host")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
b, err := s.readMemoryBounds(r.Context(), vmid)
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Error("local-api: guest-memory read", "vmid", vmid, "err", err)
|
||||||
|
writeErr(w, http.StatusBadGateway, "could not read guest memory: "+err.Error())
|
||||||
|
return
|
||||||
|
}
|
||||||
|
writeOK(w, b.info(vmid))
|
||||||
|
}
|
||||||
|
|
||||||
|
// handleGuestMemoryResize serves POST /guest/memory — validate FRESH against the ruled bounds (the UI's
|
||||||
|
// numbers are decoration), then apply via SetConfig (live cgroup apply) and verify the new maxmem before
|
||||||
|
// claiming success. Single-flight (one customer per host). SetConfig is NEVER called on a refusal path.
|
||||||
|
func (s *Server) handleGuestMemoryResize(w http.ResponseWriter, r *http.Request, vmid int) {
|
||||||
|
if s.mem == nil {
|
||||||
|
writeErr(w, http.StatusServiceUnavailable, "guest memory resize not configured on this host")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
var req memoryResizeRequest
|
||||||
|
if !decodeBody(w, r, &req) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if !s.scopedFromBody(w, req.VMID, vmid, r.URL.Path) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if req.MemoryMB <= 0 {
|
||||||
|
writeErr(w, http.StatusBadRequest, "memory_mb must be a positive integer")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// Single-flight: one customer per host; last-write-wins at PVE is not a UX we want.
|
||||||
|
s.memMu.Lock()
|
||||||
|
defer s.memMu.Unlock()
|
||||||
|
|
||||||
|
b, err := s.readMemoryBounds(r.Context(), vmid)
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Error("local-api: guest-memory resize precheck", "vmid", vmid, "err", err)
|
||||||
|
writeErr(w, http.StatusBadGateway, "could not read guest memory: "+err.Error())
|
||||||
|
return
|
||||||
|
}
|
||||||
|
target := req.MemoryMB
|
||||||
|
|
||||||
|
// No-op: target == current allocation → success, no SetConfig call.
|
||||||
|
if target == b.allocatedMB {
|
||||||
|
s.logger.Info("local-api: guest-memory resize no-op (unchanged)", "vmid", vmid, "memory_mb", target)
|
||||||
|
writeOK(w, memoryResizeResult{VMID: vmid, OldMB: b.allocatedMB, NewMB: b.allocatedMB, Unchanged: true})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// Ruled refusals — SetConfig MUST NOT run on any of these.
|
||||||
|
switch {
|
||||||
|
case target < b.minMB:
|
||||||
|
s.logger.Warn("local-api: guest-memory resize refused (below_min)", "vmid", vmid, "target_mb", target, "min_mb", b.minMB)
|
||||||
|
writeStatus(w, http.StatusPreconditionFailed, false, b.refusal("below_min"),
|
||||||
|
fmt.Sprintf("requested %d MB is below the minimum %d MB", target, b.minMB))
|
||||||
|
return
|
||||||
|
case target > b.maxMB:
|
||||||
|
s.logger.Warn("local-api: guest-memory resize refused (above_max)", "vmid", vmid, "target_mb", target, "max_mb", b.maxMB)
|
||||||
|
writeStatus(w, http.StatusPreconditionFailed, false, b.refusal("above_max"),
|
||||||
|
fmt.Sprintf("requested %d MB exceeds the maximum %d MB (host reserve %d MB)", target, b.maxMB, hostReserveMB))
|
||||||
|
return
|
||||||
|
case target < b.allocatedMB && target < b.floorMB:
|
||||||
|
// SHRINK below the usage floor — the customer must stop applications first.
|
||||||
|
s.logger.Warn("local-api: guest-memory resize refused (below_usage_floor)",
|
||||||
|
"vmid", vmid, "target_mb", target, "usage_mb", b.usageMB, "floor_mb", b.floorMB)
|
||||||
|
writeStatus(w, http.StatusPreconditionFailed, false, b.refusal("below_usage_floor"),
|
||||||
|
fmt.Sprintf("requested %d MB is too close to current usage %d MB (floor %d MB) — stop applications first", target, b.usageMB, b.floorMB))
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// Apply. SetConfig may be synchronous (empty UPID) or return a task to wait on.
|
||||||
|
upid, err := s.mem.SetConfig(r.Context(), vmid, map[string]string{"memory": fmt.Sprintf("%d", target)})
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Error("local-api: guest-memory SetConfig", "vmid", vmid, "target_mb", target, "err", err)
|
||||||
|
writeErr(w, http.StatusBadGateway, "memory resize failed: "+err.Error())
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if upid != "" {
|
||||||
|
if _, err := s.mem.WaitTask(r.Context(), upid, proxmox.WaitOptions{}); err != nil {
|
||||||
|
s.logger.Error("local-api: guest-memory WaitTask", "vmid", vmid, "upid", upid, "err", err)
|
||||||
|
writeErr(w, http.StatusBadGateway, "memory resize task failed: "+err.Error())
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Verify: re-read and confirm maxmem reflects the target before claiming success (never trust the
|
||||||
|
// POST 200 — the config could have been rejected at task execution, or applied as pending).
|
||||||
|
st, err := s.mem.GuestStatus(r.Context(), vmid)
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Error("local-api: guest-memory post-apply status", "vmid", vmid, "err", err)
|
||||||
|
writeErr(w, http.StatusBadGateway, "resize applied but verification read failed: "+err.Error())
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if st.MaxMem != target*mib {
|
||||||
|
s.logger.Error("local-api: guest-memory resize NOT reflected after apply",
|
||||||
|
"vmid", vmid, "target_mb", target, "observed_maxmem_bytes", st.MaxMem)
|
||||||
|
writeErr(w, http.StatusBadGateway,
|
||||||
|
fmt.Sprintf("resize did not take effect (guest still reports %d MB) — a reboot may be required", st.MaxMem/mib))
|
||||||
|
return
|
||||||
|
}
|
||||||
|
s.logger.Info("local-api: guest-memory resized", "vmid", vmid, "old_mb", b.allocatedMB, "new_mb", target)
|
||||||
|
writeOK(w, memoryResizeResult{VMID: vmid, OldMB: b.allocatedMB, NewMB: target})
|
||||||
|
}
|
||||||
@@ -0,0 +1,281 @@
|
|||||||
|
package localapi
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"net/http"
|
||||||
|
"sync"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"gitea.dooplex.hu/admin/felhom-agent/internal/proxmox"
|
||||||
|
)
|
||||||
|
|
||||||
|
// fakeMemory is a MemoryOps fake: it holds the guest's allocation/usage + host total, counts
|
||||||
|
// SetConfig calls (so refusals can prove the non-effect), and — when applyReflect is set — makes a
|
||||||
|
// SetConfig update the observed maxmem so the post-apply verify read passes.
|
||||||
|
type fakeMemory struct {
|
||||||
|
mu sync.Mutex
|
||||||
|
allocMB int64
|
||||||
|
usageBytes int64
|
||||||
|
maxmemBytes int64
|
||||||
|
hostTotalB int64
|
||||||
|
status string
|
||||||
|
setCalls int
|
||||||
|
lastParams map[string]string
|
||||||
|
setUPID string
|
||||||
|
setErr error
|
||||||
|
applyReflect bool
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeMemory) GuestConfig(_ context.Context, _ int) (proxmox.GuestConfig, error) {
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
return proxmox.GuestConfig{Memory: f.allocMB}, nil
|
||||||
|
}
|
||||||
|
func (f *fakeMemory) GuestStatus(_ context.Context, vmid int) (proxmox.Guest, error) {
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
return proxmox.Guest{VMID: vmid, Mem: f.usageBytes, MaxMem: f.maxmemBytes, Status: f.status}, nil
|
||||||
|
}
|
||||||
|
func (f *fakeMemory) NodeStatus(_ context.Context) (proxmox.NodeStatus, error) {
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
var ns proxmox.NodeStatus
|
||||||
|
ns.Memory.Total = f.hostTotalB
|
||||||
|
return ns, nil
|
||||||
|
}
|
||||||
|
func (f *fakeMemory) SetConfig(_ context.Context, _ int, params map[string]string) (string, error) {
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
f.setCalls++
|
||||||
|
f.lastParams = params
|
||||||
|
if f.setErr != nil {
|
||||||
|
return "", f.setErr
|
||||||
|
}
|
||||||
|
if f.applyReflect {
|
||||||
|
if mb, ok := params["memory"]; ok {
|
||||||
|
var v int64
|
||||||
|
for _, c := range mb {
|
||||||
|
v = v*10 + int64(c-'0')
|
||||||
|
}
|
||||||
|
f.allocMB = v
|
||||||
|
f.maxmemBytes = v * mib
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return f.setUPID, nil
|
||||||
|
}
|
||||||
|
func (f *fakeMemory) WaitTask(_ context.Context, _ string, _ proxmox.WaitOptions) (proxmox.TaskStatus, error) {
|
||||||
|
return proxmox.TaskStatus{Status: "stopped", ExitStatus: "OK"}, nil
|
||||||
|
}
|
||||||
|
func (f *fakeMemory) calls() int { f.mu.Lock(); defer f.mu.Unlock(); return f.setCalls }
|
||||||
|
|
||||||
|
// memServer builds a valid server with the memory fake wired and token "A" → guest 8200.
|
||||||
|
func memServer(t *testing.T, f *fakeMemory) *Server {
|
||||||
|
t.Helper()
|
||||||
|
s := newTestServerS(t, &fakeGuests{}, &fakeBackups{}, &fakeStore{}, nil)
|
||||||
|
s.mem = f
|
||||||
|
return s
|
||||||
|
}
|
||||||
|
|
||||||
|
// stdFixture: allocated 8192 MB, usage 3000 MB, host_total 16384 MB, running.
|
||||||
|
func stdFixture() *fakeMemory {
|
||||||
|
return &fakeMemory{
|
||||||
|
allocMB: 8192,
|
||||||
|
usageBytes: 3000 * mib,
|
||||||
|
maxmemBytes: 8192 * mib,
|
||||||
|
hostTotalB: 16384 * mib,
|
||||||
|
status: "running",
|
||||||
|
applyReflect: true,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
type memResp struct {
|
||||||
|
OK bool `json:"ok"`
|
||||||
|
Data struct {
|
||||||
|
VMID int `json:"vmid"`
|
||||||
|
AllocatedMB int64 `json:"allocated_mb"`
|
||||||
|
UsageMB int64 `json:"usage_mb"`
|
||||||
|
HostTotalMB int64 `json:"host_total_mb"`
|
||||||
|
MinMB int64 `json:"min_mb"`
|
||||||
|
MaxMB int64 `json:"max_mb"`
|
||||||
|
FloorMB int64 `json:"floor_mb"`
|
||||||
|
Running bool `json:"running"`
|
||||||
|
Code string `json:"code"`
|
||||||
|
OldMB int64 `json:"old_mb"`
|
||||||
|
NewMB int64 `json:"new_mb"`
|
||||||
|
Unchanged bool `json:"unchanged"`
|
||||||
|
} `json:"data"`
|
||||||
|
Error string `json:"error"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func decodeMem(t *testing.T, body []byte) memResp {
|
||||||
|
t.Helper()
|
||||||
|
var m memResp
|
||||||
|
if err := json.Unmarshal(body, &m); err != nil {
|
||||||
|
t.Fatalf("decode response %q: %v", body, err)
|
||||||
|
}
|
||||||
|
return m
|
||||||
|
}
|
||||||
|
|
||||||
|
// Scenario C — GET /guest/memory: every field agent-computed, in MB.
|
||||||
|
func TestGuestMemory_GET(t *testing.T) {
|
||||||
|
f := stdFixture()
|
||||||
|
h := memServer(t, f).Handler()
|
||||||
|
w := do(t, h, "GET", "/guest/memory", "A", "")
|
||||||
|
if w.Code != 200 {
|
||||||
|
t.Fatalf("GET = %d (%s), want 200", w.Code, w.Body.String())
|
||||||
|
}
|
||||||
|
d := decodeMem(t, w.Body.Bytes()).Data
|
||||||
|
if d.AllocatedMB != 8192 || d.UsageMB != 3000 || d.HostTotalMB != 16384 ||
|
||||||
|
d.MinMB != 2048 || d.MaxMB != 14336 || d.FloorMB != 3512 || !d.Running {
|
||||||
|
t.Errorf("GET fields wrong: %+v", d)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Scenario A — grow + shrink happy paths: exactly one SetConfig, correct param, success.
|
||||||
|
func TestGuestMemory_GrowAndShrink(t *testing.T) {
|
||||||
|
t.Run("grow", func(t *testing.T) {
|
||||||
|
f := stdFixture()
|
||||||
|
h := memServer(t, f).Handler()
|
||||||
|
w := do(t, h, "POST", "/guest/memory", "A", `{"memory_mb":12288}`)
|
||||||
|
if w.Code != 200 {
|
||||||
|
t.Fatalf("grow = %d (%s), want 200", w.Code, w.Body.String())
|
||||||
|
}
|
||||||
|
d := decodeMem(t, w.Body.Bytes()).Data
|
||||||
|
if d.OldMB != 8192 || d.NewMB != 12288 {
|
||||||
|
t.Errorf("grow result = %+v", d)
|
||||||
|
}
|
||||||
|
if f.calls() != 1 {
|
||||||
|
t.Errorf("SetConfig calls = %d, want 1", f.calls())
|
||||||
|
}
|
||||||
|
if f.lastParams["memory"] != "12288" {
|
||||||
|
t.Errorf("SetConfig param = %q, want 12288", f.lastParams["memory"])
|
||||||
|
}
|
||||||
|
})
|
||||||
|
t.Run("shrink above floor", func(t *testing.T) {
|
||||||
|
f := stdFixture() // floor = max(2048, 3000+512) = 3512
|
||||||
|
h := memServer(t, f).Handler()
|
||||||
|
w := do(t, h, "POST", "/guest/memory", "A", `{"memory_mb":4096}`)
|
||||||
|
if w.Code != 200 {
|
||||||
|
t.Fatalf("shrink = %d (%s), want 200 (4096 >= floor 3512)", w.Code, w.Body.String())
|
||||||
|
}
|
||||||
|
if f.calls() != 1 {
|
||||||
|
t.Errorf("SetConfig calls = %d, want 1", f.calls())
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// Scenario B — the ruled refusals: 412 with the code, and SetConfig NEVER called.
|
||||||
|
func TestGuestMemory_Refusals(t *testing.T) {
|
||||||
|
cases := []struct {
|
||||||
|
name, body, code string
|
||||||
|
}{
|
||||||
|
{"below_min", `{"memory_mb":1024}`, "below_min"},
|
||||||
|
{"above_max", `{"memory_mb":15000}`, "above_max"},
|
||||||
|
{"below_usage_floor", `{"memory_mb":3300}`, "below_usage_floor"},
|
||||||
|
}
|
||||||
|
for _, c := range cases {
|
||||||
|
t.Run(c.name, func(t *testing.T) {
|
||||||
|
f := stdFixture()
|
||||||
|
h := memServer(t, f).Handler()
|
||||||
|
w := do(t, h, "POST", "/guest/memory", "A", c.body)
|
||||||
|
if w.Code != http.StatusPreconditionFailed {
|
||||||
|
t.Fatalf("%s = %d (%s), want 412", c.name, w.Code, w.Body.String())
|
||||||
|
}
|
||||||
|
d := decodeMem(t, w.Body.Bytes()).Data
|
||||||
|
if d.Code != c.code {
|
||||||
|
t.Errorf("code = %q, want %q", d.Code, c.code)
|
||||||
|
}
|
||||||
|
if f.calls() != 0 {
|
||||||
|
t.Errorf("SetConfig called %d times on a refusal — must be 0", f.calls())
|
||||||
|
}
|
||||||
|
// The refusal carries fresh bounds so the UI re-renders honestly.
|
||||||
|
if d.MinMB != 2048 || d.MaxMB != 14336 || d.FloorMB != 3512 {
|
||||||
|
t.Errorf("refusal bounds wrong: min=%d max=%d floor=%d", d.MinMB, d.MaxMB, d.FloorMB)
|
||||||
|
}
|
||||||
|
if c.code == "below_usage_floor" && d.UsageMB != 3000 {
|
||||||
|
t.Errorf("below_usage_floor usage_mb = %d, want 3000", d.UsageMB)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// B4 — cross-guest body vmid → 403, SetConfig not called.
|
||||||
|
func TestGuestMemory_CrossGuestRefused(t *testing.T) {
|
||||||
|
f := stdFixture()
|
||||||
|
h := memServer(t, f).Handler()
|
||||||
|
w := do(t, h, "POST", "/guest/memory", "A", `{"vmid":9999,"memory_mb":10000}`)
|
||||||
|
if w.Code != http.StatusForbidden {
|
||||||
|
t.Fatalf("cross-guest = %d, want 403", w.Code)
|
||||||
|
}
|
||||||
|
if f.calls() != 0 {
|
||||||
|
t.Errorf("SetConfig called on a cross-guest refusal (%d)", f.calls())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// B5 — bounds are re-read FRESH per request: raising usage between the GET and the POST makes a
|
||||||
|
// previously-valid shrink target fall below the NEW floor (a stale UI can't smuggle an old floor).
|
||||||
|
func TestGuestMemory_FreshBoundsPerRequest(t *testing.T) {
|
||||||
|
f := stdFixture() // usage 3000 → floor 3512
|
||||||
|
h := memServer(t, f).Handler()
|
||||||
|
// GET sees floor 3512; target 3600 would be a valid shrink under it.
|
||||||
|
if d := decodeMem(t, do(t, h, "GET", "/guest/memory", "A", "").Body.Bytes()).Data; d.FloorMB != 3512 {
|
||||||
|
t.Fatalf("initial floor = %d, want 3512", d.FloorMB)
|
||||||
|
}
|
||||||
|
// Usage climbs to 3600 MB → floor becomes 4112. The POST must read this FRESH.
|
||||||
|
f.mu.Lock()
|
||||||
|
f.usageBytes = 3600 * mib
|
||||||
|
f.mu.Unlock()
|
||||||
|
w := do(t, h, "POST", "/guest/memory", "A", `{"memory_mb":3700}`)
|
||||||
|
if w.Code != http.StatusPreconditionFailed {
|
||||||
|
t.Fatalf("post = %d (%s), want 412 (fresh floor 4112 > 3700)", w.Code, w.Body.String())
|
||||||
|
}
|
||||||
|
d := decodeMem(t, w.Body.Bytes()).Data
|
||||||
|
if d.Code != "below_usage_floor" || d.FloorMB != 4112 {
|
||||||
|
t.Errorf("fresh-bounds refusal = code %q floor %d, want below_usage_floor / 4112", d.Code, d.FloorMB)
|
||||||
|
}
|
||||||
|
if f.calls() != 0 {
|
||||||
|
t.Errorf("SetConfig called (%d) despite fresh-floor refusal", f.calls())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// target == current allocation → success no-op, no SetConfig.
|
||||||
|
func TestGuestMemory_UnchangedNoOp(t *testing.T) {
|
||||||
|
f := stdFixture()
|
||||||
|
h := memServer(t, f).Handler()
|
||||||
|
w := do(t, h, "POST", "/guest/memory", "A", `{"memory_mb":8192}`)
|
||||||
|
if w.Code != 200 {
|
||||||
|
t.Fatalf("no-op = %d (%s), want 200", w.Code, w.Body.String())
|
||||||
|
}
|
||||||
|
d := decodeMem(t, w.Body.Bytes()).Data
|
||||||
|
if !d.Unchanged || f.calls() != 0 {
|
||||||
|
t.Errorf("no-op should not SetConfig: unchanged=%v calls=%d", d.Unchanged, f.calls())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Verify-after-apply: SetConfig "succeeds" but the maxmem never reflects the target → 502, no false
|
||||||
|
// success (a pending/reboot-required outcome is caught, never claimed as done).
|
||||||
|
func TestGuestMemory_ApplyNotReflected(t *testing.T) {
|
||||||
|
f := stdFixture()
|
||||||
|
f.applyReflect = false // SetConfig returns ok but the guest keeps its old maxmem
|
||||||
|
h := memServer(t, f).Handler()
|
||||||
|
w := do(t, h, "POST", "/guest/memory", "A", `{"memory_mb":10000}`)
|
||||||
|
if w.Code != http.StatusBadGateway {
|
||||||
|
t.Fatalf("unreflected apply = %d (%s), want 502", w.Code, w.Body.String())
|
||||||
|
}
|
||||||
|
if f.calls() != 1 {
|
||||||
|
t.Errorf("SetConfig calls = %d, want 1 (it was attempted)", f.calls())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Not configured (nil Memory) → 503 on both routes.
|
||||||
|
func TestGuestMemory_NotConfigured(t *testing.T) {
|
||||||
|
s := newTestServerS(t, &fakeGuests{}, &fakeBackups{}, &fakeStore{}, nil) // mem stays nil
|
||||||
|
h := s.Handler()
|
||||||
|
if w := do(t, h, "GET", "/guest/memory", "A", ""); w.Code != http.StatusServiceUnavailable {
|
||||||
|
t.Errorf("GET nil-mem = %d, want 503", w.Code)
|
||||||
|
}
|
||||||
|
if w := do(t, h, "POST", "/guest/memory", "A", `{"memory_mb":9000}`); w.Code != http.StatusServiceUnavailable {
|
||||||
|
t.Errorf("POST nil-mem = %d, want 503", w.Code)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -91,6 +91,9 @@ type Options struct {
|
|||||||
// GuestAttach binds an enrolled user-data drive's felhom-data namespace into the guest (slice 10
|
// GuestAttach binds an enrolled user-data drive's felhom-data namespace into the guest (slice 10
|
||||||
// P2, Model A). OPTIONAL — when nil, POST /disks/guest-attach reports "not configured".
|
// P2, Model A). OPTIONAL — when nil, POST /disks/guest-attach reports "not configured".
|
||||||
GuestAttach GuestAttacher
|
GuestAttach GuestAttacher
|
||||||
|
// Memory is the guest-RAM-resize Proxmox surface (v0.90.0, R-24). OPTIONAL — when nil, the
|
||||||
|
// /guest/memory endpoints report "not configured". Satisfied by *proxmox.Client.
|
||||||
|
Memory MemoryOps
|
||||||
// NetStorage is the privileged network-mount (NAS) surface (Part A1). OPTIONAL — when nil, the
|
// NetStorage is the privileged network-mount (NAS) surface (Part A1). OPTIONAL — when nil, the
|
||||||
// /netstorage endpoints report "not configured". Satisfied by *storage.SudoHostOps.
|
// /netstorage endpoints report "not configured". Satisfied by *storage.SudoHostOps.
|
||||||
NetStorage NetworkStorageOps
|
NetStorage NetworkStorageOps
|
||||||
@@ -188,6 +191,8 @@ type Server struct {
|
|||||||
diskGate StorageGate // slice 8C (optional)
|
diskGate StorageGate // slice 8C (optional)
|
||||||
guestList GuestLister // slice 8C (optional)
|
guestList GuestLister // slice 8C (optional)
|
||||||
guestAttach GuestAttacher // slice 10 P2 (optional)
|
guestAttach GuestAttacher // slice 10 P2 (optional)
|
||||||
|
mem MemoryOps // v0.90.0 R-24 guest RAM resize (optional)
|
||||||
|
memMu sync.Mutex // single-flight around a resize apply (one customer per host)
|
||||||
netStorage NetworkStorageOps // Part A1: NAS network mounts (optional)
|
netStorage NetworkStorageOps // Part A1: NAS network mounts (optional)
|
||||||
netMountRoot string // the user-data namespace root for the network-mount role gate
|
netMountRoot string // the user-data namespace root for the network-mount role gate
|
||||||
smbCredsDir string // where SMB creds files are written (out-of-band, 0600)
|
smbCredsDir string // where SMB creds files are written (out-of-band, 0600)
|
||||||
@@ -300,6 +305,7 @@ func NewServer(o Options) (*Server, error) {
|
|||||||
diskGate: o.DiskGate,
|
diskGate: o.DiskGate,
|
||||||
guestList: o.Guests2,
|
guestList: o.Guests2,
|
||||||
guestAttach: o.GuestAttach,
|
guestAttach: o.GuestAttach,
|
||||||
|
mem: o.Memory,
|
||||||
netStorage: o.NetStorage,
|
netStorage: o.NetStorage,
|
||||||
netMountRoot: storage.NetworkMountRoot,
|
netMountRoot: storage.NetworkMountRoot,
|
||||||
smbCredsDir: o.SmbCredsDir,
|
smbCredsDir: o.SmbCredsDir,
|
||||||
@@ -365,6 +371,10 @@ func (s *Server) Handler() http.Handler {
|
|||||||
mux.HandleFunc("POST /disks/guest-attach", s.withGuest(s.handleDiskGuestAttach))
|
mux.HandleFunc("POST /disks/guest-attach", s.withGuest(s.handleDiskGuestAttach))
|
||||||
// Guest reboot (slice 10 P2 activation): user-triggered restart to activate pending drive binds.
|
// Guest reboot (slice 10 P2 activation): user-triggered restart to activate pending drive binds.
|
||||||
mux.HandleFunc("POST /guest/reboot", s.withGuest(s.handleGuestReboot))
|
mux.HandleFunc("POST /guest/reboot", s.withGuest(s.handleGuestReboot))
|
||||||
|
// Guest RAM resize (v0.90.0, R-24): read current allocation + bounds, and apply a bounded resize
|
||||||
|
// (live cgroup apply, no reboot). Self-scoped; the agent enforces every bound fresh per request.
|
||||||
|
mux.HandleFunc("GET /guest/memory", s.withGuest(s.handleGuestMemory))
|
||||||
|
mux.HandleFunc("POST /guest/memory", s.withGuest(s.handleGuestMemoryResize))
|
||||||
|
|
||||||
// Network storage (NAS) — Part A1: mount/list/remove a bulk-media NAS share host-side (automount
|
// Network storage (NAS) — Part A1: mount/list/remove a bulk-media NAS share host-side (automount
|
||||||
// idle-unmount; +100000 uid recipe). A distinct class from a drive — no enroll/eject/wipe.
|
// idle-unmount; +100000 uid recipe). A distinct class from a drive — no enroll/eject/wipe.
|
||||||
|
|||||||
@@ -6,6 +6,7 @@ import (
|
|||||||
"log/slog"
|
"log/slog"
|
||||||
"strconv"
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
|
"sync"
|
||||||
"sync/atomic"
|
"sync/atomic"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
@@ -42,6 +43,31 @@ type Engine struct {
|
|||||||
stateDir string
|
stateDir string
|
||||||
|
|
||||||
opSeq uint64 // atomic; makes each op id unique per attempt
|
opSeq uint64 // atomic; makes each op id unique per attempt
|
||||||
|
|
||||||
|
// lastRes records the most recent successful Reconcile Result (v0.90.0, R-28 fast-tick source).
|
||||||
|
// The fast-tick reads it to decide convergence: actionable drift is Planned − Pending > 0 (a
|
||||||
|
// destructive pending_signature refusal is EXPECTED state, not drift to hammer on). lastOK is
|
||||||
|
// false until the first successful pass.
|
||||||
|
lastMu sync.Mutex
|
||||||
|
lastRes Result
|
||||||
|
lastOK bool
|
||||||
|
}
|
||||||
|
|
||||||
|
// LastResult returns the most recent successful Reconcile Result and whether one has been recorded
|
||||||
|
// (false until the first successful pass). Read by the fast-tick convergence source; safe for
|
||||||
|
// concurrent use.
|
||||||
|
func (e *Engine) LastResult() (Result, bool) {
|
||||||
|
e.lastMu.Lock()
|
||||||
|
defer e.lastMu.Unlock()
|
||||||
|
return e.lastRes, e.lastOK
|
||||||
|
}
|
||||||
|
|
||||||
|
// recordResult stores the latest successful pass Result (called from reconcileOnce).
|
||||||
|
func (e *Engine) recordResult(res Result) {
|
||||||
|
e.lastMu.Lock()
|
||||||
|
e.lastRes = res
|
||||||
|
e.lastOK = true
|
||||||
|
e.lastMu.Unlock()
|
||||||
}
|
}
|
||||||
|
|
||||||
// EngineOptions configures a new Engine. Norm defaults to DefaultNormalizers, Logger
|
// EngineOptions configures a new Engine. Norm defaults to DefaultNormalizers, Logger
|
||||||
@@ -293,6 +319,7 @@ func (e *Engine) reconcileOnce(ctx context.Context) {
|
|||||||
e.logger.Error("reconcile: pass failed", "err", err)
|
e.logger.Error("reconcile: pass failed", "err", err)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
e.recordResult(res) // v0.90.0: fast-tick convergence source
|
||||||
if res.Planned > 0 {
|
if res.Planned > 0 {
|
||||||
e.logger.Info("reconcile: pass complete",
|
e.logger.Info("reconcile: pass complete",
|
||||||
"planned", res.Planned, "executed", res.Executed, "failed", res.Failed, "pending", res.Pending)
|
"planned", res.Planned, "executed", res.Executed, "failed", res.Failed, "pending", res.Pending)
|
||||||
|
|||||||
@@ -0,0 +1,32 @@
|
|||||||
|
package reconcile
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"gitea.dooplex.hu/admin/felhom-agent/internal/proxmox"
|
||||||
|
)
|
||||||
|
|
||||||
|
// LastResult (v0.90.0, R-28 fast-tick source): false until the first successful pass, then it
|
||||||
|
// mirrors the pass Result. The fast-tick reads Planned − Pending > 0 off it.
|
||||||
|
func TestEngine_LastResult(t *testing.T) {
|
||||||
|
api := &fakeAPI{
|
||||||
|
lxc: []proxmox.Guest{{VMID: 100, Status: "running"}},
|
||||||
|
cfg: map[int]proxmox.GuestConfig{100: {Cores: 2}},
|
||||||
|
}
|
||||||
|
e, _, _ := newEngine(t, api, EmptyProvider{})
|
||||||
|
|
||||||
|
if _, ok := e.LastResult(); ok {
|
||||||
|
t.Fatal("LastResult ok=true before any pass, want false")
|
||||||
|
}
|
||||||
|
|
||||||
|
e.reconcileOnce(context.Background())
|
||||||
|
|
||||||
|
res, ok := e.LastResult()
|
||||||
|
if !ok {
|
||||||
|
t.Fatal("LastResult not recorded after a reconcile pass")
|
||||||
|
}
|
||||||
|
if res.Planned != 0 || res.Pending != 0 {
|
||||||
|
t.Errorf("converged pass recorded drift: %+v", res)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -108,6 +108,28 @@ type Manager struct {
|
|||||||
lastResolvedIP netip.Addr // last A-record we rendered into the conf; invalid → resolve on next apply
|
lastResolvedIP netip.Addr // last A-record we rendered into the conf; invalid → resolve on next apply
|
||||||
resolveFailLogged bool // DNS-failure log throttle (reset on the next success)
|
resolveFailLogged bool // DNS-failure log throttle (reset on the next success)
|
||||||
staleSameIPLogged bool // "stale but IP unchanged" log throttle (endpoint-down, not re-IP)
|
staleSameIPLogged bool // "stale but IP unchanged" log throttle (endpoint-down, not re-IP)
|
||||||
|
|
||||||
|
// conv is the CACHED tunnel-convergence snapshot (v0.90.0, R-28 fast-tick source), refreshed at
|
||||||
|
// the end of every Apply (the tunnel's own cadence) so the fast-tick reads it WITHOUT execing
|
||||||
|
// `wg`/`systemctl` per tick. wgBlockDesired = a wireguard block for THIS key is desired;
|
||||||
|
// operational = registered (marker) AND the unit is active. desired && !operational is exactly
|
||||||
|
// the poke-undeliverable window R-28 exists to close.
|
||||||
|
conv convSnapshot
|
||||||
|
}
|
||||||
|
|
||||||
|
// convSnapshot is the cheap, cached read the fast-tick consumes.
|
||||||
|
type convSnapshot struct {
|
||||||
|
wgBlockDesired bool
|
||||||
|
operational bool
|
||||||
|
}
|
||||||
|
|
||||||
|
// TunnelConvergence returns the cached (desired, operational) snapshot without any exec (fast-tick
|
||||||
|
// source). Zero value (false, false) before the first Apply = "nothing desired yet" = converged for
|
||||||
|
// this source (the never-fetched case is covered by the desired-generation source instead).
|
||||||
|
func (m *Manager) TunnelConvergence() (desired, operational bool) {
|
||||||
|
m.mu.Lock()
|
||||||
|
defer m.mu.Unlock()
|
||||||
|
return m.conv.wgBlockDesired, m.conv.operational
|
||||||
}
|
}
|
||||||
|
|
||||||
// NewManager builds a Manager. stateDir is the agent state dir (default /var/lib/felhom-agent —
|
// NewManager builds a Manager. stateDir is the agent state dir (default /var/lib/felhom-agent —
|
||||||
@@ -307,6 +329,20 @@ func (m *Manager) Apply(ctx context.Context, fetched bool, block *hub.WireWiregu
|
|||||||
m.mu.Lock()
|
m.mu.Lock()
|
||||||
defer m.mu.Unlock()
|
defer m.mu.Unlock()
|
||||||
|
|
||||||
|
// Refresh the cached fast-tick convergence snapshot at the end of every Apply (this cadence),
|
||||||
|
// so the fast-tick never execs. Runs under the lock, before the Unlock defer (LIFO), covering
|
||||||
|
// every return path. desired = a wg block for this key is wanted; operational = registered + active.
|
||||||
|
defer func() {
|
||||||
|
desired := fetched && block != nil
|
||||||
|
operational := false
|
||||||
|
if desired {
|
||||||
|
if mk := m.loadMarker(); mk != nil {
|
||||||
|
operational = m.isActive(ctx)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
m.conv = convSnapshot{wgBlockDesired: desired, operational: operational}
|
||||||
|
}()
|
||||||
|
|
||||||
keyExists := false
|
keyExists := false
|
||||||
if _, err := os.Stat(KeyFilePath(m.stateDir)); err == nil {
|
if _, err := os.Stat(KeyFilePath(m.stateDir)); err == nil {
|
||||||
keyExists = true
|
keyExists = true
|
||||||
|
|||||||
Reference in New Issue
Block a user