hub: Restart controller (the agent's closed list on the host-report reply) and Clear hold (D64 = B, the controller's list) on the host page, each behind a confirm; controller_unresponsive (R-1344, alarm only); 04 §6.1 + 09 §3 decision 283; wire gate declares both new wires; R-1344 filed — tests red first
gates / gates (push) Successful in 3m23s
gates / gates (push) Successful in 3m23s
This commit is contained in:
1 parent
70862fa7b3
commit
3d0c907b56
20 files changed
+663
-20
No files matched your search
@@ -183,6 +183,18 @@ the deletion is the hub's), `run_job` (`fill-watch`, `offsite-integrity`, `offsi
|
||||
Anything else is refused at the hub's POST and again on the box. No action deletes data, starts a countdown or
|
||||
shortens one (`TestOpActions_ClosedList`, `TestOperatorActions_ClosedList`). Unanswered after 24 h: expired.
|
||||
|
||||
**`[DESIGN]` 2026-10-29 — D64 = B, and the agent's own list (operator ruling 12:10; `09` §3 decision 283).** The
|
||||
controller's list gains **`clear_hold <app>`** for every hold reason, with no signature: the RUNNING controller clears the
|
||||
hold in memory and saves at once (no later settings save can write it back — R-1343). The operator accepted the cost in
|
||||
words: a compromised hub could release a data-protecting hold and start an app on half-restored data; the hub's confirm
|
||||
step shows the hold's reason and, for a restore or put-back hold, that warning. **R-379 („only a person clears a hold")
|
||||
now reads „a person — on the box, or the operator on the hub".** The hub refuses `clear_hold` for an app the box does not
|
||||
report held now. **The AGENT has its own closed list on the host report's reply — `restart_controller <vmid>` only**
|
||||
(`store.ValidateHostOperatorAction` / agent `localapi.OperatorActionNames`; never a shared verb, `TestHostOperatorActions_ClosedList`):
|
||||
the supervisor's restart with its guards, the 3-in-15-minutes brake counting operator restarts too (then 30 minutes
|
||||
refused), the result in the next report (requested at once), a vmid that is not a guest of that host refused at the hub.
|
||||
Neither verb deletes data, starts a countdown or shortens one. A guest reboot from the hub is NOT on either list.
|
||||
|
||||
### 6.2 Who may press the operator's buttons: session, Basic, or a scoped token [hub v0.152.0, `09` §3 decision 251]
|
||||
|
||||
The hub's operator surface has three ways in: a browser session, Basic auth with the operator password (scripts,
|
||||
|
||||
@@ -1206,6 +1206,14 @@ Wanderer's Meilisearch went 256 → 384 MB and was measured again (it sizes itse
|
||||
`-tags faultinject` test build — never in the release image (`TestD63_TheReleaseBuildHasNoFaultSwitch`). *Operator,
|
||||
2026-10-29 08:17.* `07` §6.3.
|
||||
|
||||
### 2026-10-29 (12:10, operator) — D64 = B: the hub may clear any hold
|
||||
|
||||
283. **D64 = B — the hub's „Clear hold" works for every hold reason, with no operator signature.** The operator accepted
|
||||
that a compromised hub could clear a data-protecting hold (an app could start on half-restored data); the confirm
|
||||
step shows the hold's reason and, for a restore or put-back hold, that warning before the press. R-379 now reads
|
||||
„a person — on the box, or the operator on the hub". Built as the controller's `clear_hold` operator action
|
||||
(`04` §6.1), cleared by the running controller (R-1343). *Operator, 2026-10-29 12:10.*
|
||||
|
||||
### 2026-10-28 (day, attended) — RomM kept; the runtime cache fills itself in CI
|
||||
|
||||
277. **D61 = keep RomM offered** (R-1330) while no RomM release carries Node ≥ 24.22.1; the first one that does is taken
|
||||
|
||||
@@ -293,7 +293,7 @@ stopping line that lies.
|
||||
| **R-1285** | Box system & updates | P4 | **D43's removal of our SSH drop-in from a BYO host happens one bundle LATE: the bundle run that delivers agent 0.165.0 is planned by the box's OLD (0.164.0) wrapper, which writes the drop-in as before („same"), and only then replaces itself.** Measured 2026-10-25 on the BYO test host (VM 9371 on demo-hp, `Tester4-f967d4`; attended release run): 0.164.0 day-0 → signed `agent_update` + `agent_config_update` 0.165.0 → the planted drop-in still there; a second `agent_config_update` (same bundle) → `BUNDLE REMOVED … (ours, on a BYO host — D43)`, nothing written, `sshd -T` the owner's own `permitrootlogin yes` (`audits/release-2026-10-26/byo/byo-proof.txt`). No box is affected today (the fleet's only BYO host is this test host); a fresh BYO install on ≥ 0.165.0 runs the new wrapper from the start. | **VERIFY** — 2026-10-27: agent v0.167.0 RELEASED; in the NORMAL signed delivery the pass now runs right after the commit on all three boxes („running the bundle policy pass after the self-update commit (R-1285)” → `BUNDLE POLICY DONE install_mode=appliance removed=0`; `audits/release-2026-10-27/agent-0167-tester1-demofelhom.txt`). Not live: a BYO box coming FROM 0.164.0 (none in the fleet). **Earlier:** **VERIFY** — 2026-10-26: agent `b1bc63d` (unreleased) — the new agent runs the bundle-policy pass once right after its self-update commits (`selfupdate.SetOnCommitted`); test in the real order (commit, then one pass), red-proved. **Not live:** only a box coming FROM 0.164.0 discriminates (0.165.0's wrapper already applies D43), which needs a signed downgrade chain on a BYO box — the live proof rides the next agent release. **Earlier:** **OPEN** — 2026-10-26 (agent 0.166.0 released, attended run): the policy pass is wired, but in the normal delivery it does NOT run: the box gets `agent_update` and `agent_config_update` in ONE pass, and the bundle job runs in the OLD process (demo-hp: pid 1966780, agent 0.165.0, bundle 09:03:07–09:03:09 CET; the restart into 0.166.0 at 09:03:10) — no `bundle policy pass` line on any of the three boxes. So a BYO host coming from 0.164.0 still loses our drop-in one release late. The night proof (`audits/night-2026-10-25/rig/r1285-live.txt`) ran a bundle under an agent that was ALREADY 0.166.0-code, so it did not cover this order. No BYO host is in the fleet today. Fix options: the agent runs the pass once at start-up after `update committed`; or the release procedure sends the bundle job only after the commit. **Earlier state:** **VERIFY** — fixed on main 2026-10-25 night (agent `abcc1e6`), waits for the agent release + the vouch: the agent runs the wrapper's `bundle-policy` pass after every install (red-proved, `redproof-r1285.txt`). **Live on the rig in ONE signed run** (BYO host, staged as upgraded from 0.164.0: the new agent by hand, the old 0.164.0 wrapper on disk, our drop-in planted; one `agent_config_update` of a test bundle): the old wrapper installed the bundle and replaced itself, then „BUNDLE POLICY REMOVED … (ours, on a BYO host)", `sshd -T` the owner's own `permitrootlogin yes` (`rig/r1285-live.txt`); the test package deleted from the registry after. **Earlier state:** OPEN | — | either the wrapper plans again after it replaced itself (one agent change + a test that a 0.164.0→0.165.0 bundle on a BYO record removes our copy in ONE run, red first), or the release procedure sends a BYO box's config bundle twice; until then a BYO box upgraded from ≤ 0.164.0 keeps our drop-in until its next bundle | CC |
|
||||
|
||||
|
||||
## Monitoring & notifications — 24 rows (P2 2, P3 14, P4 8)
|
||||
## Monitoring & notifications — 25 rows (P2 2, P3 15, P4 8)
|
||||
|
||||
| ID | Category | Sev | What | State | Blocked on | Next action | Owner |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
@@ -321,6 +321,7 @@ stopping line that lies.
|
||||
| **R-1328** | Monitoring & notifications | P2 | **DooPlex's own workloads were not in the security watch at all** — about ten public apps there carry a bad Brotli/Node (HTTP2Shell), on the host that runs the hub, Gitea, the registry and the backups. | **VERIFY** — BUILT 2026-10-27 night (commit only): `scripts/dooplex_inventory.py` lists every `image:` of homelab-manifests (89 images, `documentation/security-watch/dooplex-workloads.txt`; NOT Helm-chart images — said in the file); the watch reads them as the group „dooplex" (operator mails only, own block, „Next: the homelab session"); an image in the private registry is NOT CHECKED with the reason. Tests W12, red-proved. The homelab session's list: `documentation/audits/http2shell-2026-10-27/HOMELAB-LIST.md`. Felhom changes nothing on DooPlex. | — | the first CI night shows the DooPlex block; the homelab session works its list. 2026-10-28: `minio/minio:latest` answers HTTP 401 on Docker Hub (the image is no longer published there) — for the homelab session: the pin cannot be checked anonymously any more | CC |
|
||||
| **R-1341** | Monitoring & notifications | P4 | **The OOM scan logs „container X was OOM-killed" every 30 s for as long as the container lives, though it sends the event once.** Measured 2026-10-29 on Tester 1 (controller 0.347.0, agent 0.168.0, R-528's proof): one kill → one `app_oom` and one operator mail (right), and the `[WARN] [deadapp]` line at 09:02:32, :03:02, :03:32, :04:02, :04:32 … — the same as the night's proof (`audits/night-2026-10-28/r528-oom/`). The loop (`controller/cmd/controller/main.go:1100`, since v0.243.0, R-514) logs every container `ScanOOMKilled` returns, every scan; the notifier's cooldown keeps the event to one. About 2 880 lines a day per OOM-marked container until it restarts — noise that hides the next real line. Fix: log once per (container, start time, kill count), with a test that a second scan of the same kill logs nothing and a new kill logs again. Evidence: `audits/d63-2026-10-29/run-p4-oom.log`. | OPEN | — | the fix above, red first | CC |
|
||||
| **R-528** | Monitoring & notifications | P2 | **[P2-MEDIUM] Docker does not report an OOM kill inside a Felhom LXC guest: `OOMKilled` stays false and no `oom` event fires, so the v0.243.0 OOM line is not proven live.** MEASURED 2026-09-15 on scratch 9202 (Docker 29.8.0): Paperless capped at 128M restarted 11 times with `OOMKilled=false` and zero `docker events --filter event=oom`; a memory hog inside the running container was killed (rc 137) with the same silence (`E2-oom-signal-measure-9202.txt`). BIGNIGHT VM 333 did read `oomkilled=true`, so the shape differs by case. **Fix shape:** the agent reads the guest container cgroups' `memory.events oom_kill` counters (host-side, reliable), or the controller alarms on a restart-count trend **RE-MEASURED 2026-09-16 on the DRILL box (fresh install, nested VM 334, Docker in an LXC guest, controller 0.243.0), so the finding is not a property of one machine:** the Paperless webserver was capped at 128 M with `docker update --memory`; it restarted 9-10 times, and all three signals stayed silent - `OOMKilled=false` on every inspect, `docker events --filter event=oom` EMPTY for the whole window, the container's cgroup not visible from inside the guest, and `dmesg` unreadable there. Identical to scratch 9202. So the v0.243.0 OOM line cannot fire on ANY Felhom box as shipped, on either host. Evidence: `audits/evidence-drill-0243-2026-09-16/phase2-m1-oom.txt`. | WATCHING — **released 2026-10-29:** agent 0.168.0 (vouched 10:21:56 CET) + controller 0.347.0 (the global floor). Proven on the release on Tester 1: a hog in a 128 MB Gokapi — host counter 1, one `app_oom` „seen by docker,cgroup", one operator mail (`audits/d63-2026-10-29/run-p4-oom.log`; the repeated WARN is R-1341). Closes at the dated check 2026-11-12 if it holds. | — | 2026-11-12: read the hub for any failed `oom_check` or an `app_oom` with no restart reason since 2026-10-12; none → close | CC |
|
||||
| **R-1344** | Monitoring & notifications | P3 | **A controller that RUNS but does not answer is not restarted by anything — only a stopped one is (R-523).** Operator brief 2026-10-29, Part 2: the agent may restart it only after a measurement shows a busy night (backups, updates) never looks „not answering". Built 2026-10-29 as ALARM ONLY (agent `637c92c`: 6 consecutive sweeps of Docker's health check „unhealthy" — on top of Docker's own 3 failed probes, ~4.5 min in all — set `unresponsive_since`; the hub mails `controller_unresponsive`, operator-only). The measurement: Docker's own probe record (`curl -f localhost:8080/api/health`, every 30 s, timeout 5 s) logged every 30 s on Tester 1 and demo-hp from 2026-10-29 11:14 UTC to 2026-10-30 07:00 UTC — through tonight's real night chain (a day run cannot hold the whole-guest vzdump). | OPEN — measuring | — | read the measurement (worst probe times, any 5-s timeout, any „unhealthy"); if none fired: switch the alarm to a restart (same guards and brake, `controller_unresponsive_restart`) | CC |
|
||||
|
||||
## Hub & operator — 12 rows (P3 3, P4 9)
|
||||
|
||||
|
||||
@@ -1,3 +1,23 @@
|
||||
## Unreleased (2026-10-29, afternoon) — „Restart controller" and „Clear hold" on the host page (D64 = B); controller_unresponsive (R-1344) — commit only; rides the next hub release
|
||||
|
||||
- **The AGENT's operator actions (operator brief 2026-10-29, Part 1; `04` §6.1).** `operator_actions` gains a `host_id`
|
||||
column: a row with a host id is the AGENT's (the host-report reply lists it as `operator_actions`; the agent answers
|
||||
`operator_action_results` on its next report, closed per id AND host, one `operator_action` event each); a row without
|
||||
is the controller's, as before. The agent's CLOSED list is `restart_controller <vmid>` only
|
||||
(`store.ValidateHostOperatorAction`), and a vmid must be a guest of THAT host. The host page's guest table has
|
||||
„Restart controller" behind a confirm step that names the box; the press pokes the agent (seconds over the tunnel; the
|
||||
15-min cycle otherwise). Tests `TestHostOperatorActions_*` (store), `TestHostReport_AgentOperatorActionRoundTrip` (api),
|
||||
`TestHostPage_RestartAndClearHold*` (web) — red first; the host-row filter and both web guards red by mutation.
|
||||
- **D64 = B (operator 2026-10-29 12:10) — `clear_hold` joins the controller's closed list**, for every hold reason, no
|
||||
signature. The Held apps list has „Clear hold" per row behind a confirm step showing the app, the reason and since when;
|
||||
a restore or put-back hold adds: „This hold protects data from a failed restore. If you clear it, the app may start on
|
||||
half-restored data. Check the restore copy first." Refused (400, no row) for an app the box does not report held now.
|
||||
„Who may clear it" now names the operator here too. `TestOperatorActions_ClosedList` updated deliberately.
|
||||
- **R-1344 — `controller_unresponsive`** (warning, operator-only, hub-generated like the crash-loop pair) when a guest's
|
||||
`unresponsive_since` moves in the agent's `controller_supervisor` stanza (also at first sight while unresponsive). The
|
||||
agent only alarms; it does not restart for it yet. Tests `TestControllerSupervisorChecker_Unresponsive`,
|
||||
`TestControllerUnresponsive_RegisteredAndOperatorOnly` — red first.
|
||||
|
||||
## v0.175.0 (2026-10-29) — R-1255: two escrow edges (review round 2, HUNT-3 H3/H4); D62: the restore-copy delete event — DEPLOYED on the operator's yes (image sha256:642e94b6…, from `bf69bb91`, CI 2487)
|
||||
|
||||
- **D62 (operator 2026-10-29) — `restore_copy_deleted` is accepted and operator-only.** The controller's „Törlés most"
|
||||
|
||||
@@ -809,6 +809,16 @@ type hostReportPayload struct {
|
||||
DeliberateOff *struct {
|
||||
RequestedAt string `json:"requested_at"`
|
||||
} `json:"deliberate_off"`
|
||||
// OperatorActionResults (2026-10-29, the agent's operator actions): the answers to the previous replies'
|
||||
// operator_actions — the controller's shape. Absent on an older agent.
|
||||
OperatorActionResults []hostOperatorActionResult `json:"operator_action_results"`
|
||||
}
|
||||
|
||||
// hostOperatorActionResult mirrors the agent's hub.OperatorActionResult.
|
||||
type hostOperatorActionResult struct {
|
||||
ID int64 `json:"id"`
|
||||
Outcome string `json:"outcome"`
|
||||
Message string `json:"message"`
|
||||
}
|
||||
|
||||
// hostForeignKeyArchives mirrors the agent's hub.ForeignKeyArchives (R-366 slice 2).
|
||||
@@ -1133,6 +1143,14 @@ func (h *Handler) handleHostReport(w http.ResponseWriter, r *http.Request) {
|
||||
if pending, err := h.store.PendingLogBundleRequest(hostID, store.LogBundleComponentAgent); err == nil && pending {
|
||||
resp["log_tail_requested"] = true
|
||||
}
|
||||
// The agent's operator actions (2026-10-29): the results this report carries close their rows (one event each);
|
||||
// what is still pending rides this reply. A store error drops nothing from the heartbeat.
|
||||
h.ingestHostOperatorActionResults(hostID, custID, rep.OperatorActionResults)
|
||||
if acts, err := h.store.PendingHostOperatorActions(hostID); err != nil {
|
||||
h.logger.Printf("[WARN] host operator actions for %s: %v", hostID, err)
|
||||
} else if len(acts) > 0 {
|
||||
resp["operator_actions"] = acts
|
||||
}
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.WriteHeader(http.StatusOK)
|
||||
json.NewEncoder(w).Encode(resp)
|
||||
@@ -2400,6 +2418,8 @@ var allowedEventTypes = map[string]bool{
|
||||
"controller_crashloop": true,
|
||||
// R-539 (hub v0.117.0, agent v0.132.0) — hub-generated from slow_crashloop_since; operator-only.
|
||||
"controller_slow_crashloop": true,
|
||||
// R-1344 (2026-10-29): hub-generated from the stanza's unresponsive_since; operator-only (notify.operatorOnlyEvents).
|
||||
"controller_unresponsive": true,
|
||||
// R-550 (hub v0.117.0, controller v0.246.0) — pushed by the controller at startup when it finds a
|
||||
// restore that was in flight when the box stopped. For the household: they can run it again.
|
||||
"restore_interrupted": true,
|
||||
|
||||
@@ -0,0 +1,86 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"gitea.dooplex.hu/admin/felhom-hub/internal/notify"
|
||||
"gitea.dooplex.hu/admin/felhom-hub/internal/store"
|
||||
)
|
||||
|
||||
// Operator brief 2026-10-29, Part 1: the AGENT's operator actions ride the host-report reply as `operator_actions`
|
||||
// (the controller's shape) and its answer rides the next host report as `operator_action_results`; the row closes and
|
||||
// one `operator_action` event is stored. A controller row never reaches the agent. RED-PROOF: seen failing before the
|
||||
// reply listed the agent rows (no `operator_actions` key).
|
||||
func TestHostReport_AgentOperatorActionRoundTrip(t *testing.T) {
|
||||
h, st, _ := newTestHandler(t)
|
||||
st.UpsertHost(&store.Host{HostID: "h1", CustomerID: "c1", APIKey: "HKEY"})
|
||||
if _, err := st.CreateOperatorAction("c1", "clear_hold", "radicale", "operator"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
id, err := st.CreateHostOperatorAction("c1", "h1", "restart_controller", "9201", "operator browser session from 192.168.0.5")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
rr := do(h, http.MethodPost, "/host-report", "HKEY", validReportBody("h1"))
|
||||
var env struct {
|
||||
OperatorActions []store.OperatorActionDirective `json:"operator_actions"`
|
||||
}
|
||||
if err := json.Unmarshal(rr.Body.Bytes(), &env); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(env.OperatorActions) != 1 || env.OperatorActions[0].ID != id || env.OperatorActions[0].Action != "restart_controller" ||
|
||||
env.OperatorActions[0].Arg != "9201" {
|
||||
t.Fatalf("the agent's reply lists %+v, want only the restart #%d (never the controller's clear_hold)", env.OperatorActions, id)
|
||||
}
|
||||
body := strings.Replace(validReportBody("h1"), `"audit_tail":[]`,
|
||||
`"audit_tail":[],"operator_action_results":[{"id":`+i64s(id)+`,"outcome":"done","message":"the controller answered after 14 s"}]`, 1)
|
||||
if rr := do(h, http.MethodPost, "/host-report", "HKEY", body); rr.Code != http.StatusOK {
|
||||
t.Fatalf("result heartbeat = %d %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
if left, _ := st.PendingHostOperatorActions("h1"); len(left) != 0 {
|
||||
t.Fatalf("the row is still pending: %+v", left)
|
||||
}
|
||||
rows, _ := st.ListOperatorActions("c1", 10)
|
||||
var got *store.OperatorActionRow
|
||||
for i := range rows {
|
||||
if rows[i].ID == id {
|
||||
got = &rows[i]
|
||||
}
|
||||
}
|
||||
if got == nil || got.Outcome != "done" || got.Message != "the controller answered after 14 s" {
|
||||
t.Fatalf("row = %+v", got)
|
||||
}
|
||||
evs, _ := st.GetEventsByType("c1", "operator_action", time.Time{})
|
||||
n := 0
|
||||
for _, e := range evs {
|
||||
if e.EventType == "operator_action" && strings.Contains(e.Message, "restart_controller 9201") {
|
||||
n++
|
||||
}
|
||||
}
|
||||
if n != 1 {
|
||||
t.Fatalf("operator_action events for the restart = %d, want 1: %+v", n, evs)
|
||||
}
|
||||
// once answered, the reply stops listing it
|
||||
rr = do(h, http.MethodPost, "/host-report", "HKEY", validReportBody("h1"))
|
||||
env.OperatorActions = nil
|
||||
_ = json.Unmarshal(rr.Body.Bytes(), &env)
|
||||
if len(env.OperatorActions) != 0 {
|
||||
t.Fatalf("still listed after its result: %+v", env.OperatorActions)
|
||||
}
|
||||
}
|
||||
|
||||
func i64s(n int64) string { b, _ := json.Marshal(n); return string(b) }
|
||||
|
||||
// R-1344: the hub-generated controller_unresponsive is listed (the crash-loop pair's rule) and operator-only.
|
||||
func TestControllerUnresponsive_RegisteredAndOperatorOnly(t *testing.T) {
|
||||
if !allowedEventTypes["controller_unresponsive"] {
|
||||
t.Error("controller_unresponsive is not in allowedEventTypes")
|
||||
}
|
||||
if !notify.IsOperatorOnly("controller_unresponsive") {
|
||||
t.Error("controller_unresponsive is not operator-only")
|
||||
}
|
||||
}
|
||||
@@ -55,3 +55,33 @@ func argSuffix(arg string) string {
|
||||
}
|
||||
return " " + arg
|
||||
}
|
||||
|
||||
// ingestHostOperatorActionResults closes each answered AGENT row (matched on the id AND this host) and saves one
|
||||
// hub-minted operator_action event per closed row — the controller's rule, on the agent's channel.
|
||||
func (h *Handler) ingestHostOperatorActionResults(hostID, customerID string, results []hostOperatorActionResult) {
|
||||
for _, res := range results {
|
||||
row, err := h.store.RecordHostOperatorActionResult(hostID, res.ID, res.Outcome, res.Message)
|
||||
switch {
|
||||
case err != nil:
|
||||
h.logger.Printf("[WARN] host operator action #%d result from %s not recorded: %v", res.ID, hostID, err)
|
||||
continue
|
||||
case row == nil:
|
||||
h.logger.Printf("[DEBUG] host operator action #%d result from %s ignored (already closed, or not this host's)", res.ID, hostID)
|
||||
continue
|
||||
}
|
||||
h.logger.Printf("[INFO] host operator action #%d %s%s on %s (pressed by %s): %s — %s",
|
||||
row.ID, row.Action, argSuffix(row.Arg), hostID, row.RequestedBy, row.Outcome, row.Message)
|
||||
if customerID == "" {
|
||||
continue
|
||||
}
|
||||
severity := "info"
|
||||
if row.Outcome != store.OperatorActionDone {
|
||||
severity = "warning"
|
||||
}
|
||||
if _, eerr := h.store.SaveEvent(customerID, "operator_action", severity,
|
||||
fmt.Sprintf("Operator action %s%s on host %s (pressed by %s): %s — %s", row.Action, argSuffix(row.Arg), hostID, row.RequestedBy, row.Outcome, row.Message),
|
||||
"", "hub"); eerr != nil {
|
||||
h.logger.Printf("[WARN] host operator action #%d recorded, but its event could not be saved: %v", row.ID, eerr)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -42,6 +42,7 @@ type supSeen struct {
|
||||
lastRestartAt string
|
||||
crashloopSince string
|
||||
slowCrashloopSince string
|
||||
unresponsiveSince string
|
||||
}
|
||||
|
||||
// ControllerSupervisorGuest mirrors the agent's hub.ControllerSupervisorGuest wire shape.
|
||||
@@ -57,6 +58,10 @@ type ControllerSupervisorGuest struct {
|
||||
Restarts24h int `json:"restarts_24h"`
|
||||
SlowCrashloop bool `json:"slow_crashloop"`
|
||||
SlowCrashloopSince string `json:"slow_crashloop_since"`
|
||||
// R-1344 (agent ≥ the 2026-10-29 release): the controller runs but Docker's health check has failed for several
|
||||
// sweeps. ALARM ONLY — no restart until the measurement licenses one. Absent on older agents → never an event.
|
||||
Unresponsive bool `json:"unresponsive"`
|
||||
UnresponsiveSince string `json:"unresponsive_since"`
|
||||
}
|
||||
|
||||
// ParseControllerSupervisor extracts the stanza from a host report body. A missing or malformed
|
||||
@@ -98,7 +103,8 @@ func (c *ControllerSupervisorChecker) observe(hostID, customerID string, guests
|
||||
for _, g := range guests {
|
||||
key := fmt.Sprintf("%s/%d", hostID, g.VMID)
|
||||
prev, known := c.seen[key]
|
||||
c.seen[key] = supSeen{lastRestartAt: g.LastRestartAt, crashloopSince: g.CrashloopSince, slowCrashloopSince: g.SlowCrashloopSince}
|
||||
c.seen[key] = supSeen{lastRestartAt: g.LastRestartAt, crashloopSince: g.CrashloopSince, slowCrashloopSince: g.SlowCrashloopSince,
|
||||
unresponsiveSince: g.UnresponsiveSince}
|
||||
if !known {
|
||||
if g.Crashloop && g.CrashloopSince != "" {
|
||||
c.emit(customerID, hostID, g, "controller_crashloop")
|
||||
@@ -106,6 +112,9 @@ func (c *ControllerSupervisorChecker) observe(hostID, customerID string, guests
|
||||
if g.SlowCrashloop && g.SlowCrashloopSince != "" {
|
||||
c.emit(customerID, hostID, g, "controller_slow_crashloop")
|
||||
}
|
||||
if g.Unresponsive && g.UnresponsiveSince != "" {
|
||||
c.emit(customerID, hostID, g, "controller_unresponsive")
|
||||
}
|
||||
continue
|
||||
}
|
||||
if g.LastRestartAt != "" && g.LastRestartAt != prev.lastRestartAt {
|
||||
@@ -117,6 +126,9 @@ func (c *ControllerSupervisorChecker) observe(hostID, customerID string, guests
|
||||
if g.SlowCrashloopSince != "" && g.SlowCrashloopSince != prev.slowCrashloopSince {
|
||||
c.emit(customerID, hostID, g, "controller_slow_crashloop")
|
||||
}
|
||||
if g.UnresponsiveSince != "" && g.UnresponsiveSince != prev.unresponsiveSince {
|
||||
c.emit(customerID, hostID, g, "controller_unresponsive")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -135,6 +147,10 @@ func (c *ControllerSupervisorChecker) emit(customerID, hostID string, g Controll
|
||||
severity = "warning"
|
||||
message = fmt.Sprintf("Host %s guest %d: the controller keeps dying — the agent restarted it %d times in 24 hours (each one too far apart for the 15-minute brake). It is still being restarted; this is the warning that it will not stay up. Last reason: %s",
|
||||
hostID, g.VMID, g.Restarts24h, g.LastReason)
|
||||
case "controller_unresponsive":
|
||||
severity = "warning"
|
||||
message = fmt.Sprintf("Host %s guest %d: the controller runs but does not answer its health check (since %s). The agent does NOT restart it yet (R-1344: the measurement must first show a busy night never looks like this) — use „Restart controller\" on the host page if it stays so.",
|
||||
hostID, g.VMID, g.UnresponsiveSince)
|
||||
default:
|
||||
return
|
||||
}
|
||||
@@ -143,6 +159,7 @@ func (c *ControllerSupervisorChecker) emit(customerID, hostID string, g Controll
|
||||
"last_restart_at": g.LastRestartAt, "last_reason": g.LastReason,
|
||||
"crashloop_since": g.CrashloopSince, "parked": g.Parked,
|
||||
"restarts_24h": g.Restarts24h, "slow_crashloop_since": g.SlowCrashloopSince,
|
||||
"unresponsive_since": g.UnresponsiveSince,
|
||||
})
|
||||
c.logger.Printf("[INFO] Controller supervisor: %s %s/%d (%s)", eventType, hostID, g.VMID, g.LastReason)
|
||||
if _, err := c.store.SaveEvent(customerID, eventType, severity, message, string(details), "hub"); err != nil {
|
||||
|
||||
@@ -146,3 +146,32 @@ func TestControllerSupervisorChecker_SlowCrashloopAtFirstSight(t *testing.T) {
|
||||
t.Fatalf("first sight in a slow crash loop: got %v, want exactly [controller_slow_crashloop/warning]", evs)
|
||||
}
|
||||
}
|
||||
|
||||
// R-1344 (operator brief 2026-10-29, Part 2 — ALARM ONLY until the measurement licenses a restart): a controller that
|
||||
// runs but does not answer reaches the operator once per new unresponsive_since, as controller_unresponsive (warning),
|
||||
// also at first sight; an answering one never. RED-PROOF: seen failing before the branch existed.
|
||||
func TestControllerSupervisorChecker_Unresponsive(t *testing.T) {
|
||||
st := newCapStore(t)
|
||||
rec := &evRec{}
|
||||
c := NewControllerSupervisorChecker(st, rec.fn, log.New(io.Discard, "", 0))
|
||||
st.SaveHostReport("h1", "c1", supReport(`[{"vmid":9201,"restarts_total":0,"crashloop":false,"parked":false,"unresponsive":false}]`), store.HostReportDenorm{})
|
||||
c.Check()
|
||||
if evs := rec.take(); len(evs) != 0 {
|
||||
t.Fatalf("an answering controller emitted %v", evs)
|
||||
}
|
||||
st.SaveHostReport("h1", "c1", supReport(`[{"vmid":9201,"restarts_total":0,"crashloop":false,"parked":false,"unresponsive":true,"unresponsive_since":"2026-10-29T13:00:00Z"}]`), store.HostReportDenorm{})
|
||||
c.Check()
|
||||
if evs := rec.take(); len(evs) != 1 || evs[0] != "controller_unresponsive/warning" {
|
||||
t.Fatalf("got %v, want exactly [controller_unresponsive/warning]", evs)
|
||||
}
|
||||
c.Check()
|
||||
if evs := rec.take(); len(evs) != 0 {
|
||||
t.Fatalf("the same unresponsive_since emitted again: %v", evs)
|
||||
}
|
||||
// a hub restarted while it is unresponsive says so once
|
||||
c2 := NewControllerSupervisorChecker(st, rec.fn, log.New(io.Discard, "", 0))
|
||||
c2.Check()
|
||||
if evs := rec.take(); len(evs) != 1 || evs[0] != "controller_unresponsive/warning" {
|
||||
t.Fatalf("first sight while unresponsive: %v", evs)
|
||||
}
|
||||
}
|
||||
@@ -58,7 +58,7 @@ func HeldAppWhoClears(reason string) string {
|
||||
case "putback_pending":
|
||||
return "nobody — it clears itself when the drive with the pre-restore copy comes back"
|
||||
default:
|
||||
return "a person only (support: the controller's -clear-restore-hold)"
|
||||
return "a person — support on the box (--clear-restore-hold) or the operator here (Clear hold, D64)"
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -744,6 +744,8 @@ var recordOnlyEvents = map[string]bool{
|
||||
// is customer-grade and would be DISCARDED by a template). Turning the fallback into a gate would
|
||||
// change behaviour well outside this concern.
|
||||
var operatorOnlyEvents = map[string]bool{
|
||||
// R-1344 (2026-10-29): the controller runs but does not answer (agent stanza, alarm only) — the operator's.
|
||||
"controller_unresponsive": true,
|
||||
// R-97a. A customer can take no action on a failed whole-guest backup, and being told it failed
|
||||
// while it is still retrying behind the R-88 breaker is alarming without being actionable.
|
||||
"whole_guest_backup_failed": true,
|
||||
|
||||
@@ -3,8 +3,10 @@ package store
|
||||
import (
|
||||
"database/sql"
|
||||
"fmt"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
@@ -28,6 +30,14 @@ const (
|
||||
OperatorActionAbandonStop = "abandon_stop"
|
||||
OperatorActionAbandonExtend = "abandon_extend"
|
||||
OperatorActionRunJob = "run_job"
|
||||
// OperatorActionClearHold (D64 = B, operator 2026-10-29 12:10; `04` §6.1): the RUNNING controller clears one app's
|
||||
// hold (any reason) and saves at once. No signature: the operator accepted that a compromised hub could release a
|
||||
// data-protecting hold; the hub shows the hold's reason and a warning before the press.
|
||||
OperatorActionClearHold = "clear_hold"
|
||||
|
||||
// HostOperatorActionRestartController (operator brief 2026-10-29, Part 1): the AGENT restarts a guest's controller
|
||||
// (the supervisor's own act and guards). The agent's list holds this one verb.
|
||||
HostOperatorActionRestartController = "restart_controller"
|
||||
|
||||
// Outcomes: the box's three, plus two the hub sets itself.
|
||||
OperatorActionDone = "done"
|
||||
@@ -50,7 +60,35 @@ const (
|
||||
operatorActionsListed = 10
|
||||
)
|
||||
|
||||
var operatorActionNames = []string{OperatorActionOffsiteBackupNow, OperatorActionAbandonStop, OperatorActionAbandonExtend, OperatorActionRunJob}
|
||||
var operatorActionNames = []string{OperatorActionOffsiteBackupNow, OperatorActionAbandonStop, OperatorActionAbandonExtend, OperatorActionRunJob, OperatorActionClearHold}
|
||||
|
||||
// hostOperatorActionNames is the AGENT's closed list (the host report's reply). Pinned by
|
||||
// TestHostOperatorActions_ClosedList; never shares a verb with the controller's list.
|
||||
var hostOperatorActionNames = []string{HostOperatorActionRestartController}
|
||||
|
||||
// stackNameRe is an app's stack name as the controller names it (a catalog slug).
|
||||
var stackNameRe = regexp.MustCompile(`^[a-z0-9][a-z0-9_-]{0,63}$`)
|
||||
|
||||
// HostOperatorActionNames returns the agent's closed list (sorted copy).
|
||||
func HostOperatorActionNames() []string { return sortedStrings(hostOperatorActionNames) }
|
||||
|
||||
// IsHostOperatorAction reports whether the verb belongs to the agent's list.
|
||||
func IsHostOperatorAction(action string) bool { return inList(hostOperatorActionNames, action) }
|
||||
|
||||
// ValidateHostOperatorAction refuses anything outside the agent's closed list. restart_controller takes the guest's
|
||||
// vmid (Proxmox: 100–999999999).
|
||||
func ValidateHostOperatorAction(action, arg string) error {
|
||||
switch action {
|
||||
case HostOperatorActionRestartController:
|
||||
v, err := strconv.Atoi(arg)
|
||||
if err != nil || strconv.Itoa(v) != arg || v < 100 || v > 999999999 {
|
||||
return fmt.Errorf("the guest id must be a number 100-999999999")
|
||||
}
|
||||
default:
|
||||
return fmt.Errorf("unknown host action %q", action)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// operatorJobNames is the fixed set run_job may name — the controller's scheduler job names.
|
||||
var operatorJobNames = []string{"fill-watch", "offsite-integrity", "offsite-proof", "disk-health-check"}
|
||||
@@ -93,6 +131,10 @@ func ValidateOperatorAction(action, arg string) error {
|
||||
if !inList(operatorJobNames, arg) {
|
||||
return fmt.Errorf("unknown job %q", arg)
|
||||
}
|
||||
case OperatorActionClearHold:
|
||||
if !stackNameRe.MatchString(arg) {
|
||||
return fmt.Errorf("not an app name: %q", arg)
|
||||
}
|
||||
default:
|
||||
return fmt.Errorf("unknown action %q", action)
|
||||
}
|
||||
@@ -112,6 +154,7 @@ type OperatorActionDirective struct {
|
||||
type OperatorActionRow struct {
|
||||
ID int64
|
||||
CustomerID string
|
||||
HostID string // "" = a controller action; else the agent of this host acts
|
||||
Action string
|
||||
Arg string
|
||||
RequestedAt time.Time
|
||||
@@ -135,7 +178,83 @@ func (s *Store) migrateOperatorActions() error {
|
||||
message TEXT NOT NULL DEFAULT ''
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_operator_actions_customer ON operator_actions(customer_id, done_at);`)
|
||||
return err
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
// The agent's rows (2026-10-29): host_id names the host whose agent acts; "" = the controller's row.
|
||||
if _, err := s.db.Exec(`ALTER TABLE operator_actions ADD COLUMN host_id TEXT NOT NULL DEFAULT ''`); err != nil &&
|
||||
!strings.Contains(err.Error(), "duplicate column") {
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// CreateHostOperatorAction validates against the AGENT's closed list and stores a pending row for hostID (listed on the
|
||||
// customer's host page beside the controller's rows).
|
||||
func (s *Store) CreateHostOperatorAction(customerID, hostID, action, arg, by string) (int64, error) {
|
||||
if hostID == "" {
|
||||
return 0, fmt.Errorf("host operator action: empty host id")
|
||||
}
|
||||
if err := ValidateHostOperatorAction(action, arg); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
res, err := s.db.Exec(`INSERT INTO operator_actions (customer_id, host_id, action, arg, requested_at, requested_by)
|
||||
VALUES (?, ?, ?, ?, ?, ?)`, customerID, hostID, action, arg, time.Now().UTC(), by)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return res.LastInsertId()
|
||||
}
|
||||
|
||||
// PendingHostOperatorActions is what the next host-report reply lists for hostID's agent, oldest first; rows older
|
||||
// than OperatorActionTTL are closed as expired first.
|
||||
func (s *Store) PendingHostOperatorActions(hostID string) ([]OperatorActionDirective, error) {
|
||||
now := time.Now().UTC()
|
||||
if _, err := s.db.Exec(`UPDATE operator_actions SET done_at = ?, outcome = ?, message = ?
|
||||
WHERE host_id = ? AND done_at IS NULL AND requested_at < ?`,
|
||||
now, OperatorActionExpired, "the agent did not answer within a day", hostID, now.Add(-OperatorActionTTL)); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
rows, err := s.db.Query(`SELECT id, action, arg FROM operator_actions
|
||||
WHERE host_id = ? AND done_at IS NULL ORDER BY id LIMIT ?`, hostID, operatorActionsListed)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
var out []OperatorActionDirective
|
||||
for rows.Next() {
|
||||
var d OperatorActionDirective
|
||||
if err := rows.Scan(&d.ID, &d.Action, &d.Arg); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, d)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
// RecordHostOperatorActionResult closes a pending AGENT row with the agent's result — matched on the id AND the
|
||||
// reporting host, so another host's (or the controller's) result changes nothing.
|
||||
func (s *Store) RecordHostOperatorActionResult(hostID string, id int64, outcome, message string) (*OperatorActionRow, error) {
|
||||
switch outcome {
|
||||
case OperatorActionDone, OperatorActionRefused, OperatorActionFailed:
|
||||
default:
|
||||
return nil, fmt.Errorf("unknown outcome %q", outcome)
|
||||
}
|
||||
if hostID == "" {
|
||||
return nil, nil
|
||||
}
|
||||
if r := []rune(message); len(r) > operatorActionMessageMax {
|
||||
message = string(r[:operatorActionMessageMax])
|
||||
}
|
||||
res, err := s.db.Exec(`UPDATE operator_actions SET done_at = ?, outcome = ?, message = ?
|
||||
WHERE id = ? AND host_id = ? AND done_at IS NULL`, time.Now().UTC(), outcome, message, id, hostID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if n, _ := res.RowsAffected(); n == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
return s.getOperatorAction(id)
|
||||
}
|
||||
|
||||
// CreateOperatorAction validates against the closed list and stores a pending row. by names who
|
||||
@@ -160,12 +279,12 @@ func (s *Store) CreateOperatorAction(customerID, action, arg, by string) (int64,
|
||||
func (s *Store) PendingOperatorActions(customerID string) ([]OperatorActionDirective, error) {
|
||||
now := time.Now().UTC()
|
||||
if _, err := s.db.Exec(`UPDATE operator_actions SET done_at = ?, outcome = ?, message = ?
|
||||
WHERE customer_id = ? AND done_at IS NULL AND requested_at < ?`,
|
||||
WHERE customer_id = ? AND host_id = '' AND done_at IS NULL AND requested_at < ?`,
|
||||
now, OperatorActionExpired, "the box did not answer within a day", customerID, now.Add(-OperatorActionTTL)); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
rows, err := s.db.Query(`SELECT id, action, arg FROM operator_actions
|
||||
WHERE customer_id = ? AND done_at IS NULL ORDER BY id LIMIT ?`, customerID, operatorActionsListed)
|
||||
WHERE customer_id = ? AND host_id = '' AND done_at IS NULL ORDER BY id LIMIT ?`, customerID, operatorActionsListed)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@@ -194,7 +313,7 @@ func (s *Store) RecordOperatorActionResult(customerID string, id int64, outcome,
|
||||
message = string(r[:operatorActionMessageMax])
|
||||
}
|
||||
res, err := s.db.Exec(`UPDATE operator_actions SET done_at = ?, outcome = ?, message = ?
|
||||
WHERE id = ? AND customer_id = ? AND done_at IS NULL`, time.Now().UTC(), outcome, message, id, customerID)
|
||||
WHERE id = ? AND customer_id = ? AND host_id = '' AND done_at IS NULL`, time.Now().UTC(), outcome, message, id, customerID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@@ -205,7 +324,7 @@ func (s *Store) RecordOperatorActionResult(customerID string, id int64, outcome,
|
||||
}
|
||||
|
||||
func (s *Store) getOperatorAction(id int64) (*OperatorActionRow, error) {
|
||||
rows, err := s.db.Query(`SELECT id, customer_id, action, arg, requested_at, requested_by, done_at, outcome, message
|
||||
rows, err := s.db.Query(`SELECT id, customer_id, host_id, action, arg, requested_at, requested_by, done_at, outcome, message
|
||||
FROM operator_actions WHERE id = ?`, id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@@ -220,7 +339,7 @@ func (s *Store) getOperatorAction(id int64) (*OperatorActionRow, error) {
|
||||
|
||||
// ListOperatorActions returns the customer's newest rows (pending and closed), newest first.
|
||||
func (s *Store) ListOperatorActions(customerID string, limit int) ([]OperatorActionRow, error) {
|
||||
rows, err := s.db.Query(`SELECT id, customer_id, action, arg, requested_at, requested_by, done_at, outcome, message
|
||||
rows, err := s.db.Query(`SELECT id, customer_id, host_id, action, arg, requested_at, requested_by, done_at, outcome, message
|
||||
FROM operator_actions WHERE customer_id = ? ORDER BY id DESC LIMIT ?`, customerID, limit)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@@ -234,7 +353,7 @@ func scanOperatorActions(rows *sql.Rows) ([]OperatorActionRow, error) {
|
||||
for rows.Next() {
|
||||
var r OperatorActionRow
|
||||
var done sql.NullTime
|
||||
if err := rows.Scan(&r.ID, &r.CustomerID, &r.Action, &r.Arg, &r.RequestedAt, &r.RequestedBy, &done, &r.Outcome, &r.Message); err != nil {
|
||||
if err := rows.Scan(&r.ID, &r.CustomerID, &r.HostID, &r.Action, &r.Arg, &r.RequestedAt, &r.RequestedBy, &done, &r.Outcome, &r.Message); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if done.Valid {
|
||||
|
||||
@@ -11,7 +11,8 @@ import (
|
||||
// The list is CLOSED, on the hub side too — this fails when an action or a job is added, by design. The
|
||||
// controller pins the same four (internal/report TestOpActions_ClosedList).
|
||||
func TestOperatorActions_ClosedList(t *testing.T) {
|
||||
if got, want := OperatorActionNames(), []string{"abandon_extend", "abandon_stop", "offsite_backup_now", "run_job"}; !reflect.DeepEqual(got, want) {
|
||||
// D64 = B (operator 2026-10-29 12:10): clear_hold joins the controller's list.
|
||||
if got, want := OperatorActionNames(), []string{"abandon_extend", "abandon_stop", "clear_hold", "offsite_backup_now", "run_job"}; !reflect.DeepEqual(got, want) {
|
||||
t.Fatalf("operator actions = %v, want exactly %v — a new action needs the operator's word (decision 185)", got, want)
|
||||
}
|
||||
if got, want := OperatorJobNames(), []string{"disk-health-check", "fill-watch", "offsite-integrity", "offsite-proof"}; !reflect.DeepEqual(got, want) {
|
||||
@@ -108,3 +109,84 @@ func TestOperatorActions_ExpireAndResetCancel(t *testing.T) {
|
||||
t.Fatalf("fresh row after reset = %+v, want cancelled (kept as a record)", r)
|
||||
}
|
||||
}
|
||||
|
||||
// The AGENT's closed list (operator brief 2026-10-29, Part 1): one verb. RED-PROOF: seen failing before the list existed.
|
||||
func TestHostOperatorActions_ClosedList(t *testing.T) {
|
||||
if got, want := HostOperatorActionNames(), []string{"restart_controller"}; !reflect.DeepEqual(got, want) {
|
||||
t.Fatalf("host operator actions = %v, want exactly %v — a new verb needs the operator's word", got, want)
|
||||
}
|
||||
for _, a := range OperatorActionNames() {
|
||||
if IsHostOperatorAction(a) {
|
||||
t.Errorf("%s is on BOTH lists — a controller action must never reach the agent", a)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Each list refuses the other's verbs and every bad argument BEFORE storing.
|
||||
func TestHostOperatorActions_RefusedBeforeStoring(t *testing.T) {
|
||||
s := newTestStore(t)
|
||||
bad := []struct{ action, arg string }{
|
||||
{"restart_controller", ""}, {"restart_controller", "abc"}, {"restart_controller", "99"}, {"restart_controller", "9201 "},
|
||||
{"restart_controller", "-1"}, {"restart_controller", "10000000000"}, {"reboot_guest", "9201"}, {"clear_hold", "radicale"},
|
||||
{"offsite_backup_now", ""},
|
||||
}
|
||||
for _, b := range bad {
|
||||
if _, err := s.CreateHostOperatorAction("c1", "h1", b.action, b.arg, "operator"); err == nil {
|
||||
t.Errorf("host action %s(%q) was accepted", b.action, b.arg)
|
||||
}
|
||||
}
|
||||
for _, b := range []struct{ action, arg string }{{"clear_hold", ""}, {"clear_hold", "../x"}, {"clear_hold", "Rad icale"},
|
||||
{"clear_hold", "a/b"}, {"restart_controller", "9201"}} {
|
||||
if _, err := s.CreateOperatorAction("c1", b.action, b.arg, "operator"); err == nil {
|
||||
t.Errorf("controller action %s(%q) was accepted", b.action, b.arg)
|
||||
}
|
||||
}
|
||||
if _, err := s.CreateOperatorAction("c1", "clear_hold", "radicale", "operator"); err != nil {
|
||||
t.Fatalf("clear_hold radicale refused: %v", err)
|
||||
}
|
||||
if _, err := s.CreateHostOperatorAction("c1", "h1", "restart_controller", "9201", "operator"); err != nil {
|
||||
t.Fatalf("restart_controller 9201 refused: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// An agent row reaches only its host's envelope, a controller row only the customer's ACK; a result closes only a row of
|
||||
// the reporting host. RED-PROOF: drop `AND host_id = ”` from PendingOperatorActions → the restart reaches the controller.
|
||||
func TestHostOperatorActions_EachSideSeesOnlyItsOwn(t *testing.T) {
|
||||
s := newTestStore(t)
|
||||
cid, err := s.CreateOperatorAction("c1", "clear_hold", "radicale", "operator")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
hid, err := s.CreateHostOperatorAction("c1", "h1", "restart_controller", "9201", "operator")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctl, _ := s.PendingOperatorActions("c1")
|
||||
if len(ctl) != 1 || ctl[0].ID != cid {
|
||||
t.Fatalf("the controller ACK lists %+v, want only #%d", ctl, cid)
|
||||
}
|
||||
agent, _ := s.PendingHostOperatorActions("h1")
|
||||
if len(agent) != 1 || agent[0].ID != hid || agent[0].Action != "restart_controller" || agent[0].Arg != "9201" {
|
||||
t.Fatalf("the agent envelope lists %+v, want only #%d", agent, hid)
|
||||
}
|
||||
if other, _ := s.PendingHostOperatorActions("h2"); len(other) != 0 {
|
||||
t.Fatalf("another host sees %+v", other)
|
||||
}
|
||||
if row, _ := s.RecordHostOperatorActionResult("h2", hid, "done", "x"); row != nil {
|
||||
t.Fatal("another host closed the row")
|
||||
}
|
||||
if row, _ := s.RecordOperatorActionResult("c1", hid, "done", "x"); row != nil {
|
||||
t.Fatal("the controller's result channel closed an agent row")
|
||||
}
|
||||
row, err := s.RecordHostOperatorActionResult("h1", hid, "done", "the controller answered after 14 s")
|
||||
if err != nil || row == nil || row.Outcome != "done" || row.HostID != "h1" {
|
||||
t.Fatalf("result: %+v %v", row, err)
|
||||
}
|
||||
if left, _ := s.PendingHostOperatorActions("h1"); len(left) != 0 {
|
||||
t.Fatalf("a closed row is still listed: %+v", left)
|
||||
}
|
||||
list, _ := s.ListOperatorActions("c1", 10)
|
||||
if len(list) != 2 {
|
||||
t.Fatalf("the host page lists %d rows, want both", len(list))
|
||||
}
|
||||
}
|
||||
@@ -18,6 +18,18 @@ type heldAppRow struct {
|
||||
WhoClears string
|
||||
Reminded bool
|
||||
Stale bool // the box has not repeated it for heldAppStale — it may be down
|
||||
// ProtectsData (D64 = B): a restore or put-back hold — the confirm step warns before „Clear hold" (`07` §8).
|
||||
ProtectsData bool
|
||||
}
|
||||
|
||||
// holdProtectsData: the reasons whose hold keeps an app off half-restored data (`07` §8): a failed restore (the
|
||||
// reason is "" or restore_failed on older rows), a mixed restore, a put-back waiting for its drive.
|
||||
func holdProtectsData(reason string) bool {
|
||||
switch reason {
|
||||
case "", "restore_failed", "restore_mixed", "putback_pending":
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// heldAppStaleAfter mirrors monitor's heldAppStale: a box repeats app_held daily.
|
||||
@@ -51,7 +63,7 @@ func toHeldAppRows(held []store.HeldApp, hostOf map[string]string, now time.Time
|
||||
}
|
||||
out = append(out, heldAppRow{CustomerID: h.CustomerID, HostID: hostOf[h.CustomerID], Stack: h.Stack, Reason: h.Reason,
|
||||
Since: since, Why: h.Why, WhoClears: monitor.HeldAppWhoClears(h.Reason), Reminded: !h.RemindedAt.IsZero(),
|
||||
Stale: now.Sub(h.LastSeen) > heldAppStaleAfter})
|
||||
Stale: now.Sub(h.LastSeen) > heldAppStaleAfter, ProtectsData: holdProtectsData(h.Reason)})
|
||||
}
|
||||
return out
|
||||
}
|
||||
@@ -0,0 +1,94 @@
|
||||
package web
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"gitea.dooplex.hu/admin/felhom-hub/internal/store"
|
||||
)
|
||||
|
||||
// Operator brief 2026-10-29: „Restart controller" (the AGENT's closed list) and „Clear hold" (D64 = B, the
|
||||
// controller's list) on the host page — each behind a confirm step, CSRF like every form.
|
||||
|
||||
func heldFixture(t *testing.T) (*Server, *store.Store) {
|
||||
t.Helper()
|
||||
s, st := newTestServer(t)
|
||||
if err := st.UpsertHost(&store.Host{HostID: "h1", CustomerID: "c1", APIKey: "k"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
now := time.Now()
|
||||
if err := st.UpsertGuestFromReport(&store.Guest{GuestID: "h1-9201", CustomerID: "c1", HostID: "h1", VMID: 9201, DisplayName: "felhom",
|
||||
Status: "running", ControllerVersion: "0.347.0", LastSeenAt: &now}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := st.UpsertHeldApp("c1", "radicale", "restore_mixed", now.Add(-time.Hour), "a restore stopped half-way", now); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := st.UpsertHeldApp("c1", "gokapi", "unhealthy_stop", now.Add(-time.Hour), "stopped in a crash loop", now); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return s, st
|
||||
}
|
||||
|
||||
// RED-PROOF: seen failing before the forms existed.
|
||||
func TestHostPage_RestartAndClearHoldRenderBehindAConfirm(t *testing.T) {
|
||||
s, _ := heldFixture(t)
|
||||
body := renderHost(t, s, "h1")
|
||||
for _, want := range []string{
|
||||
`data-restart-controller="9201"`, `name="action" value="restart_controller"`, `name="arg" value="9201"`,
|
||||
"Restart the controller of guest 9201 on h1 (c1)?",
|
||||
`data-clear-hold="radicale"`, `name="action" value="clear_hold"`, `name="arg" value="radicale"`,
|
||||
"This hold protects data from a failed restore. If you clear it, the app may start on half-restored data. Check the restore copy first.",
|
||||
`data-clear-hold="gokapi"`,
|
||||
} {
|
||||
if !strings.Contains(body, want) {
|
||||
t.Errorf("host page lacks %q", want)
|
||||
}
|
||||
}
|
||||
// the warning only on the data-protecting hold
|
||||
gk := body[strings.Index(body, `data-clear-hold="gokapi"`):]
|
||||
gk = gk[:strings.Index(gk, "</details>")]
|
||||
if strings.Contains(gk, "half-restored") {
|
||||
t.Error("a crash-loop hold shows the restore warning")
|
||||
}
|
||||
if !strings.Contains(gk, "unhealthy_stop") {
|
||||
t.Error("the confirm step does not name the hold's reason")
|
||||
}
|
||||
}
|
||||
|
||||
// restart_controller: a row for the AGENT of this host (never the controller's ACK); clear_hold only for an app that
|
||||
// IS held; every other verb or argument refused with no row. RED-PROOF: seen failing before the handler routed them.
|
||||
func TestHostPage_RestartAndClearHoldPosts(t *testing.T) {
|
||||
s, st := heldFixture(t)
|
||||
if rr := postOperatorAction(t, s, "h1", url.Values{"action": {"restart_controller"}, "arg": {"9201"}}); rr.Code != http.StatusSeeOther {
|
||||
t.Fatalf("restart: %d %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
if a, _ := st.PendingHostOperatorActions("h1"); len(a) != 1 || a[0].Arg != "9201" {
|
||||
t.Fatalf("the agent's reply would list %+v", a)
|
||||
}
|
||||
if c, _ := st.PendingOperatorActions("c1"); len(c) != 0 {
|
||||
t.Fatalf("the restart reached the controller's ACK: %+v", c)
|
||||
}
|
||||
if rr := postOperatorAction(t, s, "h1", url.Values{"action": {"clear_hold"}, "arg": {"radicale"}}); rr.Code != http.StatusSeeOther {
|
||||
t.Fatalf("clear_hold: %d %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
if c, _ := st.PendingOperatorActions("c1"); len(c) != 1 || c[0].Action != "clear_hold" || c[0].Arg != "radicale" {
|
||||
t.Fatalf("the controller's ACK would list %+v", c)
|
||||
}
|
||||
for _, f := range []url.Values{
|
||||
{"action": {"clear_hold"}, "arg": {"vikunja"}}, // not held
|
||||
{"action": {"restart_controller"}, "arg": {"9202"}}, // not a guest of this host
|
||||
{"action": {"restart_controller"}, "arg": {"abc"}},
|
||||
{"action": {"reboot_guest"}, "arg": {"9201"}},
|
||||
} {
|
||||
if rr := postOperatorAction(t, s, "h1", f); rr.Code != http.StatusBadRequest {
|
||||
t.Errorf("%v: status %d, want 400", f, rr.Code)
|
||||
}
|
||||
}
|
||||
if rows, _ := st.ListOperatorActions("c1", 20); len(rows) != 2 {
|
||||
t.Fatalf("rows = %d, want the two accepted presses only", len(rows))
|
||||
}
|
||||
}
|
||||
@@ -174,11 +174,17 @@ func TestHandleHostDetail(t *testing.T) {
|
||||
// Decision 185 (2026-10-08): plus the four operator-action buttons — a closed list, none of which
|
||||
// deletes or mutates the host (TestOperatorActions_* pin what they may do). Every OTHER button
|
||||
// remains absent: 2 log requests + 4 operator actions, and the 4 all post to /operator-action.
|
||||
if got := strings.Count(strings.ToLower(body), "<button"); got != 6 {
|
||||
t.Errorf("host detail has %d buttons, want exactly the 2 log-request + 4 operator-action buttons", got)
|
||||
// 2026-10-29 (operator brief, Part 1): plus ONE „Restart controller" per guest that runs a controller (the AGENT's
|
||||
// closed list, behind a confirm step; this fixture has one such guest). „Clear hold" appears only per held app (none
|
||||
// here). No other button.
|
||||
if got := strings.Count(strings.ToLower(body), "<button"); got != 7 {
|
||||
t.Errorf("host detail has %d buttons, want exactly the 2 log-request + 4 operator-action + 1 restart-controller buttons", got)
|
||||
}
|
||||
if got := strings.Count(body, `action="/hosts/demo-felhom-01/operator-action"`); got != 4 {
|
||||
t.Errorf("operator-action forms = %d, want 4", got)
|
||||
if got := strings.Count(body, `action="/hosts/demo-felhom-01/operator-action"`); got != 5 {
|
||||
t.Errorf("operator-action forms = %d, want 5", got)
|
||||
}
|
||||
if got := strings.Count(body, `value="restart_controller"`); got != 1 {
|
||||
t.Errorf("restart_controller forms = %d, want 1 (one guest with a controller)", got)
|
||||
}
|
||||
if strings.Count(body, `action="/hosts/demo-felhom-01/request-logs"`) != 2 {
|
||||
t.Error("the request-logs forms are missing — every button must be a log-bundle request")
|
||||
|
||||
@@ -2,6 +2,7 @@ package web
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"gitea.dooplex.hu/admin/felhom-hub/internal/store"
|
||||
@@ -42,11 +43,22 @@ func (s *Server) handleOperatorAction(w http.ResponseWriter, r *http.Request, ho
|
||||
}
|
||||
action := strings.TrimSpace(r.FormValue("action"))
|
||||
arg := strings.TrimSpace(r.FormValue("arg"))
|
||||
// The AGENT's closed list (2026-10-29): its own channel, the host report's reply.
|
||||
if store.IsHostOperatorAction(action) {
|
||||
s.handleHostOperatorAction(w, r, host, action, arg)
|
||||
return
|
||||
}
|
||||
if err := store.ValidateOperatorAction(action, arg); err != nil {
|
||||
s.logger.Printf("[INFO] operator action refused for host %s: %v", hostID, err)
|
||||
http.Error(w, "Refused: "+err.Error(), http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
// D64 = B: Clear hold only for an app the box reports held now — a stale page or a typo stores nothing.
|
||||
if action == store.OperatorActionClearHold && !s.customerHoldsApp(host.CustomerID, arg) {
|
||||
s.logger.Printf("[INFO] clear_hold refused for host %s: %s is not held on this box", hostID, arg)
|
||||
http.Error(w, "Refused: "+arg+" is not held on this box", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
by := operatorActor(r)
|
||||
id, err := s.store.CreateOperatorAction(host.CustomerID, action, arg, by)
|
||||
if err != nil {
|
||||
@@ -80,3 +92,58 @@ func (s *Server) hostOperatorActionRows(host *store.Host) []store.OperatorAction
|
||||
}
|
||||
return rows
|
||||
}
|
||||
|
||||
// customerHoldsApp reports whether the box's held-apps mirror lists the app now.
|
||||
func (s *Server) customerHoldsApp(customerID, app string) bool {
|
||||
held, err := s.store.ListHeldApps(customerID)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
for _, h := range held {
|
||||
if h.Stack == app {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// handleHostOperatorAction stores an AGENT row (the agent's closed list) and pokes the box, so its out-of-cycle
|
||||
// report carries the action in seconds; the result comes on the report after (the agent requests it at once).
|
||||
// restart_controller names a guest of THIS host — another vmid is refused before anything is stored.
|
||||
func (s *Server) handleHostOperatorAction(w http.ResponseWriter, r *http.Request, host *store.Host, action, arg string) {
|
||||
if err := store.ValidateHostOperatorAction(action, arg); err != nil {
|
||||
s.logger.Printf("[INFO] host operator action refused for host %s: %v", host.HostID, err)
|
||||
http.Error(w, "Refused: "+err.Error(), http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
if !s.hostHasGuest(host.HostID, arg) {
|
||||
s.logger.Printf("[INFO] host operator action %s refused for host %s: guest %s is not on this host", action, host.HostID, arg)
|
||||
http.Error(w, "Refused: guest "+arg+" is not on this host", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
by := operatorActor(r)
|
||||
id, err := s.store.CreateHostOperatorAction(host.CustomerID, host.HostID, action, arg, by)
|
||||
if err != nil {
|
||||
s.logger.Printf("[ERROR] host operator action %s for %s: %v", action, host.HostID, err)
|
||||
http.Error(w, "Internal error", http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
s.logger.Printf("[INFO] host operator action #%d %s%s requested for host %s by %s — the agent acts on its next report (poked now)",
|
||||
id, action, opArgSuffix(arg), host.HostID, by)
|
||||
s.poke.PokeHost(host.HostID) // nil-safe; without the tunnel the 15-min cycle carries it
|
||||
http.Redirect(w, r, "/hosts/"+host.HostID+"#operator-actions", http.StatusSeeOther)
|
||||
}
|
||||
|
||||
// hostHasGuest reports whether vmid is one of the host's guests as its reports name them.
|
||||
func (s *Server) hostHasGuest(hostID, vmid string) bool {
|
||||
guests, err := s.store.ListGuestsForHost(hostID)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
for _, g := range guests {
|
||||
if strconv.Itoa(g.VMID) == vmid {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
@@ -133,7 +133,19 @@
|
||||
<td>{{.VMID}}</td>
|
||||
<td>{{if .DisplayName}}{{.DisplayName}}{{else}}—{{end}}</td>
|
||||
<td>{{if eq .Status "running"}}<span style="color: var(--green)">{{.Status}}</span>{{else if eq .Status "stopped"}}<span style="color: var(--red)">{{.Status}}</span>{{else}}{{.Status}}{{end}}</td>
|
||||
<td>{{if .ControllerVersion}}<code>{{.ControllerVersion}}</code>{{else}}—{{end}}</td>
|
||||
<td>{{if .ControllerVersion}}<code>{{.ControllerVersion}}</code>{{else}}—{{end}}
|
||||
{{if and .ControllerVersion $.CustomerID}}{{/* 2026-10-29: the AGENT restarts it (the supervisor's act and guards; 3 in 15 min, then refused for 30) — works with the controller stopped */}}
|
||||
<details class="op-confirm" data-restart-controller="{{.VMID}}">
|
||||
<summary class="btn btn-sm btn-outline">Restart controller</summary>
|
||||
<form method="POST" action="/hosts/{{$.HostID}}/operator-action">
|
||||
<input type="hidden" name="_csrf" value="{{$.CSRFToken}}">
|
||||
<input type="hidden" name="action" value="restart_controller">
|
||||
<input type="hidden" name="arg" value="{{.VMID}}">
|
||||
<p>Restart the controller of guest {{.VMID}} on {{$.HostID}} ({{$.CustomerID}})? The box's agent restarts it, even when the controller does not answer. The household's dashboard is away for about a minute. At most 3 restarts in 15 minutes; the result shows under Operator Actions.</p>
|
||||
<button type="submit" class="btn btn-sm btn-danger">Restart the controller</button>
|
||||
</form>
|
||||
</details>{{end}}
|
||||
</td>
|
||||
<td>{{timeAgoPtr .LastSeenAt}}</td>
|
||||
</tr>
|
||||
{{end}}
|
||||
@@ -146,7 +158,8 @@
|
||||
{{end}}
|
||||
</section>
|
||||
|
||||
<!-- Held apps (D53, `08` §10): apps the box keeps stopped until a person acts. The hub never clears one. -->
|
||||
<!-- Held apps (D53, `08` §10): apps the box keeps stopped until a person acts. D64 = B (operator 2026-10-29): the
|
||||
operator may clear any hold here; the confirm step shows the reason and, for a data-protecting hold, a warning. -->
|
||||
<section class="card" id="held-apps" style="padding: 0; overflow: hidden;">
|
||||
<h2 style="padding: 1.25rem 1.25rem 0.5rem;">Held apps</h2>
|
||||
{{if .HeldApps}}
|
||||
@@ -167,7 +180,19 @@
|
||||
<td><span class="severity-badge severity-warning">{{.Reason}}</span>{{if .Stale}} <span class="text-muted">(not confirmed by the box for 2 days)</span>{{end}}</td>
|
||||
<td>{{timeAgo .Since}}</td>
|
||||
<td>{{.Why}}</td>
|
||||
<td>{{.WhoClears}}{{if .Reminded}} <span class="text-muted">(operator reminded)</span>{{end}}</td>
|
||||
<td>{{.WhoClears}}{{if .Reminded}} <span class="text-muted">(operator reminded)</span>{{end}}
|
||||
<details class="op-confirm" data-clear-hold="{{.Stack}}">
|
||||
<summary class="btn btn-sm btn-outline">Clear hold</summary>
|
||||
<form method="POST" action="/hosts/{{$.HostID}}/operator-action">
|
||||
<input type="hidden" name="_csrf" value="{{$.CSRFToken}}">
|
||||
<input type="hidden" name="action" value="clear_hold">
|
||||
<input type="hidden" name="arg" value="{{.Stack}}">
|
||||
<p>Clear the hold of <code>{{.Stack}}</code> ({{.Reason}}, since {{.Since.Format "2006-01-02 15:04"}})? The box's controller clears it at once; the household can then start the app.</p>
|
||||
{{if .ProtectsData}}<p class="op-warning">This hold protects data from a failed restore. If you clear it, the app may start on half-restored data. Check the restore copy first.</p>{{end}}
|
||||
<button type="submit" class="btn btn-sm btn-danger">Clear the hold</button>
|
||||
</form>
|
||||
</details>
|
||||
</td>
|
||||
</tr>
|
||||
{{end}}
|
||||
</tbody>
|
||||
|
||||
@@ -1031,3 +1031,10 @@ body.js-tabs .tab-panel:not(.tab-panel-active) { display: none; }
|
||||
}
|
||||
.mail-hold-release .text-muted { font-size: 0.8rem; }
|
||||
#mail-hold a { color: var(--text-1); }
|
||||
|
||||
/* 2026-10-29: the confirm fold-out of „Restart controller" and „Clear hold" (no script; the final button is inside). */
|
||||
.op-confirm { margin-top: 0.35rem; }
|
||||
.op-confirm summary { list-style: none; display: inline-block; cursor: pointer; }
|
||||
.op-confirm form { margin-top: 0.5rem; max-width: 34rem; }
|
||||
.op-confirm p { font-size: 0.85rem; margin: 0.25rem 0 0.5rem; }
|
||||
.op-warning { color: var(--warn); }
|
||||
@@ -106,6 +106,9 @@ ROOTS = [
|
||||
# ACK's operator_actions list. Named type for the same reason as R-311's root above.
|
||||
("hub -> controller (report ACK, `operator_actions` entry)",
|
||||
"hub", "internal/store", "OperatorActionDirective", "controller"),
|
||||
# 2026-10-29 (operator brief, Part 1): the AGENT's operator actions — the same entry type on the host-report reply.
|
||||
("hub -> agent (host-report reply, `operator_actions` entry)",
|
||||
"hub", "internal/store", "OperatorActionDirective", "agent"),
|
||||
# R-922 (2026-10-09): the notification-prefs push. Declared when the household's deliberate clear
|
||||
# (`email_cleared`) joined it — a flag the hub cannot decode would leave the cleared address stored.
|
||||
("controller -> hub (POST /preferences)",
|
||||
@@ -124,6 +127,7 @@ MIRRORS = {
|
||||
"hub -> agent (GET /hosts/<id>/escrow/retained)": ("internal/hub", "RetainedEscrowResponse"),
|
||||
"hub -> controller (report ACK, `escrow` object)": ("internal/report", "EscrowStatus"),
|
||||
"hub -> controller (report ACK, `operator_actions` entry)": ("internal/report", "OperatorAction"),
|
||||
"hub -> agent (host-report reply, `operator_actions` entry)": ("internal/hub", "OperatorAction"),
|
||||
}
|
||||
|
||||
# R-444 (2026-10-06): a SUBTREE of a name-checked root whose receiver decodes it into one named mirror type gets the
|
||||
@@ -136,6 +140,8 @@ MIRRORS = {
|
||||
SUBTREE_MIRRORS = {
|
||||
("agent -> hub (POST /host-report)", "guest_disk_trim"): ("internal/sysfacts", "DiskTrim"),
|
||||
("agent -> hub (POST /host-report)", "line_test"): ("internal/sysfacts", "LineTest"),
|
||||
# 2026-10-29: the agent's answers to the operator's actions (hub api.hostOperatorActionResult).
|
||||
("agent -> hub (POST /host-report)", "operator_action_results"): ("internal/api", "hostOperatorActionResult"),
|
||||
}
|
||||
|
||||
# Tag names whose literal string carries no information in a repo-wide search. NOT CHECKED.
|
||||
|
||||
Reference in new issue
Block a user