From f2596ea433e9197bb725e9a43614b227ad36a407 Mon Sep 17 00:00:00 2001 From: kisfenyo Date: Sun, 14 Jun 2026 20:08:52 +0200 Subject: [PATCH] B2b: decommission orchestration + missing-storage indicator + re-enroll fix (v0.65.0) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit agentapi.Decommission + handleStorageDecommission (migrate-all-or-none, Change 2): migrate-then-decommission via the migration done-hook, or decommission-anyway (stop apps, keep HDD_PATH). 'Hiányzó tárhely' badge on dashboard/stacks/app card when an app's drive is decommissioned/disconnected/absent. Change 4: registerStoragePath clears the decommissioned marker on re-enroll (ClearDecommissioned had no callers). Non-hollow tests incl. mutation-proven Change-4 companion. Co-Authored-By: Claude Opus 4.8 (1M context) --- CHANGELOG.md | 43 ++++++ controller/cmd/controller/main.go | 7 +- controller/internal/agentapi/client.go | 28 +++- controller/internal/stacks/manager.go | 31 ++-- controller/internal/stacks/migrate.go | 33 ++++- controller/internal/web/decommission_test.go | 110 ++++++++++++++ controller/internal/web/handlers.go | 43 +++++- controller/internal/web/storage_handlers.go | 138 ++++++++++++++++++ .../internal/web/storage_handlers_test.go | 21 ++- .../internal/web/templates/app_info.html | 7 + .../internal/web/templates/dashboard.html | 1 + controller/internal/web/templates/stacks.html | 1 + controller/internal/web/templates/style.css | 7 + 13 files changed, 435 insertions(+), 35 deletions(-) create mode 100644 controller/internal/web/decommission_test.go diff --git a/CHANGELOG.md b/CHANGELOG.md index 70b4075..3ebf6b4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,48 @@ ## Changelog +### v0.65.0 — data migration + self-serve decommission (B1+B2) (2026-06-14) + +Customer-self-serve storage **migration** (move app data between drives) and **decommission** (retire +a drive), implemented trunk-based with the locked spike design +(`felhom.eu/documentation/audits/SPIKE-decommission-migration-2026-06-14.md`). Pairs with agent +v0.32.0 (the self-serve `/disks/decommission` endpoint + intent-aware re-assert). Built + deployed to +demo guest 9201. **Live decommission/migration of real data is NOT yet validated — that is the +supervised B3 session.** + +- **B1 — migration engine** (`internal/stacks/migrate.go`). In-process over the controller's + `/mnt:/mnt:rslave` RW mount; crash-safe + resumable via a single journal (`/migration.json`). + Two entry points share one pipeline: `MigrateAll` (whole namespace — every app + a conflict-merge + walk for non-app/customer content) and `MigrateApp` (one app subtree; handles drive→drive AND + SSD→drive). Pipeline: validate → stop → copy (`rsync -a --checksum`, additive, NO `--delete`) → verify + (`rsync -ani --checksum`, zero pending) → flip+redeploy (`RedeployFromEnv`, one idempotent unit) → + cleanup. **CLEANUP is the only destructive step and is gated on every unit verified AND every app + redeployed.** Conflict-merge: skip-identical (checksum vs the target file AND its `(N)` siblings), + rename-on-differ to the lowest-free `(N)`, never overwrite; idempotent (no `(1)(1)`). + Single-flight; **mutual exclusion with the backup orchestrator** (Change 3 — migration refuses while a + backup runs; the scheduled DB-dump/Tier-2 skip while a migration runs). +- **B1 UI** — `POST /api/storage/migrate` (whole-namespace), `POST /api/storage/migrate-app` (per-app), + `GET /api/storage/migrate/status` (poll). The greyed migrate-all `` in settings.html is now a + real target-select + button; app_info.html gains a per-app "Áthelyezés másik tárhelyre" control; both + share a Hungarian progress panel. +- **B2b — decommission orchestration** (`handleStorageDecommission`, `POST /api/storage/decommission`). + Two choices, no partial (Change 2): **migrate-all-then-decommission** (runs `MigrateAll`; the + migration done-hook soft-marks the source + calls the agent once every app has moved) or + **decommission-anyway** (type-to-confirm; stops the apps but KEEPS their `HDD_PATH` so they show + "missing storage"). `agentapi.Decommission` added; both branches end at `SetDecommissioned` (soft + marker retained — blocks A1 resurrection) + agent `Decommission`. +- **"Hiányzó tárhely" indicator** — a deployed app whose `HDD_PATH` resolves to a decommissioned/ + disconnected/absent registry path now shows a distinct warning badge on the dashboard, stacks page, + and app card (label via `GetStorageLabel`); persists until re-enroll or migrate. +- **Change 4 — re-enroll clears the marker.** `registerStoragePath` now un-retires a re-plugged + decommissioned drive (`ClearDecommissioned` + restore `Schedulable`) — previously `AddStoragePath` + deduped the re-register into a no-op and the soft marker (and the apps' missing-storage badge) would + persist forever. (`ClearDecommissioned` had zero callers before this.) +- Non-hollow tests across `internal/stacks` (engine: collision-refuse, merge dedup/idempotency, + cleanup-only-after-redeploy, verify-catches-corruption, resume, single-flight, SSD→drive, backup + exclusion), `internal/backup` (scheduled backup skipped while migrating), and `internal/web` + (finalize soft-mark+agent, re-enroll clears marker, missing-storage label). Companions for the + collision guard, cleanup gate, and Change-4 clearing were mutation-proven to fail on the pre-fix code. + ### v0.64.0 — storage-lifecycle cleanups (2026-06-14) Two settings-layer cleanups from the F9 storage-registration diagnosis diff --git a/controller/cmd/controller/main.go b/controller/cmd/controller/main.go index d3a6e8a..5af754a 100644 --- a/controller/cmd/controller/main.go +++ b/controller/cmd/controller/main.go @@ -231,7 +231,8 @@ func main() { if backupMgr != nil { backupMgr.SetMigrationRunningCheck(stackMgr.IsMigrating) } - stackMgr.RecoverMigration(ctx) + // RecoverMigration is started AFTER the web server's done-hook is wired (below), so a resumed + // decommission-migration still finalizes the source decommission on completion. // --- Initialize alert manager --- alertMgr := web.NewAlertManager(logger) @@ -637,6 +638,10 @@ func main() { // --- Initialize web server --- webServer := web.NewServer(cfg, stackMgr, cpuCollector, backupMgr, sched, sett, alertMgr, notifier, updater, logger, Version) + // Migration done-hook: a decommission-initiated migration finalizes the source decommission on + // success (soft-mark + agent). Wire it before RecoverMigration so a resumed one still finalizes. + stackMgr.SetMigrationDoneHook(webServer.OnMigrationDone) + stackMgr.RecoverMigration(ctx) webServer.SetEncryptionKey(encKey) webServer.SetAppExporter(appExporter) webServer.SetIntegrationManager(integrationMgr) diff --git a/controller/internal/agentapi/client.go b/controller/internal/agentapi/client.go index 8d6c8b7..78c20e3 100644 --- a/controller/internal/agentapi/client.go +++ b/controller/internal/agentapi/client.go @@ -134,10 +134,10 @@ type StatusResponse struct { // BackupRecord mirrors the agent's hub.Backup — one whole-guest vzdump/PBS backup result. The // controller renders it read-only (it does NOT own whole-guest backup; the agent does). type BackupRecord struct { - TargetID string `json:"target_id"` // backup storage name (e.g. "local", "felhom-pbs") + TargetID string `json:"target_id"` // backup storage name (e.g. "local", "felhom-pbs") VMID int `json:"vmid"` - Archive string `json:"archive"` // produced vzdump volid (e.g. "local:backup/vzdump-lxc-…") - Mode string `json:"mode"` // snapshot | stop + Archive string `json:"archive"` // produced vzdump volid (e.g. "local:backup/vzdump-lxc-…") + Mode string `json:"mode"` // snapshot | stop CrashConsistent bool `json:"crash_consistent"` SizeBytes int64 `json:"size_bytes"` Success bool `json:"success"` @@ -368,6 +368,28 @@ func (c *Client) EjectDisk(ctx context.Context, where string) (EjectResult, erro return out, nil } +// DecommissionResult mirrors POST /disks/decommission. +type DecommissionResult struct { + VMID int `json:"vmid"` + Decommissioned string `json:"decommissioned"` + DependentGuests []int `json:"dependent_guests"` +} + +// Decommission permanently removes a user-data drive (self-serve, non-destructive — the agent records +// IntentDecommissioned, prunes the bind record, and unmounts; it NEVER formats). Data stays on the +// drive. The agent role-gates to user-data and refuses a system/backup mount regardless. +func (c *Client) Decommission(ctx context.Context, where string) (DecommissionResult, error) { + var out DecommissionResult + body, err := c.post(ctx, "/disks/decommission", map[string]string{"where": where}) + if err != nil { + return out, err + } + if err := json.Unmarshal(body, &out); err != nil { + return out, fmt.Errorf("agentapi: decode /disks/decommission: %w", err) + } + return out, nil +} + // FormatDisk asks the agent to format a device. The AGENT inspects the device and tiers it by ROLE // (its own classification, never the controller's claim): // - blank device → formatted. diff --git a/controller/internal/stacks/manager.go b/controller/internal/stacks/manager.go index 38944a4..ffcb8dc 100644 --- a/controller/internal/stacks/manager.go +++ b/controller/internal/stacks/manager.go @@ -23,8 +23,8 @@ type ContainerState string const ( StateRunning ContainerState = "running" - StateStarting ContainerState = "starting" // running but health: starting - StateUnhealthy ContainerState = "unhealthy" // running but health: unhealthy + StateStarting ContainerState = "starting" // running but health: starting + StateUnhealthy ContainerState = "unhealthy" // running but health: unhealthy StateStopped ContainerState = "stopped" StateRestarting ContainerState = "restarting" StateExited ContainerState = "exited" @@ -52,12 +52,12 @@ type HealthProbeResult struct { // HealthCheckDetail holds the result of a single health check item. type HealthCheckDetail struct { - Type string `json:"type"` // "http", "api", "tcp" - Target string `json:"target"` // e.g. ":3456/api/v1/info" + Type string `json:"type"` // "http", "api", "tcp" + Target string `json:"target"` // e.g. ":3456/api/v1/info" Healthy bool `json:"healthy"` - Status int `json:"status,omitempty"` // HTTP status code (for http/api) - Latency string `json:"latency"` // e.g. "45ms" - Error string `json:"error,omitempty"` // error message if unhealthy + Status int `json:"status,omitempty"` // HTTP status code (for http/api) + Latency string `json:"latency"` // e.g. "45ms" + Error string `json:"error,omitempty"` // error message if unhealthy } // Stack represents a docker compose stack on disk. @@ -66,14 +66,14 @@ type Stack struct { Meta Metadata `json:"meta"` ComposePath string `json:"compose_path"` State ContainerState `json:"state"` - Deployed bool `json:"deployed"` // Has app.yaml with deployed=true + Deployed bool `json:"deployed"` // Has app.yaml with deployed=true Protected bool `json:"protected"` - Orphaned bool `json:"orphaned"` // Deployed but no catalog template + Orphaned bool `json:"orphaned"` // Deployed but no catalog template Containers []ContainerInfo `json:"containers"` AppConfig *AppConfig `json:"app_config,omitempty"` - Deploying bool `json:"deploying"` // compose up in progress - DeployError string `json:"deploy_error,omitempty"` // last async deploy error - HealthProbe *HealthProbeResult `json:"health_probe,omitempty"` // controller-side probe result + Deploying bool `json:"deploying"` // compose up in progress + DeployError string `json:"deploy_error,omitempty"` // last async deploy error + HealthProbe *HealthProbeResult `json:"health_probe,omitempty"` // controller-side probe result LastUpdated time.Time `json:"last_updated"` } @@ -93,8 +93,9 @@ type Manager struct { migJob *MigrationJob settings *settings.Settings sysDataPath string - backupRunning func() bool // mutual exclusion with the backup orchestrator (Change 3) - testSeams *migSeams // nil in production; tests inject fakes + backupRunning func() bool // mutual exclusion with the backup orchestrator (Change 3) + migDoneHook func(*MigrationJob) // fired on successful completion (decommission policy lives in caller) + testSeams *migSeams // nil in production; tests inject fakes } // NewManager creates a new stack manager. @@ -1105,4 +1106,4 @@ func (m *Manager) getCatalogTemplateSlugs() map[string]bool { m.logger.Printf("[DEBUG] [stacks] getCatalogTemplateSlugs: found %d template slugs in %s", len(slugs), cacheDir) } return slugs -} \ No newline at end of file +} diff --git a/controller/internal/stacks/migrate.go b/controller/internal/stacks/migrate.go index 98e0cf7..7caaf51 100644 --- a/controller/internal/stacks/migrate.go +++ b/controller/internal/stacks/migrate.go @@ -115,6 +115,11 @@ type MigrationJob struct { StartedAt time.Time `json:"started_at"` UpdatedAt time.Time `json:"updated_at"` FinishedAt time.Time `json:"finished_at,omitempty"` + + // DecommissionOnDone: when set, the migration was started by the decommission flow — on successful + // completion the done-hook soft-marks the source registry path + tells the agent to decommission it. + // The engine itself does NOT decommission; it only fires the hook (the policy lives in the caller). + DecommissionOnDone bool `json:"decommission_on_done,omitempty"` } func (j *MigrationJob) clone() *MigrationJob { @@ -185,17 +190,27 @@ func (m *Manager) MigrationStatus() *MigrationJob { return m.migJob.clone() } +// SetMigrationDoneHook registers a callback fired (in the migration goroutine) when a migration +// completes successfully. The decommission flow uses it to soft-mark + agent-decommission the source. +func (m *Manager) SetMigrationDoneHook(fn func(*MigrationJob)) { m.migDoneHook = fn } + // MigrateAll moves the whole namespace off sourcePath onto targetPath. func (m *Manager) MigrateAll(ctx context.Context, sourcePath, targetPath string) (string, error) { - return m.startMigration("all", sourcePath, "", targetPath) + return m.startMigration("all", sourcePath, "", targetPath, false) +} + +// MigrateAllAndDecommission moves the whole namespace then (on success) fires the done-hook so the +// caller decommissions the now-empty source drive. +func (m *Manager) MigrateAllAndDecommission(ctx context.Context, sourcePath, targetPath string) (string, error) { + return m.startMigration("all", sourcePath, "", targetPath, true) } // MigrateApp moves a single app's data subtree onto targetPath. func (m *Manager) MigrateApp(ctx context.Context, appName, targetPath string) (string, error) { - return m.startMigration("app", "", appName, targetPath) + return m.startMigration("app", "", appName, targetPath, false) } -func (m *Manager) startMigration(scope, sourcePath, appName, targetPath string) (string, error) { +func (m *Manager) startMigration(scope, sourcePath, appName, targetPath string, decommission bool) (string, error) { if err := m.acquireMigrating(); err != nil { return "", err } @@ -207,10 +222,11 @@ func (m *Manager) startMigration(scope, sourcePath, appName, targetPath string) }() j := &MigrationJob{ - Scope: scope, - Target: filepath.Clean(targetPath), - Units: map[string]*MigUnit{}, - StartedAt: time.Now().UTC(), + Scope: scope, + Target: filepath.Clean(targetPath), + Units: map[string]*MigUnit{}, + StartedAt: time.Now().UTC(), + DecommissionOnDone: decommission, } j.ID = "mig-" + j.StartedAt.Format("20060102-150405") @@ -389,6 +405,9 @@ func (m *Manager) runMigration(ctx context.Context, j *MigrationJob) { } if j.Phase == PhaseDone { m.logger.Printf("[INFO] [migrate] %s complete: %s → %s (%d app(s))", j.ID, j.Source, j.Target, len(j.Apps)) + if m.migDoneHook != nil { + m.migDoneHook(j.clone()) // decommission policy (soft-mark + agent) lives in the hook + } return } } diff --git a/controller/internal/web/decommission_test.go b/controller/internal/web/decommission_test.go new file mode 100644 index 0000000..c12055c --- /dev/null +++ b/controller/internal/web/decommission_test.go @@ -0,0 +1,110 @@ +package web + +import ( + "context" + "testing" + + "gitea.dooplex.hu/admin/felhom-controller/internal/settings" + "gitea.dooplex.hu/admin/felhom-controller/internal/stacks" +) + +// TestFinalizeDecommission_SoftMarksAndCallsAgent: finalizeDecommissionWith soft-marks the registry +// (entry retained, MigratedTo set) and calls the agent's Decommission. Used by both the migrate-done +// hook (MigratedTo=target) and decommission-anyway (MigratedTo=""). +func TestFinalizeDecommission_SoftMarksAndCallsAgent(t *testing.T) { + s := testServer(t) + _ = s.settings.AddStoragePath(settings.StoragePath{Path: "/mnt/old", Schedulable: true, IsDefault: true}) + agent := &mockAgent{} + + if err := s.finalizeDecommissionWith(context.Background(), agent, "/mnt/old", "/mnt/new"); err != nil { + t.Fatalf("finalize: %v", err) + } + // soft-marked, entry retained + if !s.settings.IsDecommissioned("/mnt/old") { + t.Errorf("path not soft-marked decommissioned") + } + found := false + for _, sp := range s.settings.GetStoragePaths() { + if sp.Path == "/mnt/old" { + found = true + if sp.MigratedTo != "/mnt/new" { + t.Errorf("MigratedTo = %q, want /mnt/new", sp.MigratedTo) + } + if sp.Schedulable || sp.IsDefault { + t.Errorf("decommissioned entry should not be schedulable/default") + } + } + } + if !found { + t.Errorf("entry was REMOVED — soft marker must retain it (blocks A1 resurrection)") + } + // agent told to decommission + if len(agent.decommissionCalls) != 1 || agent.decommissionCalls[0] != "/mnt/old" { + t.Errorf("agent.Decommission calls = %v, want [/mnt/old]", agent.decommissionCalls) + } +} + +// TestOnMigrationDone_UnflaggedIsNoop: a plain migrate-all completion (DecommissionOnDone=false) must +// NOT decommission the source. (The flagged path's effects are covered by the finalize test; it would +// require a live agent client here.) Companion: if the early-return guard were removed, this would try +// to reach agentClient() and the path would change — so the no-op assertion pins the guard. +func TestOnMigrationDone_UnflaggedIsNoop(t *testing.T) { + s := testServer(t) + _ = s.settings.AddStoragePath(settings.StoragePath{Path: "/mnt/old", Schedulable: true}) + s.OnMigrationDone(&stacks.MigrationJob{Source: "/mnt/old", Target: "/mnt/new", DecommissionOnDone: false}) + if s.settings.IsDecommissioned("/mnt/old") { + t.Errorf("unflagged migration must not decommission the source") + } +} + +// TestReEnrollClearMarker_Change4: re-enrolling a decommissioned path clears the marker + restores +// Schedulable so its apps' missing-storage indicator clears. (Companion: a non-decommissioned path is +// untouched — returns false.) +func TestReEnrollClearMarker_Change4(t *testing.T) { + s := testServer(t) + _ = s.settings.AddStoragePath(settings.StoragePath{Path: "/mnt/usb", Schedulable: true}) + _ = s.settings.SetDecommissioned("/mnt/usb", "") + if !s.settings.IsDecommissioned("/mnt/usb") { + t.Fatal("precondition: should be decommissioned") + } + + cleared, err := s.reEnrollClearMarker("/mnt/usb") + if err != nil { + t.Fatalf("reEnrollClearMarker: %v", err) + } + if !cleared { + t.Errorf("expected cleared=true for a decommissioned path") + } + if s.settings.IsDecommissioned("/mnt/usb") { + t.Errorf("marker not cleared") + } + if !s.settings.IsStoragePathSchedulable("/mnt/usb") { + t.Errorf("Schedulable not restored") + } + // companion: a non-decommissioned path is a no-op + if c, _ := s.reEnrollClearMarker("/mnt/usb"); c { + t.Errorf("non-decommissioned path should return cleared=false") + } +} + +// TestMissingStorageLabel: an app's HDD_PATH on a decommissioned/disconnected/absent registry path is +// "missing"; on a present+schedulable path it is not; an empty HDD_PATH (SSD) is never missing. +func TestMissingStorageLabel(t *testing.T) { + s := testServer(t) + _ = s.settings.AddStoragePath(settings.StoragePath{Path: "/mnt/live", Label: "Élő", Schedulable: true}) + _ = s.settings.AddStoragePath(settings.StoragePath{Path: "/mnt/dead", Label: "Holt", Schedulable: true}) + _ = s.settings.SetDecommissioned("/mnt/dead", "") + + if _, missing := s.missingStorageLabel(""); missing { + t.Errorf("empty HDD_PATH (SSD) must not be missing") + } + if _, missing := s.missingStorageLabel("/mnt/live"); missing { + t.Errorf("present+schedulable path must not be missing") + } + if label, missing := s.missingStorageLabel("/mnt/dead"); !missing || label != "Holt" { + t.Errorf("decommissioned path: missing=%v label=%q, want true/Holt", missing, label) + } + if label, missing := s.missingStorageLabel("/mnt/gone"); !missing || label == "" { + t.Errorf("unregistered path: missing=%v label=%q, want true/non-empty", missing, label) + } +} diff --git a/controller/internal/web/handlers.go b/controller/internal/web/handlers.go index 0cb4666..6a1292a 100644 --- a/controller/internal/web/handlers.go +++ b/controller/internal/web/handlers.go @@ -151,6 +151,7 @@ func (s *Server) dashboardHandler(w http.ResponseWriter, r *http.Request) { data := s.baseData("dashboard", "Vezérlőpult") data["Stacks"] = deployedStacks + data["MissingStorage"] = s.missingStorageMap(deployedStacks) data["RunningCount"] = running data["StoppedCount"] = stopped data["TotalCount"] = len(stackList) @@ -195,7 +196,9 @@ func (s *Server) dashboardHandler(w http.ResponseWriter, r *http.Request) { func (s *Server) stacksHandler(w http.ResponseWriter, r *http.Request) { data := s.baseData("stacks", "Alkalmazások") - data["Stacks"] = s.stackMgr.GetStacks() + allStacks := s.stackMgr.GetStacks() + data["Stacks"] = allStacks + data["MissingStorage"] = s.missingStorageMap(allStacks) // Build storage label lookup for deployed apps storageLabels := make(map[string]string) // stack name → storage label @@ -486,6 +489,9 @@ func (s *Server) appDetailHandler(w http.ResponseWriter, r *http.Request, slug s } data["MigrateTargets"] = targets data["MigrateCurrent"] = current + if label, missing := s.missingStorageLabel(current); missing { + data["MissingStorageLabel"] = label + } } s.executeTemplate(w, r, "app_info", data) @@ -1116,6 +1122,41 @@ func (s *Server) appsUsingPath(storagePath string) []string { return appsUsingPathIn(s.stackMgr.GetStacks(), s.stackMgr.LoadAppConfigByName, storagePath) } +// missingStorageLabel reports whether a deployed app's HDD_PATH resolves to an UNAVAILABLE registry +// path (decommissioned, disconnected, or no longer registered) and returns its human label. An app +// with no HDD_PATH (SSD-resident) is never "missing". +func (s *Server) missingStorageLabel(hddPath string) (string, bool) { + if hddPath == "" { + return "", false + } + for _, sp := range s.settings.GetStoragePaths() { + if sp.Path == hddPath { + if sp.Decommissioned || sp.Disconnected { + return s.settings.GetStorageLabel(hddPath), true + } + return "", false // present + available + } + } + return s.settings.GetStorageLabel(hddPath), true // not in registry → its drive is gone +} + +// missingStorageMap returns stack-name → storage label for every deployed app whose data drive is +// currently unavailable (drives the "Hiányzó tárhely" dashboard/stacks/app-card indicator). +func (s *Server) missingStorageMap(list []stacks.Stack) map[string]string { + out := map[string]string{} + for _, st := range list { + if !st.Deployed { + continue + } + if cfg := s.stackMgr.LoadAppConfigByName(st.Name); cfg != nil { + if label, missing := s.missingStorageLabel(cfg.Env["HDD_PATH"]); missing { + out[st.Name] = label + } + } + } + return out +} + // appsUsingPathIn is the pure core of appsUsingPath (testable without a live stacks.Manager): the // deployed apps whose data dir (app.yaml HDD_PATH) is exactly storagePath, by display name. This is // the "name the apps that break" list for the type-to-confirm wipe/eject UI. diff --git a/controller/internal/web/storage_handlers.go b/controller/internal/web/storage_handlers.go index 0eaf0a5..c3ce52d 100644 --- a/controller/internal/web/storage_handlers.go +++ b/controller/internal/web/storage_handlers.go @@ -16,6 +16,7 @@ import ( "gitea.dooplex.hu/admin/felhom-controller/internal/agentapi" "gitea.dooplex.hu/admin/felhom-controller/internal/appbackup" "gitea.dooplex.hu/admin/felhom-controller/internal/settings" + "gitea.dooplex.hu/admin/felhom-controller/internal/stacks" "gitea.dooplex.hu/admin/felhom-controller/internal/system" ) @@ -32,6 +33,7 @@ type diskAgent interface { FormatDisk(ctx context.Context, device, fstype string, confirmed bool, durableID string) (agentapi.FormatResult, error) AssignDisk(ctx context.Context, uuid, where, fstype, options string) error EjectDisk(ctx context.Context, where string) (agentapi.EjectResult, error) + Decommission(ctx context.Context, where string) (agentapi.DecommissionResult, error) GuestAttach(ctx context.Context, where string) error } @@ -195,12 +197,39 @@ func (s *Server) pendingActivationDrives() []string { return pending } +// reEnrollClearMarker un-retires a re-plugged decommissioned drive (Change 4): clears the soft marker +// and restores Schedulable so its apps' "missing storage" indicator clears. Returns true if it acted. +func (s *Server) reEnrollClearMarker(where string) (bool, error) { + if !s.settings.IsDecommissioned(where) { + return false, nil + } + if err := s.settings.ClearDecommissioned(where); err != nil { + return false, fmt.Errorf("leszerelés visszavonása sikertelen: %w", err) + } + if err := s.settings.SetSchedulable(where, true); err != nil { + return false, fmt.Errorf("ütemezhetőség visszaállítása sikertelen: %w", err) + } + s.logger.Printf("[INFO] [web] re-enrolled decommissioned drive — marker cleared: %s", where) + return true, nil +} + // registerStoragePath records a freshly-mounted path in the StoragePath registry (schedulable by // default) and refreshes the FileBrowser mounts so it's usable immediately. func (s *Server) registerStoragePath(where, label string, setDefault bool) error { if strings.TrimSpace(label) == "" { label = settings.InferStorageLabel(where) } + // Change 4: re-enrolling a previously-DECOMMISSIONED drive must un-retire it. AddStoragePath + // dedups a re-register into a no-op, so without this the soft marker would persist forever and the + // apps' "missing storage" indicator would never clear. + if cleared, err := s.reEnrollClearMarker(where); err != nil { + return err + } else if cleared { + if s.stackMgr != nil { + go s.SyncFileBrowserMounts() + } + return nil + } sp := settings.StoragePath{ Path: where, Label: label, @@ -251,6 +280,8 @@ func (s *Server) ServeStorageAPI(w http.ResponseWriter, r *http.Request) { s.handleStorageMigrateApp(w, r) case r.URL.Path == "/api/storage/migrate/status" && r.Method == http.MethodGet: s.handleStorageMigrateStatus(w, r) + case r.URL.Path == "/api/storage/decommission" && r.Method == http.MethodPost: + s.handleStorageDecommission(w, r) default: http.NotFound(w, r) } @@ -300,6 +331,113 @@ func (s *Server) handleStorageMigrateStatus(w http.ResponseWriter, r *http.Reque writeDiskJSON(w, http.StatusOK, true, "", map[string]any{"job": s.stackMgr.MigrationStatus()}) } +// handleStorageDecommission is the SELF-SERVE drive decommission (B2b). Exactly two choices, no partial +// (Change 2): mode="migrate" moves ALL apps to a target then decommissions the now-empty source (the +// done-hook finalizes); mode="anyway" decommissions immediately, stopping the apps (keeping their +// HDD_PATH so they show "missing storage"). Non-destructive — the drive's data is never formatted. +func (s *Server) handleStorageDecommission(w http.ResponseWriter, r *http.Request) { + var req struct { + Where string `json:"where"` + Mode string `json:"mode"` // "migrate" | "anyway" + Target string `json:"target"` // required for mode=migrate + MountName string `json:"mount_name"` // type-to-confirm for mode=anyway + } + if err := json.NewDecoder(r.Body).Decode(&req); err != nil { + writeDiskJSON(w, http.StatusBadRequest, false, "érvénytelen kérés", nil) + return + } + req.Where = path.Clean(strings.TrimSpace(req.Where)) + if req.Where == "" || req.Where == "." || !strings.HasPrefix(req.Where, "/mnt/") { + writeDiskJSON(w, http.StatusBadRequest, false, "érvénytelen csatlakoztatási pont", nil) + return + } + + switch req.Mode { + case "migrate": + if strings.TrimSpace(req.Target) == "" { + writeDiskJSON(w, http.StatusBadRequest, false, "céltároló kötelező az áthelyezéshez", nil) + return + } + // Start the migration; the done-hook (onMigrationDone) soft-marks + agent-decommissions the + // source once every app has moved and come up on the target. A VALIDATE refusal returns here. + id, err := s.stackMgr.MigrateAllAndDecommission(r.Context(), req.Where, strings.TrimSpace(req.Target)) + if err != nil { + writeDiskJSON(w, http.StatusConflict, false, err.Error(), nil) + return + } + writeDiskJSON(w, http.StatusOK, true, "", map[string]any{"started": true, "id": id, "mode": "migrate"}) + + case "anyway": + // Type-to-confirm: the typed name must match the mount basename exactly (mirrors wipe). + if strings.TrimSpace(req.MountName) != path.Base(req.Where) { + writeDiskJSON(w, http.StatusBadRequest, false, "a beírt név nem egyezik a csatlakoztatási névvel", nil) + return + } + // Stop the apps that live on this drive — but KEEP their HDD_PATH so the dashboard can name the + // drive in the "missing storage" indicator until the customer re-enrolls or migrates. + var stopped []string + for _, st := range s.stackMgr.GetStacks() { + if !st.Deployed { + continue + } + if cfg := s.stackMgr.LoadAppConfigByName(st.Name); cfg != nil && cfg.Env["HDD_PATH"] == req.Where { + if err := s.stackMgr.StopStack(st.Name); err != nil { + s.logger.Printf("[WARN] [web] decommission: stop %s failed: %v", st.Name, err) + } + stopped = append(stopped, st.Meta.DisplayName) + } + } + if err := s.finalizeDecommission(r.Context(), req.Where, ""); err != nil { + writeDiskJSON(w, http.StatusBadGateway, false, err.Error(), nil) + return + } + writeDiskJSON(w, http.StatusOK, true, "", map[string]any{"decommissioned": true, "where": req.Where, "stopped_apps": stopped}) + + default: + writeDiskJSON(w, http.StatusBadRequest, false, "ismeretlen mód (migrate vagy anyway)", nil) + } +} + +// finalizeDecommission resolves the agent client then soft-marks + decommissions (see *With). +func (s *Server) finalizeDecommission(ctx context.Context, where, migratedTo string) error { + agent, err := s.agentClient() + if err != nil { + return err + } + return s.finalizeDecommissionWith(ctx, agent, where, migratedTo) +} + +// finalizeDecommissionWith soft-marks the registry path (keeping the entry — blocks A1 resurrection) +// and tells the agent to decommission the drive (intent + unmount; never formats). migratedTo is the +// target path for a migrate-then-decommission, or "" for decommission-anyway. The agent is injected so +// the orchestration is testable without a live agent. +func (s *Server) finalizeDecommissionWith(ctx context.Context, agent diskAgent, where, migratedTo string) error { + if err := s.settings.SetDecommissioned(where, migratedTo); err != nil { + return fmt.Errorf("nyilvántartás frissítése sikertelen: %w", err) + } + if _, err := agent.Decommission(ctx, where); err != nil { + return fmt.Errorf("a meghajtó leszerelése sikertelen: %w", err) + } + if s.stackMgr != nil { + go s.SyncFileBrowserMounts() + } + s.logger.Printf("[INFO] [web] storage decommissioned: %s (migrated_to=%q)", where, migratedTo) + return nil +} + +// onMigrationDone is the migration completion hook: when a decommission-initiated migration succeeds, +// the source drive is now empty (all apps flipped to the target), so soft-mark + agent-decommission it. +func (s *Server) OnMigrationDone(j *stacks.MigrationJob) { + if j == nil || !j.DecommissionOnDone { + return + } + if err := s.finalizeDecommission(context.Background(), j.Source, j.Target); err != nil { + s.logger.Printf("[ERROR] [web] post-migration decommission of %s failed: %v", j.Source, err) + return + } + s.logger.Printf("[INFO] [web] source %s decommissioned after migration to %s", j.Source, j.Target) +} + type storageProvReq struct { Device string `json:"device"` FSType string `json:"fstype"` diff --git a/controller/internal/web/storage_handlers_test.go b/controller/internal/web/storage_handlers_test.go index af01027..fd23f74 100644 --- a/controller/internal/web/storage_handlers_test.go +++ b/controller/internal/web/storage_handlers_test.go @@ -27,14 +27,15 @@ func TestTemplatesParse(t *testing.T) { // mockAgent records calls so tests can assert the refusal path performs NO mount/destructive action. type mockAgent struct { - disks agentapi.DisksResponse - formatRes agentapi.FormatResult - formatErr error - assignErr error - assignCalls []assignCall - disksCalls int - formatCalls []formatCall - guestAttachCalls []string + disks agentapi.DisksResponse + formatRes agentapi.FormatResult + formatErr error + assignErr error + assignCalls []assignCall + disksCalls int + formatCalls []formatCall + guestAttachCalls []string + decommissionCalls []string } type assignCall struct{ uuid, where, fstype string } @@ -58,6 +59,10 @@ func (m *mockAgent) AssignDisk(_ context.Context, uuid, where, fstype, _ string) func (m *mockAgent) EjectDisk(_ context.Context, where string) (agentapi.EjectResult, error) { return agentapi.EjectResult{Ejected: where}, nil } +func (m *mockAgent) Decommission(_ context.Context, where string) (agentapi.DecommissionResult, error) { + m.decommissionCalls = append(m.decommissionCalls, where) + return agentapi.DecommissionResult{Decommissioned: where}, nil +} func (m *mockAgent) GuestAttach(_ context.Context, where string) error { m.guestAttachCalls = append(m.guestAttachCalls, where) return nil diff --git a/controller/internal/web/templates/app_info.html b/controller/internal/web/templates/app_info.html index 96082c7..42719d9 100644 --- a/controller/internal/web/templates/app_info.html +++ b/controller/internal/web/templates/app_info.html @@ -24,6 +24,13 @@ +{{if .MissingStorageLabel}} +
+ ⚠ Hiányzó tárhely: {{.MissingStorageLabel}}
+ Ennek az alkalmazásnak az adattárolója jelenleg nem elérhető, ezért le van állítva. Csatlakoztasd újra a meghajtót, vagy helyezd át az adatokat egy másik tárhelyre. +
+{{end}} +
{{stateLabel .State}} {{if .Orphaned}}Elavult{{end}} + {{$ms := index $.MissingStorage .Name}}{{if $ms}}⚠ Hiányzó tárhely: {{$ms}}{{end}} {{if and .Deployed (routeUnpublished .State)}}⚠ URL nem elérhető{{end}} {{if .Protected}} diff --git a/controller/internal/web/templates/stacks.html b/controller/internal/web/templates/stacks.html index 9c468f4..5523e95 100644 --- a/controller/internal/web/templates/stacks.html +++ b/controller/internal/web/templates/stacks.html @@ -38,6 +38,7 @@
{{stateLabel .State}} {{if .Orphaned}}Elavult{{end}} + {{$ms := index $.MissingStorage .Name}}{{if $ms}}⚠ Hiányzó tárhely: {{$ms}}{{end}} {{if .Meta.Description}} diff --git a/controller/internal/web/templates/style.css b/controller/internal/web/templates/style.css index 767e1ed..4b78cff 100644 --- a/controller/internal/web/templates/style.css +++ b/controller/internal/web/templates/style.css @@ -1249,6 +1249,13 @@ a.stat-card:hover { color: var(--orange); } +/* B2b: a deployed app whose data drive was decommissioned/disconnected — distinct warning, not error-spam. */ +.badge-missing-storage { + background: var(--orange-bg); + color: var(--orange); + white-space: nowrap; +} + /* F5: an unhealthy/restarting deployed app has its public route withheld by Traefik (404 at the URL). */ .badge-route-unpublished { background: var(--orange-bg);