From 99bf3ca7a809e1ebb6f38f9e4dfc67476580175f Mon Sep 17 00:00:00 2001 From: kisfenyo Date: Thu, 19 Feb 2026 21:49:14 +0100 Subject: [PATCH] feat: drive migration & Tier 2 restic deprecation (v0.18.0) Phase 1: Deprecate restic as Tier 2 method (rsync only), auto-migrate on startup Phase 2: Enhanced per-app migration with backup awareness, DB dump copy, auto-cleanup Phase 3: Full drive migration with decommissioned state, rollback support, wizard UI Phase 4: Hub report includes decommissioned drive state Co-Authored-By: Claude Opus 4.6 --- CHANGELOG.md | 44 ++ controller/cmd/controller/main.go | 109 +++- controller/internal/api/router.go | 14 +- controller/internal/backup/backup.go | 224 ++++---- controller/internal/backup/crossdrive.go | 143 +---- controller/internal/monitor/healthcheck.go | 5 + controller/internal/monitor/watchdog.go | 8 + controller/internal/report/builder.go | 9 + controller/internal/report/infra_backup.go | 6 - controller/internal/report/types.go | 14 +- controller/internal/settings/settings.go | 162 ++++-- controller/internal/storage/migrate.go | 167 ++++++ controller/internal/storage/migrate_drive.go | 497 ++++++++++++++++++ controller/internal/web/handlers.go | 78 ++- controller/internal/web/server.go | 13 +- controller/internal/web/storage_handlers.go | 290 +++++++++- .../internal/web/templates/backups.html | 28 +- controller/internal/web/templates/deploy.html | 25 - .../internal/web/templates/migrate.html | 43 +- .../internal/web/templates/migrate_drive.html | 218 ++++++++ .../internal/web/templates/settings.html | 23 +- controller/internal/web/templates/style.css | 7 +- 22 files changed, 1725 insertions(+), 402 deletions(-) create mode 100644 controller/internal/storage/migrate_drive.go create mode 100644 controller/internal/web/templates/migrate_drive.html diff --git a/CHANGELOG.md b/CHANGELOG.md index e332705..e86d7c0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,49 @@ ## Changelog +### What was just completed (2026-02-19 session 60) +- **v0.18.0 — Drive Migration & Tier 2 Restic Deprecation:** + + Full drive replacement workflow with decommissioned state, enhanced per-app migration with backup awareness, and deprecation of restic as a Tier 2 cross-drive backup method (rsync only). + + **Phase 1 — Restic Tier 2 Deprecation:** + - `settings.go`: Auto-migrate restic→rsync on startup via `migrateResticToRsync()` in `Load()` + - `crossdrive.go`: Removed `runResticBackup()`, `pruneResticRepo()`, `ensureResticRepo()`; `RunAppBackup()` calls rsync directly + - `backup.go`: Removed Tier 2 secondary restic scanning from `ListAllSnapshots()` + - `settings.go`: Removed cross-drive restic password methods (`GetOrCreateCrossDrivePassword`, etc.) + - `deploy.html`: Removed method dropdown (rsync/restic selector) + - `handlers.go`: Simplified `Tier2DriveGroup` (flat `Items` list), removed method handling from `settingsCrossBackupHandler()` + - `backups.html`: Removed method split in Tier 2 details section + - `router.go`: Always set method to "rsync" in cross-backup API + - `infra_backup.go`: Removed cross-drive password block from `CollectInfraBackup()` + - `main.go`: Removed `SetCrossDriveResticPassword` restore block + + **Phase 2 — Enhanced Per-App Migration:** + - `backup.go`: Extracted `backupDrive()` from `runBackupInternal()` loop; added `TryRunDriveBackup()` with non-blocking lock + - `crossdrive.go`: Added `AnyRunning()` method + - `migrate.go`: Added `BackupTrigger` interface, `MigrateOrchestrator`, `RunEnhancedMigration()` with post-migration steps (DB dump copy, Tier 2 conflict clearing, auto-delete stale data, immediate Tier 1 backup) + - `storage_handlers.go`: Wired orchestrator into migration handler with `auto_delete_stale` support + - `migrate.html`: Added auto-delete checkbox, "cleaning" + "backing_up" progress steps + + **Phase 3 — Full Drive Migration:** + - `settings.go`: Added `Decommissioned`/`DecommissionedAt`/`MigratedTo` fields to `StoragePath`; added `SetDecommissioned()`, `ClearDecommissioned()`, `IsDecommissioned()`, `GetDecommissionedPaths()`, `GetStorageLabel()`; `GetConnectedPaths()`/`GetSchedulableStoragePaths()` exclude decommissioned + - `migrate_drive.go` (NEW): `DriveMigrator` with `MigrateDrive()` 10-step flow (validate→stop→rsync→verify→configure→decommission→Tier2→start→backup→notify), `migrationTx` rollback pattern, excludes restic repos from rsync + - `settings.html`: Decommissioned card variant with "Kiváltva" badge, "Összes adat átköltöztetése" button on connected cards + - `migrate_drive.html` (NEW): Drive migration wizard (form + progress + done cards) + - `storage_handlers.go`: Added `/api/storage/migrate-drive`, `/api/storage/migrate-drive/status`, `/api/storage/decommission/remove` endpoints + - `server.go`: Added `/settings/storage/migrate-drive` route, `SetDriveMigrator()` setter + - `watchdog.go`: Skip decommissioned drives in `Check()`; block `SafeDisconnect()` for decommissioned + - `healthcheck.go`: Skip decommissioned paths in `checkStoragePaths()` + - `backup.go`: Skip decommissioned drives in `backupDrive()`/`runDBDumpsInternal()`; added `MigrationActiveCheck` callback to skip nightly backup during migration + - `crossdrive.go`: Reject decommissioned destinations in `ValidateDestination()`; skip decommissioned paths in `AutoEnableSmallApps()` + - `handlers.go`: Skip decommissioned drives in `buildStorageBars()`; made `SyncFileBrowserMounts()` public + - `main.go`: Added `driveMigrateStackAdapter`, wired `DriveMigrator` with all dependencies + + **Phase 4 — Hub Changes:** + - `report/types.go`: Added `Decommissioned`/`MigratedTo` fields to `StorageReport` + - `report/builder.go`: Include decommissioned drives in report with flag + + **Files modified:** 21 files modified + 2 new files (`migrate_drive.go`, `migrate_drive.html`). + ### What was just completed (2026-02-19 session 59) - **v0.16.1 + hub v0.1.8 — Hub Update Trigger + Controller URL Reporting:** diff --git a/controller/cmd/controller/main.go b/controller/cmd/controller/main.go index 80e531b..4d62162 100644 --- a/controller/cmd/controller/main.go +++ b/controller/cmd/controller/main.go @@ -29,6 +29,7 @@ import ( "gitea.dooplex.hu/admin/felhom-controller/internal/stacks" catalogsync "gitea.dooplex.hu/admin/felhom-controller/internal/sync" "gitea.dooplex.hu/admin/felhom-controller/internal/system" + "gitea.dooplex.hu/admin/felhom-controller/internal/storage" "gitea.dooplex.hu/admin/felhom-controller/internal/web" ) @@ -545,6 +546,48 @@ func main() { webServer := web.NewServer(cfg, stackMgr, cpuCollector, backupMgr, crossDriveRunner, sched, sett, alertMgr, notifier, updater, logger, Version) webServer.SetStorageWatchdog(storageWatchdog) + // --- Initialize drive migrator --- + driveMigrator := &storage.DriveMigrator{ + Sett: sett, + StackProvider: &driveMigrateStackAdapter{mgr: stackMgr}, + Logger: logger, + } + // Only set BackupTrigger if backup is enabled (avoid non-nil interface with nil concrete value) + if backupMgr != nil { + driveMigrator.BackupTrigger = backupMgr + } + driveMigrator.AlertRefresh = func() { + healthReport := monitor.RunHealthCheck(cfg, cpuCollector, sett.GetStoragePaths()) + updateAvailable := false + latestVersion := "" + if updater != nil { + status := updater.GetStatus() + if status.LastCheck != nil { + updateAvailable = status.LastCheck.UpdateAvailable + latestVersion = status.LastCheck.LatestVersion + } + } + alertMgr.Refresh(healthReport, cfg, backupMgr, updateAvailable, latestVersion, sett.GetStoragePaths()) + } + if hubPusher != nil { + driveMigrator.PushHubReport = func() { + r := report.BuildReport(cfg, stackMgr, backupMgr, cpuCollector, metricsStore, Version, sett.GetStoragePaths()) + hubPusher.Push(r) + } + driveMigrator.PushInfraBackup = func() { + pushInfraBackup(cfg, sett, stackProv, hubPusher, logger) + } + } + driveMigrator.SyncFBMounts = func() { + webServer.SyncFileBrowserMounts() + } + webServer.SetDriveMigrator(driveMigrator) + + // Wire migration-active check into backup manager + if backupMgr != nil { + backupMgr.MigrationActiveCheck = driveMigrator.IsActive + } + // Phase 3: Set DR restore mode if a restore plan was built if restorePlan != nil && len(restorePlan.Apps) > 0 { webServer.SetRestoreState(restorePlan) @@ -741,6 +784,65 @@ func (a *watchdogStackAdapter) StartStack(name string) error { return a.mgr.StartStack(name) } +// driveMigrateStackAdapter implements storage.StackProviderForMigration using stacks.Manager. +type driveMigrateStackAdapter struct { + mgr *stacks.Manager +} + +func (a *driveMigrateStackAdapter) ListDeployedStacks() []storage.StackSummaryForMigration { + var result []storage.StackSummaryForMigration + for _, s := range a.mgr.GetStacks() { + if !s.Deployed { + continue + } + result = append(result, storage.StackSummaryForMigration{ + Name: s.Name, + DisplayName: s.Meta.DisplayName, + }) + } + return result +} + +func (a *driveMigrateStackAdapter) GetStackHDDPath(name string) string { + s, ok := a.mgr.GetStack(name) + if !ok { + return "" + } + stackDir := filepath.Dir(s.ComposePath) + appCfg := stacks.LoadAppConfig(stackDir) + if appCfg != nil && appCfg.Env["HDD_PATH"] != "" { + return filepath.Clean(appCfg.Env["HDD_PATH"]) + } + return "" +} + +func (a *driveMigrateStackAdapter) StopStack(name string) error { + return a.mgr.StopStack(name) +} + +func (a *driveMigrateStackAdapter) StartStack(name string) error { + return a.mgr.StartStack(name) +} + +func (a *driveMigrateStackAdapter) UpdateStackHDDPath(name, newPath string) error { + s, ok := a.mgr.GetStack(name) + if !ok { + return fmt.Errorf("stack not found: %s", name) + } + stackDir := filepath.Dir(s.ComposePath) + appCfg := stacks.LoadAppConfig(stackDir) + if appCfg == nil { + return fmt.Errorf("app.yaml not found for stack: %s", name) + } + appCfg.Env["HDD_PATH"] = newPath + return stacks.SaveAppConfig(stackDir, appCfg) +} + +func (a *driveMigrateStackAdapter) StackExists(name string) bool { + _, ok := a.mgr.GetStack(name) + return ok +} + // pushInfraBackup builds and sends the infrastructure snapshot to the Hub. func pushInfraBackup(cfg *config.Config, sett *settings.Settings, stackProv *stackAdapter, pusher *report.Pusher, logger *log.Logger) { @@ -793,13 +895,6 @@ func restorePasswordsFromHub(ib *report.InfraBackup, cfg *config.Config, } } - if ib.CrossDrivePassword != "" { - if err := sett.SetCrossDriveResticPassword(ib.CrossDrivePassword); err == nil { - logger.Println("[INFO] Cross-drive restic password restored from Hub") - } else { - logger.Printf("[WARN] Failed to set cross-drive password: %v", err) - } - } } // restoreSettingsFromHub restores settings.json from a Hub infra backup. diff --git a/controller/internal/api/router.go b/controller/internal/api/router.go index be6ba35..cc603fc 100644 --- a/controller/internal/api/router.go +++ b/controller/internal/api/router.go @@ -595,7 +595,6 @@ func (r *Router) saveCrossBackupConfig(w http.ResponseWriter, req *http.Request, var body struct { Enabled bool `json:"enabled"` - Method string `json:"method"` DestinationPath string `json:"destination_path"` Schedule string `json:"schedule"` } @@ -604,11 +603,6 @@ func (r *Router) saveCrossBackupConfig(w http.ResponseWriter, req *http.Request, return } - // Validate method - if body.Method != "rsync" && body.Method != "restic" { - writeJSON(w, http.StatusBadRequest, apiResponse{OK: false, Error: "method must be 'rsync' or 'restic'"}) - return - } // Validate schedule if body.Schedule != "daily" && body.Schedule != "weekly" && body.Schedule != "manual" { writeJSON(w, http.StatusBadRequest, apiResponse{OK: false, Error: "schedule must be 'daily', 'weekly', or 'manual'"}) @@ -640,7 +634,7 @@ func (r *Router) saveCrossBackupConfig(w http.ResponseWriter, req *http.Request, cfg := &settings.CrossDriveBackup{ Enabled: body.Enabled, - Method: body.Method, + Method: "rsync", DestinationPath: body.DestinationPath, Schedule: body.Schedule, LastRun: lastRun, @@ -656,8 +650,8 @@ func (r *Router) saveCrossBackupConfig(w http.ResponseWriter, req *http.Request, return } - r.logger.Printf("[API] Cross-drive backup config saved for %s: method=%s dest=%s schedule=%s", - name, body.Method, body.DestinationPath, body.Schedule) + r.logger.Printf("[API] Cross-drive backup config saved for %s: dest=%s schedule=%s", + name, body.DestinationPath, body.Schedule) writeJSON(w, http.StatusOK, apiResponse{OK: true, Message: "Cross-drive backup configuration saved"}) } @@ -690,7 +684,7 @@ func (r *Router) getCrossBackupStatus(w http.ResponseWriter, _ *http.Request, na writeJSON(w, http.StatusOK, apiResponse{OK: true, Data: map[string]interface{}{ "configured": true, "enabled": cfg.Enabled, - "method": cfg.Method, + "method": "rsync", "schedule": cfg.Schedule, "running": r.crossDriveRunner != nil && r.crossDriveRunner.IsRunning(name), "last_run": cfg.LastRun, diff --git a/controller/internal/backup/backup.go b/controller/internal/backup/backup.go index 19d332f..13cf115 100644 --- a/controller/internal/backup/backup.go +++ b/controller/internal/backup/backup.go @@ -43,6 +43,10 @@ type Manager struct { // AfterBackup is called after a backup completes to refresh the cache. // Set by main.go to avoid circular import with scheduler. AfterBackup func() + + // MigrationActiveCheck returns true if a full drive migration is in progress. + // Set by main.go to coordinate with DriveMigrator. + MigrationActiveCheck func() bool } // SnapshotRecord combines restic snapshot metadata with our run stats. @@ -243,12 +247,17 @@ func (m *Manager) runDBDumpsInternal(ctx context.Context) error { for _, db := range dbs { drivePath := m.GetAppDrivePath(db.StackName) - // Skip if drive is disconnected + // Skip if drive is disconnected or decommissioned if m.settings != nil && m.settings.IsDisconnected(drivePath) { m.logger.Printf("[WARN] Skipping DB dump for %s — drive disconnected: %s", db.StackName, drivePath) summary = append(summary, fmt.Sprintf("SKIP %s (drive disconnected)", db.ContainerName)) continue } + if m.settings != nil && m.settings.IsDecommissioned(drivePath) { + m.logger.Printf("[WARN] Skipping DB dump for %s — drive decommissioned: %s", db.StackName, drivePath) + summary = append(summary, fmt.Sprintf("SKIP %s (drive decommissioned)", db.ContainerName)) + continue + } dumpDir := AppDBDumpPath(drivePath, db.StackName) @@ -319,6 +328,12 @@ func (m *Manager) RunBackup(ctx context.Context) error { // runBackupInternal is the implementation of RunBackup. Caller must hold the running flag. func (m *Manager) runBackupInternal(ctx context.Context) error { + // Skip if a full drive migration is in progress + if m.MigrationActiveCheck != nil && m.MigrationActiveCheck() { + m.logger.Printf("[WARN] Skipping nightly backup — drive migration in progress") + return nil + } + start := time.Now() m.logger.Printf("[INFO] Starting restic backup (per-drive)") @@ -339,68 +354,14 @@ func (m *Manager) runBackupInternal(ctx context.Context) error { driveCount := 0 for drivePath, stacks := range driveStacks { - // Skip disconnected drives - if m.settings != nil && m.settings.IsDisconnected(drivePath) { - m.logger.Printf("[WARN] Skipping backup for drive %s — disconnected", drivePath) - continue - } - - repoPath := PrimaryResticRepoPath(drivePath) - - // Ensure repo is initialized - if err := m.restic.EnsureInitialized(repoPath); err != nil { - m.logger.Printf("[ERROR] Restic init failed for %s: %v", repoPath, err) - anyErr = err - continue - } - - // Build paths for this drive - var paths []string - paths = append(paths, infraPaths...) - - for _, stack := range stacks { - // App data (appdata//) - appData := AppDataDir(drivePath, stack.Name) - if _, err := os.Stat(appData); err == nil { - paths = append(paths, appData) - } - // HDD mounts (for apps with custom mount points) - if m.stackProvider != nil { - for _, mount := range m.stackProvider.GetStackHDDMounts(stack.Name) { - if _, err := os.Stat(mount); err == nil { - paths = append(paths, mount) - } - } - } - // DB dumps for this stack - dumpDir := AppDBDumpPath(drivePath, stack.Name) - if _, err := os.Stat(dumpDir); err == nil { - paths = append(paths, dumpDir) - } - } - - // Deduplicate paths - paths = dedup(paths) - - tags := []string{"felhom", m.cfg.Customer.ID, filepath.Base(drivePath)} - m.logger.Printf("[INFO] Backing up drive %s (%d apps, %d paths)", drivePath, len(stacks), len(paths)) - - result, err := m.restic.Snapshot(repoPath, paths, tags) + result, err := m.backupDrive(ctx, drivePath, stacks, infraPaths) if err != nil { - m.logger.Printf("[ERROR] Restic backup failed for drive %s: %v", drivePath, err) anyErr = err continue } - - lastResult = result - driveCount++ - - // Prune check (weekly — Sunday) - if shouldPrune(m.cfg.Backup.PruneSchedule) { - m.logger.Printf("[INFO] Running weekly prune for %s", repoPath) - if err := m.restic.Prune(repoPath, m.cfg.Backup.Retention); err != nil { - m.logger.Printf("[WARN] Restic prune failed for %s: %v", repoPath, err) - } + if result != nil { + lastResult = result + driveCount++ } } @@ -463,6 +424,120 @@ func (m *Manager) runBackupInternal(ctx context.Context) error { return anyErr } +// backupDrive runs restic backup for a single drive. Returns nil result if skipped. +// Caller must hold the running flag. +func (m *Manager) backupDrive(ctx context.Context, drivePath string, stacks []StackSummary, infraPaths []string) (*SnapshotResult, error) { + // Skip disconnected or decommissioned drives + if m.settings != nil && m.settings.IsDisconnected(drivePath) { + m.logger.Printf("[WARN] Skipping backup for drive %s — disconnected", drivePath) + return nil, nil + } + if m.settings != nil && m.settings.IsDecommissioned(drivePath) { + m.logger.Printf("[WARN] Skipping backup for drive %s — decommissioned", drivePath) + return nil, nil + } + + repoPath := PrimaryResticRepoPath(drivePath) + + // Ensure repo is initialized + if err := m.restic.EnsureInitialized(repoPath); err != nil { + m.logger.Printf("[ERROR] Restic init failed for %s: %v", repoPath, err) + return nil, err + } + + // Build paths for this drive + var paths []string + paths = append(paths, infraPaths...) + + for _, stack := range stacks { + // App data (appdata//) + appData := AppDataDir(drivePath, stack.Name) + if _, err := os.Stat(appData); err == nil { + paths = append(paths, appData) + } + // HDD mounts (for apps with custom mount points) + if m.stackProvider != nil { + for _, mount := range m.stackProvider.GetStackHDDMounts(stack.Name) { + if _, err := os.Stat(mount); err == nil { + paths = append(paths, mount) + } + } + } + // DB dumps for this stack + dumpDir := AppDBDumpPath(drivePath, stack.Name) + if _, err := os.Stat(dumpDir); err == nil { + paths = append(paths, dumpDir) + } + } + + // Deduplicate paths + paths = dedup(paths) + + tags := []string{"felhom", m.cfg.Customer.ID, filepath.Base(drivePath)} + m.logger.Printf("[INFO] Backing up drive %s (%d apps, %d paths)", drivePath, len(stacks), len(paths)) + + result, err := m.restic.Snapshot(repoPath, paths, tags) + if err != nil { + m.logger.Printf("[ERROR] Restic backup failed for drive %s: %v", drivePath, err) + return nil, err + } + + // Prune check (weekly — Sunday) + if shouldPrune(m.cfg.Backup.PruneSchedule) { + m.logger.Printf("[INFO] Running weekly prune for %s", repoPath) + if err := m.restic.Prune(repoPath, m.cfg.Backup.Retention); err != nil { + m.logger.Printf("[WARN] Restic prune failed for %s: %v", repoPath, err) + } + } + + return result, nil +} + +// tryAcquireRunning attempts to set the running flag without blocking. +// Returns true if acquired, false if already running. +func (m *Manager) tryAcquireRunning() bool { + m.mu.Lock() + defer m.mu.Unlock() + if m.running { + return false + } + m.running = true + return true +} + +// TryRunDriveBackup runs a backup for a single drive if no other backup is in progress. +// Returns error if the backup lock cannot be acquired or if backup fails. +func (m *Manager) TryRunDriveBackup(ctx context.Context, drivePath string) error { + if !m.tryAcquireRunning() { + return fmt.Errorf("backup already in progress") + } + defer m.releaseRunning() + + driveStacks := m.groupStacksByDrive() + stacks, ok := driveStacks[drivePath] + if !ok || len(stacks) == 0 { + m.logger.Printf("[INFO] No deployed stacks on drive %s — skipping backup", drivePath) + return nil + } + + infraPaths := []string{ + m.cfg.Paths.StacksDir, + "/opt/docker/felhom-controller/controller.yaml", + } + + result, err := m.backupDrive(ctx, drivePath, stacks, infraPaths) + if err != nil { + return err + } + + if result != nil { + m.logger.Printf("[INFO] Single-drive backup for %s: snapshot %s, %d new, %d changed, %s added", + drivePath, result.SnapshotID, result.FilesNew, result.FilesChanged, result.DataAdded) + } + + return nil +} + // RunIntegrityCheck runs restic check on all primary repos and pings healthchecks. func (m *Manager) RunIntegrityCheck(ctx context.Context) error { m.logger.Printf("[INFO] Starting restic integrity check") @@ -596,13 +671,12 @@ func (m *Manager) ListSnapshots(limit int) ([]SnapshotInfo, error) { return allSnapshots, nil } -// ListAllSnapshots returns snapshots from both primary and secondary restic repos. -// Primary snapshots get Tier=1, secondary snapshots get Tier=2. +// ListAllSnapshots returns snapshots from primary restic repos across all active drives. +// All snapshots get Tier=1. func (m *Manager) ListAllSnapshots(limit int) ([]SnapshotInfo, error) { drives := m.activeDrives() var allSnapshots []SnapshotInfo - // Tier 1: primary repos (same as ListSnapshots) for _, drive := range drives { repoPath := PrimaryResticRepoPath(drive) if !m.restic.RepoExists(repoPath) { @@ -620,32 +694,6 @@ func (m *Manager) ListAllSnapshots(limit int) ([]SnapshotInfo, error) { allSnapshots = append(allSnapshots, snapshots...) } - // Tier 2: secondary restic repos on cross-drive destinations - if m.settings != nil { - destPaths := make(map[string]bool) - for _, cfg := range m.settings.GetAllCrossDriveConfigs() { - if cfg != nil && cfg.Method == "restic" && cfg.DestinationPath != "" { - destPaths[cfg.DestinationPath] = true - } - } - for destPath := range destPaths { - repoPath := SecondaryResticRepoPath(destPath) - if !m.restic.RepoExists(repoPath) { - continue - } - snapshots, err := m.restic.ListSnapshots(repoPath, 0) - if err != nil { - m.logger.Printf("[WARN] Could not list secondary snapshots from %s: %v", repoPath, err) - continue - } - for i := range snapshots { - snapshots[i].RepoPath = repoPath - snapshots[i].Tier = 2 - } - allSnapshots = append(allSnapshots, snapshots...) - } - } - // Sort newest first sort.Slice(allSnapshots, func(i, j int) bool { return allSnapshots[i].Time.After(allSnapshots[j].Time) diff --git a/controller/internal/backup/crossdrive.go b/controller/internal/backup/crossdrive.go index 8357bdb..6b9c619 100644 --- a/controller/internal/backup/crossdrive.go +++ b/controller/internal/backup/crossdrive.go @@ -102,8 +102,8 @@ func (r *CrossDriveRunner) RunAppBackup(ctx context.Context, stackName string) e }) start := time.Now() - r.logger.Printf("[INFO] Cross-drive backup starting: %s → %s (method: %s)", - stackName, cfg.DestinationPath, cfg.Method) + r.logger.Printf("[INFO] Cross-drive backup starting: %s → %s (rsync)", + stackName, cfg.DestinationPath) // Trigger fresh DB dump for this app before cross-drive backup if r.dbDumper != nil { @@ -130,15 +130,7 @@ func (r *CrossDriveRunner) RunAppBackup(ctx context.Context, stackName string) e } } - var runErr error - switch cfg.Method { - case "rsync": - runErr = r.runRsyncBackup(ctx, stackName, cfg.DestinationPath, mounts) - case "restic": - runErr = r.runResticBackup(ctx, stackName, cfg.DestinationPath, mounts) - default: - runErr = fmt.Errorf("unknown backup method: %s", cfg.Method) - } + runErr := r.runRsyncBackup(ctx, stackName, cfg.DestinationPath, mounts) duration := time.Since(start) @@ -150,11 +142,9 @@ func (r *CrossDriveRunner) RunAppBackup(ctx context.Context, stackName string) e // Calculate backup size var sizeHuman string - if cfg.Method == "rsync" { - destDir := AppSecondaryRsyncPath(cfg.DestinationPath, stackName) - if sz, err := dirSizeBytes(destDir); err == nil { - sizeHuman = humanizeBytes(sz) - } + destDir := AppSecondaryRsyncPath(cfg.DestinationPath, stackName) + if sz, err := dirSizeBytes(destDir); err == nil { + sizeHuman = humanizeBytes(sz) } r.logger.Printf("[INFO] Cross-drive backup completed: %s (%s)", stackName, duration.Round(time.Second)) @@ -209,6 +199,18 @@ func (r *CrossDriveRunner) IsRunning(stackName string) bool { return r.running[stackName] } +// AnyRunning returns true if any cross-drive backup is currently in progress. +func (r *CrossDriveRunner) AnyRunning() bool { + r.mu.Lock() + defer r.mu.Unlock() + for _, running := range r.running { + if running { + return true + } + } + return false +} + // ValidateDestination checks that the destination path exists, is writable, // and has sufficient free space. System-drive destinations get stricter limits // (≥10 GB free, <90% used) to protect OS stability; external drives just need @@ -217,6 +219,9 @@ func (r *CrossDriveRunner) ValidateDestination(path string) error { if path == "" { return fmt.Errorf("destination path is empty") } + if r.sett.IsDecommissioned(path) { + return fmt.Errorf("destination %s is decommissioned — choose an active drive", path) + } if _, err := os.Stat(path); os.IsNotExist(err) { return fmt.Errorf("destination %s does not exist", path) } @@ -326,108 +331,6 @@ func (r *CrossDriveRunner) runRsyncBackup(ctx context.Context, stackName, destBa return nil } -// --- restic --- - -func (r *CrossDriveRunner) runResticBackup(ctx context.Context, stackName, destBase string, mounts []string) error { - repoPath := SecondaryResticRepoPath(destBase) - - // Get or create the cross-drive restic password - password, err := r.sett.GetOrCreateCrossDrivePassword() - if err != nil { - return fmt.Errorf("getting restic password: %w", err) - } - - // H6: Write password to temp file with safe cleanup order (close before deferred remove). - pwFile, err := os.CreateTemp("", "felhom-crossdrive-pw-*") - if err != nil { - return fmt.Errorf("creating password file: %w", err) - } - pwPath := pwFile.Name() - if _, err := pwFile.WriteString(password); err != nil { - pwFile.Close() - os.Remove(pwPath) - return fmt.Errorf("writing password file: %w", err) - } - pwFile.Close() - defer os.Remove(pwPath) - - // Ensure repo is initialized - if err := r.ensureResticRepo(ctx, repoPath, pwPath); err != nil { - return err - } - - // Run restic backup - args := []string{ - "backup", "--repo", repoPath, - "--password-file", pwPath, - "--tag", stackName, - "--tag", "cross-drive", - } - // Include user data (HDD mounts) - args = append(args, mounts...) - // Include app config dir (compose + app.yaml + .felhom.yml) - if composePath, ok := r.stackProvider.GetStackComposePath(stackName); ok { - args = append(args, filepath.Dir(composePath)) - } - // Include DB dump dir for this app (from its home drive) - appDrive := r.getAppDrivePath(stackName) - dumpDir := AppDBDumpPath(appDrive, stackName) - if _, err := os.Stat(dumpDir); err == nil { - args = append(args, dumpDir) - } - - // Include infrastructure paths (same as primary restic) - args = append(args, r.stacksDir) - if _, err := os.Stat(r.controllerYAMLPath); err == nil { - args = append(args, r.controllerYAMLPath) - } - - cmd := exec.CommandContext(ctx, "restic", args...) - r.logger.Printf("[DEBUG] restic backup: %v", args) - if out, err := cmd.CombinedOutput(); err != nil { - return fmt.Errorf("restic backup failed: %v (%s)", err, strings.TrimSpace(string(out))) - } - - // H5: Prune old snapshots to prevent unbounded accumulation. - return r.pruneResticRepo(ctx, repoPath, pwPath) -} - -// pruneResticRepo forgets old snapshots in a cross-drive restic repo, keeping recent ones. -func (r *CrossDriveRunner) pruneResticRepo(ctx context.Context, repoPath, pwPath string) error { - args := []string{ - "forget", "--repo", repoPath, - "--password-file", pwPath, - "--keep-daily", "7", - "--keep-weekly", "4", - "--prune", - } - cmd := exec.CommandContext(ctx, "restic", args...) - r.logger.Printf("[DEBUG] restic forget (prune): %s", repoPath) - if out, err := cmd.CombinedOutput(); err != nil { - // Non-fatal: log warning but don't fail the backup - r.logger.Printf("[WARN] restic forget failed for %s: %v (%s)", repoPath, err, strings.TrimSpace(string(out))) - } - return nil -} - -func (r *CrossDriveRunner) ensureResticRepo(ctx context.Context, repoPath, pwFile string) error { - // Check if repo config exists - if _, err := os.Stat(filepath.Join(repoPath, "config")); err == nil { - return nil // already initialized - } - - if err := os.MkdirAll(repoPath, 0755); err != nil { - return fmt.Errorf("creating restic repo dir: %w", err) - } - - cmd := exec.CommandContext(ctx, "restic", "init", "--repo", repoPath, "--password-file", pwFile) - r.logger.Printf("[INFO] Initializing cross-drive restic repo at %s", repoPath) - if out, err := cmd.CombinedOutput(); err != nil { - return fmt.Errorf("restic init failed: %v (%s)", err, strings.TrimSpace(string(out))) - } - return nil -} - // copyStackDBDumps copies DB dump files for the given stack from its home drive. // DB dumps are at /backups/primary//db-dumps/_.sql. func (r *CrossDriveRunner) copyStackDBDumps(stackName, destDir string) error { @@ -537,11 +440,11 @@ func (r *CrossDriveRunner) AutoEnableSmallApps() { continue } - // Find destination: first storage path that differs from the app's home drive + // Find destination: first active storage path that differs from the app's home drive appDrive := r.getAppDrivePath(stack.Name) var destPath string for _, sp := range storagePaths { - if sp.Path != appDrive { + if sp.Path != appDrive && !sp.Disconnected && !sp.Decommissioned { destPath = sp.Path break } diff --git a/controller/internal/monitor/healthcheck.go b/controller/internal/monitor/healthcheck.go index 76e2cac..41430f0 100644 --- a/controller/internal/monitor/healthcheck.go +++ b/controller/internal/monitor/healthcheck.go @@ -172,6 +172,11 @@ func checkProtectedContainers(protected []string) []string { func checkStoragePaths(paths []settings.StoragePath) (issues, warnings []string) { for _, sp := range paths { + // Skip decommissioned paths — no longer in active use + if sp.Decommissioned { + continue + } + // Skip disconnected paths — handled by the storage watchdog if sp.Disconnected { warnings = append(warnings, fmt.Sprintf("Meghajtó leválasztva: %s (%s)", sp.Label, sp.Path)) diff --git a/controller/internal/monitor/watchdog.go b/controller/internal/monitor/watchdog.go index 4b73e8c..4c22c28 100644 --- a/controller/internal/monitor/watchdog.go +++ b/controller/internal/monitor/watchdog.go @@ -133,6 +133,11 @@ func (w *StorageWatchdog) Check(ctx context.Context) error { } state.lastProbeTime = time.Now() + // Skip decommissioned drives entirely — no apps reference them + if sp.Decommissioned { + continue + } + if sp.Disconnected { w.handleReconnectCheck(ctx, sp) } else { @@ -434,6 +439,9 @@ func (w *StorageWatchdog) SafeDisconnect(ctx context.Context, path string) (stop if sp.Disconnected { return nil, fmt.Errorf("drive already disconnected") } + if sp.Decommissioned { + return nil, fmt.Errorf("drive is decommissioned — no apps to stop") + } label := sp.Label if label == "" { diff --git a/controller/internal/report/builder.go b/controller/internal/report/builder.go index c7d735b..023cbc0 100644 --- a/controller/internal/report/builder.go +++ b/controller/internal/report/builder.go @@ -69,6 +69,15 @@ func BuildReport( {Mount: "/", Label: "SSD", TotalGB: sysInfo.DiskTotalGB, UsedGB: sysInfo.DiskUsedGB, Percent: sysInfo.DiskPercent}, } for _, sp := range storagePaths { + if sp.Decommissioned { + r.Storage = append(r.Storage, StorageReport{ + Mount: sp.Path, + Label: sp.Label, + Decommissioned: true, + MigratedTo: sp.MigratedTo, + }) + continue + } if sp.Disconnected { r.Storage = append(r.Storage, StorageReport{ Mount: sp.Path, diff --git a/controller/internal/report/infra_backup.go b/controller/internal/report/infra_backup.go index 1597548..67f7be5 100644 --- a/controller/internal/report/infra_backup.go +++ b/controller/internal/report/infra_backup.go @@ -75,12 +75,6 @@ func BuildInfraBackup( logger.Printf("[WARN] Infra backup: could not read restic password file: %v", err) } - // Cross-drive password is stored as plain text (not base64) because it's - // already a string in settings, unlike ResticPassword which comes from a file. - if pw := sett.GetCrossDriveResticPassword(); pw != "" { - ib.CrossDrivePassword = pw - } - // Collect disk layout from fstab + blkid ib.DiskLayout = collectDiskLayout(systemDataPath) diff --git a/controller/internal/report/types.go b/controller/internal/report/types.go index c9f218d..3a88920 100644 --- a/controller/internal/report/types.go +++ b/controller/internal/report/types.go @@ -39,12 +39,14 @@ type SystemReport struct { // StorageReport holds disk usage for a mount point. type StorageReport struct { - Mount string `json:"mount"` - Label string `json:"label,omitempty"` - TotalGB float64 `json:"total_gb"` - UsedGB float64 `json:"used_gb"` - Percent float64 `json:"percent"` - Disconnected bool `json:"disconnected,omitempty"` + Mount string `json:"mount"` + Label string `json:"label,omitempty"` + TotalGB float64 `json:"total_gb"` + UsedGB float64 `json:"used_gb"` + Percent float64 `json:"percent"` + Disconnected bool `json:"disconnected,omitempty"` + Decommissioned bool `json:"decommissioned,omitempty"` + MigratedTo string `json:"migrated_to,omitempty"` } // ContainerReport holds aggregate and per-container status. diff --git a/controller/internal/settings/settings.go b/controller/internal/settings/settings.go index 6048b0d..ff253e5 100644 --- a/controller/internal/settings/settings.go +++ b/controller/internal/settings/settings.go @@ -1,10 +1,8 @@ package settings import ( - "crypto/rand" "encoding/json" "fmt" - "io" "log" "os" "path/filepath" @@ -13,9 +11,6 @@ import ( "time" ) -// cryptoRandRead is a var so tests can stub it. -var cryptoRandRead = func(b []byte) (int, error) { return io.ReadFull(rand.Reader, b) } - // Settings holds customer-modifiable overrides and cached state. // Persisted as a single JSON file (settings.json) in the data directory. type Settings struct { @@ -68,14 +63,17 @@ type CrossDriveBackup struct { // StoragePath represents a registered external storage location. type StoragePath struct { - Path string `json:"path"` // e.g., "/mnt/hdd_1" - Label string `json:"label,omitempty"` // e.g., "Külső HDD 1TB" - IsDefault bool `json:"is_default,omitempty"` // new apps use this by default - Schedulable bool `json:"schedulable"` // whether new apps can be deployed here - AddedAt string `json:"added_at"` // RFC3339 - Disconnected bool `json:"disconnected,omitempty"` // true when drive detected as disconnected - DisconnectedAt string `json:"disconnected_at,omitempty"` // RFC3339 timestamp of disconnect detection - StoppedStacks []string `json:"stopped_stacks,omitempty"` // stacks auto-stopped on disconnect + Path string `json:"path"` // e.g., "/mnt/hdd_1" + Label string `json:"label,omitempty"` // e.g., "Külső HDD 1TB" + IsDefault bool `json:"is_default,omitempty"` // new apps use this by default + Schedulable bool `json:"schedulable"` // whether new apps can be deployed here + AddedAt string `json:"added_at"` // RFC3339 + Disconnected bool `json:"disconnected,omitempty"` // true when drive detected as disconnected + DisconnectedAt string `json:"disconnected_at,omitempty"` // RFC3339 timestamp of disconnect detection + StoppedStacks []string `json:"stopped_stacks,omitempty"` // stacks auto-stopped on disconnect + Decommissioned bool `json:"decommissioned,omitempty"` // true when drive data migrated to another + DecommissionedAt string `json:"decommissioned_at,omitempty"` // RFC3339 timestamp + MigratedTo string `json:"migrated_to,omitempty"` // path of target drive } // NotificationPrefs holds customer notification preferences. @@ -124,9 +122,31 @@ func Load(path string, logger *log.Logger) (*Settings, error) { } logger.Printf("[DEBUG] Settings loaded from %s", path) + s.migrateResticToRsync() return s, nil } +// migrateResticToRsync converts any cross-drive backup configs using restic to rsync. +// Called once during Load() before the mutex is exposed. +func (s *Settings) migrateResticToRsync() { + changed := false + for name, prefs := range s.AppBackup { + if prefs.CrossDrive != nil && prefs.CrossDrive.Method == "restic" { + prefs.CrossDrive.Method = "rsync" + s.AppBackup[name] = prefs + if s.log != nil { + s.log.Printf("[INFO] Migrated cross-drive backup for %s from restic to rsync", name) + } + changed = true + } + } + if changed { + if err := s.save(); err != nil && s.log != nil { + s.log.Printf("[ERROR] Failed to save restic→rsync migration: %v", err) + } + } +} + // Save writes settings to disk atomically (write to .tmp, rename). // Caller must hold the write lock or call this from a method that does. func (s *Settings) save() error { @@ -297,42 +317,10 @@ func (s *Settings) GetAllCrossDriveConfigs() map[string]*CrossDriveBackup { return result } -// GetCrossDriveResticPassword returns the cross-drive restic password (read-only). -// Returns empty string if not yet generated. -func (s *Settings) GetCrossDriveResticPassword() string { - s.mu.RLock() - defer s.mu.RUnlock() - return s.CrossDriveResticPassword -} - -// SetCrossDriveResticPassword sets the cross-drive restic password (e.g., during DR restore). -func (s *Settings) SetCrossDriveResticPassword(password string) error { - s.mu.Lock() - defer s.mu.Unlock() - s.CrossDriveResticPassword = password - return s.save() -} - -// GetOrCreateCrossDrivePassword returns the cross-drive restic password, -// generating and persisting one if it doesn't exist yet. -func (s *Settings) GetOrCreateCrossDrivePassword() (string, error) { - s.mu.Lock() - defer s.mu.Unlock() - if s.CrossDriveResticPassword != "" { - return s.CrossDriveResticPassword, nil - } - // Generate a random 32-byte password - buf := make([]byte, 32) - _, err := cryptoRandRead(buf) - if err != nil { - return "", fmt.Errorf("generating cross-drive restic password: %w", err) - } - s.CrossDriveResticPassword = fmt.Sprintf("%x", buf) - if err := s.save(); err != nil { - return "", err - } - return s.CrossDriveResticPassword, nil -} +// NOTE: GetCrossDriveResticPassword, SetCrossDriveResticPassword, and +// GetOrCreateCrossDrivePassword were removed in the Tier 2 restic deprecation. +// The CrossDriveResticPassword field is kept in the struct for backward-compat +// JSON loading but is no longer used. // --- Storage Paths --- @@ -360,13 +348,25 @@ func (s *Settings) GetDefaultStoragePath() string { return "" } +// GetStorageLabel returns the label for a storage path, or the base name if not found. +func (s *Settings) GetStorageLabel(path string) string { + s.mu.RLock() + defer s.mu.RUnlock() + for _, sp := range s.StoragePaths { + if sp.Path == path && sp.Label != "" { + return sp.Label + } + } + return filepath.Base(path) +} + // GetSchedulableStoragePaths returns paths available for new deployments. func (s *Settings) GetSchedulableStoragePaths() []StoragePath { s.mu.RLock() defer s.mu.RUnlock() var result []StoragePath for _, sp := range s.StoragePaths { - if sp.Schedulable { + if sp.Schedulable && !sp.Decommissioned { result = append(result, sp) } } @@ -568,13 +568,13 @@ func (s *Settings) GetDisconnectedPaths() []StoragePath { return result } -// GetConnectedPaths returns a copy of all storage paths that are NOT disconnected. +// GetConnectedPaths returns a copy of all storage paths that are NOT disconnected and NOT decommissioned. func (s *Settings) GetConnectedPaths() []StoragePath { s.mu.RLock() defer s.mu.RUnlock() var result []StoragePath for _, sp := range s.StoragePaths { - if !sp.Disconnected { + if !sp.Disconnected && !sp.Decommissioned { result = append(result, sp) } } @@ -610,3 +610,61 @@ func (s *Settings) ClearStoppedStacks(path string) error { } return fmt.Errorf("storage path %q not found", path) } + +// SetDecommissioned marks a storage path as decommissioned with migration target. +// Clears IsDefault and Schedulable. +func (s *Settings) SetDecommissioned(path, migratedTo string) error { + s.mu.Lock() + defer s.mu.Unlock() + for i := range s.StoragePaths { + if s.StoragePaths[i].Path == path { + s.StoragePaths[i].Decommissioned = true + s.StoragePaths[i].DecommissionedAt = time.Now().UTC().Format(time.RFC3339) + s.StoragePaths[i].MigratedTo = migratedTo + s.StoragePaths[i].IsDefault = false + s.StoragePaths[i].Schedulable = false + return s.save() + } + } + return fmt.Errorf("storage path %q not found", path) +} + +// ClearDecommissioned removes the decommissioned state from a storage path. +func (s *Settings) ClearDecommissioned(path string) error { + s.mu.Lock() + defer s.mu.Unlock() + for i := range s.StoragePaths { + if s.StoragePaths[i].Path == path { + s.StoragePaths[i].Decommissioned = false + s.StoragePaths[i].DecommissionedAt = "" + s.StoragePaths[i].MigratedTo = "" + return s.save() + } + } + return fmt.Errorf("storage path %q not found", path) +} + +// IsDecommissioned returns whether a storage path is marked as decommissioned. +func (s *Settings) IsDecommissioned(path string) bool { + s.mu.RLock() + defer s.mu.RUnlock() + for _, sp := range s.StoragePaths { + if sp.Path == path { + return sp.Decommissioned + } + } + return false +} + +// GetDecommissionedPaths returns a copy of all decommissioned storage paths. +func (s *Settings) GetDecommissionedPaths() []StoragePath { + s.mu.RLock() + defer s.mu.RUnlock() + var result []StoragePath + for _, sp := range s.StoragePaths { + if sp.Decommissioned { + result = append(result, sp) + } + } + return result +} diff --git a/controller/internal/storage/migrate.go b/controller/internal/storage/migrate.go index dd2c4fb..dd02733 100644 --- a/controller/internal/storage/migrate.go +++ b/controller/internal/storage/migrate.go @@ -2,14 +2,18 @@ package storage import ( "bufio" + "context" "fmt" "io" + "log" "os" "os/exec" "path/filepath" "strings" "sync" "time" + + "gitea.dooplex.hu/admin/felhom-controller/internal/settings" ) // MigrateRequest holds parameters for migrating app data. @@ -300,3 +304,166 @@ func bytesHuman(b int64) string { return fmt.Sprintf("%d B", b) } } + +// BackupTrigger allows triggering backup operations without importing the backup package. +type BackupTrigger interface { + TryRunDriveBackup(ctx context.Context, drivePath string) error +} + +// MigrateOptions holds optional configuration for enhanced migration. +type MigrateOptions struct { + AutoDeleteStale bool // delete old data from source after success (default true) +} + +// MigrateOrchestrator wraps MigrateAppData with backup-aware pre/post steps. +type MigrateOrchestrator struct { + Sett *settings.Settings + BackupTrigger BackupTrigger // nil if backup disabled + Logger *log.Logger +} + +// RunEnhancedMigration runs MigrateAppData with additional pre/post-migration steps: +// - Copy DB dumps from source to destination drive +// - Clear Tier 2 config if destination conflicts with cross-drive target +// - Optionally delete stale data from source drive +// - Trigger immediate Tier 1 backup on destination drive +func (o *MigrateOrchestrator) RunEnhancedMigration( + req MigrateRequest, + stopFn StopFunc, + startFn StartFunc, + updateFn UpdateHDDPathFunc, + opts MigrateOptions, + progress chan<- MigrateProgress, +) error { + start := time.Now() + + // Pre-flight: detect Tier 2 conflict + var tier2WillClear bool + cfg := o.Sett.GetCrossDriveConfig(req.StackName) + if cfg != nil && cfg.Enabled && cfg.DestinationPath != "" { + // If destination is under the target drive, the Tier 2 backup would point + // to the same drive the app now lives on — no redundancy, so we clear it. + if strings.HasPrefix(cfg.DestinationPath, req.TargetPath) || cfg.DestinationPath == req.TargetPath { + tier2WillClear = true + o.Logger.Printf("[INFO] Migration %s: Tier 2 will be cleared (dest %s is under target %s)", + req.StackName, cfg.DestinationPath, req.TargetPath) + } + } + + // Run core migration (stop, rsync, update config, start). + // Intercept the "done" step from MigrateAppData — we have post-steps to run. + innerCh := make(chan MigrateProgress, 64) + innerDone := make(chan struct{}) + go func() { + for p := range innerCh { + if p.Step == "done" { + // Suppress MigrateAppData's "done" — we'll send our own after post-steps. + continue + } + progress <- p + } + close(innerDone) + }() + + if err := MigrateAppData(req, stopFn, startFn, updateFn, innerCh); err != nil { + close(innerCh) + <-innerDone + return err + } + close(innerCh) + <-innerDone // wait for forwarding goroutine to finish + + // --- Post-migration steps (all non-fatal) --- + + // 1. Copy DB dumps from source to destination + srcDBDumps := filepath.Join(req.CurrentHDDPath, "backups", "primary", req.StackName, "db-dumps") + dstDBDumps := filepath.Join(req.TargetPath, "backups", "primary", req.StackName, "db-dumps") + if _, err := os.Stat(srcDBDumps); err == nil { + if err := os.MkdirAll(filepath.Dir(dstDBDumps), 0755); err != nil { + o.Logger.Printf("[WARN] Migration %s: failed to create DB dump dir: %v", req.StackName, err) + } else { + cmd := exec.Command("rsync", "-a", srcDBDumps+"/", dstDBDumps+"/") + if out, err := cmd.CombinedOutput(); err != nil { + o.Logger.Printf("[WARN] Migration %s: DB dump copy failed: %v — %s", req.StackName, err, string(out)) + } else { + o.Logger.Printf("[INFO] Migration %s: DB dumps copied to %s", req.StackName, dstDBDumps) + } + } + } + + // 2. Clear Tier 2 if conflict + if tier2WillClear { + if err := o.Sett.SetCrossDriveConfig(req.StackName, nil); err != nil { + o.Logger.Printf("[WARN] Migration %s: failed to clear Tier 2 config: %v", req.StackName, err) + } else { + o.Logger.Printf("[INFO] Migration %s: Tier 2 cross-drive config cleared (dest was on same drive)", req.StackName) + } + } + + // 3. Auto-delete stale data from source + if opts.AutoDeleteStale { + progress <- MigrateProgress{ + Step: "cleaning", + Message: "Régi adatok törlése a forrás meghajtóról...", + Percent: 92, + ElapsedSeconds: int(time.Since(start).Seconds()), + } + + // Delete app data from source + for _, srcPath := range req.HDDMounts { + if !strings.HasPrefix(srcPath, req.CurrentHDDPath+"/") && srcPath != req.CurrentHDDPath { + continue + } + if err := os.RemoveAll(srcPath); err != nil { + o.Logger.Printf("[WARN] Migration %s: failed to delete stale data %s: %v", req.StackName, srcPath, err) + } else { + o.Logger.Printf("[INFO] Migration %s: deleted stale data %s", req.StackName, srcPath) + } + } + + // Delete DB dumps from source + if _, err := os.Stat(srcDBDumps); err == nil { + if err := os.RemoveAll(srcDBDumps); err != nil { + o.Logger.Printf("[WARN] Migration %s: failed to delete stale DB dumps %s: %v", req.StackName, srcDBDumps, err) + } else { + o.Logger.Printf("[INFO] Migration %s: deleted stale DB dumps %s", req.StackName, srcDBDumps) + } + } + } + + // 4. Trigger immediate Tier 1 backup on destination drive + if o.BackupTrigger != nil { + progress <- MigrateProgress{ + Step: "backing_up", + Message: "Biztonsági mentés indítása az új meghajtón...", + Percent: 95, + ElapsedSeconds: int(time.Since(start).Seconds()), + } + + if err := o.BackupTrigger.TryRunDriveBackup(context.Background(), req.TargetPath); err != nil { + o.Logger.Printf("[WARN] Migration %s: post-migration backup failed: %v", req.StackName, err) + progress <- MigrateProgress{ + Step: "backing_up", + Message: "Biztonsági mentés nem indítható (másik mentés fut)", + Percent: 96, + ElapsedSeconds: int(time.Since(start).Seconds()), + } + } else { + o.Logger.Printf("[INFO] Migration %s: post-migration backup completed for %s", req.StackName, req.TargetPath) + } + } + + // Final done step with enhanced info + msg := fmt.Sprintf("Áthelyezés kész! Az alkalmazás az új tárolóról fut. (idő: %ds)", int(time.Since(start).Seconds())) + if tier2WillClear { + msg += " A 2. szintű mentés törlésre került." + } + progress <- MigrateProgress{ + Step: "done", + Message: msg, + Percent: 100, + ElapsedSeconds: int(time.Since(start).Seconds()), + } + + return nil +} diff --git a/controller/internal/storage/migrate_drive.go b/controller/internal/storage/migrate_drive.go new file mode 100644 index 0000000..40f336a --- /dev/null +++ b/controller/internal/storage/migrate_drive.go @@ -0,0 +1,497 @@ +package storage + +import ( + "bufio" + "context" + "fmt" + "log" + "os" + "os/exec" + "path/filepath" + "strings" + "sync" + "time" + + "gitea.dooplex.hu/admin/felhom-controller/internal/settings" +) + +// StackProviderForMigration abstracts stack operations needed by drive migration. +type StackProviderForMigration interface { + ListDeployedStacks() []StackSummaryForMigration + GetStackHDDPath(name string) string + StopStack(name string) error + StartStack(name string) error + UpdateStackHDDPath(name, newPath string) error + StackExists(name string) bool +} + +// StackSummaryForMigration holds minimal stack info for drive migration. +type StackSummaryForMigration struct { + Name string + DisplayName string +} + +// DriveMigrateRequest holds parameters for migrating all apps from one drive to another. +type DriveMigrateRequest struct { + SourcePath string // e.g., "/mnt/hdd_1" + DestPath string // e.g., "/mnt/hdd_2" +} + +// DriveMigrateProgress tracks drive migration state. +type DriveMigrateProgress struct { + Step string // "validating","stopping","copying","verifying","configuring","starting","backup","done","error","rolling_back" + Message string + BytesCopied int64 + BytesTotal int64 + Percent int + Error string + ElapsedSeconds int + Detail string // sub-step detail (e.g., which app is being configured) +} + +// DriveMigrator orchestrates full drive migration. +type DriveMigrator struct { + Sett *settings.Settings + StackProvider StackProviderForMigration + BackupTrigger BackupTrigger + AlertRefresh func() + PushHubReport func() + PushInfraBackup func() + SyncFBMounts func() + Logger *log.Logger + + mu sync.Mutex + active bool // global migration lock +} + +// IsActive returns whether a full drive migration is currently in progress. +func (dm *DriveMigrator) IsActive() bool { + dm.mu.Lock() + defer dm.mu.Unlock() + return dm.active +} + +// rollbackAction describes a reversible action in the migration transaction. +type rollbackAction struct { + description string + undo func() error +} + +// migrationTx is a transaction log that enables reverse-order rollback. +type migrationTx struct { + actions []rollbackAction + logger *log.Logger +} + +func (tx *migrationTx) add(desc string, undoFn func() error) { + tx.actions = append(tx.actions, rollbackAction{description: desc, undo: undoFn}) +} + +func (tx *migrationTx) rollback() { + for i := len(tx.actions) - 1; i >= 0; i-- { + a := tx.actions[i] + tx.logger.Printf("[ROLLBACK] %s", a.description) + if err := a.undo(); err != nil { + tx.logger.Printf("[ROLLBACK-ERROR] %s: %v", a.description, err) + } + } +} + +// MigrateDrive performs a full drive migration, moving all apps from source to dest. +func (dm *DriveMigrator) MigrateDrive(ctx context.Context, req DriveMigrateRequest, progress chan<- DriveMigrateProgress) error { + start := time.Now() + + send := func(step, msg string, pct int) { + progress <- DriveMigrateProgress{ + Step: step, + Message: msg, + Percent: pct, + ElapsedSeconds: int(time.Since(start).Seconds()), + } + } + sendDetail := func(step, msg, detail string, pct int) { + progress <- DriveMigrateProgress{ + Step: step, + Message: msg, + Detail: detail, + Percent: pct, + ElapsedSeconds: int(time.Since(start).Seconds()), + } + } + fail := func(msg string, err error) error { + errStr := "" + if err != nil { + errStr = err.Error() + } + progress <- DriveMigrateProgress{ + Step: "error", + Message: msg, + Error: errStr, + ElapsedSeconds: int(time.Since(start).Seconds()), + } + return fmt.Errorf("%s: %w", msg, err) + } + + // Acquire global migration lock + dm.mu.Lock() + if dm.active { + dm.mu.Unlock() + return fail("Egy másik meghajtó-migráció folyamatban van", fmt.Errorf("migration already active")) + } + dm.active = true + dm.mu.Unlock() + defer func() { + dm.mu.Lock() + dm.active = false + dm.mu.Unlock() + }() + + // --- Pre-validation --- + send("validating", "Ellenőrzés...", 1) + + srcLabel := dm.Sett.GetStorageLabel(req.SourcePath) + dstLabel := dm.Sett.GetStorageLabel(req.DestPath) + + if dm.Sett.IsDisconnected(req.SourcePath) { + return fail("A forrás meghajtó le van választva", fmt.Errorf("source disconnected")) + } + if dm.Sett.IsDecommissioned(req.SourcePath) { + return fail("A forrás meghajtó már kiváltott", fmt.Errorf("source decommissioned")) + } + if dm.Sett.IsDisconnected(req.DestPath) { + return fail("A cél meghajtó le van választva", fmt.Errorf("dest disconnected")) + } + if dm.Sett.IsDecommissioned(req.DestPath) { + return fail("A cél meghajtó már kiváltott", fmt.Errorf("dest decommissioned")) + } + + // Find apps on source drive + var appsToMigrate []StackSummaryForMigration + for _, stack := range dm.StackProvider.ListDeployedStacks() { + hddPath := dm.StackProvider.GetStackHDDPath(stack.Name) + if hddPath == req.SourcePath { + appsToMigrate = append(appsToMigrate, stack) + } + } + + if len(appsToMigrate) == 0 { + return fail("A forrás meghajtón nincs telepített alkalmazás", fmt.Errorf("no apps on source")) + } + + // Check for conflicts on destination + for _, app := range appsToMigrate { + destAppData := filepath.Join(req.DestPath, "appdata", app.Name) + if info, err := os.Stat(destAppData); err == nil && info.IsDir() { + entries, _ := os.ReadDir(destAppData) + if len(entries) > 0 { + return fail( + fmt.Sprintf("A cél meghajtón már létezik adat: %s/%s", req.DestPath, app.Name), + fmt.Errorf("conflict: %s already exists on destination", app.Name), + ) + } + } + } + + // Estimate total size (exclude restic repos) + var totalBytes int64 + entries, _ := os.ReadDir(req.SourcePath) + for _, entry := range entries { + entryPath := filepath.Join(req.SourcePath, entry.Name()) + if entry.IsDir() { + // Skip restic repos in size estimate + if entry.Name() == "backups" { + totalBytes += dirSizeExcluding(entryPath, "restic") + } else { + totalBytes += dirSize(entryPath) + } + } + } + + // Check free space on destination + freeBytes := getFreeBytes(req.DestPath) + if freeBytes > 0 && totalBytes > 0 && int64(float64(totalBytes)*1.05) > freeBytes { + return fail( + fmt.Sprintf("Nincs elég szabad hely: szükséges ~%s, szabad %s", + bytesHuman(totalBytes), bytesHuman(freeBytes)), + fmt.Errorf("insufficient disk space"), + ) + } + + dm.Logger.Printf("[INFO] Drive migration: %s (%s) → %s (%s), %d apps, ~%s data", + req.SourcePath, srcLabel, req.DestPath, dstLabel, len(appsToMigrate), bytesHuman(totalBytes)) + + tx := &migrationTx{logger: dm.Logger} + + // --- Step 1: Stop all affected apps --- + send("stopping", fmt.Sprintf("Alkalmazások leállítása (%d db)...", len(appsToMigrate)), 5) + + var stoppedApps []string + for _, app := range appsToMigrate { + sendDetail("stopping", "Leállítás: "+app.DisplayName, app.Name, 5) + if err := dm.StackProvider.StopStack(app.Name); err != nil { + dm.Logger.Printf("[ERROR] Drive migration: failed to stop %s: %v", app.Name, err) + // Rollback: restart already stopped apps + send("rolling_back", "Hiba a leállításnál, visszagörgetés...", 0) + for _, name := range stoppedApps { + _ = dm.StackProvider.StartStack(name) + } + return fail("Alkalmazás leállítása sikertelen: "+app.DisplayName, err) + } + stoppedApps = append(stoppedApps, app.Name) + } + tx.add("Restart all stopped apps", func() error { + for _, name := range stoppedApps { + if err := dm.StackProvider.StartStack(name); err != nil { + dm.Logger.Printf("[ROLLBACK-WARN] Failed to restart %s: %v", name, err) + } + } + return nil + }) + + // --- Step 2: rsync entire drive (excluding restic repos) --- + send("copying", "Adatok másolása...", 10) + + rsyncCmd := exec.CommandContext(ctx, "rsync", "-a", "--info=progress2", + "--exclude=backups/primary/restic/", + "--exclude=backups/secondary/restic/", + req.SourcePath+"/", req.DestPath+"/", + ) + + stdout, err := rsyncCmd.StdoutPipe() + if err != nil { + send("rolling_back", "rsync indítása sikertelen, visszagörgetés...", 0) + tx.rollback() + return fail("rsync pipe hiba", err) + } + stderr, err := rsyncCmd.StderrPipe() + if err != nil { + send("rolling_back", "rsync indítása sikertelen, visszagörgetés...", 0) + tx.rollback() + return fail("rsync stderr pipe hiba", err) + } + + if err := rsyncCmd.Start(); err != nil { + send("rolling_back", "rsync indítása sikertelen, visszagörgetés...", 0) + tx.rollback() + return fail("rsync indítás sikertelen", err) + } + + // Parse rsync progress + go func() { + scanner := bufio.NewScanner(stdout) + for scanner.Scan() { + line := scanner.Text() + if b, pct, ok := parseRsyncProgress(line); ok { + scaledPct := 10 + pct*50/100 // scale to 10-60% + if scaledPct > 60 { + scaledPct = 60 + } + progress <- DriveMigrateProgress{ + Step: "copying", + Message: fmt.Sprintf("Adatok másolása... %s / %s", bytesHuman(b), bytesHuman(totalBytes)), + BytesCopied: b, + BytesTotal: totalBytes, + Percent: scaledPct, + ElapsedSeconds: int(time.Since(start).Seconds()), + } + } + } + }() + + var stderrBuf strings.Builder + go func() { + buf := make([]byte, 4096) + for { + n, err := stderr.Read(buf) + if n > 0 { + stderrBuf.Write(buf[:n]) + } + if err != nil { + break + } + } + }() + + if err := rsyncCmd.Wait(); err != nil { + send("rolling_back", "rsync sikertelen, visszagörgetés...", 0) + tx.rollback() + return fail("Adatmásolás sikertelen", fmt.Errorf("rsync failed: %w — %s", err, stderrBuf.String())) + } + + // --- Step 3: Verify copy --- + send("verifying", "Másolat ellenőrzése...", 62) + + for _, app := range appsToMigrate { + destAppData := filepath.Join(req.DestPath, "appdata", app.Name) + if _, err := os.Stat(destAppData); os.IsNotExist(err) { + // appdata might not exist for all apps (SSD-only apps that share the drive) + // Only warn, don't fail + dm.Logger.Printf("[WARN] Drive migration: %s/appdata/%s not found on destination (may be SSD-only)", req.DestPath, app.Name) + } + } + + // --- Step 4: Update all app configs --- + send("configuring", "Konfiguráció frissítése...", 65) + + var configuredApps []string + for i, app := range appsToMigrate { + // Guard: verify app still exists + if !dm.StackProvider.StackExists(app.Name) { + dm.Logger.Printf("[WARN] Drive migration: app %s no longer exists, skipping config update", app.Name) + continue + } + + pct := 65 + (i * 10 / len(appsToMigrate)) + sendDetail("configuring", "Konfiguráció: "+app.DisplayName, app.Name, pct) + + oldPath := dm.StackProvider.GetStackHDDPath(app.Name) + if err := dm.StackProvider.UpdateStackHDDPath(app.Name, req.DestPath); err != nil { + dm.Logger.Printf("[ERROR] Drive migration: failed to update HDD_PATH for %s: %v", app.Name, err) + send("rolling_back", "Konfiguráció frissítése sikertelen, visszagörgetés...", 0) + // Rollback config changes + for _, name := range configuredApps { + _ = dm.StackProvider.UpdateStackHDDPath(name, req.SourcePath) + } + tx.rollback() + return fail("HDD_PATH frissítés sikertelen: "+app.DisplayName, err) + } + configuredApps = append(configuredApps, app.Name) + tx.add("Revert HDD_PATH for "+app.Name, func() error { + return dm.StackProvider.UpdateStackHDDPath(app.Name, oldPath) + }) + } + + // --- Step 5: Update storage registry --- + send("configuring", "Tárolóregiszter frissítése...", 76) + + // Transfer IsDefault + allPaths := dm.Sett.GetStoragePaths() + var srcWasDefault bool + var srcWasSchedulable bool + for _, sp := range allPaths { + if sp.Path == req.SourcePath { + srcWasDefault = sp.IsDefault + srcWasSchedulable = sp.Schedulable + } + } + + if srcWasDefault { + _ = dm.Sett.SetDefaultStoragePath(req.DestPath) + } + if srcWasSchedulable { + _ = dm.Sett.SetSchedulable(req.DestPath, true) + } + + // Mark source as decommissioned + if err := dm.Sett.SetDecommissioned(req.SourcePath, req.DestPath); err != nil { + dm.Logger.Printf("[WARN] Drive migration: failed to mark source as decommissioned: %v", err) + } + tx.add("Clear decommissioned on source", func() error { + return dm.Sett.ClearDecommissioned(req.SourcePath) + }) + + // --- Step 6: Update Tier 2 cross-drive configs --- + send("configuring", "Mentési beállítások frissítése...", 78) + + allCrossConfigs := dm.Sett.GetAllCrossDriveConfigs() + for name, cfg := range allCrossConfigs { + if cfg == nil { + continue + } + // Apps that moved (source→dest) with Tier 2 pointing to dest: clear (no redundancy) + appHDD := dm.StackProvider.GetStackHDDPath(name) + if appHDD == req.DestPath && cfg.DestinationPath == req.DestPath { + dm.Logger.Printf("[INFO] Drive migration: clearing Tier 2 for %s (dest same as app drive)", name) + _ = dm.Sett.SetCrossDriveConfig(name, nil) + continue + } + // Apps on OTHER drives with Tier 2 pointing to source: redirect to dest + if cfg.DestinationPath == req.SourcePath { + dm.Logger.Printf("[INFO] Drive migration: redirecting Tier 2 for %s from %s to %s", name, req.SourcePath, req.DestPath) + cfg.DestinationPath = req.DestPath + _ = dm.Sett.SetCrossDriveConfig(name, cfg) + } + } + + // --- Step 7: Start all migrated apps --- + send("starting", "Alkalmazások indítása...", 80) + + for i, app := range appsToMigrate { + if !dm.StackProvider.StackExists(app.Name) { + continue + } + pct := 80 + (i * 8 / len(appsToMigrate)) + sendDetail("starting", "Indítás: "+app.DisplayName, app.Name, pct) + + if err := dm.StackProvider.StartStack(app.Name); err != nil { + dm.Logger.Printf("[WARN] Drive migration: failed to start %s after migration: %v", app.Name, err) + // Non-fatal — log but continue + } + } + + // At this point, migration is considered successful — no more rollback. + + // --- Step 8: Trigger immediate backup --- + send("backup", "Biztonsági mentés indítása...", 90) + + if dm.BackupTrigger != nil { + if err := dm.BackupTrigger.TryRunDriveBackup(ctx, req.DestPath); err != nil { + dm.Logger.Printf("[WARN] Drive migration: post-migration backup failed: %v", err) + } + } + + // --- Step 9: Post-migration notifications --- + send("configuring", "Befejező lépések...", 95) + + if dm.SyncFBMounts != nil { + dm.SyncFBMounts() + } + if dm.AlertRefresh != nil { + dm.AlertRefresh() + } + if dm.PushHubReport != nil { + dm.PushHubReport() + } + if dm.PushInfraBackup != nil { + dm.PushInfraBackup() + } + + elapsed := time.Since(start) + dm.Logger.Printf("[INFO] Drive migration complete: %s → %s, %d apps, %s elapsed", + req.SourcePath, req.DestPath, len(appsToMigrate), elapsed.Round(time.Second)) + + // --- Step 10: Done --- + appNames := make([]string, len(appsToMigrate)) + for i, app := range appsToMigrate { + appNames[i] = app.DisplayName + } + + progress <- DriveMigrateProgress{ + Step: "done", + Message: fmt.Sprintf("A %s meghajtó sikeresen kiváltva! %d alkalmazás átköltöztetve ide: %s (%s). Idő: %s", + srcLabel, len(appsToMigrate), dstLabel, req.DestPath, elapsed.Round(time.Second)), + Percent: 100, + ElapsedSeconds: int(elapsed.Seconds()), + Detail: strings.Join(appNames, ", "), + } + + return nil +} + +// dirSizeExcluding returns the total bytes in a directory, excluding subdirectories named excludeName. +func dirSizeExcluding(path, excludeName string) int64 { + var total int64 + filepath.Walk(path, func(p string, info os.FileInfo, err error) error { + if err != nil { + return nil + } + if info.IsDir() && info.Name() == excludeName { + return filepath.SkipDir + } + if !info.IsDir() { + total += info.Size() + } + return nil + }) + return total +} diff --git a/controller/internal/web/handlers.go b/controller/internal/web/handlers.go index c64315f..ab68b86 100644 --- a/controller/internal/web/handlers.go +++ b/controller/internal/web/handlers.go @@ -34,6 +34,10 @@ type StorageBarInfo struct { func (s *Server) buildStorageBars() []StorageBarInfo { var bars []StorageBarInfo for _, sp := range s.settings.GetStoragePaths() { + // Skip decommissioned drives — they are no longer in active use + if sp.Decommissioned { + continue + } if sp.Disconnected { bars = append(bars, StorageBarInfo{ Label: sp.Label, @@ -74,13 +78,15 @@ type StorageAppDetail struct { // StoragePathView extends StoragePath with display data for the settings page. type StoragePathView struct { settings.StoragePath - DiskInfo *system.DiskUsageInfo - AppCount int - IsMounted bool - AppDetails []StorageAppDetail - FSInfo *system.FSInfo - IsUSB bool // true if this is a USB-attached device (safe disconnect available) - StoppedApps []string // stacks auto-stopped due to disconnect (for restart UI) + DiskInfo *system.DiskUsageInfo + AppCount int + IsMounted bool + AppDetails []StorageAppDetail + FSInfo *system.FSInfo + IsUSB bool // true if this is a USB-attached device (safe disconnect available) + StoppedApps []string // stacks auto-stopped due to disconnect (for restart UI) + MigratedToLabel string // label of the drive data was migrated to + HasOtherPaths bool // true if other connected non-decommissioned paths exist } func (s *Server) baseData(page, title string) map[string]interface{} { @@ -606,14 +612,7 @@ func (s *Server) backupsHandler(w http.ResponseWriter, r *http.Request) { } tier2GroupMap[item.DestPath] = grp } - switch item.Method { - case "restic": - grp.ResticItems = append(grp.ResticItems, item) - case "rsync": - grp.RsyncItems = append(grp.RsyncItems, item) - default: - grp.RsyncItems = append(grp.RsyncItems, item) - } + grp.Items = append(grp.Items, item) } var tier2Groups []Tier2DriveGroup for _, grp := range tier2GroupMap { @@ -629,10 +628,9 @@ func (s *Server) backupsHandler(w http.ResponseWriter, r *http.Request) { // Tier2DriveGroup holds grouped Tier 2 cross-drive backup items for one destination drive. type Tier2DriveGroup struct { - DestPath string - DestLabel string - ResticItems []backup.CrossDriveSummaryItem - RsyncItems []backup.CrossDriveSummaryItem + DestPath string + DestLabel string + Items []backup.CrossDriveSummaryItem } // AppBackupRow holds per-tier backup information for one app on the backup page. @@ -659,8 +657,6 @@ type AppBackupRow struct { // Tier 2: Cross-drive backup (configurable for all apps) Tier2Configured bool - Tier2Method string // "rsync", "restic" - Tier2MethodLabel string // "rsync", "restic" Tier2Dest string // destination label Tier2Schedule string // "Naponta", "Hetente" Tier2LastRun string @@ -668,7 +664,6 @@ type AppBackupRow struct { Tier2LastError string Tier2StatusBadge string // "Sikeres", "Hiba", "Fut...", "—" Tier2SizeHuman string - Tier2Browsable bool // true for rsync (plain files), false for restic // Drive disconnected — app's home drive is currently disconnected DriveDisconnected bool @@ -777,9 +772,6 @@ func (s *Server) buildAppBackupRows( row.Tier2Configured = false } else { row.Tier2Configured = true - row.Tier2Method = cfg.Method - row.Tier2MethodLabel = cfg.Method // "rsync" or "restic" - row.Tier2Browsable = cfg.Method == "rsync" row.Tier2Dest = destLabels[cfg.DestinationPath] if row.Tier2Dest == "" { row.Tier2Dest = cfg.DestinationPath @@ -854,21 +846,15 @@ func (s *Server) settingsCrossBackupHandler(w http.ResponseWriter, r *http.Reque // Preserve existing runtime status fields and config when disabling existing := s.settings.GetCrossDriveConfig(name) - var method, destPath, schedule string + var destPath, schedule string if enabled { - method = r.FormValue("cross_drive_method") destPath = r.FormValue("cross_drive_dest") schedule = r.FormValue("cross_drive_schedule") - // Validate method and schedule - if method != "rsync" && method != "restic" { - method = "rsync" - } if schedule != "daily" && schedule != "weekly" { schedule = "daily" } } else if existing != nil { // Preserve existing settings when disabling - method = existing.Method destPath = existing.DestinationPath schedule = existing.Schedule } @@ -877,7 +863,7 @@ func (s *Server) settingsCrossBackupHandler(w http.ResponseWriter, r *http.Reque if destPath != "" || existing != nil { cfg = &settings.CrossDriveBackup{ Enabled: enabled, - Method: method, + Method: "rsync", DestinationPath: destPath, Schedule: schedule, } @@ -896,8 +882,8 @@ func (s *Server) settingsCrossBackupHandler(w http.ResponseWriter, r *http.Reque return } - s.logger.Printf("[INFO] Cross-drive backup config saved for %s: method=%s dest=%s schedule=%s enabled=%v", - name, method, destPath, schedule, enabled) + s.logger.Printf("[INFO] Cross-drive backup config saved for %s: dest=%s schedule=%s enabled=%v", + name, destPath, schedule, enabled) http.Redirect(w, r, "/stacks/"+name+"/deploy?flash=Ment%C3%A9si+be%C3%A1ll%C3%ADt%C3%A1s+mentve.", http.StatusFound) } @@ -966,15 +952,25 @@ func (s *Server) settingsData() map[string]interface{} { // Storage paths with display data storagePaths := s.settings.GetStoragePaths() + connectedCount := 0 + for _, sp := range storagePaths { + if !sp.Disconnected && !sp.Decommissioned { + connectedCount++ + } + } var storageViews []StoragePathView for _, sp := range storagePaths { view := StoragePathView{ - StoragePath: sp, - StoppedApps: sp.StoppedStacks, + StoragePath: sp, + StoppedApps: sp.StoppedStacks, + HasOtherPaths: connectedCount > 1, } if sp.Disconnected { // Skip I/O calls on disconnected drives — they'd hang or fail view.IsMounted = false + } else if sp.Decommissioned { + view.IsMounted = false + view.MigratedToLabel = s.settings.GetStorageLabel(sp.MigratedTo) } else { view.IsMounted = system.IsMountPoint(sp.Path) view.AppDetails = s.appDetailsForPath(sp.Path) @@ -1297,7 +1293,7 @@ func (s *Server) settingsStorageAddHandler(w http.ResponseWriter, r *http.Reques } s.logger.Printf("[INFO] Storage path added: %s (%s)", path, label) - go s.syncFileBrowserMounts() + go s.SyncFileBrowserMounts() http.Redirect(w, r, "/settings?storage_msg=success&storage_detail="+url.QueryEscape("Adattároló sikeresen hozzáadva: "+path), http.StatusFound) } @@ -1339,7 +1335,7 @@ func (s *Server) settingsStorageRemoveHandler(w http.ResponseWriter, r *http.Req s.logger.Printf("[INFO] Storage path removed: %s", path) // Sync FileBrowser mounts after storage path removal - go s.syncFileBrowserMounts() + go s.SyncFileBrowserMounts() http.Redirect(w, r, "/settings?storage_msg=success&storage_detail="+url.QueryEscape("Adattároló eltávolítva: "+path), http.StatusFound) } @@ -1392,9 +1388,9 @@ func (s *Server) settingsStorageLabelHandler(w http.ResponseWriter, r *http.Requ http.Redirect(w, r, "/settings?storage_msg=success&storage_detail="+url.QueryEscape("Megnevezés módosítva: "+label), http.StatusFound) } -// syncFileBrowserMounts regenerates FileBrowser's docker-compose.yml and config.yaml +// SyncFileBrowserMounts regenerates FileBrowser's docker-compose.yml and config.yaml // with volume mounts and sources for all registered storage paths, then recreates the container. -func (s *Server) syncFileBrowserMounts() { +func (s *Server) SyncFileBrowserMounts() { stackDir := "/opt/docker/stacks/filebrowser" composePath := stackDir + "/docker-compose.yml" diff --git a/controller/internal/web/server.go b/controller/internal/web/server.go index 36eb4c2..19fc9a0 100644 --- a/controller/internal/web/server.go +++ b/controller/internal/web/server.go @@ -18,6 +18,7 @@ import ( "gitea.dooplex.hu/admin/felhom-controller/internal/selfupdate" "gitea.dooplex.hu/admin/felhom-controller/internal/settings" "gitea.dooplex.hu/admin/felhom-controller/internal/stacks" + "gitea.dooplex.hu/admin/felhom-controller/internal/storage" "gitea.dooplex.hu/admin/felhom-controller/internal/system" ) @@ -47,6 +48,9 @@ type Server struct { // Active raw mount for the attach wizard (empty when not in use) activeRawMount string + // Drive migration + driveMigrator *storage.DriveMigrator + // DR restore mode state restoreMu sync.RWMutex restorePlan *backup.RestorePlan @@ -85,7 +89,7 @@ func NewServer(cfg *config.Config, stackMgr *stacks.Manager, cpuCollector *syste } // Sync FileBrowser config on startup to ensure mounts and sources are current - go s.syncFileBrowserMounts() + go s.SyncFileBrowserMounts() return s } @@ -108,6 +112,11 @@ func (s *Server) SetStorageWatchdog(w *monitor.StorageWatchdog) { s.storageWatchdog = w } +// SetDriveMigrator sets the drive migration engine for full drive migration. +func (s *Server) SetDriveMigrator(dm *storage.DriveMigrator) { + s.driveMigrator = dm +} + // InRestoreMode returns true if the server is in DR restore mode. func (s *Server) InRestoreMode() bool { s.restoreMu.RLock() @@ -179,6 +188,8 @@ func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) { s.storageInitHandler(w, r) case path == "/settings/storage/attach": s.storageAttachHandler(w, r) + case path == "/settings/storage/migrate-drive": + s.migrateDrivePageHandler(w, r) case strings.HasPrefix(path, "/stacks/") && strings.HasSuffix(path, "/migrate"): name := strings.TrimPrefix(path, "/stacks/") name = strings.TrimSuffix(name, "/migrate") diff --git a/controller/internal/web/storage_handlers.go b/controller/internal/web/storage_handlers.go index 96dda9d..f8c42d3 100644 --- a/controller/internal/web/storage_handlers.go +++ b/controller/internal/web/storage_handlers.go @@ -1,6 +1,7 @@ package web import ( + "context" "encoding/json" "fmt" "net/http" @@ -18,11 +19,12 @@ import ( // activeDiskJob tracks an in-progress disk operation (format or migrate). type activeDiskJob struct { - mu sync.RWMutex - jobType string // "format" or "migrate" - done bool - fmtProg []storage.FormatProgress - migProg []storage.MigrateProgress + mu sync.RWMutex + jobType string // "format", "migrate", or "migrate-drive" + done bool + fmtProg []storage.FormatProgress + migProg []storage.MigrateProgress + driveMigProg []storage.DriveMigrateProgress } // DeployStorageInfo holds storage info for the deploy page (already-deployed apps). @@ -54,6 +56,26 @@ func (j *activeDiskJob) appendMigProg(p storage.MigrateProgress) { } } +// appendDriveMigProg adds a drive migration progress update to the job. +func (j *activeDiskJob) appendDriveMigProg(p storage.DriveMigrateProgress) { + j.mu.Lock() + defer j.mu.Unlock() + j.driveMigProg = append(j.driveMigProg, p) + if p.Step == "done" || p.Step == "error" { + j.done = true + } +} + +// lastDriveMigProg returns the most recent drive migration progress. +func (j *activeDiskJob) lastDriveMigProg() (storage.DriveMigrateProgress, bool) { + j.mu.RLock() + defer j.mu.RUnlock() + if len(j.driveMigProg) == 0 { + return storage.DriveMigrateProgress{}, false + } + return j.driveMigProg[len(j.driveMigProg)-1], true +} + // lastFmtProg returns the most recent format progress snapshot. func (j *activeDiskJob) lastFmtProg() (storage.FormatProgress, bool) { j.mu.RLock() @@ -162,6 +184,12 @@ func (s *Server) storageAPIHandler(w http.ResponseWriter, r *http.Request) { s.storageRestartAppsHandler(w, r) case path == "/api/storage/status" && r.Method == http.MethodGet: s.storageStatusHandler(w, r) + case path == "/api/storage/migrate-drive" && r.Method == http.MethodPost: + s.driveMigrateAPIHandler(w, r) + case path == "/api/storage/migrate-drive/status" && r.Method == http.MethodGet: + s.driveMigrateStatusHandler(w, r) + case path == "/api/storage/decommission/remove" && r.Method == http.MethodPost: + s.decommissionRemoveHandler(w, r) default: http.NotFound(w, r) } @@ -273,7 +301,7 @@ func (s *Server) storageInitAPIHandler(w http.ResponseWriter, r *http.Request) { } else { s.logger.Printf("[INFO] Storage path registered: %s (%s)", mountPath, label) // Sync FileBrowser mounts with new storage path - s.syncFileBrowserMounts() + s.SyncFileBrowserMounts() } }() @@ -393,8 +421,9 @@ func (s *Server) migratePageHandler(w http.ResponseWriter, r *http.Request, stac // storageMigrateAPIHandler handles POST /api/storage/migrate — starts migration job. func (s *Server) storageMigrateAPIHandler(w http.ResponseWriter, r *http.Request) { var req struct { - StackName string `json:"stack_name"` - TargetPath string `json:"target_path"` + StackName string `json:"stack_name"` + TargetPath string `json:"target_path"` + AutoDeleteStale *bool `json:"auto_delete_stale"` // default true } if err := json.NewDecoder(r.Body).Decode(&req); err != nil { jsonError(w, "Érvénytelen kérés", http.StatusBadRequest) @@ -476,6 +505,20 @@ func (s *Server) storageMigrateAPIHandler(w http.ResponseWriter, r *http.Request return s.updateStackHDDPath(name, newPath) } + autoDelete := true + if req.AutoDeleteStale != nil { + autoDelete = *req.AutoDeleteStale + } + + orch := &storage.MigrateOrchestrator{ + Sett: s.settings, + BackupTrigger: s.backupMgr, + Logger: s.logger, + } + opts := storage.MigrateOptions{ + AutoDeleteStale: autoDelete, + } + go func() { progressCh := make(chan storage.MigrateProgress, 64) go func() { @@ -484,12 +527,12 @@ func (s *Server) storageMigrateAPIHandler(w http.ResponseWriter, r *http.Request } }() - if err := storage.MigrateAppData(migrReq, stopFn, startFn, updateFn, progressCh); err != nil { + if err := orch.RunEnhancedMigration(migrReq, stopFn, startFn, updateFn, opts, progressCh); err != nil { s.logger.Printf("[ERROR] Migration failed: stack=%s: %v", req.StackName, err) } else { s.logger.Printf("[INFO] Migration complete: stack=%s → %s", req.StackName, req.TargetPath) // Sync FileBrowser mounts (storage paths may now have new app data) - go s.syncFileBrowserMounts() + go s.SyncFileBrowserMounts() } close(progressCh) }() @@ -1033,7 +1076,7 @@ func (s *Server) storageAttachAPIHandler(w http.ResponseWriter, r *http.Request) s.logger.Printf("[WARN] Failed to register storage path after attach: %v", err) } else { s.logger.Printf("[INFO] Storage path registered: %s (%s)", mountPath, label) - s.syncFileBrowserMounts() + s.SyncFileBrowserMounts() } }() @@ -1251,3 +1294,228 @@ func (s *Server) storageStatusHandler(w http.ResponseWriter, r *http.Request) { "data": result, }) } + +// migrateDrivePageHandler handles GET /settings/storage/migrate-drive. +func (s *Server) migrateDrivePageHandler(w http.ResponseWriter, r *http.Request) { + sourcePath := r.URL.Query().Get("source") + if sourcePath == "" { + http.Redirect(w, r, "/settings", http.StatusFound) + return + } + + data := s.baseData("settings", "Meghajtó kiváltása") + data["SourcePath"] = sourcePath + data["SourceLabel"] = s.settings.GetStorageLabel(sourcePath) + data["SourceDiskInfo"] = system.GetDiskUsage(sourcePath) + + // Find apps on source drive + type appInfo struct { + Name string + DisplayName string + } + var appsOnSource []appInfo + for _, stack := range s.stackMgr.GetStacks() { + if !stack.Deployed { + continue + } + cfg := s.stackMgr.LoadAppConfigByName(stack.Name) + if cfg != nil && cfg.Env["HDD_PATH"] == sourcePath { + appsOnSource = append(appsOnSource, appInfo{ + Name: stack.Name, + DisplayName: stack.Meta.DisplayName, + }) + } + } + data["AppsOnSource"] = appsOnSource + + // Available destination paths + type destPathInfo struct { + Path string + Label string + FreeHuman string + } + var destPaths []destPathInfo + for _, sp := range s.settings.GetConnectedPaths() { + if sp.Path == sourcePath { + continue + } + free := "" + if di := system.GetDiskUsage(sp.Path); di != nil { + free = fmt.Sprintf("%.1f GB", di.AvailGB) + } + destPaths = append(destPaths, destPathInfo{ + Path: sp.Path, + Label: sp.Label, + FreeHuman: free, + }) + } + data["DestPaths"] = destPaths + + // Tier 2 impact analysis + var tier2Impact []string + allCrossConfigs := s.settings.GetAllCrossDriveConfigs() + for _, app := range appsOnSource { + if cfg := allCrossConfigs[app.Name]; cfg != nil && cfg.Enabled { + tier2Impact = append(tier2Impact, fmt.Sprintf("%s: 2. szintű mentés → %s (automatikusan átirányításra kerül)", + app.DisplayName, s.settings.GetStorageLabel(cfg.DestinationPath))) + } + } + data["Tier2Impact"] = tier2Impact + + s.render(w, "migrate_drive", data) +} + +// driveMigrateAPIHandler handles POST /api/storage/migrate-drive — starts drive migration. +func (s *Server) driveMigrateAPIHandler(w http.ResponseWriter, r *http.Request) { + if s.driveMigrator == nil { + jsonError(w, "Meghajtó-migráció nem elérhető", http.StatusServiceUnavailable) + return + } + + var req struct { + SourcePath string `json:"source_path"` + DestPath string `json:"dest_path"` + } + if err := json.NewDecoder(r.Body).Decode(&req); err != nil { + jsonError(w, "Érvénytelen kérés", http.StatusBadRequest) + return + } + + if req.SourcePath == "" || req.DestPath == "" { + jsonError(w, "Hiányos paraméterek", http.StatusBadRequest) + return + } + + // Validate against registered paths + registeredPaths := s.settings.GetStoragePaths() + validSrc, validDst := false, false + for _, sp := range registeredPaths { + if sp.Path == req.SourcePath { + validSrc = true + } + if sp.Path == req.DestPath { + validDst = true + } + } + if !validSrc || !validDst { + jsonError(w, "Érvénytelen útvonal: nem regisztrált adattároló", http.StatusBadRequest) + return + } + + job, ok := s.tryStartDiskJob("migrate-drive") + if !ok { + jsonError(w, "Egy másik lemezművelet folyamatban van", http.StatusConflict) + return + } + + s.logger.Printf("[INFO] Drive migration started: %s → %s by %s", req.SourcePath, req.DestPath, r.RemoteAddr) + + go func() { + ctx := context.Background() + progressCh := make(chan storage.DriveMigrateProgress, 64) + go func() { + for p := range progressCh { + job.appendDriveMigProg(p) + } + }() + + migrReq := storage.DriveMigrateRequest{ + SourcePath: req.SourcePath, + DestPath: req.DestPath, + } + if err := s.driveMigrator.MigrateDrive(ctx, migrReq, progressCh); err != nil { + s.logger.Printf("[ERROR] Drive migration failed: %v", err) + } else { + s.logger.Printf("[INFO] Drive migration complete: %s → %s", req.SourcePath, req.DestPath) + go s.SyncFileBrowserMounts() + } + close(progressCh) + }() + + jsonResponse(w, map[string]interface{}{ + "ok": true, + "msg": "Meghajtó kiváltás elindítva", + }) +} + +// driveMigrateStatusHandler handles GET /api/storage/migrate-drive/status. +func (s *Server) driveMigrateStatusHandler(w http.ResponseWriter, r *http.Request) { + job := s.currentDiskJob() + if job == nil || job.jobType != "migrate-drive" { + jsonResponse(w, map[string]interface{}{ + "ok": true, + "active": false, + }) + return + } + + p, ok := job.lastDriveMigProg() + if !ok { + jsonResponse(w, map[string]interface{}{ + "ok": true, + "active": true, + "step": "validating", + "msg": "Meghajtó kiváltás elindult...", + "pct": 0, + }) + return + } + + jsonResponse(w, map[string]interface{}{ + "ok": true, + "active": !job.isDone(), + "step": p.Step, + "msg": p.Message, + "detail": p.Detail, + "pct": p.Percent, + "error": p.Error, + "done": job.isDone(), + "elapsed_sec": p.ElapsedSeconds, + }) +} + +// decommissionRemoveHandler handles POST /api/storage/decommission/remove. +func (s *Server) decommissionRemoveHandler(w http.ResponseWriter, r *http.Request) { + var req struct { + Path string `json:"storage_path"` + } + + // Support both form and JSON + if r.Header.Get("Content-Type") == "application/json" { + if err := json.NewDecoder(r.Body).Decode(&req); err != nil { + jsonError(w, "Érvénytelen kérés", http.StatusBadRequest) + return + } + } else { + _ = r.ParseForm() + req.Path = r.FormValue("storage_path") + } + + if req.Path == "" { + jsonError(w, "Hiányzó útvonal", http.StatusBadRequest) + return + } + + if !s.settings.IsDecommissioned(req.Path) { + jsonError(w, "A meghajtó nincs kiváltva állapotban", http.StatusBadRequest) + return + } + + if err := s.settings.RemoveStoragePath(req.Path); err != nil { + s.logger.Printf("[ERROR] Failed to remove decommissioned path %s: %v", req.Path, err) + jsonError(w, "Eltávolítás sikertelen: "+err.Error(), http.StatusInternalServerError) + return + } + + s.logger.Printf("[INFO] Decommissioned storage path removed: %s", req.Path) + + // For form submissions, redirect back to settings + if r.Header.Get("Content-Type") != "application/json" { + http.Redirect(w, r, "/settings?storage_msg=success&storage_detail=Meghajtó+eltávolítva+a+rendszerből.", http.StatusFound) + return + } + jsonResponse(w, map[string]interface{}{ + "ok": true, + "msg": "Meghajtó eltávolítva a rendszerből", + }) +} diff --git a/controller/internal/web/templates/backups.html b/controller/internal/web/templates/backups.html index bd69f6b..7dab908 100644 --- a/controller/internal/web/templates/backups.html +++ b/controller/internal/web/templates/backups.html @@ -296,7 +296,7 @@
2. mentés {{if .Tier2Configured}} - {{.Tier2MethodLabel}} + rsync → {{.Tier2Dest}} {{.Tier2Schedule}} {{if .Tier2LastRun}} @@ -308,7 +308,7 @@ {{end}} {{if .Tier2SizeHuman}}{{.Tier2SizeHuman}}{{end}} {{.BackupContents}} - {{if .Tier2Browsable}}📁{{end}} + 📁
Beállítás
-
- - Módszer - - i - - Egyszerű másolat (rsync): Tükörszerű másolat, a fájlok közvetlenül böngészhetők. - Nem titkosított, nem verziózott — mindig a legfrissebb állapotot tartalmazza. -

- Titkosított mentés (restic): Titkosított, tömörített, verziózott mentés. - Korábbi állapotok visszaállíthatók. Nem böngészhető közvetlenül — - visszaállításhoz a vezérlőpult szükséges. - - - - -
Ütemezés
diff --git a/controller/internal/web/templates/migrate.html b/controller/internal/web/templates/migrate.html index da74f53..b3f5426 100644 --- a/controller/internal/web/templates/migrate.html +++ b/controller/internal/web/templates/migrate.html @@ -38,10 +38,19 @@
  • Az alkalmazás a mozgatás idejére leáll
  • Nagy adatmennyiségnél ez percekig tarthat
    • -
  • A régi adatok megmaradnak biztonsági másolatként
    • +
  • DB mentés fájlok is átkerülnek
    • +
  • A migráció után azonnal lefut egy biztonsági mentés az új meghajtón
+
+ + Ha bekapcsolva, a forrás meghajtóról az alkalmazás adatai és DB mentései automatikusan törlődnek a sikeres áthelyezés után. +
+
@@ -58,6 +67,8 @@
○ Adatok másolása
○ Konfiguráció frissítése
○ Alkalmazás indítása
+
○ Régi adatok törlése
+
○ Biztonsági mentés
○ Kész
@@ -75,16 +86,18 @@