diff --git a/controller/README.md b/controller/README.md index 95a0ef0..dc25239 100644 --- a/controller/README.md +++ b/controller/README.md @@ -85,7 +85,7 @@ A single, lightweight Go container that replaces Portainer + scattered systemd s | **Backup** | `internal/backup/` | Per-drive 3-layer backup: DB dumps → restic snapshots → cross-drive copies, restore | | **Storage** | `internal/storage/` | Disk scanning (`lsblk`), partitioning (`sfdisk`), formatting (`mkfs.ext4`), mounting, data migration (`rsync`) | | **System** | `internal/system/` | System info (`/proc`), CPU collector, mount points, disk usage, FS info | -| **Monitor** | `internal/monitor/` | Healthchecks.io pinger, system health checks | +| **Monitor** | `internal/monitor/` | Healthchecks.io pinger, system health checks, storage watchdog | | **Metrics** | `internal/metrics/` | SQLite time-series store, system + container metric collection | | **Scheduler** | `internal/scheduler/` | Central job scheduler (periodic + daily, skip-if-running, panic recovery) | | **SelfUpdate** | `internal/selfupdate/` | Version checking (registry), update trigger, state persistence, startup verification | @@ -403,8 +403,9 @@ Multiple external storage paths supported with: - **Label**: Human-readable name (editable inline) - **Default flag**: New deploys use this path by default - **Schedulable flag**: Path appears in deploy dropdown +- **Disconnected state**: `Disconnected`, `DisconnectedAt`, `StoppedStacks` — set by watchdog or safe-disconnect API, cleared on reconnect - **Auto-discovery**: On startup, scans deployed apps' `HDD_PATH` values and registers unknown paths -- Thread-safe CRUD: Add, Remove, SetDefault, SetSchedulable, SetLabel +- Thread-safe CRUD: Add, Remove, SetDefault, SetSchedulable, SetLabel, SetDisconnected, ClearDisconnected #### Data Migration (`internal/storage/migrate.go`) @@ -431,6 +432,39 @@ After migration, the deploy page detects leftover data on previous storage paths When storage paths are added or removed, `syncFileBrowserMounts()` auto-regenerates FileBrowser's `docker-compose.yml` with volume mounts for all registered paths, then recreates the container. +#### Storage Watchdog (`internal/monitor/watchdog.go`) + +Continuously monitors registered storage paths for disconnection/reconnection (primarily USB drives): + +- **Probe loop**: `ProbeStoragePath()` calls `syscall.Statfs()` with 3-second timeout in a goroutine. Runs every 5s per connected path, 30s per disconnected path. +- **Debouncing**: 3 consecutive probe failures required before declaring a drive disconnected (prevents false positives from transient I/O). +- **Disconnect reaction** (automatic, ~15s detection): + 1. Stops all deployed stacks whose `HDD_PATH` is under the disconnected drive (skips protected stacks) + 2. Persists `Disconnected`, `DisconnectedAt`, `StoppedStacks` to `settings.json` + 3. Lazy-unmounts stale VFS entries (`umount -l`) — for attach-wizard drives, unmounts bind first, then raw + 4. Fires alert refresh (red banner on all pages), notification (`storage_disconnected`), and immediate hub report push +- **Auto-reconnect** (for UUID-based fstab entries): + 1. Checks `/host-dev/disk/by-uuid/` for device reappearance + 2. Cleans stale mounts, then `mount -T /host-fstab ` (raw + bind for attach-wizard drives) + 3. Verifies with a post-mount probe + 4. Runs `restic unlock` if stale lock files exist + 5. Validates `StoppedStacks` (filters to actually-stopped stacks), clears `Disconnected` flag + 6. Fires alert refresh, notification (`storage_reconnected`), hub report push + +**Safe disconnect UI** (manual, Settings page): + +- "Leválasztás" button shown for USB drives (detected via sysfs symlink path containing `/usb`) +- Confirmation dialog lists affected apps +- Flow: stop apps → `sync` → `umount` (fallback `umount -l`) → mark disconnected → notification +- Disconnected card: dashed border, red badge, timestamp, stopped apps list, "Csatlakoztatás" (reconnect) button +- After reconnect: "Alkalmazások indítása" button to restart auto-stopped stacks + +**USB detection** (`system.IsUSBDevice`): Reads `/host/sys/block/` symlink — if target path contains `/usb`, it's a USB device. The `removable` sysfs flag is unreliable for USB HDDs (returns 0). + +**Backup guards**: Nightly DB dumps, restic snapshots, and cross-drive backups all skip disconnected drives with WARN log (not treated as failures). + +**UI integration**: Disconnected drives show with hatched red bars on dashboard, monitoring, and backup pages. Per-app backup rows show "Meghajtó leválasztva" badge. Health check emits warnings for disconnected paths. + --- ### 4. Monitoring & Health @@ -446,7 +480,7 @@ When storage paths are added or removed, `syncFileBrowserMounts()` auto-regenera | CPU temperature | >= 75C | >= 85C | | Docker daemon | — | unreachable | | Protected containers | — | not running | -| Storage paths | not a mount point (data on SSD) | path inaccessible, disk >= 95% | +| Storage paths | not a mount point (data on SSD), drive disconnected | path inaccessible, disk >= 95% | Backup destination validation (`CheckBackupDestination`) has tiered checks: - Path doesn't exist → critical/blocked @@ -694,7 +728,7 @@ Runtime-mutable settings in `settings.json` (separate from infrastructure config | `notifications` | email, enabled events, cooldown hours | | `db_validations` | per-DB dump validation results (survives restarts) | | `app_backup` | per-app map: enabled flag, cross-drive config (method, dest, schedule, runtime status) | -| `storage_paths` | registered paths with label, default flag, schedulable flag | +| `storage_paths` | registered paths with label, default flag, schedulable flag, disconnected state | | `cross_drive_restic_password` | auto-generated restic password for cross-drive repos | All public methods use `sync.RWMutex`. File writes are atomic (`.tmp` + rename). @@ -704,7 +738,7 @@ All public methods use `sync.RWMutex`. File writes are atomic (`.tmp` + rename). Five sections: 1. **System config** — read-only display of `controller.yaml` values 2. **Version & update** — current/latest version, check/update buttons, auto-update status, last update result -3. **Storage paths** — add/remove, edit labels, set default, toggle schedulable, per-path app list with sizes +3. **Storage paths** — add/remove, edit labels, set default, toggle schedulable, per-path app list with sizes, safe disconnect/reconnect for USB drives 4. **Password change** — current + new + confirm, min 8 chars 5. **Notifications** — email, event checkboxes, cooldown hours, test email button @@ -805,10 +839,11 @@ controller/ │ ├── system/ │ │ ├── info.go, info_linux.go # RAM, disk, CPU, temperature, load average │ │ ├── cpu_linux.go # Background /proc/stat sampling -│ │ └── mounts_linux.go # Mount points, disk usage, FS info, backup dest checks +│ │ └── mounts_linux.go # Mount points, disk usage, FS info, backup dest checks, storage probing, USB detection │ ├── monitor/ │ │ ├── pinger.go # Healthchecks.io HTTP ping client -│ │ └── healthcheck.go # System health checks (disk, mem, CPU, temp, Docker) +│ │ ├── healthcheck.go # System health checks (disk, mem, CPU, temp, Docker) +│ │ └── watchdog.go # Storage watchdog (probe, disconnect/reconnect, safe eject) │ ├── metrics/ │ │ ├── store.go # SQLite time-series (WAL mode, downsampled queries) │ │ ├── collector.go # Background collector (60s, system + docker stats) @@ -827,7 +862,7 @@ controller/ │ ├── auth.go # Session auth, login/logout, session cleanup │ ├── handlers.go # Page handlers (dashboard, stacks, deploy, backups, etc.) │ ├── handler_restore.go # DR: restore page handler + APIs (scan, restore all, skip) -│ ├── storage_handlers.go # Storage API handlers (scan, format, attach, migrate, cleanup) +│ ├── storage_handlers.go # Storage API handlers (scan, format, attach, migrate, cleanup, disconnect/reconnect) │ ├── alerts.go # State-based alert generation │ ├── funcmap.go # Template functions (state colors, Hungarian formatting) │ ├── embed.go # go:embed for templates + Chart.js @@ -973,6 +1008,10 @@ All daily jobs use Europe/Budapest timezone. Skip-if-running prevents concurrent | POST | `/api/storage/attach/cancel` | Cleanup temp raw mount | | POST | `/api/storage/migrate` | Start app data migration | | GET | `/api/storage/migrate/status` | Migration progress | +| POST | `/api/storage/disconnect` | Safe disconnect (stop apps, unmount) | +| POST | `/api/storage/reconnect` | Reconnect disconnected drive | +| POST | `/api/storage/restart-apps` | Restart auto-stopped apps | +| GET | `/api/storage/status` | All storage paths with connection state | ### Self-Update @@ -1060,6 +1099,7 @@ See `docker-compose.yml` for the full volume configuration. - [x] Storage management (scan, format, mount, registry) - [x] Attach existing drive wizard (v0.15.0) — bind-mount subfolder from pre-formatted drive, directory browser - [x] App data migration between storage paths +- [x] Storage watchdog (v0.17.0) — USB disconnect detection (~15s), auto-stop apps, auto-remount on reconnect, safe eject UI - [x] Central hub reporting - [x] Email notifications via hub relay - [x] Settings persistence and password management diff --git a/controller/cmd/controller/main.go b/controller/cmd/controller/main.go index 55a0143..80e531b 100644 --- a/controller/cmd/controller/main.go +++ b/controller/cmd/controller/main.go @@ -284,7 +284,7 @@ func main() { latestVersion = status.LastCheck.LatestVersion } } - alertMgr.Refresh(healthReport, cfg, backupMgr, updateAvailable, latestVersion) + alertMgr.Refresh(healthReport, cfg, backupMgr, updateAvailable, latestVersion, sett.GetStoragePaths()) // Notify on health status changes notifier.NotifyHealthChange(healthReport.Status, healthReport.Issues, healthReport.Warnings) return nil @@ -409,6 +409,36 @@ func main() { } } + // --- Storage watchdog --- + storageWatchdog := monitor.NewStorageWatchdog(sett, &watchdogStackAdapter{mgr: stackMgr}, notifier, cfg, logger) + storageWatchdog.SetAlertRefresh(func() { + healthReport := monitor.RunHealthCheck(cfg, cpuCollector, sett.GetStoragePaths()) + updateAvailable := false + latestVersion := "" + if updater != nil { + status := updater.GetStatus() + if status.LastCheck != nil { + updateAvailable = status.LastCheck.UpdateAvailable + latestVersion = status.LastCheck.LatestVersion + } + } + alertMgr.Refresh(healthReport, cfg, backupMgr, updateAvailable, latestVersion, sett.GetStoragePaths()) + }) + if hubPusher != nil { + storageWatchdog.SetHubReportPusher(func() { + r := report.BuildReport(cfg, stackMgr, backupMgr, cpuCollector, metricsStore, Version, sett.GetStoragePaths()) + hubPusher.Push(r) + }) + } + if backupMgr != nil { + storageWatchdog.SetRepoUnlocker(func(ctx context.Context, repoPath string) error { + return backupMgr.UnlockRepo(ctx, repoPath) + }) + } + sched.Every("storage-watchdog", 5*time.Second, func(ctx context.Context) error { + return storageWatchdog.Check(ctx) + }) + sched.Start(ctx) defer sched.Stop() @@ -513,6 +543,7 @@ func main() { // --- Initialize web server --- webServer := web.NewServer(cfg, stackMgr, cpuCollector, backupMgr, crossDriveRunner, sched, sett, alertMgr, notifier, updater, logger, Version) + webServer.SetStorageWatchdog(storageWatchdog) // Phase 3: Set DR restore mode if a restore plan was built if restorePlan != nil && len(restorePlan.Apps) > 0 { @@ -673,6 +704,43 @@ func (a *stackAdapter) GetStackHDDPath(name string) string { return "" } +// watchdogStackAdapter implements monitor.WatchdogStackProvider using stacks.Manager. +type watchdogStackAdapter struct { + mgr *stacks.Manager +} + +func (a *watchdogStackAdapter) ListDeployedStacks() []monitor.WatchdogStackInfo { + var result []monitor.WatchdogStackInfo + for _, s := range a.mgr.GetStacks() { + if !s.Deployed { + continue + } + result = append(result, monitor.WatchdogStackInfo{Name: s.Name}) + } + return result +} + +func (a *watchdogStackAdapter) GetStackHDDPath(name string) string { + s, ok := a.mgr.GetStack(name) + if !ok { + return "" + } + stackDir := filepath.Dir(s.ComposePath) + appCfg := stacks.LoadAppConfig(stackDir) + if appCfg != nil && appCfg.Env["HDD_PATH"] != "" { + return filepath.Clean(appCfg.Env["HDD_PATH"]) + } + return "" +} + +func (a *watchdogStackAdapter) StopStack(name string) error { + return a.mgr.StopStack(name) +} + +func (a *watchdogStackAdapter) StartStack(name string) error { + return a.mgr.StartStack(name) +} + // pushInfraBackup builds and sends the infrastructure snapshot to the Hub. func pushInfraBackup(cfg *config.Config, sett *settings.Settings, stackProv *stackAdapter, pusher *report.Pusher, logger *log.Logger) { diff --git a/controller/internal/backup/backup.go b/controller/internal/backup/backup.go index 10cff81..19d332f 100644 --- a/controller/internal/backup/backup.go +++ b/controller/internal/backup/backup.go @@ -242,6 +242,14 @@ func (m *Manager) runDBDumpsInternal(ctx context.Context) error { for _, db := range dbs { drivePath := m.GetAppDrivePath(db.StackName) + + // Skip if drive is disconnected + if m.settings != nil && m.settings.IsDisconnected(drivePath) { + m.logger.Printf("[WARN] Skipping DB dump for %s — drive disconnected: %s", db.StackName, drivePath) + summary = append(summary, fmt.Sprintf("SKIP %s (drive disconnected)", db.ContainerName)) + continue + } + dumpDir := AppDBDumpPath(drivePath, db.StackName) result := DumpOne(ctx, db, dumpDir, m.logger) @@ -331,6 +339,12 @@ func (m *Manager) runBackupInternal(ctx context.Context) error { driveCount := 0 for drivePath, stacks := range driveStacks { + // Skip disconnected drives + if m.settings != nil && m.settings.IsDisconnected(drivePath) { + m.logger.Printf("[WARN] Skipping backup for drive %s — disconnected", drivePath) + continue + } + repoPath := PrimaryResticRepoPath(drivePath) // Ensure repo is initialized @@ -650,6 +664,20 @@ func (m *Manager) SetStackProvider(provider StackDataProvider) { m.mu.Unlock() } +// UnlockRepo runs restic unlock on the given repo path. +func (m *Manager) UnlockRepo(ctx context.Context, repoPath string) error { + if !m.restic.RepoExists(repoPath) { + return nil // no repo to unlock + } + cmd := m.restic.UnlockCommand(ctx, repoPath) + out, err := cmd.CombinedOutput() + if err != nil { + return fmt.Errorf("restic unlock: %v (%s)", err, strings.TrimSpace(string(out))) + } + m.logger.Printf("[INFO] Restic repo unlocked: %s", repoPath) + return nil +} + // GetStackHDDMounts returns HDD mount paths for the named stack via the stack provider. func (m *Manager) GetStackHDDMounts(name string) []string { if m.stackProvider == nil { diff --git a/controller/internal/backup/crossdrive.go b/controller/internal/backup/crossdrive.go index 2b83a2e..8357bdb 100644 --- a/controller/internal/backup/crossdrive.go +++ b/controller/internal/backup/crossdrive.go @@ -81,6 +81,21 @@ func (r *CrossDriveRunner) RunAppBackup(ctx context.Context, stackName string) e r.mu.Unlock() }() + // Check if source or destination drive is disconnected + srcDrive := r.stackProvider.GetStackHDDPath(stackName) + if srcDrive != "" && r.sett.IsDisconnected(srcDrive) { + r.mu.Lock() + r.running[stackName] = false + r.mu.Unlock() + return fmt.Errorf("source drive disconnected: %s", srcDrive) + } + if r.sett.IsDisconnected(cfg.DestinationPath) { + r.mu.Lock() + r.running[stackName] = false + r.mu.Unlock() + return fmt.Errorf("destination drive disconnected: %s", cfg.DestinationPath) + } + // Mark as running in settings _ = r.sett.UpdateCrossDriveStatus(stackName, func(c *settings.CrossDriveBackup) { c.LastStatus = "running" diff --git a/controller/internal/backup/restic.go b/controller/internal/backup/restic.go index 63e7c91..dd9f405 100644 --- a/controller/internal/backup/restic.go +++ b/controller/internal/backup/restic.go @@ -348,6 +348,11 @@ func (r *ResticManager) RepoExists(repoPath string) bool { return err == nil } +// UnlockCommand returns an exec.Cmd that runs restic unlock on the given repo. +func (r *ResticManager) UnlockCommand(ctx context.Context, repoPath string) *exec.Cmd { + return r.command(ctx, repoPath, "unlock") +} + func (r *ResticManager) command(ctx context.Context, repoPath string, args ...string) *exec.Cmd { cmd := exec.CommandContext(ctx, "restic", args...) cmd.Env = append(os.Environ(), diff --git a/controller/internal/monitor/healthcheck.go b/controller/internal/monitor/healthcheck.go index 66a3d95..76e2cac 100644 --- a/controller/internal/monitor/healthcheck.go +++ b/controller/internal/monitor/healthcheck.go @@ -172,6 +172,12 @@ func checkProtectedContainers(protected []string) []string { func checkStoragePaths(paths []settings.StoragePath) (issues, warnings []string) { for _, sp := range paths { + // Skip disconnected paths — handled by the storage watchdog + if sp.Disconnected { + warnings = append(warnings, fmt.Sprintf("Meghajtó leválasztva: %s (%s)", sp.Label, sp.Path)) + continue + } + // Path accessible? if _, err := os.Stat(sp.Path); err != nil { warnings = append(warnings, fmt.Sprintf("Adattároló nem elérhető: %s", sp.Path)) diff --git a/controller/internal/monitor/watchdog.go b/controller/internal/monitor/watchdog.go new file mode 100644 index 0000000..4b73e8c --- /dev/null +++ b/controller/internal/monitor/watchdog.go @@ -0,0 +1,612 @@ +package monitor + +import ( + "context" + "fmt" + "log" + "os" + "os/exec" + "path/filepath" + "strings" + "sync" + "time" + + "gitea.dooplex.hu/admin/felhom-controller/internal/config" + "gitea.dooplex.hu/admin/felhom-controller/internal/notify" + "gitea.dooplex.hu/admin/felhom-controller/internal/settings" + "gitea.dooplex.hu/admin/felhom-controller/internal/system" +) + +const ( + // probeThreshold is the number of consecutive probe failures before declaring disconnected. + probeThreshold = 3 + + // defaultProbeInterval is the normal probe interval for connected drives. + defaultProbeInterval = 5 * time.Second + + // disconnectedProbeInterval is the slower probe interval for disconnected drives + // (checking for UUID reappearance, not I/O probing). + disconnectedProbeInterval = 30 * time.Second + + // hostFstabPath is where the host's fstab is mounted inside the container. + hostFstabPath = "/host-fstab" + + // hostDevUUIDPath is where the host's /dev/disk/by-uuid is accessible. + hostDevUUIDPath = "/host-dev/disk/by-uuid" + + // primaryResticSubpath is the relative path to the primary restic repo under a drive. + primaryResticSubpath = "backups/primary/restic" +) + +// WatchdogStackInfo holds minimal stack info for the watchdog. +type WatchdogStackInfo struct { + Name string +} + +// WatchdogStackProvider provides stack operations needed by the watchdog. +// Defined here to avoid circular imports with the backup package. +type WatchdogStackProvider interface { + ListDeployedStacks() []WatchdogStackInfo + GetStackHDDPath(name string) string + StopStack(name string) error + StartStack(name string) error +} + +// pathProbeState tracks in-memory probe state for a single storage path. +type pathProbeState struct { + consecutiveFailures int + lastStatus string // "connected", "disconnected" + lastProbeTime time.Time + probeInterval time.Duration +} + +// StorageWatchdog monitors registered storage paths and reacts to disconnection/reconnection. +type StorageWatchdog struct { + settings *settings.Settings + stackProvider WatchdogStackProvider + notifier *notify.Notifier + cfg *config.Config + logger *log.Logger + + // Callbacks to break import cycles — set via SetXxx methods after construction + alertRefresh func() + pushHubReport func() + unlockRepo func(ctx context.Context, repoPath string) error + + mu sync.Mutex + pathState map[string]*pathProbeState +} + +// NewStorageWatchdog creates a new storage watchdog. +func NewStorageWatchdog( + sett *settings.Settings, + stackProvider WatchdogStackProvider, + notifier *notify.Notifier, + cfg *config.Config, + logger *log.Logger, +) *StorageWatchdog { + return &StorageWatchdog{ + settings: sett, + stackProvider: stackProvider, + notifier: notifier, + cfg: cfg, + logger: logger, + pathState: make(map[string]*pathProbeState), + } +} + +// SetAlertRefresh sets the callback to trigger alert refresh. +func (w *StorageWatchdog) SetAlertRefresh(fn func()) { + w.alertRefresh = fn +} + +// SetHubReportPusher sets the callback to push an immediate hub report. +func (w *StorageWatchdog) SetHubReportPusher(fn func()) { + w.pushHubReport = fn +} + +// SetRepoUnlocker sets the callback to unlock a restic repo on reconnect. +func (w *StorageWatchdog) SetRepoUnlocker(fn func(ctx context.Context, repoPath string) error) { + w.unlockRepo = fn +} + +// Check probes all registered storage paths and reacts to state changes. +// Called by the scheduler every 5 seconds. +func (w *StorageWatchdog) Check(ctx context.Context) error { + paths := w.settings.GetStoragePaths() + if len(paths) == 0 { + return nil + } + + for _, sp := range paths { + select { + case <-ctx.Done(): + return ctx.Err() + default: + } + + state := w.getOrCreateState(sp.Path) + + // Rate-limit per-path probes + if time.Since(state.lastProbeTime) < state.probeInterval { + continue + } + state.lastProbeTime = time.Now() + + if sp.Disconnected { + w.handleReconnectCheck(ctx, sp) + } else { + w.handleConnectedProbe(sp, state) + } + } + return nil +} + +// getOrCreateState returns the in-memory probe state for a path, creating if needed. +func (w *StorageWatchdog) getOrCreateState(path string) *pathProbeState { + w.mu.Lock() + defer w.mu.Unlock() + if s, ok := w.pathState[path]; ok { + return s + } + s := &pathProbeState{ + lastStatus: "connected", + probeInterval: defaultProbeInterval, + } + w.pathState[path] = s + return s +} + +// handleConnectedProbe probes a connected drive and triggers disconnect if needed. +func (w *StorageWatchdog) handleConnectedProbe(sp settings.StoragePath, state *pathProbeState) { + result := system.ProbeStoragePath(sp.Path) + if result.Status == system.ProbeConnected { + if state.consecutiveFailures > 0 { + w.logger.Printf("[DEBUG] [STORAGE] Probe recovered for %s after %d failures", sp.Path, state.consecutiveFailures) + } + state.consecutiveFailures = 0 + state.lastStatus = "connected" + return + } + + state.consecutiveFailures++ + w.logger.Printf("[WARN] [STORAGE] Probe failed for %s (%d/%d): %v", + sp.Path, state.consecutiveFailures, probeThreshold, result.Err) + + if state.consecutiveFailures >= probeThreshold { + w.handleDisconnect(sp, state, result) + } +} + +// handleDisconnect reacts to a confirmed drive disconnection. +func (w *StorageWatchdog) handleDisconnect(sp settings.StoragePath, state *pathProbeState, probe system.ProbeResult) { + label := sp.Label + if label == "" { + label = sp.Path + } + w.logger.Printf("[ERROR] [STORAGE] Drive disconnected: %s (%s)", sp.Path, label) + + // 1. Find and stop affected stacks + stoppedStacks := w.stopAffectedStacks(sp.Path) + + // 2. Mark disconnected in settings (persists to settings.json) + if err := w.settings.SetDisconnected(sp.Path, true, stoppedStacks); err != nil { + w.logger.Printf("[ERROR] [STORAGE] Failed to mark disconnected: %v", err) + } + + // 3. Lazy unmount stale mount (if probe timed out — mount is likely hanging) + if probe.Status == system.ProbeTimeout { + w.lazyUnmount(sp.Path) + } + + // 4. Update in-memory state + state.lastStatus = "disconnected" + state.probeInterval = disconnectedProbeInterval + state.consecutiveFailures = 0 + + // 5. Trigger alert refresh + if w.alertRefresh != nil { + w.alertRefresh() + } + + // 6. Send notification + w.notifier.NotifyStorageDisconnected(label, stoppedStacks) + + // 7. Push immediate hub report + if w.pushHubReport != nil { + go w.pushHubReport() + } +} + +// handleReconnectCheck checks if a disconnected drive has been reconnected. +func (w *StorageWatchdog) handleReconnectCheck(ctx context.Context, sp settings.StoragePath) { + // Find the UUID for this path from fstab + // For attach-wizard drives, the UUID is on the raw mount, not the bind mount + mountPath := sp.Path + rawPath, isAttachWizard := system.HasFelhomRawMount(hostFstabPath, sp.Path) + if isAttachWizard { + mountPath = rawPath + } + + uuid := system.ParseFstabUUID(hostFstabPath, mountPath) + if uuid == "" { + // No UUID in fstab — can't detect reconnection automatically + return + } + + // Check if the UUID block device is present + uuidPath := filepath.Join(hostDevUUIDPath, uuid) + if _, err := os.Stat(uuidPath); err != nil { + return // Drive not reconnected yet + } + + label := sp.Label + if label == "" { + label = sp.Path + } + w.logger.Printf("[INFO] [STORAGE] Drive reconnected (UUID found), attempting remount: %s (%s)", sp.Path, label) + + // Attempt remount + if err := w.remount(sp.Path, rawPath, isAttachWizard); err != nil { + w.logger.Printf("[ERROR] [STORAGE] Remount failed for %s: %v", sp.Path, err) + return // Try again next cycle + } + + // Verify with a probe + verifyResult := system.ProbeStoragePath(sp.Path) + if verifyResult.Status != system.ProbeConnected { + w.logger.Printf("[ERROR] [STORAGE] Post-remount probe failed for %s: %v", sp.Path, verifyResult.Err) + return + } + + w.logger.Printf("[INFO] [STORAGE] Drive successfully remounted: %s (%s)", sp.Path, label) + + // Clean stale restic locks + w.cleanResticLocks(ctx, sp.Path) + + // Validate stopped stacks — filter to only actually stopped ones + filteredStacks := w.filterStoppedStacks(sp.StoppedStacks) + + // Clear disconnected but preserve StoppedStacks for the restart UI + if err := w.settings.SetDisconnected(sp.Path, false, filteredStacks); err != nil { + w.logger.Printf("[ERROR] [STORAGE] Failed to clear disconnected: %v", err) + } + + // Update in-memory state + state := w.getOrCreateState(sp.Path) + state.lastStatus = "connected" + state.probeInterval = defaultProbeInterval + state.consecutiveFailures = 0 + + // Trigger alert refresh + if w.alertRefresh != nil { + w.alertRefresh() + } + + // Send notification + w.notifier.NotifyStorageReconnected(label) + + // Push immediate hub report + if w.pushHubReport != nil { + go w.pushHubReport() + } +} + +// stopAffectedStacks stops all deployed stacks whose HDD_PATH matches the disconnected drive. +func (w *StorageWatchdog) stopAffectedStacks(drivePath string) []string { + if w.stackProvider == nil { + return nil + } + + var stopped []string + cleanDrive := filepath.Clean(drivePath) + + for _, stack := range w.stackProvider.ListDeployedStacks() { + hddPath := w.stackProvider.GetStackHDDPath(stack.Name) + if hddPath == "" { + continue + } + cleanHDD := filepath.Clean(hddPath) + if cleanHDD != cleanDrive && !strings.HasPrefix(cleanHDD, cleanDrive+"/") { + continue + } + + // Don't stop protected stacks + if w.cfg.IsProtectedStack(stack.Name) { + w.logger.Printf("[WARN] [STORAGE] Skipping protected stack: %s", stack.Name) + continue + } + + w.logger.Printf("[INFO] [STORAGE] Stopping stack %s (drive disconnected: %s)", stack.Name, drivePath) + if err := w.stackProvider.StopStack(stack.Name); err != nil { + w.logger.Printf("[ERROR] [STORAGE] Failed to stop stack %s: %v", stack.Name, err) + continue // Don't add to stopped list if stop failed + } + stopped = append(stopped, stack.Name) + } + + if len(stopped) > 0 { + w.logger.Printf("[INFO] [STORAGE] Stopped %d stack(s) due to drive disconnect: %v", len(stopped), stopped) + } + return stopped +} + +// lazyUnmount performs a lazy unmount of a path and its raw mount (if attach-wizard). +func (w *StorageWatchdog) lazyUnmount(path string) { + // For attach-wizard, unmount bind first, then raw + rawPath, isAttachWizard := system.HasFelhomRawMount(hostFstabPath, path) + + // Unmount the bind/main path + cmd := exec.Command("umount", "-l", path) + if out, err := cmd.CombinedOutput(); err != nil { + w.logger.Printf("[WARN] [STORAGE] umount -l %s: %v (%s)", path, err, strings.TrimSpace(string(out))) + } else { + w.logger.Printf("[INFO] [STORAGE] Lazy unmounted: %s", path) + } + + // Then unmount the raw path if it's an attach-wizard drive + if isAttachWizard && rawPath != "" { + cmd = exec.Command("umount", "-l", rawPath) + if out, err := cmd.CombinedOutput(); err != nil { + w.logger.Printf("[WARN] [STORAGE] umount -l %s: %v (%s)", rawPath, err, strings.TrimSpace(string(out))) + } else { + w.logger.Printf("[INFO] [STORAGE] Lazy unmounted raw: %s", rawPath) + } + } +} + +// remount attempts to remount a storage path using fstab entries. +func (w *StorageWatchdog) remount(path, rawPath string, isAttachWizard bool) error { + // Clean any stale mount entries first + exec.Command("umount", "-l", path).Run() + if isAttachWizard && rawPath != "" { + exec.Command("umount", "-l", rawPath).Run() + } + + if isAttachWizard && rawPath != "" { + // Mount raw first, then bind + cmd := exec.Command("mount", "-T", hostFstabPath, rawPath) + if out, err := cmd.CombinedOutput(); err != nil { + return fmt.Errorf("mount raw %s: %v (%s)", rawPath, err, strings.TrimSpace(string(out))) + } + w.logger.Printf("[INFO] [STORAGE] Mounted raw: %s", rawPath) + + cmd = exec.Command("mount", "-T", hostFstabPath, path) + if out, err := cmd.CombinedOutput(); err != nil { + return fmt.Errorf("mount bind %s: %v (%s)", path, err, strings.TrimSpace(string(out))) + } + w.logger.Printf("[INFO] [STORAGE] Mounted bind: %s", path) + } else { + cmd := exec.Command("mount", "-T", hostFstabPath, path) + if out, err := cmd.CombinedOutput(); err != nil { + return fmt.Errorf("mount %s: %v (%s)", path, err, strings.TrimSpace(string(out))) + } + w.logger.Printf("[INFO] [STORAGE] Mounted: %s", path) + } + return nil +} + +// cleanResticLocks runs restic unlock on the primary repo for a drive path. +func (w *StorageWatchdog) cleanResticLocks(ctx context.Context, drivePath string) { + repoPath := filepath.Join(drivePath, primaryResticSubpath) + locksDir := filepath.Join(repoPath, "locks") + entries, err := os.ReadDir(locksDir) + if err != nil || len(entries) == 0 { + return // No locks dir or no lock files + } + + w.logger.Printf("[INFO] [STORAGE] Found %d restic lock file(s) in %s, running unlock", len(entries), repoPath) + + if w.unlockRepo != nil { + if err := w.unlockRepo(ctx, repoPath); err != nil { + w.logger.Printf("[WARN] [STORAGE] Restic unlock failed for %s: %v", repoPath, err) + } + } +} + +// filterStoppedStacks validates that stacks in the list still exist as deployed stacks. +func (w *StorageWatchdog) filterStoppedStacks(stackNames []string) []string { + if w.stackProvider == nil || len(stackNames) == 0 { + return nil + } + + deployed := make(map[string]bool) + for _, s := range w.stackProvider.ListDeployedStacks() { + deployed[s.Name] = true + } + + var result []string + for _, name := range stackNames { + if deployed[name] { + result = append(result, name) + } + } + return result +} + +// SafeDisconnect performs a safe disconnect of a storage path. +// Stops affected apps, syncs filesystem, and unmounts the drive. +func (w *StorageWatchdog) SafeDisconnect(ctx context.Context, path string) (stoppedStacks []string, err error) { + sp := w.findStoragePath(path) + if sp == nil { + return nil, fmt.Errorf("storage path %q not found", path) + } + if sp.Disconnected { + return nil, fmt.Errorf("drive already disconnected") + } + + label := sp.Label + if label == "" { + label = sp.Path + } + w.logger.Printf("[INFO] [STORAGE] Safe disconnect requested: %s (%s)", path, label) + + // 1. Stop affected stacks + stoppedStacks = w.stopAffectedStacks(path) + + // 2. Sync filesystem + exec.Command("sync").Run() + + // 3. Unmount + rawPath, isAttachWizard := system.HasFelhomRawMount(hostFstabPath, path) + + // Unmount bind/main + cmd := exec.Command("umount", path) + if out, umountErr := cmd.CombinedOutput(); umountErr != nil { + // Try lazy unmount as fallback + w.logger.Printf("[WARN] [STORAGE] umount %s failed, trying lazy: %v", path, umountErr) + cmd = exec.Command("umount", "-l", path) + if out, umountErr = cmd.CombinedOutput(); umountErr != nil { + return stoppedStacks, fmt.Errorf("umount %s failed: %v (%s)", path, umountErr, strings.TrimSpace(string(out))) + } + } + + // Unmount raw if attach-wizard + if isAttachWizard && rawPath != "" { + cmd = exec.Command("umount", rawPath) + if out, umountErr := cmd.CombinedOutput(); umountErr != nil { + cmd = exec.Command("umount", "-l", rawPath) + if out, umountErr = cmd.CombinedOutput(); umountErr != nil { + w.logger.Printf("[WARN] [STORAGE] umount raw %s failed: %v (%s)", rawPath, umountErr, strings.TrimSpace(string(out))) + } + } + } + + // 4. Mark disconnected + if setErr := w.settings.SetDisconnected(path, true, stoppedStacks); setErr != nil { + w.logger.Printf("[ERROR] [STORAGE] Failed to mark disconnected: %v", setErr) + } + + // 5. Update in-memory state + state := w.getOrCreateState(path) + state.lastStatus = "disconnected" + state.probeInterval = disconnectedProbeInterval + state.consecutiveFailures = 0 + + // 6. Trigger alert refresh + if w.alertRefresh != nil { + w.alertRefresh() + } + + // 7. Notify and push hub report + w.notifier.Notify("storage_safe_disconnect", "info", + fmt.Sprintf("Meghajtó biztonságosan leválasztva: %s", label), "") + if w.pushHubReport != nil { + go w.pushHubReport() + } + + w.logger.Printf("[INFO] [STORAGE] Safe disconnect completed: %s — drive can be removed", path) + return stoppedStacks, nil +} + +// Reconnect attempts to remount a disconnected storage path. +func (w *StorageWatchdog) Reconnect(ctx context.Context, path string) (stoppedStacks []string, err error) { + sp := w.findStoragePath(path) + if sp == nil { + return nil, fmt.Errorf("storage path %q not found", path) + } + if !sp.Disconnected { + return nil, fmt.Errorf("drive is not disconnected") + } + + label := sp.Label + if label == "" { + label = sp.Path + } + + // Check UUID availability + mountPath := sp.Path + rawPath, isAttachWizard := system.HasFelhomRawMount(hostFstabPath, sp.Path) + if isAttachWizard { + mountPath = rawPath + } + uuid := system.ParseFstabUUID(hostFstabPath, mountPath) + if uuid != "" { + uuidPath := filepath.Join(hostDevUUIDPath, uuid) + if _, statErr := os.Stat(uuidPath); statErr != nil { + return nil, fmt.Errorf("drive not detected (UUID %s not found) — ensure the drive is physically connected", uuid) + } + } + + // Attempt remount + if mountErr := w.remount(path, rawPath, isAttachWizard); mountErr != nil { + return nil, fmt.Errorf("mount failed: %w", mountErr) + } + + // Verify + verifyResult := system.ProbeStoragePath(path) + if verifyResult.Status != system.ProbeConnected { + return nil, fmt.Errorf("mount appeared to succeed but probe failed: %v", verifyResult.Err) + } + + // Clean restic locks + w.cleanResticLocks(ctx, path) + + // Validate stopped stacks + filteredStacks := w.filterStoppedStacks(sp.StoppedStacks) + + // Clear disconnected, preserve stopped stacks for restart UI + if setErr := w.settings.SetDisconnected(path, false, filteredStacks); setErr != nil { + w.logger.Printf("[ERROR] [STORAGE] Failed to clear disconnected: %v", setErr) + } + + // Update in-memory state + state := w.getOrCreateState(path) + state.lastStatus = "connected" + state.probeInterval = defaultProbeInterval + state.consecutiveFailures = 0 + + // Trigger alert refresh + if w.alertRefresh != nil { + w.alertRefresh() + } + + // Notify + w.notifier.NotifyStorageReconnected(label) + if w.pushHubReport != nil { + go w.pushHubReport() + } + + w.logger.Printf("[INFO] [STORAGE] Reconnect completed: %s", path) + return filteredStacks, nil +} + +// RestartStoppedApps restarts apps that were auto-stopped due to a drive disconnect. +func (w *StorageWatchdog) RestartStoppedApps(path string) (started, failed []string) { + sp := w.findStoragePath(path) + if sp == nil || sp.Disconnected { + return nil, nil + } + + stacks := w.settings.GetStoppedStacks(path) + if len(stacks) == 0 { + return nil, nil + } + + for _, name := range stacks { + w.logger.Printf("[INFO] [STORAGE] Starting stack %s (drive reconnected: %s)", name, path) + if err := w.stackProvider.StartStack(name); err != nil { + w.logger.Printf("[ERROR] [STORAGE] Failed to start stack %s: %v", name, err) + failed = append(failed, name) + } else { + started = append(started, name) + } + } + + // Clear stopped stacks list + if err := w.settings.ClearStoppedStacks(path); err != nil { + w.logger.Printf("[ERROR] [STORAGE] Failed to clear stopped stacks: %v", err) + } + + return started, failed +} + +// findStoragePath returns the storage path entry for a given path, or nil. +func (w *StorageWatchdog) findStoragePath(path string) *settings.StoragePath { + for _, sp := range w.settings.GetStoragePaths() { + if sp.Path == path { + return &sp + } + } + return nil +} diff --git a/controller/internal/notify/notifier.go b/controller/internal/notify/notifier.go index 689e9b7..6113a33 100644 --- a/controller/internal/notify/notifier.go +++ b/controller/internal/notify/notifier.go @@ -25,8 +25,9 @@ type Notifier struct { enabled bool settings *settings.Settings - mu sync.Mutex - cooldowns map[string]time.Time // event_type -> last notification time + mu sync.Mutex + cooldowns map[string]time.Time // event_type -> last notification time + perEventCooldown map[string]time.Duration // per-event override cooldown durations // prevHealthStatus tracks the previous health check status for change detection prevHealthStatus string @@ -50,6 +51,10 @@ func New(hubURL, apiKey, customerID string, sett *settings.Settings, logger *log enabled: enabled, settings: sett, cooldowns: make(map[string]time.Time), + perEventCooldown: map[string]time.Duration{ + "storage_disconnected": 1 * time.Hour, + "storage_reconnected": 1 * time.Hour, + }, } } @@ -141,11 +146,14 @@ func (n *Notifier) Notify(eventType, severity, message, details string) { return } - // Check cooldown + // Check cooldown — per-event override takes priority over global cooldownDuration := time.Duration(prefs.CooldownHours) * time.Hour if cooldownDuration == 0 { cooldownDuration = 6 * time.Hour } + if override, ok := n.perEventCooldown[eventType]; ok { + cooldownDuration = override + } n.mu.Lock() lastSent, exists := n.cooldowns[eventType] @@ -329,3 +337,19 @@ func classifyWarning(message string) string { func contains(s, substr string) bool { return strings.Contains(s, substr) } + +// NotifyStorageDisconnected sends a notification about a drive disconnection. +func (n *Notifier) NotifyStorageDisconnected(label string, stoppedApps []string) { + msg := fmt.Sprintf("Meghajtó váratlanul leválasztva: %s", label) + details := "" + if len(stoppedApps) > 0 { + details = fmt.Sprintf("Leállított alkalmazások: %s", strings.Join(stoppedApps, ", ")) + } + n.Notify("storage_disconnected", "critical", msg, details) +} + +// NotifyStorageReconnected sends a notification about a drive reconnection. +func (n *Notifier) NotifyStorageReconnected(label string) { + n.Notify("storage_reconnected", "info", + fmt.Sprintf("Meghajtó újra csatlakoztatva: %s. Az alkalmazások manuálisan indíthatók.", label), "") +} diff --git a/controller/internal/report/builder.go b/controller/internal/report/builder.go index 8428257..c7d735b 100644 --- a/controller/internal/report/builder.go +++ b/controller/internal/report/builder.go @@ -69,6 +69,14 @@ func BuildReport( {Mount: "/", Label: "SSD", TotalGB: sysInfo.DiskTotalGB, UsedGB: sysInfo.DiskUsedGB, Percent: sysInfo.DiskPercent}, } for _, sp := range storagePaths { + if sp.Disconnected { + r.Storage = append(r.Storage, StorageReport{ + Mount: sp.Path, + Label: sp.Label, + Disconnected: true, + }) + continue + } di := system.GetDiskUsage(sp.Path) if di == nil { continue diff --git a/controller/internal/report/types.go b/controller/internal/report/types.go index fd4d13c..c9f218d 100644 --- a/controller/internal/report/types.go +++ b/controller/internal/report/types.go @@ -39,11 +39,12 @@ type SystemReport struct { // StorageReport holds disk usage for a mount point. type StorageReport struct { - Mount string `json:"mount"` - Label string `json:"label,omitempty"` - TotalGB float64 `json:"total_gb"` - UsedGB float64 `json:"used_gb"` - Percent float64 `json:"percent"` + Mount string `json:"mount"` + Label string `json:"label,omitempty"` + TotalGB float64 `json:"total_gb"` + UsedGB float64 `json:"used_gb"` + Percent float64 `json:"percent"` + Disconnected bool `json:"disconnected,omitempty"` } // ContainerReport holds aggregate and per-container status. diff --git a/controller/internal/settings/settings.go b/controller/internal/settings/settings.go index 2bb737c..6048b0d 100644 --- a/controller/internal/settings/settings.go +++ b/controller/internal/settings/settings.go @@ -68,11 +68,14 @@ type CrossDriveBackup struct { // StoragePath represents a registered external storage location. type StoragePath struct { - Path string `json:"path"` // e.g., "/mnt/hdd_1" - Label string `json:"label,omitempty"` // e.g., "Külső HDD 1TB" - IsDefault bool `json:"is_default,omitempty"` // new apps use this by default - Schedulable bool `json:"schedulable"` // whether new apps can be deployed here - AddedAt string `json:"added_at"` // RFC3339 + Path string `json:"path"` // e.g., "/mnt/hdd_1" + Label string `json:"label,omitempty"` // e.g., "Külső HDD 1TB" + IsDefault bool `json:"is_default,omitempty"` // new apps use this by default + Schedulable bool `json:"schedulable"` // whether new apps can be deployed here + AddedAt string `json:"added_at"` // RFC3339 + Disconnected bool `json:"disconnected,omitempty"` // true when drive detected as disconnected + DisconnectedAt string `json:"disconnected_at,omitempty"` // RFC3339 timestamp of disconnect detection + StoppedStacks []string `json:"stopped_stacks,omitempty"` // stacks auto-stopped on disconnect } // NotificationPrefs holds customer notification preferences. @@ -87,6 +90,8 @@ var DefaultEnabledEvents = []string{ "disk_warning", "backup_failed", "update_available", + "storage_disconnected", + "storage_reconnected", } // DBValidationCache holds cached DB dump validation results. @@ -499,3 +504,109 @@ func InferStorageLabel(path string) string { } return fmt.Sprintf("Tárhely (%s)", base) } + +// SetDisconnected marks a storage path as disconnected (or connected) and records which stacks were stopped. +func (s *Settings) SetDisconnected(path string, disconnected bool, stoppedStacks []string) error { + s.mu.Lock() + defer s.mu.Unlock() + for i := range s.StoragePaths { + if s.StoragePaths[i].Path == path { + s.StoragePaths[i].Disconnected = disconnected + if disconnected { + s.StoragePaths[i].DisconnectedAt = time.Now().UTC().Format(time.RFC3339) + s.StoragePaths[i].StoppedStacks = stoppedStacks + } else { + s.StoragePaths[i].DisconnectedAt = "" + // Preserve StoppedStacks on reconnect so the UI can offer restart + if stoppedStacks != nil { + s.StoragePaths[i].StoppedStacks = stoppedStacks + } + } + return s.save() + } + } + return fmt.Errorf("storage path %q not found", path) +} + +// ClearDisconnected marks a path as connected and clears all disconnect-related fields. +func (s *Settings) ClearDisconnected(path string) error { + s.mu.Lock() + defer s.mu.Unlock() + for i := range s.StoragePaths { + if s.StoragePaths[i].Path == path { + s.StoragePaths[i].Disconnected = false + s.StoragePaths[i].DisconnectedAt = "" + s.StoragePaths[i].StoppedStacks = nil + return s.save() + } + } + return fmt.Errorf("storage path %q not found", path) +} + +// IsDisconnected returns whether a storage path is marked as disconnected. +func (s *Settings) IsDisconnected(path string) bool { + s.mu.RLock() + defer s.mu.RUnlock() + for _, sp := range s.StoragePaths { + if sp.Path == path { + return sp.Disconnected + } + } + return false +} + +// GetDisconnectedPaths returns a copy of all storage paths that are marked disconnected. +func (s *Settings) GetDisconnectedPaths() []StoragePath { + s.mu.RLock() + defer s.mu.RUnlock() + var result []StoragePath + for _, sp := range s.StoragePaths { + if sp.Disconnected { + result = append(result, sp) + } + } + return result +} + +// GetConnectedPaths returns a copy of all storage paths that are NOT disconnected. +func (s *Settings) GetConnectedPaths() []StoragePath { + s.mu.RLock() + defer s.mu.RUnlock() + var result []StoragePath + for _, sp := range s.StoragePaths { + if !sp.Disconnected { + result = append(result, sp) + } + } + return result +} + +// GetStoppedStacks returns the list of stacks that were auto-stopped for a storage path. +func (s *Settings) GetStoppedStacks(path string) []string { + s.mu.RLock() + defer s.mu.RUnlock() + for _, sp := range s.StoragePaths { + if sp.Path == path { + if len(sp.StoppedStacks) == 0 { + return nil + } + result := make([]string, len(sp.StoppedStacks)) + copy(result, sp.StoppedStacks) + return result + } + } + return nil +} + +// ClearStoppedStacks removes the stopped stacks list for a storage path (e.g., after restart). +func (s *Settings) ClearStoppedStacks(path string) error { + s.mu.Lock() + defer s.mu.Unlock() + for i := range s.StoragePaths { + if s.StoragePaths[i].Path == path { + s.StoragePaths[i].StoppedStacks = nil + return s.save() + } + } + return fmt.Errorf("storage path %q not found", path) +} diff --git a/controller/internal/system/mounts_linux.go b/controller/internal/system/mounts_linux.go index c4d384f..21eee9d 100644 --- a/controller/internal/system/mounts_linux.go +++ b/controller/internal/system/mounts_linux.go @@ -3,12 +3,14 @@ package system import ( + "bufio" "fmt" "os" "os/exec" "path/filepath" "strings" "syscall" + "time" ) // IsMountPoint checks if a path is on a different device than its parent. @@ -212,21 +214,22 @@ func isSameBlockDevice(pathA, pathB string) bool { return statA.Dev == statB.Dev } -// diskModel reads the disk model from /sys/block//device/model. -func diskModel(device string) string { - // /dev/sda1 → sda, /dev/nvme0n1p1 → nvme0n1 - base := filepath.Base(device) - // Strip partition number: sda1 → sda, nvme0n1p1 → nvme0n1 - disk := base +// stripPartition strips the partition suffix from a device name. +// e.g., "sda1" → "sda", "nvme0n1p1" → "nvme0n1". +func stripPartition(base string) string { if strings.HasPrefix(base, "nvme") { - // nvme0n1p1 → find last 'p' followed by digits if idx := strings.LastIndex(base, "p"); idx > 4 { - disk = base[:idx] + return base[:idx] } } else { - // sda1 → sda: strip trailing digits - disk = strings.TrimRight(base, "0123456789") + return strings.TrimRight(base, "0123456789") } + return base +} + +// diskModel reads the disk model from /sys/block//device/model. +func diskModel(device string) string { + disk := stripPartition(filepath.Base(device)) modelPath := "/sys/block/" + disk + "/device/model" data, err := os.ReadFile(modelPath) if err != nil { @@ -234,3 +237,151 @@ func diskModel(device string) string { } return strings.TrimSpace(string(data)) } + +// ProbeStatus represents the result of a storage path probe. +type ProbeStatus int + +const ( + ProbeConnected ProbeStatus = iota + ProbeDisconnected + ProbeTimeout +) + +// ProbeResult holds the outcome of a storage path probe. +type ProbeResult struct { + Status ProbeStatus + Err error +} + +// ProbeStoragePath checks if a storage path is responsive. +// Uses a goroutine with a 3-second timeout to avoid blocking on dead mounts. +func ProbeStoragePath(path string) ProbeResult { + // Quick check: does the path exist at all? + if _, err := os.Lstat(path); os.IsNotExist(err) { + return ProbeResult{Status: ProbeDisconnected, Err: err} + } + + type statResult struct { + err error + } + ch := make(chan statResult, 1) + go func() { + var stat syscall.Statfs_t + err := syscall.Statfs(path, &stat) + ch <- statResult{err: err} + }() + + select { + case res := <-ch: + if res.err == nil { + return ProbeResult{Status: ProbeConnected} + } + errStr := res.err.Error() + if strings.Contains(errStr, "transport endpoint") || + strings.Contains(errStr, "input/output error") || + strings.Contains(errStr, "no such device") { + return ProbeResult{Status: ProbeDisconnected, Err: res.err} + } + return ProbeResult{Status: ProbeDisconnected, Err: res.err} + case <-time.After(3 * time.Second): + return ProbeResult{Status: ProbeTimeout, Err: fmt.Errorf("stat timed out after 3s")} + } +} + +// IsUSBDevice checks if a block device is connected via USB. +// devicePath should be like "/dev/sdb" or "/dev/sdb1". +// Checks the sysfs symlink for the disk — if the path contains "/usb", it's a USB device. +func IsUSBDevice(devicePath string) bool { + disk := stripPartition(filepath.Base(devicePath)) + if disk == "" { + return false + } + // Try /host/sys first (Docker mount), then /sys (native) + for _, prefix := range []string{"/host/sys", "/sys"} { + link, err := os.Readlink(prefix + "/block/" + disk) + if err != nil { + continue + } + if strings.Contains(link, "/usb") { + return true + } + return false // found the sysfs entry, but not USB + } + return false +} + +// ParseFstabUUID extracts the UUID for a given mount point from an fstab file. +// Returns empty string if the mount point is not found or has no UUID. +func ParseFstabUUID(fstabPath, mountPath string) string { + f, err := os.Open(fstabPath) + if err != nil { + return "" + } + defer f.Close() + + cleanMount := filepath.Clean(mountPath) + scanner := bufio.NewScanner(f) + for scanner.Scan() { + line := strings.TrimSpace(scanner.Text()) + if line == "" || strings.HasPrefix(line, "#") { + continue + } + fields := strings.Fields(line) + if len(fields) < 2 { + continue + } + if filepath.Clean(fields[1]) != cleanMount { + continue + } + source := fields[0] + if strings.HasPrefix(source, "UUID=") { + return strings.TrimPrefix(source, "UUID=") + } + } + return "" +} + +// HasFelhomRawMount checks if a mount path was set up via the attach wizard +// (which uses a raw mount at /mnt/.felhom-raw/ + a bind mount). +// Returns the raw mount path if found, e.g., "/mnt/.felhom-raw/hdd_1". +func HasFelhomRawMount(fstabPath, mountPath string) (rawPath string, ok bool) { + f, err := os.Open(fstabPath) + if err != nil { + return "", false + } + defer f.Close() + + cleanMount := filepath.Clean(mountPath) + scanner := bufio.NewScanner(f) + for scanner.Scan() { + line := strings.TrimSpace(scanner.Text()) + if line == "" || strings.HasPrefix(line, "#") { + continue + } + fields := strings.Fields(line) + if len(fields) < 4 { + continue + } + // Look for a bind mount line targeting our mount path + // Format: /mnt/.felhom-raw/hdd_1/subfolder /mnt/hdd_1 none bind,... + target := filepath.Clean(fields[1]) + if target != cleanMount { + continue + } + source := fields[0] + if !strings.Contains(source, ".felhom-raw") { + continue + } + // Extract the raw mount path: /mnt/.felhom-raw/hdd_1/subfolder → /mnt/.felhom-raw/hdd_1 + // The raw mount is the first two components after /mnt/.felhom-raw/ + parts := strings.Split(filepath.Clean(source), string(os.PathSeparator)) + // parts: ["", "mnt", ".felhom-raw", "hdd_1", "subfolder"] + for i, p := range parts { + if p == ".felhom-raw" && i+1 < len(parts) { + rawPath = string(os.PathSeparator) + filepath.Join(parts[1:i+2]...) + return rawPath, true + } + } + } + return "", false +} diff --git a/controller/internal/system/mounts_other.go b/controller/internal/system/mounts_other.go index fa61dbf..85ad46b 100644 --- a/controller/internal/system/mounts_other.go +++ b/controller/internal/system/mounts_other.go @@ -80,3 +80,30 @@ func CheckBackupDestination(path string) DestinationHealth { Severity: "ok", } } + +// ProbeStatus represents the result of a storage path probe. +type ProbeStatus int + +const ( + ProbeConnected ProbeStatus = iota + ProbeDisconnected + ProbeTimeout +) + +// ProbeResult holds the outcome of a storage path probe. +type ProbeResult struct { + Status ProbeStatus + Err error +} + +// ProbeStoragePath always returns connected on non-Linux. +func ProbeStoragePath(_ string) ProbeResult { return ProbeResult{Status: ProbeConnected} } + +// IsUSBDevice always returns false on non-Linux. +func IsUSBDevice(_ string) bool { return false } + +// ParseFstabUUID always returns empty on non-Linux. +func ParseFstabUUID(_, _ string) string { return "" } + +// HasFelhomRawMount always returns false on non-Linux. +func HasFelhomRawMount(_, _ string) (string, bool) { return "", false } diff --git a/controller/internal/web/alerts.go b/controller/internal/web/alerts.go index 44ffe3d..63ed501 100644 --- a/controller/internal/web/alerts.go +++ b/controller/internal/web/alerts.go @@ -9,6 +9,7 @@ import ( "gitea.dooplex.hu/admin/felhom-controller/internal/backup" "gitea.dooplex.hu/admin/felhom-controller/internal/config" "gitea.dooplex.hu/admin/felhom-controller/internal/monitor" + "gitea.dooplex.hu/admin/felhom-controller/internal/settings" ) // Alert represents a persistent dashboard alert banner. @@ -39,10 +40,29 @@ func NewAlertManager(logger *log.Logger) *AlertManager { } // Refresh regenerates alerts from the latest health check report and config state. -// Called after each health check cycle (every 5 minutes). -func (am *AlertManager) Refresh(report *monitor.HealthReport, cfg *config.Config, backupMgr *backup.Manager, updateAvailable bool, latestVersion string) { +// Called after each health check cycle (every 5 minutes) and on storage state changes. +func (am *AlertManager) Refresh(report *monitor.HealthReport, cfg *config.Config, backupMgr *backup.Manager, updateAvailable bool, latestVersion string, storagePaths ...[]settings.StoragePath) { var alerts []Alert + // Disconnected storage alerts (top-level error banners on all pages) + if len(storagePaths) > 0 { + for _, sp := range storagePaths[0] { + if sp.Disconnected { + label := sp.Label + if label == "" { + label = sp.Path + } + alerts = append(alerts, Alert{ + ID: "storage-disconnected-" + simpleHash(sp.Path), + Level: "error", + Message: fmt.Sprintf("Meghajtó leválasztva: %s (%s)", label, sp.Path), + Link: "/settings", + LinkText: "Beállítások", + }) + } + } + } + // From health check issues (critical) for _, issue := range report.Issues { alerts = append(alerts, Alert{ diff --git a/controller/internal/web/handlers.go b/controller/internal/web/handlers.go index 5aaa2cb..c64315f 100644 --- a/controller/internal/web/handlers.go +++ b/controller/internal/web/handlers.go @@ -22,17 +22,26 @@ import ( // StorageBarInfo holds data for rendering a storage usage bar on dashboard/monitoring. type StorageBarInfo struct { - Label string // e.g., "USB HDD 1TB", "SYS Storage 350G" - Path string // e.g., "/mnt/hdd_1" - TotalGB float64 - UsedGB float64 - Percent float64 + Label string // e.g., "USB HDD 1TB", "SYS Storage 350G" + Path string // e.g., "/mnt/hdd_1" + TotalGB float64 + UsedGB float64 + Percent float64 + Disconnected bool } // buildStorageBars returns usage bars for all registered storage paths. func (s *Server) buildStorageBars() []StorageBarInfo { var bars []StorageBarInfo for _, sp := range s.settings.GetStoragePaths() { + if sp.Disconnected { + bars = append(bars, StorageBarInfo{ + Label: sp.Label, + Path: sp.Path, + Disconnected: true, + }) + continue + } di := system.GetDiskUsage(sp.Path) if di == nil { continue @@ -65,11 +74,13 @@ type StorageAppDetail struct { // StoragePathView extends StoragePath with display data for the settings page. type StoragePathView struct { settings.StoragePath - DiskInfo *system.DiskUsageInfo - AppCount int - IsMounted bool - AppDetails []StorageAppDetail - FSInfo *system.FSInfo + DiskInfo *system.DiskUsageInfo + AppCount int + IsMounted bool + AppDetails []StorageAppDetail + FSInfo *system.FSInfo + IsUSB bool // true if this is a USB-attached device (safe disconnect available) + StoppedApps []string // stacks auto-stopped due to disconnect (for restart UI) } func (s *Server) baseData(page, title string) map[string]interface{} { @@ -659,6 +670,9 @@ type AppBackupRow struct { Tier2SizeHuman string Tier2Browsable bool // true for rsync (plain files), false for restic + // Drive disconnected — app's home drive is currently disconnected + DriveDisconnected bool + // Warnings accumulated for this app Warnings []string } @@ -700,10 +714,32 @@ func (s *Server) buildAppBackupRows( } } + // Build disconnected paths set for drive-disconnected detection + disconnectedPaths := make(map[string]bool) + for _, dp := range s.settings.GetDisconnectedPaths() { + disconnectedPaths[dp.Path] = true + } + var rows []AppBackupRow for _, app := range status.AppDataInfo { hasDB := dbStacks[app.StackName] || app.HasDBDump + // Check if this app's home drive is disconnected + driveDisconnected := false + if app.HasHDDData && len(app.HDDPaths) > 0 { + for dp := range disconnectedPaths { + for _, hp := range app.HDDPaths { + if strings.HasPrefix(hp.HostPath, dp+"/") || hp.HostPath == dp { + driveDisconnected = true + break + } + } + if driveDisconnected { + break + } + } + } + // Build backup contents label var parts []string if hasDB { @@ -716,10 +752,11 @@ func (s *Server) buildAppBackupRows( contents := strings.Join(parts, " + ") row := AppBackupRow{ - StackName: app.StackName, - DisplayName: app.DisplayName, - HasHDDData: app.HasHDDData, - HasDB: hasDB, + StackName: app.StackName, + DisplayName: app.DisplayName, + HasHDDData: app.HasHDDData, + HasDB: hasDB, + DriveDisconnected: driveDisconnected, StorageLabel: app.StorageLabel, HDDSizeHuman: app.HDDSizeHuman, BackupContents: contents, @@ -933,13 +970,23 @@ func (s *Server) settingsData() map[string]interface{} { for _, sp := range storagePaths { view := StoragePathView{ StoragePath: sp, - IsMounted: system.IsMountPoint(sp.Path), - AppDetails: s.appDetailsForPath(sp.Path), - FSInfo: system.GetFSInfo(sp.Path), + StoppedApps: sp.StoppedStacks, } - view.AppCount = len(view.AppDetails) - if di := system.GetDiskUsage(sp.Path); di != nil { - view.DiskInfo = di + if sp.Disconnected { + // Skip I/O calls on disconnected drives — they'd hang or fail + view.IsMounted = false + } else { + view.IsMounted = system.IsMountPoint(sp.Path) + view.AppDetails = s.appDetailsForPath(sp.Path) + view.FSInfo = system.GetFSInfo(sp.Path) + view.AppCount = len(view.AppDetails) + if di := system.GetDiskUsage(sp.Path); di != nil { + view.DiskInfo = di + } + // Detect USB for safe disconnect button + if view.FSInfo != nil && view.FSInfo.Device != "" { + view.IsUSB = system.IsUSBDevice(view.FSInfo.Device) + } } storageViews = append(storageViews, view) } diff --git a/controller/internal/web/server.go b/controller/internal/web/server.go index 94bd2a7..36eb4c2 100644 --- a/controller/internal/web/server.go +++ b/controller/internal/web/server.go @@ -12,6 +12,7 @@ import ( "gitea.dooplex.hu/admin/felhom-controller/internal/backup" "gitea.dooplex.hu/admin/felhom-controller/internal/config" + "gitea.dooplex.hu/admin/felhom-controller/internal/monitor" "gitea.dooplex.hu/admin/felhom-controller/internal/notify" "gitea.dooplex.hu/admin/felhom-controller/internal/scheduler" "gitea.dooplex.hu/admin/felhom-controller/internal/selfupdate" @@ -49,6 +50,9 @@ type Server struct { // DR restore mode state restoreMu sync.RWMutex restorePlan *backup.RestorePlan + + // Storage watchdog (set after construction to break init ordering) + storageWatchdog *monitor.StorageWatchdog } func NewServer(cfg *config.Config, stackMgr *stacks.Manager, cpuCollector *system.CPUCollector, backupMgr *backup.Manager, crossDrive *backup.CrossDriveRunner, sched *scheduler.Scheduler, sett *settings.Settings, alertMgr *AlertManager, notif *notify.Notifier, updater *selfupdate.Updater, logger *log.Logger, version string) *Server { @@ -99,6 +103,11 @@ func (s *Server) SetRestoreState(plan *backup.RestorePlan) { s.restorePlan = plan } +// SetStorageWatchdog sets the storage watchdog for disconnect/reconnect operations. +func (s *Server) SetStorageWatchdog(w *monitor.StorageWatchdog) { + s.storageWatchdog = w +} + // InRestoreMode returns true if the server is in DR restore mode. func (s *Server) InRestoreMode() bool { s.restoreMu.RLock() diff --git a/controller/internal/web/storage_handlers.go b/controller/internal/web/storage_handlers.go index 79c7ca8..96dda9d 100644 --- a/controller/internal/web/storage_handlers.go +++ b/controller/internal/web/storage_handlers.go @@ -154,6 +154,14 @@ func (s *Server) storageAPIHandler(w http.ResponseWriter, r *http.Request) { s.storageAttachStatusAPIHandler(w, r) case path == "/api/storage/attach/cancel" && r.Method == http.MethodPost: s.storageAttachCancelHandler(w, r) + case path == "/api/storage/disconnect" && r.Method == http.MethodPost: + s.storageDisconnectHandler(w, r) + case path == "/api/storage/reconnect" && r.Method == http.MethodPost: + s.storageReconnectHandler(w, r) + case path == "/api/storage/restart-apps" && r.Method == http.MethodPost: + s.storageRestartAppsHandler(w, r) + case path == "/api/storage/status" && r.Method == http.MethodGet: + s.storageStatusHandler(w, r) default: http.NotFound(w, r) } @@ -1091,3 +1099,155 @@ func (s *Server) storageAttachCancelHandler(w http.ResponseWriter, r *http.Reque jsonResponse(w, map[string]interface{}{"ok": true}) } + +// storageDisconnectHandler handles POST /api/storage/disconnect. +// Performs a safe disconnect: stops affected apps, syncs, unmounts. +func (s *Server) storageDisconnectHandler(w http.ResponseWriter, r *http.Request) { + var req struct { + Path string `json:"path"` + } + if err := json.NewDecoder(r.Body).Decode(&req); err != nil { + jsonError(w, "Érvénytelen kérés", http.StatusBadRequest) + return + } + if req.Path == "" { + jsonError(w, "Hiányzó útvonal", http.StatusBadRequest) + return + } + + if s.storageWatchdog == nil { + jsonError(w, "Szolgáltatás nem elérhető", http.StatusServiceUnavailable) + return + } + + // Check if USB device (only USB drives can be safely disconnected) + fsInfo := system.GetFSInfo(req.Path) + if fsInfo != nil && fsInfo.Device != "" && !system.IsUSBDevice(fsInfo.Device) { + jsonError(w, "Csak USB meghajtó választható le biztonságosan", http.StatusBadRequest) + return + } + + stoppedStacks, err := s.storageWatchdog.SafeDisconnect(r.Context(), req.Path) + if err != nil { + s.logger.Printf("[ERROR] Safe disconnect %s: %v", req.Path, err) + jsonError(w, fmt.Sprintf("Leválasztás sikertelen: %v", err), http.StatusInternalServerError) + return + } + + jsonResponse(w, map[string]interface{}{ + "ok": true, + "message": "A meghajtó biztonságosan eltávolítható.", + "stopped_stacks": stoppedStacks, + }) +} + +// storageReconnectHandler handles POST /api/storage/reconnect. +// Attempts to remount a disconnected drive. +func (s *Server) storageReconnectHandler(w http.ResponseWriter, r *http.Request) { + var req struct { + Path string `json:"path"` + } + if err := json.NewDecoder(r.Body).Decode(&req); err != nil { + jsonError(w, "Érvénytelen kérés", http.StatusBadRequest) + return + } + if req.Path == "" { + jsonError(w, "Hiányzó útvonal", http.StatusBadRequest) + return + } + + if s.storageWatchdog == nil { + jsonError(w, "Szolgáltatás nem elérhető", http.StatusServiceUnavailable) + return + } + + stoppedStacks, err := s.storageWatchdog.Reconnect(r.Context(), req.Path) + if err != nil { + s.logger.Printf("[ERROR] Reconnect %s: %v", req.Path, err) + jsonError(w, fmt.Sprintf("Csatlakoztatás sikertelen: %v", err), http.StatusInternalServerError) + return + } + + jsonResponse(w, map[string]interface{}{ + "ok": true, + "message": "Meghajtó sikeresen csatlakoztatva.", + "stopped_stacks": stoppedStacks, + }) +} + +// storageRestartAppsHandler handles POST /api/storage/restart-apps. +// Restarts apps that were auto-stopped due to a drive disconnect. +func (s *Server) storageRestartAppsHandler(w http.ResponseWriter, r *http.Request) { + var req struct { + Path string `json:"path"` + } + if err := json.NewDecoder(r.Body).Decode(&req); err != nil { + jsonError(w, "Érvénytelen kérés", http.StatusBadRequest) + return + } + if req.Path == "" { + jsonError(w, "Hiányzó útvonal", http.StatusBadRequest) + return + } + + if s.storageWatchdog == nil { + jsonError(w, "Szolgáltatás nem elérhető", http.StatusServiceUnavailable) + return + } + + // Validate drive is connected + if s.settings.IsDisconnected(req.Path) { + jsonError(w, "A meghajtó jelenleg leválasztva — először csatlakoztassa", http.StatusBadRequest) + return + } + + started, failed := s.storageWatchdog.RestartStoppedApps(req.Path) + jsonResponse(w, map[string]interface{}{ + "ok": true, + "started": started, + "failed": failed, + }) +} + +// storageStatusHandler handles GET /api/storage/status. +// Returns status of all storage paths including connection state and USB detection. +func (s *Server) storageStatusHandler(w http.ResponseWriter, r *http.Request) { + paths := s.settings.GetStoragePaths() + + type pathStatus struct { + Path string `json:"path"` + Label string `json:"label"` + Connected bool `json:"connected"` + IsUSB bool `json:"is_usb"` + DisconnectedAt string `json:"disconnected_at"` + StoppedStacks []string `json:"stopped_stacks"` + } + + result := make([]pathStatus, 0, len(paths)) + for _, sp := range paths { + ps := pathStatus{ + Path: sp.Path, + Label: sp.Label, + Connected: !sp.Disconnected, + DisconnectedAt: sp.DisconnectedAt, + StoppedStacks: sp.StoppedStacks, + } + if ps.StoppedStacks == nil { + ps.StoppedStacks = []string{} + } + + // Detect USB for connected drives + if !sp.Disconnected { + if fsInfo := system.GetFSInfo(sp.Path); fsInfo != nil && fsInfo.Device != "" { + ps.IsUSB = system.IsUSBDevice(fsInfo.Device) + } + } + + result = append(result, ps) + } + + jsonResponse(w, map[string]interface{}{ + "ok": true, + "data": result, + }) +} diff --git a/controller/internal/web/templates/backups.html b/controller/internal/web/templates/backups.html index 4214a34..bd69f6b 100644 --- a/controller/internal/web/templates/backups.html +++ b/controller/internal/web/templates/backups.html @@ -38,6 +38,15 @@ {{range $.StorageBars}} + {{if .Disconnected}} +
+
+ {{.Label}} + Leválasztva +
+
+
+ {{else}}
{{.Label}} @@ -49,6 +58,7 @@
{{end}} {{end}} + {{end}}
{{if .Backup.RepoStats}} @@ -253,7 +263,9 @@ {{.DisplayName}}
- {{if .HasHDDData}} + {{if .DriveDisconnected}} + Meghajtó leválasztva + {{else if .HasHDDData}} {{if .StorageLabel}}{{.StorageLabel}}{{end}} {{.HDDSizeHuman}} {{else}} diff --git a/controller/internal/web/templates/dashboard.html b/controller/internal/web/templates/dashboard.html index ba25a13..acc8001 100644 --- a/controller/internal/web/templates/dashboard.html +++ b/controller/internal/web/templates/dashboard.html @@ -66,6 +66,15 @@
{{range .StorageBars}} + {{if .Disconnected}} +
+
+ {{.Label}} + Leválasztva +
+
+
+ {{else}}
{{.Label}} @@ -76,6 +85,7 @@
{{end}} + {{end}} {{if .DiskWarnings}}
diff --git a/controller/internal/web/templates/monitoring.html b/controller/internal/web/templates/monitoring.html index a112d3d..bcce5f1 100644 --- a/controller/internal/web/templates/monitoring.html +++ b/controller/internal/web/templates/monitoring.html @@ -51,6 +51,15 @@
{{range $.StorageBars}} + {{if .Disconnected}} +
+
+ {{.Label}} + Leválasztva +
+
+
+ {{else}}
{{.Label}} @@ -62,6 +71,7 @@
{{end}} {{end}} + {{end}}
{{if .DiskWarnings}}
diff --git a/controller/internal/web/templates/settings.html b/controller/internal/web/templates/settings.html index a8ed77d..f5a9cd4 100644 --- a/controller/internal/web/templates/settings.html +++ b/controller/internal/web/templates/settings.html @@ -205,21 +205,38 @@ function pollUntilBack() { {{if .StoragePaths}}
{{range .StoragePaths}} -
+
{{.Label}} - + {{if not .Disconnected}}{{end}}
{{.Path}}
+ {{if .Disconnected}} + Leválasztva + {{else}} {{if .IsDefault}}Alapértelmezett{{end}} {{if .Schedulable}}Aktív{{else}}Inaktív{{end}} {{if not .IsMounted}}Rendszermeghajtón{{end}} + {{end}}
+ {{if .Disconnected}} +
+
+ {{if .DisconnectedAt}}Leválasztva: {{.DisconnectedAt}}{{end}} + {{if .StoppedApps}} + Leállított alkalmazások: {{range $i, $name := .StoppedApps}}{{if $i}}, {{end}}{{$name}}{{end}} + {{end}} +
+
+
+ +
+ {{else}}
{{if .DiskInfo}}
@@ -237,6 +254,12 @@ function pollUntilBack() { {{.FSInfo.FSType}} · {{.FSInfo.Device}}{{if .FSInfo.Model}} · {{.FSInfo.Model}}{{end}}
{{end}} + {{if .StoppedApps}} +
+ Újraindításra váró alkalmazások: {{range $i, $name := .StoppedApps}}{{if $i}}, {{end}}{{$name}}{{end}} + +
+ {{end}}
{{if .AppDetails}}
@@ -278,6 +301,9 @@ function pollUntilBack() { {{end}} + {{if .IsUSB}} + + {{end}} {{if and (not .IsDefault) (eq .AppCount 0)}}
@@ -286,6 +312,7 @@ function pollUntilBack() {
{{end}}
+ {{end}}
{{end}}
@@ -420,6 +447,60 @@ function editStorageLabel(path, currentLabel) { ''; wrap.querySelector('input[name=storage_label]').focus(); } +function storageDisconnect(path, label, appCount) { + var msg = 'Biztos leválasztja a meghajtót: ' + label + '?'; + if (appCount > 0) msg += '\n\nA rajta futó ' + appCount + ' alkalmazás le fog állni.'; + msg += '\n\nA meghajtó ezután biztonságosan eltávolítható.'; + if (!confirm(msg)) return; + fetch('/api/storage/disconnect', { + method: 'POST', + headers: {'Content-Type': 'application/json'}, + body: JSON.stringify({path: path}) + }).then(function(r) { return r.json(); }).then(function(data) { + if (data.ok) { + alert('A meghajtó biztonságosan eltávolítható.'); + location.reload(); + } else { + alert('Hiba: ' + (data.error || 'ismeretlen')); + } + }).catch(function(e) { alert('Hiba: ' + e); }); +} +function storageReconnect(path) { + var actionsDiv = document.getElementById('storage-actions-' + path); + if (actionsDiv) actionsDiv.innerHTML = 'Csatlakoztatás...'; + fetch('/api/storage/reconnect', { + method: 'POST', + headers: {'Content-Type': 'application/json'}, + body: JSON.stringify({path: path}) + }).then(function(r) { return r.json(); }).then(function(data) { + if (data.ok) { + location.reload(); + } else { + alert('Hiba: ' + (data.error || 'ismeretlen')); + if (actionsDiv) actionsDiv.innerHTML = ''; + } + }).catch(function(e) { + alert('Hiba: ' + e); + if (actionsDiv) actionsDiv.innerHTML = ''; + }); +} +function storageRestartApps(path) { + fetch('/api/storage/restart-apps', { + method: 'POST', + headers: {'Content-Type': 'application/json'}, + body: JSON.stringify({path: path}) + }).then(function(r) { return r.json(); }).then(function(data) { + if (data.ok) { + var msg = ''; + if (data.started && data.started.length) msg += 'Elindítva: ' + data.started.join(', '); + if (data.failed && data.failed.length) msg += (msg ? '\n' : '') + 'Sikertelen: ' + data.failed.join(', '); + if (msg) alert(msg); + location.reload(); + } else { + alert('Hiba: ' + (data.error || 'ismeretlen')); + } + }).catch(function(e) { alert('Hiba: ' + e); }); +} function cancelEditLabel(path, label) { var wrap = document.getElementById('label-wrap-' + path); if (!wrap) return; diff --git a/controller/internal/web/templates/style.css b/controller/internal/web/templates/style.css index 5f083df..132e51f 100644 --- a/controller/internal/web/templates/style.css +++ b/controller/internal/web/templates/style.css @@ -2254,6 +2254,41 @@ a.stat-card:hover { background: rgba(250, 204, 21, 0.15); color: #facc15; } +.badge-error { + background: rgba(239, 68, 68, 0.15); + color: #ef4444; +} + +/* Disconnected storage path card */ +.storage-disconnected { + opacity: 0.75; + border-style: dashed; +} +.storage-disconnected .storage-disconnected-info { + display: flex; + flex-direction: column; + gap: .25rem; +} +.storage-stopped-apps-info { + display: flex; + align-items: center; + flex-wrap: wrap; + gap: .25rem; + margin-bottom: .5rem; +} + +/* Disconnected bar on dashboard/monitoring */ +.system-bar-disconnected { + background: repeating-linear-gradient( + -45deg, + rgba(239, 68, 68, 0.15), + rgba(239, 68, 68, 0.15) 5px, + transparent 5px, + transparent 10px + ); + height: 100%; + border-radius: 4px; +} /* Task progress bar (storage init — not disk usage zone gradient) */ .progress-bar-task {