From 2fb2c6e1ae82872bd723dc8aebf5b11fe3bf95c6 Mon Sep 17 00:00:00 2001 From: kisfenyo Date: Tue, 17 Feb 2026 10:27:18 +0100 Subject: [PATCH] =?UTF-8?q?v0.11.0=20=E2=80=94=20Phase=20C:=20Storage=20In?= =?UTF-8?q?it=20Wizard,=20Data=20Migration=20&=20Startup=20Fix?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Startup ping: fire heartbeat + health + hub report immediately on boot (5s delay after scheduler start, instead of waiting 5-15 min for first tick) - Storage init wizard: new internal/storage/ package with disk scanning (lsblk -J), format+mount pipeline (sfdisk → mkfs.ext4 → blkid → fstab → mount → chown), safety guards (system disk detection, confirmation "FORMÁZÁS"), progress channel, auto-register in settings.json - Data migration: MigrateAppData() with rsync --info=progress2 progress parsing, stop/rsync/update-config/start flow, rollback on failure, old data preserved - New pages: /settings/storage/init (wizard), /stacks/{name}/migrate (migration) - New API routes: /api/storage/{scan,init,init/status,migrate,migrate/status} - Deploy page: storage info section for deployed apps (path, size, free, migrate link) - Settings page: "Mozgatás" button per app in storage path details - Container: privileged: true, /dev:/dev, /etc/fstab:/host-fstab, /run/udev:/run/udev:ro - Dockerfile: add util-linux, e2fsprogs, rsync, parted for disk ops Co-Authored-By: Claude Opus 4.6 --- CHANGELOG.md | 12 + CONTEXT.md | 5 +- controller/Dockerfile | 8 + controller/cmd/controller/main.go | 37 +- controller/docker-compose.yml | 9 +- controller/internal/storage/format.go | 50 ++ controller/internal/storage/format_linux.go | 181 ++++++ controller/internal/storage/format_other.go | 20 + controller/internal/storage/migrate.go | 298 +++++++++ controller/internal/storage/safety.go | 27 + controller/internal/storage/safety_linux.go | 102 +++ controller/internal/storage/safety_other.go | 25 + controller/internal/storage/scan.go | 32 + controller/internal/storage/scan_linux.go | 168 +++++ controller/internal/storage/scan_other.go | 10 + controller/internal/web/handlers.go | 9 + controller/internal/web/server.go | 15 + controller/internal/web/storage_handlers.go | 603 ++++++++++++++++++ controller/internal/web/templates/deploy.html | 28 + .../internal/web/templates/migrate.html | 190 ++++++ .../internal/web/templates/settings.html | 7 +- .../internal/web/templates/storage_init.html | 325 ++++++++++ controller/internal/web/templates/style.css | 81 +++ 23 files changed, 2236 insertions(+), 6 deletions(-) create mode 100644 controller/internal/storage/format.go create mode 100644 controller/internal/storage/format_linux.go create mode 100644 controller/internal/storage/format_other.go create mode 100644 controller/internal/storage/migrate.go create mode 100644 controller/internal/storage/safety.go create mode 100644 controller/internal/storage/safety_linux.go create mode 100644 controller/internal/storage/safety_other.go create mode 100644 controller/internal/storage/scan.go create mode 100644 controller/internal/storage/scan_linux.go create mode 100644 controller/internal/storage/scan_other.go create mode 100644 controller/internal/web/storage_handlers.go create mode 100644 controller/internal/web/templates/migrate.html create mode 100644 controller/internal/web/templates/storage_init.html diff --git a/CHANGELOG.md b/CHANGELOG.md index 6665b63..c43c235 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,17 @@ ## Changelog +### What was just completed (2026-02-17 session 28) +- **v0.11.0 — Phase C: Storage Init, Data Migration & Startup Fixes:** + - **Step 0: Startup ping + hub report** — Controller now fires heartbeat ping, system_health ping, and hub report immediately on startup (5s delay) instead of waiting for first scheduler tick (5-15 min). `hubPusher` instance created once and reused for both startup and periodic reports. Prevents Healthchecks showing stale "Last Ping: X ago" after restarts. + - **Step 1-3: Storage initialization wizard** — New `internal/storage/` package (`scan.go`, `format.go`, `safety.go`, `format_linux.go`, `safety_linux.go`, `scan_linux.go` + non-linux stubs). `ScanDisks()` via `lsblk -J`. `FormatAndMount()` with progress channel (partition via sfdisk → mkfs.ext4 → blkid UUID → fstab backup + UUID-based entry → mount → chown + subdirs). Safety guards: system disk detection via major device numbers, mount path conflict, confirmation "FORMÁZÁS" required. New wizard page at `/settings/storage/init`. JSON API endpoints at `/api/storage/scan`, `/api/storage/init`, `/api/storage/init/status`. Auto-registers storage path in settings.json after success. + - **Step 4-5: Data migration** — New `MigrateAppData()` in `internal/storage/migrate.go`. Per-app "Mozgatás" button on deploy page (for deployed apps with HDD data) and settings page storage app list. Migration flow: stop app → rsync with `--info=progress2` progress parsing → update `app.yaml` HDD_PATH → start app. Rollback on failure (revert config + restart with original path). Old data preserved. New migration page at `/stacks/{name}/migrate`. JSON API at `/api/storage/migrate`, `/api/storage/migrate/status`. + - **Step 6: Per-app storage display** — Deploy page (read-only mode) now shows "Adattárolás" section for deployed apps: current path + label, data size, free space. "Mozgatás" link shown when other storage paths exist. + - **Step 7: Container setup** — Added `privileged: true` to `docker-compose.yml`. New volume mounts: `/dev:/dev`, `/etc/fstab:/host-fstab`, `/run/udev:/run/udev:ro`. Docker socket changed from `:ro` to writable. `Dockerfile` adds: `util-linux`, `e2fsprogs`, `rsync`, `parted`. + - **Storage API routing** — New `/api/storage/` prefix registered in `main.go` before `/api/` catch-all (longer prefix takes priority in Go ServeMux). `ServeStorageAPI` method on web.Server handles all storage JSON endpoints. + - **CSS additions** — `.disk-step`, `.disk-step-active`, `.disk-step-done`, `.disk-progress-steps`, `.disk-progress-bar-wrap`, `.deploy-storage-info` styles. + - **Files created (13):** `storage/scan.go`, `storage/scan_linux.go`, `storage/scan_other.go`, `storage/safety.go`, `storage/safety_linux.go`, `storage/safety_other.go`, `storage/format.go`, `storage/format_linux.go`, `storage/format_other.go`, `storage/migrate.go`, `web/storage_handlers.go`, `templates/storage_init.html`, `templates/migrate.html` + - **Files modified (8):** `main.go`, `web/server.go`, `web/handlers.go`, `templates/settings.html`, `templates/deploy.html`, `templates/style.css`, `docker-compose.yml`, `Dockerfile` + ### What was just completed (2026-02-17 session 27) - **v0.10.0 — Phase B: Storage Management UI Polish & Health Severity Fix:** - **Step 0: Health severity fix** — `checkStoragePaths()` mount-point check reclassified from **issue** (FAIL) to **warning** (WARN). All storage health messages translated to Hungarian. Added `.monitoring-banner-warn` CSS class for yellow warning banners. Prevents false FAIL status on demo/test environments where storage is intentionally on SSD. diff --git a/CONTEXT.md b/CONTEXT.md index 238445a..df223b1 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -7,7 +7,7 @@ > > Ask Claude Code: "Please update CONTEXT.md with what we did today" -Last updated: 2026-02-17 (session 27) +Last updated: 2026-02-17 (session 28) --- @@ -20,7 +20,7 @@ Last updated: 2026-02-17 (session 27) - Customer deployments use Docker Compose (not Kubernetes) for simplicity ### felhom-controller (this repo) -- **Version:** v0.10.0 +- **Version:** v0.11.0 - **Phase 1:** ✅ COMPLETE — Stack Manager + Deploy Flow - **Phase 2:** ✅ COMPLETE — Monitoring & Health (scheduler, CPU/temp, healthchecks.io pings) - **Phase 3:** ✅ COMPLETE — Backups (DB dumps, restic integration, manual trigger, **dedicated backup page**) @@ -30,6 +30,7 @@ Last updated: 2026-02-17 (session 27) - **Phase 7:** ✅ COMPLETE — Storage Overview, Per-App Backup Toggles & Limited Restore - **Phase A:** ✅ COMPLETE — Storage Paths Foundation (registry, auto-discovery, per-app HDD_PATH, deploy dropdown, health monitoring) - **Phase B:** ✅ COMPLETE — Storage Management UI Polish & Health Severity Fix (flash messages, label editing, app details, FS info, deploy free space, backup context) +- **Phase C:** ✅ COMPLETE — Storage Init Wizard, Data Migration & Startup Fix (disk scan/format/mount wizard, rsync-based migration, startup pings) - **First app deployed:** Paperless-ngx on demo-felhom.eu (2026-02-13) - **Running on:** demo-felhom (N100 mini PC) at 192.168.0.162:8080 - **All Phase 1-5 features working:** deploy, start/stop/restart/update, logs, health-aware states, auth, monitoring, backups, backup detail page, system monitoring page, settings page diff --git a/controller/Dockerfile b/controller/Dockerfile index 2524aa4..f1dbbed 100644 --- a/controller/Dockerfile +++ b/controller/Dockerfile @@ -45,6 +45,10 @@ FROM debian:bookworm-slim # - sqlite3: for SQLite backup # - git: for stack sync from Gitea # - curl: for health pings and debugging +# - util-linux: lsblk, blkid, sfdisk, mount (storage init) +# - e2fsprogs: mkfs.ext4 (filesystem formatting) +# - rsync: for data migration between storage paths +# - parted: partprobe (partition table re-read after sfdisk) RUN apt-get update && apt-get install -y --no-install-recommends \ ca-certificates \ curl \ @@ -54,6 +58,10 @@ RUN apt-get update && apt-get install -y --no-install-recommends \ postgresql-client \ default-mysql-client \ sqlite3 \ + util-linux \ + e2fsprogs \ + rsync \ + parted \ && rm -rf /var/lib/apt/lists/* # Install docker-cli (without daemon) diff --git a/controller/cmd/controller/main.go b/controller/cmd/controller/main.go index 9671957..5a8ab23 100644 --- a/controller/cmd/controller/main.go +++ b/controller/cmd/controller/main.go @@ -225,15 +225,16 @@ func main() { } // --- Central hub reporting --- + var hubPusher *report.Pusher if cfg.Hub.Enabled && cfg.Hub.URL != "" { pushInterval, err := time.ParseDuration(cfg.Hub.PushInterval) if err != nil { pushInterval = 15 * time.Minute } - pusher := report.NewPusher(&cfg.Hub, logger) + hubPusher = report.NewPusher(&cfg.Hub, logger) sched.Every("hub-report", pushInterval, func(ctx context.Context) error { r := report.BuildReport(cfg, stackMgr, backupMgr, cpuCollector, metricsStore, Version, sett.GetStoragePaths()) - return pusher.Push(r) + return hubPusher.Push(r) }) logger.Printf("[INFO] Hub reporting enabled (every %s to %s)", pushInterval, cfg.Hub.URL) } @@ -241,6 +242,36 @@ func main() { sched.Start(ctx) defer sched.Stop() + // Fire startup pings + hub report immediately (don't wait for first scheduler tick) + go func() { + time.Sleep(5 * time.Second) // Let all subsystems fully initialize + + // Heartbeat ping + pinger.Ping(cfg.Monitoring.PingUUIDs.Heartbeat, "startup") + logger.Println("[INFO] Startup heartbeat ping sent") + + // System health ping + healthReport := monitor.RunHealthCheck(cfg, cpuCollector, sett.GetStoragePaths()) + body := healthReport.FormatMessage() + healthUUID := cfg.Monitoring.PingUUIDs.SystemHealth + if healthReport.Status == "fail" { + pinger.Fail(healthUUID, body) + } else { + pinger.Ping(healthUUID, body) + } + logger.Printf("[INFO] Startup health ping sent (status: %s)", healthReport.Status) + + // Hub report + if hubPusher != nil { + r := report.BuildReport(cfg, stackMgr, backupMgr, cpuCollector, metricsStore, Version, sett.GetStoragePaths()) + if err := hubPusher.Push(r); err != nil { + logger.Printf("[WARN] Startup hub report failed: %v", err) + } else { + logger.Println("[INFO] Startup hub report sent") + } + } + }() + // Initial backup cache population (don't block startup) if cfg.Backup.Enabled && backupMgr != nil { go func() { @@ -279,6 +310,8 @@ func main() { // API routes (no auth for health endpoint, auth for everything else) mux.HandleFunc("/api/health", apiRouter.HealthHandler) + // Storage API routes handled by web server (longer prefix takes precedence over /api/) + mux.Handle("/api/storage/", webServer.RequireAuth(http.HandlerFunc(webServer.ServeStorageAPI))) mux.Handle("/api/", webServer.RequireAuth(http.HandlerFunc(apiRouter.ServeHTTP))) // Web UI routes (auth required) diff --git a/controller/docker-compose.yml b/controller/docker-compose.yml index 6bae4e6..ecff697 100644 --- a/controller/docker-compose.yml +++ b/controller/docker-compose.yml @@ -8,11 +8,12 @@ services: image: gitea.dooplex.hu/admin/felhom-controller:latest container_name: felhom-controller restart: unless-stopped + privileged: true # Required for disk operations (mkfs, mount, sfdisk) ports: - "8080:8080" volumes: # Docker socket — required for compose operations + DB dumps (docker exec) - - /var/run/docker.sock:/var/run/docker.sock:ro + - /var/run/docker.sock:/var/run/docker.sock # Controller config - /opt/docker/felhom-controller/controller.yaml:/opt/docker/felhom-controller/controller.yaml:ro # Controller persistent data (sessions, restic cache, restic password) @@ -29,6 +30,12 @@ services: - /etc/os-release:/host/etc/os-release:ro # Host hostname — for monitoring page (os.Hostname() returns container ID) - /etc/hostname:/host/etc/hostname:ro + # Block devices — required for storage init (lsblk, mkfs, sfdisk) + - /dev:/dev + # Host fstab — UUID-based mount persistence (mounted as /host-fstab inside container) + - /etc/fstab:/host-fstab + # udev metadata — for blkid/lsblk device model info + - /run/udev:/run/udev:ro environment: - TZ=Europe/Budapest labels: diff --git a/controller/internal/storage/format.go b/controller/internal/storage/format.go new file mode 100644 index 0000000..b37b1ec --- /dev/null +++ b/controller/internal/storage/format.go @@ -0,0 +1,50 @@ +package storage + +import ( + "bufio" + "fmt" + "strings" +) + +// FormatRequest holds parameters for formatting and mounting a disk. +type FormatRequest struct { + DevicePath string // "/dev/sdb" or "/dev/sdb1" + MountName string // "hdd_1" → mounts at /mnt/hdd_1 + Label string // Display label for the UI + CreatePartition bool // If true, create a single partition first (wipes disk) + SetDefault bool // Register as default storage path +} + +// FormatProgress tracks the formatting/mounting progress. +type FormatProgress struct { + Step string // "validating","partitioning","formatting","mounting","permissions","done","error" + Message string // Human-readable status + Error string // Non-empty if Step == "error" + Percent int // 0–100 +} + +// parseRsyncProgress parses a single line of rsync --info=progress2 output. +// Returns (bytesCopied, percent, ok). +func parseRsyncProgress(line string) (int64, int, bool) { + // Format: " 45,678,901 49% 12.34MB/s 0:00:30" + scanner := bufio.NewScanner(strings.NewReader(line)) + scanner.Split(bufio.ScanWords) + var tokens []string + for scanner.Scan() { + tokens = append(tokens, scanner.Text()) + } + if len(tokens) < 2 { + return 0, 0, false + } + bytesStr := strings.ReplaceAll(tokens[0], ",", "") + var bytesCopied int64 + if _, err := fmt.Sscanf(bytesStr, "%d", &bytesCopied); err != nil { + return 0, 0, false + } + pctStr := strings.TrimSuffix(tokens[1], "%") + var pct int + if _, err := fmt.Sscanf(pctStr, "%d", &pct); err != nil { + return 0, 0, false + } + return bytesCopied, pct, true +} diff --git a/controller/internal/storage/format_linux.go b/controller/internal/storage/format_linux.go new file mode 100644 index 0000000..fb1f7e1 --- /dev/null +++ b/controller/internal/storage/format_linux.go @@ -0,0 +1,181 @@ +//go:build linux + +package storage + +import ( + "bytes" + "fmt" + "os" + "os/exec" + "path/filepath" + "strings" + "time" +) + +// FormatAndMount formats a disk/partition and mounts it. +// Progress updates are sent on the progress channel. +// Returns the final mount path on success. +func FormatAndMount(req FormatRequest, progress chan<- FormatProgress) (string, error) { + send := func(step, msg string, pct int) { + progress <- FormatProgress{Step: step, Message: msg, Percent: pct} + } + fail := func(step, msg string, err error) error { + errStr := "" + if err != nil { + errStr = err.Error() + } + progress <- FormatProgress{Step: "error", Message: msg, Error: errStr, Percent: 0} + return fmt.Errorf("%s: %w", msg, err) + } + + mountPath := "/mnt/" + req.MountName + + // --- Step 1: Validate --- + send("validating", "Eszköz ellenőrzése...", 5) + + if err := ValidateMountName(req.MountName); err != nil { + return "", fail("validating", "Érvénytelen csatlakoztatási név", err) + } + if _, err := os.Stat(req.DevicePath); err != nil { + return "", fail("validating", "Az eszköz nem létezik: "+req.DevicePath, err) + } + + isSystem, err := IsSystemDisk(req.DevicePath) + if err != nil { + return "", fail("validating", "Rendszermeghajtó ellenőrzése sikertelen", err) + } + if isSystem { + return "", fail("validating", "Ez a rendszermeghajtó — nem formázható!", fmt.Errorf("device is system disk")) + } + + mounted, err := IsDeviceMounted(req.DevicePath) + if err != nil { + return "", fail("validating", "Csatlakoztatási állapot ellenőrzése sikertelen", err) + } + if mounted { + return "", fail("validating", "Az eszköz már csatlakoztatva van", fmt.Errorf("device already mounted")) + } + + inUse, err := IsMountPathInUse(mountPath) + if err != nil { + return "", fail("validating", "Csatlakoztatási útvonal ellenőrzése sikertelen", err) + } + if inUse { + return "", fail("validating", "A csatlakoztatási útvonal már használatban van: "+mountPath, fmt.Errorf("mount path in use")) + } + + send("validating", "Ellenőrzés kész", 10) + + // --- Step 2: Partition (if requested) --- + partDev := req.DevicePath + if req.CreatePartition { + send("partitioning", "Partíció létrehozása...", 15) + + sfdiskInput := "label: gpt\n,,,L\n" + cmd := exec.Command("sfdisk", req.DevicePath) + cmd.Stdin = strings.NewReader(sfdiskInput) + if out, err := cmd.CombinedOutput(); err != nil { + return "", fail("partitioning", "Partícionálás sikertelen: "+string(out), err) + } + + _ = exec.Command("partprobe", req.DevicePath).Run() + time.Sleep(2 * time.Second) + + partDev = req.DevicePath + "1" + if strings.Contains(req.DevicePath, "nvme") { + partDev = req.DevicePath + "p1" + } + if _, err := os.Stat(partDev); err != nil { + return "", fail("partitioning", "Partíció nem található a létrehozás után: "+partDev, err) + } + + send("partitioning", "Partíció létrehozva: "+partDev, 25) + } + + // --- Step 3: Format --- + send("formatting", "Fájlrendszer formázása (ext4)...", 30) + + label := req.Label + if label == "" { + label = req.MountName + } + if len(label) > 16 { + label = label[:16] + } + + mkfsCmd := exec.Command("mkfs.ext4", "-L", label, "-F", partDev) + var mkfsOut bytes.Buffer + mkfsCmd.Stdout = &mkfsOut + mkfsCmd.Stderr = &mkfsOut + if err := mkfsCmd.Run(); err != nil { + return "", fail("formatting", "Formázás sikertelen: "+mkfsOut.String(), err) + } + + send("formatting", "Formázás kész", 60) + + // --- Step 4: Mount --- + send("mounting", "Csatlakoztatás: "+mountPath+"...", 65) + + if err := os.MkdirAll(mountPath, 0755); err != nil { + return "", fail("mounting", "Csatlakoztatási mappa nem hozható létre: "+mountPath, err) + } + + uuidOut, err := exec.Command("blkid", "-s", "UUID", "-o", "value", partDev).Output() + if err != nil { + return "", fail("mounting", "UUID lekérése sikertelen", err) + } + uuid := strings.TrimSpace(string(uuidOut)) + if uuid == "" { + return "", fail("mounting", "UUID üres a formázás után", fmt.Errorf("empty UUID")) + } + + // Backup fstab (non-fatal) + _ = BackupFstab(FstabPath) + + if err := AppendFstabEntry(FstabPath, uuid, mountPath, "ext4", "defaults,nofail,noatime"); err != nil { + return "", fail("mounting", "fstab bejegyzés hozzáadása sikertelen", err) + } + + if out, err := exec.Command("mount", mountPath).CombinedOutput(); err != nil { + return "", fail("mounting", "Csatlakoztatás sikertelen: "+string(out), err) + } + + send("mounting", "Csatlakoztatva: "+mountPath, 80) + + // --- Step 5: Permissions + subdirs --- + send("permissions", "Mappák létrehozása és jogosultságok beállítása...", 85) + + _ = exec.Command("chown", "1000:1000", mountPath).Run() + + for _, subdir := range []string{"storage", "Dokumentumok"} { + dir := filepath.Join(mountPath, subdir) + if err := os.MkdirAll(dir, 0755); err == nil { + _ = exec.Command("chown", "1000:1000", dir).Run() + } + } + + send("done", "Meghajtó sikeresen inicializálva: "+mountPath, 100) + + return mountPath, nil +} + +// GetDeviceUUID returns the UUID of a block device/partition. +func GetDeviceUUID(devicePath string) (string, error) { + out, err := exec.Command("blkid", "-s", "UUID", "-o", "value", devicePath).Output() + if err != nil { + return "", err + } + return strings.TrimSpace(string(out)), nil +} + +// ReadFstab reads the current fstab content. +func ReadFstab() (string, error) { + data, err := os.ReadFile(FstabPath) + if err != nil { + data, err = os.ReadFile("/etc/fstab") + if err != nil { + return "", err + } + } + return string(data), nil +} diff --git a/controller/internal/storage/format_other.go b/controller/internal/storage/format_other.go new file mode 100644 index 0000000..1a1362d --- /dev/null +++ b/controller/internal/storage/format_other.go @@ -0,0 +1,20 @@ +//go:build !linux + +package storage + +import "fmt" + +// FormatAndMount is not supported on non-Linux platforms. +func FormatAndMount(req FormatRequest, progress chan<- FormatProgress) (string, error) { + return "", fmt.Errorf("storage init is only supported on Linux") +} + +// GetDeviceUUID is not supported on non-Linux platforms. +func GetDeviceUUID(devicePath string) (string, error) { + return "", fmt.Errorf("storage init is only supported on Linux") +} + +// ReadFstab is not supported on non-Linux platforms. +func ReadFstab() (string, error) { + return "", fmt.Errorf("storage init is only supported on Linux") +} diff --git a/controller/internal/storage/migrate.go b/controller/internal/storage/migrate.go new file mode 100644 index 0000000..57e5841 --- /dev/null +++ b/controller/internal/storage/migrate.go @@ -0,0 +1,298 @@ +package storage + +import ( + "bufio" + "fmt" + "io" + "os" + "os/exec" + "path/filepath" + "strings" + "sync" + "time" +) + +// MigrateRequest holds parameters for migrating app data. +type MigrateRequest struct { + StackName string // e.g., "immich" + DisplayName string // e.g., "Immich" + CurrentHDDPath string // e.g., "/mnt/hdd_placeholder" + TargetPath string // e.g., "/mnt/hdd_1" + HDDMounts []string // host-side paths to rsync (e.g., ["/mnt/hdd_placeholder/storage/immich"]) +} + +// MigrateProgress tracks migration state. +type MigrateProgress struct { + Step string // "stopping","copying","updating","starting","done","error","rolling_back" + Message string + BytesCopied int64 + BytesTotal int64 + Percent int + Error string + ElapsedSeconds int +} + +// StopFunc stops an app's containers. Returns error if stop fails. +type StopFunc func(stackName string) error + +// StartFunc starts an app's containers. Returns error if start fails. +type StartFunc func(stackName string) error + +// UpdateHDDPathFunc updates the HDD_PATH in app.yaml. Returns error on failure. +type UpdateHDDPathFunc func(stackName, newPath string) error + +// MigrateAppData moves app data from current to target storage path. +// stopFn and startFn are called to stop/start the app containers. +// updateFn is called to update the app's HDD_PATH configuration. +func MigrateAppData( + req MigrateRequest, + stopFn StopFunc, + startFn StartFunc, + updateFn UpdateHDDPathFunc, + progress chan<- MigrateProgress, +) error { + start := time.Now() + + send := func(step, msg string, pct int, bytesCopied, bytesTotal int64) { + progress <- MigrateProgress{ + Step: step, + Message: msg, + Percent: pct, + BytesCopied: bytesCopied, + BytesTotal: bytesTotal, + ElapsedSeconds: int(time.Since(start).Seconds()), + } + } + + fail := func(step, msg string, err error) error { + errStr := "" + if err != nil { + errStr = err.Error() + } + progress <- MigrateProgress{ + Step: "error", + Message: msg, + Error: errStr, + ElapsedSeconds: int(time.Since(start).Seconds()), + } + return fmt.Errorf("%s: %w", msg, err) + } + + // --- Step 1: Validate --- + if req.CurrentHDDPath == "" { + return fail("validating", "A jelenlegi tárhely nem megadott", fmt.Errorf("empty current HDD path")) + } + if req.TargetPath == "" { + return fail("validating", "A cél tárhely nem megadott", fmt.Errorf("empty target path")) + } + if req.CurrentHDDPath == req.TargetPath { + return fail("validating", "A forrás és a cél tárhely azonos", fmt.Errorf("source equals target")) + } + if _, err := os.Stat(req.TargetPath); err != nil { + return fail("validating", "A cél tárhely nem létezik: "+req.TargetPath, err) + } + if len(req.HDDMounts) == 0 { + return fail("validating", "Nincsenek HDD csatlakozások az alkalmazáshoz", fmt.Errorf("no HDD mounts")) + } + + // Estimate total size + var totalBytes int64 + for _, m := range req.HDDMounts { + if info, err := os.Stat(m); err == nil && info.IsDir() { + totalBytes += dirSize(m) + } + } + + // Check free space on target + freeBytes := getFreeBytes(req.TargetPath) + if freeBytes > 0 && totalBytes > 0 && int64(float64(totalBytes)*1.05) > freeBytes { + return fail("validating", fmt.Sprintf( + "Nincs elég szabad hely a céltárolón: szükséges ~%s, szabad %s", + bytesHuman(totalBytes), bytesHuman(freeBytes), + ), fmt.Errorf("insufficient disk space")) + } + + send("stopping", "Alkalmazás leállítása...", 5, 0, totalBytes) + + // --- Step 2: Stop app --- + if err := stopFn(req.StackName); err != nil { + return fail("stopping", "Alkalmazás leállítása sikertelen", err) + } + + send("stopping", "Alkalmazás leállítva", 10, 0, totalBytes) + + // --- Step 3: rsync --- + var bytesCopied int64 + for i, srcPath := range req.HDDMounts { + // Determine destination path: replace CurrentHDDPath prefix with TargetPath + if !strings.HasPrefix(srcPath, req.CurrentHDDPath) { + continue + } + relPath := strings.TrimPrefix(srcPath, req.CurrentHDDPath) + dstPath := filepath.Join(req.TargetPath, relPath) + + // Ensure destination parent exists + if err := os.MkdirAll(filepath.Dir(dstPath), 0755); err != nil { + // Rollback + send("rolling_back", "Hiba: mappa létrehozása sikertelen, visszagörgetés...", 0, bytesCopied, totalBytes) + _ = startFn(req.StackName) + return fail("copying", "Cél mappa létrehozása sikertelen: "+filepath.Dir(dstPath), err) + } + + mountPct := 10 + (i * 60 / len(req.HDDMounts)) + + send("copying", fmt.Sprintf("Adatok másolása (%d/%d): %s...", i+1, len(req.HDDMounts), filepath.Base(srcPath)), + mountPct, bytesCopied, totalBytes) + + var rsyncErr error + bytesCopied, rsyncErr = runRsync(srcPath, dstPath, totalBytes, bytesCopied, mountPct, progress, start) + if rsyncErr != nil { + // Rollback + send("rolling_back", "rsync sikertelen, alkalmazás visszaállítása az eredeti tárolóra...", 0, bytesCopied, totalBytes) + _ = startFn(req.StackName) + return fail("copying", "Adatmásolás sikertelen", rsyncErr) + } + } + + send("updating", "Konfiguráció frissítése...", 75, bytesCopied, totalBytes) + + // --- Step 4: Update app.yaml HDD_PATH --- + if err := updateFn(req.StackName, req.TargetPath); err != nil { + send("rolling_back", "Konfiguráció frissítése sikertelen, visszaállítás...", 0, bytesCopied, totalBytes) + _ = startFn(req.StackName) + return fail("updating", "HDD_PATH frissítése sikertelen", err) + } + + send("starting", "Alkalmazás indítása az új tárolóról...", 85, bytesCopied, totalBytes) + + // --- Step 5: Start app --- + if err := startFn(req.StackName); err != nil { + // Revert config and restart with old path + _ = updateFn(req.StackName, req.CurrentHDDPath) + _ = startFn(req.StackName) + return fail("starting", "Alkalmazás indítása sikertelen az új tárolóról", err) + } + + elapsed := int(time.Since(start).Seconds()) + send("done", + fmt.Sprintf("Áthelyezés kész! Az alkalmazás az új tárolóról fut. (Régi adat: %s)", req.CurrentHDDPath), + 100, bytesCopied, totalBytes) + _ = elapsed + + return nil +} + +// runRsync runs rsync from srcPath to dstPath and reports progress. +func runRsync(srcPath, dstPath string, totalBytes, prevCopied int64, basePct int, progress chan<- MigrateProgress, start time.Time) (int64, error) { + // Ensure src ends with / for rsync to sync contents (not the directory itself) + if !strings.HasSuffix(srcPath, "/") { + srcPath += "/" + } + + cmd := exec.Command( + "rsync", "-a", "--info=progress2", "--human-readable", + srcPath, dstPath, + ) + + stdout, err := cmd.StdoutPipe() + if err != nil { + return prevCopied, err + } + stderr, err := cmd.StderrPipe() + if err != nil { + return prevCopied, err + } + + if err := cmd.Start(); err != nil { + return prevCopied, fmt.Errorf("rsync start failed: %w", err) + } + + var bytesCopied int64 = prevCopied + var mu sync.Mutex + + // Parse stdout progress + go func() { + scanner := bufio.NewScanner(stdout) + for scanner.Scan() { + line := scanner.Text() + if b, pct, ok := parseRsyncProgress(line); ok { + mu.Lock() + bytesCopied = prevCopied + b + // Scale pct into our range + scaledPct := basePct + pct*40/100 + if scaledPct > 99 { + scaledPct = 99 + } + mu.Unlock() + progress <- MigrateProgress{ + Step: "copying", + Message: fmt.Sprintf("Adatok másolása... %s / %s", bytesHuman(b), bytesHuman(totalBytes)), + Percent: scaledPct, + BytesCopied: bytesCopied, + BytesTotal: totalBytes, + ElapsedSeconds: int(time.Since(start).Seconds()), + } + } + } + }() + + var stderrBuf strings.Builder + io.Copy(&stderrBuf, stderr) + + if err := cmd.Wait(); err != nil { + return bytesCopied, fmt.Errorf("rsync failed: %w — %s", err, stderrBuf.String()) + } + + mu.Lock() + finalCopied := bytesCopied + mu.Unlock() + return finalCopied, nil +} + +// dirSize returns the total bytes in a directory (best-effort). +func dirSize(path string) int64 { + var total int64 + filepath.Walk(path, func(_ string, info os.FileInfo, err error) error { + if err != nil || info.IsDir() { + return nil + } + total += info.Size() + return nil + }) + return total +} + +// getFreeBytes returns available bytes on the filesystem at path. +func getFreeBytes(path string) int64 { + // Use df to get available bytes — works cross-platform within Linux container + out, err := exec.Command("df", "-B1", "--output=avail", path).Output() + if err != nil { + return 0 + } + lines := strings.Split(strings.TrimSpace(string(out)), "\n") + if len(lines) < 2 { + return 0 + } + var avail int64 + fmt.Sscanf(strings.TrimSpace(lines[1]), "%d", &avail) + return avail +} + +// bytesHuman converts a byte count to human-readable string. +func bytesHuman(b int64) string { + const ( + KB = 1024 + MB = KB * 1024 + GB = MB * 1024 + ) + switch { + case b >= GB: + return fmt.Sprintf("%.1f GB", float64(b)/float64(GB)) + case b >= MB: + return fmt.Sprintf("%.0f MB", float64(b)/float64(MB)) + case b >= KB: + return fmt.Sprintf("%.0f KB", float64(b)/float64(KB)) + default: + return fmt.Sprintf("%d B", b) + } +} diff --git a/controller/internal/storage/safety.go b/controller/internal/storage/safety.go new file mode 100644 index 0000000..32a2eeb --- /dev/null +++ b/controller/internal/storage/safety.go @@ -0,0 +1,27 @@ +package storage + +import ( + "fmt" + "regexp" +) + +// mountNameRe validates mount names: only alphanumeric + underscore. +var mountNameRe = regexp.MustCompile(`^[a-zA-Z0-9_]+$`) + +// FstabPath is the path to the host fstab inside the container. +// The compose file mounts /etc/fstab → /host-fstab. +const FstabPath = "/host-fstab" + +// ValidateMountName returns an error if the mount name is invalid. +func ValidateMountName(name string) error { + if name == "" { + return fmt.Errorf("a csatlakoztatási névnek nem szabad üresnek lennie") + } + if !mountNameRe.MatchString(name) { + return fmt.Errorf("a csatlakoztatási néven csak betűk, számok és alávonás megengedett") + } + if len(name) > 32 { + return fmt.Errorf("a csatlakoztatási néven legfeljebb 32 karakter megengedett") + } + return nil +} diff --git a/controller/internal/storage/safety_linux.go b/controller/internal/storage/safety_linux.go new file mode 100644 index 0000000..c90b8b4 --- /dev/null +++ b/controller/internal/storage/safety_linux.go @@ -0,0 +1,102 @@ +//go:build linux + +package storage + +import ( + "fmt" + "os" + "path/filepath" + "strings" + "syscall" + "time" +) + +// IsSystemDisk checks if the given device path overlaps with the root filesystem device. +// Returns true if the device is (or is the parent of) the system disk. +func IsSystemDisk(devicePath string) (bool, error) { + // Get the block device major number of the root filesystem + var rootStat syscall.Stat_t + if err := syscall.Stat("/", &rootStat); err != nil { + return false, fmt.Errorf("cannot stat /: %w", err) + } + + // Get block device info of the target device + var devStat syscall.Stat_t + if err := syscall.Stat(devicePath, &devStat); err != nil { + return false, fmt.Errorf("cannot stat %s: %w", devicePath, err) + } + + // Compare major device numbers + rootMajor := rootStat.Dev >> 8 & 0xff + devMajor := devStat.Rdev >> 8 & 0xff + if rootMajor == devMajor { + return true, nil + } + + return false, nil +} + +// IsDeviceMounted checks if a device or any of its partitions is currently mounted. +func IsDeviceMounted(devicePath string) (bool, error) { + data, err := os.ReadFile("/proc/mounts") + if err != nil { + return false, fmt.Errorf("cannot read /proc/mounts: %w", err) + } + + base := filepath.Base(devicePath) + for _, line := range strings.Split(string(data), "\n") { + fields := strings.Fields(line) + if len(fields) < 2 { + continue + } + dev := fields[0] + devBase := filepath.Base(dev) + if devBase == base || strings.HasPrefix(devBase, base) { + return true, nil + } + } + return false, nil +} + +// IsMountPathInUse checks if a path is already used as a mount point. +func IsMountPathInUse(mountPath string) (bool, error) { + data, err := os.ReadFile("/proc/mounts") + if err != nil { + return false, fmt.Errorf("cannot read /proc/mounts: %w", err) + } + mountPath = filepath.Clean(mountPath) + for _, line := range strings.Split(string(data), "\n") { + fields := strings.Fields(line) + if len(fields) < 2 { + continue + } + if filepath.Clean(fields[1]) == mountPath { + return true, nil + } + } + return false, nil +} + +// BackupFstab creates a dated backup of the fstab file. +func BackupFstab(fstabPath string) error { + data, err := os.ReadFile(fstabPath) + if err != nil { + return fmt.Errorf("cannot read %s: %w", fstabPath, err) + } + backupPath := fstabPath + ".bak." + time.Now().Format("20060102") + return os.WriteFile(backupPath, data, 0644) +} + +// AppendFstabEntry appends a UUID-based fstab entry. +func AppendFstabEntry(fstabPath, uuid, mountPoint, fsType, options string) error { + entry := fmt.Sprintf("\nUUID=%s\t%s\t%s\t%s\t0 2\n", uuid, mountPoint, fsType, options) + f, err := os.OpenFile(fstabPath, os.O_APPEND|os.O_WRONLY, 0644) + if err != nil { + return fmt.Errorf("cannot open fstab for writing: %w", err) + } + defer f.Close() + if _, err := f.WriteString(entry); err != nil { + return fmt.Errorf("cannot write fstab entry: %w", err) + } + return nil +} diff --git a/controller/internal/storage/safety_other.go b/controller/internal/storage/safety_other.go new file mode 100644 index 0000000..2d9fef9 --- /dev/null +++ b/controller/internal/storage/safety_other.go @@ -0,0 +1,25 @@ +//go:build !linux + +package storage + +import "fmt" + +func IsSystemDisk(devicePath string) (bool, error) { + return false, fmt.Errorf("storage init is only supported on Linux") +} + +func IsDeviceMounted(devicePath string) (bool, error) { + return false, fmt.Errorf("storage init is only supported on Linux") +} + +func IsMountPathInUse(mountPath string) (bool, error) { + return false, fmt.Errorf("storage init is only supported on Linux") +} + +func BackupFstab(fstabPath string) error { + return fmt.Errorf("storage init is only supported on Linux") +} + +func AppendFstabEntry(fstabPath, uuid, mountPoint, fsType, options string) error { + return fmt.Errorf("storage init is only supported on Linux") +} diff --git a/controller/internal/storage/scan.go b/controller/internal/storage/scan.go new file mode 100644 index 0000000..4430c40 --- /dev/null +++ b/controller/internal/storage/scan.go @@ -0,0 +1,32 @@ +package storage + +// BlockDevice represents a detected physical disk. +type BlockDevice struct { + Name string // "sdb" + Path string // "/dev/sdb" + Size string // "931.5G" + SizeBytes int64 // raw bytes from lsblk + Model string // "WD Elements 25A2" + Type string // "disk" + Removable bool // true for USB + Partitions []Partition // child partitions + Mounted bool // any partition is mounted +} + +// Partition represents a partition on a block device. +type Partition struct { + Name string // "sdb1" + Path string // "/dev/sdb1" + Size string // "931.5G" + SizeBytes int64 + FSType string // "ext4", "" for no filesystem + Label string // filesystem label + UUID string + MountPoint string // "" if not mounted +} + +// ScanResult from disk detection. +type ScanResult struct { + AvailableDisks []BlockDevice // Unmounted, non-system disks + SystemDisks []BlockDevice // System/mounted disks (display only) +} diff --git a/controller/internal/storage/scan_linux.go b/controller/internal/storage/scan_linux.go new file mode 100644 index 0000000..64445d4 --- /dev/null +++ b/controller/internal/storage/scan_linux.go @@ -0,0 +1,168 @@ +//go:build linux + +package storage + +import ( + "encoding/json" + "fmt" + "os/exec" + "strings" +) + +// lsblkOutput matches the top-level JSON from lsblk -J. +type lsblkOutput struct { + BlockDevices []lsblkDevice `json:"blockdevices"` +} + +// lsblkDevice is the raw JSON structure from lsblk. +type lsblkDevice struct { + Name string `json:"name"` + Path string `json:"path"` + Size interface{} `json:"size"` // may be float64 or string + Type string `json:"type"` + FSType *string `json:"fstype"` + MountPoint *string `json:"mountpoint"` + Model *string `json:"model"` + RM interface{} `json:"rm"` // removable: bool or "0"/"1" + Children []lsblkDevice `json:"children"` +} + +func (d *lsblkDevice) sizeBytes() int64 { + switch v := d.Size.(type) { + case float64: + return int64(v) + } + return 0 +} + +func (d *lsblkDevice) sizeHuman() string { + bytes := d.sizeBytes() + const ( + GB = 1024 * 1024 * 1024 + TB = GB * 1024 + ) + switch { + case bytes >= TB: + return fmt.Sprintf("%.1f TB", float64(bytes)/float64(TB)) + case bytes >= GB: + return fmt.Sprintf("%.1f GB", float64(bytes)/float64(GB)) + default: + return fmt.Sprintf("%d MB", bytes/(1024*1024)) + } +} + +func (d *lsblkDevice) isRemovable() bool { + switch v := d.RM.(type) { + case bool: + return v + case float64: + return v != 0 + case string: + return v == "1" || strings.EqualFold(v, "true") + } + return false +} + +func (d *lsblkDevice) fsType() string { + if d.FSType != nil { + return *d.FSType + } + return "" +} + +func (d *lsblkDevice) mountPoint() string { + if d.MountPoint != nil { + return *d.MountPoint + } + return "" +} + +func (d *lsblkDevice) model() string { + if d.Model != nil { + return strings.TrimSpace(*d.Model) + } + return "" +} + +// ScanDisks detects all block devices and classifies them into +// available (not mounted, not system) and system/mounted disks. +func ScanDisks() (*ScanResult, error) { + out, err := exec.Command( + "lsblk", "-J", "-b", + "-o", "NAME,PATH,SIZE,TYPE,FSTYPE,MOUNTPOINT,MODEL,RM", + ).Output() + if err != nil { + return nil, fmt.Errorf("lsblk failed: %w", err) + } + + var parsed lsblkOutput + if err := json.Unmarshal(out, &parsed); err != nil { + return nil, fmt.Errorf("lsblk JSON parse failed: %w", err) + } + + result := &ScanResult{} + + for _, dev := range parsed.BlockDevices { + if dev.Type != "disk" { + continue + } + + bd := BlockDevice{ + Name: dev.Name, + Path: dev.Path, + Size: dev.sizeHuman(), + SizeBytes: dev.sizeBytes(), + Model: dev.model(), + Type: dev.Type, + Removable: dev.isRemovable(), + } + if bd.Path == "" { + bd.Path = "/dev/" + bd.Name + } + + isSystem := false + anyMounted := false + for _, child := range dev.Children { + if child.Type != "part" && child.Type != "lvm" && child.Type != "crypt" { + continue + } + part := Partition{ + Name: child.Name, + Path: child.Path, + Size: child.sizeHuman(), + SizeBytes: child.sizeBytes(), + FSType: child.fsType(), + MountPoint: child.mountPoint(), + } + if part.Path == "" { + part.Path = "/dev/" + part.Name + } + bd.Partitions = append(bd.Partitions, part) + if part.MountPoint != "" { + anyMounted = true + if part.MountPoint == "/" || part.MountPoint == "/boot" || part.MountPoint == "/boot/efi" { + isSystem = true + } + } + } + + // Also check if the disk itself is directly mounted (no partition table) + if dev.mountPoint() != "" { + anyMounted = true + mp := dev.mountPoint() + if mp == "/" || mp == "/boot" { + isSystem = true + } + } + + bd.Mounted = anyMounted + + if isSystem { + result.SystemDisks = append(result.SystemDisks, bd) + } else { + result.AvailableDisks = append(result.AvailableDisks, bd) + } + } + + return result, nil +} diff --git a/controller/internal/storage/scan_other.go b/controller/internal/storage/scan_other.go new file mode 100644 index 0000000..011cdca --- /dev/null +++ b/controller/internal/storage/scan_other.go @@ -0,0 +1,10 @@ +//go:build !linux + +package storage + +import "fmt" + +// ScanDisks is not supported on non-Linux platforms. +func ScanDisks() (*ScanResult, error) { + return nil, fmt.Errorf("storage init is only supported on Linux") +} diff --git a/controller/internal/web/handlers.go b/controller/internal/web/handlers.go index c5525f0..42114b4 100644 --- a/controller/internal/web/handlers.go +++ b/controller/internal/web/handlers.go @@ -187,6 +187,15 @@ func (s *Server) deployHandler(w http.ResponseWriter, r *http.Request, name stri } data["StoragePaths"] = deployPaths + // Storage info for already-deployed apps with HDD data + if alreadyDeployed { + storageInfo := s.storageInfoForStack(name) + if storageInfo != nil { + data["StorageInfo"] = storageInfo + data["OtherStoragePaths"] = s.otherStoragePathsForStack(name) + } + } + // Memory info for deploy page (only for non-deployed apps) if !alreadyDeployed { memInfo := map[string]interface{}{"Available": false} diff --git a/controller/internal/web/server.go b/controller/internal/web/server.go index 73aa719..54abfcc 100644 --- a/controller/internal/web/server.go +++ b/controller/internal/web/server.go @@ -35,6 +35,10 @@ type Server struct { sessions map[string]*session sessionsMu sync.RWMutex done chan struct{} + + // Disk operation state (format/migrate jobs) + diskJobMu sync.Mutex + diskJob *activeDiskJob } func NewServer(cfg *config.Config, stackMgr *stacks.Manager, cpuCollector *system.CPUCollector, backupMgr *backup.Manager, sched *scheduler.Scheduler, sett *settings.Settings, alertMgr *AlertManager, notif *notify.Notifier, logger *log.Logger, version string) *Server { @@ -108,6 +112,12 @@ func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) { s.settingsAppBackupHandler(w, r) case path == "/backup/restore" && r.Method == http.MethodPost: s.backupRestoreHandler(w, r) + case path == "/settings/storage/init": + s.storageInitHandler(w, r) + case strings.HasPrefix(path, "/stacks/") && strings.HasSuffix(path, "/migrate"): + name := strings.TrimPrefix(path, "/stacks/") + name = strings.TrimSuffix(name, "/migrate") + s.migratePageHandler(w, r, name) case strings.HasPrefix(path, "/stacks/") && strings.HasSuffix(path, "/logs"): name := strings.TrimPrefix(path, "/stacks/") name = strings.TrimSuffix(name, "/logs") @@ -132,6 +142,11 @@ func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) { } } +// ServeStorageAPI handles /api/storage/* routes (JSON API for disk operations). +func (s *Server) ServeStorageAPI(w http.ResponseWriter, r *http.Request) { + s.storageAPIHandler(w, r) +} + // primaryHDDPath returns the first registered storage path, or the legacy config value. func (s *Server) primaryHDDPath() string { if paths := s.settings.GetStoragePaths(); len(paths) > 0 { diff --git a/controller/internal/web/storage_handlers.go b/controller/internal/web/storage_handlers.go new file mode 100644 index 0000000..a943de5 --- /dev/null +++ b/controller/internal/web/storage_handlers.go @@ -0,0 +1,603 @@ +package web + +import ( + "encoding/json" + "fmt" + "net/http" + "os" + "path/filepath" + "strings" + "sync" + "time" + + "gitea.dooplex.hu/admin/felhom-controller/internal/settings" + "gitea.dooplex.hu/admin/felhom-controller/internal/stacks" + "gitea.dooplex.hu/admin/felhom-controller/internal/storage" + "gitea.dooplex.hu/admin/felhom-controller/internal/system" +) + +// activeDiskJob tracks an in-progress disk operation (format or migrate). +type activeDiskJob struct { + mu sync.RWMutex + jobType string // "format" or "migrate" + done bool + fmtProg []storage.FormatProgress + migProg []storage.MigrateProgress +} + +// DeployStorageInfo holds storage info for the deploy page (already-deployed apps). +type DeployStorageInfo struct { + Path string + Label string + DataSizeHuman string + FreeHuman string + FreePercent float64 +} + +// appendFmtProg adds a format progress update to the job. +func (j *activeDiskJob) appendFmtProg(p storage.FormatProgress) { + j.mu.Lock() + defer j.mu.Unlock() + j.fmtProg = append(j.fmtProg, p) + if p.Step == "done" || p.Step == "error" { + j.done = true + } +} + +// appendMigProg adds a migration progress update to the job. +func (j *activeDiskJob) appendMigProg(p storage.MigrateProgress) { + j.mu.Lock() + defer j.mu.Unlock() + j.migProg = append(j.migProg, p) + if p.Step == "done" || p.Step == "error" { + j.done = true + } +} + +// lastFmtProg returns the most recent format progress snapshot. +func (j *activeDiskJob) lastFmtProg() (storage.FormatProgress, bool) { + j.mu.RLock() + defer j.mu.RUnlock() + if len(j.fmtProg) == 0 { + return storage.FormatProgress{}, false + } + return j.fmtProg[len(j.fmtProg)-1], true +} + +// lastMigProg returns the most recent migration progress snapshot. +func (j *activeDiskJob) lastMigProg() (storage.MigrateProgress, bool) { + j.mu.RLock() + defer j.mu.RUnlock() + if len(j.migProg) == 0 { + return storage.MigrateProgress{}, false + } + return j.migProg[len(j.migProg)-1], true +} + +// isDone returns true if the job has finished. +func (j *activeDiskJob) isDone() bool { + j.mu.RLock() + defer j.mu.RUnlock() + return j.done +} + +// jsonResponse writes a JSON response. +func jsonResponse(w http.ResponseWriter, v interface{}) { + w.Header().Set("Content-Type", "application/json; charset=utf-8") + json.NewEncoder(w).Encode(v) +} + +// jsonError writes a JSON error response. +func jsonError(w http.ResponseWriter, msg string, code int) { + w.Header().Set("Content-Type", "application/json; charset=utf-8") + w.WriteHeader(code) + json.NewEncoder(w).Encode(map[string]interface{}{ + "ok": false, + "error": msg, + }) +} + +// tryStartDiskJob attempts to start a new disk operation job. +// Returns false if another job is already active. +func (s *Server) tryStartDiskJob(jobType string) (*activeDiskJob, bool) { + s.diskJobMu.Lock() + defer s.diskJobMu.Unlock() + if s.diskJob != nil && !s.diskJob.isDone() { + return nil, false + } + job := &activeDiskJob{jobType: jobType} + s.diskJob = job + return job, true +} + +// currentDiskJob returns the current disk job (may be nil or done). +func (s *Server) currentDiskJob() *activeDiskJob { + s.diskJobMu.Lock() + defer s.diskJobMu.Unlock() + return s.diskJob +} + +// --- Storage Init Wizard --- + +// storageInitHandler serves the storage init wizard page. +func (s *Server) storageInitHandler(w http.ResponseWriter, r *http.Request) { + data := s.baseData("settings", "Meghajtó inicializálása") + s.render(w, "storage_init", data) +} + +// storageAPIHandler is the main handler for /api/storage/* routes. +func (s *Server) storageAPIHandler(w http.ResponseWriter, r *http.Request) { + path := r.URL.Path + + switch { + case path == "/api/storage/scan" && r.Method == http.MethodPost: + s.storageScanAPIHandler(w, r) + case path == "/api/storage/init" && r.Method == http.MethodPost: + s.storageInitAPIHandler(w, r) + case path == "/api/storage/init/status" && r.Method == http.MethodGet: + s.storageInitStatusAPIHandler(w, r) + case path == "/api/storage/migrate" && r.Method == http.MethodPost: + s.storageMigrateAPIHandler(w, r) + case path == "/api/storage/migrate/status" && r.Method == http.MethodGet: + s.storageMigrateStatusAPIHandler(w, r) + default: + http.NotFound(w, r) + } +} + +// storageScanAPIHandler handles POST /api/storage/scan. +func (s *Server) storageScanAPIHandler(w http.ResponseWriter, r *http.Request) { + result, err := storage.ScanDisks() + if err != nil { + s.logger.Printf("[ERROR] storageScan: %v", err) + jsonError(w, "Meghajtók keresése sikertelen: "+err.Error(), http.StatusInternalServerError) + return + } + jsonResponse(w, map[string]interface{}{ + "ok": true, + "available": result.AvailableDisks, + "system": result.SystemDisks, + "available_count": len(result.AvailableDisks), + }) +} + +// storageInitAPIHandler handles POST /api/storage/init — starts format+mount job. +func (s *Server) storageInitAPIHandler(w http.ResponseWriter, r *http.Request) { + var req struct { + DevicePath string `json:"device_path"` + MountName string `json:"mount_name"` + Label string `json:"label"` + CreatePartition bool `json:"create_partition"` + SetDefault bool `json:"set_default"` + Confirm string `json:"confirm"` + } + if err := json.NewDecoder(r.Body).Decode(&req); err != nil { + jsonError(w, "Érvénytelen kérés", http.StatusBadRequest) + return + } + + if req.Confirm != "FORMÁZÁS" { + jsonError(w, "Megerősítés szükséges: írja be 'FORMÁZÁS'", http.StatusBadRequest) + return + } + if req.DevicePath == "" || req.MountName == "" { + jsonError(w, "Hiányos paraméterek", http.StatusBadRequest) + return + } + + job, ok := s.tryStartDiskJob("format") + if !ok { + jsonError(w, "Egy másik lemezművelet folyamatban van", http.StatusConflict) + return + } + + s.logger.Printf("[INFO] Storage init started: device=%s mountName=%s by %s", req.DevicePath, req.MountName, r.RemoteAddr) + + fmtReq := storage.FormatRequest{ + DevicePath: req.DevicePath, + MountName: req.MountName, + Label: req.Label, + CreatePartition: req.CreatePartition, + SetDefault: req.SetDefault, + } + + go func() { + progressCh := make(chan storage.FormatProgress, 32) + // Collect progress + go func() { + for p := range progressCh { + job.appendFmtProg(p) + } + }() + + mountPath, err := storage.FormatAndMount(fmtReq, progressCh) + close(progressCh) + + if err != nil { + s.logger.Printf("[ERROR] Storage init failed: %v", err) + return + } + + // Auto-register the new storage path + label := req.Label + if label == "" { + label = settings.InferStorageLabel(mountPath) + } + sp := settings.StoragePath{ + Path: mountPath, + Label: label, + IsDefault: req.SetDefault, + Schedulable: true, + AddedAt: time.Now().UTC().Format(time.RFC3339), + } + if err := s.settings.AddStoragePath(sp); err != nil { + s.logger.Printf("[WARN] Failed to register storage path after init: %v", err) + } else { + s.logger.Printf("[INFO] Storage path registered: %s (%s)", mountPath, label) + } + }() + + jsonResponse(w, map[string]interface{}{ + "ok": true, + "msg": "Inicializálás elindítva", + }) +} + +// storageInitStatusAPIHandler handles GET /api/storage/init/status. +func (s *Server) storageInitStatusAPIHandler(w http.ResponseWriter, r *http.Request) { + job := s.currentDiskJob() + if job == nil || job.jobType != "format" { + jsonResponse(w, map[string]interface{}{ + "ok": true, + "active": false, + }) + return + } + + p, ok := job.lastFmtProg() + if !ok { + jsonResponse(w, map[string]interface{}{ + "ok": true, + "active": true, + "step": "starting", + "msg": "Inicializálás elindult...", + "pct": 0, + }) + return + } + + jsonResponse(w, map[string]interface{}{ + "ok": true, + "active": !job.isDone(), + "step": p.Step, + "msg": p.Message, + "pct": p.Percent, + "error": p.Error, + "done": job.isDone(), + }) +} + +// --- Migration --- + +// migratePageHandler serves the migration page for an app. +func (s *Server) migratePageHandler(w http.ResponseWriter, r *http.Request, stackName string) { + stack, ok := s.stackMgr.GetStack(stackName) + if !ok { + http.NotFound(w, r) + return + } + + appCfg := s.stackMgr.LoadAppConfigByName(stackName) + if appCfg == nil || !appCfg.Deployed { + http.NotFound(w, r) + return + } + + currentHDDPath := appCfg.Env["HDD_PATH"] + if currentHDDPath == "" { + http.Error(w, "Ez az alkalmazás nem tárol adatot külső meghajtón.", http.StatusBadRequest) + return + } + + // Other storage paths (exclude current) + var otherPaths []DeployStoragePath + for _, sp := range s.settings.GetStoragePaths() { + if sp.Path == currentHDDPath { + continue + } + dp := DeployStoragePath{StoragePath: sp} + if di := system.GetDiskUsage(sp.Path); di != nil { + dp.FreeHuman = formatFreeSpace(di.AvailGB) + if di.TotalGB > 0 { + dp.FreePercent = di.AvailGB / di.TotalGB * 100 + } + } + otherPaths = append(otherPaths, dp) + } + + if len(otherPaths) == 0 { + http.Error(w, "Nincs más elérhető tárhely az áthelyezéshez.", http.StatusBadRequest) + return + } + + // Current path label + currentLabel := settings.InferStorageLabel(currentHDDPath) + for _, sp := range s.settings.GetStoragePaths() { + if sp.Path == currentHDDPath { + currentLabel = sp.Label + break + } + } + + // Estimate current data size + mounts := stacks.ParseComposeHDDMounts(stack.ComposePath, currentHDDPath) + var totalSizeHuman string + if len(mounts) > 0 { + var total int64 + for _, m := range mounts { + total += dirSizeInt64(m) + } + totalSizeHuman = dirSizeBytesHuman(total) + } + + data := s.baseData("stacks", stack.Meta.DisplayName+" — Adatáthelyezés") + data["Stack"] = stack + data["Meta"] = stack.Meta + data["CurrentHDDPath"] = currentHDDPath + data["CurrentLabel"] = currentLabel + data["OtherPaths"] = otherPaths + data["DataSizeHuman"] = totalSizeHuman + s.render(w, "migrate", data) +} + +// storageMigrateAPIHandler handles POST /api/storage/migrate — starts migration job. +func (s *Server) storageMigrateAPIHandler(w http.ResponseWriter, r *http.Request) { + var req struct { + StackName string `json:"stack_name"` + TargetPath string `json:"target_path"` + } + if err := json.NewDecoder(r.Body).Decode(&req); err != nil { + jsonError(w, "Érvénytelen kérés", http.StatusBadRequest) + return + } + + if req.StackName == "" || req.TargetPath == "" { + jsonError(w, "Hiányos paraméterek", http.StatusBadRequest) + return + } + + stack, ok := s.stackMgr.GetStack(req.StackName) + if !ok { + jsonError(w, "Alkalmazás nem található: "+req.StackName, http.StatusNotFound) + return + } + + appCfg := s.stackMgr.LoadAppConfigByName(req.StackName) + if appCfg == nil || !appCfg.Deployed { + jsonError(w, "Az alkalmazás nincs telepítve", http.StatusBadRequest) + return + } + + currentHDDPath := appCfg.Env["HDD_PATH"] + if currentHDDPath == "" { + jsonError(w, "Az alkalmazásnak nincs HDD_PATH beállítva", http.StatusBadRequest) + return + } + + if currentHDDPath == req.TargetPath { + jsonError(w, "A forrás és a cél tárhely azonos", http.StatusBadRequest) + return + } + + mounts := stacks.ParseComposeHDDMounts(stack.ComposePath, currentHDDPath) + if len(mounts) == 0 { + jsonError(w, "Az alkalmazáshoz nem találhatók HDD csatlakozások", http.StatusBadRequest) + return + } + + job, ok := s.tryStartDiskJob("migrate") + if !ok { + jsonError(w, "Egy másik lemezművelet folyamatban van", http.StatusConflict) + return + } + + s.logger.Printf("[INFO] Migration started: stack=%s from=%s to=%s by %s", + req.StackName, currentHDDPath, req.TargetPath, r.RemoteAddr) + + migrReq := storage.MigrateRequest{ + StackName: req.StackName, + DisplayName: stack.Meta.DisplayName, + CurrentHDDPath: currentHDDPath, + TargetPath: req.TargetPath, + HDDMounts: mounts, + } + + stopFn := func(name string) error { + return s.stackMgr.StopStack(name) + } + startFn := func(name string) error { + return s.stackMgr.StartStack(name) + } + updateFn := func(name, newPath string) error { + return s.updateStackHDDPath(name, newPath) + } + + go func() { + progressCh := make(chan storage.MigrateProgress, 64) + go func() { + for p := range progressCh { + job.appendMigProg(p) + } + }() + + if err := storage.MigrateAppData(migrReq, stopFn, startFn, updateFn, progressCh); err != nil { + s.logger.Printf("[ERROR] Migration failed: stack=%s: %v", req.StackName, err) + } else { + s.logger.Printf("[INFO] Migration complete: stack=%s → %s", req.StackName, req.TargetPath) + } + close(progressCh) + }() + + jsonResponse(w, map[string]interface{}{ + "ok": true, + "msg": "Áthelyezés elindítva", + }) +} + +// storageMigrateStatusAPIHandler handles GET /api/storage/migrate/status. +func (s *Server) storageMigrateStatusAPIHandler(w http.ResponseWriter, r *http.Request) { + job := s.currentDiskJob() + if job == nil || job.jobType != "migrate" { + jsonResponse(w, map[string]interface{}{ + "ok": true, + "active": false, + }) + return + } + + p, ok := job.lastMigProg() + if !ok { + jsonResponse(w, map[string]interface{}{ + "ok": true, + "active": true, + "step": "starting", + "msg": "Áthelyezés elindult...", + "pct": 0, + }) + return + } + + jsonResponse(w, map[string]interface{}{ + "ok": true, + "active": !job.isDone(), + "step": p.Step, + "msg": p.Message, + "pct": p.Percent, + "error": p.Error, + "done": job.isDone(), + "bytes_copied": p.BytesCopied, + "bytes_total": p.BytesTotal, + "elapsed_sec": p.ElapsedSeconds, + }) +} + +// updateStackHDDPath updates the HDD_PATH in a stack's app.yaml. +func (s *Server) updateStackHDDPath(stackName, newPath string) error { + stack, ok := s.stackMgr.GetStack(stackName) + if !ok { + return fmt.Errorf("stack not found: %s", stackName) + } + stackDir := filepath.Dir(stack.ComposePath) + appCfg := stacks.LoadAppConfig(stackDir) + if appCfg == nil { + return fmt.Errorf("app.yaml not found for stack: %s", stackName) + } + appCfg.Env["HDD_PATH"] = newPath + return stacks.SaveAppConfig(stackDir, appCfg) +} + +// storageInfoForStack returns deploy storage info for a deployed stack. +func (s *Server) storageInfoForStack(stackName string) *DeployStorageInfo { + appCfg := s.stackMgr.LoadAppConfigByName(stackName) + if appCfg == nil { + return nil + } + hddPath := appCfg.Env["HDD_PATH"] + if hddPath == "" { + return nil + } + + info := &DeployStorageInfo{Path: hddPath} + + // Find label + for _, sp := range s.settings.GetStoragePaths() { + if sp.Path == hddPath { + info.Label = sp.Label + break + } + } + if info.Label == "" { + info.Label = settings.InferStorageLabel(hddPath) + } + + // Data size + stack, ok := s.stackMgr.GetStack(stackName) + if ok { + mounts := stacks.ParseComposeHDDMounts(stack.ComposePath, hddPath) + var total int64 + for _, m := range mounts { + total += dirSizeInt64(m) + } + if total > 0 { + info.DataSizeHuman = dirSizeBytesHuman(total) + } + } + + // Free space + if di := system.GetDiskUsage(hddPath); di != nil { + info.FreeHuman = formatFreeSpace(di.AvailGB) + if di.TotalGB > 0 { + info.FreePercent = di.AvailGB / di.TotalGB * 100 + } + } + + return info +} + +// dirSizeInt64 returns total bytes in a directory. +func dirSizeInt64(path string) int64 { + var total int64 + filepath.Walk(path, func(_ string, info os.FileInfo, err error) error { + if err != nil || info.IsDir() { + return nil + } + total += info.Size() + return nil + }) + return total +} + +// dirSizeBytesHuman formats bytes as human-readable. +func dirSizeBytesHuman(b int64) string { + const ( + KB = 1024 + MB = KB * 1024 + GB = MB * 1024 + ) + switch { + case b >= GB: + return fmt.Sprintf("%.1f GB", float64(b)/float64(GB)) + case b >= MB: + return fmt.Sprintf("%.0f MB", float64(b)/float64(MB)) + case b >= KB: + return fmt.Sprintf("%.0f KB", float64(b)/float64(KB)) + default: + return fmt.Sprintf("%d B", b) + } +} + +// otherStoragePathsForStack returns storage paths excluding the one the app is on. +func (s *Server) otherStoragePathsForStack(stackName string) []settings.StoragePath { + appCfg := s.stackMgr.LoadAppConfigByName(stackName) + if appCfg == nil { + return nil + } + currentHDDPath := appCfg.Env["HDD_PATH"] + var others []settings.StoragePath + for _, sp := range s.settings.GetStoragePaths() { + if sp.Path != currentHDDPath { + others = append(others, sp) + } + } + return others +} + +// storageSectionLabel returns the label for a given path. +func (s *Server) storageLabelForPath(path string) string { + for _, sp := range s.settings.GetStoragePaths() { + if sp.Path == path { + return sp.Label + } + } + return strings.TrimPrefix(path, "/mnt/") +} diff --git a/controller/internal/web/templates/deploy.html b/controller/internal/web/templates/deploy.html index 69119b5..dccabe0 100644 --- a/controller/internal/web/templates/deploy.html +++ b/controller/internal/web/templates/deploy.html @@ -30,6 +30,34 @@
Ez az alkalmazás már telepítve van. Az alábbi beállítások csak olvashatók.
+ {{if .StorageInfo}} +
+

Adattárolás

+
+
+ Tárhely + {{.StorageInfo.Label}} ({{.StorageInfo.Path}}) +
+ {{if .StorageInfo.DataSizeHuman}} +
+ Adatméret + {{.StorageInfo.DataSizeHuman}} +
+ {{end}} + {{if .StorageInfo.FreeHuman}} +
+ Szabad hely + {{.StorageInfo.FreeHuman}} ({{printf "%.0f" .StorageInfo.FreePercent}}% szabad) +
+ {{end}} +
+ {{if .OtherStoragePaths}} + + 📦 Mozgatás másik tárolóra + + {{end}} +
+ {{end}} {{end}} {{if and (not .AlreadyDeployed) .MemoryInfo}} diff --git a/controller/internal/web/templates/migrate.html b/controller/internal/web/templates/migrate.html new file mode 100644 index 0000000..32185bb --- /dev/null +++ b/controller/internal/web/templates/migrate.html @@ -0,0 +1,190 @@ +{{define "migrate"}} +{{template "layout_start" .}} + +
+
+ ← Vissza +

{{.Meta.DisplayName}} — Adatáthelyezés

+
+
+ +
+

Adatok áthelyezése másik tárolóra

+ +
+
+ Jelenlegi tárhely + {{.CurrentLabel}} ({{.CurrentHDDPath}}) +
+ {{if .DataSizeHuman}} +
+ Adatméret + {{.DataSizeHuman}} +
+ {{end}} +
+ +
+ + +
+ +
+ Figyelmeztetések: +
    +
  • Az alkalmazás a mozgatás idejére leáll
    • +
  • Nagy adatmennyiségnél ez percekig tarthat
    • +
  • A régi adatok megmaradnak biztonsági másolatként
    • +
+
+ + + +
+ + Mégsem +
+
+ + + + + + + +{{template "layout_end" .}} +{{end}} diff --git a/controller/internal/web/templates/settings.html b/controller/internal/web/templates/settings.html index 895a8f9..9b53e2e 100644 --- a/controller/internal/web/templates/settings.html +++ b/controller/internal/web/templates/settings.html @@ -117,6 +117,7 @@
{{.Name}} {{if .SizeHuman}}{{.SizeHuman}}{{end}} + 📦 Mozgatás
{{end}} @@ -163,8 +164,12 @@ {{end}} +
+ 🔧 Új meghajtó inicializálása +
+
- Új adattároló hozzáadása + Már csatlakoztatott tárhely hozzáadása kézzel
diff --git a/controller/internal/web/templates/storage_init.html b/controller/internal/web/templates/storage_init.html new file mode 100644 index 0000000..657f0bf --- /dev/null +++ b/controller/internal/web/templates/storage_init.html @@ -0,0 +1,325 @@ +{{define "storage_init"}} +{{template "layout_start" .}} + +
+
+ ← Vissza +

Új meghajtó inicializálása

+
+
+ +
+

1. Meghajtók keresése

+

Keresse meg a rendszerhez csatlakoztatott, még nem inicializált meghajtókat.

+ + + + + +
+ + + + + + + + + +{{template "layout_end" .}} +{{end}} diff --git a/controller/internal/web/templates/style.css b/controller/internal/web/templates/style.css index 06aaa14..245d9e2 100644 --- a/controller/internal/web/templates/style.css +++ b/controller/internal/web/templates/style.css @@ -2127,3 +2127,84 @@ a.stat-card:hover { .restore-label { min-width: auto; } .restore-select { max-width: 100%; } } + +/* =================================================================== + Storage Init Wizard & Migration UI + =================================================================== */ + +.disk-progress-steps { + display: flex; + flex-direction: column; + gap: .5rem; + margin-top: 1rem; +} + +.disk-step { + display: flex; + align-items: center; + gap: .75rem; + padding: .5rem .75rem; + border-radius: 8px; + color: var(--text-secondary); + font-size: .9rem; + background: var(--bg-primary); +} + +.disk-step-icon { + font-size: 1.1rem; + min-width: 1.4rem; +} + +.disk-step.disk-step-done { + color: var(--text-primary); +} + +.disk-step.disk-step-active { + background: rgba(0, 136, 204, 0.08); + color: var(--accent-light); + border: 1px solid rgba(0, 136, 204, 0.2); +} + +.disk-progress-bar-wrap { + display: flex; + align-items: center; + gap: 1rem; +} + +.disk-progress-bar-wrap .system-bar { + flex: 1; +} + +.deploy-storage-info { + background: var(--bg-primary); + border: 1px solid var(--border-color); + border-radius: var(--radius); + padding: 1.25rem; + margin-bottom: 1rem; +} + +.deploy-storage-info h4 { + font-size: .95rem; + font-weight: 600; + color: var(--text-secondary); + text-transform: uppercase; + letter-spacing: .05em; + margin-bottom: .75rem; +} + +.storage-app-row { + display: flex; + align-items: center; + gap: .75rem; + padding: .35rem 0; +} + +.storage-app-link { + color: var(--accent-light); + text-decoration: none; + font-size: .9rem; +} + +.storage-app-link:hover { + text-decoration: underline; +}