v0.26.0: Storage namespace felhom-data/ + test node wipe script
All felhom-managed data on external drives now lives under felhom-data/ subdirectory, cleanly separating controller data from user files. - backup/paths.go: add FelhomDataDir constant, update 8 path helpers - stacks/delete.go: add local felhomDataDir constant (circular import boundary), update ProtectedHDDPaths + GetStackBackupData - storage/migrate_drive.go: import backup pkg, fix conflict check, verify, rsync excludes (felhom-data/backups/*/restic/), size estimation - storage/migrate.go: import backup pkg, fix DB dump paths - web/handlers.go: fix legacy 'storage' path -> backup.AppDataDir() - storage/format_linux.go: create felhom-data/ instead of storage/ - storage/attach_linux.go: create felhom-data/ instead of storage/ - scripts/felhom-wipe.sh: new multi-level test node wipe script (soft/controller/full/nuclear) - CHANGELOG.md, controller/README.md, scripts/README.md: updated docs Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
+31
-17
@@ -4,7 +4,7 @@
|
||||
|
||||
A single, lightweight Go container that replaces Portainer + scattered systemd scripts with a unified, Hungarian-language web dashboard for managing Docker Compose stacks, backups, storage, monitoring, and notifications on customer hardware.
|
||||
|
||||
**Current version: v0.25.0**
|
||||
**Current version: v0.26.0**
|
||||
|
||||
---
|
||||
|
||||
@@ -227,21 +227,35 @@ self-sufficient backup** — any single tier can fully restore an app.
|
||||
|
||||
The nightly backup has two phases that run sequentially. All paths are **per-drive** — each physical drive gets its own restic repo and per-app DB dump directories.
|
||||
|
||||
**Drive layout (v0.14.1):**
|
||||
**Drive layout (v0.26.0):**
|
||||
```
|
||||
<drive>/
|
||||
├── appdata/<app>/ ← app user data
|
||||
└── backups/
|
||||
└── primary/
|
||||
├── restic/ ← one restic repo per drive (all apps on this drive)
|
||||
└── <app>/db-dumps/ ← per-app DB dump files
|
||||
├── felhom-data/ ← all controller-managed data (namespace, v0.26.0+)
|
||||
│ ├── appdata/<app>/ ← app user data
|
||||
│ └── backups/
|
||||
│ ├── primary/
|
||||
│ │ ├── restic/ ← one restic repo per drive (all apps on this drive)
|
||||
│ │ └── <app>/db-dumps/ ← per-app DB dump files
|
||||
│ └── secondary/
|
||||
│ ├── restic/ ← secondary restic repo (cross-drive)
|
||||
│ ├── _infra/ ← infra config mirror
|
||||
│ └── <app>/rsync/ ← per-app rsync data
|
||||
├── .felhom-infra-backup/ ← DR marker (stays at drive root for scanner)
|
||||
├── Dokumentumok/ ← user files (not controller-managed)
|
||||
└── media/ ← user files (not controller-managed)
|
||||
```
|
||||
|
||||
Path computation is centralized in `backup/paths.go`:
|
||||
- `PrimaryResticRepoPath(drivePath)` → `<drive>/backups/primary/restic/`
|
||||
- `AppDBDumpPath(drivePath, stackName)` → `<drive>/backups/primary/<stack>/db-dumps/`
|
||||
- `AppDataDir(drivePath, stackName)` → `<drive>/appdata/<stack>/`
|
||||
- `SecondaryInfraPath(drivePath)` → `<drive>/backups/secondary/_infra/`
|
||||
> **Note:** `HDD_PATH` env var in `app.yaml` is still the mount point (e.g., `/mnt/hdd_1`). The `felhom-data` segment is embedded in path helpers — not in `HDD_PATH`.
|
||||
> Pre-v0.26.0 installations use `<drive>/appdata/` and `<drive>/backups/` directly (no `felhom-data/` namespace).
|
||||
|
||||
Path computation is centralized in `backup/paths.go` via the `FelhomDataDir = "felhom-data"` constant:
|
||||
- `PrimaryResticRepoPath(drivePath)` → `<drive>/felhom-data/backups/primary/restic/`
|
||||
- `AppDBDumpPath(drivePath, stackName)` → `<drive>/felhom-data/backups/primary/<stack>/db-dumps/`
|
||||
- `AppDataDir(drivePath, stackName)` → `<drive>/felhom-data/appdata/<stack>/`
|
||||
- `SecondaryResticRepoPath(drivePath)` → `<drive>/felhom-data/backups/secondary/restic/`
|
||||
- `AppSecondaryRsyncPath(drivePath, stackName)` → `<drive>/felhom-data/backups/secondary/<stack>/rsync/`
|
||||
- `SecondaryInfraPath(drivePath)` → `<drive>/felhom-data/backups/secondary/_infra/`
|
||||
- `InfraBackupDir(mountPath)` → `<drive>/.felhom-infra-backup/` (**unchanged** — stays at drive root for DR scanner)
|
||||
|
||||
**Phase 1 — Database Dumps** (`internal/backup/dbdump.go`, scheduled 02:30)
|
||||
|
||||
@@ -405,7 +419,7 @@ A step-by-step UI at `/settings/storage/init`:
|
||||
1. **Scan** — Lists available disks with model, size, partition info
|
||||
2. **Select** — User picks a disk and enters a mount name (e.g., `hdd_1`)
|
||||
3. **Confirm** — User types "FORMAZAS" to confirm destructive operation
|
||||
4. **Format pipeline**: `wipefs` → `sfdisk` (GPT) → `mkfs.ext4` → `blkid` UUID → backup fstab → append UUID-based fstab entry → mount → `findmnt` verification → `chown 1000:1000` → create `appdata/`, `backups/`, and `Dokumentumok/` subdirectories
|
||||
4. **Format pipeline**: `wipefs` → `sfdisk` (GPT) → `mkfs.ext4` → `blkid` UUID → backup fstab → append UUID-based fstab entry → mount → `findmnt` verification → `chown 1000:1000` → create `felhom-data/` and `Dokumentumok/` subdirectories
|
||||
5. Auto-registers new storage path in settings.json
|
||||
6. Smart partition detection: skips repartitioning for existing empty partitions
|
||||
|
||||
@@ -415,7 +429,7 @@ Safety guards: system disk detection, mount path conflict check, confirmation re
|
||||
|
||||
A step-by-step UI at `/settings/storage/attach` for drives that already have a filesystem (e.g., a previously used ext4 drive). Unlike the init wizard, this does **not** format the drive — existing data is preserved.
|
||||
|
||||
**Problem solved:** Mounting a whole drive at `/mnt/<name>` would mix existing user data with the controller's directory structure (`storage/`, `Dokumentumok/`, backup repos). The bind-mount approach isolates the controller's working directory from other data on the drive.
|
||||
**Problem solved:** Mounting a whole drive at `/mnt/<name>` would mix existing user data with the controller's directory structure (`felhom-data/`, `Dokumentumok/`, etc.). The bind-mount approach isolates the controller's working directory from other data on the drive.
|
||||
|
||||
1. **Scan** — Lists available disks, filtered to partitions that have an existing filesystem (FSType != "")
|
||||
2. **Mount raw** — Partition is mounted read-only at a hidden staging path (`/mnt/.felhom-raw/<label>`)
|
||||
@@ -424,7 +438,7 @@ A step-by-step UI at `/settings/storage/attach` for drives that already have a f
|
||||
5. **Finalize** — Bind-mounts the selected subfolder at `/mnt/<name>`. Two fstab entries are created (both with `nofail`):
|
||||
- Raw mount: `UUID=<uuid> /mnt/.felhom-raw/<x> <fstype> defaults,nofail,noatime 0 2`
|
||||
- Bind mount: `/mnt/.felhom-raw/<x>/<subfolder> /mnt/<name> none bind,nofail 0 0`
|
||||
6. Sets permissions (`chown 1000:1000`), creates `storage/` and `Dokumentumok/` subdirectories
|
||||
6. Sets permissions (`chown 1000:1000`), creates `felhom-data/` and `Dokumentumok/` subdirectories
|
||||
7. Auto-registers the storage path in settings.json + syncs FileBrowser mounts
|
||||
|
||||
Cancel at any point cleans up the temporary raw mount. The bind mount path (`/mnt/<name>`) is a real mount point, so all existing code (disk usage, IsMountPoint checks, etc.) works unchanged.
|
||||
@@ -458,7 +472,7 @@ Progress UI at `/stacks/{name}/migrate` with byte counter and percentage.
|
||||
After migration, the deploy page detects leftover data on previous storage paths:
|
||||
- Shows path, size, and a delete button
|
||||
- Two-step confirmation required
|
||||
- Protected paths (appdata, backups, media, Dokumentumok) cannot be deleted
|
||||
- Protected paths (`felhom-data/`, `felhom-data/appdata/`, `felhom-data/backups/`, `media/`, `Dokumentumok/`) cannot be deleted
|
||||
|
||||
#### FileBrowser Mount Sync
|
||||
|
||||
@@ -1044,7 +1058,7 @@ controller/
|
||||
│ │ └── *_other.go # Non-Linux stubs for cross-compilation
|
||||
│ ├── backup/
|
||||
│ │ ├── backup.go # Orchestrator (per-drive dumps + restic + cross-drive chain)
|
||||
│ │ ├── paths.go # Per-drive path helpers (PrimaryResticRepoPath, InfraBackupDir, etc.)
|
||||
│ │ ├── paths.go # Per-drive path helpers (FelhomDataDir constant, PrimaryResticRepoPath, AppDataDir, InfraBackupDir, etc.)
|
||||
│ │ ├── local_infra.go # Local infra backup to all drives (.felhom-infra-backup/)
|
||||
│ │ ├── dbdump.go # DB auto-discovery + dump (pg_dump, mariadb-dump)
|
||||
│ │ ├── restic.go # Restic operations (init, snapshot, prune, check) — repoPath as param
|
||||
|
||||
@@ -2,44 +2,47 @@ package backup
|
||||
|
||||
import "path/filepath"
|
||||
|
||||
// FelhomDataDir is the namespace directory on storage drives for all felhom-managed data.
|
||||
const FelhomDataDir = "felhom-data"
|
||||
|
||||
// PrimaryBackupPath returns the root primary backup directory for a drive.
|
||||
func PrimaryBackupPath(drivePath string) string {
|
||||
return filepath.Join(drivePath, "backups", "primary")
|
||||
return filepath.Join(drivePath, FelhomDataDir, "backups", "primary")
|
||||
}
|
||||
|
||||
// PrimaryResticRepoPath returns the restic repo path on a drive's primary backup.
|
||||
func PrimaryResticRepoPath(drivePath string) string {
|
||||
return filepath.Join(drivePath, "backups", "primary", "restic")
|
||||
return filepath.Join(drivePath, FelhomDataDir, "backups", "primary", "restic")
|
||||
}
|
||||
|
||||
// AppDBDumpPath returns the DB dump directory for an app on its home drive.
|
||||
func AppDBDumpPath(drivePath, stackName string) string {
|
||||
return filepath.Join(drivePath, "backups", "primary", stackName, "db-dumps")
|
||||
return filepath.Join(drivePath, FelhomDataDir, "backups", "primary", stackName, "db-dumps")
|
||||
}
|
||||
|
||||
// SecondaryBackupPath returns the root secondary backup directory for a drive.
|
||||
func SecondaryBackupPath(drivePath string) string {
|
||||
return filepath.Join(drivePath, "backups", "secondary")
|
||||
return filepath.Join(drivePath, FelhomDataDir, "backups", "secondary")
|
||||
}
|
||||
|
||||
// AppSecondaryRsyncPath returns the rsync destination for an app's secondary backup.
|
||||
func AppSecondaryRsyncPath(drivePath, stackName string) string {
|
||||
return filepath.Join(drivePath, "backups", "secondary", stackName, "rsync")
|
||||
return filepath.Join(drivePath, FelhomDataDir, "backups", "secondary", stackName, "rsync")
|
||||
}
|
||||
|
||||
// SecondaryResticRepoPath returns the restic repo path on a drive's secondary backup.
|
||||
func SecondaryResticRepoPath(drivePath string) string {
|
||||
return filepath.Join(drivePath, "backups", "secondary", "restic")
|
||||
return filepath.Join(drivePath, FelhomDataDir, "backups", "secondary", "restic")
|
||||
}
|
||||
|
||||
// SecondaryInfraPath returns the infrastructure config mirror directory on a drive's secondary backup.
|
||||
func SecondaryInfraPath(drivePath string) string {
|
||||
return filepath.Join(drivePath, "backups", "secondary", "_infra")
|
||||
return filepath.Join(drivePath, FelhomDataDir, "backups", "secondary", "_infra")
|
||||
}
|
||||
|
||||
// AppDataDir returns the app data directory path on a drive.
|
||||
func AppDataDir(drivePath, stackName string) string {
|
||||
return filepath.Join(drivePath, "appdata", stackName)
|
||||
return filepath.Join(drivePath, FelhomDataDir, "appdata", stackName)
|
||||
}
|
||||
|
||||
// InfraBackupDir returns the hidden infra backup directory on a drive.
|
||||
|
||||
@@ -11,6 +11,9 @@ import (
|
||||
"time"
|
||||
)
|
||||
|
||||
// felhomDataDir matches backup.FelhomDataDir — duplicated to avoid circular import via StackDataProvider.
|
||||
const felhomDataDir = "felhom-data"
|
||||
|
||||
// DeleteResponse holds the result of a stack deletion (orphan delete).
|
||||
type DeleteResponse struct {
|
||||
Deleted string `json:"deleted"`
|
||||
@@ -56,11 +59,12 @@ func ProtectedHDDPaths(hddPath string) map[string]bool {
|
||||
return nil
|
||||
}
|
||||
return map[string]bool{
|
||||
hddPath: true,
|
||||
filepath.Join(hddPath, "appdata"): true,
|
||||
filepath.Join(hddPath, "backups"): true,
|
||||
filepath.Join(hddPath, "media"): true,
|
||||
filepath.Join(hddPath, "Dokumentumok"): true,
|
||||
hddPath: true,
|
||||
filepath.Join(hddPath, felhomDataDir): true,
|
||||
filepath.Join(hddPath, felhomDataDir, "appdata"): true,
|
||||
filepath.Join(hddPath, felhomDataDir, "backups"): true,
|
||||
filepath.Join(hddPath, "media"): true,
|
||||
filepath.Join(hddPath, "Dokumentumok"): true,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -351,12 +355,12 @@ func (m *Manager) GetStackBackupData(name string, drivePath string) (*BackupData
|
||||
return resp, nil
|
||||
}
|
||||
|
||||
// Check DB dump directory: <drive>/backups/primary/<stack>/db-dumps
|
||||
dbDumpPath := filepath.Join(drivePath, "backups", "primary", name, "db-dumps")
|
||||
// Check DB dump directory: <drive>/felhom-data/backups/primary/<stack>/db-dumps
|
||||
dbDumpPath := filepath.Join(drivePath, felhomDataDir, "backups", "primary", name, "db-dumps")
|
||||
resp.BackupPaths = append(resp.BackupPaths, buildPathInfo(dbDumpPath))
|
||||
|
||||
// Check cross-drive rsync directory: <drive>/backups/secondary/<stack>/rsync
|
||||
rsyncPath := filepath.Join(drivePath, "backups", "secondary", name, "rsync")
|
||||
// Check cross-drive rsync directory: <drive>/felhom-data/backups/secondary/<stack>/rsync
|
||||
rsyncPath := filepath.Join(drivePath, felhomDataDir, "backups", "secondary", name, "rsync")
|
||||
resp.BackupPaths = append(resp.BackupPaths, buildPathInfo(rsyncPath))
|
||||
|
||||
for _, p := range resp.BackupPaths {
|
||||
|
||||
@@ -310,7 +310,7 @@ func FinalizeAttach(req AttachRequest, progress chan<- FormatProgress) (string,
|
||||
|
||||
_ = exec.Command("chown", "1000:1000", mountPath).Run()
|
||||
|
||||
for _, subdir := range []string{"storage", "Dokumentumok"} {
|
||||
for _, subdir := range []string{"felhom-data", "Dokumentumok"} {
|
||||
dir := filepath.Join(mountPath, subdir)
|
||||
if err := os.MkdirAll(dir, 0755); err == nil {
|
||||
_ = exec.Command("chown", "1000:1000", dir).Run()
|
||||
|
||||
@@ -208,7 +208,7 @@ func FormatAndMount(req FormatRequest, progress chan<- FormatProgress) (string,
|
||||
|
||||
_ = exec.Command("chown", "1000:1000", mountPath).Run()
|
||||
|
||||
for _, subdir := range []string{"storage", "Dokumentumok"} {
|
||||
for _, subdir := range []string{"felhom-data", "Dokumentumok"} {
|
||||
dir := filepath.Join(mountPath, subdir)
|
||||
if err := os.MkdirAll(dir, 0755); err == nil {
|
||||
_ = exec.Command("chown", "1000:1000", dir).Run()
|
||||
|
||||
@@ -13,6 +13,7 @@ import (
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/backup"
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/settings"
|
||||
)
|
||||
|
||||
@@ -394,8 +395,8 @@ func (o *MigrateOrchestrator) RunEnhancedMigration(
|
||||
// --- Post-migration steps (all non-fatal) ---
|
||||
|
||||
// 1. Copy DB dumps from source to destination
|
||||
srcDBDumps := filepath.Join(req.CurrentHDDPath, "backups", "primary", req.StackName, "db-dumps")
|
||||
dstDBDumps := filepath.Join(req.TargetPath, "backups", "primary", req.StackName, "db-dumps")
|
||||
srcDBDumps := backup.AppDBDumpPath(req.CurrentHDDPath, req.StackName)
|
||||
dstDBDumps := backup.AppDBDumpPath(req.TargetPath, req.StackName)
|
||||
if _, err := os.Stat(srcDBDumps); err == nil {
|
||||
if err := os.MkdirAll(filepath.Dir(dstDBDumps), 0755); err != nil {
|
||||
o.Logger.Printf("[WARN] Migration %s: failed to create DB dump dir: %v", req.StackName, err)
|
||||
|
||||
@@ -12,6 +12,7 @@ import (
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/backup"
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/settings"
|
||||
)
|
||||
|
||||
@@ -180,7 +181,7 @@ func (dm *DriveMigrator) MigrateDrive(ctx context.Context, req DriveMigrateReque
|
||||
|
||||
// Check for conflicts on destination
|
||||
for _, app := range appsToMigrate {
|
||||
destAppData := filepath.Join(req.DestPath, "appdata", app.Name)
|
||||
destAppData := backup.AppDataDir(req.DestPath, app.Name)
|
||||
if info, err := os.Stat(destAppData); err == nil && info.IsDir() {
|
||||
entries, _ := os.ReadDir(destAppData)
|
||||
if len(entries) > 0 {
|
||||
@@ -192,18 +193,30 @@ func (dm *DriveMigrator) MigrateDrive(ctx context.Context, req DriveMigrateReque
|
||||
}
|
||||
}
|
||||
|
||||
// Estimate total size (exclude restic repos)
|
||||
// Estimate total size (exclude restic repos inside felhom-data/backups/)
|
||||
var totalBytes int64
|
||||
entries, _ := os.ReadDir(req.SourcePath)
|
||||
for _, entry := range entries {
|
||||
if !entry.IsDir() {
|
||||
continue
|
||||
}
|
||||
entryPath := filepath.Join(req.SourcePath, entry.Name())
|
||||
if entry.IsDir() {
|
||||
// Skip restic repos in size estimate
|
||||
if entry.Name() == "backups" {
|
||||
totalBytes += dirSizeExcluding(entryPath, "restic")
|
||||
} else {
|
||||
totalBytes += dirSize(entryPath)
|
||||
if entry.Name() == backup.FelhomDataDir {
|
||||
// Scan inside namespace dir, excluding restic repos from estimate
|
||||
subEntries, _ := os.ReadDir(entryPath)
|
||||
for _, sub := range subEntries {
|
||||
if !sub.IsDir() {
|
||||
continue
|
||||
}
|
||||
subPath := filepath.Join(entryPath, sub.Name())
|
||||
if sub.Name() == "backups" {
|
||||
totalBytes += dirSizeExcluding(subPath, "restic")
|
||||
} else {
|
||||
totalBytes += dirSize(subPath)
|
||||
}
|
||||
}
|
||||
} else {
|
||||
totalBytes += dirSize(entryPath)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -252,8 +265,8 @@ func (dm *DriveMigrator) MigrateDrive(ctx context.Context, req DriveMigrateReque
|
||||
send("copying", "Adatok másolása...", 10)
|
||||
|
||||
rsyncCmd := exec.CommandContext(ctx, "rsync", "-a", "--info=progress2",
|
||||
"--exclude=backups/primary/restic/",
|
||||
"--exclude=backups/secondary/restic/",
|
||||
"--exclude=felhom-data/backups/primary/restic/",
|
||||
"--exclude=felhom-data/backups/secondary/restic/",
|
||||
req.SourcePath+"/", req.DestPath+"/",
|
||||
)
|
||||
|
||||
@@ -322,11 +335,11 @@ func (dm *DriveMigrator) MigrateDrive(ctx context.Context, req DriveMigrateReque
|
||||
send("verifying", "Másolat ellenőrzése...", 62)
|
||||
|
||||
for _, app := range appsToMigrate {
|
||||
destAppData := filepath.Join(req.DestPath, "appdata", app.Name)
|
||||
destAppData := backup.AppDataDir(req.DestPath, app.Name)
|
||||
if _, err := os.Stat(destAppData); os.IsNotExist(err) {
|
||||
// appdata might not exist for all apps (SSD-only apps that share the drive)
|
||||
// Only warn, don't fail
|
||||
dm.Logger.Printf("[WARN] Drive migration: %s/appdata/%s not found on destination (may be SSD-only)", req.DestPath, app.Name)
|
||||
dm.Logger.Printf("[WARN] Drive migration: %s not found on destination (may be SSD-only)", destAppData)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -1220,7 +1220,7 @@ func (s *Server) appDetailsForPath(storagePath string) []StorageAppDetail {
|
||||
Stack: stack.Meta.Slug,
|
||||
}
|
||||
// Try to get data size from the storage subdirectory
|
||||
appDataDir := filepath.Join(storagePath, "storage", stack.Name)
|
||||
appDataDir := backup.AppDataDir(storagePath, stack.Name)
|
||||
if fi, err := os.Stat(appDataDir); err == nil && fi.IsDir() {
|
||||
detail.SizeHuman = dirSizeHuman(appDataDir)
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user