705 lines
30 KiB
Markdown
705 lines
30 KiB
Markdown
# TASK: Phase C — Storage Initialization, Data Migration & Startup Fixes
|
||
|
||
**Version target:** controller 0.11.0
|
||
**Repo:** `deploy-felhom-compose` (controller)
|
||
|
||
## Overview
|
||
|
||
Three features in this phase:
|
||
|
||
1. **Startup ping + hub report** — Controller should announce itself immediately on start, not wait 5–15 minutes for the first scheduler tick
|
||
2. **Storage initialization** — Detect unformatted/unmounted disks, format (ext4), mount, and register as storage path — all from the web UI
|
||
3. **Data migration** — Per-app "Mozgatás" button to move app data between storage paths with rsync + progress
|
||
|
||
---
|
||
|
||
## 0. Startup Ping & Hub Report (Quick Fix)
|
||
|
||
### 0.1 Problem
|
||
|
||
After controller starts (e.g., after `docker restart felhom-controller` or system reboot), the first heartbeat fires after 5 minutes, first system_health after 5 minutes, and first hub report after 15 minutes. During this gap, Healthchecks shows stale "Last Ping: X minutes ago" and hub has no fresh data.
|
||
|
||
### 0.2 Fix: Fire initial pings + report immediately after scheduler starts
|
||
|
||
Add to `main.go` after `sched.Start(ctx)`, inside a goroutine (non-blocking):
|
||
|
||
```go
|
||
// Fire startup pings + hub report immediately (don't wait for first scheduler tick)
|
||
go func() {
|
||
time.Sleep(5 * time.Second) // Let all subsystems fully initialize
|
||
|
||
// Heartbeat ping
|
||
pinger.Ping(cfg.Monitoring.PingUUIDs.Heartbeat, "startup")
|
||
logger.Println("[INFO] Startup heartbeat ping sent")
|
||
|
||
// System health ping
|
||
healthReport := monitor.RunHealthCheck(cfg, cpuCollector, sett.GetStoragePaths())
|
||
body := healthReport.FormatMessage()
|
||
healthUUID := cfg.Monitoring.PingUUIDs.SystemHealth
|
||
if healthReport.Status == "fail" {
|
||
pinger.Fail(healthUUID, body)
|
||
} else {
|
||
pinger.Ping(healthUUID, body)
|
||
}
|
||
logger.Printf("[INFO] Startup health ping sent (status: %s)", healthReport.Status)
|
||
|
||
// Hub report
|
||
if cfg.Hub.Enabled && cfg.Hub.URL != "" {
|
||
pusher := report.NewPusher(&cfg.Hub, logger)
|
||
r := report.BuildReport(cfg, stackMgr, backupMgr, cpuCollector, metricsStore, Version, sett.GetStoragePaths())
|
||
if err := pusher.Push(r); err != nil {
|
||
logger.Printf("[WARN] Startup hub report failed: %v", err)
|
||
} else {
|
||
logger.Println("[INFO] Startup hub report sent")
|
||
}
|
||
}
|
||
}()
|
||
```
|
||
|
||
**Note:** The existing `go func()` that runs `alertMgr.Refresh()` at startup already does a health check but doesn't ping Healthchecks or hub. Merge this logic or add alongside it. The 5-second delay gives Docker, metrics, and backup subsystems time to initialize.
|
||
|
||
### 0.3 Reuse existing pusher instance
|
||
|
||
The hub-report scheduler task already creates a `pusher` — consider creating it once at init and reusing:
|
||
|
||
```go
|
||
var pusher *report.Pusher
|
||
if cfg.Hub.Enabled && cfg.Hub.URL != "" {
|
||
pusher = report.NewPusher(&cfg.Hub, logger)
|
||
sched.Every("hub-report", pushInterval, func(ctx context.Context) error {
|
||
r := report.BuildReport(...)
|
||
return pusher.Push(r)
|
||
})
|
||
}
|
||
// ... then in startup goroutine:
|
||
if pusher != nil {
|
||
// use same pusher
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## 1. Storage Initialization Feature
|
||
|
||
### 1.1 Concept
|
||
|
||
The controller UI provides a guided wizard to:
|
||
1. **Scan** for block devices that are not mounted (like `sdb` in the demo system)
|
||
2. **Show** disk info: size, model, existing partitions/filesystems
|
||
3. **Format** with ext4 after explicit confirmation with safety warnings
|
||
4. **Mount** at a user-specified path under `/mnt/`
|
||
5. **Register** automatically as a storage path in settings.json
|
||
|
||
This replaces the need to SSH in and run `hdd-setup.sh` manually. The existing `scripts/hdd-setup.sh` in the repo provides the proven logic — the controller wraps this as a web-based flow.
|
||
|
||
### 1.2 Architecture: Go package, not bash wrapper
|
||
|
||
Implement in Go directly (not shelling out to `hdd-setup.sh`) for:
|
||
- Structured error handling and progress feedback
|
||
- Testability
|
||
- No dependency on bash script being present in the container
|
||
|
||
Use `exec.Command` for: `lsblk`, `blkid`, `mkfs.ext4`, `mount`, and fstab editing.
|
||
|
||
### 1.3 New package: `internal/storage/`
|
||
|
||
```go
|
||
package storage
|
||
|
||
// BlockDevice represents a detected physical disk.
|
||
type BlockDevice struct {
|
||
Name string // "sdb"
|
||
Path string // "/dev/sdb"
|
||
Size string // "931.5G"
|
||
SizeBytes int64
|
||
Model string // "WD Elements 25A2"
|
||
Serial string // "WX..." (if available)
|
||
Type string // "disk"
|
||
Removable bool // true for USB
|
||
Partitions []Partition
|
||
Mounted bool // any partition mounted
|
||
}
|
||
|
||
// Partition represents a partition on a block device.
|
||
type Partition struct {
|
||
Name string // "sdb1"
|
||
Path string // "/dev/sdb1"
|
||
Size string // "931.5G"
|
||
FSType string // "ext4", "" (no filesystem)
|
||
Label string // filesystem label
|
||
UUID string
|
||
MountPoint string // "" if not mounted
|
||
}
|
||
|
||
// ScanResult from disk detection.
|
||
type ScanResult struct {
|
||
AvailableDisks []BlockDevice // Unmounted disks/partitions
|
||
SystemDisks []BlockDevice // Mounted/system disks (for display only, not selectable)
|
||
}
|
||
|
||
// FormatRequest parameters for formatting a disk.
|
||
type FormatRequest struct {
|
||
DevicePath string // "/dev/sdb" (whole disk) or "/dev/sdb1" (partition)
|
||
MountName string // "hdd_1" → will mount at /mnt/hdd_1
|
||
Label string // Display label for storage path registry
|
||
CreatePartition bool // If true, wipe and create single partition first
|
||
}
|
||
|
||
// FormatProgress tracks the formatting/mounting progress.
|
||
type FormatProgress struct {
|
||
Step string // "partitioning", "formatting", "mounting", "fstab", "registering", "done", "error"
|
||
Message string // Human-readable status
|
||
Error string // Error message if Step == "error"
|
||
Percent int // 0-100
|
||
}
|
||
```
|
||
|
||
### 1.4 Core functions
|
||
|
||
```go
|
||
// ScanDisks detects all block devices and classifies them.
|
||
func ScanDisks() (*ScanResult, error) {
|
||
// Run: lsblk -J -o NAME,SIZE,TYPE,FSTYPE,MOUNTPOINT,MODEL,SERIAL,RM,PKNAME -b
|
||
// Parse JSON output
|
||
// Classify: system disk (has / or /boot mount) vs available
|
||
// A disk is "available" if NO partition is mounted AND it's not the system disk
|
||
}
|
||
|
||
// FormatAndMount formats a partition and mounts it.
|
||
// This is a long-running operation — use a channel for progress updates.
|
||
func FormatAndMount(req FormatRequest, progress chan<- FormatProgress) error {
|
||
// Step 1: Validate
|
||
// - Device exists
|
||
// - Not already mounted
|
||
// - Not system disk (compare with / device)
|
||
// - Mount name valid (alphanumeric + underscore, no spaces)
|
||
// - Mount path doesn't already exist OR is empty dir
|
||
|
||
// Step 2: Partition (if CreatePartition)
|
||
// - sfdisk: create single Linux partition filling whole disk
|
||
// - Wait for kernel to re-read partition table (partprobe)
|
||
// - Update device path to new partition (e.g., /dev/sdb1)
|
||
|
||
// Step 3: Format
|
||
// - mkfs.ext4 -L <label> <partition>
|
||
// - This is the slow step for large disks
|
||
|
||
// Step 4: Mount
|
||
// - mkdir -p /mnt/<mount_name>
|
||
// - Get UUID via blkid
|
||
// - Backup /etc/fstab to /etc/fstab.bak.YYYYMMDD
|
||
// - Append fstab entry: UUID=<uuid> /mnt/<name> ext4 defaults,nofail,noatime 0 2
|
||
// - mount /mnt/<name>
|
||
// - Verify mount succeeded
|
||
|
||
// Step 5: Set permissions
|
||
// - chown 1000:1000 /mnt/<name>
|
||
// - Create standard Felhom subdirs: storage/, Dokumentumok/
|
||
|
||
// Step 6: Register
|
||
// - Return mount path for caller to register in settings.json
|
||
}
|
||
```
|
||
|
||
### 1.5 UI: Wizard on Settings Page
|
||
|
||
Add a new section on the Beállítások page (or as a separate page `/settings/storage/init`):
|
||
|
||
**Step 1: Scan**
|
||
```
|
||
┌─────────────────────────────────────────────┐
|
||
│ Új meghajtó inicializálása │
|
||
│ │
|
||
│ [🔍 Meghajtók keresése] │
|
||
│ │
|
||
│ Talált meghajtók: │
|
||
│ ┌──────────────────────────────────────┐ │
|
||
│ │ ○ /dev/sdb — 931.5 GB │ │
|
||
│ │ WD Elements 25A2 │ │
|
||
│ │ 1 partíció (sdb1) — nincs fájlrendszer│ │
|
||
│ └──────────────────────────────────────┘ │
|
||
│ │
|
||
│ A rendszermeghajtó (sda) nem választható. │
|
||
└─────────────────────────────────────────────┘
|
||
```
|
||
|
||
**Step 2: Configure**
|
||
```
|
||
┌─────────────────────────────────────────────┐
|
||
│ Meghajtó: /dev/sdb (931.5 GB, WD Elements) │
|
||
│ │
|
||
│ Csatlakoztatási név: [hdd_1 ] │
|
||
│ (A meghajtó a /mnt/hdd_1 útvonalra kerül) │
|
||
│ │
|
||
│ Megnevezés: [Külső HDD 1TB ] │
|
||
│ │
|
||
│ ☑ Beállítás alapértelmezett adattárolóként │
|
||
│ │
|
||
│ ⚠️ FIGYELEM: A meghajtó ÖSSZES adata │
|
||
│ törlődik! Ez a művelet NEM vonható vissza. │
|
||
│ │
|
||
│ A folytatáshoz írja be: FORMÁZÁS │
|
||
│ [ ] │
|
||
│ │
|
||
│ [Inicializálás indítása] [Mégsem] │
|
||
└─────────────────────────────────────────────┘
|
||
```
|
||
|
||
**Step 3: Progress (live updates via polling)**
|
||
```
|
||
┌─────────────────────────────────────────────┐
|
||
│ Meghajtó inicializálás folyamatban... │
|
||
│ │
|
||
│ ✅ Partíció létrehozása │
|
||
│ ✅ Fájlrendszer formázása (ext4) │
|
||
│ ⏳ Csatlakoztatás (/mnt/hdd_1)... │
|
||
│ ○ Mappák létrehozása │
|
||
│ ○ Regisztráció │
|
||
│ │
|
||
│ ████████████████░░░░░░░░ 60% │
|
||
└─────────────────────────────────────────────┘
|
||
```
|
||
|
||
**Step 4: Done**
|
||
```
|
||
┌─────────────────────────────────────────────┐
|
||
│ ✅ Meghajtó sikeresen inicializálva! │
|
||
│ │
|
||
│ Útvonal: /mnt/hdd_1 │
|
||
│ Méret: 931.5 GB │
|
||
│ Fájlrendszer: ext4 │
|
||
│ Állapot: Alapértelmezett adattároló │
|
||
│ │
|
||
│ [Vissza a Beállításokhoz] │
|
||
└─────────────────────────────────────────────┘
|
||
```
|
||
|
||
### 1.6 Routes
|
||
|
||
| Method | Path | Auth? | Description |
|
||
|--------|------|-------|-------------|
|
||
| GET | `/settings/storage/init` | Yes | Storage init wizard page |
|
||
| POST | `/api/storage/scan` | Yes | Scan for available disks (JSON response) |
|
||
| POST | `/api/storage/init` | Yes | Start format+mount (returns job ID) |
|
||
| GET | `/api/storage/init/status` | Yes | Poll format progress (JSON) |
|
||
|
||
### 1.7 Safety guardrails
|
||
|
||
1. **Never touch system disk** — compare block device of target with block device of `/`. If same parent disk, hard reject.
|
||
2. **Confirmation typing** — user must type "FORMÁZÁS" (Hungarian for "FORMAT") before proceeding. Not a simple checkbox.
|
||
3. **Pre-flight checks:**
|
||
- Device still exists when init starts (not unplugged between scan and confirm)
|
||
- Device still not mounted (not mounted by something else in between)
|
||
- Mount path doesn't conflict with existing registered storage paths
|
||
- No duplicate mount names
|
||
4. **fstab safety:**
|
||
- Backup fstab before editing
|
||
- Use UUID-based mount (not device path — device names can change)
|
||
- Use `nofail` option (system boots even if drive missing)
|
||
- Use `noatime` option (reduces write wear)
|
||
5. **Container requirements:** The controller container needs:
|
||
- `/dev:/dev:ro` mount (to see block devices) — actually needs `:rw` for mkfs
|
||
- `/mnt:/mnt:rw` (already have this from v0.9.0)
|
||
- `--privileged` or specific `--cap-add` for disk operations
|
||
- Access to host's `/etc/fstab` — mount as `/host-fstab:/etc/fstab:rw` or similar
|
||
|
||
### 1.8 Container privilege escalation
|
||
|
||
**This is the biggest architectural question.** Currently the controller runs as a regular container with Docker socket and `/mnt:/mnt:rw`. Disk formatting requires:
|
||
- Access to `/dev/sdb*` (block devices)
|
||
- Permission to run `mkfs.ext4`, `sfdisk`, `mount`, `blkid`
|
||
- Write access to host's `/etc/fstab`
|
||
|
||
**Options:**
|
||
|
||
**Option A: `--privileged` controller container**
|
||
- Simplest. Full access to everything.
|
||
- Already has Docker socket (full host control anyway).
|
||
- Risk: Broader attack surface if container is compromised.
|
||
- For a managed home server where the controller IS the management layer, this is pragmatic.
|
||
|
||
**Option B: Specific capabilities**
|
||
- `--cap-add SYS_ADMIN --cap-add MKNOD`
|
||
- Mount `/dev:/dev:rw`, host's `/etc/fstab`
|
||
- More surgical but still significant privilege.
|
||
|
||
**Option C: Helper script on host**
|
||
- Controller calls a script on the host via Docker exec into a sidecar or host process.
|
||
- Complex, fragile. Not recommended.
|
||
|
||
**Recommended: Option A (`--privileged`)** — the controller already has Docker socket which is equivalent to root anyway. Adding `--privileged` just makes the disk operations possible without workaround complexity. Update the controller's docker-compose.yml.
|
||
|
||
### 1.9 Required container compose changes
|
||
|
||
```yaml
|
||
services:
|
||
felhom-controller:
|
||
# ... existing ...
|
||
privileged: true # NEW: for disk operations
|
||
volumes:
|
||
- /var/run/docker.sock:/var/run/docker.sock
|
||
- /opt/docker:/opt/docker
|
||
- /mnt:/mnt:rw
|
||
- /dev:/dev # NEW: block device access
|
||
- /etc/fstab:/host-fstab # NEW: fstab read/write
|
||
- /run/udev:/run/udev:ro # NEW: for blkid/lsblk metadata
|
||
```
|
||
|
||
Note: Mount host fstab at `/host-fstab` inside the container to avoid shadowing the container's own `/etc/fstab`. The Go code reads/writes `/host-fstab`.
|
||
|
||
### 1.10 Prerequisite packages in controller Docker image
|
||
|
||
The controller's Dockerfile needs: `lsblk`, `blkid`, `mkfs.ext4`, `sfdisk`, `mount`, `partprobe`
|
||
|
||
These come from: `util-linux`, `e2fsprogs`, `parted` (or `fdisk`). Add to the Dockerfile's `apt-get install` if not already present.
|
||
|
||
Alternatively, the controller can shell out to the HOST's tools via `nsenter` if running with `--pid=host`. This avoids adding packages to the container image. But `--privileged` + installing the tools is simpler.
|
||
|
||
### 1.11 After successful init: auto-register
|
||
|
||
After format+mount succeeds:
|
||
```go
|
||
sett.AddStoragePath(settings.StoragePath{
|
||
Path: "/mnt/" + req.MountName,
|
||
Label: req.Label,
|
||
IsDefault: req.SetDefault,
|
||
Schedulable: true,
|
||
AddedAt: time.Now().UTC().Format(time.RFC3339),
|
||
})
|
||
```
|
||
|
||
This makes the new drive immediately available for app deployment.
|
||
|
||
---
|
||
|
||
## 2. Data Migration Feature
|
||
|
||
### 2.1 Concept
|
||
|
||
Per-app "Mozgatás" button to move an app's user data from one storage path to another. This is essential when:
|
||
- Adding a larger drive and wanting to move photos/documents to it
|
||
- A drive is filling up
|
||
- Reorganizing storage
|
||
|
||
### 2.2 Migration flow
|
||
|
||
```
|
||
User clicks "Mozgatás" on app detail or settings page
|
||
↓
|
||
Select target storage path from dropdown
|
||
↓
|
||
Pre-flight: check free space, estimate size, show warnings
|
||
↓
|
||
Confirm (app will be stopped during migration)
|
||
↓
|
||
Stop app containers
|
||
↓
|
||
rsync data (with progress reporting)
|
||
↓
|
||
Update app.yaml HDD_PATH to new path
|
||
↓
|
||
Start app containers
|
||
↓
|
||
Success: old data kept as safety net (user can delete later)
|
||
```
|
||
|
||
### 2.3 UI: Migration dialog
|
||
|
||
Accessible from two places:
|
||
- **Settings page**: per-app row in storage path details → "Mozgatás" button
|
||
- **App detail page** (deploy page in read-only mode): storage info section → "Mozgatás" button
|
||
|
||
Dialog/page:
|
||
|
||
```
|
||
┌─────────────────────────────────────────────────┐
|
||
│ Immich adatok áthelyezése │
|
||
│ │
|
||
│ Jelenlegi tárhely: Külső tárhely (hdd_placeholder) │
|
||
│ Adatméret: 92 MB (/mnt/hdd_placeholder/storage/immich) │
|
||
│ │
|
||
│ Cél tárhely: │
|
||
│ [▼ Külső HDD 1TB (/mnt/hdd_1) — 800 GB szabad] │
|
||
│ │
|
||
│ ⚠️ Figyelmeztetések: │
|
||
│ • Az alkalmazás a mozgatás idejére leáll │
|
||
│ • Nagy adatmennyiségnél ez percekig tarthat │
|
||
│ • A régi adatok megmaradnak biztonsági másolatként │
|
||
│ │
|
||
│ [Mozgatás indítása] [Mégsem] │
|
||
└─────────────────────────────────────────────────┘
|
||
```
|
||
|
||
Progress:
|
||
```
|
||
┌─────────────────────────────────────────────────┐
|
||
│ Adatok áthelyezése... │
|
||
│ │
|
||
│ ✅ Alkalmazás leállítva │
|
||
│ ⏳ Adatok másolása... 45 MB / 92 MB │
|
||
│ ○ Konfiguráció frissítése │
|
||
│ ○ Alkalmazás indítása │
|
||
│ │
|
||
│ ████████████████░░░░░░░░ 49% │
|
||
│ ~30 másodperc hátravan │
|
||
└─────────────────────────────────────────────────┘
|
||
```
|
||
|
||
### 2.4 Implementation: `internal/storage/migrate.go`
|
||
|
||
```go
|
||
// MigrateRequest parameters.
|
||
type MigrateRequest struct {
|
||
StackName string // e.g., "immich"
|
||
TargetPath string // e.g., "/mnt/hdd_1"
|
||
}
|
||
|
||
// MigrateProgress tracks migration state.
|
||
type MigrateProgress struct {
|
||
Step string // "stopping", "copying", "updating", "starting", "done", "error", "rolling_back"
|
||
Message string
|
||
BytesCopied int64
|
||
BytesTotal int64
|
||
Percent int
|
||
Error string
|
||
ElapsedSeconds int
|
||
}
|
||
|
||
// MigrateAppData moves an app's data from current to target storage path.
|
||
func MigrateAppData(req MigrateRequest, stackMgr *stacks.Manager, sett *settings.Settings, progress chan<- MigrateProgress) error {
|
||
// 1. Load app config, get current HDD_PATH
|
||
// 2. Parse compose to find HDD mount mappings (e.g., ${HDD_PATH}/storage/immich)
|
||
// 3. Validate target path: exists, mounted, sufficient space, different from current
|
||
// 4. Stop app: docker compose -f ... stop
|
||
// 5. rsync each HDD mount:
|
||
// rsync -avP /mnt/hdd_placeholder/storage/immich/ /mnt/hdd_1/storage/immich/
|
||
// Parse rsync output for progress (--info=progress2)
|
||
// 6. Update app.yaml: change HDD_PATH to target path
|
||
// 7. Start app: docker compose -f ... up -d
|
||
// 8. Verify containers running
|
||
// 9. Log migration: "Moved immich from /mnt/hdd_placeholder to /mnt/hdd_1"
|
||
// 10. Keep old data (don't delete — user cleanup later)
|
||
}
|
||
```
|
||
|
||
### 2.5 rsync progress parsing
|
||
|
||
Use `rsync --info=progress2` which outputs lines like:
|
||
```
|
||
45,678,901 49% 12.34MB/s 0:00:30
|
||
```
|
||
|
||
Parse with regex:
|
||
```go
|
||
progressRe := regexp.MustCompile(`([\d,]+)\s+(\d+)%\s+([\d.]+\w+/s)\s+(\d+:\d+:\d+)`)
|
||
```
|
||
|
||
Feed into `MigrateProgress.Percent` and `MigrateProgress.BytesCopied`.
|
||
|
||
### 2.6 Rollback on failure
|
||
|
||
If rsync fails or app doesn't start after migration:
|
||
1. Revert `app.yaml` HDD_PATH to original value
|
||
2. Start app with original config
|
||
3. Log error but keep partially copied data (don't delete)
|
||
4. Show error to user: "Áthelyezés sikertelen. Az alkalmazás az eredeti tárolóról fut. A részlegesen másolt adatok: /mnt/hdd_1/storage/immich"
|
||
|
||
### 2.7 Routes
|
||
|
||
| Method | Path | Auth? | Description |
|
||
|--------|------|-------|-------------|
|
||
| GET | `/stacks/{name}/migrate` | Yes | Migration page for app |
|
||
| POST | `/api/storage/migrate` | Yes | Start migration (returns job ID) |
|
||
| GET | `/api/storage/migrate/status` | Yes | Poll migration progress (JSON) |
|
||
|
||
### 2.8 Mutex / concurrency
|
||
|
||
Only one migration or format operation at a time. Use a global mutex:
|
||
|
||
```go
|
||
var diskOpMutex sync.Mutex
|
||
var diskOpActive atomic.Bool
|
||
```
|
||
|
||
Check `diskOpActive` before starting. Return error if another operation is running. Show "Egy másik lemezművelet folyamatban van" message.
|
||
|
||
### 2.9 Old data cleanup
|
||
|
||
After successful migration, the old data remains. Options for the user:
|
||
- **Manual:** User can SSH in and delete, or use FileBrowser
|
||
- **UI button (future):** "Régi adatok törlése" button with confirmation — Phase D
|
||
|
||
For now, just leave old data and show a note on the app detail page: "Korábbi adatok a(z) /mnt/hdd_placeholder/storage/immich útvonalon továbbra is elérhetők."
|
||
|
||
---
|
||
|
||
## 3. Per-App Storage Display Enhancements
|
||
|
||
### 3.1 App detail page (deploy page read-only mode)
|
||
|
||
Currently shows deployed config fields as disabled inputs. Add a storage info section for deployed apps:
|
||
|
||
```html
|
||
{{if and .AlreadyDeployed .StorageInfo}}
|
||
<div class="deploy-storage-info">
|
||
<h4>Adattárolás</h4>
|
||
<div class="settings-grid">
|
||
<div class="settings-row">
|
||
<span class="settings-label">Tárhely</span>
|
||
<span class="settings-value">{{.StorageInfo.Label}} ({{.StorageInfo.Path}})</span>
|
||
</div>
|
||
<div class="settings-row">
|
||
<span class="settings-label">Adatméret</span>
|
||
<span class="settings-value mono">{{.StorageInfo.DataSizeHuman}}</span>
|
||
</div>
|
||
<div class="settings-row">
|
||
<span class="settings-label">Szabad hely</span>
|
||
<span class="settings-value mono">{{.StorageInfo.FreeHuman}} ({{printf "%.0f" .StorageInfo.FreePercent}}%)</span>
|
||
</div>
|
||
</div>
|
||
{{if gt (len $.OtherStoragePaths) 0}}
|
||
<a href="/stacks/{{.Meta.Slug}}/migrate" class="btn btn-sm btn-outline" style="margin-top:0.5rem">
|
||
📦 Mozgatás másik tárolóra
|
||
</a>
|
||
{{end}}
|
||
</div>
|
||
{{end}}
|
||
```
|
||
|
||
### 3.2 Data struct
|
||
|
||
```go
|
||
type DeployStorageInfo struct {
|
||
Path string
|
||
Label string
|
||
DataSizeHuman string
|
||
FreeHuman string
|
||
FreePercent float64
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## 4. Implementation Steps
|
||
|
||
### Step 0: Startup ping + hub report
|
||
- Add startup goroutine in `main.go` after `sched.Start(ctx)`
|
||
- Fire heartbeat ping, system_health ping, and hub report with 5s delay
|
||
- Reuse existing `pinger` and create `pusher` once (not per-task)
|
||
- **Test:** Restart controller → Healthchecks shows fresh ping within 10 seconds. Hub shows fresh report.
|
||
|
||
### Step 1: Storage initialization — backend
|
||
- Create `internal/storage/` package with `BlockDevice`, `Partition`, `ScanResult` structs
|
||
- Implement `ScanDisks()` using `lsblk -J`
|
||
- Implement `FormatAndMount()` with progress channel
|
||
- Implement safety checks: system disk detection, device validation, mount conflict check
|
||
- Implement fstab editing (read from `/host-fstab`, backup, append, write)
|
||
- **Test:** Call `ScanDisks()` → returns sdb as available. FormatAndMount (dry-run or test partition).
|
||
|
||
### Step 2: Storage initialization — container setup
|
||
- Update controller's `docker-compose.yml`:
|
||
- Add `privileged: true`
|
||
- Add `/dev:/dev` volume
|
||
- Add `/etc/fstab:/host-fstab` volume
|
||
- Add `/run/udev:/run/udev:ro` volume
|
||
- Update controller Dockerfile: install `util-linux`, `e2fsprogs` (for `mkfs.ext4`, `lsblk`, `blkid`, `sfdisk`, `mount`)
|
||
- Update `docker-setup.sh` to generate compose with new mounts
|
||
- **Test:** `docker exec felhom-controller lsblk` shows host block devices. `mkfs.ext4 --version` works.
|
||
|
||
### Step 3: Storage initialization — UI
|
||
- Create `/settings/storage/init` page with wizard flow
|
||
- Implement `/api/storage/scan` endpoint (JSON)
|
||
- Implement `/api/storage/init` endpoint (start format, return job ID)
|
||
- Implement `/api/storage/init/status` endpoint (poll progress)
|
||
- JS: scan button → display disks → configure form → confirmation typing → progress polling → done
|
||
- Auto-register new storage path in settings.json on completion
|
||
- **Test:** Full wizard: scan → select sdb → name "hdd_1" → type "FORMÁZÁS" → format → mount → registered. `lsblk` shows `/mnt/hdd_1` mounted.
|
||
|
||
### Step 4: Data migration — backend
|
||
- Create `internal/storage/migrate.go`
|
||
- Implement `MigrateAppData()` with progress channel
|
||
- rsync with `--info=progress2` output parsing
|
||
- Rollback on failure (revert app.yaml, restart with original config)
|
||
- Global mutex for disk operations
|
||
- **Test:** Migrate Paperless-ngx from `/mnt/hdd_placeholder` to `/mnt/hdd_1` → data appears at new location, app restarts.
|
||
|
||
### Step 5: Data migration — UI
|
||
- Create `/stacks/{name}/migrate` page
|
||
- Target storage dropdown (exclude current path)
|
||
- Pre-flight: size estimate, free space check
|
||
- Progress polling via `/api/storage/migrate/status`
|
||
- Add "Mozgatás" button to app detail page (deploy page read-only mode) and settings storage details
|
||
- **Test:** Navigate to Immich detail → "Mozgatás" → select new drive → confirm → progress → done → Immich running from new path.
|
||
|
||
### Step 6: Per-app storage display on deploy page
|
||
- Add `DeployStorageInfo` to deploy page template data for deployed apps
|
||
- Show current storage path, data size, free space
|
||
- Show "Mozgatás" link if other storage paths exist
|
||
- **Test:** Visit deployed Immich page → shows "Adattárolás" section with path, size, free space, migrate button.
|
||
|
||
### Step 7: Version bump & cleanup
|
||
- Update CHANGELOG.md / CONTEXT.md / CLAUDE.md / README.md for controller
|
||
- Bump to 0.11.0
|
||
- Build + deploy
|
||
- **Full test cycle:**
|
||
1. Controller starts → Healthchecks shows ping within 10s ✓
|
||
2. Navigate to settings → "Új meghajtó inicializálása" → scan → sdb visible ✓
|
||
3. Configure hdd_1, type FORMÁZÁS → format + mount succeeds ✓
|
||
4. Settings shows /mnt/hdd_1 as registered storage path ✓
|
||
5. Deploy something (or use existing app) → migrate to /mnt/hdd_1 ✓
|
||
6. App running from new storage path ✓
|
||
|
||
---
|
||
|
||
## 5. Files to Create / Modify
|
||
|
||
### New files:
|
||
- `controller/internal/storage/scan.go` — `ScanDisks()`, `BlockDevice`, `Partition` structs
|
||
- `controller/internal/storage/format.go` — `FormatAndMount()`, `FormatRequest`, `FormatProgress`
|
||
- `controller/internal/storage/migrate.go` — `MigrateAppData()`, `MigrateRequest`, `MigrateProgress`
|
||
- `controller/internal/storage/safety.go` — System disk detection, mount conflict check, fstab helpers
|
||
- `controller/internal/web/templates/storage_init.html` — Storage init wizard page
|
||
- `controller/internal/web/templates/migrate.html` — Migration page
|
||
|
||
### Modified files:
|
||
- `controller/cmd/controller/main.go` — Startup pings + hub report, reuse pusher instance
|
||
- `controller/internal/web/handlers.go` — Storage init handlers, migration handlers, deploy page storage info
|
||
- `controller/internal/web/server.go` — Register new routes
|
||
- `controller/internal/web/templates/deploy.html` — Storage info section for deployed apps, "Mozgatás" link
|
||
- `controller/internal/web/templates/settings.html` — "Új meghajtó inicializálása" link/button, "Mozgatás" buttons per app
|
||
- `controller/internal/web/templates/style.css` — Wizard styles, progress bars, migration UI
|
||
- `controller/docker-compose.yml` — `privileged: true`, new volume mounts
|
||
- `controller/Dockerfile` — Install `util-linux`, `e2fsprogs`
|
||
- `scripts/docker-setup.sh` — Updated compose generation
|
||
|
||
---
|
||
|
||
## 6. Design Decisions
|
||
|
||
### Why Go package instead of wrapping hdd-setup.sh?
|
||
The bash script has interactive prompts, color codes, and reads stdin — none of which work from a web handler. Reimplementing the core operations in Go gives structured errors, progress channels, and clean API integration. The bash script remains useful for manual SSH access.
|
||
|
||
### Why `--privileged` instead of fine-grained capabilities?
|
||
The controller already has Docker socket access (root-equivalent). Adding `--privileged` adds no meaningful additional risk but greatly simplifies disk operations. Fine-grained caps (`SYS_ADMIN`, `MKNOD`, device cgroup rules) are complex to get right and test.
|
||
|
||
### Why keep old data after migration?
|
||
Deleting data is irreversible. The old copy serves as an implicit backup during the transition period. Users can clean up manually when they're confident the migration worked. A future "cleanup" button can be added.
|
||
|
||
### Why rsync instead of cp/mv?
|
||
rsync has progress reporting (`--info=progress2`), handles interruptions gracefully (can resume), preserves permissions/timestamps, and works across filesystems. `mv` can't cross filesystem boundaries (would fall back to copy+delete). `cp -a` works but has no progress output.
|
||
|
||
### Why a separate `/settings/storage/init` page instead of inline?
|
||
The format wizard has multiple steps and safety confirmations that don't fit well inside the existing settings page layout. A dedicated page with clear back-navigation keeps the settings page clean.
|
||
|
||
### Why UUID-based fstab instead of device path?
|
||
Device paths (`/dev/sdb1`) can change between boots (especially with USB devices). UUID-based mounting is stable regardless of device enumeration order. This is standard best practice for Linux.
|
||
|
||
### Why mount under `/mnt/`?
|
||
Standard convention for additional mount points on Linux. The controller already mounts `/mnt:/mnt:rw`. Keeps all storage in a predictable, manageable location.
|
||
|
||
### Future (NOT in scope):
|
||
- **Old data cleanup button** — "Régi adatok törlése" after successful migration
|
||
- **SMART health monitoring** — smartmontools integration
|
||
- **Auto-detect new drives** — udev event watching
|
||
- **Batch migration** — move multiple apps at once
|
||
- **Filesystem check** — `fsck` integration for maintenance |