686 lines
28 KiB
Markdown
686 lines
28 KiB
Markdown
# TASK: Phase A — Storage Paths Foundation & Backup Toggle Fix
|
|
|
|
**Version target:** controller 0.9.0
|
|
**Repos:** `deploy-felhom-compose` (controller), `app-catalog-felhom.eu` (compose mount change)
|
|
|
|
## Overview
|
|
|
|
Per-app backup toggles (implemented in v0.8.0) don't appear on the backup page. Root cause is two-layered:
|
|
|
|
1. **`controller.yaml` has no `paths.hdd_path`** → `m.cfg.Paths.HDDPath` is `""` → `ParseComposeHDDMounts` returns nil → `DiscoverAppData` finds zero HDD data → no backup toggles shown
|
|
2. **Even with a global hdd_path, the parser uses ONE path** → apps deployed with a different `HDD_PATH` (e.g., `/mnt/hdd_placeholder` vs `/mnt/hdd_1`) won't match
|
|
|
|
The fix: stop relying on a single global HDD path. Instead, read each app's **own** `HDD_PATH` from its `app.yaml` env section, and introduce a storage paths registry in `settings.json` that auto-discovers paths from deployed apps.
|
|
|
|
This phase also lays the foundation for multi-storage management (Phase B: UI polish, Phase C: migration wizard).
|
|
|
|
---
|
|
|
|
## 1. Root Cause Analysis
|
|
|
|
### Current broken flow
|
|
|
|
```
|
|
controller.yaml → paths.hdd_path = "" (missing)
|
|
↓
|
|
backup.Manager.stackProvider.GetStackHDDMounts(name)
|
|
→ stacks.ParseComposeHDDMounts(composePath, hddPath="")
|
|
→ hddPath == "" → return nil ← BUG: exits early
|
|
↓
|
|
DiscoverAppData → info.HasHDDData = false for ALL apps
|
|
↓
|
|
backups.html → {{if .HasHDDData}} → false → no checkbox rendered
|
|
```
|
|
|
|
### Second layer (would hit even with hdd_path set)
|
|
|
|
```
|
|
controller.yaml → paths.hdd_path = "/mnt/hdd_1"
|
|
↓
|
|
ParseComposeHDDMounts(composePath, "/mnt/hdd_1")
|
|
→ reads compose: "- ${HDD_PATH}/storage/immich:/usr/src/app/upload"
|
|
→ resolves ${HDD_PATH} → "/mnt/hdd_1/storage/immich"
|
|
→ BUT app.yaml has HDD_PATH="/mnt/hdd_placeholder"
|
|
→ actual data is at /mnt/hdd_placeholder/storage/immich
|
|
→ os.Stat("/mnt/hdd_1/storage/immich") → not found
|
|
→ path.Exists = false, SizeBytes = 0
|
|
```
|
|
|
|
### Correct approach
|
|
|
|
Read each app's actual `HDD_PATH` from its `app.yaml` env section. The compose template uses `${HDD_PATH}` which gets resolved at `docker compose up` time from the environment. We need to resolve it the same way during discovery.
|
|
|
|
---
|
|
|
|
## 2. Storage Paths Registry
|
|
|
|
### 2.1 New struct in `settings.go`
|
|
|
|
```go
|
|
// StoragePath represents a registered external storage location.
|
|
type StoragePath struct {
|
|
Path string `json:"path"` // e.g., "/mnt/hdd_1"
|
|
Label string `json:"label,omitempty"` // e.g., "Külső HDD 1TB"
|
|
IsDefault bool `json:"is_default,omitempty"` // new apps use this by default
|
|
Schedulable bool `json:"schedulable"` // whether new apps can be deployed here
|
|
AddedAt string `json:"added_at"` // RFC3339
|
|
}
|
|
```
|
|
|
|
### 2.2 Add to Settings struct
|
|
|
|
```go
|
|
type Settings struct {
|
|
// ... existing fields ...
|
|
|
|
// Storage paths registry
|
|
StoragePaths []StoragePath `json:"storage_paths,omitempty"`
|
|
}
|
|
```
|
|
|
|
### 2.3 Helper methods on Settings
|
|
|
|
```go
|
|
// GetStoragePaths returns all registered storage paths.
|
|
func (s *Settings) GetStoragePaths() []StoragePath
|
|
|
|
// GetDefaultStoragePath returns the default storage path, or "" if none.
|
|
func (s *Settings) GetDefaultStoragePath() string
|
|
|
|
// GetSchedulableStoragePaths returns paths where new apps can be deployed.
|
|
func (s *Settings) GetSchedulableStoragePaths() []StoragePath
|
|
|
|
// AddStoragePath registers a new storage path after validation.
|
|
func (s *Settings) AddStoragePath(path, label string, isDefault bool) error
|
|
|
|
// RemoveStoragePath removes a path only if no apps reference it.
|
|
// Returns error with list of apps using this path if removal is blocked.
|
|
func (s *Settings) RemoveStoragePath(path string, appChecker func(string) []string) error
|
|
|
|
// SetDefaultStoragePath changes which path is the default.
|
|
func (s *Settings) SetDefaultStoragePath(path string) error
|
|
|
|
// SetSchedulable enables/disables a path for new deployments.
|
|
func (s *Settings) SetSchedulable(path string, schedulable bool) error
|
|
```
|
|
|
|
### 2.4 Validation rules (in AddStoragePath)
|
|
|
|
1. **Must be a real mount point** — compare device ID of path vs parent using `syscall.Stat`. Reject paths that are just directories on the boot SSD. Error message: "Ez az útvonal nem külön csatlakoztatott meghajtó. Adatok az SSD-re kerülnének."
|
|
2. **Must exist and be a directory** — `os.Stat` check
|
|
3. **Must be writable** — attempt to create + remove a temp file
|
|
4. **No overlapping paths** — reject if new path is a parent or child of an existing path (e.g., `/mnt/hdd_1` and `/mnt/hdd_1/subdir`)
|
|
5. **No duplicates** — reject if path already registered (normalized with `filepath.Clean`)
|
|
6. **Must be under `/mnt/`** — soft warning, not hard block (log a WARN for edge cases)
|
|
|
|
### 2.5 Auto-discovery on startup
|
|
|
|
On controller startup, if `StoragePaths` is empty in settings.json, auto-discover from deployed apps:
|
|
|
|
```go
|
|
func (s *Settings) AutoDiscoverStoragePaths(stacksDir string, fallbackHDDPath string, logger *log.Logger) {
|
|
if len(s.StoragePaths) > 0 {
|
|
return // already configured, don't override
|
|
}
|
|
|
|
// Scan all deployed apps' app.yaml for HDD_PATH values
|
|
seen := map[string]bool{}
|
|
entries, _ := os.ReadDir(stacksDir)
|
|
for _, e := range entries {
|
|
if !e.IsDir() { continue }
|
|
appCfg := LoadAppConfig(filepath.Join(stacksDir, e.Name()))
|
|
if appCfg == nil || !appCfg.Deployed { continue }
|
|
if hddPath, ok := appCfg.Env["HDD_PATH"]; ok && hddPath != "" {
|
|
cleaned := filepath.Clean(hddPath)
|
|
if !seen[cleaned] {
|
|
seen[cleaned] = true
|
|
}
|
|
}
|
|
}
|
|
|
|
// Also use controller.yaml paths.hdd_path as fallback seed
|
|
if fallbackHDDPath != "" {
|
|
cleaned := filepath.Clean(fallbackHDDPath)
|
|
if !seen[cleaned] {
|
|
seen[cleaned] = true
|
|
}
|
|
}
|
|
|
|
// Register discovered paths
|
|
isFirst := true
|
|
for path := range seen {
|
|
sp := StoragePath{
|
|
Path: path,
|
|
Label: inferStorageLabel(path),
|
|
IsDefault: isFirst,
|
|
Schedulable: true,
|
|
AddedAt: time.Now().UTC().Format(time.RFC3339),
|
|
}
|
|
s.StoragePaths = append(s.StoragePaths, sp)
|
|
isFirst = false
|
|
}
|
|
|
|
if len(s.StoragePaths) > 0 {
|
|
s.Save()
|
|
logger.Printf("[INFO] Auto-discovered %d storage path(s)", len(s.StoragePaths))
|
|
for _, sp := range s.StoragePaths {
|
|
logger.Printf("[INFO] %s (%s) default=%v", sp.Path, sp.Label, sp.IsDefault)
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
Helper `inferStorageLabel(path)`:
|
|
- `/mnt/hdd_1` → "Külső tárhely (hdd_1)"
|
|
- `/mnt/hdd_placeholder` → "Külső tárhely (hdd_placeholder)"
|
|
- Anything else → path basename
|
|
|
|
Note: `LoadAppConfig` is in the `stacks` package. To avoid circular imports, either pass a scanner function or do the scanning in `main.go` and pass the discovered paths in. Use whichever approach avoids import issues.
|
|
|
|
---
|
|
|
|
## 3. Fix `GetStackHDDMounts` — Per-App HDD_PATH Resolution
|
|
|
|
### 3.1 Change the adapter
|
|
|
|
Currently:
|
|
```go
|
|
func (a *stackAdapter) GetStackHDDMounts(name string) []string {
|
|
s, ok := a.mgr.GetStack(name)
|
|
if !ok { return nil }
|
|
return stacks.ParseComposeHDDMounts(s.ComposePath, a.hddPath) // ← uses global path
|
|
}
|
|
```
|
|
|
|
Change to:
|
|
```go
|
|
func (a *stackAdapter) GetStackHDDMounts(name string) []string {
|
|
s, ok := a.mgr.GetStack(name)
|
|
if !ok { return nil }
|
|
|
|
// Priority 1: Read the app's own HDD_PATH from its app.yaml
|
|
stackDir := filepath.Dir(s.ComposePath)
|
|
appCfg := stacks.LoadAppConfig(stackDir)
|
|
if appCfg != nil && appCfg.Env["HDD_PATH"] != "" {
|
|
return stacks.ParseComposeHDDMounts(s.ComposePath, appCfg.Env["HDD_PATH"])
|
|
}
|
|
|
|
// Priority 2: Try all registered storage paths (fallback)
|
|
var allMounts []string
|
|
seen := map[string]bool{}
|
|
for _, sp := range a.getStoragePaths() {
|
|
mounts := stacks.ParseComposeHDDMounts(s.ComposePath, sp.Path)
|
|
for _, m := range mounts {
|
|
if !seen[m] {
|
|
seen[m] = true
|
|
allMounts = append(allMounts, m)
|
|
}
|
|
}
|
|
}
|
|
return allMounts
|
|
}
|
|
```
|
|
|
|
### 3.2 Update adapter struct
|
|
|
|
```go
|
|
type stackAdapter struct {
|
|
mgr *stacks.Manager
|
|
getStoragePaths func() []settings.StoragePath // getter from settings
|
|
}
|
|
```
|
|
|
|
Wire in `main.go`:
|
|
```go
|
|
adapter := &stackAdapter{
|
|
mgr: stackMgr,
|
|
getStoragePaths: func() []settings.StoragePath { return settingsMgr.GetStoragePaths() },
|
|
}
|
|
```
|
|
|
|
Remove the old `hddPath string` field from the adapter.
|
|
|
|
### 3.3 `resolveAppBackupPaths()` in backup Manager
|
|
|
|
No changes needed — it already calls `m.stackProvider.GetStackHDDMounts(stackName)` which we're fixing above. The fix propagates automatically.
|
|
|
|
---
|
|
|
|
## 4. Controller Docker Compose Mount Change
|
|
|
|
### 4.1 Change in `controller/docker-compose.yml`
|
|
|
|
Replace:
|
|
```yaml
|
|
- ${HDD_PATH:-/mnt/hdd_placeholder}:${HDD_PATH:-/mnt/hdd_placeholder}:ro
|
|
```
|
|
|
|
With:
|
|
```yaml
|
|
# All external storage — covers /mnt/* paths for multi-storage support + restore
|
|
- /mnt:/mnt:rw
|
|
```
|
|
|
|
**Why `:rw`:** Required for restore feature (already implemented). Also needed for mount-point validation (write test in AddStoragePath).
|
|
|
|
**Why `/mnt:/mnt`:** All standard storage mounts are under `/mnt`. Covers current and future drives with one entry. No controller restart when adding new drives.
|
|
|
|
### 4.2 Also update `docker-setup.sh`
|
|
|
|
If `docker-setup.sh` generates the controller compose, update it to emit `/mnt:/mnt:rw` instead of the old `${HDD_PATH}` line.
|
|
|
|
---
|
|
|
|
## 5. Deploy Page: Storage Path Dropdown
|
|
|
|
### 5.1 When app has a `path` type deploy field (HDD_PATH)
|
|
|
|
Currently renders as a plain text input. Change to dropdown when storage paths exist:
|
|
|
|
```html
|
|
{{if eq .Type "path"}}
|
|
{{if $.StoragePaths}}
|
|
<select id="field-{{.EnvVar}}" name="{{.EnvVar}}" class="form-control"
|
|
{{if $.AlreadyDeployed}}disabled{{end}}>
|
|
{{range $.StoragePaths}}
|
|
<option value="{{.Path}}" {{if .IsDefault}}selected{{end}}>
|
|
{{.Label}} ({{.Path}}) {{if .IsDefault}}— alapértelmezett{{end}}
|
|
</option>
|
|
{{end}}
|
|
</select>
|
|
{{else}}
|
|
<input type="text" id="field-{{.EnvVar}}" name="{{.EnvVar}}"
|
|
class="form-control" value="{{.Default}}"
|
|
placeholder="{{.Placeholder}}"
|
|
{{if .Required}}required{{end}}
|
|
{{if $.AlreadyDeployed}}disabled{{end}}>
|
|
<span class="form-hint form-hint-warn">
|
|
Nincs regisztrált adattároló. Adjon hozzá egyet a Beállítások oldalon.
|
|
</span>
|
|
{{end}}
|
|
{{end}}
|
|
```
|
|
|
|
### 5.2 Pass storage paths to deploy page template data
|
|
|
|
In the deploy handler:
|
|
```go
|
|
data["StoragePaths"] = settingsMgr.GetSchedulableStoragePaths()
|
|
```
|
|
|
|
### 5.3 Already-deployed: show current path read-only
|
|
|
|
When `AlreadyDeployed == true`, the field is disabled. Show the actual `HDD_PATH` from `app.yaml` as the displayed value (existing behavior works as-is with a disabled select or text input).
|
|
|
|
### 5.4 Edge case: no schedulable paths but app needs HDD
|
|
|
|
If `resources.needs_hdd: true` and zero schedulable paths:
|
|
- Show warning: "Nincs elérhető adattároló. Csatlakoztasson külső meghajtót és adja hozzá a Beállítások oldalon."
|
|
- Disable deploy button (add `data-needs-storage="true"` to form, JS disables submit)
|
|
|
|
---
|
|
|
|
## 6. Storage Path Monitoring Integration
|
|
|
|
### 6.1 Periodic mount-point check
|
|
|
|
Add to existing system health check (runs every `system_health_interval`, default 5m):
|
|
|
|
```go
|
|
func checkStoragePaths(paths []settings.StoragePath, logger *log.Logger) []string {
|
|
var warnings []string
|
|
for _, sp := range paths {
|
|
// Check 1: path exists and is directory
|
|
fi, err := os.Stat(sp.Path)
|
|
if err != nil {
|
|
warnings = append(warnings, fmt.Sprintf("Adattároló nem elérhető: %s", sp.Path))
|
|
continue
|
|
}
|
|
if !fi.IsDir() {
|
|
warnings = append(warnings, fmt.Sprintf("Adattároló nem mappa: %s", sp.Path))
|
|
continue
|
|
}
|
|
|
|
// Check 2: still a real mount point (not writing to SSD)
|
|
if !isMountPoint(sp.Path) {
|
|
warnings = append(warnings,
|
|
fmt.Sprintf("FIGYELEM: %s nincs felcsatolva (leválasztva?) — "+
|
|
"az adatok az SSD-re íródnának!", sp.Path))
|
|
}
|
|
|
|
// Check 3: disk space (reuse existing threshold config)
|
|
usage := getDiskUsage(sp.Path)
|
|
if usage.UsedPercent >= 90 {
|
|
warnings = append(warnings,
|
|
fmt.Sprintf("Adattároló majdnem tele: %s (%.0f%%)", sp.Path, usage.UsedPercent))
|
|
}
|
|
}
|
|
return warnings
|
|
}
|
|
```
|
|
|
|
### 6.2 `isMountPoint()` implementation
|
|
|
|
New file: `controller/internal/system/mounts.go`
|
|
|
|
```go
|
|
// IsMountPoint checks if a path is a mount point by comparing device IDs.
|
|
// Returns true if path is on a different device than its parent.
|
|
func IsMountPoint(path string) bool {
|
|
var pathStat, parentStat syscall.Stat_t
|
|
if err := syscall.Stat(path, &pathStat); err != nil {
|
|
return false
|
|
}
|
|
parent := filepath.Dir(path)
|
|
if err := syscall.Stat(parent, &parentStat); err != nil {
|
|
return false
|
|
}
|
|
return pathStat.Dev != parentStat.Dev
|
|
}
|
|
```
|
|
|
|
Platform-specific: needs build tag `//go:build linux` (consistent with existing system package). Add stub for non-Linux that always returns true.
|
|
|
|
### 6.3 Surface warnings
|
|
|
|
- Add storage warnings to monitoring page warnings section (existing infrastructure)
|
|
- Include in hub report `SystemWarnings` field (already exists)
|
|
- Fire `disk_warning` notification event for enabled customers
|
|
|
|
---
|
|
|
|
## 7. Beállítások Page: Storage Section
|
|
|
|
Add between "Rendszer konfiguráció" and "Jelszó módosítás" sections.
|
|
|
|
### 7.1 Template: "Adattárolók" section
|
|
|
|
```html
|
|
<!-- Section: Storage Paths -->
|
|
<div class="settings-card">
|
|
<h3>Adattárolók</h3>
|
|
<p class="settings-card-desc">
|
|
Külső meghajtók, ahol az alkalmazások felhasználói adatai tárolódnak.
|
|
</p>
|
|
|
|
{{if .StorageError}}<div class="alert alert-error">{{.StorageError}}</div>{{end}}
|
|
{{if .StorageSuccess}}<div class="alert alert-info">{{.StorageSuccess}}</div>{{end}}
|
|
|
|
{{if .StoragePaths}}
|
|
<div class="storage-paths-list">
|
|
{{range .StoragePaths}}
|
|
<div class="storage-path-item {{if not .Schedulable}}storage-path-disabled{{end}}">
|
|
<div class="storage-path-header">
|
|
<div class="storage-path-info">
|
|
<span class="storage-path-label">{{.Label}}</span>
|
|
<span class="storage-path-path mono">{{.Path}}</span>
|
|
</div>
|
|
<div class="storage-path-badges">
|
|
{{if .IsDefault}}<span class="meta-badge meta-badge-ok">alapértelmezett</span>{{end}}
|
|
{{if not .Schedulable}}<span class="meta-badge meta-badge-warn">nem ütemezhet</span>{{end}}
|
|
{{if .DiskInfo}}
|
|
<span class="mono storage-path-usage">
|
|
{{.DiskInfo.UsedHuman}} / {{.DiskInfo.TotalHuman}} ({{.DiskInfo.UsedPercent}}%)
|
|
</span>
|
|
{{end}}
|
|
</div>
|
|
</div>
|
|
{{if .AppCount}}
|
|
<div class="storage-path-apps">
|
|
{{.AppCount}} alkalmazás használja
|
|
</div>
|
|
{{end}}
|
|
<div class="storage-path-actions">
|
|
{{if not .IsDefault}}
|
|
<form method="POST" action="/settings/storage/default" style="display:inline">
|
|
<input type="hidden" name="storage_path" value="{{.Path}}">
|
|
<button type="submit" class="btn btn-xs btn-outline">Alapértelmezetté tétel</button>
|
|
</form>
|
|
{{end}}
|
|
<form method="POST" action="/settings/storage/schedulable" style="display:inline">
|
|
<input type="hidden" name="storage_path" value="{{.Path}}">
|
|
<input type="hidden" name="schedulable" value="{{if .Schedulable}}false{{else}}true{{end}}">
|
|
<button type="submit" class="btn btn-xs btn-outline">
|
|
{{if .Schedulable}}Letiltás{{else}}Engedélyezés{{end}}
|
|
</button>
|
|
</form>
|
|
{{if and (not .IsDefault) (eq .AppCount 0)}}
|
|
<form method="POST" action="/settings/storage/remove" style="display:inline"
|
|
onsubmit="return confirm('Biztosan törli a(z) {{.Path}} adattárolót?')">
|
|
<input type="hidden" name="storage_path" value="{{.Path}}">
|
|
<button type="submit" class="btn btn-xs btn-danger-outline">Törlés</button>
|
|
</form>
|
|
{{end}}
|
|
</div>
|
|
</div>
|
|
{{end}}
|
|
</div>
|
|
{{else}}
|
|
<div class="alert alert-info">
|
|
Nincs regisztrált adattároló. Csatlakoztasson külső meghajtót és adja hozzá alább.
|
|
</div>
|
|
{{end}}
|
|
|
|
<!-- Add new path -->
|
|
<details class="storage-add-section">
|
|
<summary class="btn btn-sm btn-outline" style="margin-top:1rem">+ Új adattároló hozzáadása</summary>
|
|
<form method="POST" action="/settings/storage/add" class="storage-add-form">
|
|
<div class="form-group">
|
|
<label for="storage_path">Útvonal</label>
|
|
<input type="text" id="storage_path" name="storage_path"
|
|
class="form-control" placeholder="/mnt/hdd_1" required>
|
|
<span class="form-hint">A felcsatolt meghajtó elérési útja (pl. /mnt/hdd_1)</span>
|
|
</div>
|
|
<div class="form-group">
|
|
<label for="storage_label">Megnevezés (opcionális)</label>
|
|
<input type="text" id="storage_label" name="storage_label"
|
|
class="form-control" placeholder="Külső HDD 1TB">
|
|
</div>
|
|
<div class="form-group">
|
|
<label class="toggle">
|
|
<input type="checkbox" name="storage_default" value="true">
|
|
<span class="toggle-label">Beállítás alapértelmezettként</span>
|
|
</label>
|
|
</div>
|
|
<button type="submit" class="btn btn-primary">Hozzáadás</button>
|
|
</form>
|
|
</details>
|
|
</div>
|
|
```
|
|
|
|
### 7.2 Routes
|
|
|
|
| Method | Path | Auth? | Handler |
|
|
|--------|------|-------|---------|
|
|
| POST | `/settings/storage/add` | Yes | Add storage path with validation |
|
|
| POST | `/settings/storage/remove` | Yes | Remove (blocked if apps use it) |
|
|
| POST | `/settings/storage/default` | Yes | Set default storage path |
|
|
| POST | `/settings/storage/schedulable` | Yes | Toggle schedulable on/off |
|
|
|
|
### 7.3 Handler: `/settings/storage/add`
|
|
|
|
1. Parse form: `storage_path`, `storage_label`, `storage_default`
|
|
2. Clean path with `filepath.Clean`
|
|
3. Validate (Section 2.4):
|
|
- Exists + is directory → error: "Az útvonal nem létezik vagy nem mappa."
|
|
- Is mount point → error: "Ez az útvonal nem külön csatlakoztatott meghajtó. Adatok az SSD-re kerülnének!"
|
|
- Writable → error: "Az útvonal nem írható."
|
|
- No overlap → error: "Az útvonal átfedi a már regisztrált XYZ útvonalat."
|
|
- No duplicate → error: "Ez az útvonal már regisztrálva van."
|
|
4. If `storage_default`, unset previous default
|
|
5. Add to settings.json via `AddStoragePath()`
|
|
6. Redirect to `/settings` with flash: "Adattároló sikeresen hozzáadva: /mnt/hdd_1"
|
|
7. On error: redirect with error flash
|
|
|
|
### 7.4 Handler: `/settings/storage/remove`
|
|
|
|
1. Parse form: `storage_path`
|
|
2. Count apps using this path (scan app.yaml files for `HDD_PATH`)
|
|
3. If apps found → error: "Nem törölhető: az alábbi alkalmazások használják: Immich, Paperless-ngx"
|
|
4. Cannot be the default path → error: "Az alapértelmezett adattároló nem törölhető."
|
|
5. If only one path left → error: "Az utolsó adattároló nem törölhető."
|
|
6. Remove from settings.json
|
|
7. Redirect with success flash
|
|
|
|
### 7.5 Template data for settings page
|
|
|
|
Add to the settings handler:
|
|
```go
|
|
type StoragePathView struct {
|
|
settings.StoragePath
|
|
DiskInfo *DiskUsageInfo // total, used, percent
|
|
AppCount int // number of deployed apps with HDD_PATH matching this
|
|
IsMounted bool // mount-point check result
|
|
}
|
|
```
|
|
|
|
Populate by scanning `app.yaml` files for `HDD_PATH` and matching against registered paths.
|
|
|
|
---
|
|
|
|
## 8. Deprecate `paths.hdd_path` from controller.yaml
|
|
|
|
### 8.1 Backward compatibility
|
|
|
|
Keep reading `paths.hdd_path` — use it only as fallback seed for auto-discovery on first run.
|
|
|
|
Startup order in `main.go`:
|
|
```go
|
|
// 1. Load settings.json
|
|
settingsMgr, _ := settings.Load(...)
|
|
|
|
// 2. Auto-discover storage paths (if settings.json has none)
|
|
settingsMgr.AutoDiscoverStoragePaths(cfg.Paths.StacksDir, cfg.Paths.HDDPath, logger)
|
|
|
|
// 3. Wire adapter with storage paths getter
|
|
adapter := &stackAdapter{
|
|
mgr: stackMgr,
|
|
getStoragePaths: func() []settings.StoragePath { return settingsMgr.GetStoragePaths() },
|
|
}
|
|
```
|
|
|
|
### 8.2 Update configs/controller.yaml.example
|
|
|
|
```yaml
|
|
paths:
|
|
stacks_dir: "/opt/docker/stacks"
|
|
# hdd_path is DEPRECATED — storage paths are managed via web UI (Beállítások > Adattárolók)
|
|
# Existing value is auto-migrated to settings.json on first startup
|
|
# hdd_path: "/mnt/hdd_1"
|
|
```
|
|
|
|
---
|
|
|
|
## 9. Implementation Steps
|
|
|
|
### Step 1: Storage paths in settings.json
|
|
- Add `StoragePath` struct and `StoragePaths` field to `Settings`
|
|
- Add all getter/setter methods
|
|
- Add `AutoDiscoverStoragePaths()` — note: may need to pass a scanner function or do scanning in main.go to avoid circular import with stacks package
|
|
- Add `inferStorageLabel()` helper
|
|
- **Test:** Manually create a `settings.json` with `storage_paths` → Load → verify GetStoragePaths returns them
|
|
|
|
### Step 2: Mount-point validation utilities
|
|
- Create `controller/internal/system/mounts.go` with `IsMountPoint()` (Linux + stub)
|
|
- Add overlap check helper: `PathsOverlap(a, b string) bool`
|
|
- Add writable check helper: `IsWritable(path string) bool`
|
|
- **Test:** `IsMountPoint("/mnt/hdd_1")` → true, `IsMountPoint("/tmp")` → false
|
|
|
|
### Step 3: Wire auto-discovery on startup
|
|
- In `main.go`: call `AutoDiscoverStoragePaths` after loading settings
|
|
- Pass `cfg.Paths.HDDPath` as fallback
|
|
- **Test:** Start controller on demo-felhom with empty settings.json → should auto-discover `/mnt/hdd_placeholder` from deployed apps' app.yaml
|
|
|
|
### Step 4: Fix per-app HDD_PATH resolution (THE CORE FIX)
|
|
- Update `stackAdapter` struct: replace `hddPath string` with `getStoragePaths func()`
|
|
- Update `GetStackHDDMounts()` to read per-app HDD_PATH from app.yaml first, fallback to registered paths
|
|
- Wire updated adapter in `main.go`
|
|
- **Test:** Backup page → "Alkalmazás adatok" section now shows with correct paths (e.g., `/mnt/hdd_placeholder/storage/immich`). Backup toggles are visible and functional.
|
|
|
|
### Step 5: Controller compose mount change
|
|
- Change `controller/docker-compose.yml`: `/mnt:/mnt:rw`
|
|
- Update `docker-setup.sh` controller compose generation
|
|
- **Test:** Recreate controller container → verify both `/mnt/hdd_1` and `/mnt/hdd_placeholder` accessible inside container
|
|
|
|
### Step 6: Beállítások storage section
|
|
- Add "Adattárolók" section to `settings.html`
|
|
- Add handlers: `POST /settings/storage/add`, `/remove`, `/default`, `/schedulable`
|
|
- Add `StoragePathView` type for template data with disk info + app count
|
|
- Pass storage data to settings page handler
|
|
- Add CSS for storage path items
|
|
- **Test:** Add path via UI → validation catches non-mount-point. Remove path with apps → blocked with app list. Set default → badge updates.
|
|
|
|
### Step 7: Deploy page dropdown
|
|
- Modify `deploy.html`: path field becomes dropdown when storage paths exist
|
|
- Pass `StoragePaths` to deploy page template data
|
|
- Fall back to text input if no paths registered
|
|
- Block deploy if app needs HDD but no schedulable paths
|
|
- **Test:** Deploy page for Immich → dropdown shows registered paths. Already-deployed shows current path read-only.
|
|
|
|
### Step 8: Storage monitoring integration
|
|
- Add `checkStoragePaths()` to system health check
|
|
- Surface warnings on monitoring page
|
|
- Include in hub report
|
|
- Fire `disk_warning` notification for unmounted drives
|
|
- **Test:** (Simulated) If path doesn't exist → warning appears on monitoring page within one health check cycle
|
|
|
|
### Step 9: Cleanup & version bump
|
|
- Deprecate `paths.hdd_path` in controller.yaml.example
|
|
- Create new CHANGELOG.md, changelogs will be updated there, not in CONTEXT.md. CONTEXT.md will only have information about architecture decisions, roadmap, information about the project
|
|
- Regenerate CONTEXT.md with current architecture, deparating different sections, with detailed descriptions how it should work, what is planned, what is the architecture. Sections like: Storage management, Notifications, Backup management, App management, Monitoring, Infra, Settings management, etc-etc..
|
|
- Update CONTEXT.md / CHANGELOG.md / CLAUDE.md
|
|
- Bump to 0.9.0
|
|
- Build + deploy
|
|
- **Test end-to-end:** Auto-discover → backup toggles visible → enable Immich backup → manual backup → verify HDD data in restic snapshot → storage management in Beállítások works
|
|
|
|
---
|
|
|
|
## 10. Files to Create / Modify
|
|
|
|
### New files:
|
|
- `controller/internal/system/mounts.go` — `IsMountPoint()`, `IsWritable()`, `PathsOverlap()` + non-Linux stub
|
|
- (No new packages — StoragePath goes in existing `settings` package)
|
|
|
|
### Modified files:
|
|
- `controller/internal/settings/settings.go` — `StoragePath` struct, `StoragePaths` field, all getter/setter methods, `AutoDiscoverStoragePaths()`, `inferStorageLabel()`, validation logic
|
|
- `controller/cmd/controller/main.go` — Wire auto-discovery, update stackAdapter struct (replace `hddPath` with `getStoragePaths`), update adapter constructor
|
|
- `controller/internal/web/handlers.go` — Storage management handlers (`/settings/storage/*`), pass StoragePaths to deploy + settings templates
|
|
- `controller/internal/web/server.go` — Register new storage routes
|
|
- `controller/internal/web/templates/settings.html` — New "Adattárolók" section
|
|
- `controller/internal/web/templates/deploy.html` — Path field → dropdown, edge case warnings
|
|
- `controller/internal/web/templates/style.css` — Storage path item styles, badges
|
|
- `controller/internal/monitoring/health.go` (or wherever health checks live) — Add `checkStoragePaths()`
|
|
- `controller/docker-compose.yml` — `/mnt:/mnt:rw`
|
|
- `controller/configs/controller.yaml.example` — Deprecation comment
|
|
- `docker-setup.sh` — Update controller compose generation
|
|
|
|
---
|
|
|
|
## 11. Design Decisions & Notes
|
|
|
|
### Why settings.json, not controller.yaml?
|
|
controller.yaml is operator-controlled (read-only from customer perspective). User-configurable state lives in settings.json. Clear separation: operator configures infrastructure, customer configures preferences.
|
|
|
|
### Why auto-discover from app.yaml?
|
|
Existing deployments already have apps with HDD_PATH set. Auto-discovery makes the upgrade seamless — backup toggles appear without manual intervention after the update.
|
|
|
|
### Why `/mnt:/mnt:rw` instead of individual mounts?
|
|
Single mount covers all current and future storage devices. No controller restart when adding new drives. Required for restore + validation. Trade-off: broader access, but controller already has Docker socket access and is a trusted component.
|
|
|
|
### Why validate mount points?
|
|
If a USB drive is disconnected but the empty directory remains (created by Docker or `nofail` fstab), data silently writes to the SSD boot drive. Mount-point check (device ID comparison) catches this. This is the #1 gotcha for home server setups.
|
|
|
|
### Why no overlapping paths?
|
|
If `/mnt/hdd_1` and `/mnt/hdd_1/storage` are both registered, discovery counts files twice and backup may include data twice. One entry per physical mount point keeps it simple.
|
|
|
|
### Why fallback chain in GetStackHDDMounts?
|
|
1. App's own HDD_PATH from app.yaml (most accurate — the actual deployed value)
|
|
2. Try all registered storage paths (handles edge cases: missing app.yaml, new template before deploy)
|
|
Ensures backup discovery works even in degraded states.
|
|
|
|
### Why `Schedulable` toggle?
|
|
An "unschedulable" path means existing apps stay put but no new apps can use it. Useful for drives that are filling up or being deprecated — prevents new deployments while allowing existing apps to continue operating.
|
|
|
|
### Future phases (NOT in scope)
|
|
- **Phase B:** Storage management UI polish — disk usage bars per path, per-app storage column on apps page
|
|
- **Phase C:** Migration wizard — "Mozgatás" button per app, rsync with progress reporting, automatic app.yaml HDD_PATH update + restart, old data cleanup option |