From a3af7c6a2deb5fa9e477a39cef60c805c84f95e3 Mon Sep 17 00:00:00 2001 From: kisfenyo Date: Mon, 16 Feb 2026 21:05:51 +0100 Subject: [PATCH] =?UTF-8?q?Phase=203=20=E2=80=94=20Storage=20Overview=20&?= =?UTF-8?q?=20Per-App=20Backup=20Toggles?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- TASK.md | 1022 +++++++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 774 insertions(+), 248 deletions(-) diff --git a/TASK.md b/TASK.md index 6491532..2e6d741 100644 --- a/TASK.md +++ b/TASK.md @@ -1,327 +1,853 @@ -# TASK: Fix Notification Preferences Sync (Controller → Hub) +# TASK: Phase 3 — Storage Overview & Per-App Backup Toggles -**Version target:** controller 0.7.2, hub 0.1.5 -**Repos:** `deploy-felhom-compose` (controller) + `felhom.eu` (hub) +**Version target:** controller 0.8.0 +**Repos:** `deploy-felhom-compose` (controller), `app-catalog-felhom.eu` (metadata) -## Problem +## Overview -When a customer saves notification settings (email + enabled events) on the controller settings page, the preferences are only stored locally in `settings.json`. The hub's `customer_notifications` SQLite table remains empty, so when the controller sends a notification event via `POST /api/v1/notify`, the hub responds with "No email configured for demo-felhom, skipping notification." +Currently, restic backs up three fixed paths: the stacks directory (compose files), DB dumps, and `controller.yaml`. User data stored on the HDD (Paperless documents, media files, etc.) and Docker named volumes are **not** included. -The email sending infrastructure works (Resend API key is configured, `sendResendEmail` is proven). The gap is that preferences never reach the hub. - -## Solution - -Add a preferences sync endpoint to the hub (`POST /api/v1/preferences`) and have the controller call it when the user saves notification settings. +Phase 3 adds: +1. **Storage overview** on the backup page — SSD, HDD, and backup repo usage at a glance +2. **Per-app backup discovery** — controller discovers each app's user data (HDD bind mounts + Docker volumes) +3. **Per-app backup toggles** — customer enables/disables backup per app on the backup page +4. **Dynamic backup paths** — `RunBackup()` includes enabled app data paths in the restic snapshot +5. **Restic password visibility** — password shown on backup page (behind toggle) + synced to hub for disaster recovery +6. **Limited app restore** — per-app restore from snapshot with warnings, self-service emergency recovery --- -## 1. Hub: Add `POST /api/v1/preferences` Endpoint +## 1. Current State -### 1.1 Route - -Add to `hub/internal/api/handler.go` routing: +### What restic backs up today ```go -case r.Method == http.MethodPost && path == "/preferences": - h.handleSavePreferences(w, r) +// In backup.go RunBackup(): +paths := []string{ + m.cfg.Paths.StacksDir, // /opt/docker/stacks (compose files + .felhom.yml) + m.cfg.Paths.DBDumpDir, // /srv/backups/db-dumps (nightly dumps) + "/opt/docker/felhom-controller/controller.yaml", +} ``` -### 1.2 Handler: `handleSavePreferences` +### What's NOT backed up +- HDD user data (e.g., `/mnt/hdd_1/paperless/media`, `/mnt/hdd_1/romm/...`) +- Docker named volumes (e.g., `actualbudget_data`, `docmost-postgres_data`) + +### Existing infrastructure we can reuse +- `parseComposeHDDMounts(composePath, hddPath)` — already discovers HDD bind mount paths per stack by parsing compose files +- `GetStackHDDData(name)` — returns HDD paths + sizes for a stack +- `HDDPath` struct with `Path`, `SizeBytes`, `SizeHuman`, `Exists` +- HDD is already mounted into the controller container (read-only, fine for restic) +- `.felhom.yml` has `resources.needs_hdd: true` flag +- Monitoring page already shows SSD/HDD disk usage + +### Controller container volume mounts (relevant) +```yaml +- /opt/docker/stacks:/opt/docker/stacks # compose files +- /srv/backups:/srv/backups # restic repo + db dumps +- ${HDD_PATH}:${HDD_PATH}:ro # HDD (read-only) +- /var/run/docker.sock:/var/run/docker.sock:ro # Docker socket +``` + +HDD data is accessible from the controller container for restic reads. Docker volume paths (`/var/lib/docker/volumes/`) are NOT mounted — volume backup requires a different approach (deferred). + +--- + +## 2. Storage Overview (Backup Page Enhancement) + +### 2.1 New section: "Tárhely áttekintés" (Storage Overview) + +Add as the **first section** on the backup page, above the current "Ütemezés" section. Shows storage utilization relevant to backups: + +``` +┌─────────────────────────────────────────────────────────────┐ +│ Tárhely áttekintés │ +│ │ +│ SSD (/) ████████░░░░░░░░░░░░ 42.1 / 512.0 GB (8%) │ +│ Külső HDD ██████████████░░░░░░ 680.2 / 1000.0 GB │ +│ (68%) │ +│ │ +│ Mentési tároló 2.4 GB (/srv/backups/restic-repo) │ +│ DB mentések 142 MB (/srv/backups/db-dumps) │ +│ Pillanatképek 14 │ +└─────────────────────────────────────────────────────────────┘ +``` + +### 2.2 Data source + +Reuse `system.GetInfo()` for SSD/HDD disk stats (already available in `SystemInfo`). Add backup repo stats from existing `RepoStats`. DB dump dir size from `ListDumpFiles()`. + +No new backend code needed — just template work. The `FullBackupStatus` already contains `RepoStats` and the monitoring page already has `SystemInfo`. + +Add `SystemInfo` to the backup page template data if not already present: ```go -func (h *Handler) handleSavePreferences(w http.ResponseWriter, r *http.Request) { - // Same bearer token auth as /report and /notify - if h.apiKey != "" { - auth := r.Header.Get("Authorization") - if !strings.HasPrefix(auth, "Bearer ") || strings.TrimPrefix(auth, "Bearer ") != h.apiKey { - http.Error(w, "Unauthorized", http.StatusUnauthorized) - return +type BackupPageData struct { + // existing fields... + SystemInfo *system.Info // for SSD/HDD bars +} +``` + +--- + +## 3. Per-App Data Discovery + +### 3.1 App data classification + +For each deployed app, classify its data into three categories: + +| Category | Example | Discovery method | Backup support | +|---|---|---|---| +| **HDD bind mounts** | `/mnt/hdd_1/paperless/media` | `parseComposeHDDMounts()` (existing) | ✅ This phase | +| **Named Docker volumes** | `paperless-ngx_postgres_data` | Parse compose `volumes:` section | ⏳ Future (not mounted in controller) | +| **Config (stacks dir)** | `/opt/docker/stacks/paperless-ngx/` | Always backed up | ✅ Already done | + +### 3.2 New struct: `AppBackupInfo` + +```go +// AppBackupInfo holds backup-relevant data paths for a deployed app. +type AppBackupInfo struct { + StackName string // e.g., "paperless-ngx" + DisplayName string // e.g., "Paperless-ngx" + NeedsHDD bool // from .felhom.yml resources.needs_hdd + + // HDD bind mounts (backupable now) + HDDPaths []AppDataPath + HDDTotalSize int64 // bytes + HDDSizeHuman string + + // Docker named volumes (info only, not backupable yet) + DockerVolumes []AppDockerVolume + + // Backup state + BackupEnabled bool // from settings.json + HasHDDData bool // HDDPaths is non-empty + HasDBDump bool // app has a database container (already covered by DB dump) +} + +type AppDataPath struct { + HostPath string // e.g., "/mnt/hdd_1/paperless/media" + Exists bool + SizeHuman string + SizeBytes int64 +} + +type AppDockerVolume struct { + Name string // e.g., "paperless-ngx_postgres_data" + Contains string // Human description (from .felhom.yml, or empty) +} +``` + +### 3.3 Discovery: `DiscoverAppData()` + +New function in `internal/backup/appdata.go`: + +```go +// DiscoverAppData discovers backup-relevant data for all deployed apps. +func DiscoverAppData(stackProvider StackDataProvider, hddPath string, backupPrefs map[string]bool, discoveredDBs []DiscoveredDB) []AppBackupInfo { + var result []AppBackupInfo + + for _, stack := range stackProvider.ListDeployedStacks() { + info := AppBackupInfo{ + StackName: stack.Name, + DisplayName: stack.DisplayName, + NeedsHDD: stack.NeedsHDD, + } + + // Discover HDD bind mounts (reuse existing parser) + hddMounts := parseComposeHDDMounts(stack.ComposePath, hddPath) + for _, mount := range hddMounts { + path := AppDataPath{HostPath: mount} + if fi, err := os.Stat(mount); err == nil && fi.IsDir() { + path.Exists = true + path.SizeBytes = getDirSizeBytes(mount) + path.SizeHuman = getDirSizeHuman(mount) + } + info.HDDPaths = append(info.HDDPaths, path) + info.HDDTotalSize += path.SizeBytes + } + info.HDDSizeHuman = humanizeBytes(info.HDDTotalSize) + info.HasHDDData = len(info.HDDPaths) > 0 + + // Discover Docker named volumes (from compose file) + info.DockerVolumes = parseComposeNamedVolumes(stack.ComposePath) + + // Check if app has a DB container (already backed up via DB dump) + for _, db := range discoveredDBs { + if db.StackName == stack.Name { + info.HasDBDump = true + break + } + } + + info.BackupEnabled = backupPrefs[stack.Name] + + if info.HasHDDData || len(info.DockerVolumes) > 0 { + result = append(result, info) } } - body, err := io.ReadAll(io.LimitReader(r.Body, 1<<20)) - if err != nil { - http.Error(w, "Bad request", http.StatusBadRequest) - return - } - - var payload struct { - CustomerID string `json:"customer_id"` - Email string `json:"email"` - EnabledEvents []string `json:"enabled_events"` - } - if err := json.Unmarshal(body, &payload); err != nil || payload.CustomerID == "" { - http.Error(w, "Invalid payload: customer_id required", http.StatusBadRequest) - return - } - - if err := h.store.SaveNotificationPrefs(payload.CustomerID, payload.Email, payload.EnabledEvents); err != nil { - h.logger.Printf("[ERROR] Failed to save notification prefs for %s: %v", payload.CustomerID, err) - http.Error(w, "Internal error", http.StatusInternalServerError) - return - } - - h.logger.Printf("[INFO] Notification preferences updated for %s: email=%s, events=%v", payload.CustomerID, payload.Email, payload.EnabledEvents) - w.WriteHeader(http.StatusOK) - w.Write([]byte(`{"status":"ok"}`)) + return result } ``` -**Note:** `store.SaveNotificationPrefs` already exists and uses `INSERT ... ON CONFLICT DO UPDATE`. No store changes needed. +### 3.4 Named volume parser: `parseComposeNamedVolumes()` -### 1.3 Edge case: empty email +```go +// parseComposeNamedVolumes extracts named Docker volumes from a docker-compose.yml. +// Parses the top-level `volumes:` key. Skips external volumes. +func parseComposeNamedVolumes(composePath string) []AppDockerVolume { + data, err := os.ReadFile(composePath) + if err != nil { return nil } -If the customer clears their email and saves, the controller should still sync — this effectively disables notifications for that customer. The hub accepts empty email and stores it (the notify handler already handles `prefs.Email == ""` gracefully). + var compose struct { + Volumes map[string]interface{} `yaml:"volumes"` + } + if err := yaml.Unmarshal(data, &compose); err != nil { return nil } + + var volumes []AppDockerVolume + for name, cfg := range compose.Volumes { + // Skip external volumes (networks like traefik-public) + if cfgMap, ok := cfg.(map[string]interface{}); ok { + if ext, ok := cfgMap["external"]; ok && ext == true { + continue + } + } + volumes = append(volumes, AppDockerVolume{Name: name}) + } + return volumes +} +``` --- -## 2. Controller: Sync Preferences on Save +## 4. Per-App Backup Toggles -### 2.1 Add `SyncPreferences` method to `internal/notify/notifier.go` +### 4.1 Storage in `settings.json` + +Add to the `Settings` struct: ```go -// preferencesRequest is the JSON payload sent to the hub preferences endpoint. -type preferencesRequest struct { - CustomerID string `json:"customer_id"` - Email string `json:"email"` - EnabledEvents []string `json:"enabled_events"` +type Settings struct { + // ... existing fields ... + + // Per-app backup preferences + AppBackup map[string]AppBackupPrefs `json:"app_backup,omitempty"` } -// SyncPreferences pushes the current notification preferences to the hub. -// Called after the user saves notification settings on the settings page. -// Returns error for the handler to display to the user. -func (n *Notifier) SyncPreferences(email string, enabledEvents []string) error { - if !n.enabled { - return fmt.Errorf("hub nem konfigurált") +type AppBackupPrefs struct { + Enabled bool `json:"enabled"` +} +``` + +Add getter/setter methods: + +```go +func (s *Settings) IsAppBackupEnabled(stackName string) bool +func (s *Settings) SetAppBackup(stackName string, enabled bool) error +func (s *Settings) GetAppBackupMap() map[string]bool // stack_name -> enabled +``` + +### 4.2 Backup page: "Alkalmazás adatok" section + +New section on the backup page, below "Tároló" (repo info), above snapshot history: + +``` +┌─────────────────────────────────────────────────────────────┐ +│ Alkalmazás adatok │ +│ │ +│ Az alkalmazások felhasználói adatainak biztonsági mentése. │ +│ │ +│ ┌──────────────────────────────────────────────────────────┐ │ +│ │ [✅] Paperless-ngx 2.4 GB (HDD) │ │ +│ │ /mnt/hdd_1/paperless/media (2.1 GB) │ │ +│ │ /mnt/hdd_1/paperless/consume (312 MB) │ │ +│ │ 📦 Docker kötet: paperless-ngx_data (nem mentett) │ │ +│ ├──────────────────────────────────────────────────────────┤ │ +│ │ [☐] RoMM 8.7 GB (HDD) │ │ +│ │ /mnt/hdd_1/romm/library (8.5 GB) │ │ +│ │ /mnt/hdd_1/romm/assets (200 MB) │ │ +│ ├──────────────────────────────────────────────────────────┤ │ +│ │ [—] ActualBudget │ │ +│ │ 📦 Docker kötet: actualbudget_data (nem mentett) │ │ +│ │ ℹ️ Adatbázis mentés naponta (DB dump) │ │ +│ └──────────────────────────────────────────────────────────┘ │ +│ │ +│ [Mentés] │ +│ │ +│ ⚠️ Docker kötetek mentése jelenleg nem támogatott. │ +│ Az adatbázisokat az automatikus DB dump menti naponta. │ +└─────────────────────────────────────────────────────────────┘ +``` + +**UI behavior:** +- Toggle checkbox per app — only enabled for apps with HDD data +- Apps with only Docker volumes show info state (no checkbox) +- Apps with databases show "DB dump naponta" note +- "Mentés" button saves all toggles at once via `POST /settings/app-backup` +- Sizes update on page refresh (from the cached `RefreshCache`) + +### 4.3 Route + +| Method | Path | Auth? | Handler | +|--------|------|-------|---------| +| POST | `/settings/app-backup` | Yes | Save per-app backup toggles | + +**Handler:** +1. Parse form: checkboxes named `backup_{stack_name}` (checked = enabled) +2. For each app with HDD data: set enabled/disabled in settings +3. Save to `settings.json` +4. Refresh backup cache +5. Redirect to `/backups` with success flash: "Alkalmazás mentési beállítások mentve." + +--- + +## 5. Dynamic Backup Paths in RunBackup + +### 5.1 Modify `RunBackup()` in `backup.go` + +```go +// Base paths (always backed up) +paths := []string{ + m.cfg.Paths.StacksDir, + m.cfg.Paths.DBDumpDir, + "/opt/docker/felhom-controller/controller.yaml", +} + +// Per-app HDD data paths (from settings) +appPaths := m.resolveAppBackupPaths() +paths = append(paths, appPaths...) + +m.logger.Printf("[INFO] Backup paths (%d total): %v", len(paths), paths) +``` + +### 5.2 Path resolver + +```go +func (m *Manager) resolveAppBackupPaths() []string { + appBackupMap := m.settings.GetAppBackupMap() + if len(appBackupMap) == 0 { + return nil } - payload := preferencesRequest{ - CustomerID: n.customerID, - Email: email, - EnabledEvents: enabledEvents, - } + var paths []string + seen := make(map[string]bool) - jsonData, err := json.Marshal(payload) + for stackName, enabled := range appBackupMap { + if !enabled { + continue + } + composePath, ok := m.stackProvider.GetStackComposePath(stackName) + if !ok { + m.logger.Printf("[WARN] App backup enabled for %s but stack not found", stackName) + continue + } + hddMounts := parseComposeHDDMounts(composePath, m.cfg.Paths.HDDPath) + for _, mount := range hddMounts { + if seen[mount] { + continue + } + if _, err := os.Stat(mount); err == nil { + paths = append(paths, mount) + seen[mount] = true + m.logger.Printf("[DEBUG] Including app data: %s (from %s)", mount, stackName) + } + } + } + return paths +} +``` + +### 5.3 Stack data provider interface + +To avoid circular imports between `backup` and `stacks` packages: + +```go +// In backup package +type StackDataProvider interface { + GetStackComposePath(name string) (composePath string, ok bool) + ListDeployedStacks() []StackSummary +} + +type StackSummary struct { + Name string + DisplayName string + ComposePath string + NeedsHDD bool +} +``` + +Implement this interface with a thin wrapper in `main.go` or as an adapter: + +```go +// In main.go +type stackAdapter struct{ mgr *stacks.Manager } + +func (a *stackAdapter) GetStackComposePath(name string) (string, bool) { + s, ok := a.mgr.GetStack(name) + if !ok { return "", false } + return s.ComposePath, true +} +func (a *stackAdapter) ListDeployedStacks() []backup.StackSummary { ... } +``` + +### 5.4 Update `FullBackupStatus` + +```go +type FullBackupStatus struct { + // ... existing ... + AppDataInfo []AppBackupInfo // per-app backup info for backup page + AppDataPaths []string // resolved app data paths (for "Mentett útvonalak" display) + AppDataSizeHuman string // total size of enabled app data +} +``` + +Populate `AppDataInfo` during `RefreshCache()` (runs periodically). + +--- + +## 6. App Catalog Metadata Enhancement (Optional) + +### 6.1 Add `backup` section to `.felhom.yml` + +While the controller can discover HDD paths from compose files, explicit metadata gives better descriptions: + +```yaml +# In .felhom.yml for paperless-ngx +backup: + description: "Dokumentumok, beolvasott fájlok és feldolgozási queue" + data_paths: + - "${HDD_PATH}/paperless/media" + - "${HDD_PATH}/paperless/consume" + - "${HDD_PATH}/paperless/export" + docker_volumes: + - name: "paperless-ngx_data" + contains: "Alkalmazás belső adatai" + - name: "paperless-ngx_postgres_data" + contains: "Adatbázis (DB dump menti)" +``` + +**Optional** — the controller falls back to `parseComposeHDDMounts()` auto-discovery if the `backup` section doesn't exist. + +### 6.2 Update `Metadata` struct + +```go +type Metadata struct { + // ... existing fields ... + Backup BackupMetadata `yaml:"backup" json:"backup"` +} + +type BackupMetadata struct { + Description string `yaml:"description" json:"description"` + DataPaths []string `yaml:"data_paths" json:"data_paths"` + DockerVolumes []VolumeDescription `yaml:"docker_volumes" json:"docker_volumes"` +} +``` + +### 6.3 Discovery priority + +1. If `.felhom.yml` has `backup.data_paths` → use those (resolve `${HDD_PATH}`) +2. Else → fall back to `parseComposeHDDMounts()` auto-discovery +3. Docker volumes: merge `.felhom.yml` descriptions with parsed compose volumes + +--- + +## 7. Restic Password Visibility + +### 7.1 Problem + +The restic password is auto-generated on first backup and stored at `/opt/docker/felhom-controller/data/restic-password` inside the `controller-data` Docker named volume. If the SSD dies, that password is gone and ALL backup snapshots become permanently inaccessible. The customer currently has zero visibility into this password. + +### 7.2 Password display on backup page + +Add a "Titkosítási kulcs" (Encryption Key) section on the backup page, within the storage/repo area: + +``` +┌─────────────────────────────────────────────────────────────┐ +│ Titkosítási kulcs │ +│ │ +│ A mentések titkosítva vannak. A visszaállításhoz szükség │ +│ van erre a kulcsra. │ +│ │ +│ [••••••••••••••••••••••] [👁 Megjelenítés] [📋 Másolás] │ +│ │ +│ ⚠️ Mentse el biztonságos helyre! A kulcs nélkül a │ +│ biztonsági mentések NEM állíthatók vissza. │ +└─────────────────────────────────────────────────────────────┘ +``` + +**Implementation:** + +```go +// Add to ResticManager +func (r *ResticManager) GetPassword() (string, error) { + data, err := os.ReadFile(r.passwordFile) if err != nil { - return fmt.Errorf("marshal: %w", err) + return "", fmt.Errorf("reading restic password: %w", err) + } + return strings.TrimSpace(string(data)), nil +} +``` + +**Template behavior:** +- Password field is masked by default (`type="password"`) +- "Megjelenítés" button toggles visibility (JS, `type="text"`) +- "Másolás" button copies to clipboard (JS, `navigator.clipboard.writeText()`) +- Password is loaded in the template data (served over existing auth-protected page) + +### 7.3 Password sync to hub + +Add password to the periodic hub report so the operator has a backup copy: + +```go +// In the hub report payload (already sent every 15m) +type ReportPayload struct { + // ... existing fields ... + ResticPassword string `json:"restic_password,omitempty"` // encrypted key for recovery +} +``` + +**Hub side** — store the password in the customer record. The hub is operator-controlled and already stores customer metadata. Add a `restic_password` field to the hub database for each customer. + +**Sync trigger:** +- On every periodic report (piggyback on existing 15-minute push) +- On startup (controller init) + +**Security note:** The hub API is already authenticated with `Bearer` token. The password travels over HTTPS (Cloudflare Tunnel). This is safer than the password being in a single Docker named volume with no redundancy. + +--- + +## 8. Limited App Restore (Self-Service Emergency) + +### 8.1 Approach + +Provide a per-app "Visszaállítás" (Restore) feature on the backup page that restores an entire app's HDD data from a selected snapshot. This is a self-service emergency tool — customers can recover from accidental deletions without waiting for support. + +**Scope:** Only restore HDD bind mount data for apps that have backup enabled. Compose files and controller.yaml are always restorable (they're in every snapshot). + +### 8.2 UI: Restore section on backup page + +Add a "Visszaállítás" section below the snapshot history: + +``` +┌─────────────────────────────────────────────────────────────┐ +│ Visszaállítás │ +│ │ +│ Alkalmazás: [▾ Paperless-ngx ] │ +│ Pillanatkép: [▾ 2026-02-16 03:00 (legutóbbi) ] │ +│ │ +│ Visszaállítandó útvonalak: │ +│ /mnt/hdd_1/paperless/media │ +│ /mnt/hdd_1/paperless/consume │ +│ │ +│ ⚠️ FIGYELMEZTETÉS │ +│ A visszaállítás FELÜLÍRJA a kiválasztott alkalmazás │ +│ jelenlegi adatait a mentés pillanatának állapotával. │ +│ Ez a művelet NEM vonható vissza! │ +│ │ +│ Javasoljuk az alkalmazás leállítását a visszaállítás előtt. │ +│ │ +│ [☐] Megértettem, visszaállítás saját felelősségre. │ +│ │ +│ [🔄 Visszaállítás indítása] (disabled until checkbox) │ +└─────────────────────────────────────────────────────────────┘ +``` + +### 8.3 Restore flow + +1. User selects app + snapshot from dropdowns +2. JS fetches app's HDD paths (already in `AppBackupInfo`) and shows them +3. User checks the "saját felelősségre" checkbox — this enables the button +4. `POST /backup/restore` with `stack_name`, `snapshot_id` +5. Handler validates: + - Stack exists and has backup enabled + - Snapshot ID exists in restic repo + - HDD paths are valid +6. Controller runs `restic restore` for the specific paths +7. Redirect to backup page with result flash message + +### 8.4 Backend: `RestoreAppData()` + +```go +// In ResticManager +func (r *ResticManager) RestoreAppData(snapshotID string, paths []string) error { + // Build --include flags for each HDD path + args := []string{ + "restore", snapshotID, + "--target", "/", // restore to original absolute paths + "--password-file", r.passwordFile, + "--repo", r.repoPath, + "--cache-dir", r.cacheDir, + } + for _, p := range paths { + args = append(args, "--include", p) } - url := n.hubURL + "/api/v1/preferences" - req, err := http.NewRequest("POST", url, bytes.NewReader(jsonData)) + r.logger.Printf("[WARN] RESTORE started: snapshot=%s, paths=%v", snapshotID, paths) + + cmd := exec.Command("restic", args...) + output, err := cmd.CombinedOutput() if err != nil { - return fmt.Errorf("request: %w", err) - } - req.Header.Set("Authorization", "Bearer "+n.apiKey) - req.Header.Set("Content-Type", "application/json") - - resp, err := n.httpClient.Do(req) - if err != nil { - return fmt.Errorf("hub elérhetetlen: %w", err) - } - defer resp.Body.Close() - - if resp.StatusCode >= 400 { - body, _ := io.ReadAll(io.LimitReader(resp.Body, 512)) - return fmt.Errorf("hub hiba (%d): %s", resp.StatusCode, string(body)) + r.logger.Printf("[ERROR] Restore failed: %v, output: %s", err, string(output)) + return fmt.Errorf("restic restore failed: %w", err) } - n.logger.Printf("[INFO] Notification preferences synced to hub: email=%s, events=%v", email, enabledEvents) + r.logger.Printf("[INFO] RESTORE completed: snapshot=%s, paths=%v", snapshotID, paths) return nil } ``` -**Note:** This is a synchronous call (not goroutine) because the handler needs to know if it succeeded to show feedback to the user. - -### 2.2 Call `SyncPreferences` in the notification settings POST handler - -In `internal/web/handlers.go`, the `handleNotificationSettings` handler currently: -1. Parses form data -2. Saves to `settings.json` -3. Redirects with success message - -Add step 2.5: sync to hub. +### 8.5 Backup Manager: `RestoreApp()` ```go -// After saving to settings.json: -if err := s.notifier.SyncPreferences(email, enabledEvents); err != nil { - s.logger.Printf("[WARN] Failed to sync preferences to hub: %v", err) - // Don't block the save — local settings are saved, sync failed - // Show warning instead of error - // Redirect with: "Értesítési beállítások mentve, de a központi szinkronizálás sikertelen: " -} -// If sync succeeded: "Értesítési beállítások mentve." -``` - -**Important:** Local save always succeeds even if hub sync fails. The user sees a warning but their settings are saved. This prevents a hub outage from blocking local config changes. - -### 2.3 Flash message variants - -| Scenario | Message | Color | -|---|---|---| -| Save + sync OK | "Értesítési beállítások mentve." | Green (success) | -| Save OK, sync failed | "Értesítési beállítások mentve (helyi). A központi szinkronizálás sikertelen: [error]" | Yellow (warning) | -| Save failed | "Hiba az értesítési beállítások mentésekor: [error]" | Red (error) | - -### 2.4 Sync on startup (recommended) - -When the controller starts and has notification preferences in `settings.json`, sync them to the hub once. This handles the case where the hub was rebuilt (lost its DB) or the controller was moved to a new hub. - -In `main.go`, after loading settings and creating the notifier: - -```go -prefs := sett.GetNotificationPrefs() -if prefs.Email != "" && notifier.IsEnabled() { - if err := notifier.SyncPreferences(prefs.Email, prefs.EnabledEvents); err != nil { - logger.Printf("[WARN] Failed to sync notification preferences on startup: %v", err) +func (m *Manager) RestoreApp(stackName, snapshotID string) error { + // Validate app has backup enabled + if !m.settings.IsAppBackupEnabled(stackName) { + return fmt.Errorf("backup not enabled for %s", stackName) } + + // Resolve HDD paths for this app + composePath, ok := m.stackProvider.GetStackComposePath(stackName) + if !ok { + return fmt.Errorf("stack %s not found", stackName) + } + hddMounts := parseComposeHDDMounts(composePath, m.cfg.Paths.HDDPath) + if len(hddMounts) == 0 { + return fmt.Errorf("no HDD data paths found for %s", stackName) + } + + // Validate snapshot exists + snapshots, err := m.restic.ListSnapshots(100) + if err != nil { + return fmt.Errorf("listing snapshots: %w", err) + } + found := false + for _, s := range snapshots { + if s.ID == snapshotID { + found = true + break + } + } + if !found { + return fmt.Errorf("snapshot %s not found", snapshotID) + } + + // Send notification before restore + m.notify("restore_started", fmt.Sprintf("Visszaállítás indult: %s (snapshot: %s)", stackName, snapshotID)) + + // Execute restore + if err := m.restic.RestoreAppData(snapshotID, hddMounts); err != nil { + m.notify("restore_failed", fmt.Sprintf("Visszaállítás sikertelen: %s — %v", stackName, err)) + return err + } + + m.notify("restore_completed", fmt.Sprintf("Visszaállítás kész: %s (snapshot: %s)", stackName, snapshotID)) + return nil } ``` -Add `IsEnabled() bool` method to Notifier if it doesn't exist (simple getter for `n.enabled`). +### 8.6 Route ---- +| Method | Path | Auth? | Handler | +|--------|------|-------|---------| +| POST | `/backup/restore` | Yes | Restore app HDD data from snapshot | +| GET | `/api/backup/snapshots` | Yes | List snapshots (JSON, for restore dropdown) | -## 3. Hub: Display Customer Notification Settings (Dashboard) - -### 3.1 Customer detail page - -On the existing hub customer detail page (`/customers/{id}`), add a small "Notifications" section showing: -- Email: `nagyfenyvesi.viktor@gmail.com` (or "Not set") -- Enabled events: comma-separated list -- Recent notification log entries (last 10) - -### 3.2 Store: Add `GetRecentNotifications` method +**Restore handler:** +1. Parse form: `stack_name`, `snapshot_id` +2. Validate both fields present +3. Call `backupManager.RestoreApp(stackName, snapshotID)` +4. On success: redirect to `/backups` with flash "✅ {app} visszaállítva ({snapshot})." +5. On error: redirect to `/backups` with error flash "❌ Visszaállítás sikertelen: {error}" +**Snapshots JSON handler** (for dynamic dropdown population): ```go -type NotificationLogEntry struct { - EventType string - Severity string - Message string - Status string // "sent", "skipped", "failed" - ErrorMessage string - CreatedAt time.Time -} - -func (s *Store) GetRecentNotifications(customerID string, limit int) ([]NotificationLogEntry, error) { - rows, err := s.db.Query(` - SELECT event_type, severity, message, status, error_message, created_at - FROM notification_log - WHERE customer_id = ? - ORDER BY created_at DESC - LIMIT ?`, customerID, limit) - // ... scan rows into []NotificationLogEntry +func (s *Server) apiBackupSnapshotsHandler(w http.ResponseWriter, r *http.Request) { + snapshots, err := s.backupManager.ListSnapshots(50) + // return as JSON array } ``` -### 3.3 Customer detail handler update +### 8.7 Important constraints -In the web handler that renders the customer detail page, load notification prefs and recent log: +- **HDD must NOT be read-only for restore.** The controller currently mounts HDD as `:ro`. For restore to work, the mount must be `:rw` OR a separate restore path must be used. + - Change HDD mount to `:rw` (simpler, but less safe for normal operation) -```go -notifPrefs, _ := s.store.GetNotificationPrefs(customerID) -recentNotifs, _ := s.store.GetRecentNotifications(customerID, 10) -// Pass both to template data -``` +- **Restore only for apps with enabled backup** — if the user never enabled backup for an app, there's no data to restore. -### 3.4 Customer detail template addition +- **No concurrent restore + backup** — add a mutex/lock to prevent backup and restore running simultaneously. -In `hub/internal/web/templates/customer.html`, add after the Health section: - -```html - -
-

Notifications

-
-
- Email - {{if .NotifPrefs}}{{if .NotifPrefs.Email}}{{.NotifPrefs.Email}}{{else}}Not set{{end}}{{else}}Not configured{{end}} -
- {{if .NotifPrefs}} -
- Events - {{joinStrings .NotifPrefs.EnabledEvents ", "}} -
- {{end}} -
- {{if .RecentNotifications}} -

Recent (last 10)

- - - - {{range .RecentNotifications}} - - - - - - - {{end}} - -
TimeEventStatusMessage
{{.CreatedAt.Format "Jan 02 15:04"}}{{.EventType}}{{.Status}}{{.Message}}
- {{end}} -
-``` - -Add `joinStrings` template function if not already available. +- **Notification events:** Add `restore_started`, `restore_completed`, `restore_failed` event types to the notification system. --- -## 4. Implementation Order +## 9. Implementation Order -### Step 1: Hub — Add preferences endpoint -- Add `POST /api/v1/preferences` route + handler in `handler.go` -- No store changes needed (`SaveNotificationPrefs` already exists) -- Build hub 0.1.5, deploy to k3s -- **Test:** `curl -X POST https://hub.felhom.eu/api/v1/preferences -H "Authorization: Bearer " -H "Content-Type: application/json" -d '{"customer_id":"demo-felhom","email":"nagyfenyvesi.viktor@gmail.com","enabled_events":["disk_warning","backup_failed","update_available"]}'` → check hub logs + SQLite +### Step 1: Storage overview on backup page +- Add `SystemInfo` to backup page template data +- Add "Tárhely áttekintés" section to `backups.html` with SSD/HDD bars + backup repo stats +- Reuse existing CSS for storage bars (from monitoring page) +- **Test:** Backup page shows storage usage bars and repo stats -### Step 2: Controller — Sync on settings save -- Add `SyncPreferences` method to `notifier.go` -- Add `IsEnabled()` method to `notifier.go` if missing -- Call from notification settings POST handler in `handlers.go` -- Update flash messages for success/warning/error -- Build controller 0.7.2, deploy -- **Test:** Save notification settings on controller → hub logs "Notification preferences updated" → press "Teszt email küldése" → email arrives in inbox +### Step 2: Restic password visibility +- Add `GetPassword()` to ResticManager +- Add "Titkosítási kulcs" section to `backups.html` with masked password, show/copy buttons +- Add password to hub report payload (`restic_password` field) +- Hub side: store password in customer record (new DB column or JSON field) +- **Test:** Password visible on backup page → copy works → hub DB has password after report cycle -### Step 3: Controller — Sync on startup -- Add startup sync in `main.go` -- Rebuild + deploy -- **Test:** Restart controller → hub logs show preference sync on startup +### Step 3: App data discovery + settings struct +- Create `internal/backup/appdata.go` with `AppBackupInfo`, `DiscoverAppData()`, `parseComposeNamedVolumes()` +- Define `StackDataProvider` interface in backup package +- Add `AppBackupPrefs` + getter/setter to settings.go +- Create stack adapter in main.go +- **Test:** Call `DiscoverAppData()`, verify it finds paperless HDD paths -### Step 4: Hub — Display notification info on customer page -- Add `GetRecentNotifications` to store -- Load notification prefs + recent log in customer detail handler -- Update `customer.html` template -- Add `joinStrings` template function -- Rebuild + deploy hub -- **Test:** Open hub customer detail page → see email, events, notification log +### Step 4: Per-app toggles on backup page +- Add "Alkalmazás adatok" section to `backups.html` with toggle checkboxes per app +- Add `POST /settings/app-backup` handler +- Include `AppDataInfo` in backup page template data (populate during `RefreshCache`) +- **Test:** Toggle paperless backup on → save → check settings.json → page shows updated state + +### Step 5: Dynamic backup paths in RunBackup +- Add `resolveAppBackupPaths()` to backup Manager +- Modify `RunBackup()` to include enabled app HDD paths +- Update `BackupPaths` display in `FullBackupStatus` +- **Test:** Enable paperless backup → trigger manual backup → verify restic snapshot includes HDD data paths via `restic snapshots --json` + +### Step 6: Limited app restore +- Change HDD container mount from `:ro` to `:rw` in controller docker-compose.yml +- Add `RestoreAppData()` to ResticManager +- Add `RestoreApp()` to backup Manager with validation and notifications +- Add `POST /backup/restore` handler and `GET /api/backup/snapshots` JSON endpoint +- Add "Visszaállítás" section to `backups.html` with app/snapshot dropdowns, warnings, confirmation checkbox +- Add restore notification events (`restore_started`, `restore_completed`, `restore_failed`) +- Add restore mutex to prevent concurrent backup/restore operations +- **Test:** Enable paperless backup → backup → delete test file → restore → verify file is back + +### Step 7: Metadata enhancement (optional) +- Add `backup` section to `.felhom.yml` for relevant apps in app-catalog repo +- Update `Metadata` struct in `metadata.go` +- Update discovery to prefer metadata over compose parsing +- **Test:** Apps with `backup.data_paths` show correct descriptions and paths + +### Step 8: Cleanup & version bump +- Update CONTEXT.md / CHANGELOG +- Bump to 0.8.0 +- Build + deploy +- **Test end-to-end:** Full cycle — enable app backup → nightly backup runs → restore works → password visible and synced to hub --- -## 5. Files to Create / Modify +## 10. Files to Create / Modify -### Hub (`felhom.eu`): -- `hub/internal/api/handler.go` — Add `handleSavePreferences` handler + route -- `hub/internal/store/store.go` — Add `GetRecentNotifications` method + `NotificationLogEntry` struct -- `hub/internal/web/server.go` — Load notification prefs + log in customer detail handler, add `joinStrings` template func -- `hub/internal/web/templates/customer.html` — Add notification section +### New files: +- `controller/internal/backup/appdata.go` — `AppBackupInfo`, `DiscoverAppData()`, `parseComposeNamedVolumes()`, `StackDataProvider` interface +- `controller/internal/backup/restore.go` — `RestoreAppData()`, `RestoreApp()`, restore mutex -### Controller (`deploy-felhom-compose`): -- `controller/internal/notify/notifier.go` — Add `SyncPreferences` method, `preferencesRequest` struct, `IsEnabled()` method -- `controller/internal/web/handlers.go` — Call `SyncPreferences` after saving notification settings, update flash messages -- `controller/cmd/controller/main.go` — Add startup preferences sync +### Modified files: +- `controller/internal/backup/backup.go` — `RunBackup()` uses dynamic paths, `RefreshCache()` builds `AppDataInfo`, add `StackDataProvider` field + `resolveAppBackupPaths()`, restore mutex integration +- `controller/internal/backup/restic.go` — Add `GetPassword()`, `RestoreAppData()` methods to ResticManager +- `controller/internal/settings/settings.go` — `AppBackupPrefs` struct, getter/setter methods +- `controller/internal/web/handlers.go` — `POST /settings/app-backup` handler, `POST /backup/restore` handler, `GET /api/backup/snapshots` JSON endpoint, pass `AppDataInfo` + password to backup page +- `controller/internal/web/templates/backups.html` — "Tárhely áttekintés" section, "Titkosítási kulcs" section, "Alkalmazás adatok" section, "Visszaállítás" section +- `controller/internal/web/templates/style.css` — Styles for app backup cards, toggle rows, restore section, password field +- `controller/internal/notify/notifier.go` — Add `restore_started`, `restore_completed`, `restore_failed` event types +- `controller/internal/stacks/metadata.go` — Add `BackupMetadata` to `Metadata` struct (optional) +- `controller/cmd/controller/main.go` — Create stack adapter, wire into backup manager +- Controller `docker-compose.yml` — Change HDD mount from `:ro` to `:rw` +- Hub report payload struct — Add `restic_password` field +- Hub API/DB — Store `restic_password` per customer (hub-side change) + +### App catalog (optional): +- `paperless-ngx/.felhom.yml` — Add `backup` section with data paths and volume descriptions +- Other apps with HDD data as applicable --- -## 6. Notes +## 11. Design Decisions & Notes -### Deploy order matters -Deploy hub first — the controller needs the `/api/v1/preferences` endpoint to exist. If controller is deployed first, the sync will fail gracefully (warning logged, local save still works). Next save attempt after hub is deployed will succeed. +### Why only HDD data in this phase? -### Hub DB rebuild recovery -The startup sync ensures that if the hub's SQLite is lost (hub redeployment, PVC wipe), the next controller restart will re-populate preferences. This makes the system self-healing. +Docker named volumes are stored at `/var/lib/docker/volumes/` which is NOT mounted into the controller container. Backing them up would require either: +- Mounting `/var/lib/docker/volumes` into the controller (security concern, large mount) +- Running restic via a temporary Docker container (complex orchestration) +- Using `docker cp` to export data (slow, no incremental) -### Resend API key -The hub already has the Resend API key configured — the "DOWN | TEST" email to admin@felhom.eu (visible in Resend dashboard, 11 days ago) proves this. If the key is missing in hub.yaml, the notify handler already logs "Resend API key not configured." That's a config issue, not a code issue. +For most apps, the important user data is on HDD (documents, media, photos). Database data in named volumes is covered by the nightly DB dump. Config-only volumes are small and recoverable from compose + deploy fields. -### Hub dashboard expansion (future, out of scope) -The notification section on the customer detail page is read-only. Future hub features like editing notification preferences from the hub side, customer management, and controller.yaml generation are out of scope but this task lays the data model and API groundwork. \ No newline at end of file +### Why reuse `parseComposeHDDMounts` over explicit metadata? + +Auto-discovery from compose files is zero-config for existing apps. The parser is already proven (used in the orphan delete workflow). Adding `backup.data_paths` to `.felhom.yml` is optional polish that gives better descriptions. + +### Why toggles on the backup page instead of per-app detail? + +Backup is a cross-cutting concern. Having all toggles in one place on the backup page gives the customer a complete picture of what's protected. Individual app detail pages could show backup status and link to the backup page. + +### Backup size awareness + +Enabling app data backup can dramatically increase backup duration and repo size (especially media files with low dedup potential). The repo lives on SSD (`/srv/backups/restic-repo`). Show an info banner: + +"ℹ️ Az alkalmazás adatok mentésének bekapcsolása megnöveli a mentési időt és a tárhelyigényt." + +The existing health check already warns on SSD disk usage > 80%. + +### HDD read-only mount is fine + +The controller mounts HDD as `:ro`. Restic only reads files to create snapshots — read-only access is sufficient and more secure. + +### Circular import avoidance + +The backup package needs stack data but shouldn't import the stacks package. Use a `StackDataProvider` interface defined in the backup package, implemented by a thin adapter in `main.go`. This keeps the dependency graph clean. + +### Future considerations +- **Docker volume backup** — Phase 4: mount `/var/lib/docker/volumes:ro` into controller, or use a sidecar approach +- **Offsite backup** — the "Távoli másolat" placeholder: second restic repo to B2/S3/SFTP +- **Selective file restore** — browse individual files within snapshots (restic `ls` + `restore --include`). Phase 3 does whole-app restore only. +- **Backup scheduling per app** — different retention or frequency for different apps +- **Backup size quota warnings** — alert when backup repo exceeds a configurable threshold + +### Encryption architecture + +All restic backups are encrypted at rest — restic has no unencrypted mode. This is a feature for a managed service ("your backups are encrypted even if someone accesses the drive"). The password is: + +1. Auto-generated (32 random bytes, base64url encoded) on first backup +2. Stored locally at `/opt/docker/felhom-controller/data/restic-password` (Docker named volume) +3. Displayed on the backup page behind a toggle (customer can copy/save it) +4. Synced to hub via periodic report (operator has recovery copy) + +If the SSD fails, the password can be retrieved from hub to access the offsite/HDD backup repo. + +### Why limited restore instead of full file browser? + +A full file browser (restic `ls` per snapshot, pick individual files) adds significant UI complexity: directory tree rendering, file selection, path handling. For Phase 3, whole-app restore covers the primary emergency use case (accidental deletion, corruption). File-level restore can be added later as a power-user feature. + +### Why show the password to the customer? + +The managed service model means the operator (Viktor) has hub access and can recover the password. But showing it to the customer: +- Enables self-service disaster recovery if the customer is technically capable +- Builds trust — the customer isn't locked out of their own backups +- Reduces support burden for simple restore scenarios + +The "behind toggle + clipboard" pattern prevents accidental exposure while making it accessible when needed. + +### Why sync password to hub? + +The password lives in a Docker named volume on the SSD. If the SSD fails: +- Without hub sync: password is permanently lost → all backups inaccessible +- With hub sync: operator retrieves from hub → restores access + +This is the single most critical piece of information for backup recovery. Hub sync provides the necessary redundancy. + +### HDD mount mode for restore + +Changing from `:ro` to `:rw` is acceptable because: +- The controller is a trusted, operator-managed component +- It already has Docker socket access (much more privileged than filesystem write) +- The `:ro` mount was a defense-in-depth measure, not a hard security boundary +- Without `:rw`, restore would require either a sidecar container or host-level orchestration, adding significant complexity for minimal security benefit + +### Restore concurrency + +Backup and restore both use the same restic repository. Running them concurrently can cause lock contention or corruption. A simple `sync.Mutex` in the backup Manager prevents this. The mutex is held for the duration of either operation. \ No newline at end of file