7abd1c5954
All felhom-managed data on external drives now lives under felhom-data/ subdirectory, cleanly separating controller data from user files. - backup/paths.go: add FelhomDataDir constant, update 8 path helpers - stacks/delete.go: add local felhomDataDir constant (circular import boundary), update ProtectedHDDPaths + GetStackBackupData - storage/migrate_drive.go: import backup pkg, fix conflict check, verify, rsync excludes (felhom-data/backups/*/restic/), size estimation - storage/migrate.go: import backup pkg, fix DB dump paths - web/handlers.go: fix legacy 'storage' path -> backup.AppDataDir() - storage/format_linux.go: create felhom-data/ instead of storage/ - storage/attach_linux.go: create felhom-data/ instead of storage/ - scripts/felhom-wipe.sh: new multi-level test node wipe script (soft/controller/full/nuclear) - CHANGELOG.md, controller/README.md, scripts/README.md: updated docs Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
844 lines
27 KiB
Markdown
844 lines
27 KiB
Markdown
# TASK: Storage Namespace (`felhom-data/`) + Test Node Wipe Script (v0.26.0)
|
|
|
|
**Controller:** v0.25.0 → v0.26.0
|
|
|
|
## Overview
|
|
|
|
All felhom-managed data on external drives moves under a `felhom-data/` subdirectory, cleanly separating our data from user files. Plus a wipe script for repeatable testing.
|
|
|
|
**Key design principle:** `HDD_PATH` env var stays as the mount point (e.g., `/mnt/hdd_1`). The `felhom-data` segment is embedded in path helpers and compose templates, not in `HDD_PATH`.
|
|
|
|
---
|
|
|
|
## Phase 1: Path Helpers (`internal/backup/paths.go`)
|
|
|
|
This is the foundation. All other changes depend on this.
|
|
|
|
### File: `controller/internal/backup/paths.go`
|
|
|
|
**Add constant at top of file (after imports):**
|
|
```go
|
|
// FelhomDataDir is the namespace directory on storage drives for all felhom-managed data.
|
|
const FelhomDataDir = "felhom-data"
|
|
```
|
|
|
|
**Update 8 functions** to include `FelhomDataDir` in the path. Each function gets `FelhomDataDir` inserted between `drivePath` and the next segment:
|
|
|
|
| Function | Current first segment | New first segment |
|
|
|----------|----------------------|-------------------|
|
|
| `PrimaryBackupPath` | `"backups"` | `FelhomDataDir, "backups"` |
|
|
| `PrimaryResticRepoPath` | `"backups"` | `FelhomDataDir, "backups"` |
|
|
| `AppDBDumpPath` | `"backups"` | `FelhomDataDir, "backups"` |
|
|
| `SecondaryBackupPath` | `"backups"` | `FelhomDataDir, "backups"` |
|
|
| `AppSecondaryRsyncPath` | `"backups"` | `FelhomDataDir, "backups"` |
|
|
| `SecondaryResticRepoPath` | `"backups"` | `FelhomDataDir, "backups"` |
|
|
| `SecondaryInfraPath` | `"backups"` | `FelhomDataDir, "backups"` |
|
|
| `AppDataDir` | `"appdata"` | `FelhomDataDir, "appdata"` |
|
|
|
|
**DO NOT change `InfraBackupDir`** — it stays at drive root (DR scanner needs it).
|
|
|
|
Example transformation for each function — replace:
|
|
```go
|
|
return filepath.Join(drivePath, "backups", "primary")
|
|
```
|
|
with:
|
|
```go
|
|
return filepath.Join(drivePath, FelhomDataDir, "backups", "primary")
|
|
```
|
|
|
|
And for `AppDataDir` replace:
|
|
```go
|
|
return filepath.Join(drivePath, "appdata", stackName)
|
|
```
|
|
with:
|
|
```go
|
|
return filepath.Join(drivePath, FelhomDataDir, "appdata", stackName)
|
|
```
|
|
|
|
---
|
|
|
|
## Phase 2: Fix Hardcoded Paths
|
|
|
|
### 2a. `controller/internal/stacks/delete.go` — `ProtectedHDDPaths()`
|
|
|
|
**Line 54-65.** Cannot import `backup` package (architectural boundary via `StackDataProvider`).
|
|
|
|
Add local constant at top of file (after imports, before types):
|
|
```go
|
|
// felhomDataDir matches backup.FelhomDataDir — duplicated to avoid circular import via StackDataProvider.
|
|
const felhomDataDir = "felhom-data"
|
|
```
|
|
|
|
Update `ProtectedHDDPaths()`:
|
|
```go
|
|
func ProtectedHDDPaths(hddPath string) map[string]bool {
|
|
if hddPath == "" {
|
|
return nil
|
|
}
|
|
return map[string]bool{
|
|
hddPath: true,
|
|
filepath.Join(hddPath, felhomDataDir): true,
|
|
filepath.Join(hddPath, felhomDataDir, "appdata"): true,
|
|
filepath.Join(hddPath, felhomDataDir, "backups"): true,
|
|
filepath.Join(hddPath, "media"): true,
|
|
filepath.Join(hddPath, "Dokumentumok"): true,
|
|
}
|
|
}
|
|
```
|
|
|
|
### 2b. `controller/internal/stacks/delete.go` — `GetStackBackupData()`
|
|
|
|
**Line 355 and 359.** Replace hardcoded paths with the local constant:
|
|
|
|
Replace:
|
|
```go
|
|
dbDumpPath := filepath.Join(drivePath, "backups", "primary", name, "db-dumps")
|
|
```
|
|
with:
|
|
```go
|
|
dbDumpPath := filepath.Join(drivePath, felhomDataDir, "backups", "primary", name, "db-dumps")
|
|
```
|
|
|
|
Replace:
|
|
```go
|
|
rsyncPath := filepath.Join(drivePath, "backups", "secondary", name, "rsync")
|
|
```
|
|
with:
|
|
```go
|
|
rsyncPath := filepath.Join(drivePath, felhomDataDir, "backups", "secondary", name, "rsync")
|
|
```
|
|
|
|
### 2c. `controller/internal/storage/migrate_drive.go` — Conflict check & verify
|
|
|
|
**Import `backup` package** (safe — `backup` does NOT import `storage`):
|
|
```go
|
|
"gitea.dooplex.hu/admin/felhom-controller/internal/backup"
|
|
```
|
|
|
|
**Line 183** — conflict check before migration. Replace:
|
|
```go
|
|
destAppData := filepath.Join(req.DestPath, "appdata", app.Name)
|
|
```
|
|
with:
|
|
```go
|
|
destAppData := backup.AppDataDir(req.DestPath, app.Name)
|
|
```
|
|
|
|
**Line 325** — verify after copy. Replace:
|
|
```go
|
|
destAppData := filepath.Join(req.DestPath, "appdata", app.Name)
|
|
```
|
|
with:
|
|
```go
|
|
destAppData := backup.AppDataDir(req.DestPath, app.Name)
|
|
```
|
|
|
|
### 2d. `controller/internal/storage/migrate_drive.go` — rsync exclude paths
|
|
|
|
**CRITICAL BUG FIX.** Line 254-256. The rsync copies the entire drive root. The exclude patterns are relative paths. After namespace change, restic repos are under `felhom-data/backups/*/restic/`.
|
|
|
|
Replace:
|
|
```go
|
|
rsyncCmd := exec.CommandContext(ctx, "rsync", "-a", "--info=progress2",
|
|
"--exclude=backups/primary/restic/",
|
|
"--exclude=backups/secondary/restic/",
|
|
req.SourcePath+"/", req.DestPath+"/",
|
|
)
|
|
```
|
|
with:
|
|
```go
|
|
rsyncCmd := exec.CommandContext(ctx, "rsync", "-a", "--info=progress2",
|
|
"--exclude=felhom-data/backups/primary/restic/",
|
|
"--exclude=felhom-data/backups/secondary/restic/",
|
|
req.SourcePath+"/", req.DestPath+"/",
|
|
)
|
|
```
|
|
|
|
Use `backup.FelhomDataDir` constant in the string construction if preferred, but since these are string literals for rsync args, hardcoding `"felhom-data"` is acceptable.
|
|
|
|
### 2e. `controller/internal/storage/migrate_drive.go` — Size estimation
|
|
|
|
**BUG FIX.** Lines 196-208. This scans the drive root for size estimation. After namespace change, `entry.Name() == "backups"` never matches because backups are now under `felhom-data/`.
|
|
|
|
Replace the size estimation block:
|
|
```go
|
|
// Estimate total size (exclude restic repos)
|
|
var totalBytes int64
|
|
entries, _ := os.ReadDir(req.SourcePath)
|
|
for _, entry := range entries {
|
|
entryPath := filepath.Join(req.SourcePath, entry.Name())
|
|
if entry.IsDir() {
|
|
// Skip restic repos in size estimate
|
|
if entry.Name() == "backups" {
|
|
totalBytes += dirSizeExcluding(entryPath, "restic")
|
|
} else {
|
|
totalBytes += dirSize(entryPath)
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
with:
|
|
```go
|
|
// Estimate total size (exclude restic repos inside felhom-data/backups/)
|
|
var totalBytes int64
|
|
entries, _ := os.ReadDir(req.SourcePath)
|
|
for _, entry := range entries {
|
|
if !entry.IsDir() {
|
|
continue
|
|
}
|
|
entryPath := filepath.Join(req.SourcePath, entry.Name())
|
|
if entry.Name() == backup.FelhomDataDir {
|
|
// Scan inside namespace dir, excluding restic repos from estimate
|
|
subEntries, _ := os.ReadDir(entryPath)
|
|
for _, sub := range subEntries {
|
|
if !sub.IsDir() {
|
|
continue
|
|
}
|
|
subPath := filepath.Join(entryPath, sub.Name())
|
|
if sub.Name() == "backups" {
|
|
totalBytes += dirSizeExcluding(subPath, "restic")
|
|
} else {
|
|
totalBytes += dirSize(subPath)
|
|
}
|
|
}
|
|
} else {
|
|
totalBytes += dirSize(entryPath)
|
|
}
|
|
}
|
|
```
|
|
|
|
### 2f. `controller/internal/storage/migrate.go` — Post-migration DB dump copy
|
|
|
|
**Lines 397-398.** Add import for `backup` package (same import section as 2c — `storage` package can safely import `backup`).
|
|
|
|
Replace:
|
|
```go
|
|
srcDBDumps := filepath.Join(req.CurrentHDDPath, "backups", "primary", req.StackName, "db-dumps")
|
|
dstDBDumps := filepath.Join(req.TargetPath, "backups", "primary", req.StackName, "db-dumps")
|
|
```
|
|
with:
|
|
```go
|
|
srcDBDumps := backup.AppDBDumpPath(req.CurrentHDDPath, req.StackName)
|
|
dstDBDumps := backup.AppDBDumpPath(req.TargetPath, req.StackName)
|
|
```
|
|
|
|
Add to the import block:
|
|
```go
|
|
"gitea.dooplex.hu/admin/felhom-controller/internal/backup"
|
|
```
|
|
|
|
### 2g. `controller/internal/web/handlers.go` — Legacy "storage" path (pre-existing bug)
|
|
|
|
**Line 1223.** Replace legacy dead code that uses `"storage"` instead of `"appdata"`.
|
|
|
|
The `backup` package is already imported in `handlers.go`, so use the helper directly:
|
|
|
|
Replace:
|
|
```go
|
|
appDataDir := filepath.Join(storagePath, "storage", stack.Name)
|
|
```
|
|
with:
|
|
```go
|
|
appDataDir := backup.AppDataDir(storagePath, stack.Name)
|
|
```
|
|
|
|
No import addition needed.
|
|
|
|
---
|
|
|
|
## Phase 3: Format & Attach — Create `felhom-data/` Directory
|
|
|
|
### 3a. `controller/internal/storage/format_linux.go`
|
|
|
|
**Line 211.** Replace `"storage"` with `"felhom-data"` in the subdirectory creation list:
|
|
|
|
Replace:
|
|
```go
|
|
for _, subdir := range []string{"storage", "Dokumentumok"} {
|
|
```
|
|
with:
|
|
```go
|
|
for _, subdir := range []string{"felhom-data", "Dokumentumok"} {
|
|
```
|
|
|
|
### 3b. `controller/internal/storage/attach_linux.go`
|
|
|
|
**Line 313.** Same change:
|
|
|
|
Replace:
|
|
```go
|
|
for _, subdir := range []string{"storage", "Dokumentumok"} {
|
|
```
|
|
with:
|
|
```go
|
|
for _, subdir := range []string{"felhom-data", "Dokumentumok"} {
|
|
```
|
|
|
|
---
|
|
|
|
## Phase 4: Verification Grep
|
|
|
|
After making all the above changes, run these greps to verify no hardcoded paths remain:
|
|
|
|
```bash
|
|
grep -rn '"appdata"' controller/internal/ --include='*.go' | grep -v '_test.go\|paths.go\|CLAUDE\|README'
|
|
grep -rn '"backups"' controller/internal/ --include='*.go' | grep -v '_test.go\|paths.go\|CLAUDE\|README'
|
|
```
|
|
|
|
**Expected remaining hits** (all OK — not filesystem path construction):
|
|
- `web/handlers.go` — `"backups"` as page name for template routing (display context)
|
|
- `web/templates/layout.html` — navigation link text
|
|
- `web/templates/backups.html` — template define name
|
|
- Any template/HTML display contexts
|
|
|
|
**Any remaining `filepath.Join(*, "appdata", *)` or `filepath.Join(*, "backups", *)` outside of `paths.go` is a BUG** — fix it.
|
|
|
|
---
|
|
|
|
## Phase 5: Verification of Auto-Propagating Changes
|
|
|
|
These components use path helpers and automatically get the namespace change from Phase 1:
|
|
|
|
### 5a. `controller/internal/backup/crossdrive.go`
|
|
|
|
The `syncInfraConfig()` method (line 504) uses `SecondaryInfraPath(dest)` — auto-updated.
|
|
The `runRsyncBackup()` method (line 367) uses `AppSecondaryRsyncPath()` — auto-updated.
|
|
The `copyStackDBDumps()` method (line 455) uses `AppDBDumpPath()` — auto-updated.
|
|
|
|
**No changes needed.** Just verify by reading the file.
|
|
|
|
### 5b. `controller/internal/backup/restore.go`
|
|
|
|
Line 65: `AppDBDumpPath(drivePath, stackName)` — auto-updated.
|
|
Line 84: `PrimaryResticRepoPath(drivePath)` — auto-updated.
|
|
|
|
**No changes needed.**
|
|
|
|
### 5c. `controller/internal/backup/backup.go`
|
|
|
|
Multiple uses of `AppDataDir()`, `AppDBDumpPath()`, `PrimaryResticRepoPath()` — all auto-updated.
|
|
|
|
**No changes needed.**
|
|
|
|
---
|
|
|
|
## Phase 6: Build & Verify
|
|
|
|
```bash
|
|
cd controller && go build ./... && go vet ./...
|
|
```
|
|
|
|
Must compile cleanly with no errors or warnings.
|
|
|
|
---
|
|
|
|
## Phase 7: Wipe Script
|
|
|
|
### File: `scripts/felhom-wipe.sh`
|
|
|
|
Create the following script. It must be executable (`chmod +x`).
|
|
|
|
```bash
|
|
#!/usr/bin/env bash
|
|
set -euo pipefail
|
|
|
|
# ===================================================================
|
|
# felhom-wipe.sh — Clean felhom data from a test node
|
|
# Usage: ./felhom-wipe.sh --level <soft|controller|full|nuclear> [--yes]
|
|
# ===================================================================
|
|
|
|
# --- Colors ---
|
|
RED='\033[0;31m'
|
|
YELLOW='\033[1;33m'
|
|
GREEN='\033[0;32m'
|
|
CYAN='\033[0;36m'
|
|
BOLD='\033[1m'
|
|
NC='\033[0m'
|
|
|
|
# --- Defaults ---
|
|
LEVEL=""
|
|
DRY_RUN=true
|
|
INCLUDE_PROTECTED=false
|
|
|
|
# --- Configuration (auto-detected) ---
|
|
CONTROLLER_YAML="/opt/docker/felhom-controller/controller.yaml"
|
|
DATA_DIR="/opt/docker/felhom-controller/data"
|
|
COMPOSE_DIR="/opt/docker/felhom-controller"
|
|
STACKS_DIR="/opt/docker/stacks"
|
|
SETTINGS_JSON="$DATA_DIR/settings.json"
|
|
|
|
# --- Helpers ---
|
|
die() { echo -e "${RED}ERROR: $1${NC}" >&2; exit 1; }
|
|
info() { echo -e "${GREEN}$1${NC}"; }
|
|
warn() { echo -e "${YELLOW}$1${NC}"; }
|
|
bold() { echo -e "${BOLD}$1${NC}"; }
|
|
|
|
human_size() {
|
|
local path="$1"
|
|
if [ -e "$path" ]; then
|
|
du -sh "$path" 2>/dev/null | cut -f1 || echo "?"
|
|
else
|
|
echo "n/a"
|
|
fi
|
|
}
|
|
|
|
usage() {
|
|
cat <<EOF
|
|
Usage: $(basename "$0") --level <level> [--yes] [--include-protected]
|
|
|
|
Levels:
|
|
soft Controller state only (settings.json, metrics.db, session data)
|
|
controller Soft + remove all app containers, volumes, stack dirs, app.yaml files
|
|
full Controller + felhom-data/ on all drives (appdata, backups)
|
|
nuclear Full + controller.yaml, controller container, Traefik, Portainer, all Docker data
|
|
|
|
Options:
|
|
--yes Execute the wipe (default: dry run)
|
|
--include-protected Also remove protected stacks (controller level only)
|
|
|
|
EOF
|
|
exit 1
|
|
}
|
|
|
|
# --- Parse Args ---
|
|
parse_args() {
|
|
while [[ $# -gt 0 ]]; do
|
|
case "$1" in
|
|
--level) LEVEL="$2"; shift 2 ;;
|
|
--yes) DRY_RUN=false; shift ;;
|
|
--include-protected) INCLUDE_PROTECTED=true; shift ;;
|
|
-h|--help) usage ;;
|
|
*) die "Unknown argument: $1" ;;
|
|
esac
|
|
done
|
|
|
|
[[ -z "$LEVEL" ]] && usage
|
|
case "$LEVEL" in
|
|
soft|controller|full|nuclear) ;;
|
|
*) die "Invalid level: $LEVEL (must be soft|controller|full|nuclear)" ;;
|
|
esac
|
|
}
|
|
|
|
# --- Detect Paths ---
|
|
detect_paths() {
|
|
# Auto-detect from controller.yaml if present
|
|
if [ -f "$CONTROLLER_YAML" ]; then
|
|
local sd
|
|
sd=$(grep -oP 'stacks_dir:\s*\K\S+' "$CONTROLLER_YAML" 2>/dev/null || true)
|
|
[ -n "$sd" ] && STACKS_DIR="$sd"
|
|
local dd
|
|
dd=$(grep -oP 'data_dir:\s*\K\S+' "$CONTROLLER_YAML" 2>/dev/null || true)
|
|
[ -n "$dd" ] && DATA_DIR="$dd" && SETTINGS_JSON="$dd/settings.json"
|
|
fi
|
|
}
|
|
|
|
# --- Detect Storage Paths ---
|
|
declare -a STORAGE_PATHS=()
|
|
|
|
detect_storage_paths() {
|
|
# From settings.json
|
|
if [ -f "$SETTINGS_JSON" ]; then
|
|
while IFS= read -r p; do
|
|
[ -n "$p" ] && STORAGE_PATHS+=("$p")
|
|
done < <(python3 -c "
|
|
import json, sys
|
|
try:
|
|
d = json.load(open('$SETTINGS_JSON'))
|
|
for sp in d.get('storage_paths', []):
|
|
print(sp.get('path', ''))
|
|
except: pass
|
|
" 2>/dev/null || true)
|
|
fi
|
|
|
|
# Also scan /mnt/* for felhom-data dirs not in registry
|
|
for d in /mnt/*/; do
|
|
[ -d "${d}felhom-data" ] || [ -d "${d}appdata" ] || continue
|
|
local already=false
|
|
for sp in "${STORAGE_PATHS[@]:-}"; do
|
|
[ "$sp" = "${d%/}" ] && already=true && break
|
|
done
|
|
$already || STORAGE_PATHS+=("${d%/}")
|
|
done
|
|
}
|
|
|
|
# --- List App Containers (non-infra) ---
|
|
list_app_containers() {
|
|
docker ps -a --format '{{.Names}}' 2>/dev/null | grep -v -E '^(felhom-controller|traefik|cloudflared|portainer)$' || true
|
|
}
|
|
|
|
# --- List App Volumes (non-infra) ---
|
|
list_app_volumes() {
|
|
docker volume ls -q 2>/dev/null | grep -v -E '^(portainer_data)$' || true
|
|
}
|
|
|
|
# --- Protected Stacks ---
|
|
get_protected_stacks() {
|
|
if [ -f "$CONTROLLER_YAML" ]; then
|
|
grep -A 20 'protected_stacks:' "$CONTROLLER_YAML" 2>/dev/null | grep -oP '^\s*-\s*\K\S+' || true
|
|
fi
|
|
}
|
|
|
|
# --- Print Plan ---
|
|
print_plan() {
|
|
echo ""
|
|
bold "=== Felhom Wipe — Level: $LEVEL ==="
|
|
echo ""
|
|
|
|
# State files
|
|
echo -e "${CYAN}Controller state:${NC}"
|
|
local state_files=("$DATA_DIR/settings.json" "$DATA_DIR/metrics.db" "$DATA_DIR/setup-state.json" "$DATA_DIR/update-state.json" "$DATA_DIR/session-data.json" "$DATA_DIR/snapshot-history.json")
|
|
for f in "${state_files[@]}"; do
|
|
if [ -f "$f" ]; then
|
|
echo -e " ${YELLOW}DELETE${NC} $f ($(human_size "$f"))"
|
|
fi
|
|
done
|
|
|
|
if [[ "$LEVEL" == "controller" || "$LEVEL" == "full" || "$LEVEL" == "nuclear" ]]; then
|
|
echo ""
|
|
echo -e "${CYAN}Docker containers:${NC}"
|
|
local containers
|
|
containers=$(list_app_containers)
|
|
if [ -n "$containers" ]; then
|
|
echo "$containers" | while read -r c; do
|
|
echo -e " ${YELLOW}REMOVE${NC} $c"
|
|
done
|
|
else
|
|
echo -e " ${GREEN}(none)${NC}"
|
|
fi
|
|
|
|
echo ""
|
|
echo -e "${CYAN}Docker volumes:${NC}"
|
|
local volumes
|
|
volumes=$(list_app_volumes)
|
|
if [ -n "$volumes" ]; then
|
|
echo "$volumes" | while read -r v; do
|
|
echo -e " ${YELLOW}REMOVE${NC} $v"
|
|
done
|
|
else
|
|
echo -e " ${GREEN}(none)${NC}"
|
|
fi
|
|
|
|
echo ""
|
|
echo -e "${CYAN}Stack directories:${NC}"
|
|
if [ -d "$STACKS_DIR" ]; then
|
|
for sd in "$STACKS_DIR"/*/; do
|
|
[ -d "$sd" ] || continue
|
|
local stack_name
|
|
stack_name=$(basename "$sd")
|
|
local protected_stacks
|
|
protected_stacks=$(get_protected_stacks)
|
|
if echo "$protected_stacks" | grep -qx "$stack_name" && ! $INCLUDE_PROTECTED; then
|
|
echo -e " ${GREEN}KEEP${NC} $sd (protected)"
|
|
else
|
|
echo -e " ${YELLOW}DELETE${NC} $sd"
|
|
fi
|
|
done
|
|
else
|
|
echo -e " ${GREEN}(not found)${NC}"
|
|
fi
|
|
fi
|
|
|
|
if [[ "$LEVEL" == "full" || "$LEVEL" == "nuclear" ]]; then
|
|
echo ""
|
|
echo -e "${CYAN}Storage data:${NC}"
|
|
if [ ${#STORAGE_PATHS[@]} -gt 0 ]; then
|
|
for sp in "${STORAGE_PATHS[@]}"; do
|
|
if [ -d "$sp/felhom-data" ]; then
|
|
echo -e " ${YELLOW}DELETE${NC} $sp/felhom-data/ ($(human_size "$sp/felhom-data"))"
|
|
fi
|
|
# Old-style paths
|
|
if [ -d "$sp/appdata" ]; then
|
|
echo -e " ${YELLOW}DELETE${NC} $sp/appdata/ ($(human_size "$sp/appdata")) [old-style]"
|
|
fi
|
|
if [ -d "$sp/backups" ]; then
|
|
echo -e " ${YELLOW}DELETE${NC} $sp/backups/ ($(human_size "$sp/backups")) [old-style]"
|
|
fi
|
|
done
|
|
else
|
|
echo -e " ${GREEN}(no storage paths found)${NC}"
|
|
fi
|
|
fi
|
|
|
|
if [[ "$LEVEL" == "nuclear" ]]; then
|
|
echo ""
|
|
echo -e "${RED}Nuclear:${NC}"
|
|
echo -e " ${RED}DELETE${NC} controller.yaml"
|
|
echo -e " ${RED}DELETE${NC} controller container + image"
|
|
echo -e " ${RED}DELETE${NC} Traefik container"
|
|
echo -e " ${RED}DELETE${NC} Cloudflared container"
|
|
echo -e " ${RED}DELETE${NC} Portainer container + volume"
|
|
echo -e " ${RED}DELETE${NC} .felhom-infra-backup/ (DR markers on all drives)"
|
|
echo -e " ${RED}DELETE${NC} All Docker data (docker system prune -af --volumes)"
|
|
fi
|
|
|
|
echo ""
|
|
echo -e "${CYAN}Will preserve:${NC}"
|
|
echo -e " ${GREEN}- OS and system files${NC}"
|
|
if [[ "$LEVEL" != "nuclear" ]]; then
|
|
echo -e " ${GREEN}- Controller container (felhom-controller)${NC}"
|
|
echo -e " ${GREEN}- Controller image${NC}"
|
|
echo -e " ${GREEN}- Traefik, Cloudflare Tunnel${NC}"
|
|
echo -e " ${GREEN}- controller.yaml${NC}"
|
|
echo -e " ${GREEN}- .felhom-infra-backup/ (DR markers on drives)${NC}"
|
|
fi
|
|
if [[ "$LEVEL" != "full" && "$LEVEL" != "nuclear" ]]; then
|
|
echo -e " ${GREEN}- Storage data on drives${NC}"
|
|
fi
|
|
echo -e " ${GREEN}- User files (Dokumentumok, media, etc.)${NC}"
|
|
echo ""
|
|
}
|
|
|
|
# --- Wipe Functions ---
|
|
|
|
do_soft_wipe() {
|
|
info "Soft wipe: removing controller state..."
|
|
local state_files=("$DATA_DIR/settings.json" "$DATA_DIR/metrics.db" "$DATA_DIR/setup-state.json" "$DATA_DIR/update-state.json" "$DATA_DIR/session-data.json" "$DATA_DIR/snapshot-history.json")
|
|
for f in "${state_files[@]}"; do
|
|
[ -f "$f" ] && rm -f "$f" && info " Removed: $f"
|
|
done
|
|
}
|
|
|
|
do_controller_wipe() {
|
|
do_soft_wipe
|
|
|
|
info "Controller wipe: stopping and removing app containers..."
|
|
|
|
# Stop and remove app containers
|
|
local containers
|
|
containers=$(list_app_containers)
|
|
if [ -n "$containers" ]; then
|
|
echo "$containers" | while read -r c; do
|
|
docker rm -f "$c" 2>/dev/null && info " Removed container: $c" || warn " Failed to remove: $c"
|
|
done
|
|
fi
|
|
|
|
# Remove app volumes
|
|
info "Removing app volumes..."
|
|
local volumes
|
|
volumes=$(list_app_volumes)
|
|
if [ -n "$volumes" ]; then
|
|
echo "$volumes" | while read -r v; do
|
|
docker volume rm "$v" 2>/dev/null && info " Removed volume: $v" || warn " Failed to remove: $v"
|
|
done
|
|
fi
|
|
|
|
# Remove stack directories
|
|
info "Removing stack directories..."
|
|
if [ -d "$STACKS_DIR" ]; then
|
|
local protected_stacks
|
|
protected_stacks=$(get_protected_stacks)
|
|
for sd in "$STACKS_DIR"/*/; do
|
|
[ -d "$sd" ] || continue
|
|
local stack_name
|
|
stack_name=$(basename "$sd")
|
|
if echo "$protected_stacks" | grep -qx "$stack_name" && ! $INCLUDE_PROTECTED; then
|
|
warn " Skipping protected stack: $stack_name"
|
|
continue
|
|
fi
|
|
rm -rf "$sd" && info " Removed: $sd"
|
|
done
|
|
fi
|
|
|
|
# NOTE: No restart here — callers handle restart after all cleanup is done.
|
|
}
|
|
|
|
do_full_wipe() {
|
|
do_controller_wipe
|
|
|
|
info "Full wipe: removing storage data..."
|
|
for sp in "${STORAGE_PATHS[@]}"; do
|
|
# New-style namespace
|
|
if [ -d "$sp/felhom-data" ]; then
|
|
rm -rf "$sp/felhom-data" && info " Removed: $sp/felhom-data/"
|
|
fi
|
|
# Old-style paths
|
|
if [ -d "$sp/appdata" ]; then
|
|
rm -rf "$sp/appdata" && info " Removed: $sp/appdata/ [old-style]"
|
|
fi
|
|
if [ -d "$sp/backups" ]; then
|
|
rm -rf "$sp/backups" && info " Removed: $sp/backups/ [old-style]"
|
|
fi
|
|
done
|
|
|
|
# Restart controller after all cleanup is done
|
|
info "Restarting controller..."
|
|
docker restart felhom-controller 2>/dev/null || warn "Could not restart controller"
|
|
}
|
|
|
|
do_nuclear_wipe() {
|
|
do_full_wipe
|
|
|
|
info "Nuclear wipe: removing all infrastructure..."
|
|
|
|
# Stop infrastructure containers
|
|
for c in felhom-controller traefik cloudflared portainer; do
|
|
docker rm -f "$c" 2>/dev/null && info " Removed: $c" || true
|
|
done
|
|
|
|
# Remove controller.yaml
|
|
[ -f "$CONTROLLER_YAML" ] && rm -f "$CONTROLLER_YAML" && info " Removed: controller.yaml"
|
|
|
|
# Remove DR markers (nuclear = brand-new machine simulation)
|
|
for sp in "${STORAGE_PATHS[@]}"; do
|
|
if [ -d "$sp/.felhom-infra-backup" ]; then
|
|
rm -rf "$sp/.felhom-infra-backup" && info " Removed: $sp/.felhom-infra-backup/"
|
|
fi
|
|
done
|
|
|
|
# Remove all Docker data
|
|
warn "Pruning all Docker data..."
|
|
docker system prune -af --volumes 2>/dev/null || warn "Docker prune failed"
|
|
|
|
echo ""
|
|
info "Nuclear wipe complete."
|
|
echo -e "${CYAN}To redeploy, run:${NC}"
|
|
echo " curl -fsSL https://gitea.dooplex.hu/admin/deploy-felhom-compose/raw/branch/main/scripts/docker-setup.sh | bash"
|
|
}
|
|
|
|
# --- Main ---
|
|
main() {
|
|
# Must run as root
|
|
if [ "$(id -u)" -ne 0 ]; then
|
|
die "Must run as root (use sudo)"
|
|
fi
|
|
|
|
# Check Docker
|
|
if ! docker info >/dev/null 2>&1; then
|
|
die "Docker is not running"
|
|
fi
|
|
|
|
parse_args "$@"
|
|
detect_paths
|
|
detect_storage_paths
|
|
print_plan
|
|
|
|
if $DRY_RUN; then
|
|
warn "Dry run — nothing deleted. Use --yes to execute."
|
|
exit 0
|
|
fi
|
|
|
|
# Confirmation
|
|
echo -e "${RED}${BOLD}This will permanently delete the data listed above.${NC}"
|
|
read -rp "Type YES to confirm: " confirm
|
|
if [ "$confirm" != "YES" ]; then
|
|
echo "Aborted."
|
|
exit 1
|
|
fi
|
|
|
|
echo ""
|
|
case "$LEVEL" in
|
|
soft) do_soft_wipe ;;
|
|
controller) do_controller_wipe
|
|
info "Restarting controller..."
|
|
docker restart felhom-controller 2>/dev/null || warn "Could not restart controller"
|
|
;;
|
|
full) do_full_wipe ;;
|
|
nuclear) do_nuclear_wipe ;;
|
|
esac
|
|
|
|
echo ""
|
|
info "Wipe complete (level: $LEVEL)."
|
|
}
|
|
|
|
main "$@"
|
|
```
|
|
|
|
---
|
|
|
|
## Phase 8: Version Bump
|
|
|
|
### File: `controller/cmd/controller/main.go`
|
|
|
|
Find the `Version` constant and update:
|
|
```go
|
|
const Version = "0.26.0"
|
|
```
|
|
|
|
---
|
|
|
|
## Phase 9: Documentation
|
|
|
|
### 9a. CHANGELOG.md
|
|
|
|
Read first 30 lines for format reference, then insert new entry at top of changelog:
|
|
|
|
```markdown
|
|
## v0.26.0 — 2026-02-XX
|
|
|
|
### Changed
|
|
- **Storage namespace**: All felhom-managed data on external drives now lives under `felhom-data/` subdirectory, cleanly separating from user files
|
|
- Path helpers (`paths.go`) updated: 8 functions now include `felhom-data` segment
|
|
- `InfraBackupDir()` unchanged — stays at drive root for DR scanner
|
|
- `ProtectedHDDPaths()` updated with namespace-aware paths
|
|
- Fixed hardcoded paths in `delete.go`, `migrate.go`, `migrate_drive.go`
|
|
- Format and attach wizards now create `felhom-data/` instead of legacy `storage/` subdirectory
|
|
- Fixed legacy `"storage"` path reference in `handlers.go` (was dead code)
|
|
- Drive migration rsync excludes updated for new path structure
|
|
- Drive migration size estimation updated for namespace directory
|
|
|
|
### Added
|
|
- `scripts/felhom-wipe.sh` — Test node cleanup script with 4 wipe levels (soft, controller, full, nuclear)
|
|
- `backup.FelhomDataDir` constant for namespace directory name
|
|
|
|
### Notes
|
|
- Pre-v0.26.0 restic snapshots use old path structure without `felhom-data/`
|
|
- App-catalog compose templates need separate update (see post-implementation task)
|
|
- `HDD_PATH` env var value unchanged — still the mount point (e.g., `/mnt/hdd_1`)
|
|
```
|
|
|
|
### 9b. Controller README.md
|
|
|
|
Update the "Storage Layout" section to show the new `felhom-data/` structure. Update path helper documentation. Add wipe script to scripts section. Add note about pre-v0.26.0 restic snapshot paths.
|
|
|
|
---
|
|
|
|
## Complete File Change List
|
|
|
|
| # | File | Action | Phase |
|
|
|---|------|--------|-------|
|
|
| 1 | `internal/backup/paths.go` | Edit: add constant + update 8 functions | 1 |
|
|
| 2 | `internal/stacks/delete.go` | Edit: add local constant + fix `ProtectedHDDPaths()` + fix `GetStackBackupData()` | 2a, 2b |
|
|
| 3 | `internal/storage/migrate_drive.go` | Edit: add import + fix conflict check + fix verify + fix rsync excludes + fix size estimation | 2c, 2d, 2e |
|
|
| 4 | `internal/storage/migrate.go` | Edit: add import + fix DB dump copy paths | 2f |
|
|
| 5 | `internal/web/handlers.go` | Edit: fix legacy `"storage"` path | 2g |
|
|
| 6 | `internal/storage/format_linux.go` | Edit: replace `"storage"` with `"felhom-data"` | 3a |
|
|
| 7 | `internal/storage/attach_linux.go` | Edit: replace `"storage"` with `"felhom-data"` | 3b |
|
|
| 8 | `scripts/felhom-wipe.sh` | **New file** | 7 |
|
|
| 9 | `cmd/controller/main.go` | Edit: version bump | 8 |
|
|
| 10 | `CHANGELOG.md` | Edit: add v0.26.0 entry | 9a |
|
|
| 11 | `README.md` | Edit: update storage layout + paths + scripts sections | 9b |
|
|
|
|
---
|
|
|
|
## Testing Checklist
|
|
|
|
1. `go build ./...` succeeds
|
|
2. `go vet ./...` passes
|
|
3. `grep -rn '"appdata"' controller/internal/ --include='*.go' | grep -v paths.go` — only display/template contexts
|
|
4. `grep -rn '"backups"' controller/internal/ --include='*.go' | grep -v paths.go` — only display/template contexts
|
|
5. Verify `felhom-wipe.sh` has correct bash syntax: `bash -n scripts/felhom-wipe.sh`
|
|
|
|
---
|
|
|
|
## Post-Implementation: App Catalog Update (SEPARATE TASK)
|
|
|
|
After this task, update `app-catalog-felhom.eu` compose templates:
|
|
```
|
|
${HDD_PATH}/appdata/ → ${HDD_PATH}/felhom-data/appdata/
|
|
${HDD_PATH}/backups/ → ${HDD_PATH}/felhom-data/backups/
|
|
```
|
|
|
|
Media paths (`${HDD_PATH}/media/`) stay at drive root (user data, not felhom-managed).
|
|
|
|
---
|
|
|
|
## Important Reminders
|
|
|
|
- `FelhomDataDir` = `"felhom-data"` (hyphen, not underscore)
|
|
- `InfraBackupDir()` does NOT get namespace — stays at drive root
|
|
- `stacks/delete.go` duplicates the constant (can't import `backup`)
|
|
- `storage/migrate.go` and `storage/migrate_drive.go` CAN import `backup` (no cycle)
|
|
- Wipe script handles BOTH old-style and new-style paths
|
|
- `HDD_PATH` env var in app.yaml stays as mount point — does NOT include `felhom-data`
|