diff --git a/CHANGELOG.md b/CHANGELOG.md index 509c5d9..8c078fe 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,46 @@ +## v0.243.0 — the file manager gets a real password, the backup page tells the truth per tier, an absent tier stops no app, an OOM-killed worker is seen (2026-09-15, R-513 / R-517 / R-518 / R-514) + +**MinAgent: 0.131.0** (the per-tier backup status and tier storage presence are agent v0.131.0; the controller supervisor rides the same release) + +- **R-513 (P1, SECURITY) — FileBrowser no longer accepts `admin` / `admin`.** Measured first + (2026-09-15, `gtstef/filebrowser:1.3.3-stable`, `felhom.eu/documentation/audits/evidence-p1fixes-2026-09-15/B1`): + the config key `auth.adminPassword` / env `FILEBROWSER_ADMIN_PASSWORD` sets the password on a fresh + and an existing database — but RE-APPLIES IT ON EVERY START, overwriting a password set by hand; the + API (`PUT /api/users?id=` with `X-Password: `) changes it once and it sticks. So ONE + mechanism for fresh and existing boxes, the API: every base-stack tick until decided, + `Manager.EnsureFileBrowserAdminPassword` logs in as admin/admin against `http://filebrowser:80`; + **200** → generates `password:16`, sets it, verifies new=200 AND admin=401, stores it AES-encrypted in + settings (`filebrowser_admin_state: generated`); **401** → records `operator` and never touches it + (the HP and the N100 were changed by hand on 2026-09-15). Unreachable → nothing recorded, retried. + The FileBrowser app page shows „Kezdeti belépési adatok": user `admin` and the password behind + „Megjelenítés" (the R-254 reveal: POST, no-store, logged, never in the page), or „az üzemeltető + állította be". Cost, stated: on a brand-new box admin/admin works from FileBrowser's first start until + the next tick. Red-proofs: without the PUT, admin/admin still logs in; without the operator branch, the + page does not say who set it. +- **R-517 (P1) — „Rendszermentés" speaks per tier.** The tile read the agent's single LATEST record, so + a failed 0-byte PBS attempt on absent storage became „✗ · 0 B · PBS · Naprakész" and ticked + „Távoli rendszermentés — külön hardveren". With agent ≥ 0.131.0 the page shows, per tier, the newest + SUCCESSFUL backup (date, size — unknown when read back from storage after a restart), a failed attempt + ✗ „sikertelen" UNDER it, and a tier whose storage does not exist as „nincs beállítva". „Naprakész" is + computed from successes (a set-up tier whose newest success is older than 1.5 × its cadence is + „Esedékes"); the remote tick needs a current PBS SUCCESS. An older agent renders exactly as before. + Red-proof: reading a tier's success from its last attempt loses the local backup's size. +- **R-518 (cheap half) — a tier whose storage does not exist is not attempted, and no app is stopped for + it.** `quiesce.skipAbsentTiers` drops tiers the agent reports `storage: absent` from the manual and the + scheduled run (unknown/legacy never skipped), logs every skip and pushes `backup_tier_skipped` + (warning, operator-only, once per absence). Button copy now tells the truth: „A mentés alatt az + alkalmazások leállnak — általában néhány perc, nagyobb adatnál több." Per-tier quiesce (the other half) + stays open. Red-proof: without the skip, the manual run starts felhom-pbs and the scheduled run stops an + app for it. +- **R-514 — an OOM-killed worker inside a running container is visible.** BIGNIGHT: Paperless's worker + was killed by the memory limit, the container ran on, the app read „Fut". The 30 s dead-app check reads + `State.OOMKilled` for running app containers in one `docker inspect`; the dashboard row shows „Memória + elfogyott" (with a what-to-do title) and the hub gets `app_oom` (warning, operator-only), once per + container run. **The task named the tag „memória elfogyott — újraindítva"; the controller does NOT + restart the app** (an automatic restart is an unmeasured mechanism that could cut a household's + upload), so the tag does not claim it. Red-proof: skipping the `true` lines hides the killed worker. +- Hub ≥ v0.114.0 registers `backup_tier_skipped` and `app_oom`; deploy the hub first. + ## v0.242.0 — a removed app is listed with its kept backup, and five small ones (2026-09-13/14, R-487 / R-491 / R-490 / R-489 / R-476 / R-456) **MinAgent: 0.129.0** (unchanged) diff --git a/controller/README.md b/controller/README.md index 832eacb..f1b56c8 100644 --- a/controller/README.md +++ b/controller/README.md @@ -829,6 +829,8 @@ cloudflared is only deployed when a tunnel token is configured. **Triggers**: a > **Mount prerequisite (Section-G):** the controller writes these stacks under `/opt/docker/stacks` *inside its container*, but `docker compose up` runs on the **guest** Docker daemon. The golden's controller-bootstrap (`felhom-agent` `build-golden.sh`) therefore bind-mounts that path **same-path** (`-v /opt/docker/stacks:/opt/docker/stacks`) so the daemon resolves every relative bind source — without it, all bind-mounted stacks (base infra and customer apps) silently break. +**FileBrowser admin password (v0.243.0, R-513).** FileBrowser used to accept `admin`/`admin` on every box. After `ensureFileBrowser`, every tick until decided, `Manager.EnsureFileBrowserAdminPassword` (`internal/stacks/filebrowser_password.go`) probes `http://filebrowser:80/api/auth/login` with admin/admin: **200** → a generated `password:16` is set through FileBrowser's API (never the `auth.adminPassword` config key — measured to overwrite a hand-set password on every start), verified both ways, and stored AES-encrypted in `settings.json` (`filebrowser_admin_state: generated`); **401** → `operator`, never touched. The FileBrowser app page shows user `admin` and the password behind the R-254 reveal (`POST /apps/filebrowser/initial-credentials/reveal`), or „az üzemeltető állította be". + **Controller routing + the wildcard cert anchor (`wireController` → `RenderControllerRoute`, v0.41.1 / v0.42.1).** filebrowser self-registers with traefik via Docker labels + `traefik-public` membership baked into its compose; the controller can't (it's started by the golden bootstrap *before* `traefik-public` exists, and the v2 `bootstrap.json` carries no domain — that comes from the hub pull). So `EnsureBaseStack` wires the controller **post-pull**: it `docker network connect traefik-public felhom-controller` and writes a traefik file-provider route `dynamic/controller.yml` (`Host(felhom.) → http://felhom-controller:8080`, write-if-changed). When DNS-01 ACME is configured, that route is **also the wildcard-cert anchor**: its router-level `tls.domains: *.` makes traefik **proactively obtain the wildcard `*.` + apex via Cloudflare DNS-01 at startup** (an entrypoint-level `http.tls.domains` does *not* trigger issuance in traefik v3 — only a router-level `tls.domains` does). Every other router then serves that one real wildcard cert by SNI — no per-app `certresolver` labels. This is what lets a LAN client reach the box directly at `*.` with the real cert (the `felhom-agent` split-horizon resolver depends on it). #### Missing Field Injection (`deploy.go`) @@ -983,7 +985,7 @@ height with no magic number to drift as item counts change. | Route | Page | Sections | |-------|------|----------| -| `/backups` | Áttekintés | storage overview, whole-guest Rendszermentés, status stat cards, single-copy warning, **backup-target banner + offer (v0.186.0)** | +| `/backups` | Áttekintés | storage overview, whole-guest Rendszermentés (per tier since v0.243.0, R-517: newest success, a failed attempt under it, „nincs beállítva" for absent storage; „Naprakész" and the remote tick from successes only), status stat cards, single-copy warning, **backup-target banner + offer (v0.186.0)** | | `/backups/remote` | Távoli mentés | Felhom-offsite status card (3 states, display-only), tier-3 status block + quota, participation toggles (+ zero-toggle hint; the persisted zero-toggle run-warning is DISPLAY-replaced by a "kijelölés módosult" note once ≥1 app is toggled — `offboxWarningDisplay`, v0.126.0), manual-target form (`#offbox-section`) | | `/backups/apps` | Alkalmazások | schedule, Adatbázisok table, per-app 1./2./3. tier rows (tier-2 config entry; tier-3 actions deep-link to `/backups/remote#offbox-section`); **since v0.242.0 a REMOVED app whose recovery unit was kept is listed after the deployed rows („Eltávolítva — visszaállítható") with one action, the unit restore that reinstalls it (R-487) — the list is keyed on the drives, not on what is deployed** | | `/backups/restore` | Visszaállítás | restore panel, offsite restore list (**one „Visszaállítás…" entry per app** since v0.154.0), existing verification copies, .fab download/import loop | @@ -1067,6 +1069,10 @@ The nightly backup has two phases that run sequentially. All paths are **per-dri > the three daily legs via `scheduler.UpdateDaily` and takes effect **without a restart**. Precedence: > settings > controller.yaml `db_dump_schedule` > "02:30". See `internal/backupwindow`. +> **Absent-storage tier skip (v0.243.0, R-518).** A tier whose storage the agent (≥ 0.131.0) reports `absent` is dropped from the manual and the scheduled run before anything is stopped (`quiesce.skipAbsentTiers`), logged, and reported once as `backup_tier_skipped`. `unknown` or a legacy agent is never skipped. + +> **OOM visibility (v0.243.0, R-514).** The 30 s dead-app check also reads `State.OOMKilled` for running app containers (`Manager.ScanOOMKilled`); the dashboard shows „Memória elfogyott" and the hub gets `app_oom` once per container run. The controller does not restart the app. + > **Multi-tier whole-guest backup (v0.174.0, R-82 Slice B).** The agent can serve SEVERAL whole-guest > backup tiers with independent cadences — "local daily + PBS weekly" (agent >= v0.97.0, > `GET /backup/tiers`). The controller owns quiescing, so it reconciles them: it collects EVERY due diff --git a/controller/cmd/controller/main.go b/controller/cmd/controller/main.go index b67301d..be68663 100644 --- a/controller/cmd/controller/main.go +++ b/controller/cmd/controller/main.go @@ -788,6 +788,15 @@ func main() { dead, states := scanDeployedAppRunStates(stackMgr, quiesceLoop, appStopGuard) alertMgr.SetDeadAppAlerts(dead) notifier.NotifyAppStartFailures(states) + // R-514: a worker OOM-killed inside a running container leaves the app „Fut". Surface it. + if ooms, oerr := stackMgr.ScanOOMKilled(); oerr != nil { + logger.Printf("[WARN] [deadapp] OOM scan failed: %v", oerr) + } else { + for _, o := range ooms { + logger.Printf("[WARN] [deadapp] %s: container %s was OOM-killed (started %s)", o.Stack, o.Container, o.StartedAt) + notifier.NotifyAppOOM(o.Stack, o.Container, o.StartedAt) + } + } deadAppScans++ noteDeadAppScan(logger, deadAppScans, len(states), len(dead)) noteUnknownIntentSuppressions(logger, states) @@ -2972,7 +2981,7 @@ func (b quiesceBackend) Tiers(ctx context.Context) ([]quiesce.BackupTier, error) } out := make([]quiesce.BackupTier, 0, len(r.Tiers)) for _, t := range r.Tiers { - out = append(out, quiesce.BackupTier{Target: t.Target, Primary: t.Primary}) + out = append(out, quiesce.BackupTier{Target: t.Target, Primary: t.Primary, StorageAbsent: t.Storage == "absent"}) } return out, nil } @@ -3032,6 +3041,11 @@ func (q quiesceTierNotifier) BackupRecovered(tier, message string) { q.n.NotifyWholeGuestBackupRecovered(tier, message) } +// BackupTierSkipped satisfies quiesce.TierSkipNotifier (R-518). +func (q quiesceTierNotifier) BackupTierSkipped(tier, message string) { + q.n.NotifyBackupTierSkipped(tier, message) +} + // startQuiesceLoop wires + starts the slice-8B quiesce loop when the local API is configured and // quiesce is enabled. It Recovers (restarts stacks left stopped by a mid-quiesce crash) before // starting the loop goroutine. Non-fatal: any misconfig disables the loop with a log line. diff --git a/controller/internal/agentapi/backup_tiers.go b/controller/internal/agentapi/backup_tiers.go index 6602b7f..50ac3d7 100644 --- a/controller/internal/agentapi/backup_tiers.go +++ b/controller/internal/agentapi/backup_tiers.go @@ -25,6 +25,9 @@ type BackupTierInfo struct { Target string `json:"target"` CadenceSeconds int64 `json:"cadence_seconds"` Primary bool `json:"primary"` + // Storage (agent >= v0.131.0, R-518): "present" | "absent" | "unknown"; "" on an older agent. + // Only "absent" licenses skipping a tier — unknown and legacy are treated as present. + Storage string `json:"storage,omitempty"` } // TiersResponse mirrors the agent's GET /backup/tiers payload. diff --git a/controller/internal/agentapi/client.go b/controller/internal/agentapi/client.go index 69ef681..2c79dd6 100644 --- a/controller/internal/agentapi/client.go +++ b/controller/internal/agentapi/client.go @@ -199,6 +199,26 @@ type StatusResponse struct { JobID string `json:"job_id"` Error string `json:"error"` Backup *BackupRecord `json:"backup,omitempty"` + // Tiers (agent >= v0.131.0, R-517) — per tier: newest success, last attempt, storage presence. + // Absent on an older agent; the page then falls back to the single latest record. + Tiers []TierBackupState `json:"tiers,omitempty"` +} + +// TierBackupState mirrors the agent's localapi.TierBackupState. +type TierBackupState struct { + Target string `json:"target"` + Primary bool `json:"primary"` + Storage string `json:"storage"` // present | absent | unknown + LastSuccess *BackupRecord `json:"last_success,omitempty"` + LastSuccessSource string `json:"last_success_source,omitempty"` // record | storage + LastAttempt *TierAttempt `json:"last_attempt,omitempty"` +} + +// TierAttempt is a tier's newest recorded attempt. +type TierAttempt struct { + StartedAt string `json:"started_at"` + Success bool `json:"success"` + Error string `json:"error,omitempty"` } // BackupRecord mirrors the agent's hub.Backup — one whole-guest vzdump/PBS backup result. The diff --git a/controller/internal/notify/notifier.go b/controller/internal/notify/notifier.go index e5843c2..717d596 100644 --- a/controller/internal/notify/notifier.go +++ b/controller/internal/notify/notifier.go @@ -40,6 +40,8 @@ type Notifier struct { mu sync.Mutex prevHealthStatus string // tracks previous health check status for change detection + // oomSeen (R-514) remembers container runs already reported as OOM-killed. + oomSeen map[string]bool // appDown tracks which deployed apps are currently in the DOWN state so app_start_failed fires // ONCE per running→down transition, not every health cycle (fix-3 anti-spam). In-memory: a // controller restart re-notifies once (acceptable — better than missing). The hub owns the real @@ -581,6 +583,27 @@ func (n *Notifier) NotifyAppStartFailures(apps []AppRunState) { } } +// NotifyAppOOM — R-514 (v0.243.0). A process inside a running app container was killed by the memory +// limit (Docker State.OOMKilled). Fires ONCE per container run (keyed by StartedAt), so a container +// that stays OOM-marked does not repeat. "warning" — the hub vocabulary (R-329). Operator-only +// hub-side (hub >= v0.114.0 registers it): the household sees the dashboard tag. +func (n *Notifier) NotifyAppOOM(stack, container, startedAt string) { + key := container + "|" + startedAt + n.mu.Lock() + if n.oomSeen == nil { + n.oomSeen = map[string]bool{} + } + if n.oomSeen[key] { + n.mu.Unlock() + return + } + n.oomSeen[key] = true + n.mu.Unlock() + n.emit("app_oom", "warning", + fmt.Sprintf("Alkalmazás memóriája elfogyott: %s (%s) — egy folyamatát a memóriakorlát leállította", stack, container), + AppDetails{StackName: stack, DisplayName: container}) +} + // DiskHealthDetails is the event-detail payload for disk_health_degraded. type DiskHealthDetails struct { Disk string `json:"disk"` @@ -981,6 +1004,13 @@ func (n *Notifier) NotifyWholeGuestBackupFailed(tier, message, errMsg string) { WholeGuestBackupDetails{Tier: tier, Error: errMsg}) } +// NotifyBackupTierSkipped — R-518 (v0.243.0). A whole-guest tier was not attempted because its +// storage does not exist on the host. Operator-only hub-side (hub >= v0.114.0 registers the type in +// allowedEventTypes AND operatorOnlyEvents): the household can do nothing about an unprovisioned DR tier. +func (n *Notifier) NotifyBackupTierSkipped(tier, message string) { + n.PushEvent("backup_tier_skipped", "warning", message, WholeGuestBackupDetails{Tier: tier}) +} + func (n *Notifier) NotifyWholeGuestBackupRecovered(tier, message string) { n.PushEvent("whole_guest_backup_recovered", "info", message, WholeGuestBackupDetails{Tier: tier}) diff --git a/controller/internal/quiesce/quiesce.go b/controller/internal/quiesce/quiesce.go index ca48a3c..80080eb 100644 --- a/controller/internal/quiesce/quiesce.go +++ b/controller/internal/quiesce/quiesce.go @@ -112,6 +112,9 @@ type Loop struct { // tierNotify (R-97a) reports a tier's backup outcome to the hub. nil = not wired (pre-provisioning). // Init-only: set once at startup via SetTierNotifier, before Run. tierNotify TierNotifier + // R-518: tiers skipped for absent storage, noted once per absence (skipAbsentTiers). + skipMu sync.Mutex + skipNoted map[string]bool // suppressed (R-97b) is stack name → grace expiry (zero = still quiesced). Read by // SuppressedStacks so an app WE stopped is not reported to the customer as broken. suppressMu sync.Mutex @@ -431,8 +434,9 @@ func (l *Loop) allTiersForManualRun(ctx context.Context) []dueTier { } return []dueTier{{target: ""}} } - out := make([]dueTier, 0, len(tiers)) - for _, t := range tiers { + kept := l.skipAbsentTiers(tiers) + out := make([]dueTier, 0, len(kept)) + for _, t := range kept { out = append(out, dueTier{target: t.Target}) } return out diff --git a/controller/internal/quiesce/tier_skip_test.go b/controller/internal/quiesce/tier_skip_test.go new file mode 100644 index 0000000..345e27a --- /dev/null +++ b/controller/internal/quiesce/tier_skip_test.go @@ -0,0 +1,81 @@ +package quiesce + +import ( + "context" + "sync" + "testing" + "time" +) + +// R-518 (cheap half) — a tier whose storage the agent reports ABSENT is not attempted, and no app is +// stopped for it. BIGNIGHT: the apps stayed stopped through a felhom-pbs tier whose storage did not +// exist, then that tier failed. + +type skipRecorder struct { + mu sync.Mutex + skipped []string +} + +func (r *skipRecorder) BackupFailed(string, string, string) {} +func (r *skipRecorder) BackupRecovered(string, string) {} +func (r *skipRecorder) BackupTierSkipped(tier, _ string) { + r.mu.Lock() + defer r.mu.Unlock() + r.skipped = append(r.skipped, tier) +} + +// RED-PROOF (run 2026-09-15, recorded in REPORT.md): with skipAbsentTiers returning its input +// unchanged, the manual run started [local felhom-pbs] and this failed at "absent tier was attempted". +func TestManualRun_AbsentTierSkipped(t *testing.T) { + be := newTierBackend() + be.tiers = []BackupTier{{Target: "local", Primary: true}, {Target: "felhom-pbs", StorageAbsent: true}} + be.phases["local"] = []string{phaseDone} + st := &fakeStacks{running: []string{"immich"}} + l := newTierLoop(t, be, st, nil) + rec := &skipRecorder{} + l.SetTierNotifier(rec) + + if err := l.quiesceAndPollTiers(context.Background(), l.allTiersForManualRun(context.Background())); err != nil { + t.Fatalf("manual cycle: %v", err) + } + if got := be.startedTargets(); len(got) != 1 || got[0] != "local" { + t.Fatalf("absent tier was attempted: started=%v", got) + } + if stops, starts := len(st.stoppedNames()), len(st.startedNames()); stops != 1 || starts != 1 { + t.Fatalf("want one stop/start for the local tier, got %d/%d", stops, starts) + } + // A second resolution in the same absence does not notify again. + _ = l.allTiersForManualRun(context.Background()) + if len(rec.skipped) != 1 || rec.skipped[0] != "felhom-pbs" { + t.Fatalf("want exactly one skip notice for felhom-pbs, got %v", rec.skipped) + } +} + +// Only the absent tier is due → no quiesce at all. +func TestScheduled_OnlyAbsentTierDue_NoQuiesce(t *testing.T) { + be := newTierBackend() + be.tiers = []BackupTier{{Target: "local", Primary: true}, {Target: "felhom-pbs", StorageAbsent: true}} + be.dueSet["felhom-pbs"] = true + st := &fakeStacks{running: []string{"immich"}} + l := newTierLoop(t, be, st, nil) + l.now = func() time.Time { return time.Date(2026, 9, 15, 1, 0, 0, 0, time.UTC) } + _ = l.runOnce(context.Background()) + if stops := len(st.stoppedNames()); stops != 0 { + t.Fatalf("an app was stopped for a tier whose storage does not exist (stops=%d)", stops) + } + if got := be.startedTargets(); len(got) != 0 { + t.Fatalf("absent tier started: %v", got) + } +} + +// Unknown/legacy storage (StorageAbsent=false) is never skipped — fail toward backing up. +func TestManualRun_UnknownStorageNotSkipped(t *testing.T) { + be := newTierBackend() + be.tiers = []BackupTier{{Target: "local", Primary: true}, {Target: "felhom-pbs"}} + be.phases["local"] = []string{phaseDone} + be.phases["felhom-pbs"] = []string{phaseDone} + l := newTierLoop(t, be, &fakeStacks{running: []string{"immich"}}, nil) + if got := l.allTiersForManualRun(context.Background()); len(got) != 2 { + t.Fatalf("a tier without an absent verdict was dropped: %v", got) + } +} diff --git a/controller/internal/quiesce/tiers.go b/controller/internal/quiesce/tiers.go index ff00d15..79e7c0c 100644 --- a/controller/internal/quiesce/tiers.go +++ b/controller/internal/quiesce/tiers.go @@ -35,6 +35,43 @@ var ErrTiersUnsupported = errors.New("quiesce: agent does not serve /backup/tier type BackupTier struct { Target string Primary bool + // StorageAbsent (R-518, agent >= v0.131.0): the agent POSITIVELY determined the tier's storage + // does not exist. Only then is the tier skipped; unknown/legacy stay false (fail toward backing up). + StorageAbsent bool +} + +// TierSkipNotifier is the OPTIONAL notifier extension for a tier skipped because its storage does not +// exist (R-518). Optional so every existing TierNotifier keeps compiling. +type TierSkipNotifier interface { + BackupTierSkipped(tier, message string) +} + +// skipAbsentTiers drops the tiers whose storage the agent reports absent (R-518's cheap half, +// BIGNIGHT: the apps stayed stopped through a PBS tier whose storage did not exist, then it failed). +// It logs every skip and notifies ONCE per tier per absence (a scheduled loop re-resolves every poll; +// the note re-arms when the storage reappears). +func (l *Loop) skipAbsentTiers(tiers []BackupTier) []BackupTier { + out := make([]BackupTier, 0, len(tiers)) + l.skipMu.Lock() + defer l.skipMu.Unlock() + if l.skipNoted == nil { + l.skipNoted = map[string]bool{} + } + for _, t := range tiers { + if !t.StorageAbsent { + delete(l.skipNoted, t.Target) + out = append(out, t) + continue + } + l.logger.Printf("[WARN] [quiesce] tier %s skipped: its storage does not exist on the host — no app is stopped for it", tierLabel(t.Target)) + if !l.skipNoted[t.Target] { + l.skipNoted[t.Target] = true + if n, ok := l.tierNotify.(TierSkipNotifier); ok && n != nil { + n.BackupTierSkipped(t.Target, "Whole-guest backup tier "+t.Target+" skipped: its storage does not exist on the host (never provisioned or removed). No app was stopped for it.") + } + } + } + return out } // TieredBackend is the OPTIONAL R-82 extension to Backend. A backend that does not implement it @@ -120,6 +157,14 @@ func (l *Loop) resolveDueTiers(ctx context.Context) (due []dueTier, degraded boo if terr != nil { return nil, false, terr } + if len(tiers) > 0 { + if kept := l.skipAbsentTiers(tiers); len(kept) == 0 { + l.logger.Printf("[WARN] [quiesce] every advertised tier's storage is absent — nothing to back up this cycle") + return nil, false, nil + } else { + tiers = kept + } + } if len(tiers) == 0 { // An agent that advertises no tiers cannot be backed up per-tier, but it can still be // backed up untargeted. Fail toward DOING the backup, never toward skipping it. diff --git a/controller/internal/settings/settings.go b/controller/internal/settings/settings.go index fec6ecb..9560eec 100644 --- a/controller/internal/settings/settings.go +++ b/controller/internal/settings/settings.go @@ -36,6 +36,15 @@ type Settings struct { LauncherShareToken string `json:"launcher_share_token,omitempty"` LauncherSharePasswordHash string `json:"launcher_share_password_hash,omitempty"` + // FileBrowser admin login (R-513, v0.243.0). Every box used to accept admin/admin. The controller + // sets a generated password ONCE through FileBrowser's own API when admin/admin still works, and + // records "set by the operator" when it does not (a hand-set password is never overwritten). + // FileBrowserAdminPasswordEnc is AES-encrypted with the controller's app key (crypto.Encrypt), + // never plaintext. FileBrowserAdminState: "" (not yet decided) | "generated" | "operator". + FileBrowserAdminPasswordEnc string `json:"filebrowser_admin_password_enc,omitempty"` + FileBrowserAdminState string `json:"filebrowser_admin_state,omitempty"` + FileBrowserAdminDecidedAt string `json:"filebrowser_admin_decided_at,omitempty"` // RFC3339 + // Customer-claim arc (v0.122.0, F-4). Claimed is SET-ONLY (a claim or reset completed at // least once — never cleared). ClaimCode* cache the freshest hub-delivered code state (report // ACK; beats controller.yaml when its generation is newer). ClaimConsumedGeneration records @@ -834,6 +843,31 @@ func (s *Settings) SetLauncherSharePasswordHash(hash string) error { return s.save() } +// ── FileBrowser admin login (R-513) ───────────────────────────────────────────── + +const ( + FileBrowserAdminGenerated = "generated" + FileBrowserAdminOperator = "operator" +) + +// GetFileBrowserAdmin returns the recorded decision: state ("" = undecided), the ENCRYPTED password +// (empty unless state is generated) and when it was decided. +func (s *Settings) GetFileBrowserAdmin() (state, passwordEnc, decidedAt string) { + s.mu.RLock() + defer s.mu.RUnlock() + return s.FileBrowserAdminState, s.FileBrowserAdminPasswordEnc, s.FileBrowserAdminDecidedAt +} + +// SetFileBrowserAdmin records the decision and saves. passwordEnc must already be encrypted. +func (s *Settings) SetFileBrowserAdmin(state, passwordEnc, decidedAt string) error { + s.mu.Lock() + defer s.mu.Unlock() + s.FileBrowserAdminState = state + s.FileBrowserAdminPasswordEnc = passwordEnc + s.FileBrowserAdminDecidedAt = decidedAt + return s.save() +} + // ── Hub-held recovery package (v0.199.0, R-204 item 4) ───────────────────────── // GetHubEscrowIdentityPresent reports whether the hub is holding a sealed identity/recovery package diff --git a/controller/internal/stacks/filebrowser_password.go b/controller/internal/stacks/filebrowser_password.go new file mode 100644 index 0000000..65a42c2 --- /dev/null +++ b/controller/internal/stacks/filebrowser_password.go @@ -0,0 +1,166 @@ +package stacks + +import ( + "bytes" + "encoding/json" + "fmt" + "io" + "net/http" + "strings" + "time" + + "gitea.dooplex.hu/admin/felhom-controller/internal/crypto" + "gitea.dooplex.hu/admin/felhom-controller/internal/settings" +) + +// R-513 — FileBrowser's admin password. +// +// MEASURED FIRST (2026-09-15, gtstef/filebrowser:1.3.3-stable, evidence-p1fixes-2026-09-15/B1): +// - with no `auth.adminPassword` key and no FILEBROWSER_ADMIN_PASSWORD env — the controller's +// render — a new database accepts admin/admin; +// - the config key or the env var DOES set the password, on a fresh AND on an existing database — +// but it RE-APPLIES ON EVERY START: a password changed by hand afterwards is overwritten at the +// next restart; +// - the API changes it once and it sticks: `PUT /api/users?id=` with +// `{"which":["password"],"data":{"id":,"username":"admin","password":}}`, the session +// token, and `X-Password: ` → 204. +// +// THE DECISION (one mechanism for fresh and existing boxes): the API path, never the config key. The +// key would silently undo the password the operator set by hand on the HP and the N100 (2026-09-15) +// and any a household sets later. Cost: on a brand-new box admin/admin works from FileBrowser's first +// start until the next base-stack tick sets the password (seconds; before a claim there is no tunnel). +// +// The probe: login admin/admin → 200 ⇒ generate (password:16), PUT, verify new=200 AND admin=401, then +// record "generated" with the encrypted value; 401 ⇒ record "operator" (somebody set it — leave it). +// Anything else (container starting, network) ⇒ record nothing and try again next tick. + +const fileBrowserBaseURL = "http://filebrowser:80" + +// fbHTTPDo is the network seam (tests inject a fake FileBrowser). +type fbHTTPDo func(req *http.Request) (*http.Response, error) + +func (m *Manager) fbDo() fbHTTPDo { + if m.fbHTTP != nil { + return m.fbHTTP + } + c := &http.Client{Timeout: 10 * time.Second} + return c.Do +} + +// fbLogin returns (token, status, err). status 200 → token set; 401 → wrong password. +func fbLogin(do fbHTTPDo, base, password string) (string, int, error) { + req, _ := http.NewRequest(http.MethodPost, base+"/api/auth/login?username=admin", nil) + req.Header.Set("X-Password", password) + resp, err := do(req) + if err != nil { + return "", 0, err + } + defer resp.Body.Close() + b, _ := io.ReadAll(io.LimitReader(resp.Body, 1<<16)) + if resp.StatusCode != http.StatusOK { + return "", resp.StatusCode, nil + } + return strings.Trim(strings.TrimSpace(string(b)), `"`), resp.StatusCode, nil +} + +// EnsureFileBrowserAdminPassword makes the one-time decision. Safe to call every tick: it returns at +// once when a decision is recorded. Returns an error only for logging. +func (m *Manager) EnsureFileBrowserAdminPassword() error { + if m.settings == nil || len(m.encKey) == 0 { + return nil + } + if state, _, _ := m.settings.GetFileBrowserAdmin(); state != "" { + return nil + } + do := m.fbDo() + base := fileBrowserBaseURL + if m.fbBaseURL != "" { + base = m.fbBaseURL + } + now := time.Now().UTC().Format(time.RFC3339) + + token, code, err := fbLogin(do, base, "admin") + if err != nil { + return fmt.Errorf("filebrowser admin probe: %w (will retry)", err) + } + switch code { + case http.StatusOK: + // default login still works — set a generated password below + case http.StatusUnauthorized, http.StatusForbidden: + m.logger.Printf("[INFO] [infra] filebrowser: admin/admin is refused (HTTP %d) — the password was set by someone; leaving it and recording \"operator\"", code) + return m.settings.SetFileBrowserAdmin(settings.FileBrowserAdminOperator, "", now) + default: + return fmt.Errorf("filebrowser admin probe: HTTP %d (will retry)", code) + } + + id, err := fbSelfID(do, base, token) + if err != nil { + return fmt.Errorf("filebrowser: read admin user id: %w (will retry)", err) + } + pw, err := generateValue("password:16") + if err != nil { + return fmt.Errorf("filebrowser: generate password: %w", err) + } + body, _ := json.Marshal(map[string]any{ + "which": []string{"password"}, + "data": map[string]any{"id": id, "username": "admin", "password": pw}, + }) + req, _ := http.NewRequest(http.MethodPut, fmt.Sprintf("%s/api/users?id=%d", base, id), bytes.NewReader(body)) + req.Header.Set("Content-Type", "application/json") + req.Header.Set("X-Auth", token) + req.Header.Set("Authorization", "Bearer "+token) + req.Header.Set("X-Password", "admin") + resp, err := do(req) + if err != nil { + return fmt.Errorf("filebrowser: set password: %w (will retry)", err) + } + io.Copy(io.Discard, io.LimitReader(resp.Body, 1<<16)) + resp.Body.Close() + if resp.StatusCode/100 != 2 { + return fmt.Errorf("filebrowser: set password: HTTP %d (will retry)", resp.StatusCode) + } + // Verify the consequence, both directions, before recording anything. + if _, c, err := fbLogin(do, base, pw); err != nil || c != http.StatusOK { + return fmt.Errorf("filebrowser: new password does not log in (HTTP %d, err %v) — NOT recorded, will retry", c, err) + } + if _, c, err := fbLogin(do, base, "admin"); err != nil || c == http.StatusOK { + return fmt.Errorf("filebrowser: admin/admin still logs in after the change (HTTP %d, err %v) — NOT recorded", c, err) + } + enc, err := crypto.Encrypt(m.encKey, pw) + if err != nil { + return fmt.Errorf("filebrowser: encrypt password: %w", err) + } + if err := m.settings.SetFileBrowserAdmin(settings.FileBrowserAdminGenerated, enc, now); err != nil { + // The password IS changed and we could not record it: say so loudly — the household cannot + // be shown a password that is not stored. Recoverable by the operator (FileBrowser CLI). + m.logger.Printf("[ERROR] [infra] filebrowser: password CHANGED but settings save FAILED: %v — the generated password is lost; reset it with the filebrowser CLI", err) + return err + } + m.logger.Printf("[INFO] [infra] filebrowser: admin/admin replaced by a generated password (value never logged); verified new=200 admin=401") + return nil +} + +// fbSelfID reads the logged-in user's id (GET /api/users?id=self). +func fbSelfID(do fbHTTPDo, base, token string) (int, error) { + req, _ := http.NewRequest(http.MethodGet, base+"/api/users?id=self", nil) + req.Header.Set("X-Auth", token) + req.Header.Set("Authorization", "Bearer "+token) + resp, err := do(req) + if err != nil { + return 0, err + } + defer resp.Body.Close() + if resp.StatusCode != http.StatusOK { + return 0, fmt.Errorf("HTTP %d", resp.StatusCode) + } + var u struct { + ID int `json:"id"` + } + if err := json.NewDecoder(io.LimitReader(resp.Body, 1<<16)).Decode(&u); err != nil { + return 0, err + } + if u.ID <= 0 { + return 0, fmt.Errorf("no user id in response") + } + return u.ID, nil +} diff --git a/controller/internal/stacks/filebrowser_password_test.go b/controller/internal/stacks/filebrowser_password_test.go new file mode 100644 index 0000000..496efa9 --- /dev/null +++ b/controller/internal/stacks/filebrowser_password_test.go @@ -0,0 +1,147 @@ +package stacks + +import ( + "encoding/json" + "io" + "log" + "net/http" + "net/http/httptest" + "path/filepath" + "sync" + "testing" + + "gitea.dooplex.hu/admin/felhom-controller/internal/crypto" + "gitea.dooplex.hu/admin/felhom-controller/internal/settings" +) + +// R-513 — FileBrowser accepted admin/admin on every box. The fake below behaves like the measured +// Quantum 1.3.3 API (evidence-p1fixes-2026-09-15/B1): login by X-Password, PUT /api/users with the +// current password in X-Password. + +type fakeFB struct { + mu sync.Mutex + password string + puts int +} + +func (f *fakeFB) handler() http.Handler { + mux := http.NewServeMux() + mux.HandleFunc("/api/auth/login", func(w http.ResponseWriter, r *http.Request) { + f.mu.Lock() + defer f.mu.Unlock() + if r.Header.Get("X-Password") != f.password { + w.WriteHeader(http.StatusUnauthorized) + return + } + io.WriteString(w, "tok-123") + }) + mux.HandleFunc("/api/users", func(w http.ResponseWriter, r *http.Request) { + f.mu.Lock() + defer f.mu.Unlock() + if r.Header.Get("X-Auth") != "tok-123" { + w.WriteHeader(http.StatusUnauthorized) + return + } + switch r.Method { + case http.MethodGet: + io.WriteString(w, `{"id":1,"username":"admin"}`) + case http.MethodPut: + if r.Header.Get("X-Password") != f.password { + w.WriteHeader(http.StatusUnauthorized) + return + } + var body struct { + Which []string `json:"which"` + Data struct { + Password string `json:"password"` + } `json:"data"` + } + _ = json.NewDecoder(r.Body).Decode(&body) + f.password = body.Data.Password + f.puts++ + w.WriteHeader(http.StatusNoContent) + } + }) + return mux +} + +func fbManager(t *testing.T, base string) (*Manager, *settings.Settings, []byte) { + t.Helper() + st, err := settings.Load(filepath.Join(t.TempDir(), "settings.json"), log.New(io.Discard, "", 0)) + if err != nil { + t.Fatal(err) + } + key := make([]byte, 32) + for i := range key { + key[i] = byte(i + 1) + } + return &Manager{logger: log.New(io.Discard, "", 0), settings: st, encKey: key, fbBaseURL: base}, st, key +} + +// The consequence: after one tick admin/admin no longer logs in, the stored (decrypted) password +// does, and the state is "generated". A second tick changes nothing. +// +// RED-PROOF (run 2026-09-15, recorded in REPORT.md): with EnsureFileBrowserAdminPassword returning +// nil before the PUT, admin/admin stayed valid and this failed at "admin/admin still logs in". +func TestFileBrowserAdmin_DefaultLoginReplaced(t *testing.T) { + fb := &fakeFB{password: "admin"} + srv := httptest.NewServer(fb.handler()) + defer srv.Close() + m, st, key := fbManager(t, srv.URL) + + if err := m.EnsureFileBrowserAdminPassword(); err != nil { + t.Fatalf("ensure: %v", err) + } + if _, c, _ := fbLogin(srv.Client().Do, srv.URL, "admin"); c == http.StatusOK { + t.Fatalf("admin/admin still logs in — R-513 not fixed") + } + state, enc, at := st.GetFileBrowserAdmin() + if state != settings.FileBrowserAdminGenerated || enc == "" || at == "" { + t.Fatalf("decision not recorded: state=%q enc=%v at=%q", state, enc != "", at) + } + if enc == fb.password { + t.Fatal("the password was stored in plaintext") + } + pw, err := crypto.Decrypt(key, enc) + if err != nil || len(pw) != 16 { + t.Fatalf("stored password does not decrypt to a 16-char value (len=%d err=%v)", len(pw), err) + } + if _, c, _ := fbLogin(srv.Client().Do, srv.URL, pw); c != http.StatusOK { + t.Fatalf("the stored password does not log in (HTTP %d)", c) + } + _ = m.EnsureFileBrowserAdminPassword() + if fb.puts != 1 { + t.Fatalf("a recorded decision was acted on again (puts=%d)", fb.puts) + } +} + +// A password set by hand (admin/admin refused) is NEVER overwritten. +func TestFileBrowserAdmin_HandSetPasswordLeftAlone(t *testing.T) { + fb := &fakeFB{password: "operator-set-by-hand"} + srv := httptest.NewServer(fb.handler()) + defer srv.Close() + m, st, _ := fbManager(t, srv.URL) + if err := m.EnsureFileBrowserAdminPassword(); err != nil { + t.Fatal(err) + } + if fb.puts != 0 || fb.password != "operator-set-by-hand" { + t.Fatalf("hand-set password was changed (puts=%d)", fb.puts) + } + if state, enc, _ := st.GetFileBrowserAdmin(); state != settings.FileBrowserAdminOperator || enc != "" { + t.Fatalf("want state operator with no stored value, got %q enc=%v", state, enc != "") + } +} + +// FileBrowser not reachable → nothing recorded, so the next tick tries again. +func TestFileBrowserAdmin_UnreachableRecordsNothing(t *testing.T) { + srv := httptest.NewServer(http.NotFoundHandler()) + url := srv.URL + srv.Close() + m, st, _ := fbManager(t, url) + if err := m.EnsureFileBrowserAdminPassword(); err == nil { + t.Fatal("want an error (for the log) when FileBrowser is unreachable") + } + if state, _, _ := st.GetFileBrowserAdmin(); state != "" { + t.Fatalf("an unreachable FileBrowser recorded a decision: %q", state) + } +} diff --git a/controller/internal/stacks/infra.go b/controller/internal/stacks/infra.go index cfa43a8..d283a06 100644 --- a/controller/internal/stacks/infra.go +++ b/controller/internal/stacks/infra.go @@ -69,6 +69,9 @@ func (m *Manager) EnsureBaseStack() error { if err := m.ensureFileBrowser(filepath.Join(base, "filebrowser")); err != nil { errs = append(errs, fmt.Sprintf("filebrowser: %v", err)) + } else if err := m.EnsureFileBrowserAdminPassword(); err != nil { + // R-513: one-time; retried every tick until decided. Logged, never fatal to the base stack. + m.logger.Printf("[WARN] [infra] %v", err) } // samba (LAN network-sharing, R-7) — conditional deploy, same shape as cloudflared: only when the diff --git a/controller/internal/stacks/manager.go b/controller/internal/stacks/manager.go index 34b2187..19c7636 100644 --- a/controller/internal/stacks/manager.go +++ b/controller/internal/stacks/manager.go @@ -211,6 +211,12 @@ type Manager struct { // answering from a stale entry; pruned every refresh to the live container set. Guarded by mu // (every read/write happens under refreshStatusLocked's write lock). restartPolicyCache map[string]string + // R-514: last OOM scan (oom.go), for the dashboard. + oomMu sync.Mutex + oomCache map[string][]string + // R-513: FileBrowser admin-password seams (filebrowser_password.go); nil/"" in production. + fbHTTP fbHTTPDo + fbBaseURL string // execFn replaces execCommand's process boundary in tests; nil in production. execFn func(name string, args ...string) (string, error) diff --git a/controller/internal/stacks/oom.go b/controller/internal/stacks/oom.go new file mode 100644 index 0000000..59910be --- /dev/null +++ b/controller/internal/stacks/oom.go @@ -0,0 +1,85 @@ +package stacks + +import ( + "sort" + "strings" +) + +// R-514 (v0.243.0) — an OOM-killed worker inside a RUNNING container is invisible to the state model. +// +// BIGNIGHT: Paperless's celery worker was killed by the memory cgroup inside the webserver container; +// the container kept running (`docker inspect` → OOMKilled=true, RestartCount=0), the app read „Fut", +// 11 documents failed and 8 waited forever. Docker's State.OOMKilled is set when ANY process in the +// container's cgroup was OOM-killed and stays set until the container restarts — so it is the +// observable, read for the running containers of deployed stacks. + +// OOMContainer is one container whose cgroup had a process OOM-killed since it started. +type OOMContainer struct { + Stack string + Container string + StartedAt string // identifies the container run — one event per run +} + +// ScanOOMKilled inspects the containers of every deployed, running-ish stack in ONE docker call and +// returns those with State.OOMKilled=true. It also caches the per-stack result for the dashboard. +func (m *Manager) ScanOOMKilled() ([]OOMContainer, error) { + m.mu.RLock() + owner := map[string]string{} + var names []string + for name, st := range m.stacks { + if !st.Deployed { + continue + } + for _, c := range st.Containers { + if c.State == StateRunning || c.State == StateUnhealthy || c.State == StateRestarting { + owner[c.Name] = name + names = append(names, c.Name) + } + } + } + m.mu.RUnlock() + if len(names) == 0 { + m.setOOMCache(nil) + return nil, nil + } + sort.Strings(names) + args := append([]string{"inspect", "-f", "{{.Name}}|{{.State.OOMKilled}}|{{.State.StartedAt}}"}, names...) + out, err := m.execCommand("docker", args...) + if err != nil && strings.TrimSpace(out) == "" { + return nil, err + } + var found []OOMContainer + for _, line := range strings.Split(strings.TrimSpace(out), "\n") { + f := strings.SplitN(strings.TrimSpace(line), "|", 3) + if len(f) != 3 || f[1] != "true" { + continue + } + cname := strings.TrimPrefix(f[0], "/") + if st, ok := owner[cname]; ok { + found = append(found, OOMContainer{Stack: st, Container: cname, StartedAt: f[2]}) + } + } + m.setOOMCache(found) + return found, nil +} + +func (m *Manager) setOOMCache(found []OOMContainer) { + cache := map[string][]string{} + for _, o := range found { + cache[o.Stack] = append(cache[o.Stack], o.Container) + } + m.oomMu.Lock() + m.oomCache = cache + m.oomMu.Unlock() +} + +// OOMKilledStacks returns stack name → OOM-killed container names from the last scan. +func (m *Manager) OOMKilledStacks() map[string][]string { + m.oomMu.Lock() + defer m.oomMu.Unlock() + out := make(map[string][]string, len(m.oomCache)) + for k, v := range m.oomCache { + out[k] = append([]string(nil), v...) + } + return out +} diff --git a/controller/internal/stacks/oom_test.go b/controller/internal/stacks/oom_test.go new file mode 100644 index 0000000..d6e2770 --- /dev/null +++ b/controller/internal/stacks/oom_test.go @@ -0,0 +1,55 @@ +package stacks + +import ( + "io" + "log" + "strings" + "testing" +) + +// R-514 — a worker OOM-killed inside a RUNNING container must be seen. BIGNIGHT: paperless-webserver +// `oomkilled=true restarts=0`, app „Fut", no event. +// +// RED-PROOF (run 2026-09-15, recorded in REPORT.md): with the `f[1] != "true"` filter inverted to +// skip every "true" line, the scan returned nothing and this failed at "OOM-killed worker not seen". +func TestScanOOMKilled_SeesKilledWorkerInRunningContainer(t *testing.T) { + var gotArgs []string + m := &Manager{ + logger: log.New(io.Discard, "", 0), + stacks: map[string]*Stack{ + "paperless-ngx": {Name: "paperless-ngx", Deployed: true, Containers: []ContainerInfo{ + {Name: "paperless-webserver", State: StateRunning}, + {Name: "paperless-redis", State: StateRunning}, + }}, + "bookstack": {Name: "bookstack", Deployed: true, Containers: []ContainerInfo{{Name: "bookstack", State: StateRunning}}}, + "stopped": {Name: "stopped", Deployed: true, Containers: []ContainerInfo{{Name: "stopped-c", State: StateExited}}}, + "undeployed": {Name: "undeployed", Deployed: false, Containers: []ContainerInfo{{Name: "u", State: StateRunning}}}, + }, + } + m.execFn = func(name string, args ...string) (string, error) { + gotArgs = args + return "/bookstack|false|2026-09-14T18:00:00Z\n/paperless-redis|false|2026-09-14T18:00:00Z\n/paperless-webserver|true|2026-09-14T18:00:01Z\n", nil + } + ooms, err := m.ScanOOMKilled() + if err != nil { + t.Fatal(err) + } + if len(ooms) != 1 || ooms[0].Stack != "paperless-ngx" || ooms[0].Container != "paperless-webserver" || ooms[0].StartedAt == "" { + t.Fatalf("OOM-killed worker not seen: %+v", ooms) + } + joined := strings.Join(gotArgs, " ") + if strings.Contains(joined, "stopped-c") || strings.Contains(joined, " u") { + t.Fatalf("inspected a stopped or undeployed container: %v", gotArgs) + } + if got := m.OOMKilledStacks(); len(got["paperless-ngx"]) != 1 || len(got) != 1 { + t.Fatalf("dashboard cache wrong: %v", got) + } + // Next scan clean → cache cleared (a restarted container resets OOMKilled). + m.execFn = func(string, ...string) (string, error) { + return "/bookstack|false|x\n/paperless-redis|false|x\n/paperless-webserver|false|y\n", nil + } + _, _ = m.ScanOOMKilled() + if got := m.OOMKilledStacks(); len(got) != 0 { + t.Fatalf("cache not cleared after a clean scan: %v", got) + } +} diff --git a/controller/internal/web/backup_handlers.go b/controller/internal/web/backup_handlers.go index aaa574f..97ce0c1 100644 --- a/controller/internal/web/backup_handlers.go +++ b/controller/internal/web/backup_handlers.go @@ -113,6 +113,85 @@ type guestBackupView struct { RestoreTestedAt time.Time CanTrigger bool // a backup trigger (quiesce loop) is wired + + // Tiers (R-517, agent >= v0.131.0) is the per-tier truth. Empty on an older agent — the tile then + // renders the single latest record as before. + Tiers []guestTierView +} + +// guestTierView is one whole-guest tier as the customer sees it. A failed attempt is shown UNDER the +// newest success, never instead of it ("presence is not success"). +type guestTierView struct { + Target string + Label string // Hungarian tier name + IsPBS bool + NotSetUp bool // the tier's storage does not exist on the host → „nincs beállítva" + HasSuccess bool + SuccessAt time.Time + SizeKnown bool // false when the success was read back from storage after a restart + SizeBytes int64 + Current bool // newest success is inside the tier's window + FailedAfter bool // the last attempt failed and is newer than the newest success + FailedAt time.Time +} + +// tierWindow is how old a tier's newest success may be and still count as current: its cadence plus +// half again (a scheduled run lands inside the nightly window, not on the exact second). A tier with +// no advertised cadence uses a day. +func tierWindow(cadenceSecs int64) time.Duration { + if cadenceSecs <= 0 { + cadenceSecs = 24 * 3600 + } + return time.Duration(cadenceSecs) * time.Second * 3 / 2 +} + +// buildTierViews turns the agent's per-tier state into page rows and derives the three legacy tile +// fields from SUCCESSES only: HasBackup/Success/Size/Target/StartedAt from the primary tier's newest +// success; Due when any set-up tier has no current success; Offsite only on a current PBS success. +func buildTierViews(v *guestBackupView, tiers []agentapi.TierBackupState, cadence map[string]int64, now time.Time) { + v.Tiers = nil + v.Due, v.Offsite, v.HasBackup = false, false, false + var newestOK time.Time + for _, t := range tiers { + tv := guestTierView{Target: t.Target, IsPBS: strings.Contains(strings.ToLower(t.Target), "pbs")} + if tv.IsPBS { + tv.Label = "Biztonsági szerver – külön hardver (PBS)" + } else { + tv.Label = "Helyi tároló (" + t.Target + ")" + } + tv.NotSetUp = t.Storage == "absent" + if t.LastSuccess != nil { + if ts, err := time.Parse(time.RFC3339, t.LastSuccess.StartedAt); err == nil { + tv.HasSuccess, tv.SuccessAt = true, ts + tv.SizeKnown = t.LastSuccessSource != "storage" + tv.SizeBytes = t.LastSuccess.SizeBytes + tv.Current = now.Sub(ts) <= tierWindow(cadence[t.Target]) + } + } + if a := t.LastAttempt; a != nil && !a.Success { + if ts, err := time.Parse(time.RFC3339, a.StartedAt); err == nil && (!tv.HasSuccess || ts.After(tv.SuccessAt)) { + tv.FailedAfter, tv.FailedAt = true, ts + } + } + if !tv.NotSetUp && !tv.Current { + v.Due = true + } + if tv.IsPBS && tv.Current && !tv.NotSetUp { + v.Offsite = true + } + if tv.HasSuccess && (t.Primary || !v.HasBackup) && (t.Primary || tv.SuccessAt.After(newestOK)) { + v.HasBackup, v.Success = true, true + v.StartedAt, v.SizeBytes, v.Target = tv.SuccessAt, tv.SizeBytes, tv.Label + newestOK = tv.SuccessAt + } + v.Tiers = append(v.Tiers, tv) + } + if v.Due { + v.DueReason = "tier_not_current" + } + if v.HasBackup { + v.AgeHours = int64(now.Sub(v.StartedAt).Hours()) + } } // loadGuestBackup fetches the agent's whole-guest backup view (best-effort). Returns a view with @@ -153,6 +232,17 @@ func (s *Server) loadGuestBackup(ctx context.Context) *guestBackupView { v.AgeHours = *due.AgeSecs / 3600 } } + // R-517: an agent that speaks per tier replaces the single-record fields with per-tier truth. + // Done AFTER the legacy fill so an older agent keeps exactly the old rendering. + if len(st.Tiers) > 0 { + cadence := map[string]int64{} + if tr, terr := client.BackupTiers(ctx); terr == nil { + for _, t := range tr.Tiers { + cadence[t.Target] = t.CadenceSeconds + } + } + buildTierViews(v, st.Tiers, cadence, time.Now()) + } // Restore-test (the "verified restorable" trust signal; nil until one runs). if rt, rerr := client.RestoreTestStatus(ctx); rerr == nil && rt != nil { v.HasRestoreTest = true diff --git a/controller/internal/web/backup_tier_view_test.go b/controller/internal/web/backup_tier_view_test.go new file mode 100644 index 0000000..c13e768 --- /dev/null +++ b/controller/internal/web/backup_tier_view_test.go @@ -0,0 +1,77 @@ +package web + +import ( + "testing" + "time" + + "gitea.dooplex.hu/admin/felhom-controller/internal/agentapi" +) + +// R-517 — the whole-guest tile speaks from SUCCESSES per tier. The BIGNIGHT page read, after a local +// success and a failed PBS attempt on absent storage: "✗ · 0 B · PBS · Naprakész" and ticked +// "Távoli rendszermentés — külön hardveren". + +var tvNow = time.Date(2026, 9, 14, 19, 15, 36, 0, time.UTC) + +// RED-PROOF (run 2026-09-15, recorded in REPORT.md): with buildTierViews reading a tier's "success" +// from LastAttempt instead of LastSuccess, the shown backup lost its size and this failed at "the +// successful local backup is not the one shown: has=true size=0". +func TestBuildTierViews_BignightFailedPBSOnAbsentStorage(t *testing.T) { + v := &guestBackupView{} + tiers := []agentapi.TierBackupState{ + {Target: "local", Primary: true, Storage: "present", + LastSuccess: &agentapi.BackupRecord{TargetID: "local", Success: true, SizeBytes: 8877619753, StartedAt: "2026-09-14T19:03:23Z"}, + LastAttempt: &agentapi.TierAttempt{StartedAt: "2026-09-14T19:03:23Z", Success: true}}, + {Target: "felhom-pbs", Storage: "absent", + LastAttempt: &agentapi.TierAttempt{StartedAt: "2026-09-14T19:09:59Z", Success: false, Error: "storage 'felhom-pbs' does not exist"}}, + } + buildTierViews(v, tiers, map[string]int64{"local": 86400, "felhom-pbs": 604800}, tvNow) + + if v.Offsite { + t.Fatalf("remote tick shown without a PBS success: %+v", v.Tiers) + } + if !v.HasBackup || v.SizeBytes != 8877619753 || v.Target != "Helyi tároló (local)" { + t.Fatalf("the successful local backup is not the one shown: has=%v size=%d target=%q", v.HasBackup, v.SizeBytes, v.Target) + } + if v.Due { + t.Fatalf("a current local success with an unprovisioned PBS tier must read up to date, got Due") + } + pbs := v.Tiers[1] + if !pbs.NotSetUp || pbs.HasSuccess || !pbs.FailedAfter { + t.Fatalf("pbs row wrong (want not set up, no success, failed attempt shown): %+v", pbs) + } +} + +// A set-up PBS tier whose last attempt failed: ✗ under the last success, and not current if the +// success is outside the window → Due, no remote tick. +func TestBuildTierViews_PBSFailedUnderOldSuccess(t *testing.T) { + v := &guestBackupView{} + tiers := []agentapi.TierBackupState{ + {Target: "local", Primary: true, Storage: "present", + LastSuccess: &agentapi.BackupRecord{Success: true, SizeBytes: 10, StartedAt: "2026-09-14T01:00:00Z"}}, + {Target: "felhom-pbs", Storage: "present", + LastSuccess: &agentapi.BackupRecord{Success: true, SizeBytes: 20, StartedAt: "2026-08-20T01:00:00Z"}, + LastAttempt: &agentapi.TierAttempt{StartedAt: "2026-09-14T02:00:00Z", Success: false}}, + } + buildTierViews(v, tiers, map[string]int64{"local": 86400, "felhom-pbs": 604800}, tvNow) + pbs := v.Tiers[1] + if !pbs.HasSuccess || !pbs.FailedAfter || pbs.Current { + t.Fatalf("pbs row: want old success kept, failure shown under it, not current: %+v", pbs) + } + if v.Offsite || !v.Due { + t.Fatalf("want no remote tick and Due, got Offsite=%v Due=%v", v.Offsite, v.Due) + } +} + +// After an agent restart the success is read back from storage: shown, size unknown. +func TestBuildTierViews_SuccessFromStorageAfterRestart(t *testing.T) { + v := &guestBackupView{} + tiers := []agentapi.TierBackupState{ + {Target: "local", Primary: true, Storage: "present", LastSuccessSource: "storage", + LastSuccess: &agentapi.BackupRecord{Success: true, StartedAt: "2026-09-14T19:03:23Z"}}, + } + buildTierViews(v, tiers, map[string]int64{"local": 86400}, tvNow) + if !v.HasBackup || v.Due || v.Tiers[0].SizeKnown { + t.Fatalf("after restart: want the local success shown, up to date, size unknown: %+v / due=%v", v.Tiers[0], v.Due) + } +} diff --git a/controller/internal/web/filebrowser_login_test.go b/controller/internal/web/filebrowser_login_test.go new file mode 100644 index 0000000..412d511 --- /dev/null +++ b/controller/internal/web/filebrowser_login_test.go @@ -0,0 +1,96 @@ +package web + +import ( + "bytes" + "encoding/json" + "net/http" + "net/http/httptest" + "strings" + "testing" + + "gitea.dooplex.hu/admin/felhom-controller/internal/crypto" + "gitea.dooplex.hu/admin/felhom-controller/internal/settings" + "gitea.dooplex.hu/admin/felhom-controller/internal/stacks" +) + +// R-513 — the file manager's login is shown where app logins are, under the R-254 rule: the value is +// never in the page, only behind the reveal act. + +const fbTestPW = "TESTONLYfbPw7Kq2" + +func renderFBPage(t *testing.T, operatorSet bool) string { + t.Helper() + s := securityHarness(t) + s.loadTemplates() + data := map[string]interface{}{ + "Page": "stacks", "Title": "FileBrowser", "Domain": "example.hu", + "Stack": stacks.Stack{Name: "filebrowser", Deployed: true, State: "running"}, + "Meta": stacks.Metadata{DisplayName: "FileBrowser", Slug: "filebrowser"}, + "HasAppInfo": true, + "InitialCreds": &stacks.ExtractedCreds{Available: true, Username: "admin"}, + } + if operatorSet { + data["InitialCredsOperatorSet"] = true + } else { + data["InitialCredsHasPassword"] = true + } + var buf bytes.Buffer + if err := s.tmpl.ExecuteTemplate(&buf, "app_info", data); err != nil { + t.Fatalf("render: %v", err) + } + return buf.String() +} + +func TestFileBrowserLoginCard_Generated(t *testing.T) { + html := renderFBPage(t, false) + if !strings.Contains(html, "initial-credentials/reveal") || !strings.Contains(html, "admin") { + t.Fatal("generated state: the login card has no reveal call or no username") + } + if strings.Contains(html, "zemeltet") { // „üzemeltető" — ASCII fragment + t.Fatal("generated state shows the operator-set text") + } +} + +// RED-PROOF (run 2026-09-15, recorded in REPORT.md): with the {{if .InitialCredsOperatorSet}} branch +// removed from app_info.html, the operator page rendered the reveal button and this failed at +// "operator state still offers a reveal". +func TestFileBrowserLoginCard_OperatorSet(t *testing.T) { + html := renderFBPage(t, true) + if !strings.Contains(html, "zemeltet") { + t.Fatal("operator state does not say the operator set the password") + } + if strings.Contains(html, `id="initcred-reveal"`) { + t.Fatal("operator state still offers a reveal for a password the Felhom does not hold") + } +} + +func revealFB(t *testing.T, s *Server) (int, map[string]any) { + t.Helper() + w := httptest.NewRecorder() + s.fileBrowserPasswordReveal(w, httptest.NewRequest(http.MethodPost, "/apps/filebrowser/initial-credentials/reveal", nil)) + var body map[string]any + _ = json.Unmarshal(w.Body.Bytes(), &body) + return w.Code, body +} + +func TestFileBrowserPasswordReveal(t *testing.T) { + s := securityHarness(t) + s.encKey = bytes.Repeat([]byte{7}, 32) + if code, _ := revealFB(t, s); code != http.StatusNotFound { + t.Fatalf("undecided: want 404, got %d", code) + } + _ = s.settings.SetFileBrowserAdmin(settings.FileBrowserAdminOperator, "", "2026-09-15T00:00:00Z") + if code, _ := revealFB(t, s); code != http.StatusNotFound { + t.Fatalf("operator-set: want 404, got %d", code) + } + enc, err := crypto.Encrypt(s.encKey, fbTestPW) + if err != nil { + t.Fatal(err) + } + _ = s.settings.SetFileBrowserAdmin(settings.FileBrowserAdminGenerated, enc, "2026-09-15T00:00:00Z") + code, body := revealFB(t, s) + data, _ := body["data"].(map[string]any) + if code != http.StatusOK || data["password"] != fbTestPW { + t.Fatalf("generated: want 200 + the decrypted password, got %d %v", code, body) + } +} diff --git a/controller/internal/web/handlers.go b/controller/internal/web/handlers.go index e402b2f..17d46b7 100644 --- a/controller/internal/web/handlers.go +++ b/controller/internal/web/handlers.go @@ -180,6 +180,7 @@ func (s *Server) dashboardHandler(w http.ResponseWriter, r *http.Request) { data["SettingsWarning"] = s.settings.LoadWarning // non-empty if settings.json was recovered from corruption data["Stacks"] = deployedStacks data["MissingStorage"] = s.missingStorageMap(deployedStacks) + data["OOMKilled"] = s.stackMgr.OOMKilledStacks() // R-514 nw, ns := s.networkStorageWarnings(deployedStacks) // NAS unreachable (recoverable) / guest-side stub (defect) data["NetworkWarnings"] = nw data["NetworkStubs"] = ns @@ -741,6 +742,24 @@ func (s *Server) appDetailHandler(w http.ResponseWriter, r *http.Request, slug s } } + // R-513 (v0.243.0): the file manager's login lives with the other app logins. The controller set a + // generated password (state "generated") or found one set by hand ("operator"). Same R-254 rule: + // only the username and a boolean reach the page; the value comes from the reveal endpoint. + if found.Name == fileBrowserStack && s.settings != nil { + state, enc, _ := s.settings.GetFileBrowserAdmin() + switch state { + case settings.FileBrowserAdminGenerated: + data["HasAppInfo"] = true + data["InitialCreds"] = &stacks.ExtractedCreds{Available: true, Username: "admin", + Note: "A Fájlkezelő belépése. A jelszót a Felhom állította be ezen a gépen."} + data["InitialCredsHasPassword"] = enc != "" + case settings.FileBrowserAdminOperator: + data["HasAppInfo"] = true + data["InitialCreds"] = &stacks.ExtractedCreds{Available: true, Username: "admin"} + data["InitialCredsOperatorSet"] = true + } + } + // Per-app migration (B1): offer to move this app's data to another connected drive (≠ current). if found.Deployed { current := "" @@ -2261,6 +2280,11 @@ func (s *Server) appInitialCredsRevealHandler(w http.ResponseWriter, r *http.Req escrowJSON(w, http.StatusNotFound, nil, "Ismeretlen alkalmazás.") return } + // R-513: the file manager's password is the controller's own stored value, not a container file. + if found.Name == fileBrowserStack { + s.fileBrowserPasswordReveal(w, r) + return + } creds, err := s.readInitialCreds(found.Name) if err != nil { // Never swallowed, and never surfaced raw — the error can name a container/path. @@ -2279,6 +2303,33 @@ func (s *Server) appInitialCredsRevealHandler(w http.ResponseWriter, r *http.Req escrowJSON(w, http.StatusOK, map[string]any{"password": creds.Password}, "") } +// fileBrowserStack is the protected infra stack whose admin password R-513 manages. +const fileBrowserStack = "filebrowser" + +// fileBrowserPasswordReveal serves the generated FileBrowser admin password (R-513) under the same +// rules as every initial-credential reveal: POST + CSRF (router), no-store, logged as an act, and a +// refusal that says why when there is nothing to show. +func (s *Server) fileBrowserPasswordReveal(w http.ResponseWriter, r *http.Request) { + w.Header().Set("Cache-Control", "no-store") + if s.settings == nil { + escrowJSON(w, http.StatusServiceUnavailable, nil, "A beállítások nem elérhetők.") + return + } + state, enc, _ := s.settings.GetFileBrowserAdmin() + if state != settings.FileBrowserAdminGenerated || enc == "" { + escrowJSON(w, http.StatusNotFound, nil, "A Fájlkezelő jelszavát nem a Felhom állította be ezen a gépen, ezért nem tudjuk megmutatni.") + return + } + pw, err := crypto.Decrypt(s.encKey, enc) + if err != nil || strings.TrimSpace(pw) == "" { + s.logger.Printf("[ERROR] [web] filebrowser password reveal: decrypt failed: %v", err) + escrowJSON(w, http.StatusInternalServerError, nil, "A jelszó most nem olvasható ki.") + return + } + s.logger.Printf("[INFO] [web] filebrowser admin password revealed from %s (value never logged)", clientIP(r)) + escrowJSON(w, http.StatusOK, map[string]any{"password": pw}, "") +} + func (s *Server) settingsHandler(w http.ResponseWriter, r *http.Request) { s.executeTemplate(w, r, "settings_system", s.systemPageData()) } diff --git a/controller/internal/web/templates/app_info.html b/controller/internal/web/templates/app_info.html index f94edfb..8129255 100644 --- a/controller/internal/web/templates/app_info.html +++ b/controller/internal/web/templates/app_info.html @@ -187,6 +187,12 @@ function appMigrate(btn,app,label){ {{.InitialCreds.Username}} {{end}} + {{if .InitialCredsOperatorSet}} + + Jelszó + az üzemeltető állította be + + {{else}} Jelszó