diff --git a/STATUS.md b/STATUS.md index 3c46a2e4..46d8335e 100644 --- a/STATUS.md +++ b/STATUS.md @@ -34,8 +34,6 @@ Newest first. CC's pick is in the „Pick" column. Each row also has its full te |---|---|---|---|---|---|---| | D49 | R-1270 | A restore's data is back but the app does not start in 90 s. Keep the restored data and wait, or pour the old data back by itself? | **Keep and wait** (a slow first start is common; the old copy is kept, a late start turns it into a success; you get one alarm) | pour back by itself: a slow but correct restore is thrown away | the data stays restored, the old copy stays for support | — | | D46 | R-1272 | Put Homepage behind the family gate, like Glance? | **Yes** (a stranger now sees the box's CPU, memory and disk, and later the family's links; one line in the template, tried on the drill first) | leave it open: anyone who finds the address sees those | it stays open to strangers | — | -| D45 | R-1266 | Seven places where a design document says one thing and the code does another. One line each, with a pick: `documentation/audits/review-round-2-2026-10-24/D44-D45-picks.md`. Answer „all picks" or name the ones you disagree with. | **Yes — the text follows the code** (each is how the system works today; you read every changed sentence) | the code follows the text: six code changes, one of them to the box's door for you | the documents keep saying what is not true | — | -| D44 | R-1265 | 25 behaviours the code has but no document named. One line each, with a pick (21 confirm, 4 small changes): the same file. Answer „all picks" or name the ones you disagree with. | **All picks** (each was read in the code) | name the ones to change; each becomes a row | the notes stay marked „undocumented" | — | | D38 | R-1235 | Put the newest golden (637 MB) into the nightly release copy on ep0 too? | **Yes** (a new install after a DooPlex loss needs it; it fits under the 3 GB cap) | leave it out: after a loss, no new install until a golden is rebuilt and vouched | the same | — | | D37 | R-1236 | Raise Gitea's memory limit from 1 GiB to 2 GiB? It was killed for memory 4 times in 17 days, once during today's release. | **Yes** (DooPlex has the RAM; a DooPlex change, so yours) | keep 1 GiB: a restart in release hours fails box updates and CI for a minute | the same | — | | D9 | R-1062 | Split the N100's USB drive in two, for multi-drive tests? | **No — keep one drive** (the lived-in box keeps its nightly whole-box backup) | Split: the box loses its local whole-box backup, or you buy a second drive first | it stays one drive; multi-drive tests stay on Tester 1 | — | diff --git a/documentation/architecture/01-topology-and-trust.md b/documentation/architecture/01-topology-and-trust.md index 624f56f1..6eb867c6 100644 --- a/documentation/architecture/01-topology-and-trust.md +++ b/documentation/architecture/01-topology-and-trust.md @@ -23,8 +23,10 @@ records *Felhom's decisions*, not Proxmox behaviour. ## 1. Model at a glance -Three components. **Control is always box-initiated** — the hub never connects *into* a -customer box. +Three components. **Control is always box-initiated** — the hub never ~~connects~~ sends a command *into* a +customer box. **[DESIGN — corrected 2026-10-25, D45 = all picks, R-1266]** The hub may POKE a box: through ep0, one empty UDP datagram to the box's WireGuard +address (port 51822; `hub/internal/poke/client.go`, agent `internal/poke/poke.go`). A poke carries no command; it only +makes the box pull its desired state at once. The box still pulls. ``` operator side customer box (per Proxmox host) @@ -98,8 +100,12 @@ credentials. - CGNAT does **not** force this: the Cloudflare Tunnel already makes a box reachable through Cloudflare's edge. We *choose* box-initiated control for the smallest attack surface — the - box exposes no control endpoint at all. -- The agent and the controller **poll** the hub; the hub never initiates inbound. + box exposes no control endpoint ~~at all~~: no route on the box takes a command from the hub. **[DESIGN — corrected 2026-10-25, D45 = all picks, R-1266]** + Two listeners sit behind the WireGuard tunnel: the agent's poke listener (a poke carries no command; the box still + pulls) and the operator door, `felhom-sshd` on the host, reachable only from the operator's `/32` peer — the + documented break-glass (`03` §5, `06` §4.5). +- The agent and the controller **poll** the hub; the hub never initiates ~~inbound~~ a command inbound — a poke only + makes them poll at once (corrected 2026-10-25, D45, R-1266). - Operator actions are delivered as **signed jobs**: the agent verifies an operator signature before executing, so a compromised hub database alone cannot forge commands. - **[FACT, 2026-10-16, R-970]** The last contradiction is gone: until controller v0.322.0 the box's own hub API key, @@ -245,7 +251,8 @@ app. Controller down → the gated app answers an error, never the app. Measured 2026-10-08, R-900). Replacing it with our own relay is deferred until after the first customers (R-904, `09` §3 decision 184). - **Outbound only** for control/report/backup (poll to hub, push to PBS). No inbound control - endpoint exists in the chosen model. + endpoint exists in the chosen model. **[DESIGN — corrected 2026-10-25, D45 = all picks, R-1266]** A poke carries no command; the box still pulls. The operator + door (`felhom-sshd` over the WireGuard tunnel) is the documented break-glass (§4, `03` §5). - **Every customer has their OWN domain — never a name under `felhom.eu`** (operator ruling 2026-09-14). The free Cloudflare tier's certificate covers one level below a zone, so nested names such as `felhom..felhom.eu` are not covered; the customer's own domain is a few thousand @@ -330,7 +337,12 @@ app. Controller down → the gated app answers an error, never the app. Measured - **Key custody (zero-knowledge default):** three tiers the customer chooses — *customer-only* / *zero-knowledge escrow (default)* / *operator-managed*. Default escrows the **PBS passphrase-protected keyfile** in the hub, wrapped under a **customer recovery - code** the operator can't open; DR needs the customer's code. Access-notification is an + code**; DR needs the customer's code. ~~the operator can't open~~ **[DESIGN — corrected 2026-10-25, D45 = all picks, R-1266]** The hub keeps only the + locked key package, and every older one it retained; without the customer's code the operator cannot open any of + them, and from the off-site storage and the hub nobody can read the backup. Two exact limits, as the website says + since R-985: while the box runs, the operator has root access for support and with it could reach the data and the + backup key on the box; and the operator can open an OLDER off-site store from a retained package, by hand, when the + household types its old code on its own box (`runbooks/RUNBOOK-open-retained-package.md`, R-304). Access-notification is an audit signal, never the primary guard. (Don't build bespoke crypto — use PBS's native keyfile passphrase.) @@ -355,8 +367,11 @@ app. Controller down → the gated app answers an error, never the app. Measured ## 10. How this embodies the product values -- **Zero-knowledge offsite** — the operator holds the offsite backup but cannot read it. -- **Box-initiated control + signed jobs** — no standing operator backdoor; a hub compromise +- **Zero-knowledge offsite** — the operator holds the offsite backup but cannot read it without the customer's code + (the two exact limits are in §8 — corrected 2026-10-25, D45, R-1266). +- **Box-initiated control + signed jobs** — ~~no standing operator backdoor~~ no hidden operator door: the one + operator door is `felhom-sshd` over the WireGuard tunnel, the documented break-glass (`03` §5), and the website tells + the household about the operator's root access (R-985) — corrected 2026-10-25, D45, R-1266; a hub compromise alone can't forge commands. - **Customer-visible audit log** — every operator action is visible to the customer. - **Never hold data hostage** — subscriptions cover ongoing labour (monitoring, offsite, diff --git a/documentation/architecture/03-host-agent.md b/documentation/architecture/03-host-agent.md index 0060587c..6741a47b 100644 --- a/documentation/architecture/03-host-agent.md +++ b/documentation/architecture/03-host-agent.md @@ -64,7 +64,8 @@ Explicitly does **not**: - Serve application traffic or sit in the data path. **Control plane, not data plane**: if the agent dies, apps keep serving (Docker + LXC run without it); only *management* degrades — no new backups, no provisioning, hub loses the heartbeat. - Hold or proxy customer application data. - Run inside a guest. It is the thing that recovers guests and the host; it cannot be one of them. -- Manage **geo-restriction / the Cloudflare API**. Geo is hub-owned: the customer sets it in the controller UI, the controller reports the geo desired-state to the hub, and the **hub** (holding the CF API token) reconciles the WAF (S4). The agent manages only the *tunnel* service (`cloudflared`, §3/§5), never WAF rules. +- Manage **geo-restriction / the Cloudflare API**. Geo is hub-owned: the customer sets it in the controller UI, the controller reports the geo desired-state to the hub, and the **hub** (holding the CF API token) reconciles the WAF (S4). ~~The agent manages only the *tunnel* service (`cloudflared`, §3/§5), never WAF rules.~~ The agent manages no + Cloudflare piece: it only reads the guest's `cloudflared` container (§3) — corrected 2026-10-25, D45, R-1266. ## 3. Process model & host integration @@ -240,10 +241,16 @@ alerts the operator when an agent misses its expected check-in window. This is t failure mode for a managed service, so it gets first-class treatment hub-side. **Break-glass.** Standing inbound control is off. But when the poll loop *itself* is wedged -(agent hung, host sick) you cannot fix it through the poll loop. So there is an explicit, -**off-by-default, customer-consented, fully-audited** emergency path: SSH to the host via +(agent hung, host sick) you cannot fix it through the poll loop. So there is an explicit emergency path. +~~**off-by-default, customer-consented, fully-audited** emergency path: SSH to the host via the Cloudflare Tunnel behind Cloudflare Access (or on-site). Enabling it is itself a signed, -logged operation; it auto-expires. +logged operation; it auto-expires.~~ **[DESIGN — corrected 2026-10-25, D45 = all picks, R-1266]** The path is the operator door: the operator's WireGuard +peer reaches a dedicated `felhom-sshd` on the host (a non-22 port, reachable only from the operator's `/32` over +`wg-felhom`, key-only login; `06` §4.5, TASK H1). It does not use the Cloudflare Tunnel. It is installed by default on +an appliance install, and a BYO install refuses it (the owner's call; `scripts/felhom-host-install.sh`). It stays on; +it is not opened per incident. When SSH itself is broken, the hub-vaulted `root@pam` password opens the Proxmox web +console (`runbooks/break-glass.md`). The website tells the household that the operator has root access for support +(R-985). ## 6. Agent ↔ controller local API @@ -315,11 +322,11 @@ A controller can only `POST /rollback` (or snapshot/backup) **its own** guest token → guest and authorizes per guest, so a compromised controller's blast radius is **self-scoped and bounded** to its own guest. -**[DESIGN — undocumented, found 2026-10-24, review round 2 — confirmed through R-1265]:** the surface listed above is -the slice-8 design; on `main` the local API registers **41** routes (`felhom-agent/internal/localapi/server.go`; 24 of -them are not listed above — the backup tiers and target, disk candidates, decommission, format status, guest attach, -the network storage routes, the controller swap, the escrow stage/preflight/ceremony routes, the debug logs, guest -memory). Among them the controller can start a HOST action: `POST /host/line-test/run` (about 55 s; bounded by the due +**[DESIGN — confirmed by the operator 2026-10-25 (D44 = all picks, R-1265); every route listed below since +2026-10-25]:** the surface listed above is the slice-8 design; on `main` the local API registers **41** routes +(`felhom-agent/internal/localapi/server.go:562-634`). The list above names 15 of them; the other 26 are in the table +below (recounted 2026-10-25 — the review counted 24). Every route is behind the same per-guest token (`withGuest`). +Among them the controller can start a HOST action: `POST /host/line-test/run` (about 55 s; bounded by the due date, the window, 6 tries a day and the heavy-operation gate — R-1250, unreleased). The local API also listens on the R-50 island bridge (host `169.254.253.1`, guest `net1` `169.254.253.2/30`, `internal/config/config.go:260-269`). The weekly line test keeps its state in the OOB state dir (`line-test.json`) and has an off switch, `line_test.disable`. @@ -327,6 +334,37 @@ The served WireGuard block carries `oob_peer_ip` and `oob_operator_ssh_key` (`in runs only inside the tunnel watchdog, and its 15-minute count lives in memory — an agent restart starts it again (the once-an-hour bound survives in `wg/last-rekey`). +The routes the list above does not name (read from `server.go:562-634` and each handler's own comment, 2026-10-25): + +| Route | What it does | +|---|---| +| `GET /backup/tiers` | this guest's backup tiers, each with its cadence and whether its Proxmox storage exists now (`present` / `absent` / `unknown`) | +| `POST /backup/target` | moves the primary backup tier to a new storage: create → grant → config; answers `restart_required`, and the caller restarts the agent after its own in-flight check | +| `GET /host/line-test` | HOST: the weekly upload line test's last measurement; `present:false` when never measured (R-1247) | +| `POST /host/line-test/run` | HOST: starts a line test now — about 55 s, bounded as above (R-1250) | +| `GET /disks/candidates` | the host disks free for Felhom to enroll (`initialize`, and `attach` for those with a mountable file system) | +| `POST /disks/same-disk` | whether two of the guest's paths sit on the same physical disk, from the agent's own read (`16` §4) | +| `POST /disks/decommission` | takes the household's own drive out for good: a permanent intent, the guest bind pruned, unmounted; never formats; a system or backup mount is refused (403) | +| `GET /disks/format/status` | the newest or running format job, for a caller whose request timed out | +| `POST /disks/guest-attach` | binds an enrolled drive's `felhom-data` folder into this guest (read-write); the rest of the drive stays hidden | +| `GET /guest/memory` | this guest's RAM and its bounds | +| `POST /guest/memory` | a bounded RAM resize, applied live (R-24) | +| `POST /netstorage/add` | mounts a NAS share on the host and starts a verify job; a failed verify rolls it back | +| `GET /netstorage` | the NAS shares | +| `GET /netstorage/verify-status` | the verify job's state | +| `POST /netstorage/remove` | removes a NAS share | +| `POST /controller/swap` | swaps the controller image (202, then async); the agent verifies and rolls back | +| `GET /controller/swap/status` | the swap's state | +| `POST /escrow/stage-secret` | stages the off-site repository password for the escrow ceremony (0600, never logged) | +| `DELETE /escrow/stage-secret` | wipes the staged password | +| `POST /escrow/recover-offsite-password` | with the recovery code in the body, returns only the off-site repository password (§8a) | +| `GET /escrow/preflight` | the ceremony's checklist | +| `POST /escrow/ceremony` | starts the root ceremony job | +| `GET /escrow/ceremony/status` | the ceremony's state | +| `POST /escrow/ceremony/claim` | the one-shot hand-off of the new recovery code; a later claim is 410 | +| `POST /escrow/ceremony/confirm` | the household's two words; only then does the package reach the hub (R-1151) | +| `GET /debug/logs` | the agent's debug log ring, for the controller's Debug page | + ### 6a. Implementation (slice 8A — implemented) **Status: implemented** (agent v0.10.0 `internal/localapi`; controller v0.35.0 `internal/bootstrap` diff --git a/documentation/architecture/04-control-plane-authorization.md b/documentation/architecture/04-control-plane-authorization.md index e3399edb..bf3a0458 100644 --- a/documentation/architecture/04-control-plane-authorization.md +++ b/documentation/architecture/04-control-plane-authorization.md @@ -77,10 +77,10 @@ nonce (DoS-safe). Each layer is mandatory and was proven to reject independently A cannot be replayed at box B); - **time window** — `now ∈ [issued_at, expires_at]`; as built, `issued_at` may be up to **2 minutes** in the future (clock skew), `expires_at` is never extended — the agent (`felhom-agent/internal/authz/verifier.go:19-22,148-155`) and - the root wrapper (`configs/felhom-os-apply`) alike (corrected 2026-10-24). **[DESIGN — undocumented, found - 2026-10-24]:** neither caps the window's LENGTH (`expires_at − issued_at`); `felhom-opsign` defaults to 30 min - (`cmd/felhom-opsign/main.go:57`), and the nonce store keeps a record until its expiry. A register row asks the - operator to confirm or cap it; + the root wrapper (`configs/felhom-os-apply`) alike (corrected 2026-10-24). **[DESIGN — the operator picked a code change + 2026-10-25 (D44); row R-1280]:** neither caps the window's LENGTH (`expires_at − issued_at`); `felhom-opsign` defaults to 30 min + (`cmd/felhom-opsign/main.go:57`), and the nonce store keeps a record until its expiry. The pick: the agent and the + root wrapper refuse a window longer than 24 h. Not built yet; - **nonce** — unseen within the window (the nonce store **must be persistent across agent restarts** and expiry-pruned; a non-persistent store reopens the replay window after a restart). diff --git a/documentation/architecture/06-offsite-connectivity.md b/documentation/architecture/06-offsite-connectivity.md index 4353d722..2be49296 100644 --- a/documentation/architecture/06-offsite-connectivity.md +++ b/documentation/architecture/06-offsite-connectivity.md @@ -330,12 +330,16 @@ traverse `/root`) — create datastores under a world-traversable parent (`/srv` ## 6. Trust-model conformance (01 §3/§4 preserved) - The box **only ever dials out** (UDP to the endpoint; the same posture as hub-poll and - PBS-push, 01 §4/§7). No inbound listener is added to any customer box; the hub still never - initiates into a box. + PBS-push, 01 §4/§7). ~~No inbound listener is added to any customer box; the hub still never + initiates into a box.~~ **[DESIGN — corrected 2026-10-25, D45 = all picks, R-1266]** Two listeners sit on the box's tunnel side, reachable only through + the tunnel: the agent's poke listener (UDP 51822 on the box's WireGuard address — a poke carries no command, the box + still pulls) and, on an appliance install, the operator door `felhom-sshd` (only from the operator's `/32`, §4.5) — + the documented break-glass (`03` §5). The hub still never sends a command into a box. - The endpoint VM cannot reach into boxes either: it terminates tunnels whose per-peer routes (§4.5) admit only box→PBS traffic; it holds no box credentials. -- **Out of scope, explicitly:** (a) operator-inbound access to boxes — a *separate* future - problem with its own trust decision; nothing here creates or presumes it; (b) game-server +- **Out of scope, explicitly:** (a) ~~operator-inbound access to boxes — a *separate* future + problem with its own trust decision; nothing here creates or presumes it;~~ operator-inbound access to boxes was + added on 2026-07-05 (TASK H1) and rides this tunnel — §4.5 (corrected 2026-10-25, D45, R-1266); (b) game-server ingress — public players are not tunnel peers; that is a different ingress problem (public reachability), not a backup-transport one. @@ -387,7 +391,7 @@ venue; it gates nothing. Runbook: **MVP that puts offsite backup in a real customer's hands = S1 → S2 → S3; then S4.** S5/S6 complete the story. The cloudflared service (`internal/hub/cloudflared.go:9-12` — then a -read-only prober that named itself "the seam for the tunnel-management slice") adopts the -S3-piloted agent-managed-service pattern in a later, separate slice. *(Dated note 2026-10-24: cloudflared is a GUEST -container the agent only reads — four states, `cloudflared.go:12-23`; `03` §2 says the agent will not manage it. -Whether this plan is withdrawn is the operator's — a row of review round 2.)* +read-only prober that named itself "the seam for the tunnel-management slice") ~~adopts the +S3-piloted agent-managed-service pattern in a later, separate slice.~~ **[DESIGN — corrected 2026-10-25, D45 = all picks, R-1266]** This plan is withdrawn: +cloudflared is a GUEST container that the controller renders and keeps up; the agent only reads it (four states, +`cloudflared.go:12-23`) and will not manage it — `03` §2/§3 wins. diff --git a/documentation/architecture/07-backup-architecture.md b/documentation/architecture/07-backup-architecture.md index eba97409..31a3b591 100644 --- a/documentation/architecture/07-backup-architecture.md +++ b/documentation/architecture/07-backup-architecture.md @@ -1561,7 +1561,8 @@ crosses the line — **R-158**. > design, **row 14** is proven and merely never stopwatched, **row 11b** is a note not a row, and > **row 15 was an open DEFECT (R-104) that this line did NOT cover** — *R-104 was CLOSED 2026-10-05 (controller v0.297.0: a lock that survives the self-heal gets the class `locked`, its own note and a retry, `offbox.go:156-162`; -CLOSED-ITEMS.md; corrected 2026-10-24).* +CLOSED-ITEMS.md; corrected 2026-10-24).* **Row 16** (added 2026-10-25, D44 #6, R-1265 — after this banner) has a blank RTO +too: a registry rebuild from the ep0 release copy was never timed. > > **NO STATUS MOVED ON THE DAY THIS WAS WRITTEN, because nothing was proven that day.** A stopping > line that promotes a row is a stopping line that lies. Full text, and the conditions that reopen @@ -1601,6 +1602,7 @@ CLOSED-ITEMS.md; corrected 2026-10-24).* | 13 | **Customer loses R** | every byte, every tier, the Recipe | **on-premises recovery is unaffected** — Tier-1/2/3 and the whole-guest tiers all work while the guest and host live. What is lost: the ability to re-establish **host identity** and to open the escrowed offsite key material after a host loss | — | | | **NONE for host-loss** | R exists in **zero** system copies by design (INV Part C, row 24). Whether the operator should be able to recover is **open** → §11-A, §11-B. **D5 narrows this further:** Tier-1/2 app recovery needs only the drive, so losing R now costs the offsite route and host identity, never local app recovery (§7.4) | | 14 | **Host SSH management plane dies** | everything | 3-layer break-glass: tmpfiles → 60 s watchdog → hub-vaulted `root@pam` on the PVE console | operator | | | **PROVEN** | `runbooks/break-glass.md`; the original incident and its fix | | 15 | **Interrupted offsite run leaves a stale lock** | everything | manual `restic unlock --remove-all` | **operator** (SSH) | | | **FIXED 2026-10-05 (R-104, v0.297.0; was DEFECT — corrected 2026-10-24)** | the self-heal exists (`offbox.go:634-648`) but the probe fails first and `classifyResticProbe` has no lock case → fail-fast, operator told *„ismeretlen okból"* → **R-104** | +| 16 | **The container registry is lost with DooPlex** (the released images and agent packages) | the release copy on ep0 (D33): the newest 3 release tags of `felhom-controller`, `felhom-hub` and `felhom-samba`, and the newest 5 agent packages (binary + config bundle), in a plain folder outside PBS (`/srv/felhom-release-copy`) | push the copy back into a registry with the same name — `runbooks/total-loss-of-dooplex.md` „Restore the release copy” (same digests, no rebuild; `scripts/release-copy/`) | **operator** (ep0 root) | | 24 h (nightly 01:40) | **PARTIAL** | `audits/day-2026-10-23/08-release-copy-D33.txt`: the copy was pulled back, every sha256 and digest matched, and a controller image from it ran on a scratch box. Not yet walked: the push into a real new registry and the agent-package upload. Added 2026-10-25 (D44 = all picks, R-1265) | ### 8.2 The ONE exception to row 5's derived-copy rule (R-403, controller v0.230.0) @@ -1740,7 +1742,7 @@ put-back leaves (§8 D20/D22), so no new retention rule. A watcher looks 30 more turns to success („…elindult: a visszaállítás kész.") and the copy is removed, as after any successful restore; if it never runs, the copy stays for support. -### 8.2c Decisions in code that no document recorded [DESIGN — undocumented, found 2026-10-24, review round 2 — the operator confirms them through one register row] +### 8.2c Decisions in code that no document recorded [DESIGN — confirmed by the operator 2026-10-25 (D44 = all picks, R-1265)] Read at `file:line` on `main` (controller `321df1f`, hub `44d19042`) by the main session: @@ -1756,8 +1758,9 @@ Read at `file:line` on `main` (controller `321df1f`, hub `44d19042`) by the main a box that declares it waits for the household's recovery code (`awaiting_recovery_key`) is skipped — no event, no heal (R-1230, `:103-110`). - **The release copy (D33)** — nightly 01:40, the newest 3 image tags and 5 agent packages to ep0 under its own - limited user, capped by the sender (`felhom.eu/scripts/release-copy/`) — has no row in this matrix; its homes are - the poster facts and `runbooks/total-loss-of-dooplex.md`. + limited user, capped by the sender (`felhom.eu/scripts/release-copy/`). ~~has no row in this matrix~~ It is row + **16** of the matrix since 2026-10-25 (D44 = all picks, R-1265); its other homes are the poster facts and + `runbooks/total-loss-of-dooplex.md`. ### 8.1 The blank cells, listed explicitly @@ -1776,6 +1779,7 @@ Per the rule that a blank is a finding, here they are: | 13 | RTO, RPO | not a timed recovery; a capability loss | | 14 | RTO | the break-glass path is proven but was never timed | | 15 | RTO | the manual unlock was performed but not timed | +| 16 | RTO | a registry rebuild from the ep0 release copy has never been timed (row added 2026-10-25, D44 #6, R-1265) | **Also unmeasured, and not representable as a row** (INV Part F.3): any restore larger than 155.5 MB from restic; any restore of a customer data drive (no tier holds one whole); the weekly PBS diff --git a/documentation/architecture/09-update-architecture.md b/documentation/architecture/09-update-architecture.md index 0b491593..6294a7c6 100644 --- a/documentation/architecture/09-update-architecture.md +++ b/documentation/architecture/09-update-architecture.md @@ -286,8 +286,11 @@ builds them (`audits/update-rulings-2026-09-23/`). 13. **The test decides, not the tag** (Q3). **REPLACES decision 3's second half, "never across a major".** The box applies by itself every step the catalog holds, because **the catalog holds only tested steps**; the size of the version jump does not matter, the test does. The box does not - parse tags to decide. The tag rule (`stacks.CompareImageRefs`) moves to the **catalog gate** as a - push-time safety net: an image move with no test record is refused there. Two exceptions, both + parse tags to decide. ~~The tag rule (`stacks.CompareImageRefs`) moves to the **catalog gate** as a + push-time safety net:~~ **[DESIGN — corrected 2026-10-25, D45 = all picks, R-1266]** The tag rule (`stacks.CompareImageRefs`) did not move: it + stays in the controller (`internal/stacks/updateorder.go`), where it only tells whether an installed app is + AHEAD of the catalog; it never decides an update. The catalog gate is the push-time safety net, and it asks for a + proven test instead (§6.4 part 4): an image move with no test record is refused there. Two exceptions, both **marks the test sets on the step**: *files may change* → automatic only when a fresh copy holds the files, else a person (this is Q2's refinement); *needs a person* → the tester writes why. **Why:** decision 3 was a proxy. "Within a major" was standing in for "known to work", and the @@ -299,7 +302,11 @@ builds them (`audits/update-rulings-2026-09-23/`). unchanged for every other move: a step a box takes was still tested, on at least the box venue. 14. **The update ladder** (Q3). A box more than one step behind **climbs one tested step at a time, in - order, and never jumps**. Each step is the full guarded update of §6.1. **Why:** a step was tested + order, and ~~never jumps~~ the night leg never jumps**. Each step is the full guarded update of §6.1. + **[DESIGN — corrected 2026-10-25, D45 = all picks, R-1266]** An app older than the ladder (its installed version matches no entry's `from`) is never + pressed by the night leg (`internal/stacks/unattended.go`). A PERSON's press on such an app takes the catalog's + current definition, logged by name (`internal/stacks/ladder.go`, `TestLadder_UnknownInstalledJumpsAndSaysSo`); the + guarded update and its undo (decision 15) still run around it. **Why:** a step was tested from the version before it, not from three versions before it; R-40's multi-major jump is what this prevents. This is the "stepping" §6.1 already assigned to slice 6. @@ -988,8 +995,11 @@ its length, and both fixes cost something the household would notice — operato installs the vouched golden and updates itself to the floor (not yet proven end to end — R-1052). **The golden stays weekly:** a CHANGED golden selection is still refused when it is older than the fleet's newest controller (`TestVouchGoldenChangedToOlderIsStillRefused`). - 5. The push gate judges the golden by its AGE (older than 7 days while the floor is above it), not by the newest - CHANGELOG heading (`scripts/golden_currency_gate.py`). + 5. The push gate judges the golden by its AGE, not by the newest CHANGELOG heading + (`scripts/golden_currency_gate.py`). **[DESIGN — corrected 2026-10-25, D45 = all picks, R-1266]** ~~(older than 7 days while the floor is above + it)~~ The gate runs offline and cannot read the floor, so „a release is ahead of the newest bake” stands in for + „the floor is above it”: it convicts when a release is ahead AND the newest bake is more than 7 days old; ahead + and at most 7 days old is an advisory. The floor follows a release within hours, so the gate is the stricter one. 251. **Scripts use scoped hub tokens, not the operator password; CC's floor step uses its own.** *Operator ruling 2026-10-15 11:38.* Tokens with groups (read/write per group), managed on the Configuration tab, shown once, stored hashed, every use logged by name (`05` §17, hub v0.152.0). Never by any token: a secret shown or minted, a delete or @@ -2570,7 +2580,7 @@ and E are the ones that unblock a decision; leg A is the one that takes the time harness on DooPlex for the image-side ones. -### 6.4.x Decisions in code no section records [DESIGN — undocumented, found 2026-10-24, review round 2 — the operator confirms them through one register row] +### 6.4.x Decisions in code no section records [DESIGN — confirmed by the operator 2026-10-25 (D44 = all picks, R-1265)] Read at `file:line` on controller `main` (`321df1f`) by the main session: diff --git a/documentation/architecture/13-threat-model.md b/documentation/architecture/13-threat-model.md index f042b334..fe57ea5a 100644 --- a/documentation/architecture/13-threat-model.md +++ b/documentation/architecture/13-threat-model.md @@ -78,12 +78,13 @@ OS packages (exact versions, apt-signed), the installer script (by tag, R-980). | **added:** catalog ↔ every box | **design gap** | a catalog push reaches every box with no check on the box (R-971). **Re-check (2026-10-24, review round 2):** the box checks every rendered compose against its allow-list before any write since controller v0.328.0 (`internal/privpolicy/privpolicy.go`) | | **added 2026-10-23 (R-1241):** a hacked app pod on DooPlex ↔ Felhom's secrets | **does not hold**; not used in the Plex2Shell window | two no-login doors: the Longhorn NFS export (the hub's unencrypted nightly copies with the unsealed tokens of R-975; by a published file-handle escape the whole sda1 disk, the operator signing keys included — not exercised) and the Longhorn API (backup a volume to any target). Plex2Shell, 2026-10-17 → 10-23: **NOT REACHED** — 0 new NFS connections in the window, no off-schedule backup or volume; the hub, the k8s API and the kubelet refuse a caller with no credential (measured). The export was narrowed to the pod network on 2026-10-23; the pod side stays open (`audits/plex2shell-felhom-impact-2026-10-23/`). **Felhom half, 2026-10-23 night:** the last plain tokens sealed in hub 0.168.0 (deployed 2026-10-24 — corrected 2026-10-24; `audits/hub-token-seal-2026-10-23/`); the two plain Longhorn copies of the hub volume already in the folder are R-1243's decision. A household box's version of this attacker (a hacked app sending or mining) is what the upload watch (R-1244, released 2026-10-24 in controller v0.342.0 + hub v0.169.0 — corrected 2026-10-24) detects | -**[DESIGN — undocumented in `01` §5, found 2026-10-24, review round 2 — confirmed through R-1265]:** the file manager's one +**[DESIGN — the operator picked a code change 2026-10-25 (D44); row R-1281]:** the file manager's one sign-in (R-1100) — FileBrowser trusts its proxy header blindly, and every app container can reach `filebrowser:80` directly; the only guard is that the header NAME is a per-box secret, which traefik's forwardAuth strips from every client request and sets only for a browser holding the dashboard's files pass (`felhom-controller/controller/internal/infra/filebrowser_sso.go:15-22`, `internal/web/files_sso.go`). The name file's -creation race was fixed by review round 2 (controller `b01bb4b`). +creation race was fixed by review round 2 (controller `b01bb4b`). The pick: FileBrowser joins only the network traefik uses, so no app +container reaches it directly. Not built yet. ## 6. What we accept (do not re-report) diff --git a/documentation/architecture/14-host-power-off.md b/documentation/architecture/14-host-power-off.md index bd93d0b6..bf661ace 100644 --- a/documentation/architecture/14-host-power-off.md +++ b/documentation/architecture/14-host-power-off.md @@ -89,7 +89,7 @@ alternative (an out-of-band confirmation) needs a second channel the box does no - ~~The website FAQ line („to turn it off: Beállítások → Rendszer → A gép kikapcsolása") — a later website task (R-1000).~~ Published — `website/gyik.html:128,603` (corrected 2026-10-24). - The BIOS „after power loss" value of the N100 was not read (no monitor); the household text promises only the button. -## Decisions in code no section records [DESIGN — undocumented, found 2026-10-24, review round 2 — confirmed through one register row] +## Decisions in code no section records [DESIGN — confirmed by the operator 2026-10-25 (D44 = all picks, R-1265)] - **A re-enroll while the household power-off hold is active counts as „gone"** (a reinstall over the old host), not „live" (R-1154; `hub/internal/api/reenroll.go:67-68`). diff --git a/documentation/architecture/15-media-library.md b/documentation/architecture/15-media-library.md index 4954a418..c7c2d2f8 100644 --- a/documentation/architecture/15-media-library.md +++ b/documentation/architecture/15-media-library.md @@ -166,7 +166,11 @@ directory) with that tag. Facts the link code needs: no capture set (`stacks/classify_binds.go` `classifyRoot`). Measured over the live templates: Transmission's unlisted `watch` bind goes off-site; the download folders are `optional`, so a second drive gets a nightly copy of every download (R-1019). *(Fixed since: catalog `c50adfc` marks the download and watch folders `excluded` — -`app-catalog-felhom.eu/templates/transmission/.felhom.yml:46-51`; noted 2026-10-24.)* **[DESIGN]** The per-drive media bind is declared `excluded` for every media app — the family's +`app-catalog-felhom.eu/templates/transmission/.felhom.yml:46-51`; noted 2026-10-24.)* **[DESIGN — the operator picked a code change 2026-10-25 (D44); row R-1283]:** the same commit also +marks the download folders of NZBGet, SABnzbd and MeTube `excluded` (`templates/{nzbget,sabnzbd,metube}/.felhom.yml`). +NZBGet and SABnzbd stay excluded (their files can be fetched again). The pick for MeTube: `optional`, so the +second-drive copy carries it and the off-site copy does not — a MeTube video can be gone from its source site. Not +built yet. **[DESIGN]** The per-drive media bind is declared `excluded` for every media app — the family's media is no single app's data; whether the library gets its own household-level backup is a separate decision. ## 8. Removal and export assume one drive per app @@ -268,7 +272,10 @@ returns** (G7f). -**[DESIGN — undocumented, found 2026-10-24, review round 2 — confirmed through one register row]:** the „Később" +**[DESIGN — the operator picked a code change 2026-10-25 (D44); row R-1282]:** the „Később" 03:30 media restart is an IN-MEMORY timer — a controller restart forgets it and only the notice stays -(`internal/web/media_events.go` `handleMediaRestart`, `time.AfterFunc`); the Sonarr/Radarr API key is READ from `/config/config.xml`, the „set at -install" option §6 calls cleaner was not built (`internal/mediawire/mediawire.go:336-345`). +(`internal/web/media_events.go` `handleMediaRestart`, `time.AfterFunc`). The pick: the controller stores the due +time and runs the restart after a restart of its own. Not built yet. + +**[DESIGN — confirmed by the operator 2026-10-25 (D44 = all picks, R-1265)]:** the Sonarr/Radarr API key is READ from +`/config/config.xml`; the „set at install" option §6 calls cleaner was not built (`internal/mediawire/mediawire.go:336-345`). diff --git a/documentation/architecture/16-storage-physical-identity.md b/documentation/architecture/16-storage-physical-identity.md index 48d725b3..c2495fcb 100644 --- a/documentation/architecture/16-storage-physical-identity.md +++ b/documentation/architecture/16-storage-physical-identity.md @@ -240,7 +240,7 @@ Also in build B: R-1084 (the whole-box backup offer asks the same-disk question) offered to attach, by device). A bundle that adds a path needs a step bundle on boxes with an older wrapper (R16, R-880): 0.158.1 was delivered through `0.158.1-step1`. -## Decisions in code no section records [DESIGN — undocumented, found 2026-10-24, review round 2 — confirmed through one register row] +## Decisions in code no section records [DESIGN — confirmed by the operator 2026-10-25 (D44 = all picks, R-1265)] Read at `file:line` on `main` by the main session: diff --git a/documentation/audits/release-2026-10-26/d47-setaside-rc7/9202-controller.log b/documentation/audits/release-2026-10-26/d47-setaside-rc7/9202-controller.log new file mode 100644 index 00000000..190da676 --- /dev/null +++ b/documentation/audits/release-2026-10-26/d47-setaside-rc7/9202-controller.log @@ -0,0 +1,1587 @@ +2026/10/25 16:40:00 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:40:00 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 10s ago, effective interval 5m0s, healthy=true +2026/10/25 16:40:00 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping tandoor — last check 1m20s ago, effective interval 5m0s, healthy=true +2026/10/25 16:40:00 healthprobe.go:105: [DEBUG] [stacks] RunHealthProbes: paperless-ngx has a health check but no container to probe — candidates: [paperless-webserver(stopped) paperless-postgres(starting) paperless-redis(starting)] +2026/10/25 16:40:00 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (2 skipped not due, 1 skipped no container) +2026/10/25 16:40:00 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:40:00 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:40:01 manager.go:1719: [DEBUG] Command completed: docker compose up -d (took 11.5s) +2026/10/25 16:40:01 manager.go:1368: [INFO] [stacks] Stack paperless-ngx started successfully (took 11.5s) +2026/10/25 16:40:01 installed.go:409: [DEBUG] [stacks] installed-images paperless-ngx: unchanged (3 service(s)) — app.yaml not rewritten +2026/10/25 16:40:01 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:40:01 backup.go:1038: [INFO] [backup] Stopping privatebin for safe volume dump +2026/10/25 16:40:01 manager.go:1439: [DEBUG] [stacks] StopStack privatebin: current state=stopped deployed=true containers=0 +2026/10/25 16:40:01 manager.go:1442: [INFO] [stacks] Stopping stack: privatebin +2026/10/25 16:40:01 manager.go:1688: [DEBUG] Env vars for compose: [PATH, HOSTNAME, FELHOM_BOOTSTRAP_PATH, HOME, DOMAIN, DOMAIN, SUBDOMAIN, IMPORT_PATH] (8 app + 0 system) +2026/10/25 16:40:01 manager.go:1697: [DEBUG] Running: docker compose down (in /opt/docker/stacks/privatebin) +2026/10/25 16:40:01 manager.go:1719: [DEBUG] Command completed: docker compose down (took 0.1s) +2026/10/25 16:40:01 manager.go:1451: [INFO] [stacks] Stack privatebin stopped successfully (took 0.1s) +2026/10/25 16:40:01 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:40:01 backup.go:910: [DEBUG] [backup] Dumping volume privatebin_privatebin_data for privatebin +2026/10/25 16:40:02 backup.go:935: [INFO] [backup] Volume dump: privatebin/privatebin_privatebin_data → 1.5 KB +2026/10/25 16:40:02 data_versions.go:131: [DEBUG] [backup] privatebin: stamped volume-dumps/privatebin_privatebin_data.tar (1536 B) with pins [privatebin/pdo:2.0.6@sha256:4c141b2326f8b353598ce9ce7507a9cfecf2dad5c60a39fea903d430e296d8f5] +2026/10/25 16:40:02 backup.go:1050: [INFO] [backup] privatebin NOT restarted after the volume dump: the household stopped it meanwhile +2026/10/25 16:40:02 backup.go:1038: [INFO] [backup] Stopping tandoor for safe volume dump +2026/10/25 16:40:02 manager.go:1439: [DEBUG] [stacks] StopStack tandoor: current state=running deployed=true containers=2 +2026/10/25 16:40:02 manager.go:1442: [INFO] [stacks] Stopping stack: tandoor +2026/10/25 16:40:02 manager.go:1688: [DEBUG] Env vars for compose: [PATH, HOSTNAME, FELHOM_BOOTSTRAP_PATH, HOME, DOMAIN, DB_PASSWORD, DOMAIN, SECRET_KEY, SUBDOMAIN, IMPORT_PATH] (10 app + 0 system) +2026/10/25 16:40:02 manager.go:1697: [DEBUG] Running: docker compose down (in /opt/docker/stacks/tandoor) +2026/10/25 16:40:04 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/backup/status +2026/10/25 16:40:04 router.go:78: [DEBUG] [api] GET /api/backup/status (path=/backup/status) +2026/10/25 16:40:04 manager.go:1688: [DEBUG] Env vars for compose: [PATH, HOSTNAME, FELHOM_BOOTSTRAP_PATH, HOME, DOMAIN, PAPERLESS_SECRET_KEY, SUBDOMAIN, DB_PASSWORD, DOMAIN, HDD_PATH, PAPERLESS_ADMIN_PASSWORD, PAPERLESS_ADMIN_USER, PAPERLESS_OCR_LANGUAGE, USERDATA_PATH, IMPORT_PATH] (15 app + 0 system) +2026/10/25 16:40:04 manager.go:1697: [DEBUG] Running: docker compose ps -a --format table {{.Name}} {{.Image}} {{.State}} {{.Status}} (in /opt/docker/stacks/paperless-ngx) +2026/10/25 16:40:04 manager.go:1719: [DEBUG] Command completed: docker compose ps -a --format table {{.Name}} {{.Image}} {{.State}} {{.Status}} (took 0.1s) +2026/10/25 16:40:04 manager.go:1813: [INFO] [stacks] Stack paperless-ngx post-start status: +2026/10/25 16:40:04 manager.go:1816: [INFO] [stacks] paperless-postgres postgres:18-alpine running Up 14 seconds (healthy) +2026/10/25 16:40:04 manager.go:1816: [INFO] [stacks] paperless-redis redis:7-alpine running Up 14 seconds (healthy) +2026/10/25 16:40:04 manager.go:1816: [INFO] [stacks] paperless-webserver ghcr.io/paperless-ngx/paperless-ngx:2.20.15 running Up 3 seconds (health: starting) +2026/10/25 16:40:05 manager.go:1719: [DEBUG] Command completed: docker compose down (took 3.7s) +2026/10/25 16:40:05 manager.go:1451: [INFO] [stacks] Stack tandoor stopped successfully (took 3.7s) +2026/10/25 16:40:05 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:40:05 backup.go:910: [DEBUG] [backup] Dumping volume tandoor_tandoor_media for tandoor +2026/10/25 16:40:05 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting +2026/10/25 16:40:06 backup.go:935: [INFO] [backup] Volume dump: tandoor/tandoor_tandoor_media → 1.5 KB +2026/10/25 16:40:06 data_versions.go:131: [DEBUG] [backup] tandoor: stamped volume-dumps/tandoor_tandoor_media.tar (1536 B) with pins [ghcr.io/tandoorrecipes/recipes:2.6.15@sha256:2e759dd1478a2ed119ee474e28522079fb1cfa50b3fd25cba89f6b9a67abad72 postgres:17-alpine@sha256:b0f9560a2de083e2cc7382e75f808c7381a32852a7ec49117deedb300e552b24] +2026/10/25 16:40:06 backup.go:910: [DEBUG] [backup] Dumping volume tandoor_tandoor_postgres_data for tandoor +2026/10/25 16:40:07 backup.go:935: [INFO] [backup] Volume dump: tandoor/tandoor_tandoor_postgres_data → 53.9 MB +2026/10/25 16:40:07 data_versions.go:131: [DEBUG] [backup] tandoor: stamped volume-dumps/tandoor_tandoor_postgres_data.tar (56563200 B) with pins [ghcr.io/tandoorrecipes/recipes:2.6.15@sha256:2e759dd1478a2ed119ee474e28522079fb1cfa50b3fd25cba89f6b9a67abad72 postgres:17-alpine@sha256:b0f9560a2de083e2cc7382e75f808c7381a32852a7ec49117deedb300e552b24] +2026/10/25 16:40:07 backup.go:910: [DEBUG] [backup] Dumping volume tandoor_tandoor_static for tandoor +2026/10/25 16:40:09 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/backup/status +2026/10/25 16:40:09 router.go:78: [DEBUG] [api] GET /api/backup/status (path=/backup/status) +2026/10/25 16:40:10 backup.go:935: [INFO] [backup] Volume dump: tandoor/tandoor_tandoor_static → 90.0 MB +2026/10/25 16:40:10 data_versions.go:131: [DEBUG] [backup] tandoor: stamped volume-dumps/tandoor_tandoor_static.tar (94322176 B) with pins [ghcr.io/tandoorrecipes/recipes:2.6.15@sha256:2e759dd1478a2ed119ee474e28522079fb1cfa50b3fd25cba89f6b9a67abad72 postgres:17-alpine@sha256:b0f9560a2de083e2cc7382e75f808c7381a32852a7ec49117deedb300e552b24] +2026/10/25 16:40:10 backup.go:1054: [INFO] [backup] Restarting tandoor after volume dump +2026/10/25 16:40:10 manager.go:1350: [DEBUG] [stacks] StartStack tandoor: current state=stopped deployed=true +2026/10/25 16:40:10 manager.go:1353: [INFO] [stacks] Starting stack: tandoor +2026/10/25 16:40:10 manager.go:1360: [DEBUG] [stacks] StartStack tandoor: prepared 10 env vars for compose +2026/10/25 16:40:10 manager.go:1688: [DEBUG] Env vars for compose: [PATH, HOSTNAME, FELHOM_BOOTSTRAP_PATH, HOME, DOMAIN, DB_PASSWORD, DOMAIN, SECRET_KEY, SUBDOMAIN, IMPORT_PATH] (10 app + 0 system) +2026/10/25 16:40:10 manager.go:1697: [DEBUG] Running: docker compose up -d (in /opt/docker/stacks/tandoor) +2026/10/25 16:40:10 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:40:10 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 20s ago, effective interval 5m0s, healthy=true +2026/10/25 16:40:10 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (1 skipped not due, 0 skipped no container) +2026/10/25 16:40:10 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:40:10 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:40:14 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/backup/status +2026/10/25 16:40:14 router.go:78: [DEBUG] [api] GET /api/backup/status (path=/backup/status) +2026/10/25 16:40:16 manager.go:1719: [DEBUG] Command completed: docker compose up -d (took 6.4s) +2026/10/25 16:40:16 manager.go:1368: [INFO] [stacks] Stack tandoor started successfully (took 6.4s) +2026/10/25 16:40:16 installed.go:409: [DEBUG] [stacks] installed-images tandoor: unchanged (2 service(s)) — app.yaml not rewritten +2026/10/25 16:40:16 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:40:16 backup.go:737: [INFO] [backup] App-data backup completed: 3 databases (52.3 MB total), 4 volume dump(s) (1m13.057s) +2026/10/25 16:40:16 recovery_unit.go:297: [INFO] [backup] Recovery unit captured for paperless-ngx → /mnt/felhom-drives/scratch_hdd/userdata/paperless-ngx/backups/primary/paperless-ngx (images=3, secrets-referenced=3, data_keys=0, portable-carried=2/2, withheld=1) +2026/10/25 16:40:16 recovery_unit.go:297: [INFO] [backup] Recovery unit captured for privatebin → /mnt/sys_drive/felhom-data/backups/primary/privatebin (images=1, secrets-referenced=0, data_keys=0, portable-carried=0/0, withheld=0) +2026/10/25 16:40:16 recovery_unit.go:297: [INFO] [backup] Recovery unit captured for tandoor → /mnt/sys_drive/felhom-data/backups/primary/tandoor (images=2, secrets-referenced=2, data_keys=0, portable-carried=2/2, withheld=0) +2026/10/25 16:40:16 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:40:16 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:40:17 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:40:17 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:40:19 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/backup/status +2026/10/25 16:40:19 router.go:78: [DEBUG] [api] GET /api/backup/status (path=/backup/status) +2026/10/25 16:40:19 manager.go:1688: [DEBUG] Env vars for compose: [PATH, HOSTNAME, FELHOM_BOOTSTRAP_PATH, HOME, DOMAIN, DB_PASSWORD, DOMAIN, SECRET_KEY, SUBDOMAIN, IMPORT_PATH] (10 app + 0 system) +2026/10/25 16:40:19 manager.go:1697: [DEBUG] Running: docker compose ps -a --format table {{.Name}} {{.Image}} {{.State}} {{.Status}} (in /opt/docker/stacks/tandoor) +2026/10/25 16:40:19 manager.go:1719: [DEBUG] Command completed: docker compose ps -a --format table {{.Name}} {{.Image}} {{.State}} {{.Status}} (took 0.1s) +2026/10/25 16:40:19 manager.go:1813: [INFO] [stacks] Stack tandoor post-start status: +2026/10/25 16:40:19 manager.go:1816: [INFO] [stacks] tandoor ghcr.io/tandoorrecipes/recipes:2.6.15@sha256:2e759dd1478a2ed119ee474e28522079fb1cfa50b3fd25cba89f6b9a67abad72 running Up 3 seconds (health: starting) +2026/10/25 16:40:19 manager.go:1816: [INFO] [stacks] tandoor-postgres postgres:17-alpine@sha256:b0f9560a2de083e2cc7382e75f808c7381a32852a7ec49117deedb300e552b24 running Up 9 seconds (healthy) +2026/10/25 16:40:20 info.go:13: [DEBUG] [system] GetInfo starting (hddPath="/mnt/felhom-drives/scratch_hdd", hasCPUCollector=true) +2026/10/25 16:40:20 info.go:13: [DEBUG] [system] readMemInfo: guest cap=25898MB (host total was 30714380KB) → used≈13228MB avail≈12670MB +2026/10/25 16:40:20 info.go:13: [DEBUG] [system] readMemInfo: totalKB=30714380 availKB=15025740 → total=25898MB avail=12670MB used=13228MB (51.1%) +2026/10/25 16:40:20 info.go:13: [DEBUG] [system] readDiskUsage: path="/" bsize=4096 total=68.4GB used=13.0GB avail=51.8GB (20.1%) +2026/10/25 16:40:20 info.go:13: [DEBUG] [system] readDiskUsage: path="/mnt/felhom-drives/scratch_hdd" bsize=4096 total=937.8GB used=219.7GB avail=670.4GB (24.7%) +2026/10/25 16:40:20 info.go:13: [DEBUG] [system] readLoadAvg: raw="3.35 2.79 2.06 13/1818 2508" → 1m=3.35 5m=2.79 15m=2.06 +2026/10/25 16:40:20 info.go:13: [DEBUG] [system] readThermalZones: /sys — found 1 zones +2026/10/25 16:40:20 info.go:13: [DEBUG] [system] readTemperature: found via hwmon at /sys — 63.1°C (hwmon2) +2026/10/25 16:40:20 info.go:13: [DEBUG] [system] GetInfo done in 66ms — mem=13228MB/25898MB (51.1%), rootDisk=13.0GB/68.4GB (20.1%), load=3.35/2.79/2.06, temp=63.1°C (hwmon2), cpu=29.9% +2026/10/25 16:40:20 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:40:20 scheduler.go:450: [INFO] [scheduler] Running job: stack-scan +2026/10/25 16:40:20 scheduler.go:85: [DEBUG] [scheduler] job stack-scan: execution starting +2026/10/25 16:40:20 scheduler.go:85: [DEBUG] [scheduler] job ring-spill: execution starting +2026/10/25 16:40:20 scheduler.go:85: [DEBUG] [scheduler] job deadapp-check: execution starting +2026/10/25 16:40:20 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting +2026/10/25 16:40:20 scheduler.go:85: [DEBUG] [scheduler] job after-unclean-boot: execution starting +2026/10/25 16:40:20 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:40:20 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:40:20 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 30s ago, effective interval 5m0s, healthy=true +2026/10/25 16:40:20 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (1 skipped not due, 0 skipped no container) +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "actualbudget" deployed=false composePath=/opt/docker/stacks/actualbudget/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "adventurelog" deployed=false composePath=/opt/docker/stacks/adventurelog/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "audiobookshelf" deployed=false composePath=/opt/docker/stacks/audiobookshelf/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "bentopdf" deployed=false composePath=/opt/docker/stacks/bentopdf/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "bookstack" deployed=false composePath=/opt/docker/stacks/bookstack/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "calcom" deployed=false composePath=/opt/docker/stacks/calcom/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "calibre-web" deployed=false composePath=/opt/docker/stacks/calibre-web/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "chaoscrash" deployed=false composePath=/opt/docker/stacks/chaoscrash/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "chaosoom" deployed=false composePath=/opt/docker/stacks/chaosoom/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "chaosoomb" deployed=false composePath=/opt/docker/stacks/chaosoomb/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "claper" deployed=false composePath=/opt/docker/stacks/claper/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "code-server" deployed=false composePath=/opt/docker/stacks/code-server/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "crafty-controller" deployed=false composePath=/opt/docker/stacks/crafty-controller/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "dawarich" deployed=false composePath=/opt/docker/stacks/dawarich/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "docmost" deployed=false composePath=/opt/docker/stacks/docmost/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "emby" deployed=false composePath=/opt/docker/stacks/emby/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "filebrowser" deployed=false composePath=/opt/docker/stacks/filebrowser/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "ghost" deployed=false composePath=/opt/docker/stacks/ghost/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "gitea" deployed=false composePath=/opt/docker/stacks/gitea/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "glance" deployed=false composePath=/opt/docker/stacks/glance/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "gokapi" deployed=false composePath=/opt/docker/stacks/gokapi/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "grafana" deployed=false composePath=/opt/docker/stacks/grafana/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "gramps-web" deployed=false composePath=/opt/docker/stacks/gramps-web/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "grimmory" deployed=false composePath=/opt/docker/stacks/grimmory/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "grocy" deployed=false composePath=/opt/docker/stacks/grocy/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "home-assistant" deployed=false composePath=/opt/docker/stacks/home-assistant/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "homebox" deployed=false composePath=/opt/docker/stacks/homebox/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "homepage" deployed=false composePath=/opt/docker/stacks/homepage/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "immich" deployed=true composePath=/opt/docker/stacks/immich/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "jellyfin" deployed=false composePath=/opt/docker/stacks/jellyfin/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "karakeep" deployed=false composePath=/opt/docker/stacks/karakeep/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "kimai" deployed=false composePath=/opt/docker/stacks/kimai/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "komga" deployed=false composePath=/opt/docker/stacks/komga/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "lubelogger" deployed=false composePath=/opt/docker/stacks/lubelogger/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "mealie" deployed=false composePath=/opt/docker/stacks/mealie/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "metube" deployed=false composePath=/opt/docker/stacks/metube/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "n8n" deployed=false composePath=/opt/docker/stacks/n8n/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "navidrome" deployed=false composePath=/opt/docker/stacks/navidrome/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "nextcloud" deployed=false composePath=/opt/docker/stacks/nextcloud/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "nzbget" deployed=false composePath=/opt/docker/stacks/nzbget/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "onlyoffice" deployed=false composePath=/opt/docker/stacks/onlyoffice/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "opengist" deployed=false composePath=/opt/docker/stacks/opengist/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "outline" deployed=false composePath=/opt/docker/stacks/outline/docker-compose.yml +2026/10/25 16:40:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "paperless-ngx" deployed=true composePath=/opt/docker/stacks/paperless-ngx/docker-compose.yml +2026/10/25 16:40:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "papra" deployed=false composePath=/opt/docker/stacks/papra/docker-compose.yml +2026/10/25 16:40:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "plant-it" deployed=false composePath=/opt/docker/stacks/plant-it/docker-compose.yml +2026/10/25 16:40:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "plex" deployed=false composePath=/opt/docker/stacks/plex/docker-compose.yml +2026/10/25 16:40:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "privatebin" deployed=true composePath=/opt/docker/stacks/privatebin/docker-compose.yml +2026/10/25 16:40:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "prowlarr" deployed=false composePath=/opt/docker/stacks/prowlarr/docker-compose.yml +2026/10/25 16:40:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "qbittorrent" deployed=false composePath=/opt/docker/stacks/qbittorrent/docker-compose.yml +2026/10/25 16:40:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "radarr" deployed=false composePath=/opt/docker/stacks/radarr/docker-compose.yml +2026/10/25 16:40:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "radicale" deployed=false composePath=/opt/docker/stacks/radicale/docker-compose.yml +2026/10/25 16:40:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "rallly" deployed=false composePath=/opt/docker/stacks/rallly/docker-compose.yml +2026/10/25 16:40:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "recipe-importer" deployed=false composePath=/opt/docker/stacks/recipe-importer/docker-compose.yml +2026/10/25 16:40:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "romm" deployed=false composePath=/opt/docker/stacks/romm/docker-compose.yml +2026/10/25 16:40:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "sabnzbd" deployed=false composePath=/opt/docker/stacks/sabnzbd/docker-compose.yml +2026/10/25 16:40:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "seerr" deployed=false composePath=/opt/docker/stacks/seerr/docker-compose.yml +2026/10/25 16:40:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "sonarr" deployed=false composePath=/opt/docker/stacks/sonarr/docker-compose.yml +2026/10/25 16:40:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "sparkyfitness" deployed=false composePath=/opt/docker/stacks/sparkyfitness/docker-compose.yml +2026/10/25 16:40:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "tandoor" deployed=true composePath=/opt/docker/stacks/tandoor/docker-compose.yml +2026/10/25 16:40:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "termix" deployed=false composePath=/opt/docker/stacks/termix/docker-compose.yml +2026/10/25 16:40:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "traefik" deployed=false composePath=/opt/docker/stacks/traefik/docker-compose.yml +2026/10/25 16:40:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "transmission" deployed=false composePath=/opt/docker/stacks/transmission/docker-compose.yml +2026/10/25 16:40:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "uptime-kuma" deployed=false composePath=/opt/docker/stacks/uptime-kuma/docker-compose.yml +2026/10/25 16:40:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "vaultwarden" deployed=false composePath=/opt/docker/stacks/vaultwarden/docker-compose.yml +2026/10/25 16:40:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "vikunja" deployed=false composePath=/opt/docker/stacks/vikunja/docker-compose.yml +2026/10/25 16:40:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "wanderer" deployed=false composePath=/opt/docker/stacks/wanderer/docker-compose.yml +2026/10/25 16:40:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "wger" deployed=false composePath=/opt/docker/stacks/wger/docker-compose.yml +2026/10/25 16:40:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "wishlist" deployed=false composePath=/opt/docker/stacks/wishlist/docker-compose.yml +2026/10/25 16:40:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "zipline" deployed=false composePath=/opt/docker/stacks/zipline/docker-compose.yml +2026/10/25 16:40:21 manager.go:1960: [DEBUG] [stacks] getCatalogTemplateSlugs: found 65 template slugs in /opt/docker/felhom-controller/data/catalog-cache/templates +2026/10/25 16:40:21 manager.go:757: [DEBUG] [stacks] ScanStacks: catalog has 65 template slugs for orphan detection +2026/10/25 16:40:21 manager.go:788: [INFO] [stacks] ScanStacks complete: 70 stacks found (4 deployed, 66 available) +2026/10/25 16:40:21 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:40:21 scheduler.go:467: [INFO] [scheduler] Job stack-scan completed (took 179ms) +2026/10/25 16:40:23 auth.go:144: [DEBUG] [web] auth: valid session for POST /api/stacks/tandoor/stop +2026/10/25 16:40:23 router.go:78: [DEBUG] [api] POST /api/stacks/tandoor/stop (path=/stacks/tandoor/stop) +2026/10/25 16:40:23 router.go:663: [INFO] [api] stop requested for stack: tandoor +2026/10/25 16:40:23 router.go:78: [DEBUG] [api] actionStack: action=stop name=tandoor +2026/10/25 16:40:23 [DEBUG] [stacks] SaveAppConfig: saving /opt/docker/stacks/tandoor — 4 env vars, 0 encrypted, 2 sensitive fields +2026/10/25 16:40:23 [INFO] [stacks] SaveAppConfig: saved config for tandoor +2026/10/25 16:40:23 desiredstate.go:81: [INFO] [stacks] desired state for tandoor recorded as "stopped" (was "running") +2026/10/25 16:40:23 manager.go:1439: [DEBUG] [stacks] StopStack tandoor: current state=starting deployed=true containers=2 +2026/10/25 16:40:23 manager.go:1442: [INFO] [stacks] Stopping stack: tandoor +2026/10/25 16:40:23 manager.go:1688: [DEBUG] Env vars for compose: [PATH, HOSTNAME, FELHOM_BOOTSTRAP_PATH, HOME, DOMAIN, SUBDOMAIN, DB_PASSWORD, DOMAIN, SECRET_KEY, IMPORT_PATH] (10 app + 0 system) +2026/10/25 16:40:23 manager.go:1697: [DEBUG] Running: docker compose down (in /opt/docker/stacks/tandoor) +2026/10/25 16:40:24 manager.go:1719: [DEBUG] Command completed: docker compose down (took 0.9s) +2026/10/25 16:40:24 manager.go:1451: [INFO] [stacks] Stack tandoor stopped successfully (took 0.9s) +2026/10/25 16:40:24 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:40:24 lifecycle.go:24: [DEBUG] [integrations] OnStackStop: stack=tandoor integrationsFound=0 +2026/10/25 16:40:29 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:40:29 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:40:29 auth.go:144: [DEBUG] [web] auth: valid session for POST /api/stacks/tandoor/remove +2026/10/25 16:40:29 router.go:78: [DEBUG] [api] POST /api/stacks/tandoor/remove (path=/stacks/tandoor/remove) +2026/10/25 16:40:29 router.go:1025: [INFO] [api] Remove requested for stack: tandoor +2026/10/25 16:40:29 router.go:78: [DEBUG] [api] removeStack: name=tandoor +2026/10/25 16:40:29 router.go:78: [DEBUG] [api] removeStack: name=tandoor removeHDDData=false removeBackups=false +2026/10/25 16:40:29 delete.go:626: [DEBUG] [stacks] RemoveStack called: name="tandoor", removeHDDData=false, backupPathsToRemove=0 +2026/10/25 16:40:29 delete.go:640: [DEBUG] [stacks] RemoveStack tandoor: state=stopped, deployed=true, orphaned=false, deploying=false +2026/10/25 16:40:29 delete.go:697: [INFO] Removing deployed stack: tandoor (removeHDDData=false, hddDeclared=false, backupPaths=0) +2026/10/25 16:40:29 delete.go:710: [DEBUG] [stacks] RemoveStack tandoor: found 0 HDD mounts from compose file +2026/10/25 16:40:29 manager.go:1688: [DEBUG] Env vars for compose: [PATH, HOSTNAME, FELHOM_BOOTSTRAP_PATH, HOME, DOMAIN, DB_PASSWORD, DOMAIN, SECRET_KEY, SUBDOMAIN, IMPORT_PATH] (10 app + 0 system) +2026/10/25 16:40:29 manager.go:1697: [DEBUG] Running: docker compose down --volumes (in /opt/docker/stacks/tandoor) +2026/10/25 16:40:29 manager.go:1719: [DEBUG] Command completed: docker compose down --volumes (took 0.5s) +2026/10/25 16:40:29 delete.go:736: [DEBUG] [stacks] RemoveStack tandoor: compose down output: +2026/10/25 16:40:30 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:40:30 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 40s ago, effective interval 5m0s, healthy=true +2026/10/25 16:40:30 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (1 skipped not due, 0 skipped no container) +2026/10/25 16:40:30 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:40:30 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:40:35 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting +2026/10/25 16:40:40 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:40:40 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 50s ago, effective interval 5m0s, healthy=true +2026/10/25 16:40:40 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (1 skipped not due, 0 skipped no container) +2026/10/25 16:40:40 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:40:40 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:40:50 scheduler.go:85: [DEBUG] [scheduler] job after-unclean-boot: execution starting +2026/10/25 16:40:50 scheduler.go:85: [DEBUG] [scheduler] job deadapp-check: execution starting +2026/10/25 16:40:50 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting +2026/10/25 16:40:50 scheduler.go:85: [DEBUG] [scheduler] job ring-spill: execution starting +2026/10/25 16:40:50 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:40:50 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 1m0s ago, effective interval 5m0s, healthy=true +2026/10/25 16:40:50 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 1 targets (1 skipped not due, 0 skipped no container) +2026/10/25 16:40:50 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:40:50 healthprobe.go:187: [DEBUG] Health probe paperless-ngx: HTTP GET :8000/ → 302 (4ms) +2026/10/25 16:40:50 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:40:50 healthprobe.go:167: [INFO] Health probes: 1 ok (of 1 probed) +2026/10/25 16:40:56 delete.go:774: [INFO] [stacks] RemoveStack tandoor: removed volume(s) [tandoor_tandoor_media tandoor_tandoor_postgres_data tandoor_tandoor_static] +2026/10/25 16:40:56 delete.go:837: [DEBUG] [stacks] RemoveStack tandoor: processing 0 backup paths for removal (base=/mnt/sys_drive/felhom-data/backups) +2026/10/25 16:40:56 delete.go:862: [DEBUG] [stacks] RemoveStack tandoor: removing app.yaml at /opt/docker/stacks/tandoor/app.yaml +2026/10/25 16:40:56 delete.go:879: [INFO] Stack tandoor removed successfully (took 27.2s) +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "actualbudget" deployed=false composePath=/opt/docker/stacks/actualbudget/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "adventurelog" deployed=false composePath=/opt/docker/stacks/adventurelog/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "audiobookshelf" deployed=false composePath=/opt/docker/stacks/audiobookshelf/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "bentopdf" deployed=false composePath=/opt/docker/stacks/bentopdf/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "bookstack" deployed=false composePath=/opt/docker/stacks/bookstack/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "calcom" deployed=false composePath=/opt/docker/stacks/calcom/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "calibre-web" deployed=false composePath=/opt/docker/stacks/calibre-web/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "chaoscrash" deployed=false composePath=/opt/docker/stacks/chaoscrash/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "chaosoom" deployed=false composePath=/opt/docker/stacks/chaosoom/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "chaosoomb" deployed=false composePath=/opt/docker/stacks/chaosoomb/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "claper" deployed=false composePath=/opt/docker/stacks/claper/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "code-server" deployed=false composePath=/opt/docker/stacks/code-server/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "crafty-controller" deployed=false composePath=/opt/docker/stacks/crafty-controller/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "dawarich" deployed=false composePath=/opt/docker/stacks/dawarich/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "docmost" deployed=false composePath=/opt/docker/stacks/docmost/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "emby" deployed=false composePath=/opt/docker/stacks/emby/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "filebrowser" deployed=false composePath=/opt/docker/stacks/filebrowser/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "ghost" deployed=false composePath=/opt/docker/stacks/ghost/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "gitea" deployed=false composePath=/opt/docker/stacks/gitea/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "glance" deployed=false composePath=/opt/docker/stacks/glance/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "gokapi" deployed=false composePath=/opt/docker/stacks/gokapi/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "grafana" deployed=false composePath=/opt/docker/stacks/grafana/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "gramps-web" deployed=false composePath=/opt/docker/stacks/gramps-web/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "grimmory" deployed=false composePath=/opt/docker/stacks/grimmory/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "grocy" deployed=false composePath=/opt/docker/stacks/grocy/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "home-assistant" deployed=false composePath=/opt/docker/stacks/home-assistant/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "homebox" deployed=false composePath=/opt/docker/stacks/homebox/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "homepage" deployed=false composePath=/opt/docker/stacks/homepage/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "immich" deployed=true composePath=/opt/docker/stacks/immich/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "jellyfin" deployed=false composePath=/opt/docker/stacks/jellyfin/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "karakeep" deployed=false composePath=/opt/docker/stacks/karakeep/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "kimai" deployed=false composePath=/opt/docker/stacks/kimai/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "komga" deployed=false composePath=/opt/docker/stacks/komga/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "lubelogger" deployed=false composePath=/opt/docker/stacks/lubelogger/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "mealie" deployed=false composePath=/opt/docker/stacks/mealie/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "metube" deployed=false composePath=/opt/docker/stacks/metube/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "n8n" deployed=false composePath=/opt/docker/stacks/n8n/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "navidrome" deployed=false composePath=/opt/docker/stacks/navidrome/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "nextcloud" deployed=false composePath=/opt/docker/stacks/nextcloud/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "nzbget" deployed=false composePath=/opt/docker/stacks/nzbget/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "onlyoffice" deployed=false composePath=/opt/docker/stacks/onlyoffice/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "opengist" deployed=false composePath=/opt/docker/stacks/opengist/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "outline" deployed=false composePath=/opt/docker/stacks/outline/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "paperless-ngx" deployed=true composePath=/opt/docker/stacks/paperless-ngx/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "papra" deployed=false composePath=/opt/docker/stacks/papra/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "plant-it" deployed=false composePath=/opt/docker/stacks/plant-it/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "plex" deployed=false composePath=/opt/docker/stacks/plex/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "privatebin" deployed=true composePath=/opt/docker/stacks/privatebin/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "prowlarr" deployed=false composePath=/opt/docker/stacks/prowlarr/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "qbittorrent" deployed=false composePath=/opt/docker/stacks/qbittorrent/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "radarr" deployed=false composePath=/opt/docker/stacks/radarr/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "radicale" deployed=false composePath=/opt/docker/stacks/radicale/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "rallly" deployed=false composePath=/opt/docker/stacks/rallly/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "recipe-importer" deployed=false composePath=/opt/docker/stacks/recipe-importer/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "romm" deployed=false composePath=/opt/docker/stacks/romm/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "sabnzbd" deployed=false composePath=/opt/docker/stacks/sabnzbd/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "seerr" deployed=false composePath=/opt/docker/stacks/seerr/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "sonarr" deployed=false composePath=/opt/docker/stacks/sonarr/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "sparkyfitness" deployed=false composePath=/opt/docker/stacks/sparkyfitness/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "tandoor" deployed=false composePath=/opt/docker/stacks/tandoor/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "termix" deployed=false composePath=/opt/docker/stacks/termix/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "traefik" deployed=false composePath=/opt/docker/stacks/traefik/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "transmission" deployed=false composePath=/opt/docker/stacks/transmission/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "uptime-kuma" deployed=false composePath=/opt/docker/stacks/uptime-kuma/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "vaultwarden" deployed=false composePath=/opt/docker/stacks/vaultwarden/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "vikunja" deployed=false composePath=/opt/docker/stacks/vikunja/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "wanderer" deployed=false composePath=/opt/docker/stacks/wanderer/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "wger" deployed=false composePath=/opt/docker/stacks/wger/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "wishlist" deployed=false composePath=/opt/docker/stacks/wishlist/docker-compose.yml +2026/10/25 16:40:56 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "zipline" deployed=false composePath=/opt/docker/stacks/zipline/docker-compose.yml +2026/10/25 16:40:56 manager.go:1960: [DEBUG] [stacks] getCatalogTemplateSlugs: found 65 template slugs in /opt/docker/felhom-controller/data/catalog-cache/templates +2026/10/25 16:40:56 manager.go:757: [DEBUG] [stacks] ScanStacks: catalog has 65 template slugs for orphan detection +2026/10/25 16:40:56 manager.go:788: [INFO] [stacks] ScanStacks complete: 70 stacks found (3 deployed, 67 available) +2026/10/25 16:40:56 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:40:56 notifier.go:1463: [WARN] notifier disabled (no hub configured): DROPPED event app_removed (severity info) — further app_removed events are logged at DEBUG only +2026/10/25 16:40:56 lifecycle.go:176: [DEBUG] [integrations] OnStackRemove: stack=tandoor integrationsFound=0 +2026/10/25 16:40:58 image_retention.go:265: [INFO] [stacks] image retention (remove of tandoor): deleted [ghcr.io/tandoorrecipes/recipes:] (21ca08430f20, 919MB) — no container, installed app or undo names it (decision 53) +2026/10/25 16:40:59 image_retention.go:265: [INFO] [stacks] image retention (remove of tandoor): deleted [postgres:] (79bd7c99e923, 297MB) — no container, installed app or undo names it (decision 53) +2026/10/25 16:40:59 image_retention.go:238: [INFO] [stacks] image retention (remove of tandoor): pass over 45 image(s) of [ghcr.io/tandoorrecipes/recipes postgres] — 2 candidate(s), 2 deleted, the rest kept +2026/10/25 16:41:00 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:41:00 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 1m10s ago, effective interval 5m0s, healthy=true +2026/10/25 16:41:00 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 10s ago, effective interval 5m0s, healthy=true +2026/10/25 16:41:00 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (2 skipped not due, 0 skipped no container) +2026/10/25 16:41:00 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:41:00 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:41:00 setup_gate.go:492: [INFO] [stacks] tandoor: removed a setup-gate file for an app whose gate is not closed +2026/10/25 16:41:01 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:41:01 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:41:01 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:41:01 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:41:01 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor/deploy-fields +2026/10/25 16:41:01 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor/deploy-fields (path=/stacks/tandoor/deploy-fields) +2026/10/25 16:41:01 auth.go:144: [DEBUG] [web] auth: valid session for POST /api/stacks/tandoor/deploy +2026/10/25 16:41:01 router.go:78: [DEBUG] [api] POST /api/stacks/tandoor/deploy (path=/stacks/tandoor/deploy) +2026/10/25 16:41:01 router.go:442: [INFO] [api] Deploy requested for stack: tandoor +2026/10/25 16:41:01 router.go:78: [DEBUG] [api] deployStack: name=tandoor contentLength=64 +2026/10/25 16:41:01 info.go:13: [DEBUG] [system] GetDiskUsage: path="/" total=68.4 GB used=11.5 GB avail=53.3GB (17.8%) +2026/10/25 16:41:01 info.go:13: [DEBUG] [system] readMemInfo: guest cap=25898MB (host total was 30714380KB) → used≈13606MB avail≈12292MB +2026/10/25 16:41:01 info.go:13: [DEBUG] [system] readMemInfo: totalKB=30714380 availKB=14578080 → total=25898MB avail=12292MB used=13606MB (52.5%) +2026/10/25 16:41:01 deploy.go:1604: [INFO] [stacks] Memory check: total=25898MB, reserved=384MB, usable=25514MB, committed_used=2648MB, new_req=900MB, remaining=21966MB +2026/10/25 16:41:01 deploy.go:338: [DEBUG] Deploy tandoor: received 2 user values +2026/10/25 16:41:01 deploy.go:343: [DEBUG] DOMAIN = "enkisfelhom.hu" +2026/10/25 16:41:01 deploy.go:343: [DEBUG] SUBDOMAIN = "recipes" +2026/10/25 16:41:01 setup_gate.go:295: [INFO] [stacks] tandoor: setup gate CLOSED before the first start — only the household reaches [recipes.enkisfelhom.hu] until the first setup is done +2026/10/25 16:41:01 [DEBUG] [stacks] SaveAppConfig: saving /opt/docker/stacks/tandoor — 4 env vars, 2 encrypted, 2 sensitive fields +2026/10/25 16:41:01 [INFO] [stacks] SaveAppConfig: saved config for tandoor +2026/10/25 16:41:01 deploy.go:574: [INFO] [stacks] Deploying stack tandoor with 4 env vars: [DOMAIN, SUBDOMAIN, SECRET_KEY, DB_PASSWORD] +2026/10/25 16:41:01 manager.go:1852: [INFO] [stacks] Deploying stack tandoor — checking 2 images... +2026/10/25 16:41:01 manager.go:1856: [DEBUG] ghcr.io/tandoorrecipes/recipes:2.6.15@sha256:2e759dd1478a2ed119ee474e28522079fb1cfa50b3fd25cba89f6b9a67abad72 — not found locally, will pull +2026/10/25 16:41:02 manager.go:1856: [DEBUG] postgres:17-alpine@sha256:b0f9560a2de083e2cc7382e75f808c7381a32852a7ec49117deedb300e552b24 — not found locally, will pull +2026/10/25 16:41:02 notifier.go:1467: [DEBUG] notifier disabled: dropped event app_deploy_started (severity info) +2026/10/25 16:41:02 lifecycle.go:70: [DEBUG] [integrations] OnStackStart: stack=tandoor, waiting 5s for state refresh +2026/10/25 16:41:02 manager.go:1688: [DEBUG] Env vars for compose: [PATH, HOSTNAME, FELHOM_BOOTSTRAP_PATH, HOME, DB_PASSWORD, DOMAIN, SUBDOMAIN, SECRET_KEY, DOMAIN, IMPORT_PATH] (10 app + 0 system) +2026/10/25 16:41:02 manager.go:1697: [DEBUG] Running: docker compose up -d (in /opt/docker/stacks/tandoor) +2026/10/25 16:41:05 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting +2026/10/25 16:41:07 lifecycle.go:86: [DEBUG] [integrations] OnStackStart: stack=tandoor integrationsFound=0 +2026/10/25 16:41:07 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:41:07 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:41:10 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:41:10 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:41:10 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 1m20s ago, effective interval 5m0s, healthy=true +2026/10/25 16:41:10 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 20s ago, effective interval 5m0s, healthy=true +2026/10/25 16:41:10 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (2 skipped not due, 0 skipped no container) +2026/10/25 16:41:10 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:41:12 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:41:12 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:41:17 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:41:17 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:41:20 info.go:13: [DEBUG] [system] GetInfo starting (hddPath="/mnt/felhom-drives/scratch_hdd", hasCPUCollector=true) +2026/10/25 16:41:20 info.go:13: [DEBUG] [system] readMemInfo: guest cap=25898MB (host total was 30714380KB) → used≈13530MB avail≈12368MB +2026/10/25 16:41:20 info.go:13: [DEBUG] [system] readMemInfo: totalKB=30714380 availKB=14667220 → total=25898MB avail=12368MB used=13530MB (52.2%) +2026/10/25 16:41:20 info.go:13: [DEBUG] [system] readDiskUsage: path="/" bsize=4096 total=68.4GB used=12.2GB avail=52.7GB (18.8%) +2026/10/25 16:41:20 info.go:13: [DEBUG] [system] readDiskUsage: path="/mnt/felhom-drives/scratch_hdd" bsize=4096 total=937.8GB used=219.7GB avail=670.4GB (24.7%) +2026/10/25 16:41:20 info.go:13: [DEBUG] [system] readLoadAvg: raw="2.79 2.72 2.09 12/1725 3111" → 1m=2.79 5m=2.72 15m=2.09 +2026/10/25 16:41:20 info.go:13: [DEBUG] [system] readThermalZones: /sys — found 1 zones +2026/10/25 16:41:20 info.go:13: [DEBUG] [system] readTemperature: found via hwmon at /sys — 56.0°C (hwmon2) +2026/10/25 16:41:20 info.go:13: [DEBUG] [system] GetInfo done in 56ms — mem=13530MB/25898MB (52.2%), rootDisk=12.2GB/68.4GB (18.8%), load=2.79/2.72/2.09, temp=56.0°C (hwmon2), cpu=33.8% +2026/10/25 16:41:20 scheduler.go:85: [DEBUG] [scheduler] job ring-spill: execution starting +2026/10/25 16:41:20 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:41:20 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 1m30s ago, effective interval 5m0s, healthy=true +2026/10/25 16:41:20 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 30s ago, effective interval 5m0s, healthy=true +2026/10/25 16:41:20 scheduler.go:85: [DEBUG] [scheduler] job deadapp-check: execution starting +2026/10/25 16:41:20 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:41:20 scheduler.go:85: [DEBUG] [scheduler] job after-unclean-boot: execution starting +2026/10/25 16:41:20 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting +2026/10/25 16:41:20 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (2 skipped not due, 0 skipped no container) +2026/10/25 16:41:20 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:41:22 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:41:22 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:41:27 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:41:27 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:41:30 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:41:30 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:41:30 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 1m40s ago, effective interval 5m0s, healthy=true +2026/10/25 16:41:30 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 40s ago, effective interval 5m0s, healthy=true +2026/10/25 16:41:30 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (2 skipped not due, 0 skipped no container) +2026/10/25 16:41:30 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:41:32 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:41:32 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:41:35 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting +2026/10/25 16:41:37 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:41:37 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:41:40 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:41:40 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:41:40 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:41:40 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 1m50s ago, effective interval 5m0s, healthy=true +2026/10/25 16:41:40 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 50s ago, effective interval 5m0s, healthy=true +2026/10/25 16:41:40 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (2 skipped not due, 0 skipped no container) +2026/10/25 16:41:42 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:41:42 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:41:47 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:41:47 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:41:50 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting +2026/10/25 16:41:50 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:41:50 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 1m0s ago, effective interval 5m0s, healthy=true +2026/10/25 16:41:50 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 2m0s ago, effective interval 5m0s, healthy=true +2026/10/25 16:41:50 scheduler.go:85: [DEBUG] [scheduler] job deadapp-check: execution starting +2026/10/25 16:41:50 scheduler.go:85: [DEBUG] [scheduler] job ring-spill: execution starting +2026/10/25 16:41:50 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:41:50 scheduler.go:85: [DEBUG] [scheduler] job after-unclean-boot: execution starting +2026/10/25 16:41:50 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (2 skipped not due, 0 skipped no container) +2026/10/25 16:41:50 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:41:52 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:41:52 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:41:57 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:41:57 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:42:00 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:42:00 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:42:00 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:42:00 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 2m10s ago, effective interval 5m0s, healthy=true +2026/10/25 16:42:00 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 1m10s ago, effective interval 5m0s, healthy=true +2026/10/25 16:42:00 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (2 skipped not due, 0 skipped no container) +2026/10/25 16:42:02 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:42:02 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:42:05 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting +2026/10/25 16:42:07 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:42:07 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:42:10 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:42:10 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:42:10 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 2m20s ago, effective interval 5m0s, healthy=true +2026/10/25 16:42:10 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 1m20s ago, effective interval 5m0s, healthy=true +2026/10/25 16:42:10 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (2 skipped not due, 0 skipped no container) +2026/10/25 16:42:10 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:42:12 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:42:12 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:42:17 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:42:17 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:42:20 info.go:13: [DEBUG] [system] GetInfo starting (hddPath="/mnt/felhom-drives/scratch_hdd", hasCPUCollector=true) +2026/10/25 16:42:20 info.go:13: [DEBUG] [system] readMemInfo: guest cap=25898MB (host total was 30714380KB) → used≈13125MB avail≈12773MB +2026/10/25 16:42:20 info.go:13: [DEBUG] [system] readMemInfo: totalKB=30714380 availKB=15148204 → total=25898MB avail=12773MB used=13125MB (50.7%) +2026/10/25 16:42:20 info.go:13: [DEBUG] [system] readDiskUsage: path="/" bsize=4096 total=68.4GB used=12.8GB avail=52.0GB (19.8%) +2026/10/25 16:42:20 info.go:13: [DEBUG] [system] readDiskUsage: path="/mnt/felhom-drives/scratch_hdd" bsize=4096 total=937.8GB used=219.7GB avail=670.4GB (24.7%) +2026/10/25 16:42:20 info.go:13: [DEBUG] [system] readLoadAvg: raw="2.93 2.74 2.13 6/1708 3266" → 1m=2.93 5m=2.74 15m=2.13 +2026/10/25 16:42:20 info.go:13: [DEBUG] [system] readThermalZones: /sys — found 1 zones +2026/10/25 16:42:20 info.go:13: [DEBUG] [system] readTemperature: found via hwmon at /sys — 49.9°C (hwmon2) +2026/10/25 16:42:20 info.go:13: [DEBUG] [system] GetInfo done in 92ms — mem=13125MB/25898MB (50.7%), rootDisk=12.8GB/68.4GB (19.8%), load=2.93/2.74/2.13, temp=49.9°C (hwmon2), cpu=16.7% +2026/10/25 16:42:20 scheduler.go:85: [DEBUG] [scheduler] job deadapp-check: execution starting +2026/10/25 16:42:20 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:42:20 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting +2026/10/25 16:42:20 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:42:20 scheduler.go:450: [INFO] [scheduler] Running job: stack-scan +2026/10/25 16:42:20 scheduler.go:85: [DEBUG] [scheduler] job ring-spill: execution starting +2026/10/25 16:42:20 scheduler.go:85: [DEBUG] [scheduler] job stack-scan: execution starting +2026/10/25 16:42:20 scheduler.go:85: [DEBUG] [scheduler] job after-unclean-boot: execution starting +2026/10/25 16:42:20 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:42:20 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 2m30s ago, effective interval 5m0s, healthy=true +2026/10/25 16:42:20 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 1m30s ago, effective interval 5m0s, healthy=true +2026/10/25 16:42:20 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (2 skipped not due, 0 skipped no container) +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "actualbudget" deployed=false composePath=/opt/docker/stacks/actualbudget/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "adventurelog" deployed=false composePath=/opt/docker/stacks/adventurelog/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "audiobookshelf" deployed=false composePath=/opt/docker/stacks/audiobookshelf/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "bentopdf" deployed=false composePath=/opt/docker/stacks/bentopdf/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "bookstack" deployed=false composePath=/opt/docker/stacks/bookstack/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "calcom" deployed=false composePath=/opt/docker/stacks/calcom/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "calibre-web" deployed=false composePath=/opt/docker/stacks/calibre-web/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "chaoscrash" deployed=false composePath=/opt/docker/stacks/chaoscrash/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "chaosoom" deployed=false composePath=/opt/docker/stacks/chaosoom/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "chaosoomb" deployed=false composePath=/opt/docker/stacks/chaosoomb/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "claper" deployed=false composePath=/opt/docker/stacks/claper/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "code-server" deployed=false composePath=/opt/docker/stacks/code-server/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "crafty-controller" deployed=false composePath=/opt/docker/stacks/crafty-controller/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "dawarich" deployed=false composePath=/opt/docker/stacks/dawarich/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "docmost" deployed=false composePath=/opt/docker/stacks/docmost/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "emby" deployed=false composePath=/opt/docker/stacks/emby/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "filebrowser" deployed=false composePath=/opt/docker/stacks/filebrowser/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "ghost" deployed=false composePath=/opt/docker/stacks/ghost/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "gitea" deployed=false composePath=/opt/docker/stacks/gitea/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "glance" deployed=false composePath=/opt/docker/stacks/glance/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "gokapi" deployed=false composePath=/opt/docker/stacks/gokapi/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "grafana" deployed=false composePath=/opt/docker/stacks/grafana/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "gramps-web" deployed=false composePath=/opt/docker/stacks/gramps-web/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "grimmory" deployed=false composePath=/opt/docker/stacks/grimmory/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "grocy" deployed=false composePath=/opt/docker/stacks/grocy/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "home-assistant" deployed=false composePath=/opt/docker/stacks/home-assistant/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "homebox" deployed=false composePath=/opt/docker/stacks/homebox/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "homepage" deployed=false composePath=/opt/docker/stacks/homepage/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "immich" deployed=true composePath=/opt/docker/stacks/immich/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "jellyfin" deployed=false composePath=/opt/docker/stacks/jellyfin/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "karakeep" deployed=false composePath=/opt/docker/stacks/karakeep/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "kimai" deployed=false composePath=/opt/docker/stacks/kimai/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "komga" deployed=false composePath=/opt/docker/stacks/komga/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "lubelogger" deployed=false composePath=/opt/docker/stacks/lubelogger/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "mealie" deployed=false composePath=/opt/docker/stacks/mealie/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "metube" deployed=false composePath=/opt/docker/stacks/metube/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "n8n" deployed=false composePath=/opt/docker/stacks/n8n/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "navidrome" deployed=false composePath=/opt/docker/stacks/navidrome/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "nextcloud" deployed=false composePath=/opt/docker/stacks/nextcloud/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "nzbget" deployed=false composePath=/opt/docker/stacks/nzbget/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "onlyoffice" deployed=false composePath=/opt/docker/stacks/onlyoffice/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "opengist" deployed=false composePath=/opt/docker/stacks/opengist/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "outline" deployed=false composePath=/opt/docker/stacks/outline/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "paperless-ngx" deployed=true composePath=/opt/docker/stacks/paperless-ngx/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "papra" deployed=false composePath=/opt/docker/stacks/papra/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "plant-it" deployed=false composePath=/opt/docker/stacks/plant-it/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "plex" deployed=false composePath=/opt/docker/stacks/plex/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "privatebin" deployed=true composePath=/opt/docker/stacks/privatebin/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "prowlarr" deployed=false composePath=/opt/docker/stacks/prowlarr/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "qbittorrent" deployed=false composePath=/opt/docker/stacks/qbittorrent/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "radarr" deployed=false composePath=/opt/docker/stacks/radarr/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "radicale" deployed=false composePath=/opt/docker/stacks/radicale/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "rallly" deployed=false composePath=/opt/docker/stacks/rallly/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "recipe-importer" deployed=false composePath=/opt/docker/stacks/recipe-importer/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "romm" deployed=false composePath=/opt/docker/stacks/romm/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "sabnzbd" deployed=false composePath=/opt/docker/stacks/sabnzbd/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "seerr" deployed=false composePath=/opt/docker/stacks/seerr/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "sonarr" deployed=false composePath=/opt/docker/stacks/sonarr/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "sparkyfitness" deployed=false composePath=/opt/docker/stacks/sparkyfitness/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "tandoor" deployed=false composePath=/opt/docker/stacks/tandoor/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "termix" deployed=false composePath=/opt/docker/stacks/termix/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "traefik" deployed=false composePath=/opt/docker/stacks/traefik/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "transmission" deployed=false composePath=/opt/docker/stacks/transmission/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "uptime-kuma" deployed=false composePath=/opt/docker/stacks/uptime-kuma/docker-compose.yml +2026/10/25 16:42:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "vaultwarden" deployed=false composePath=/opt/docker/stacks/vaultwarden/docker-compose.yml +2026/10/25 16:42:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "vikunja" deployed=false composePath=/opt/docker/stacks/vikunja/docker-compose.yml +2026/10/25 16:42:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "wanderer" deployed=false composePath=/opt/docker/stacks/wanderer/docker-compose.yml +2026/10/25 16:42:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "wger" deployed=false composePath=/opt/docker/stacks/wger/docker-compose.yml +2026/10/25 16:42:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "wishlist" deployed=false composePath=/opt/docker/stacks/wishlist/docker-compose.yml +2026/10/25 16:42:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "zipline" deployed=false composePath=/opt/docker/stacks/zipline/docker-compose.yml +2026/10/25 16:42:21 manager.go:1960: [DEBUG] [stacks] getCatalogTemplateSlugs: found 65 template slugs in /opt/docker/felhom-controller/data/catalog-cache/templates +2026/10/25 16:42:21 manager.go:757: [DEBUG] [stacks] ScanStacks: catalog has 65 template slugs for orphan detection +2026/10/25 16:42:21 manager.go:788: [INFO] [stacks] ScanStacks complete: 70 stacks found (4 deployed, 66 available) +2026/10/25 16:42:21 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:42:21 scheduler.go:467: [INFO] [scheduler] Job stack-scan completed (took 154ms) +2026/10/25 16:42:22 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:42:22 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:42:27 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:42:27 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:42:30 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:42:30 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:42:30 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 1m40s ago, effective interval 5m0s, healthy=true +2026/10/25 16:42:30 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 2m40s ago, effective interval 5m0s, healthy=true +2026/10/25 16:42:30 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (2 skipped not due, 0 skipped no container) +2026/10/25 16:42:30 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:42:30 manager.go:921: [DEBUG] [stacks] restart-policy of down member "tandoor" = "unless-stopped" +2026/10/25 16:42:32 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:42:32 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:42:32 manager.go:1719: [DEBUG] Command completed: docker compose up -d (took 90.9s) +2026/10/25 16:42:32 deploy.go:664: [INFO] [stacks] Stack tandoor deployed successfully (took 90.9s) +2026/10/25 16:42:32 [DEBUG] [stacks] SaveAppConfig: saving /opt/docker/stacks/tandoor — 4 env vars, 2 encrypted, 2 sensitive fields +2026/10/25 16:42:32 [INFO] [stacks] SaveAppConfig: saved config for tandoor +2026/10/25 16:42:33 [DEBUG] [stacks] SaveAppConfig: saving /opt/docker/stacks/tandoor — 4 env vars, 0 encrypted, 2 sensitive fields +2026/10/25 16:42:33 [INFO] [stacks] SaveAppConfig: saved config for tandoor +2026/10/25 16:42:33 installed.go:420: [INFO] [stacks] installed-images tandoor: recorded 2 service(s) (tandoor=ghcr.io/tandoorrecipes/recipes:2.6.15 (sha256:2e759dd1478a…), tandoor-postgres=postgres:17-alpine (sha256:b0f9560a2de0…)) +2026/10/25 16:42:33 [DEBUG] [stacks] SaveAppConfig: saving /opt/docker/stacks/tandoor — 4 env vars, 0 encrypted, 2 sensitive fields +2026/10/25 16:42:33 [INFO] [stacks] SaveAppConfig: saved config for tandoor +2026/10/25 16:42:33 pin.go:93: [INFO] [stacks] pin tandoor: tandoor=ghcr.io/tandoorrecipes/recipes:2.6.15, tandoor-postgres=postgres:17-alpine +2026/10/25 16:42:33 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:42:33 [DEBUG] [backup] tandoor: off-site default not applied: no off-site target +2026/10/25 16:42:33 notifier.go:1467: [DEBUG] notifier disabled: dropped event app_deployed (severity info) +2026/10/25 16:42:35 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting +2026/10/25 16:42:36 manager.go:1688: [DEBUG] Env vars for compose: [PATH, HOSTNAME, FELHOM_BOOTSTRAP_PATH, HOME, DOMAIN, SECRET_KEY, SUBDOMAIN, DB_PASSWORD, DOMAIN, IMPORT_PATH] (10 app + 0 system) +2026/10/25 16:42:36 manager.go:1697: [DEBUG] Running: docker compose ps -a --format table {{.Name}} {{.Image}} {{.State}} {{.Status}} (in /opt/docker/stacks/tandoor) +2026/10/25 16:42:36 manager.go:1719: [DEBUG] Command completed: docker compose ps -a --format table {{.Name}} {{.Image}} {{.State}} {{.Status}} (took 0.1s) +2026/10/25 16:42:36 manager.go:1813: [INFO] [stacks] Stack tandoor post-start status: +2026/10/25 16:42:36 manager.go:1816: [INFO] [stacks] tandoor ghcr.io/tandoorrecipes/recipes:2.6.15@sha256:2e759dd1478a2ed119ee474e28522079fb1cfa50b3fd25cba89f6b9a67abad72 running Up 3 seconds (health: starting) +2026/10/25 16:42:36 manager.go:1816: [INFO] [stacks] tandoor-postgres postgres:17-alpine@sha256:b0f9560a2de083e2cc7382e75f808c7381a32852a7ec49117deedb300e552b24 running Up 9 seconds (healthy) +2026/10/25 16:42:37 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:42:37 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:42:40 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:42:40 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 2m50s ago, effective interval 5m0s, healthy=true +2026/10/25 16:42:40 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 1m50s ago, effective interval 5m0s, healthy=true +2026/10/25 16:42:40 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (2 skipped not due, 0 skipped no container) +2026/10/25 16:42:40 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:42:40 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:42:42 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:42:42 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:42:47 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:42:47 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:42:50 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:42:50 scheduler.go:85: [DEBUG] [scheduler] job deadapp-check: execution starting +2026/10/25 16:42:50 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting +2026/10/25 16:42:50 scheduler.go:85: [DEBUG] [scheduler] job after-unclean-boot: execution starting +2026/10/25 16:42:50 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:42:50 scheduler.go:85: [DEBUG] [scheduler] job ring-spill: execution starting +2026/10/25 16:42:50 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:42:50 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 2m0s ago, effective interval 5m0s, healthy=true +2026/10/25 16:42:50 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 3m0s ago, effective interval 5m0s, healthy=true +2026/10/25 16:42:50 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (2 skipped not due, 0 skipped no container) +2026/10/25 16:42:52 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:42:52 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:42:57 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:42:57 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:43:00 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:43:00 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:43:00 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:43:00 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 3m10s ago, effective interval 5m0s, healthy=true +2026/10/25 16:43:00 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 2m10s ago, effective interval 5m0s, healthy=true +2026/10/25 16:43:00 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (2 skipped not due, 0 skipped no container) +2026/10/25 16:43:02 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:43:02 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:43:05 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting +2026/10/25 16:43:07 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:43:07 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:43:10 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:43:10 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:43:10 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:43:10 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 3m20s ago, effective interval 5m0s, healthy=true +2026/10/25 16:43:10 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 2m20s ago, effective interval 5m0s, healthy=true +2026/10/25 16:43:10 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (2 skipped not due, 0 skipped no container) +2026/10/25 16:43:12 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:43:12 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:43:18 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:43:18 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:43:20 info.go:13: [DEBUG] [system] GetInfo starting (hddPath="/mnt/felhom-drives/scratch_hdd", hasCPUCollector=true) +2026/10/25 16:43:20 info.go:13: [DEBUG] [system] readMemInfo: guest cap=25898MB (host total was 30714380KB) → used≈13331MB avail≈12567MB +2026/10/25 16:43:20 info.go:13: [DEBUG] [system] readMemInfo: totalKB=30714380 availKB=14904040 → total=25898MB avail=12567MB used=13331MB (51.5%) +2026/10/25 16:43:20 info.go:13: [DEBUG] [system] readDiskUsage: path="/" bsize=4096 total=68.4GB used=12.9GB avail=51.9GB (19.9%) +2026/10/25 16:43:20 info.go:13: [DEBUG] [system] readDiskUsage: path="/mnt/felhom-drives/scratch_hdd" bsize=4096 total=937.8GB used=219.7GB avail=670.4GB (24.7%) +2026/10/25 16:43:20 info.go:13: [DEBUG] [system] readLoadAvg: raw="2.70 2.74 2.17 4/1706 3553" → 1m=2.70 5m=2.74 15m=2.17 +2026/10/25 16:43:20 info.go:13: [DEBUG] [system] readThermalZones: /sys — found 1 zones +2026/10/25 16:43:20 info.go:13: [DEBUG] [system] readTemperature: found via hwmon at /sys — 58.5°C (hwmon2) +2026/10/25 16:43:20 info.go:13: [DEBUG] [system] GetInfo done in 78ms — mem=13331MB/25898MB (51.5%), rootDisk=12.9GB/68.4GB (19.9%), load=2.70/2.74/2.17, temp=58.5°C (hwmon2), cpu=36.8% +2026/10/25 16:43:20 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:43:20 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:43:20 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting +2026/10/25 16:43:20 scheduler.go:85: [DEBUG] [scheduler] job deadapp-check: execution starting +2026/10/25 16:43:20 scheduler.go:85: [DEBUG] [scheduler] job after-unclean-boot: execution starting +2026/10/25 16:43:20 scheduler.go:85: [DEBUG] [scheduler] job ring-spill: execution starting +2026/10/25 16:43:20 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:43:20 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 3m30s ago, effective interval 5m0s, healthy=true +2026/10/25 16:43:20 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 2m30s ago, effective interval 5m0s, healthy=true +2026/10/25 16:43:20 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (2 skipped not due, 0 skipped no container) +2026/10/25 16:43:23 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:43:23 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:43:28 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:43:28 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:43:30 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:43:30 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:43:30 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:43:30 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 3m40s ago, effective interval 5m0s, healthy=true +2026/10/25 16:43:30 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 2m40s ago, effective interval 5m0s, healthy=true +2026/10/25 16:43:30 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (2 skipped not due, 0 skipped no container) +2026/10/25 16:43:33 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:43:33 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:43:35 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting +2026/10/25 16:43:38 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:43:38 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:43:40 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:43:40 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:43:40 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 3m50s ago, effective interval 5m0s, healthy=true +2026/10/25 16:43:40 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 2m50s ago, effective interval 5m0s, healthy=true +2026/10/25 16:43:40 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (2 skipped not due, 0 skipped no container) +2026/10/25 16:43:40 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:43:43 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:43:43 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:43:50 scheduler.go:85: [DEBUG] [scheduler] job after-unclean-boot: execution starting +2026/10/25 16:43:50 scheduler.go:85: [DEBUG] [scheduler] job deadapp-check: execution starting +2026/10/25 16:43:50 scheduler.go:85: [DEBUG] [scheduler] job ring-spill: execution starting +2026/10/25 16:43:50 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting +2026/10/25 16:43:50 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:43:50 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:43:50 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:43:50 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 4m0s ago, effective interval 5m0s, healthy=true +2026/10/25 16:43:50 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 3m0s ago, effective interval 5m0s, healthy=true +2026/10/25 16:43:50 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 1 targets (2 skipped not due, 0 skipped no container) +2026/10/25 16:43:51 healthprobe.go:187: [DEBUG] Health probe tandoor: HTTP GET :80/accounts/login/ → 400 (190ms) +2026/10/25 16:43:51 healthprobe.go:167: [INFO] Health probes: 1 ok (of 1 probed) +2026/10/25 16:44:00 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:44:00 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:44:00 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:44:00 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 4m10s ago, effective interval 5m0s, healthy=true +2026/10/25 16:44:00 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping tandoor — last check 10s ago, effective interval 5m0s, healthy=true +2026/10/25 16:44:00 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 3m10s ago, effective interval 5m0s, healthy=true +2026/10/25 16:44:00 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (3 skipped not due, 0 skipped no container) +2026/10/25 16:44:05 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting +2026/10/25 16:44:10 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:44:10 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:44:10 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:44:10 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 4m20s ago, effective interval 5m0s, healthy=true +2026/10/25 16:44:10 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping tandoor — last check 20s ago, effective interval 5m0s, healthy=true +2026/10/25 16:44:10 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 3m20s ago, effective interval 5m0s, healthy=true +2026/10/25 16:44:10 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (3 skipped not due, 0 skipped no container) +2026/10/25 16:44:20 info.go:13: [DEBUG] [system] GetInfo starting (hddPath="/mnt/felhom-drives/scratch_hdd", hasCPUCollector=true) +2026/10/25 16:44:20 info.go:13: [DEBUG] [system] readMemInfo: guest cap=25898MB (host total was 30714380KB) → used≈13957MB avail≈11941MB +2026/10/25 16:44:20 info.go:13: [DEBUG] [system] readMemInfo: totalKB=30714380 availKB=14161592 → total=25898MB avail=11941MB used=13957MB (53.9%) +2026/10/25 16:44:20 info.go:13: [DEBUG] [system] readDiskUsage: path="/" bsize=4096 total=68.4GB used=13.0GB avail=51.8GB (20.0%) +2026/10/25 16:44:20 info.go:13: [DEBUG] [system] readDiskUsage: path="/mnt/felhom-drives/scratch_hdd" bsize=4096 total=937.8GB used=219.7GB avail=670.4GB (24.7%) +2026/10/25 16:44:20 info.go:13: [DEBUG] [system] readLoadAvg: raw="2.50 2.79 2.23 9/1727 3714" → 1m=2.50 5m=2.79 15m=2.23 +2026/10/25 16:44:20 info.go:13: [DEBUG] [system] readThermalZones: /sys — found 1 zones +2026/10/25 16:44:20 info.go:13: [DEBUG] [system] readTemperature: found via hwmon at /sys — 48.9°C (hwmon2) +2026/10/25 16:44:20 info.go:13: [DEBUG] [system] GetInfo done in 71ms — mem=13957MB/25898MB (53.9%), rootDisk=13.0GB/68.4GB (20.0%), load=2.50/2.79/2.23, temp=48.9°C (hwmon2), cpu=12.5% +2026/10/25 16:44:20 scheduler.go:85: [DEBUG] [scheduler] job deadapp-check: execution starting +2026/10/25 16:44:20 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting +2026/10/25 16:44:20 scheduler.go:450: [INFO] [scheduler] Running job: docker-socket-users +2026/10/25 16:44:20 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-users: execution starting +2026/10/25 16:44:20 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:44:20 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 4m30s ago, effective interval 5m0s, healthy=true +2026/10/25 16:44:20 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping tandoor — last check 30s ago, effective interval 5m0s, healthy=true +2026/10/25 16:44:20 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 3m30s ago, effective interval 5m0s, healthy=true +2026/10/25 16:44:20 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (3 skipped not due, 0 skipped no container) +2026/10/25 16:44:20 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:44:20 scheduler.go:450: [INFO] [scheduler] Running job: stack-scan +2026/10/25 16:44:20 scheduler.go:85: [DEBUG] [scheduler] job stack-scan: execution starting +2026/10/25 16:44:20 scheduler.go:85: [DEBUG] [scheduler] job after-unclean-boot: execution starting +2026/10/25 16:44:20 scheduler.go:85: [DEBUG] [scheduler] job ring-spill: execution starting +2026/10/25 16:44:20 scheduler.go:450: [INFO] [scheduler] Running job: backup-cache +2026/10/25 16:44:20 scheduler.go:85: [DEBUG] [scheduler] job backup-cache: execution starting +2026/10/25 16:44:20 scheduler.go:450: [INFO] [scheduler] Running job: fill-watch-interval +2026/10/25 16:44:20 scheduler.go:85: [DEBUG] [scheduler] job fill-watch-interval: execution starting +2026/10/25 16:44:20 scheduler.go:450: [INFO] [scheduler] Running job: offsite-credential-retry +2026/10/25 16:44:20 scheduler.go:85: [DEBUG] [scheduler] job offsite-credential-retry: execution starting +2026/10/25 16:44:20 scheduler.go:467: [INFO] [scheduler] Job offsite-credential-retry completed (took 0s) +2026/10/25 16:44:20 scheduler.go:450: [INFO] [scheduler] Running job: system-health +2026/10/25 16:44:20 scheduler.go:85: [DEBUG] [scheduler] job system-health: execution starting +2026/10/25 16:44:20 info.go:13: [DEBUG] [system] GetInfo starting (hddPath="/mnt/felhom-drives/scratch_hdd", hasCPUCollector=true) +2026/10/25 16:44:20 info.go:13: [DEBUG] [system] GetDiskUsage: path="/" total=68.4 GB used=13.0 GB avail=51.8GB (20.0%) +2026/10/25 16:44:20 info.go:13: [DEBUG] [system] GetDiskUsage: path="/mnt/felhom-drives/scratch_hdd" total=937.8 GB used=219.7 GB avail=670.4GB (24.7%) +2026/10/25 16:44:20 info.go:13: [DEBUG] [system] GetDiskUsage: path="/mnt/felhom-drives/scratch_hdd/userdata/audiobookshelf" total=937.8 GB used=219.7 GB avail=670.4GB (24.7%) +2026/10/25 16:44:20 info.go:13: [DEBUG] [system] GetDiskUsage: path="/mnt/felhom-drives/scratch_hdd/userdata/calibre-web" total=937.8 GB used=219.7 GB avail=670.4GB (24.7%) +2026/10/25 16:44:20 info.go:13: [DEBUG] [system] GetDiskUsage: statfs("/mnt/felhom-drives/scratch_hdd/userdata/grimmory") failed: no such file or directory +2026/10/25 16:44:20 fillwatch.go:233: [DEBUG] [fillwatch] /mnt/felhom-drives/scratch_hdd/userdata/grimmory: usage unreadable — skipped (an unreadable filesystem is not a full one) +2026/10/25 16:44:20 info.go:13: [DEBUG] [system] GetDiskUsage: path="/mnt/felhom-drives/scratch_hdd/userdata/immich" total=937.8 GB used=219.7 GB avail=670.4GB (24.7%) +2026/10/25 16:44:20 info.go:13: [DEBUG] [system] GetDiskUsage: statfs("/mnt/felhom-drives/scratch_hdd/userdata/metube") failed: no such file or directory +2026/10/25 16:44:20 fillwatch.go:233: [DEBUG] [fillwatch] /mnt/felhom-drives/scratch_hdd/userdata/metube: usage unreadable — skipped (an unreadable filesystem is not a full one) +2026/10/25 16:44:20 info.go:13: [DEBUG] [system] GetDiskUsage: path="/mnt/felhom-drives/scratch_hdd/userdata/navidrome" total=937.8 GB used=219.7 GB avail=670.4GB (24.7%) +2026/10/25 16:44:20 info.go:13: [DEBUG] [system] GetDiskUsage: path="/mnt/felhom-drives/scratch_hdd/userdata/nextcloud" total=937.8 GB used=219.7 GB avail=670.4GB (24.7%) +2026/10/25 16:44:20 info.go:13: [DEBUG] [system] GetDiskUsage: path="/mnt/felhom-drives/scratch_hdd/userdata/paperless-ngx" total=937.8 GB used=219.7 GB avail=670.4GB (24.7%) +2026/10/25 16:44:20 info.go:13: [DEBUG] [system] GetDiskUsage: path="/mnt/felhom-drives/scratch_hdd/userdata/romm" total=937.8 GB used=219.7 GB avail=670.4GB (24.7%) +2026/10/25 16:44:20 info.go:13: [DEBUG] [system] GetDiskUsage: path="/mnt/sizing-bench" total=31.2 GB used=2.7 GB avail=26.9GB (9.1%) +2026/10/25 16:44:20 info.go:13: [DEBUG] [system] GetDiskUsage: path="/mnt/sys_drive" total=68.4 GB used=13.0 GB avail=51.8GB (20.0%) +2026/10/25 16:44:20 fillwatch.go:284: [INFO] [fillwatch] checked 11 filesystem(s), 2 unreadable/skipped, 0 notification(s); bands: all ok +2026/10/25 16:44:20 scheduler.go:467: [INFO] [scheduler] Job fill-watch-interval completed (took 2ms) +2026/10/25 16:44:20 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "actualbudget" deployed=false composePath=/opt/docker/stacks/actualbudget/docker-compose.yml +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "adventurelog" deployed=false composePath=/opt/docker/stacks/adventurelog/docker-compose.yml +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "audiobookshelf" deployed=false composePath=/opt/docker/stacks/audiobookshelf/docker-compose.yml +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "bentopdf" deployed=false composePath=/opt/docker/stacks/bentopdf/docker-compose.yml +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "bookstack" deployed=false composePath=/opt/docker/stacks/bookstack/docker-compose.yml +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "calcom" deployed=false composePath=/opt/docker/stacks/calcom/docker-compose.yml +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "calibre-web" deployed=false composePath=/opt/docker/stacks/calibre-web/docker-compose.yml +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "chaoscrash" deployed=false composePath=/opt/docker/stacks/chaoscrash/docker-compose.yml +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "chaosoom" deployed=false composePath=/opt/docker/stacks/chaosoom/docker-compose.yml +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "chaosoomb" deployed=false composePath=/opt/docker/stacks/chaosoomb/docker-compose.yml +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "claper" deployed=false composePath=/opt/docker/stacks/claper/docker-compose.yml +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "code-server" deployed=false composePath=/opt/docker/stacks/code-server/docker-compose.yml +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "crafty-controller" deployed=false composePath=/opt/docker/stacks/crafty-controller/docker-compose.yml +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "dawarich" deployed=false composePath=/opt/docker/stacks/dawarich/docker-compose.yml +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "docmost" deployed=false composePath=/opt/docker/stacks/docmost/docker-compose.yml +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "emby" deployed=false composePath=/opt/docker/stacks/emby/docker-compose.yml +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "filebrowser" deployed=false composePath=/opt/docker/stacks/filebrowser/docker-compose.yml +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "ghost" deployed=false composePath=/opt/docker/stacks/ghost/docker-compose.yml +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "gitea" deployed=false composePath=/opt/docker/stacks/gitea/docker-compose.yml +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "glance" deployed=false composePath=/opt/docker/stacks/glance/docker-compose.yml +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "gokapi" deployed=false composePath=/opt/docker/stacks/gokapi/docker-compose.yml +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "grafana" deployed=false composePath=/opt/docker/stacks/grafana/docker-compose.yml +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "gramps-web" deployed=false composePath=/opt/docker/stacks/gramps-web/docker-compose.yml +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "grimmory" deployed=false composePath=/opt/docker/stacks/grimmory/docker-compose.yml +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "grocy" deployed=false composePath=/opt/docker/stacks/grocy/docker-compose.yml +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "home-assistant" deployed=false composePath=/opt/docker/stacks/home-assistant/docker-compose.yml +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "homebox" deployed=false composePath=/opt/docker/stacks/homebox/docker-compose.yml +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "homepage" deployed=false composePath=/opt/docker/stacks/homepage/docker-compose.yml +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "immich" deployed=true composePath=/opt/docker/stacks/immich/docker-compose.yml +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "jellyfin" deployed=false composePath=/opt/docker/stacks/jellyfin/docker-compose.yml +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "karakeep" deployed=false composePath=/opt/docker/stacks/karakeep/docker-compose.yml +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "kimai" deployed=false composePath=/opt/docker/stacks/kimai/docker-compose.yml +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "komga" deployed=false composePath=/opt/docker/stacks/komga/docker-compose.yml +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "lubelogger" deployed=false composePath=/opt/docker/stacks/lubelogger/docker-compose.yml +2026/10/25 16:44:20 info.go:13: [DEBUG] [system] readMemInfo: guest cap=25898MB (host total was 30714380KB) → used≈13976MB avail≈11922MB +2026/10/25 16:44:20 info.go:13: [DEBUG] [system] readMemInfo: totalKB=30714380 availKB=14138580 → total=25898MB avail=11922MB used=13976MB (54.0%) +2026/10/25 16:44:20 info.go:13: [DEBUG] [system] readDiskUsage: path="/" bsize=4096 total=68.4GB used=13.0GB avail=51.8GB (20.0%) +2026/10/25 16:44:20 info.go:13: [DEBUG] [system] readDiskUsage: path="/mnt/felhom-drives/scratch_hdd" bsize=4096 total=937.8GB used=219.7GB avail=670.4GB (24.7%) +2026/10/25 16:44:20 info.go:13: [DEBUG] [system] readLoadAvg: raw="2.50 2.79 2.23 5/1740 3788" → 1m=2.50 5m=2.79 15m=2.23 +2026/10/25 16:44:20 info.go:13: [DEBUG] [system] readThermalZones: /sys — found 1 zones +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "mealie" deployed=false composePath=/opt/docker/stacks/mealie/docker-compose.yml +2026/10/25 16:44:20 info.go:13: [DEBUG] [system] readTemperature: found via hwmon at /sys — 58.6°C (hwmon2) +2026/10/25 16:44:20 info.go:13: [DEBUG] [system] GetInfo done in 107ms — mem=13976MB/25898MB (54.0%), rootDisk=13.0GB/68.4GB (20.0%), load=2.50/2.79/2.23, temp=58.6°C (hwmon2), cpu=12.1% +2026/10/25 16:44:20 healthcheck.go:128: [DEBUG] [monitor] Raw values: disk=20.0%, hdd=24.7% (configured=true), mem=54.0% (13976MB/25898MB), cpu=12.1%, temp=58.6°C (hwmon2) +2026/10/25 16:44:20 healthcheck.go:409: [DEBUG] [monitor] SSD disk: OK (20%) +2026/10/25 16:44:20 healthcheck.go:445: [DEBUG] [monitor] Memory: OK (54%) +2026/10/25 16:44:20 healthcheck.go:464: [DEBUG] [monitor] CPU: OK (12%) +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "metube" deployed=false composePath=/opt/docker/stacks/metube/docker-compose.yml +2026/10/25 16:44:20 healthcheck.go:483: [DEBUG] [monitor] Temperature: OK (59°C) +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "n8n" deployed=false composePath=/opt/docker/stacks/n8n/docker-compose.yml +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "navidrome" deployed=false composePath=/opt/docker/stacks/navidrome/docker-compose.yml +2026/10/25 16:44:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "nextcloud" deployed=false composePath=/opt/docker/stacks/nextcloud/docker-compose.yml +2026/10/25 16:44:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "nzbget" deployed=false composePath=/opt/docker/stacks/nzbget/docker-compose.yml +2026/10/25 16:44:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "onlyoffice" deployed=false composePath=/opt/docker/stacks/onlyoffice/docker-compose.yml +2026/10/25 16:44:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "opengist" deployed=false composePath=/opt/docker/stacks/opengist/docker-compose.yml +2026/10/25 16:44:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "outline" deployed=false composePath=/opt/docker/stacks/outline/docker-compose.yml +2026/10/25 16:44:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "paperless-ngx" deployed=true composePath=/opt/docker/stacks/paperless-ngx/docker-compose.yml +2026/10/25 16:44:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "papra" deployed=false composePath=/opt/docker/stacks/papra/docker-compose.yml +2026/10/25 16:44:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "plant-it" deployed=false composePath=/opt/docker/stacks/plant-it/docker-compose.yml +2026/10/25 16:44:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "plex" deployed=false composePath=/opt/docker/stacks/plex/docker-compose.yml +2026/10/25 16:44:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "privatebin" deployed=true composePath=/opt/docker/stacks/privatebin/docker-compose.yml +2026/10/25 16:44:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "prowlarr" deployed=false composePath=/opt/docker/stacks/prowlarr/docker-compose.yml +2026/10/25 16:44:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "qbittorrent" deployed=false composePath=/opt/docker/stacks/qbittorrent/docker-compose.yml +2026/10/25 16:44:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "radarr" deployed=false composePath=/opt/docker/stacks/radarr/docker-compose.yml +2026/10/25 16:44:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "radicale" deployed=false composePath=/opt/docker/stacks/radicale/docker-compose.yml +2026/10/25 16:44:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "rallly" deployed=false composePath=/opt/docker/stacks/rallly/docker-compose.yml +2026/10/25 16:44:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "recipe-importer" deployed=false composePath=/opt/docker/stacks/recipe-importer/docker-compose.yml +2026/10/25 16:44:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "romm" deployed=false composePath=/opt/docker/stacks/romm/docker-compose.yml +2026/10/25 16:44:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "sabnzbd" deployed=false composePath=/opt/docker/stacks/sabnzbd/docker-compose.yml +2026/10/25 16:44:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "seerr" deployed=false composePath=/opt/docker/stacks/seerr/docker-compose.yml +2026/10/25 16:44:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "sonarr" deployed=false composePath=/opt/docker/stacks/sonarr/docker-compose.yml +2026/10/25 16:44:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "sparkyfitness" deployed=false composePath=/opt/docker/stacks/sparkyfitness/docker-compose.yml +2026/10/25 16:44:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "tandoor" deployed=true composePath=/opt/docker/stacks/tandoor/docker-compose.yml +2026/10/25 16:44:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "termix" deployed=false composePath=/opt/docker/stacks/termix/docker-compose.yml +2026/10/25 16:44:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "traefik" deployed=false composePath=/opt/docker/stacks/traefik/docker-compose.yml +2026/10/25 16:44:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "transmission" deployed=false composePath=/opt/docker/stacks/transmission/docker-compose.yml +2026/10/25 16:44:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "uptime-kuma" deployed=false composePath=/opt/docker/stacks/uptime-kuma/docker-compose.yml +2026/10/25 16:44:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "vaultwarden" deployed=false composePath=/opt/docker/stacks/vaultwarden/docker-compose.yml +2026/10/25 16:44:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "vikunja" deployed=false composePath=/opt/docker/stacks/vikunja/docker-compose.yml +2026/10/25 16:44:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "wanderer" deployed=false composePath=/opt/docker/stacks/wanderer/docker-compose.yml +2026/10/25 16:44:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "wger" deployed=false composePath=/opt/docker/stacks/wger/docker-compose.yml +2026/10/25 16:44:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "wishlist" deployed=false composePath=/opt/docker/stacks/wishlist/docker-compose.yml +2026/10/25 16:44:21 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "zipline" deployed=false composePath=/opt/docker/stacks/zipline/docker-compose.yml +2026/10/25 16:44:21 manager.go:1960: [DEBUG] [stacks] getCatalogTemplateSlugs: found 65 template slugs in /opt/docker/felhom-controller/data/catalog-cache/templates +2026/10/25 16:44:21 manager.go:757: [DEBUG] [stacks] ScanStacks: catalog has 65 template slugs for orphan detection +2026/10/25 16:44:21 manager.go:788: [INFO] [stacks] ScanStacks complete: 70 stacks found (4 deployed, 66 available) +2026/10/25 16:44:21 healthcheck.go:145: [DEBUG] [monitor] Docker daemon: OK +2026/10/25 16:44:21 healthcheck.go:153: [DEBUG] [monitor] Checking 1 protected containers: [traefik] +2026/10/25 16:44:21 scheduler.go:467: [INFO] [scheduler] Job docker-socket-users completed (took 189ms) +2026/10/25 16:44:21 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:44:21 scheduler.go:467: [INFO] [scheduler] Job stack-scan completed (took 194ms) +2026/10/25 16:44:21 backup.go:540: [DEBUG] groupStacksByDrive: /mnt/sys_drive → [privatebin, tandoor] +2026/10/25 16:44:21 backup.go:540: [DEBUG] groupStacksByDrive: /mnt/sizing-bench → [immich] +2026/10/25 16:44:21 backup.go:540: [DEBUG] groupStacksByDrive: /mnt/felhom-drives/scratch_hdd/userdata/paperless-ngx → [paperless-ngx] +2026/10/25 16:44:21 backup.go:1246: [INFO] [backup] Found 4 DB dump files across drives +2026/10/25 16:44:21 dbdump.go:130: [DEBUG] DiscoverDatabases: running docker ps to find database containers +2026/10/25 16:44:21 healthcheck.go:161: [DEBUG] [monitor] All protected containers running +2026/10/25 16:44:21 info.go:13: [DEBUG] [system] GetDiskUsage: path="/mnt/felhom-drives/scratch_hdd" total=937.8 GB used=219.7 GB avail=670.4GB (24.7%) +2026/10/25 16:44:21 info.go:13: [DEBUG] [system] GetDiskUsage: path="/mnt/felhom-drives/scratch_hdd/userdata/nextcloud" total=937.8 GB used=219.7 GB avail=670.4GB (24.7%) +2026/10/25 16:44:21 info.go:13: [DEBUG] [system] GetDiskUsage: path="/mnt/felhom-drives/scratch_hdd/userdata/paperless-ngx" total=937.8 GB used=219.7 GB avail=670.4GB (24.7%) +2026/10/25 16:44:21 info.go:13: [DEBUG] [system] GetDiskUsage: path="/mnt/felhom-drives/scratch_hdd/userdata/romm" total=937.8 GB used=219.7 GB avail=670.4GB (24.7%) +2026/10/25 16:44:21 info.go:13: [DEBUG] [system] GetDiskUsage: path="/mnt/felhom-drives/scratch_hdd/userdata/navidrome" total=937.8 GB used=219.7 GB avail=670.4GB (24.7%) +2026/10/25 16:44:21 info.go:13: [DEBUG] [system] GetDiskUsage: path="/mnt/felhom-drives/scratch_hdd/userdata/audiobookshelf" total=937.8 GB used=219.7 GB avail=670.4GB (24.7%) +2026/10/25 16:44:21 info.go:13: [DEBUG] [system] GetDiskUsage: path="/mnt/felhom-drives/scratch_hdd/userdata/immich" total=937.8 GB used=219.7 GB avail=670.4GB (24.7%) +2026/10/25 16:44:21 info.go:13: [DEBUG] [system] GetDiskUsage: path="/mnt/felhom-drives/scratch_hdd/userdata/calibre-web" total=937.8 GB used=219.7 GB avail=670.4GB (24.7%) +2026/10/25 16:44:21 info.go:13: [DEBUG] [system] GetDiskUsage: path="/mnt/sizing-bench" total=31.2 GB used=2.7 GB avail=26.9GB (9.1%) +2026/10/25 16:44:21 healthcheck.go:176: [INFO] [monitor] Health check: status=warn +2026/10/25 16:44:21 healthcheck.go:180: [DEBUG] [monitor] Final status: warn (issues=0, warnings=10, info=5) +2026/10/25 16:44:21 notifier.go:1467: [DEBUG] notifier disabled: dropped event health_change (severity warning) +2026/10/25 16:44:21 scheduler.go:467: [INFO] [scheduler] Job system-health completed (took 205ms) +2026/10/25 16:44:21 dbdump.go:149: [DEBUG] DiscoverDatabases: docker ps output: 24af7267c1a4 tandoor tandoor ghcr.io/tandoorrecipes/recipes:2.6.15 +979b7ec19f49 tandoor-postgres tandoor postgres:17-alpine +74f0a38fc983 paperless-webserver paperless-ngx ghcr.io/paperless-ngx/paperless-ngx:2.20.15 +cb44703bbadd paperless-postgres paperless-ngx postgres:18-alpine +cd225421c992 paperless-redis paperless-ngx redis:7-alpine +f062a2761fab immich-server immich ghcr.io/immich-app/immich-server:v3.2.4 +6041eb882834 immich-postgres immich ghcr.io/immich-app/postgres:16-vectorchord0.4.3-pgve... +2026/10/25 16:44:21 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container tandoor (image=ghcr.io/tandoorrecipes/recipes:2.6.15, not a database) +2026/10/25 16:44:21 dbdump.go:178: [DEBUG] DiscoverDatabases: found postgres container: tandoor-postgres (id=979b7ec19f49) +2026/10/25 16:44:21 dbdump.go:198: [DEBUG] DiscoverDatabases: tandoor-postgres → stack=tandoor, dbUser=tandoor, dbName=tandoor +2026/10/25 16:44:21 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container paperless-webserver (image=ghcr.io/paperless-ngx/paperless-ngx:2.20.15, not a database) +2026/10/25 16:44:21 dbdump.go:178: [DEBUG] DiscoverDatabases: found postgres container: paperless-postgres (id=cb44703bbadd) +2026/10/25 16:44:21 dbdump.go:915: [DEBUG] DiscoverDatabases: paperless-postgres → stack "paperless-ngx" from the compose project label (the container name would have given "paperless") +2026/10/25 16:44:21 dbdump.go:198: [DEBUG] DiscoverDatabases: paperless-postgres → stack=paperless-ngx, dbUser=paperless, dbName=paperless +2026/10/25 16:44:21 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container paperless-redis (image=redis:7-alpine, not a database) +2026/10/25 16:44:21 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container immich-server (image=ghcr.io/immich-app/immich-server:v3.2.4, not a database) +2026/10/25 16:44:21 dbdump.go:178: [DEBUG] DiscoverDatabases: found postgres container: immich-postgres (id=6041eb882834) +2026/10/25 16:44:21 dbdump.go:198: [DEBUG] DiscoverDatabases: immich-postgres → stack=immich, dbUser=immich, dbName=immich +2026/10/25 16:44:21 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container immich-machine-learning (image=ghcr.io/immich-app/immich-machine-learning:v3.2.4, not a database) +2026/10/25 16:44:21 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container immich-redis (image=redis:7-alpine, not a database) +2026/10/25 16:44:21 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container felhom-controller (image=gitea.dooplex.hu/admin/felhom-controller:0.343.0-rc.7, not a database) +2026/10/25 16:44:21 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container filebrowser (image=gtstef/filebrowser:1.5.6-stable, not a database) +2026/10/25 16:44:21 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container traefik (image=traefik:v3.7.13, not a database) +2026/10/25 16:44:21 dbdump.go:205: [DEBUG] DiscoverDatabases: found 3 database(s), skipped 9 non-DB container(s) +2026/10/25 16:44:21 dbdump.go:208: [INFO] [backup] Discovered 3 databases +2026/10/25 16:44:21 [INFO] [stacks] ParseComposeHDDMounts: found 1 HDD mounts for /opt/docker/stacks/immich/docker-compose.yml +2026/10/25 16:44:21 infra.go:97: [INFO] [infra] cloudflared skipped — no cf_tunnel_token configured (LAN-only node) +2026/10/25 16:44:21 [INFO] [stacks] ParseComposeHDDMounts: found 2 HDD mounts for /opt/docker/stacks/paperless-ngx/docker-compose.yml +2026/10/25 16:44:21 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/privatebin/docker-compose.yml +2026/10/25 16:44:21 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/privatebin/docker-compose.yml +2026/10/25 16:44:21 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/privatebin/docker-compose.yml +2026/10/25 16:44:21 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/privatebin/docker-compose.yml +2026/10/25 16:44:21 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/privatebin/docker-compose.yml +2026/10/25 16:44:21 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/privatebin/docker-compose.yml +2026/10/25 16:44:21 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/privatebin/docker-compose.yml +2026/10/25 16:44:21 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/privatebin/docker-compose.yml +2026/10/25 16:44:21 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/privatebin/docker-compose.yml +2026/10/25 16:44:21 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/privatebin/docker-compose.yml +2026/10/25 16:44:21 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/privatebin/docker-compose.yml +2026/10/25 16:44:21 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/tandoor/docker-compose.yml +2026/10/25 16:44:21 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/tandoor/docker-compose.yml +2026/10/25 16:44:21 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/tandoor/docker-compose.yml +2026/10/25 16:44:21 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/tandoor/docker-compose.yml +2026/10/25 16:44:21 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/tandoor/docker-compose.yml +2026/10/25 16:44:21 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/tandoor/docker-compose.yml +2026/10/25 16:44:21 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/tandoor/docker-compose.yml +2026/10/25 16:44:21 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/tandoor/docker-compose.yml +2026/10/25 16:44:21 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/tandoor/docker-compose.yml +2026/10/25 16:44:21 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/tandoor/docker-compose.yml +2026/10/25 16:44:21 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/tandoor/docker-compose.yml +2026/10/25 16:44:21 [INFO] [backup] Discovered app data: 4 apps +2026/10/25 16:44:21 info.go:13: [DEBUG] [system] GetDiskUsage: path="/mnt/sizing-bench" total=31.2 GB used=2.7 GB avail=26.9GB (9.1%) +2026/10/25 16:44:21 info.go:13: [DEBUG] [system] GetDiskUsage: path="/mnt/felhom-drives/scratch_hdd/userdata/paperless-ngx" total=937.8 GB used=219.7 GB avail=670.4GB (24.7%) +2026/10/25 16:44:21 info.go:13: [DEBUG] [system] GetDiskUsage: path="/mnt/sys_drive" total=68.4 GB used=13.0 GB avail=51.8GB (20.0%) +2026/10/25 16:44:21 info.go:13: [DEBUG] [system] GetDiskUsage: path="/mnt/sys_drive" total=68.4 GB used=13.0 GB avail=51.8GB (20.0%) +2026/10/25 16:44:21 kept_copies.go:64: [INFO] [settings] kept copy recorded: kind=unit source="tandoor" reason=earlier_install path=/mnt/sys_drive/felhom-data/backups/set-aside/tandoor/20261025T164421Z +2026/10/25 16:44:21 settings.go:970: [DEBUG] [settings] saved to /opt/docker/felhom-controller/data/settings.json (27730 bytes) +2026/10/25 16:44:21 settings.go:973: [INFO] [settings] Settings saved +2026/10/25 16:44:21 tier2_kept.go:42: [INFO] [backup] copy KEPT (earlier_install): unit "tandoor" at /mnt/sys_drive/felhom-data/backups/set-aside/tandoor/20261025T164421Z +2026/10/25 16:44:21 unit_owner.go:182: [INFO] [backup] tandoor: the recovery unit of an earlier install (c3b2e33d5bc82152; the install ids differ) is SET ASIDE whole → /mnt/sys_drive/felhom-data/backups/set-aside/tandoor/20261025T164421Z; this install (8c1656d41509efbc) gets its own unit from its first backup (D47) +2026/10/25 16:44:21 notifier.go:1463: [WARN] notifier disabled (no hub configured): DROPPED event backup_run_failures (severity error) — further backup_run_failures events are logged at DEBUG only +2026/10/25 16:44:21 recovery_unit.go:297: [INFO] [backup] Recovery unit captured for tandoor → /mnt/sys_drive/felhom-data/backups/primary/tandoor (images=2, secrets-referenced=2, data_keys=0, portable-carried=2/2, withheld=0) +2026/10/25 16:44:21 backup.go:1317: [INFO] [backup] Backup status cache refreshed +2026/10/25 16:44:21 scheduler.go:467: [INFO] [scheduler] Job backup-cache completed (took 515ms) +2026/10/25 16:44:30 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:44:30 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:44:30 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 4m40s ago, effective interval 5m0s, healthy=true +2026/10/25 16:44:30 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping tandoor — last check 40s ago, effective interval 5m0s, healthy=true +2026/10/25 16:44:30 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 3m40s ago, effective interval 5m0s, healthy=true +2026/10/25 16:44:30 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (3 skipped not due, 0 skipped no container) +2026/10/25 16:44:30 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:44:33 auth.go:144: [DEBUG] [web] auth: valid session for GET /backups/restore +2026/10/25 16:44:33 server.go:644: [DEBUG] [web] ServeHTTP: GET /backups/restore from 172.18.0.4:53858 +2026/10/25 16:44:33 auth.go:144: [DEBUG] [web] auth: valid session for POST /backup/kept/restore +2026/10/25 16:44:33 server.go:644: [DEBUG] [web] ServeHTTP: POST /backup/kept/restore from 172.18.0.4:53858 +2026/10/25 16:44:33 kept_copies.go:108: [WARN] [web] Restore of tandoor from a set-aside unit requested (async): /mnt/sys_drive/felhom-data/backups/set-aside/tandoor/20261025T164421Z from 172.18.0.4:53858 +2026/10/25 16:44:33 unit_owner.go:324: [INFO] [backup] restore tandoor from the set-aside unit /mnt/sys_drive/felhom-data/backups/set-aside/tandoor/20261025T164421Z (an earlier install, D47) +2026/10/25 16:44:33 restore_dbcreds.go:114: [INFO] [backup] Restore tandoor: database password(s) [DB_PASSWORD] come from the backup, [] stay as installed (older than the data) — values not logged (R-1106) +2026/10/25 16:44:33 restore_unit.go:410: [INFO] [backup] Restoring tandoor from recovery unit /mnt/sys_drive/felhom-data/backups/set-aside/tandoor/20261025T164421Z: images=2, secrets recovered=2/2, data_keys=0 +2026/10/25 16:44:34 manager.go:1439: [DEBUG] [stacks] StopStack tandoor: current state=running deployed=true containers=2 +2026/10/25 16:44:34 manager.go:1442: [INFO] [stacks] Stopping stack: tandoor +2026/10/25 16:44:34 manager.go:1688: [DEBUG] Env vars for compose: [PATH, HOSTNAME, FELHOM_BOOTSTRAP_PATH, HOME, DOMAIN, DOMAIN, SECRET_KEY, SUBDOMAIN, DB_PASSWORD, IMPORT_PATH] (10 app + 0 system) +2026/10/25 16:44:34 manager.go:1697: [DEBUG] Running: docker compose down (in /opt/docker/stacks/tandoor) +2026/10/25 16:44:35 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting +2026/10/25 16:44:37 manager.go:1719: [DEBUG] Command completed: docker compose down (took 2.7s) +2026/10/25 16:44:37 manager.go:1451: [INFO] [stacks] Stack tandoor stopped successfully (took 2.7s) +2026/10/25 16:44:37 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:44:38 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/backup/restore-status +2026/10/25 16:44:38 router.go:78: [DEBUG] [api] GET /api/backup/restore-status (path=/backup/restore-status) +2026/10/25 16:44:40 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:44:40 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:44:40 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:44:40 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 3m50s ago, effective interval 5m0s, healthy=true +2026/10/25 16:44:40 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 4m50s ago, effective interval 5m0s, healthy=true +2026/10/25 16:44:40 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (2 skipped not due, 0 skipped no container) +2026/10/25 16:44:41 offbox_restore_undo.go:169: [INFO] [offbox] tandoor: the live volumes are copied before the restore (3 copied, 0 new from the snapshot) in 4.295s — /mnt/sys_drive/felhom-data/backups/restore-undo/tandoor/20261025T164437Z (R-893) +2026/10/25 16:44:41 restore_crash_putback.go:89: [INFO] [backup] tandoor: the restore copy is recorded — a power cut from here on puts the app back on start (R-1195) +2026/10/25 16:44:41 restore.go:225: [INFO] [backup] Restoring Docker volume tandoor_tandoor_media for tandoor +2026/10/25 16:44:42 restore.go:263: [DEBUG] [backup] Volume tandoor_tandoor_media restored successfully +2026/10/25 16:44:42 restore.go:225: [INFO] [backup] Restoring Docker volume tandoor_tandoor_postgres_data for tandoor +2026/10/25 16:44:43 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/backup/restore-status +2026/10/25 16:44:43 router.go:78: [DEBUG] [api] GET /api/backup/restore-status (path=/backup/restore-status) +2026/10/25 16:44:44 restore.go:263: [DEBUG] [backup] Volume tandoor_tandoor_postgres_data restored successfully +2026/10/25 16:44:44 restore.go:225: [INFO] [backup] Restoring Docker volume tandoor_tandoor_static for tandoor +2026/10/25 16:44:47 restore.go:263: [DEBUG] [backup] Volume tandoor_tandoor_static restored successfully +2026/10/25 16:44:47 restore.go:268: [INFO] [backup] Restored 3 Docker volume(s) for tandoor +2026/10/25 16:44:47 deploy.go:866: [INFO] [stacks] tandoor: the restored data belongs to install c3b2e33d5bc82152 (was 8c1656d41509efbc) — its backups follow it (D47) +2026/10/25 16:44:47 [DEBUG] [stacks] SaveAppConfig: saving /opt/docker/stacks/tandoor — 4 env vars, 2 encrypted, 2 sensitive fields +2026/10/25 16:44:47 [INFO] [stacks] SaveAppConfig: saved config for tandoor +2026/10/25 16:44:47 deploy.go:911: [INFO] [stacks] Redeploying tandoor from recovery unit with 4 env vars +2026/10/25 16:44:47 [DEBUG] [stacks] SaveAppConfig: saving /opt/docker/stacks/tandoor — 4 env vars, 0 encrypted, 2 sensitive fields +2026/10/25 16:44:47 [INFO] [stacks] SaveAppConfig: saved config for tandoor +2026/10/25 16:44:47 pin.go:93: [INFO] [stacks] pin tandoor: tandoor=ghcr.io/tandoorrecipes/recipes:2.6.15, tandoor-postgres=postgres:17-alpine +2026/10/25 16:44:47 manager.go:1406: [INFO] [stacks] Starting stack tandoor services only: [tandoor-postgres] +2026/10/25 16:44:47 manager.go:1688: [DEBUG] Env vars for compose: [PATH, HOSTNAME, FELHOM_BOOTSTRAP_PATH, HOME, DOMAIN, DOMAIN, SECRET_KEY, SUBDOMAIN, DB_PASSWORD, IMPORT_PATH] (10 app + 0 system) +2026/10/25 16:44:47 manager.go:1697: [DEBUG] Running: docker compose up -d tandoor-postgres (in /opt/docker/stacks/tandoor) +2026/10/25 16:44:47 manager.go:1719: [DEBUG] Command completed: docker compose up -d tandoor-postgres (took 0.5s) +2026/10/25 16:44:47 manager.go:1417: [INFO] [stacks] Stack tandoor services [tandoor-postgres] started (took 0.5s) +2026/10/25 16:44:47 manager.go:839: [INFO] [stacks] Status refresh: 10 containers across 70 stacks +2026/10/25 16:44:47 dbdump.go:130: [DEBUG] DiscoverDatabases: running docker ps to find database containers +2026/10/25 16:44:47 dbdump.go:149: [DEBUG] DiscoverDatabases: docker ps output: 7e6376839afe tandoor-postgres tandoor postgres:17-alpine +74f0a38fc983 paperless-webserver paperless-ngx ghcr.io/paperless-ngx/paperless-ngx:2.20.15 +cb44703bbadd paperless-postgres paperless-ngx postgres:18-alpine +cd225421c992 paperless-redis paperless-ngx redis:7-alpine +f062a2761fab immich-server immich ghcr.io/immich-app/immich-server:v3.2.4 +6041eb882834 immich-postgres immich ghcr.io/immich-app/postgres:16-vectorchord0.4.3-pgvectors0.2.0 +ac5b8fbc0332 immich-machine-learning immich ghcr.io/immi... +2026/10/25 16:44:47 dbdump.go:178: [DEBUG] DiscoverDatabases: found postgres container: tandoor-postgres (id=7e6376839afe) +2026/10/25 16:44:48 dbdump.go:198: [DEBUG] DiscoverDatabases: tandoor-postgres → stack=tandoor, dbUser=tandoor, dbName=tandoor +2026/10/25 16:44:48 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container paperless-webserver (image=ghcr.io/paperless-ngx/paperless-ngx:2.20.15, not a database) +2026/10/25 16:44:48 dbdump.go:178: [DEBUG] DiscoverDatabases: found postgres container: paperless-postgres (id=cb44703bbadd) +2026/10/25 16:44:48 dbdump.go:915: [DEBUG] DiscoverDatabases: paperless-postgres → stack "paperless-ngx" from the compose project label (the container name would have given "paperless") +2026/10/25 16:44:48 dbdump.go:198: [DEBUG] DiscoverDatabases: paperless-postgres → stack=paperless-ngx, dbUser=paperless, dbName=paperless +2026/10/25 16:44:48 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container paperless-redis (image=redis:7-alpine, not a database) +2026/10/25 16:44:48 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container immich-server (image=ghcr.io/immich-app/immich-server:v3.2.4, not a database) +2026/10/25 16:44:48 dbdump.go:178: [DEBUG] DiscoverDatabases: found postgres container: immich-postgres (id=6041eb882834) +2026/10/25 16:44:48 dbdump.go:198: [DEBUG] DiscoverDatabases: immich-postgres → stack=immich, dbUser=immich, dbName=immich +2026/10/25 16:44:48 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container immich-machine-learning (image=ghcr.io/immich-app/immich-machine-learning:v3.2.4, not a database) +2026/10/25 16:44:48 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container immich-redis (image=redis:7-alpine, not a database) +2026/10/25 16:44:48 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container felhom-controller (image=gitea.dooplex.hu/admin/felhom-controller:0.343.0-rc.7, not a database) +2026/10/25 16:44:48 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container filebrowser (image=gtstef/filebrowser:1.5.6-stable, not a database) +2026/10/25 16:44:48 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container traefik (image=traefik:v3.7.13, not a database) +2026/10/25 16:44:48 dbdump.go:205: [DEBUG] DiscoverDatabases: found 3 database(s), skipped 8 non-DB container(s) +2026/10/25 16:44:48 dbdump.go:208: [INFO] [backup] Discovered 3 databases +2026/10/25 16:44:48 restore_db.go:87: [INFO] [backup] Restore tandoor: replaying DB dump into tandoor-postgres (postgres) +2026/10/25 16:44:48 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/backup/restore-status +2026/10/25 16:44:48 router.go:78: [DEBUG] [api] GET /api/backup/restore-status (path=/backup/restore-status) +2026/10/25 16:44:50 dbdump.go:800: [DEBUG] [backup] ImportDump: importing /mnt/sys_drive/felhom-data/backups/set-aside/tandoor/20261025T164421Z/db-dumps/tandoor-postgres.sql into tandoor-postgres (postgres) +2026/10/25 16:44:50 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting +2026/10/25 16:44:50 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:44:50 scheduler.go:85: [DEBUG] [scheduler] job after-unclean-boot: execution starting +2026/10/25 16:44:50 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 4m0s ago, effective interval 5m0s, healthy=true +2026/10/25 16:44:50 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 5m0s ago, effective interval 5m0s, healthy=true +2026/10/25 16:44:50 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (2 skipped not due, 0 skipped no container) +2026/10/25 16:44:50 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:44:50 scheduler.go:85: [DEBUG] [scheduler] job ring-spill: execution starting +2026/10/25 16:44:50 scheduler.go:85: [DEBUG] [scheduler] job deadapp-check: execution starting +2026/10/25 16:44:50 manager.go:839: [INFO] [stacks] Status refresh: 10 containers across 70 stacks +2026/10/25 16:44:52 dbdump.go:820: [INFO] [backup] Imported DB dump tandoor-postgres.sql into tandoor-postgres (postgres) +2026/10/25 16:44:52 restore_db.go:97: [INFO] [backup] Restore tandoor: replayed 1 DB dump(s) +2026/10/25 16:44:52 manager.go:1350: [DEBUG] [stacks] StartStack tandoor: current state=starting deployed=true +2026/10/25 16:44:52 manager.go:1353: [INFO] [stacks] Starting stack: tandoor +2026/10/25 16:44:52 manager.go:1360: [DEBUG] [stacks] StartStack tandoor: prepared 10 env vars for compose +2026/10/25 16:44:52 manager.go:1688: [DEBUG] Env vars for compose: [PATH, HOSTNAME, FELHOM_BOOTSTRAP_PATH, HOME, DOMAIN, DB_PASSWORD, DOMAIN, SECRET_KEY, SUBDOMAIN, IMPORT_PATH] (10 app + 0 system) +2026/10/25 16:44:52 manager.go:1697: [DEBUG] Running: docker compose up -d (in /opt/docker/stacks/tandoor) +2026/10/25 16:44:53 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/backup/restore-status +2026/10/25 16:44:53 router.go:78: [DEBUG] [api] GET /api/backup/restore-status (path=/backup/restore-status) +2026/10/25 16:44:53 manager.go:1719: [DEBUG] Command completed: docker compose up -d (took 1.1s) +2026/10/25 16:44:53 manager.go:1368: [INFO] [stacks] Stack tandoor started successfully (took 1.1s) +2026/10/25 16:44:53 [DEBUG] [stacks] SaveAppConfig: saving /opt/docker/stacks/tandoor — 4 env vars, 0 encrypted, 2 sensitive fields +2026/10/25 16:44:53 [INFO] [stacks] SaveAppConfig: saved config for tandoor +2026/10/25 16:44:53 installed.go:420: [INFO] [stacks] installed-images tandoor: recorded 2 service(s) (tandoor=ghcr.io/tandoorrecipes/recipes:2.6.15 (sha256:2e759dd1478a…), tandoor-postgres=postgres:17-alpine (sha256:b0f9560a2de0…)) +2026/10/25 16:44:53 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:44:56 manager.go:1688: [DEBUG] Env vars for compose: [PATH, HOSTNAME, FELHOM_BOOTSTRAP_PATH, HOME, DOMAIN, DB_PASSWORD, DOMAIN, SECRET_KEY, SUBDOMAIN, IMPORT_PATH] (10 app + 0 system) +2026/10/25 16:44:56 manager.go:1697: [DEBUG] Running: docker compose ps -a --format table {{.Name}} {{.Image}} {{.State}} {{.Status}} (in /opt/docker/stacks/tandoor) +2026/10/25 16:44:56 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:44:56 restore.go:308: [DEBUG] [backup] Post-restore health check: tandoor not yet running, waiting... +2026/10/25 16:44:56 manager.go:1719: [DEBUG] Command completed: docker compose ps -a --format table {{.Name}} {{.Image}} {{.State}} {{.Status}} (took 0.1s) +2026/10/25 16:44:56 manager.go:1813: [INFO] [stacks] Stack tandoor post-start status: +2026/10/25 16:44:56 manager.go:1816: [INFO] [stacks] tandoor ghcr.io/tandoorrecipes/recipes:2.6.15@sha256:2e759dd1478a2ed119ee474e28522079fb1cfa50b3fd25cba89f6b9a67abad72 running Up 3 seconds (health: starting) +2026/10/25 16:44:56 manager.go:1816: [INFO] [stacks] tandoor-postgres postgres:17-alpine@sha256:b0f9560a2de083e2cc7382e75f808c7381a32852a7ec49117deedb300e552b24 running Up 9 seconds (healthy) +2026/10/25 16:44:58 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/backup/restore-status +2026/10/25 16:44:58 router.go:78: [DEBUG] [api] GET /api/backup/restore-status (path=/backup/restore-status) +2026/10/25 16:45:00 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:45:00 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:45:00 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:45:00 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 4m10s ago, effective interval 5m0s, healthy=true +2026/10/25 16:45:00 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 1 targets (1 skipped not due, 0 skipped no container) +2026/10/25 16:45:00 healthprobe.go:187: [DEBUG] Health probe immich: API GET :2283/api/server/ping → 200 (3ms) +2026/10/25 16:45:00 healthprobe.go:167: [INFO] Health probes: 1 ok (of 1 probed) +2026/10/25 16:45:01 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:45:01 restore.go:308: [DEBUG] [backup] Post-restore health check: tandoor not yet running, waiting... +2026/10/25 16:45:03 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/backup/restore-status +2026/10/25 16:45:03 router.go:78: [DEBUG] [api] GET /api/backup/restore-status (path=/backup/restore-status) +2026/10/25 16:45:05 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting +2026/10/25 16:45:07 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:45:07 restore.go:308: [DEBUG] [backup] Post-restore health check: tandoor not yet running, waiting... +2026/10/25 16:45:08 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/backup/restore-status +2026/10/25 16:45:08 router.go:78: [DEBUG] [api] GET /api/backup/restore-status (path=/backup/restore-status) +2026/10/25 16:45:10 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:45:10 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:45:10 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:45:10 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 10s ago, effective interval 5m0s, healthy=true +2026/10/25 16:45:10 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 4m20s ago, effective interval 5m0s, healthy=true +2026/10/25 16:45:10 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (2 skipped not due, 0 skipped no container) +2026/10/25 16:45:12 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:45:12 restore.go:308: [DEBUG] [backup] Post-restore health check: tandoor not yet running, waiting... +2026/10/25 16:45:13 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/backup/restore-status +2026/10/25 16:45:13 router.go:78: [DEBUG] [api] GET /api/backup/restore-status (path=/backup/restore-status) +2026/10/25 16:45:17 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:45:17 restore.go:303: [DEBUG] [backup] Post-restore health check: tandoor is running +2026/10/25 16:45:17 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:45:18 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/backup/restore-status +2026/10/25 16:45:18 router.go:78: [DEBUG] [api] GET /api/backup/restore-status (path=/backup/restore-status) +2026/10/25 16:45:20 info.go:13: [DEBUG] [system] GetInfo starting (hddPath="/mnt/felhom-drives/scratch_hdd", hasCPUCollector=true) +2026/10/25 16:45:20 info.go:13: [DEBUG] [system] readMemInfo: guest cap=25898MB (host total was 30714380KB) → used≈13971MB avail≈11927MB +2026/10/25 16:45:20 info.go:13: [DEBUG] [system] readMemInfo: totalKB=30714380 availKB=14145112 → total=25898MB avail=11927MB used=13971MB (53.9%) +2026/10/25 16:45:20 info.go:13: [DEBUG] [system] readDiskUsage: path="/" bsize=4096 total=68.4GB used=13.1GB avail=51.7GB (20.3%) +2026/10/25 16:45:20 info.go:13: [DEBUG] [system] readDiskUsage: path="/mnt/felhom-drives/scratch_hdd" bsize=4096 total=937.8GB used=219.7GB avail=670.4GB (24.7%) +2026/10/25 16:45:20 info.go:13: [DEBUG] [system] readLoadAvg: raw="2.50 2.76 2.26 7/1719 4747" → 1m=2.50 5m=2.76 15m=2.26 +2026/10/25 16:45:20 info.go:13: [DEBUG] [system] readThermalZones: /sys — found 1 zones +2026/10/25 16:45:20 info.go:13: [DEBUG] [system] readTemperature: found via hwmon at /sys — 57.8°C (hwmon2) +2026/10/25 16:45:20 info.go:13: [DEBUG] [system] GetInfo done in 62ms — mem=13971MB/25898MB (53.9%), rootDisk=13.1GB/68.4GB (20.3%), load=2.50/2.76/2.26, temp=57.8°C (hwmon2), cpu=51.0% +2026/10/25 16:45:20 scheduler.go:85: [DEBUG] [scheduler] job ring-spill: execution starting +2026/10/25 16:45:20 scheduler.go:85: [DEBUG] [scheduler] job after-unclean-boot: execution starting +2026/10/25 16:45:20 scheduler.go:85: [DEBUG] [scheduler] job deadapp-check: execution starting +2026/10/25 16:45:20 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting +2026/10/25 16:45:20 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:45:20 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 4m30s ago, effective interval 5m0s, healthy=true +2026/10/25 16:45:20 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 20s ago, effective interval 5m0s, healthy=true +2026/10/25 16:45:20 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:45:20 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 1 targets (2 skipped not due, 0 skipped no container) +2026/10/25 16:45:20 healthprobe.go:187: [DEBUG] Health probe tandoor: HTTP GET :80/accounts/login/ → 400 (9ms) +2026/10/25 16:45:20 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:45:20 healthprobe.go:167: [INFO] Health probes: 1 ok (of 1 probed) +2026/10/25 16:45:21 main.go:2394: [INFO] [deadapp] check alive: 20 scans since boot, 4 deployed app(s) evaluated, 0 currently down +2026/10/25 16:45:23 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/backup/restore-status +2026/10/25 16:45:23 router.go:78: [DEBUG] [api] GET /api/backup/restore-status (path=/backup/restore-status) +2026/10/25 16:45:27 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:45:28 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/backup/restore-status +2026/10/25 16:45:28 router.go:78: [DEBUG] [api] GET /api/backup/restore-status (path=/backup/restore-status) +2026/10/25 16:45:30 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:45:30 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 30s ago, effective interval 5m0s, healthy=true +2026/10/25 16:45:30 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping tandoor — last check 10s ago, effective interval 5m0s, healthy=true +2026/10/25 16:45:30 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 4m40s ago, effective interval 5m0s, healthy=true +2026/10/25 16:45:30 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (3 skipped not due, 0 skipped no container) +2026/10/25 16:45:30 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:45:30 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:45:33 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/backup/restore-status +2026/10/25 16:45:33 router.go:78: [DEBUG] [api] GET /api/backup/restore-status (path=/backup/restore-status) +2026/10/25 16:45:35 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting +2026/10/25 16:45:37 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:45:38 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/backup/restore-status +2026/10/25 16:45:38 router.go:78: [DEBUG] [api] GET /api/backup/restore-status (path=/backup/restore-status) +2026/10/25 16:45:40 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:45:40 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:45:40 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:45:40 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 40s ago, effective interval 5m0s, healthy=true +2026/10/25 16:45:40 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping tandoor — last check 20s ago, effective interval 5m0s, healthy=true +2026/10/25 16:45:40 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 4m50s ago, effective interval 5m0s, healthy=true +2026/10/25 16:45:40 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (3 skipped not due, 0 skipped no container) +2026/10/25 16:45:43 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/backup/restore-status +2026/10/25 16:45:43 router.go:78: [DEBUG] [api] GET /api/backup/restore-status (path=/backup/restore-status) +2026/10/25 16:45:47 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:45:48 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/backup/restore-status +2026/10/25 16:45:48 router.go:78: [DEBUG] [api] GET /api/backup/restore-status (path=/backup/restore-status) +2026/10/25 16:45:50 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:45:50 scheduler.go:85: [DEBUG] [scheduler] job after-unclean-boot: execution starting +2026/10/25 16:45:50 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting +2026/10/25 16:45:50 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 50s ago, effective interval 5m0s, healthy=true +2026/10/25 16:45:50 scheduler.go:85: [DEBUG] [scheduler] job ring-spill: execution starting +2026/10/25 16:45:50 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping tandoor — last check 30s ago, effective interval 5m0s, healthy=true +2026/10/25 16:45:50 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 1 targets (2 skipped not due, 0 skipped no container) +2026/10/25 16:45:50 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:45:50 scheduler.go:85: [DEBUG] [scheduler] job deadapp-check: execution starting +2026/10/25 16:45:50 healthprobe.go:187: [DEBUG] Health probe paperless-ngx: HTTP GET :8000/ → 302 (4ms) +2026/10/25 16:45:50 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:45:50 healthprobe.go:167: [INFO] Health probes: 1 ok (of 1 probed) +2026/10/25 16:45:53 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/backup/restore-status +2026/10/25 16:45:53 router.go:78: [DEBUG] [api] GET /api/backup/restore-status (path=/backup/restore-status) +2026/10/25 16:45:57 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:45:58 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/backup/restore-status +2026/10/25 16:45:58 router.go:78: [DEBUG] [api] GET /api/backup/restore-status (path=/backup/restore-status) +2026/10/25 16:46:00 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:46:00 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 1m0s ago, effective interval 5m0s, healthy=true +2026/10/25 16:46:00 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping tandoor — last check 40s ago, effective interval 5m0s, healthy=true +2026/10/25 16:46:00 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 10s ago, effective interval 5m0s, healthy=true +2026/10/25 16:46:00 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (3 skipped not due, 0 skipped no container) +2026/10/25 16:46:00 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:46:00 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:46:03 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/backup/restore-status +2026/10/25 16:46:03 router.go:78: [DEBUG] [api] GET /api/backup/restore-status (path=/backup/restore-status) +2026/10/25 16:46:05 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting +2026/10/25 16:46:07 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:46:07 restore_unit.go:591: [INFO] [backup] Restore-from-unit completed: tandoor — 3 volume(s) of 3 listed, 1 database(s) of 1 listed +2026/10/25 16:46:07 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:46:08 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/backup/restore-status +2026/10/25 16:46:08 router.go:78: [DEBUG] [api] GET /api/backup/restore-status (path=/backup/restore-status) +2026/10/25 16:46:10 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:46:10 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:46:10 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:46:10 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 1m10s ago, effective interval 5m0s, healthy=true +2026/10/25 16:46:10 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping tandoor — last check 50s ago, effective interval 5m0s, healthy=true +2026/10/25 16:46:10 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 20s ago, effective interval 5m0s, healthy=true +2026/10/25 16:46:10 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (3 skipped not due, 0 skipped no container) +2026/10/25 16:46:18 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:46:20 info.go:13: [DEBUG] [system] GetInfo starting (hddPath="/mnt/felhom-drives/scratch_hdd", hasCPUCollector=true) +2026/10/25 16:46:20 info.go:13: [DEBUG] [system] readMemInfo: guest cap=25898MB (host total was 30714380KB) → used≈13950MB avail≈11948MB +2026/10/25 16:46:20 info.go:13: [DEBUG] [system] readMemInfo: totalKB=30714380 availKB=14170100 → total=25898MB avail=11948MB used=13950MB (53.9%) +2026/10/25 16:46:20 info.go:13: [DEBUG] [system] readDiskUsage: path="/" bsize=4096 total=68.4GB used=13.1GB avail=51.7GB (20.3%) +2026/10/25 16:46:20 info.go:13: [DEBUG] [system] readDiskUsage: path="/mnt/felhom-drives/scratch_hdd" bsize=4096 total=937.8GB used=219.7GB avail=670.4GB (24.7%) +2026/10/25 16:46:20 info.go:13: [DEBUG] [system] readLoadAvg: raw="1.90 2.49 2.20 7/1717 5149" → 1m=1.90 5m=2.49 15m=2.20 +2026/10/25 16:46:20 info.go:13: [DEBUG] [system] readThermalZones: /sys — found 1 zones +2026/10/25 16:46:20 info.go:13: [DEBUG] [system] readTemperature: found via hwmon at /sys — 59.9°C (hwmon2) +2026/10/25 16:46:20 info.go:13: [DEBUG] [system] GetInfo done in 56ms — mem=13950MB/25898MB (53.9%), rootDisk=13.1GB/68.4GB (20.3%), load=1.90/2.49/2.20, temp=59.9°C (hwmon2), cpu=19.1% +2026/10/25 16:46:20 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:46:20 scheduler.go:450: [INFO] [scheduler] Running job: stack-scan +2026/10/25 16:46:20 scheduler.go:85: [DEBUG] [scheduler] job stack-scan: execution starting +2026/10/25 16:46:20 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 30s ago, effective interval 5m0s, healthy=true +2026/10/25 16:46:20 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting +2026/10/25 16:46:20 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:46:20 scheduler.go:85: [DEBUG] [scheduler] job deadapp-check: execution starting +2026/10/25 16:46:20 scheduler.go:85: [DEBUG] [scheduler] job ring-spill: execution starting +2026/10/25 16:46:20 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 1m20s ago, effective interval 5m0s, healthy=true +2026/10/25 16:46:20 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping tandoor — last check 1m0s ago, effective interval 5m0s, healthy=true +2026/10/25 16:46:20 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (3 skipped not due, 0 skipped no container) +2026/10/25 16:46:20 scheduler.go:85: [DEBUG] [scheduler] job after-unclean-boot: execution starting +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "actualbudget" deployed=false composePath=/opt/docker/stacks/actualbudget/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "adventurelog" deployed=false composePath=/opt/docker/stacks/adventurelog/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "audiobookshelf" deployed=false composePath=/opt/docker/stacks/audiobookshelf/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "bentopdf" deployed=false composePath=/opt/docker/stacks/bentopdf/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "bookstack" deployed=false composePath=/opt/docker/stacks/bookstack/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "calcom" deployed=false composePath=/opt/docker/stacks/calcom/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "calibre-web" deployed=false composePath=/opt/docker/stacks/calibre-web/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "chaoscrash" deployed=false composePath=/opt/docker/stacks/chaoscrash/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "chaosoom" deployed=false composePath=/opt/docker/stacks/chaosoom/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "chaosoomb" deployed=false composePath=/opt/docker/stacks/chaosoomb/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "claper" deployed=false composePath=/opt/docker/stacks/claper/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "code-server" deployed=false composePath=/opt/docker/stacks/code-server/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "crafty-controller" deployed=false composePath=/opt/docker/stacks/crafty-controller/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "dawarich" deployed=false composePath=/opt/docker/stacks/dawarich/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "docmost" deployed=false composePath=/opt/docker/stacks/docmost/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "emby" deployed=false composePath=/opt/docker/stacks/emby/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "filebrowser" deployed=false composePath=/opt/docker/stacks/filebrowser/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "ghost" deployed=false composePath=/opt/docker/stacks/ghost/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "gitea" deployed=false composePath=/opt/docker/stacks/gitea/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "glance" deployed=false composePath=/opt/docker/stacks/glance/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "gokapi" deployed=false composePath=/opt/docker/stacks/gokapi/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "grafana" deployed=false composePath=/opt/docker/stacks/grafana/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "gramps-web" deployed=false composePath=/opt/docker/stacks/gramps-web/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "grimmory" deployed=false composePath=/opt/docker/stacks/grimmory/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "grocy" deployed=false composePath=/opt/docker/stacks/grocy/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "home-assistant" deployed=false composePath=/opt/docker/stacks/home-assistant/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "homebox" deployed=false composePath=/opt/docker/stacks/homebox/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "homepage" deployed=false composePath=/opt/docker/stacks/homepage/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "immich" deployed=true composePath=/opt/docker/stacks/immich/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "jellyfin" deployed=false composePath=/opt/docker/stacks/jellyfin/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "karakeep" deployed=false composePath=/opt/docker/stacks/karakeep/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "kimai" deployed=false composePath=/opt/docker/stacks/kimai/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "komga" deployed=false composePath=/opt/docker/stacks/komga/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "lubelogger" deployed=false composePath=/opt/docker/stacks/lubelogger/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "mealie" deployed=false composePath=/opt/docker/stacks/mealie/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "metube" deployed=false composePath=/opt/docker/stacks/metube/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "n8n" deployed=false composePath=/opt/docker/stacks/n8n/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "navidrome" deployed=false composePath=/opt/docker/stacks/navidrome/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "nextcloud" deployed=false composePath=/opt/docker/stacks/nextcloud/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "nzbget" deployed=false composePath=/opt/docker/stacks/nzbget/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "onlyoffice" deployed=false composePath=/opt/docker/stacks/onlyoffice/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "opengist" deployed=false composePath=/opt/docker/stacks/opengist/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "outline" deployed=false composePath=/opt/docker/stacks/outline/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "paperless-ngx" deployed=true composePath=/opt/docker/stacks/paperless-ngx/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "papra" deployed=false composePath=/opt/docker/stacks/papra/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "plant-it" deployed=false composePath=/opt/docker/stacks/plant-it/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "plex" deployed=false composePath=/opt/docker/stacks/plex/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "privatebin" deployed=true composePath=/opt/docker/stacks/privatebin/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "prowlarr" deployed=false composePath=/opt/docker/stacks/prowlarr/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "qbittorrent" deployed=false composePath=/opt/docker/stacks/qbittorrent/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "radarr" deployed=false composePath=/opt/docker/stacks/radarr/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "radicale" deployed=false composePath=/opt/docker/stacks/radicale/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "rallly" deployed=false composePath=/opt/docker/stacks/rallly/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "recipe-importer" deployed=false composePath=/opt/docker/stacks/recipe-importer/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "romm" deployed=false composePath=/opt/docker/stacks/romm/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "sabnzbd" deployed=false composePath=/opt/docker/stacks/sabnzbd/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "seerr" deployed=false composePath=/opt/docker/stacks/seerr/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "sonarr" deployed=false composePath=/opt/docker/stacks/sonarr/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "sparkyfitness" deployed=false composePath=/opt/docker/stacks/sparkyfitness/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "tandoor" deployed=true composePath=/opt/docker/stacks/tandoor/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "termix" deployed=false composePath=/opt/docker/stacks/termix/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "traefik" deployed=false composePath=/opt/docker/stacks/traefik/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "transmission" deployed=false composePath=/opt/docker/stacks/transmission/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "uptime-kuma" deployed=false composePath=/opt/docker/stacks/uptime-kuma/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "vaultwarden" deployed=false composePath=/opt/docker/stacks/vaultwarden/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "vikunja" deployed=false composePath=/opt/docker/stacks/vikunja/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "wanderer" deployed=false composePath=/opt/docker/stacks/wanderer/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "wger" deployed=false composePath=/opt/docker/stacks/wger/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "wishlist" deployed=false composePath=/opt/docker/stacks/wishlist/docker-compose.yml +2026/10/25 16:46:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "zipline" deployed=false composePath=/opt/docker/stacks/zipline/docker-compose.yml +2026/10/25 16:46:20 manager.go:1960: [DEBUG] [stacks] getCatalogTemplateSlugs: found 65 template slugs in /opt/docker/felhom-controller/data/catalog-cache/templates +2026/10/25 16:46:20 manager.go:757: [DEBUG] [stacks] ScanStacks: catalog has 65 template slugs for orphan detection +2026/10/25 16:46:20 manager.go:788: [INFO] [stacks] ScanStacks complete: 70 stacks found (4 deployed, 66 available) +2026/10/25 16:46:21 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:46:21 scheduler.go:467: [INFO] [scheduler] Job stack-scan completed (took 119ms) +2026/10/25 16:46:21 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:46:28 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:46:28 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:46:28 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:46:30 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:46:30 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:46:30 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 40s ago, effective interval 5m0s, healthy=true +2026/10/25 16:46:30 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 1m30s ago, effective interval 5m0s, healthy=true +2026/10/25 16:46:30 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping tandoor — last check 1m10s ago, effective interval 5m0s, healthy=true +2026/10/25 16:46:30 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (3 skipped not due, 0 skipped no container) +2026/10/25 16:46:30 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:46:35 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting +2026/10/25 16:46:38 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:46:40 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:46:40 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:46:40 manager.go:839: [INFO] [stacks] Status refresh: 11 containers across 70 stacks +2026/10/25 16:46:40 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 1m40s ago, effective interval 5m0s, healthy=true +2026/10/25 16:46:40 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping tandoor — last check 1m20s ago, effective interval 5m0s, healthy=true +2026/10/25 16:46:40 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 50s ago, effective interval 5m0s, healthy=true +2026/10/25 16:46:40 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (3 skipped not due, 0 skipped no container) +2026/10/25 16:46:44 auth.go:144: [DEBUG] [web] auth: valid session for POST /api/stacks/tandoor/stop +2026/10/25 16:46:44 router.go:78: [DEBUG] [api] POST /api/stacks/tandoor/stop (path=/stacks/tandoor/stop) +2026/10/25 16:46:44 router.go:663: [INFO] [api] stop requested for stack: tandoor +2026/10/25 16:46:44 router.go:78: [DEBUG] [api] actionStack: action=stop name=tandoor +2026/10/25 16:46:44 [DEBUG] [stacks] SaveAppConfig: saving /opt/docker/stacks/tandoor — 4 env vars, 0 encrypted, 2 sensitive fields +2026/10/25 16:46:44 [INFO] [stacks] SaveAppConfig: saved config for tandoor +2026/10/25 16:46:44 desiredstate.go:81: [INFO] [stacks] desired state for tandoor recorded as "stopped" (was "running") +2026/10/25 16:46:44 manager.go:1439: [DEBUG] [stacks] StopStack tandoor: current state=running deployed=true containers=2 +2026/10/25 16:46:44 manager.go:1442: [INFO] [stacks] Stopping stack: tandoor +2026/10/25 16:46:44 manager.go:1688: [DEBUG] Env vars for compose: [PATH, HOSTNAME, FELHOM_BOOTSTRAP_PATH, HOME, DOMAIN, SECRET_KEY, SUBDOMAIN, DB_PASSWORD, DOMAIN, IMPORT_PATH] (10 app + 0 system) +2026/10/25 16:46:44 manager.go:1697: [DEBUG] Running: docker compose down (in /opt/docker/stacks/tandoor) +2026/10/25 16:46:46 manager.go:1719: [DEBUG] Command completed: docker compose down (took 2.8s) +2026/10/25 16:46:46 manager.go:1451: [INFO] [stacks] Stack tandoor stopped successfully (took 2.8s) +2026/10/25 16:46:46 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:46:46 lifecycle.go:24: [DEBUG] [integrations] OnStackStop: stack=tandoor integrationsFound=0 +2026/10/25 16:46:48 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:46:48 restore_verdict.go:172: [DEBUG] [backup] stable-start sample tandoor: not ok — the stack does not read running +2026/10/25 16:46:50 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:46:50 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:46:50 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting +2026/10/25 16:46:50 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 1m50s ago, effective interval 5m0s, healthy=true +2026/10/25 16:46:50 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 1m0s ago, effective interval 5m0s, healthy=true +2026/10/25 16:46:50 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (2 skipped not due, 0 skipped no container) +2026/10/25 16:46:50 scheduler.go:85: [DEBUG] [scheduler] job after-unclean-boot: execution starting +2026/10/25 16:46:50 scheduler.go:85: [DEBUG] [scheduler] job deadapp-check: execution starting +2026/10/25 16:46:50 scheduler.go:85: [DEBUG] [scheduler] job ring-spill: execution starting +2026/10/25 16:46:50 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:46:51 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:46:51 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:46:51 auth.go:144: [DEBUG] [web] auth: valid session for POST /api/stacks/tandoor/remove +2026/10/25 16:46:51 router.go:78: [DEBUG] [api] POST /api/stacks/tandoor/remove (path=/stacks/tandoor/remove) +2026/10/25 16:46:51 router.go:1025: [INFO] [api] Remove requested for stack: tandoor +2026/10/25 16:46:51 router.go:78: [DEBUG] [api] removeStack: name=tandoor +2026/10/25 16:46:51 router.go:78: [DEBUG] [api] removeStack: name=tandoor removeHDDData=true removeBackups=true +2026/10/25 16:46:52 delete.go:626: [DEBUG] [stacks] RemoveStack called: name="tandoor", removeHDDData=true, backupPathsToRemove=3 +2026/10/25 16:46:52 delete.go:640: [DEBUG] [stacks] RemoveStack tandoor: state=stopped, deployed=true, orphaned=false, deploying=false +2026/10/25 16:46:52 delete.go:697: [INFO] Removing deployed stack: tandoor (removeHDDData=true, hddDeclared=false, backupPaths=3) +2026/10/25 16:46:52 delete.go:710: [DEBUG] [stacks] RemoveStack tandoor: found 0 HDD mounts from compose file +2026/10/25 16:46:52 manager.go:1688: [DEBUG] Env vars for compose: [PATH, HOSTNAME, FELHOM_BOOTSTRAP_PATH, HOME, DOMAIN, DOMAIN, SECRET_KEY, SUBDOMAIN, DB_PASSWORD, IMPORT_PATH] (10 app + 0 system) +2026/10/25 16:46:52 manager.go:1697: [DEBUG] Running: docker compose down --volumes (in /opt/docker/stacks/tandoor) +2026/10/25 16:46:52 manager.go:1719: [DEBUG] Command completed: docker compose down --volumes (took 0.3s) +2026/10/25 16:46:52 delete.go:736: [DEBUG] [stacks] RemoveStack tandoor: compose down output: +2026/10/25 16:46:58 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:46:58 restore_verdict.go:172: [DEBUG] [backup] stable-start sample tandoor: not ok — the stack does not read running +2026/10/25 16:47:00 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:47:00 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:47:00 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:47:00 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 2m0s ago, effective interval 5m0s, healthy=true +2026/10/25 16:47:00 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 1m10s ago, effective interval 5m0s, healthy=true +2026/10/25 16:47:00 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (2 skipped not due, 0 skipped no container) +2026/10/25 16:47:05 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting +2026/10/25 16:47:08 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:47:08 restore_verdict.go:172: [DEBUG] [backup] stable-start sample tandoor: not ok — the stack does not read running +2026/10/25 16:47:10 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:47:10 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:47:10 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:47:10 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 2m10s ago, effective interval 5m0s, healthy=true +2026/10/25 16:47:10 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 1m20s ago, effective interval 5m0s, healthy=true +2026/10/25 16:47:10 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (2 skipped not due, 0 skipped no container) +2026/10/25 16:47:18 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:47:18 restore_verdict.go:172: [DEBUG] [backup] stable-start sample tandoor: not ok — the stack does not read running +2026/10/25 16:47:18 delete.go:774: [INFO] [stacks] RemoveStack tandoor: removed volume(s) [tandoor_tandoor_media tandoor_tandoor_postgres_data tandoor_tandoor_static] +2026/10/25 16:47:19 delete.go:837: [DEBUG] [stacks] RemoveStack tandoor: processing 3 backup paths for removal (base=/mnt/sys_drive/felhom-data/backups) +2026/10/25 16:47:19 delete.go:854: [INFO] Removed backup data: /mnt/sys_drive/felhom-data/backups/primary/tandoor (32K) +2026/10/25 16:47:19 delete.go:854: [INFO] Removed backup data: /mnt/sys_drive/felhom-data/backups/restore-undo/tandoor (144M) +2026/10/25 16:47:19 delete.go:862: [DEBUG] [stacks] RemoveStack tandoor: removing app.yaml at /opt/docker/stacks/tandoor/app.yaml +2026/10/25 16:47:19 delete.go:879: [INFO] Stack tandoor removed successfully (took 27.0s) +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "actualbudget" deployed=false composePath=/opt/docker/stacks/actualbudget/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "adventurelog" deployed=false composePath=/opt/docker/stacks/adventurelog/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "audiobookshelf" deployed=false composePath=/opt/docker/stacks/audiobookshelf/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "bentopdf" deployed=false composePath=/opt/docker/stacks/bentopdf/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "bookstack" deployed=false composePath=/opt/docker/stacks/bookstack/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "calcom" deployed=false composePath=/opt/docker/stacks/calcom/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "calibre-web" deployed=false composePath=/opt/docker/stacks/calibre-web/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "chaoscrash" deployed=false composePath=/opt/docker/stacks/chaoscrash/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "chaosoom" deployed=false composePath=/opt/docker/stacks/chaosoom/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "chaosoomb" deployed=false composePath=/opt/docker/stacks/chaosoomb/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "claper" deployed=false composePath=/opt/docker/stacks/claper/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "code-server" deployed=false composePath=/opt/docker/stacks/code-server/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "crafty-controller" deployed=false composePath=/opt/docker/stacks/crafty-controller/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "dawarich" deployed=false composePath=/opt/docker/stacks/dawarich/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "docmost" deployed=false composePath=/opt/docker/stacks/docmost/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "emby" deployed=false composePath=/opt/docker/stacks/emby/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "filebrowser" deployed=false composePath=/opt/docker/stacks/filebrowser/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "ghost" deployed=false composePath=/opt/docker/stacks/ghost/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "gitea" deployed=false composePath=/opt/docker/stacks/gitea/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "glance" deployed=false composePath=/opt/docker/stacks/glance/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "gokapi" deployed=false composePath=/opt/docker/stacks/gokapi/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "grafana" deployed=false composePath=/opt/docker/stacks/grafana/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "gramps-web" deployed=false composePath=/opt/docker/stacks/gramps-web/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "grimmory" deployed=false composePath=/opt/docker/stacks/grimmory/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "grocy" deployed=false composePath=/opt/docker/stacks/grocy/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "home-assistant" deployed=false composePath=/opt/docker/stacks/home-assistant/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "homebox" deployed=false composePath=/opt/docker/stacks/homebox/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "homepage" deployed=false composePath=/opt/docker/stacks/homepage/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "immich" deployed=true composePath=/opt/docker/stacks/immich/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "jellyfin" deployed=false composePath=/opt/docker/stacks/jellyfin/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "karakeep" deployed=false composePath=/opt/docker/stacks/karakeep/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "kimai" deployed=false composePath=/opt/docker/stacks/kimai/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "komga" deployed=false composePath=/opt/docker/stacks/komga/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "lubelogger" deployed=false composePath=/opt/docker/stacks/lubelogger/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "mealie" deployed=false composePath=/opt/docker/stacks/mealie/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "metube" deployed=false composePath=/opt/docker/stacks/metube/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "n8n" deployed=false composePath=/opt/docker/stacks/n8n/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "navidrome" deployed=false composePath=/opt/docker/stacks/navidrome/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "nextcloud" deployed=false composePath=/opt/docker/stacks/nextcloud/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "nzbget" deployed=false composePath=/opt/docker/stacks/nzbget/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "onlyoffice" deployed=false composePath=/opt/docker/stacks/onlyoffice/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "opengist" deployed=false composePath=/opt/docker/stacks/opengist/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "outline" deployed=false composePath=/opt/docker/stacks/outline/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "paperless-ngx" deployed=true composePath=/opt/docker/stacks/paperless-ngx/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "papra" deployed=false composePath=/opt/docker/stacks/papra/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "plant-it" deployed=false composePath=/opt/docker/stacks/plant-it/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "plex" deployed=false composePath=/opt/docker/stacks/plex/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "privatebin" deployed=true composePath=/opt/docker/stacks/privatebin/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "prowlarr" deployed=false composePath=/opt/docker/stacks/prowlarr/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "qbittorrent" deployed=false composePath=/opt/docker/stacks/qbittorrent/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "radarr" deployed=false composePath=/opt/docker/stacks/radarr/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "radicale" deployed=false composePath=/opt/docker/stacks/radicale/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "rallly" deployed=false composePath=/opt/docker/stacks/rallly/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "recipe-importer" deployed=false composePath=/opt/docker/stacks/recipe-importer/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "romm" deployed=false composePath=/opt/docker/stacks/romm/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "sabnzbd" deployed=false composePath=/opt/docker/stacks/sabnzbd/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "seerr" deployed=false composePath=/opt/docker/stacks/seerr/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "sonarr" deployed=false composePath=/opt/docker/stacks/sonarr/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "sparkyfitness" deployed=false composePath=/opt/docker/stacks/sparkyfitness/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "tandoor" deployed=false composePath=/opt/docker/stacks/tandoor/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "termix" deployed=false composePath=/opt/docker/stacks/termix/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "traefik" deployed=false composePath=/opt/docker/stacks/traefik/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "transmission" deployed=false composePath=/opt/docker/stacks/transmission/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "uptime-kuma" deployed=false composePath=/opt/docker/stacks/uptime-kuma/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "vaultwarden" deployed=false composePath=/opt/docker/stacks/vaultwarden/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "vikunja" deployed=false composePath=/opt/docker/stacks/vikunja/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "wanderer" deployed=false composePath=/opt/docker/stacks/wanderer/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "wger" deployed=false composePath=/opt/docker/stacks/wger/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "wishlist" deployed=false composePath=/opt/docker/stacks/wishlist/docker-compose.yml +2026/10/25 16:47:19 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "zipline" deployed=false composePath=/opt/docker/stacks/zipline/docker-compose.yml +2026/10/25 16:47:19 manager.go:1960: [DEBUG] [stacks] getCatalogTemplateSlugs: found 65 template slugs in /opt/docker/felhom-controller/data/catalog-cache/templates +2026/10/25 16:47:19 manager.go:757: [DEBUG] [stacks] ScanStacks: catalog has 65 template slugs for orphan detection +2026/10/25 16:47:19 manager.go:788: [INFO] [stacks] ScanStacks complete: 70 stacks found (3 deployed, 67 available) +2026/10/25 16:47:19 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:47:19 unit_owner.go:280: [INFO] [backup] remove tandoor: the set-aside units of earlier installs deleted with its backups: /mnt/sys_drive/felhom-data/backups/set-aside/tandoor (144.3 MB) +2026/10/25 16:47:19 settings.go:970: [DEBUG] [settings] saved to /opt/docker/felhom-controller/data/settings.json (27286 bytes) +2026/10/25 16:47:19 settings.go:973: [INFO] [settings] Settings saved +2026/10/25 16:47:19 tier2_kept.go:95: [INFO] [backup] kept copy /mnt/sys_drive/felhom-data/backups/set-aside/tandoor/20261025T164421Z was deleted with its app's backups — record dropped +2026/10/25 16:47:19 notifier.go:1467: [DEBUG] notifier disabled: dropped event app_removed (severity info) +2026/10/25 16:47:19 lifecycle.go:176: [DEBUG] [integrations] OnStackRemove: stack=tandoor integrationsFound=0 +2026/10/25 16:47:20 info.go:13: [DEBUG] [system] GetInfo starting (hddPath="/mnt/felhom-drives/scratch_hdd", hasCPUCollector=true) +2026/10/25 16:47:20 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:47:20 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:47:20 scheduler.go:85: [DEBUG] [scheduler] job deadapp-check: execution starting +2026/10/25 16:47:20 scheduler.go:85: [DEBUG] [scheduler] job ring-spill: execution starting +2026/10/25 16:47:20 scheduler.go:85: [DEBUG] [scheduler] job after-unclean-boot: execution starting +2026/10/25 16:47:20 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting +2026/10/25 16:47:20 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:47:20 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 2m20s ago, effective interval 5m0s, healthy=true +2026/10/25 16:47:20 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 1m30s ago, effective interval 5m0s, healthy=true +2026/10/25 16:47:20 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (2 skipped not due, 0 skipped no container) +2026/10/25 16:47:20 setup_gate.go:492: [INFO] [stacks] tandoor: removed a setup-gate file for an app whose gate is not closed +2026/10/25 16:47:21 image_retention.go:265: [INFO] [stacks] image retention (remove of tandoor): deleted [ghcr.io/tandoorrecipes/recipes:] (21ca08430f20, 919MB) — no container, installed app or undo names it (decision 53) +2026/10/25 16:47:21 info.go:13: [DEBUG] [system] readMemInfo: guest cap=25898MB (host total was 30714380KB) → used≈13189MB avail≈12709MB +2026/10/25 16:47:21 info.go:13: [DEBUG] [system] readMemInfo: totalKB=30714380 availKB=15071312 → total=25898MB avail=12709MB used=13189MB (50.9%) +2026/10/25 16:47:21 info.go:13: [DEBUG] [system] readDiskUsage: path="/" bsize=4096 total=68.4GB used=11.7GB avail=53.2GB (18.0%) +2026/10/25 16:47:21 info.go:13: [DEBUG] [system] readDiskUsage: path="/mnt/felhom-drives/scratch_hdd" bsize=4096 total=937.8GB used=219.7GB avail=670.4GB (24.7%) +2026/10/25 16:47:21 info.go:13: [DEBUG] [system] readLoadAvg: raw="1.39 2.24 2.12 5/1677 5826" → 1m=1.39 5m=2.24 15m=2.12 +2026/10/25 16:47:21 info.go:13: [DEBUG] [system] readThermalZones: /sys — found 1 zones +2026/10/25 16:47:21 info.go:13: [DEBUG] [system] readTemperature: found via hwmon at /sys — 54.2°C (hwmon2) +2026/10/25 16:47:21 info.go:13: [DEBUG] [system] GetInfo done in 697ms — mem=13189MB/25898MB (50.9%), rootDisk=11.7GB/68.4GB (18.0%), load=1.39/2.24/2.12, temp=54.2°C (hwmon2), cpu=18.4% +2026/10/25 16:47:21 image_retention.go:265: [INFO] [stacks] image retention (remove of tandoor): deleted [postgres:] (79bd7c99e923, 297MB) — no container, installed app or undo names it (decision 53) +2026/10/25 16:47:21 image_retention.go:238: [INFO] [stacks] image retention (remove of tandoor): pass over 45 image(s) of [ghcr.io/tandoorrecipes/recipes postgres] — 2 candidate(s), 2 deleted, the rest kept +2026/10/25 16:47:24 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/tandoor +2026/10/25 16:47:24 router.go:78: [DEBUG] [api] GET /api/stacks/tandoor (path=/stacks/tandoor) +2026/10/25 16:47:28 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:47:28 restore_verdict.go:172: [DEBUG] [backup] stable-start sample tandoor: not ok — the stack does not read running +2026/10/25 16:47:30 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:47:30 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:47:30 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:47:30 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 2m30s ago, effective interval 5m0s, healthy=true +2026/10/25 16:47:30 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 1m40s ago, effective interval 5m0s, healthy=true +2026/10/25 16:47:30 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (2 skipped not due, 0 skipped no container) +2026/10/25 16:47:35 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting +2026/10/25 16:47:38 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:47:38 restore_verdict.go:172: [DEBUG] [backup] stable-start sample tandoor: not ok — the stack does not read running +2026/10/25 16:47:40 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:47:40 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 1m50s ago, effective interval 5m0s, healthy=true +2026/10/25 16:47:40 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 2m40s ago, effective interval 5m0s, healthy=true +2026/10/25 16:47:40 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (2 skipped not due, 0 skipped no container) +2026/10/25 16:47:40 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:47:40 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:47:48 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:47:48 restore_verdict.go:172: [DEBUG] [backup] stable-start sample tandoor: not ok — the stack does not read running +2026/10/25 16:47:50 scheduler.go:85: [DEBUG] [scheduler] job ring-spill: execution starting +2026/10/25 16:47:50 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:47:50 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 2m50s ago, effective interval 5m0s, healthy=true +2026/10/25 16:47:50 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 2m0s ago, effective interval 5m0s, healthy=true +2026/10/25 16:47:50 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:47:50 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (2 skipped not due, 0 skipped no container) +2026/10/25 16:47:50 scheduler.go:85: [DEBUG] [scheduler] job after-unclean-boot: execution starting +2026/10/25 16:47:50 scheduler.go:85: [DEBUG] [scheduler] job deadapp-check: execution starting +2026/10/25 16:47:50 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting +2026/10/25 16:47:50 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:47:58 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:47:58 restore_verdict.go:172: [DEBUG] [backup] stable-start sample tandoor: not ok — the stack does not read running +2026/10/25 16:48:00 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:48:00 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:48:00 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 3m0s ago, effective interval 5m0s, healthy=true +2026/10/25 16:48:00 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 2m10s ago, effective interval 5m0s, healthy=true +2026/10/25 16:48:00 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (2 skipped not due, 0 skipped no container) +2026/10/25 16:48:00 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:48:05 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting +2026/10/25 16:48:08 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:48:08 restore_verdict.go:172: [DEBUG] [backup] stable-start sample tandoor: not ok — the stack does not read running +2026/10/25 16:48:10 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:48:10 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 3m10s ago, effective interval 5m0s, healthy=true +2026/10/25 16:48:10 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 2m20s ago, effective interval 5m0s, healthy=true +2026/10/25 16:48:10 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (2 skipped not due, 0 skipped no container) +2026/10/25 16:48:10 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:48:10 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:48:18 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:48:18 restore_verdict.go:172: [DEBUG] [backup] stable-start sample tandoor: not ok — the stack does not read running +2026/10/25 16:48:20 info.go:13: [DEBUG] [system] GetInfo starting (hddPath="/mnt/felhom-drives/scratch_hdd", hasCPUCollector=true) +2026/10/25 16:48:20 info.go:13: [DEBUG] [system] readMemInfo: guest cap=25898MB (host total was 30714380KB) → used≈13214MB avail≈12684MB +2026/10/25 16:48:20 info.go:13: [DEBUG] [system] readMemInfo: totalKB=30714380 availKB=15042476 → total=25898MB avail=12684MB used=13214MB (51.0%) +2026/10/25 16:48:20 info.go:13: [DEBUG] [system] readDiskUsage: path="/" bsize=4096 total=68.4GB used=11.4GB avail=53.4GB (17.6%) +2026/10/25 16:48:20 info.go:13: [DEBUG] [system] readDiskUsage: path="/mnt/felhom-drives/scratch_hdd" bsize=4096 total=937.8GB used=219.7GB avail=670.4GB (24.7%) +2026/10/25 16:48:20 info.go:13: [DEBUG] [system] readLoadAvg: raw="1.02 1.95 2.03 8/1673 6030" → 1m=1.02 5m=1.95 15m=2.03 +2026/10/25 16:48:20 info.go:13: [DEBUG] [system] readThermalZones: /sys — found 1 zones +2026/10/25 16:48:20 info.go:13: [DEBUG] [system] readTemperature: found via hwmon at /sys — 55.4°C (hwmon2) +2026/10/25 16:48:20 info.go:13: [DEBUG] [system] GetInfo done in 65ms — mem=13214MB/25898MB (51.0%), rootDisk=11.4GB/68.4GB (17.6%), load=1.02/1.95/2.03, temp=55.4°C (hwmon2), cpu=11.2% +2026/10/25 16:48:20 scheduler.go:85: [DEBUG] [scheduler] job deadapp-check: execution starting +2026/10/25 16:48:20 scheduler.go:450: [INFO] [scheduler] Running job: stack-scan +2026/10/25 16:48:20 scheduler.go:85: [DEBUG] [scheduler] job stack-scan: execution starting +2026/10/25 16:48:20 scheduler.go:85: [DEBUG] [scheduler] job ring-spill: execution starting +2026/10/25 16:48:20 scheduler.go:85: [DEBUG] [scheduler] job docker-socket-watch: execution starting +2026/10/25 16:48:20 scheduler.go:85: [DEBUG] [scheduler] job health-probes: execution starting +2026/10/25 16:48:20 scheduler.go:85: [DEBUG] [scheduler] job after-unclean-boot: execution starting +2026/10/25 16:48:20 scheduler.go:85: [DEBUG] [scheduler] job status-refresh: execution starting +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "actualbudget" deployed=false composePath=/opt/docker/stacks/actualbudget/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "adventurelog" deployed=false composePath=/opt/docker/stacks/adventurelog/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "audiobookshelf" deployed=false composePath=/opt/docker/stacks/audiobookshelf/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "bentopdf" deployed=false composePath=/opt/docker/stacks/bentopdf/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "bookstack" deployed=false composePath=/opt/docker/stacks/bookstack/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "calcom" deployed=false composePath=/opt/docker/stacks/calcom/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "calibre-web" deployed=false composePath=/opt/docker/stacks/calibre-web/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "chaoscrash" deployed=false composePath=/opt/docker/stacks/chaoscrash/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "chaosoom" deployed=false composePath=/opt/docker/stacks/chaosoom/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "chaosoomb" deployed=false composePath=/opt/docker/stacks/chaosoomb/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "claper" deployed=false composePath=/opt/docker/stacks/claper/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "code-server" deployed=false composePath=/opt/docker/stacks/code-server/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "crafty-controller" deployed=false composePath=/opt/docker/stacks/crafty-controller/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "dawarich" deployed=false composePath=/opt/docker/stacks/dawarich/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "docmost" deployed=false composePath=/opt/docker/stacks/docmost/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "emby" deployed=false composePath=/opt/docker/stacks/emby/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "filebrowser" deployed=false composePath=/opt/docker/stacks/filebrowser/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "ghost" deployed=false composePath=/opt/docker/stacks/ghost/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "gitea" deployed=false composePath=/opt/docker/stacks/gitea/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "glance" deployed=false composePath=/opt/docker/stacks/glance/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "gokapi" deployed=false composePath=/opt/docker/stacks/gokapi/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "grafana" deployed=false composePath=/opt/docker/stacks/grafana/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "gramps-web" deployed=false composePath=/opt/docker/stacks/gramps-web/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "grimmory" deployed=false composePath=/opt/docker/stacks/grimmory/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "grocy" deployed=false composePath=/opt/docker/stacks/grocy/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "home-assistant" deployed=false composePath=/opt/docker/stacks/home-assistant/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "homebox" deployed=false composePath=/opt/docker/stacks/homebox/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "homepage" deployed=false composePath=/opt/docker/stacks/homepage/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "immich" deployed=true composePath=/opt/docker/stacks/immich/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "jellyfin" deployed=false composePath=/opt/docker/stacks/jellyfin/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "karakeep" deployed=false composePath=/opt/docker/stacks/karakeep/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "kimai" deployed=false composePath=/opt/docker/stacks/kimai/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "komga" deployed=false composePath=/opt/docker/stacks/komga/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "lubelogger" deployed=false composePath=/opt/docker/stacks/lubelogger/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "mealie" deployed=false composePath=/opt/docker/stacks/mealie/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "metube" deployed=false composePath=/opt/docker/stacks/metube/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "n8n" deployed=false composePath=/opt/docker/stacks/n8n/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "navidrome" deployed=false composePath=/opt/docker/stacks/navidrome/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "nextcloud" deployed=false composePath=/opt/docker/stacks/nextcloud/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "nzbget" deployed=false composePath=/opt/docker/stacks/nzbget/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "onlyoffice" deployed=false composePath=/opt/docker/stacks/onlyoffice/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "opengist" deployed=false composePath=/opt/docker/stacks/opengist/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "outline" deployed=false composePath=/opt/docker/stacks/outline/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "paperless-ngx" deployed=true composePath=/opt/docker/stacks/paperless-ngx/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "papra" deployed=false composePath=/opt/docker/stacks/papra/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "plant-it" deployed=false composePath=/opt/docker/stacks/plant-it/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "plex" deployed=false composePath=/opt/docker/stacks/plex/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "privatebin" deployed=true composePath=/opt/docker/stacks/privatebin/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "prowlarr" deployed=false composePath=/opt/docker/stacks/prowlarr/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "qbittorrent" deployed=false composePath=/opt/docker/stacks/qbittorrent/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "radarr" deployed=false composePath=/opt/docker/stacks/radarr/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "radicale" deployed=false composePath=/opt/docker/stacks/radicale/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "rallly" deployed=false composePath=/opt/docker/stacks/rallly/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "recipe-importer" deployed=false composePath=/opt/docker/stacks/recipe-importer/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "romm" deployed=false composePath=/opt/docker/stacks/romm/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "sabnzbd" deployed=false composePath=/opt/docker/stacks/sabnzbd/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "seerr" deployed=false composePath=/opt/docker/stacks/seerr/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "sonarr" deployed=false composePath=/opt/docker/stacks/sonarr/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "sparkyfitness" deployed=false composePath=/opt/docker/stacks/sparkyfitness/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "tandoor" deployed=false composePath=/opt/docker/stacks/tandoor/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "termix" deployed=false composePath=/opt/docker/stacks/termix/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "traefik" deployed=false composePath=/opt/docker/stacks/traefik/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "transmission" deployed=false composePath=/opt/docker/stacks/transmission/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "uptime-kuma" deployed=false composePath=/opt/docker/stacks/uptime-kuma/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "vaultwarden" deployed=false composePath=/opt/docker/stacks/vaultwarden/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "vikunja" deployed=false composePath=/opt/docker/stacks/vikunja/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "wanderer" deployed=false composePath=/opt/docker/stacks/wanderer/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "wger" deployed=false composePath=/opt/docker/stacks/wger/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "wishlist" deployed=false composePath=/opt/docker/stacks/wishlist/docker-compose.yml +2026/10/25 16:48:20 manager.go:676: [DEBUG] [stacks] ScanStacks: found stack "zipline" deployed=false composePath=/opt/docker/stacks/zipline/docker-compose.yml +2026/10/25 16:48:20 manager.go:1960: [DEBUG] [stacks] getCatalogTemplateSlugs: found 65 template slugs in /opt/docker/felhom-controller/data/catalog-cache/templates +2026/10/25 16:48:20 manager.go:757: [DEBUG] [stacks] ScanStacks: catalog has 65 template slugs for orphan detection +2026/10/25 16:48:20 manager.go:788: [INFO] [stacks] ScanStacks complete: 70 stacks found (3 deployed, 67 available) +2026/10/25 16:48:21 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks +2026/10/25 16:48:21 scheduler.go:467: [INFO] [scheduler] Job stack-scan completed (took 129ms) +2026/10/25 16:48:21 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping immich — last check 3m20s ago, effective interval 5m0s, healthy=true +2026/10/25 16:48:21 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping paperless-ngx — last check 2m30s ago, effective interval 5m0s, healthy=true +2026/10/25 16:48:21 healthprobe.go:110: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (2 skipped not due, 0 skipped no container) +2026/10/25 16:48:21 manager.go:839: [INFO] [stacks] Status refresh: 9 containers across 70 stacks diff --git a/documentation/audits/release-2026-10-26/d47-setaside-rc7/_unitprobe2/tandoor/record.json b/documentation/audits/release-2026-10-26/d47-setaside-rc7/_unitprobe2/tandoor/record.json new file mode 100644 index 00000000..d53fe590 --- /dev/null +++ b/documentation/audits/release-2026-10-26/d47-setaside-rc7/_unitprobe2/tandoor/record.json @@ -0,0 +1,34 @@ +{ + "app": "tandoor", + "install": true, + "seed": true, + "backup": { + "code": "200", + "seconds": 75, + "status": "{\"db_dump\": {\"count\": 3, \"duration\": \"1m13.056908559s\", \"last_run\": \"2026-10-25T16:40:16.886510867Z\", \"success\": true}, \"enabled\": true, \"running\": false}" + }, + "reinstall": true, + "set_aside_line": "2026/10/25 16:44:21 unit_owner.go:182: [INFO] [backup] tandoor: the recovery unit of an earlier install (c3b2e33d5bc82152; the install ids differ) is SET ASIDE whole → /mnt/sys_drive/felhom-data/backups/set-aside/tandoor/20261025T164421Z; this install (8c1656d41509efbc) gets its own unit from its first backup (D47)", + "set_aside_dirs": [ + "20261025T164421Z" + ], + "set_aside_byte_identical": true, + "restore_page_offers": true, + "restore": { + "seconds": 95, + "status": "{\"running\": false, \"op\": \"restore\", \"stack\": \"tandoor\", \"started_at\": \"2026-10-25T16:44:33.134197317Z\", \"last\": {\"op\": \"restore\", \"stack\": \"tandoor\", \"ok\": true, \"message\": \"A(z) tandoor: 3 adatkötet és az adatbázis visszaállítva — az alkalmazás újraindult.\", \"finished_at\": \"2026-10-25T16:46:07.928283511Z\"}, \"last_recent\": true}" + }, + "state_after_restore": "running", + "item_after_restore": true, + "db_auth_errors_after_restore": "tandoor: 0\ntandoor-postgres: 0", + "controller_lines": "2026/10/25 16:39:24 recovery_unit.go:297: [INFO] [backup] Recovery unit captured for tandoor → /mnt/sys_drive/felhom-data/backups/primary/tandoor (images=2, secrets-referenced=2, data_keys=0, portable-carried=2/2, withheld=0)\n2026/10/25 16:40:16 recovery_unit.go:297: [INFO] [backup] Recovery unit captured for tandoor → /mnt/sys_drive/felhom-data/backups/primary/tandoor (images=2, secrets-referenced=2, data_keys=0, portable-carried=2/2, withheld=0)\n2026/10/25 16:44:21 unit_owner.go:182: [INFO] [backup] tandoor: the recovery unit of an earlier install (c3b2e33d5bc82152; the install ids differ) is SET ASIDE whole → /mnt/sys_drive/felhom-data/backups/set-aside/tandoor/20261025T164421Z; this install (8c1656d41509efbc) gets its own unit from its \n2026/10/25 16:44:21 recovery_unit.go:297: [INFO] [backup] Recovery unit captured for tandoor → /mnt/sys_drive/felhom-data/backups/primary/tandoor (images=2, secrets-referenced=2, data_keys=0, portable-carried=2/2, withheld=0)\n2026/10/25 16:44:33 kept_copies.go:108: [WARN] [web] Restore of tandoor from a set-aside unit requested (async): /mnt/sys_drive/felhom-data/backups/set-aside/tandoor/20261025T164421Z from 172.18.0.4:53858\n2026/10/25 16:44:33 unit_owner.go:324: [INFO] [backup] restore tandoor from the set-aside unit /mnt/sys_drive/felhom-data/backups/set-aside/tandoor/20261025T164421Z (an earlier install, D47)\n2026/10/25 16:44:33 restore_dbcreds.go:114: [INFO] [backup] Restore tandoor: database password(s) [DB_PASSWORD] come from the backup, [] stay as installed (older than the data) — values not logged (R-1106)\n2026/10/25 16:44:33 restore_unit.go:410: [INFO] [backup] Restoring tandoor from recovery unit /mnt/sys_drive/felhom-data/backups/set-aside/tandoor/20261025T164421Z: images=2, secrets recovered=2/2, data_keys=0\n2026/10/25 16:44:41 offbox_restore_undo.go:169: [INFO] [offbox] tandoor: the live volumes are copied before the restore (3 copied, 0 new from the snapshot) in 4.295s — /mnt/sys_drive/felhom-data/backups/restore-undo/tandoor/20261025T164437Z (R-893)\n2026/10/25 16:44:41 restore_crash_putback.go:89: [INFO] [backup] tandoor: the restore copy is recorded — a power cut from here on puts the app back on start (R-1195)\n2026/10/25 16:44:41 restore.go:225: [INFO] [backup] Restoring Docker volume tandoor_tandoor_media for tandoor\n2026/10/25 16:44:42 restore.go:263: [DEBUG] [backup] Volume tandoor_tandoor_media restored successfully\n2026/10/25 16:44:42 restore.go:225: [INFO] [backup] Restoring Docker volume tandoor_tandoor_postgres_data for tandoor\n2026/10/25 16:44:44 restore.go:263: [DEBUG] [backup] Volume tandoor_tandoor_postgres_data restored successfully\n2026/10/25 16:44:44 restore.go:225: [INFO] [backup] Restoring Docker volume tandoor_tandoor_static for tandoor\n2026/10/25 16:44:47 restore.go:263: [DEBUG] [backup] Volume tandoor_tandoor_static restored successfully\n2026/10/25 16:44:47 restore.go:268: [INFO] [backup] Restored 3 Docker volume(s) for tandoor\n2026/10/25 16:44:47 deploy.go:866: [INFO] [stacks] tandoor: the restored data belongs to install c3b2e33d5bc82152 (was 8c1656d41509efbc) — its backups follow it (D47)\n2026/10/25 16:44:48 restore_db.go:87: [INFO] [backup] Restore tandoor: replaying DB dump into tandoor-postgres (postgres)\n2026/10/25 16:44:52 restore_db.go:97: [INFO] [backup] Restore tandoor: replayed 1 DB dump(s)\n2026/10/25 16:44:56 restore.go:308: [DEBUG] [backup] Post-restore health check: tandoor not yet running, waiting...\n2026/10/25 16:45:01 restore.go:308: [DEBUG] [backup] Post-restore health check: tandoor not yet running, waiting...\n2026/10/25 16:45:07 restore.go:308: [DEBUG] [backup] Post-restore health check: tandoor not yet running, waiting...\n2026/10/25 16:45:12 restore.go:308: [DEBUG] [backup] Post-restore health check: tandoor not yet running, waiting...\n2026/10/25 16:45:17 restore.go:303: [DEBUG] [backup] Post-restore health check: tandoor is running\n2026/10/25 16:46:07 restore_unit.go:591: [INFO] [backup] Restore-from-unit completed: tandoor — 3 volume(s) of 3 listed, 1 database(s) of 1 listed\n", + "after_remove": { + "containers": "", + "volumes": "", + "appyaml": "", + "appdata": "", + "kept": "", + "subdomain": "404" + }, + "set_aside_after_remove": "ls: cannot access '/mnt/sys_drive/felhom-data/backups/set-aside/tandoor': No such file or directory" +} \ No newline at end of file diff --git a/documentation/audits/release-2026-10-26/d47-setaside-rc7/_unitprobe2/tandoor/restore-page-kept-section.html b/documentation/audits/release-2026-10-26/d47-setaside-rc7/_unitprobe2/tandoor/restore-page-kept-section.html new file mode 100644 index 00000000..85e82423 --- /dev/null +++ b/documentation/audits/release-2026-10-26/d47-setaside-rc7/_unitprobe2/tandoor/restore-page-kept-section.html @@ -0,0 +1,20 @@ +Megőrzött másolatok +

Ezek a második meghajtón lévő másolatok olyan mappákhoz tartoznak, amelyek most nem érhetők el: a meghajtó nincs csatlakoztatva, már nincs a gépen, vagy a mappa hiányzik. Megőrizzük őket, amíg te el nem távolítod őket.

+
+ + +
+ +
+ tandoor + a(z) sys_drive meghajtóról megőrzött másolat · egy korábbi telepítés mentése (újratelepítés előtt készült) · 144,3 MB · 2026. 10. 25. 17:44 +
+
+ +
+ + + +