Compare commits
17 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 37f7e40b18 | |||
| 90a2530689 | |||
| 5d07c4e923 | |||
| a3ed63ee9a | |||
| a1c65b3d81 | |||
| 2560cf1779 | |||
| cccb9aee84 | |||
| f24233c829 | |||
| 106325de43 | |||
| b156e4526a | |||
| dfd0f8ec53 | |||
| 1170b48e59 | |||
| 7e15a374b2 | |||
| e333477b11 | |||
| 5bd03ad152 | |||
| 20cf4bccfd | |||
| 5ce54a5076 |
@@ -1,3 +1,97 @@
|
||||
## Unreleased (next: v0.346.0) — D53: the hub learns which apps are held; an unreadable app.yaml keeps its folder (R-1220); the ceremony keeps the off-site key (R-1218, R-1219); safe .fab export/import (R-1187); holds of uninstalled apps are not reported (R-1298); Go 1.27.3 (R-972) — committed, not released
|
||||
|
||||
**MinAgent: 0.158.1** (unchanged). Operator ruling D53 (2026-10-26, felhom.eu `09` §3 decision 265): operator-only
|
||||
reminders first. Needs hub with `app_held` / `app_hold_cleared` in `allowedEventTypes` (felhom.eu, same day) — an older
|
||||
hub answers 400 to both and nothing else changes.
|
||||
|
||||
- `internal/holdreport`: ONE observer of the hold store (`settings.RestoreHolds`), ticked by the scheduler every minute
|
||||
(`hold-report`). `app_held` for a new or changed hold (details `stack_name`, `reason` — the empty legacy reason is sent
|
||||
as `restore_failed` — `since`, `why`), `app_hold_cleared` when a hold is gone; every hold is sent again once a day and
|
||||
at each start (`repeat: true`). The last-reported set is kept in `settings.reported_holds`, so a hold cleared while the
|
||||
controller was down is still reported cleared. Both severity info (the hub mails nothing on receipt).
|
||||
- **R-972: Go 1.27.3** (`go.mod` + the builder image `golang:1.27.3-*` pinned by digest; released 2026-10-19 with a
|
||||
security fix to `crypto/x509`). `govulncheck` v1.8.0: 1 called vulnerability before (GO-2026-6809), 0 after; full
|
||||
`go build/vet/test` green. `golang.org/x/crypto` v0.58.0 is already the newest.
|
||||
- **R-1220 (P2, delete path):** an app whose `app.yaml` EXISTS but cannot be read counts as INSTALLED (fail closed):
|
||||
`stacks.liveDriveBinds(drives)` adds every folder its compose binds on every given drive, so „Megőrzött adatok" never
|
||||
lists — and never offers to delete — that app's live folder; one WARN names the app. Before: a parse error gave a nil
|
||||
`AppConfig`, the app read as not installed, its folder was a „leftover". Test `TestR1220_AnUnreadableAppYamlKeepsTheFolderLive`
|
||||
(with a real leftover as the control), red-proved against the old code.
|
||||
- **R-1218 (P2, key code):** the recovery ceremony stages the off-site repository password whenever the password FILE is
|
||||
on the box (`backup.Manager.OffboxRepoPasswordOnBox`), not only when off-site is configured — a cleared or switched-off
|
||||
target keeps the file, and a ceremony that sealed no password let the hub demote the household's good package (the
|
||||
hub half refuses that demotion: felhom.eu hub, same night). Test `TestR1218_TheCeremonyStagesWhenThePasswordFileExists`
|
||||
(off, cleared, and a box with no password as the control), red-proved („preflight,start" — no stage).
|
||||
- **R-1219 (P2, key code):** during a recovery offer the top bar no longer urges the ceremony, and `POST
|
||||
/api/escrow/start` answers 409 with `api.escrow.recovery_offer_open` (hu + en) before anything is staged or started.
|
||||
Tests `TestR1219_*` (the bar with a real session, and the same box without an offer as the control), red-proved (bar
|
||||
shown, start 200).
|
||||
- **R-1187 (P2, `.fab` export/import) — (a), (b), (d), (f), (g):** (a) the export carries a symlink AS A LINK (never its
|
||||
target's bytes) and stops with `err.appexport.kulonleges_fajl` naming the path on a FIFO, socket or device (a FIFO used
|
||||
to hang it); a drive folder that cannot be packed now FAILS the export (it was left out with a WARN while the export
|
||||
ended „done"). (b) the import keeps owners (`Lchown`) and modes and restores a link that stays inside the app's folder;
|
||||
a link pointing outside is refused and listed in a WARN. (d) `StartImport` refuses (`err.appexport.import_foglalt`)
|
||||
while another operation holds the app-stop marker (`BusyWith`). (f) a bundle whose `app.yaml` carries an empty,
|
||||
relative or unclean `HDD_PATH` is refused before anything is touched (`err.appexport.hdd_path_hibas`). (g) the import
|
||||
classifies databases with the backup side's predicate (`appbackup.DBTypeForImage`: postgis, pgvector, timescaledb) and
|
||||
runs `psql -v ON_ERROR_STOP=1`. Tests `internal/appexport/r1187_abdfg_test.go` (6), red-proved with the old behaviour
|
||||
behind the new function shapes (link packing error, a 10-s hang on the FIFO, link not restored, import not refused,
|
||||
bad paths accepted, postgis missed).
|
||||
**Follow-up (the automated security review, same night):** the import writes only through `os.Root`, so no entry is
|
||||
written through a link already in the folder that points outside; such an entry is refused and listed. Test
|
||||
`TestR1187b_NoWriteThroughALinkThatLeavesTheFolder`, red-proved (the file landed outside with plain `os.Create`).
|
||||
A second review finding: a link entry whose parent path passes through a link („a" → „.", then „a/b/l" → „../..")
|
||||
passed the path-text check while really pointing above the folder — now refused and listed
|
||||
(`TestR1187b_NoLinkThroughALinkedParent`, red-proved). psql takes `--set=ON_ERROR_STOP=1` (the `-v` short form tripped
|
||||
the docker `-v` gate); the four new message keys are listed in `scripts/i18n_go_keys.json` as born keys.
|
||||
A third review finding: a link TARGET through a link already in the folder („x" → „evil/y", evil pointing out). The rule
|
||||
is now strict and simple: a restored link points only DOWNWARD — relative, no „..", no link on its path or in its
|
||||
target; everything else is refused and listed (`safeDownwardLink`; `TestR1187b_NoLinkTargetThroughALink`, red-proved).
|
||||
**Found live on 9202 the same night (grimmory), fixed:** the import ran `compose` without the controller's own path
|
||||
variables (`USERDATA_PATH`, `IMPORT_PATH` — `stacks.policyEnv` adds them at every install and start); compose printed
|
||||
„The USERDATA_PATH variable is not set" and the DB step read that WARNING as the container id, waited 60 s and left the
|
||||
app removed and not started. Now `withImportPathVars` gives the import's compose runs the same derivation (never a
|
||||
bundle's value) and `containerIDFrom` takes only a line that IS an id. Tests `TestR1187_TheContainerIDIsNeverAComposeWarning`,
|
||||
`TestR1187_TheImportEnvCarriesTheControllersPathVars`, red-proved.
|
||||
**And the second cause, found by the same live run:** the import waited with `mysqladmin` and loaded with `mysql` —
|
||||
names the MariaDB 11+ images do not ship (only `mariadb-admin` / `mariadb`; checked in `mariadb:11.4`) — and read the
|
||||
password from two fixed env names, so EVERY `.fab` import of a MariaDB 11+ app timed out and left the app down. The
|
||||
import now loads through the backup side: `appbackup.DiscoverDatabases` (the live container's own credentials) and
|
||||
`appbackup.ImportDump` (mariadb / mariadb-admin; psql `ON_ERROR_STOP` + `--single-transaction`); the import's own
|
||||
`waitForDB` / `importDBDump` are removed. `appbackup.DBTypeForImage` stays the classifier (g). Test
|
||||
`TestR1187_TheImportLoadsTheDumpThroughTheBackupSide` (seams, no Docker), red-proved with the old body.
|
||||
- **R-1298:** the hold report skips an install's hold (`unhealthy_stop`, `update_failed`, `putback_pending`) of an app that
|
||||
is not installed (`holdreport.Reporter.SetInstalled`: the app's `app.yaml` exists — an unreadable one still counts,
|
||||
R-1220) and reports it cleared if it was reported; a restore hold is always reported (R-379). 9202's four orphan holds
|
||||
from a hand removal would otherwise have mailed weekly. Test `TestR1298_OnlyHoldsOfInstalledAppsAreReported`,
|
||||
red-proved; the wiring test pins the `SetInstalled` line.
|
||||
- `settings.RestoreHold.Why()` / `WireReason()`: the hold sentence moved out of `cmd/controller` (`holdWhy` calls it), so
|
||||
`--restore-holds` and the hub's list say the same thing.
|
||||
- Tests `internal/holdreport/holdreport_test.go` (8): the report field, the daily repeat, cleared once, cleared while
|
||||
down, a changed reason, the wire reason, the wiring in main, the record across a reload. Red-proved with an inert
|
||||
`Tick` (all six behaviour tests failed) and with the scheduler line commented out (the wiring test failed).
|
||||
|
||||
## v0.345.1 (2026-10-26) — SECURITY: cloudflared 2026.10.3 (Cloudflare's eMudhra notice) — built from v0.345.0 + one commit (R-1300)
|
||||
|
||||
**MinAgent: 0.158.1** (unchanged). **This entry on `main` records the same change as the tag `v0.345.1`** (the v0.345.0
|
||||
commit `6bfb784` + one commit); the image is built from the TAG, so none of the „Unreleased" work above is in it — the
|
||||
next release from `main` carries the bump by itself.
|
||||
|
||||
- **Security:** `infra.CloudflaredImage` `cloudflare/cloudflared:2026.9.3` → `2026.10.3` (Docker Hub index digest
|
||||
`sha256:faa8e0a1475f137c910be6025f43c8e4092fdd35461b31e19fd1390657a1ff39`). Cloudflare's notice of 2026-10-20:
|
||||
cloudflared before 2026.10.2 trusts eight eMudhra (emSign) intermediate CAs whose private keys were weak and are
|
||||
recoverable — anyone holding them could pose as any website to cloudflared. The fix is the image's trust store.
|
||||
- After the controller update `ensureCloudflared` sees the compose differ (the `image:` line) and runs `compose up -d`,
|
||||
which recreates cloudflared on the new image. Test `TestEnsureCloudflared_ANewImageRecreatesTheTunnel`, red-proved.
|
||||
- Origin check (read only): every tunnel routes `*.<domain>` → `https://traefik` with `noTLSVerify: true` (R-510) and
|
||||
traefik answers with its self-signed default certificate — no eMudhra chain. felhom.eu
|
||||
`documentation/audits/cloudflared-2026.10.3-2026-10-26/`.
|
||||
- **`build.sh` (main only — the operator's yes, 2026-10-26):** a `--push` may also build from the tag `v<VERSION>` AS
|
||||
THE SERVER HAS IT when it points at HEAD — a patch release off a released commit while `main` holds unreleased work.
|
||||
A tag only on the clone, a tag on another commit, a tag of another version, or a server that cannot be asked is a
|
||||
refusal; clean-tree stays required. Four new cases in `scripts/test_build_push_once.sh` (20/20), red-proved both
|
||||
ways (the tag branch accepting anything: 4 fail; removed: 1 fails).
|
||||
|
||||
## v0.345.0 (2026-10-26) — the 50 % rule owns qBittorrent's NORMAL limit only (R-1292) — attended release run (afternoon)
|
||||
|
||||
Live before the release: Tester 1 on `0.345.0-rc.1` with a throwaway qBittorrent, alternative speed ON (the household's
|
||||
|
||||
@@ -1,5 +1,9 @@
|
||||
# CONTEXT.md — Project Memory
|
||||
|
||||
> **2026-10-26 (evening) — D53 ruled (felhom.eu `09` §3 decision 265): held apps reach the OPERATOR first.** Next here:
|
||||
> the controller reports its holds (stack, reason, since) to the hub as `app_held` / `app_hold_cleared` events plus a
|
||||
> daily re-send (felhom.eu `08` §10). Commit only; the release rides a later attended run.
|
||||
|
||||
> **2026-10-25 (Sunday, unattended) — D47, D48, R-1270, R-1271 on `main`, NOT released (next v0.343.0, MinAgent 0.158.1
|
||||
> unchanged).** D47 (`02daebe`): `stacks.AppConfig.InstallID` (minted at install/import, taken from the unit by a
|
||||
> restore, `EnsureInstallID` for older apps); `backup/unit_owner.go` `guardUnitWrite`/`guardUnitDir` in every unit writer
|
||||
|
||||
@@ -0,0 +1,40 @@
|
||||
# REPORT — controller v0.345.1: cloudflared 2026.10.3 (security), 2026-10-26 evening (attended)
|
||||
|
||||
> A sibling of `REPORT.md`, not an overwrite: the night session (D53, R-1218/1219/1220) owns `REPORT.md` tonight and
|
||||
> writes into this repo at the same time.
|
||||
|
||||
## In plain words
|
||||
|
||||
- **What:** Cloudflare said on 2026-10-20 that cloudflared before 2026.10.2 trusts eight bad certificate issuers
|
||||
(eMudhra, weak keys). Every Felhom box ran cloudflared 2026.9.3 in its guest. Controller **0.345.1** = the released
|
||||
0.345.0 + only the new cloudflared pin `2026.10.3`. Nothing from tonight's `main` is in it.
|
||||
- **Did it work:** yes. Every box with a tunnel (Tester 1, demo-hp, demo-felhom) runs controller 0.345.1 and cloudflared
|
||||
2026.10.3, healthy; the hub says „running (connected)" for all three; all 29 public names still redirect http → https.
|
||||
Tester4 moved to 0.345.1 by the global floor (it has no tunnel). Tester 2 has no box.
|
||||
- **Gaps:** the tunnel was down about 2 s (Tester 1), 4 s (demo-felhom) and 11 s (demo-hp) while cloudflared restarted.
|
||||
|
||||
## What changed
|
||||
|
||||
| Where | What |
|
||||
|---|---|
|
||||
| tag `v0.345.1` = `f3204024` (on `6bfb784`) | `infra.CloudflaredImage` → `cloudflare/cloudflared:2026.10.3`; test `TestEnsureCloudflared_ANewImageRecreatesTheTunnel`; CHANGELOG entry |
|
||||
| `main` `a1c65b3d` | the same change, plus `build.sh`: a `--push` may build from the pushed tag `v<VERSION>` when it is HEAD (operator's yes, this session); 4 new cases in `scripts/test_build_push_once.sh` |
|
||||
|
||||
- Red-proofs: the new reconcile test failed with the „compose changed" check removed; the build cases failed both ways
|
||||
(tag branch accepting anything: 4 of 20 fail; tag branch removed: 1 of 20 fails).
|
||||
- Gates `--fast` green on both commits; `go test ./...` green on the tag commit (`internal/stacks` 450 s under the night
|
||||
session's load). CI by commit: `a1c65b3d` job 2392 success; `f3204024` job 2393 success (run_attempt 1 each).
|
||||
- Image `gitea.dooplex.hu/admin/felhom-controller:0.345.1` (and `:latest`) digest
|
||||
`sha256:7cfdf7e1f835628de9c7d536e0a60c06bad5bbe19e8133765cc0a6bb732dbff6`, built from `f3204024`.
|
||||
- cloudflared image index digest `sha256:faa8e0a1475f137c910be6025f43c8e4092fdd35461b31e19fd1390657a1ff39`.
|
||||
- How a box takes it: `controller/internal/stacks/infra.go` `ensureCloudflared` — the rendered compose differs (the
|
||||
`image:` line) → `compose up -d` recreates the container. Seen on all three boxes: `[infra] cloudflared compose
|
||||
changed … recreating cloudflared` 1–2 s after `[selfupdate] Post-update startup: update successful (0.345.0 → 0.345.1)`.
|
||||
|
||||
## For the night session
|
||||
|
||||
`main` carries the bump (`a1c65b3d`). Your next release from `main` keeps cloudflared 2026.10.3 — no action needed.
|
||||
Your CHANGELOG „Unreleased" block is unchanged; the v0.345.1 section sits between it and v0.345.0.
|
||||
|
||||
Evidence: felhom.eu `documentation/audits/cloudflared-2026.10.3-2026-10-26/`. Register: R-1300 (closed), R-1301
|
||||
(opened). Full session report: felhom.eu `REPORT-cloudflared-2026-10-26.md`.
|
||||
@@ -1,30 +1,24 @@
|
||||
# REPORT — controller v0.344.0 released (2026-10-26, attended release run)
|
||||
# REPORT — D53: the hub learns which apps are held — committed, NOT released (2026-10-26 evening)
|
||||
|
||||
**Plain words first.** Every box runs 0.344.0. The global floor was set at 09:07:47 UTC (MinAgent 0.158.1) on the
|
||||
operator's yes, after the three test boxes moved through their per-box floors and the release-build checks passed. New
|
||||
installs start on it too: golden 0.344.0 was baked and vouched the same day, and a fresh install came up on 0.344.0
|
||||
with no update step. The file manager restarted once at the update, as the release notes say.
|
||||
**Plain words first.** A held app (stopped on purpose until a person acts) now reaches the hub: the box sends one
|
||||
message when a hold appears or changes, one when it goes, and repeats every hold once a day. The hub shows the list
|
||||
and mails the operator after a waiting time (felhom.eu `7e57e824`). Not released (this run allows no release).
|
||||
|
||||
## Release
|
||||
## The change (`5bd03ad`, CI 2377 ✓)
|
||||
|
||||
- Image `felhom-controller:0.344.0`, sha256 `9435165dab85b1200d94157ec49d8be065889139e1831a5476dc68f4b9474b8a`, built
|
||||
from `0a59044` (CI 2355 ✓). The rc `0.344.0-rc.4` (same commit) ran on 9202 first; 9202 is back on 0.343.0.
|
||||
- Per-box floors 0.344.0 for Tester 1, demo-hp, demo-felhom (08:15 UTC; each moved by the normal update path, healthy),
|
||||
then the global floor (read back from the hub DB with its WAL and from the hub log), then the per-box floors cleared.
|
||||
- Fleet after the floor: demo-felhom, demo-hp, tester-1 report 0.344.0; Tester-2 has no box since 2026-10-04 (R-862).
|
||||
- `internal/holdreport`: one observer of the hold store, ticked every minute by the scheduler (`hold-report`). It sends
|
||||
`app_held` (stack, reason, since, the hold's sentence) for a new or changed hold, `app_hold_cleared` when a hold is
|
||||
gone — also one cleared while the controller was down (`settings.reported_holds`) — and every hold again daily and at
|
||||
start. Severity info.
|
||||
- `settings.RestoreHold.Why()` / `WireReason()`: the hold sentence lives in one place (`holdWhy` calls it); the legacy
|
||||
empty reason is sent as `restore_failed`.
|
||||
- Tests `internal/holdreport/holdreport_test.go` (8). Red-proved: an inert `Tick` failed all six behaviour tests; the
|
||||
scheduler line commented out failed the wiring test.
|
||||
|
||||
## Checks on the release build (felhom.eu `documentation/audits/release-monday-2026-10-26/`)
|
||||
## Live proof (felhom.eu `documentation/audits/d53-held-apps-2026-10-26/`)
|
||||
|
||||
| Row | Check | Result |
|
||||
|---|---|---|
|
||||
| R-1281 | FileBrowser's networks; a forged header from apps; the household's files; OnlyOffice linked | on all three boxes FileBrowser only on `felhom-filebrowser` + `felhom-files`; qBittorrent/Sonarr/Radarr/Prowlarr → 000 (control: controller 404, traefik 301); Tester 1: dashboard → pass → cookie → listing 200, no cookie 401; OnlyOffice joined `felhom-files`, editor config 200, the OnlyOffice container fetched the document 200 — closed |
|
||||
| R-1282 | „Később", then a controller restart | the stored 03:30 kept (start-up line + settings.json) — closed |
|
||||
| R-1278 | install page with a kept backup | hint + link shown; none before or after — closed |
|
||||
| R-1286 | Kimai's install id; the hold sentences | id written 07:01:55Z; `holdWhy` released — closed |
|
||||
| R-1288 | released | proven live on the night's rig — closed |
|
||||
| R-1261 | released | the live line test is Tuesday (R-1250's dated check) — stays VERIFY |
|
||||
| D47 | upgrade count | 0 set aside on all three boxes; every app has an install id |
|
||||
|
||||
**Observation (NOT-A-FINDING):** on 9202's rc start the first FileBrowser round logged a WARN „could not join felhom-files
|
||||
(files.<domain> stays unreachable)" because FileBrowser was being re-created in the same second; it joined 2 s later.
|
||||
Log only, no event; the three release boxes logged no WARN.
|
||||
9202 ran a local-only image of this commit (`0.346.0-d53test`, never pushed to the registry) against a test hub process.
|
||||
A throwaway Radicale, crashed seven times, was stopped and held by the box (17:37:52Z); `app_held` reached the hub at the
|
||||
next tick; one operator reminder after the 3-minute test grace; the household's Start lifted the hold and
|
||||
`app_hold_cleared` removed the row. The first tick also reported four holds 9202 has carried since 2026-09-24 for apps
|
||||
that are not installed (felhom.eu R-1298). 9202 is back on 0.343.0 with its own config.
|
||||
|
||||
@@ -84,6 +84,7 @@
|
||||
| `Manager.putBackAfterFailedRestore` + `liveVolumeCopy` (R-893 option A, v0.336.0) | controller/internal/backup/offbox_restore_undo.go | `(stack, cause, live, copy, volumesMoved, filesPlaced, restart, holdKey) error` | THE restore undo — off-site (v0.336.0) and, since D22 (unreleased), the LOCAL unit restore (`RestoreFromRecoveryUnitAtWith`, so also the second drive's and a kept load): the live volumes it will replace, measured before the stop (`planLiveVolumeCopy`), copied right after it (`copyLiveVolumes`), poured back on any failure | One copy per restore under `<ns>/backups/restore-undo/<app>/` (no capture set reads `backups/`); the put-back uses the restore's own `volumeReplayFrom`/`restoreDockerVolumesFrom`, so there is one volume-replay implementation. A put-back that fails sets `keep` and HOLDS (option C) — never start on volumes that are not known to be back. Seams for tests without Docker: `undoVolumeProbeFn`, `undoVolumeTarFn`, `undoVolumeRemoveFn`; operator notice `SetRestorePutBackNotify` (leg `restore-put-back`). Pinned by `r893_putback_test.go` and `r1177_local_putback_test.go` |
|
||||
| `CrashPutBack` + `Manager.persistUndoForCrash` + `AppStopGuard.SetUndo` (D22 / R-1195, v0.338.0) | controller/internal/backup/restore_crash_putback.go | `NewCrashPutBack(stop, writeDef, logger)`; `PutBackInterruptedRestore(stack, undoDir) error` | A restore cut by a power loss is put back at start: the copy (volumes + `definition/` + `undo.json`) is recorded in the app-stop marker once whole; `Recover` pours it back BEFORE the restart | Call `persistUndoForCrash` right after `copyLiveVolumes` in any new restore path. `Recover` runs before the backup manager exists, so the put-back is wired in main.go with `stackMgr.StopStack` and the stack adapter's definition writer. A failed put-back = a FAILED recovery (app not started, marker kept). The household reads `note.restore.interrupted_put_back` (`LoadRestoreRecordAfterRecovery`). Pinned by `TestR1195_*` |
|
||||
| `tier2CaptureSetFull(...) (legs, missing, warns, classified)` + `keptCopyRels` + `dropKeptCopiesUnder` (R-1252, unreleased) | controller/internal/backup/tier2_capture.go, tier2_kept.go | `classified` = the legs come from a VALID backup block | **The second-drive reconcile deletes only under a valid classification**; a rejected or absent block keeps every dest dir; kept copies (D25) are always in the keep set; a mirror deleted by Remove drops its kept records | A new reconcile caller must take `classified` and the kept rels too — never reconcile against the legacy legs. A rejected block is told by `Manager.BackupBlockRejected` (stacks; via the adapter's optional `BackupBlockRejected` method) and signalled once a day (`SetTier2SettingsUnreadableNotify`). Pinned by `r1252_rejected_block_keeps_copies_test.go` |
|
||||
| `holdreport.Reporter` + `settings.RestoreHold.Why` / `WireReason` (D53, unreleased) | controller/internal/holdreport/holdreport.go · controller/internal/settings/settings.go | `New(store, notifier.PushEvent, logger)`; `Tick(now)` from the scheduler | Tells the hub every hold in place (`app_held`), every hold gone (`app_hold_cleared`), each again daily | **One observer of the hold store — never a send inside a hold writer.** A new hold writer needs nothing here. The hold sentence is `RestoreHold.Why()` (one per reason) — never a second copy. Pinned by `internal/holdreport/holdreport_test.go` |
|
||||
| `AppStopGuard.ResumePutBack` / `ResumeWaitingPutBacks` + `HoldReasonPutBackPending` (R-1251, unreleased) | controller/internal/backup/appstop_putback_wait.go | `ResumePutBack(stack) bool` (true = nothing owed, the caller may start); `ResumeWaitingPutBacks() []string` (puts back, then starts through the gated starter) | A cut restore's copy on a drive that is not bound is WAITING: Recover writes the restore hold `putback_pending` (copy in `SafetyDump`) and starts nothing; the put-back runs when the drive binds, before any start | **Any new path that starts an app when its drive arrives calls `ResumePutBack` first** (the boot recreate and `restartStacks` do, in web via `Server.resumePutBack`). Every other start path is covered by the hold store it already honours — do not add a second check. A failed put-back (at start or on resume) writes the D20 hold (`restore_mixed`) and drops the app from the marker. Needs `SetHolds` + `SetUndoReachable` before `Recover` (main.go). Pinned by `r1251_putback_waits_test.go`, `TestR1251_DriveStartPathsPutBackBeforeTheyStart`, `TestR1251_MainWiresThePutBackWait` |
|
||||
| `Manager.RestoreTier2Unit` + `Tier2Coverage.CanRestoreUnit` / `Tier2CopyDate` (R-102/R-103, v0.229.0) | controller/internal/backup/tier2_restore.go | `(stack string) (UnitRestoreResult, error)` / `() bool` / `() (string, bool)` | THE full restore from the SECOND DRIVE's mirror, and the predicate that gates it | **Do NOT widen `CanRestore()`** — it answers only "can the additive file restore run?" and one predicate answering two questions is R-356. Fail-closed: `tier2UnitIsOpenable` requires a parseable manifest, because a directory is not a package. The single-writer flag is taken INSIDE `RestoreFromRecoveryUnitAt` — a second `acquireRunning()` here would refuse the restore it guards. `Tier2CopyDate` prefers the SUCCESS anchor over the attempt clock (R-101) and reports which it returned |
|
||||
| `unitRestoreOutcomeMsg` + its three message constants (R-353, v0.226.0) | controller/internal/web/handlers.go | `(app string, res backup.UnitRestoreResult) string` | THE customer sentence for a completed LOCAL restore | Twin of `reconstituteOutcomeMsg`; copy its SHAPE (clauses earned by having done the thing, no filesystem path, base names only), never its text. The constants are named because `r353_unit_outcome_test.go` asserts them verbatim — a silent edit is how an honest message drifts back into a comforting one, which is the documented history of the sentence it replaces |
|
||||
|
||||
@@ -7,7 +7,7 @@
|
||||
# --- Build stage ---
|
||||
# R-972 (v0.322.0): a supported, patched Go, pinned by version AND digest (no moving tag). Go 1.24 had no security fixes
|
||||
# left (go.dev/doc/devel/release#policy). The builder's Debian does not reach the runtime (a static CGO-free binary).
|
||||
FROM golang:1.27.2-trixie@sha256:47667934874913a449b96faa7ef4777fa285d9ec29126ee80ed61c45b7cccf0e AS builder
|
||||
FROM golang:1.27.3-trixie@sha256:fbb7ae40b62ff8afe31662df570d39b4f5052ac32b90dae8e60d7657a20a74f4 AS builder
|
||||
|
||||
ARG TARGETOS=linux
|
||||
ARG TARGETARCH
|
||||
|
||||
@@ -2283,6 +2283,10 @@ that folder is never a dead end, and an install never runs into it silently (R-6
|
||||
- **R-1286 (v0.344.0).** A held app says WHY: each hold reason (a failed restore, an unhealthy stop by the crash-loop
|
||||
guard, …) has its own sentence in the start gate, the boot sweep and `--restore-holds` (an unhealthy stop used to
|
||||
read as a failed restore).
|
||||
- **D53 (next release, felhom.eu `08` §10).** The hub learns which apps are held: `internal/holdreport` reads the hold
|
||||
store every minute and sends `app_held` (stack, reason, since, the hold's sentence) for a new or changed hold and
|
||||
`app_hold_cleared` when one is gone — also one cleared while the controller was down (`settings.reported_holds`) — and
|
||||
repeats every hold daily and at start. Both are severity info; the hub's operator reminder after a grace is the mail.
|
||||
- **R-690 (fixed here).** The removed-app restore (R-487) never found a unit on a DATA drive — it asked
|
||||
`GetStackComposePath`, true for every catalog app — and restored with no env. It now asks `isStackDeployed`.
|
||||
- `.felhom.yml` **`after_load: {service, user, command: [...]}`** — one command run with `docker compose exec -T`
|
||||
|
||||
+19
-6
@@ -153,14 +153,27 @@ if [[ "${ACTION}" == "--push" || "${ACTION}" == "--multiarch" ]]; then
|
||||
fi
|
||||
head_sha=$(cd "${REPO_DIR}" && git rev-parse HEAD 2>/dev/null || echo unknown)
|
||||
main_sha=$(cd "${REPO_DIR}" && git rev-parse origin/main 2>/dev/null || echo unknown)
|
||||
# The one other source that can be fetched again: the tag v<VERSION> AS THE SERVER HAS IT, pointing at HEAD. That
|
||||
# is a patch release built off a released version while main holds unreleased work (v0.345.1, R-1300, cloudflared's
|
||||
# security bump, 2026-10-26 — operator's yes in that session). Only the tag that names THIS version counts; a tag
|
||||
# only on this clone, a tag on another commit, or a server that cannot be asked is a refusal. Pinned by
|
||||
# scripts/test_build_push_once.sh („tag" cases).
|
||||
if [[ "${head_sha}" != "${main_sha}" ]]; then
|
||||
error "${REPO_DIR} is not at origin/main — refusing to push."
|
||||
error " HEAD ${head_sha}"
|
||||
error " origin/main ${main_sha}"
|
||||
error "An unpushed change does not exist; an image built from one cannot be fetched again."
|
||||
exit 1
|
||||
tag_sha=$(cd "${REPO_DIR}" && GIT_TERMINAL_PROMPT=0 git ls-remote origin "refs/tags/v${VERSION}" "refs/tags/v${VERSION}^{}" 2>/dev/null \
|
||||
| awk '$2 ~ /\^\{\}$/ {peeled=$1} $2 !~ /\^\{\}$/ {plain=$1} END {print (peeled != "" ? peeled : plain)}') || tag_sha="" # no origin / unreachable: the error below, not set -e
|
||||
if [[ -n "${tag_sha}" && "${tag_sha}" == "${head_sha}" ]]; then
|
||||
info "Source tree is clean and at the pushed tag v${VERSION} (not origin/main)."
|
||||
else
|
||||
error "${REPO_DIR} is not at origin/main, nor at a pushed tag v${VERSION} — refusing to push."
|
||||
error " HEAD ${head_sha}"
|
||||
error " origin/main ${main_sha}"
|
||||
error " v${VERSION} on origin ${tag_sha:-<none>}"
|
||||
error "An unpushed change does not exist; an image built from one cannot be fetched again."
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
info "Source tree is clean and at origin/main."
|
||||
fi
|
||||
info "Source tree is clean and at origin/main."
|
||||
else
|
||||
error "${REPO_DIR} is not a git checkout — refusing to push a build whose source cannot be identified."
|
||||
exit 1
|
||||
|
||||
@@ -40,6 +40,7 @@ import (
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/crashboot"
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/crypto"
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/fillwatch"
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/holdreport"
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/infra"
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/integrations"
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/mailrelay"
|
||||
@@ -1063,6 +1064,20 @@ func main() {
|
||||
go func() { defer aubBusy.Unlock(); aubRunner.Tick(time.Now()) }() // a run can take minutes; never block the scheduler
|
||||
return nil
|
||||
})
|
||||
// D53 (felhom.eu `08` §10): tell the hub which apps are held — app_held for a new or changed hold, app_hold_cleared
|
||||
// when one is gone, every hold again once a day and after each start. One observer of the hold store, so every
|
||||
// writer is covered. Pinned by internal/holdreport.
|
||||
holdReporter := holdreport.New(sett, notifier.PushEvent, logger)
|
||||
// R-1298: „installed" = the app's app.yaml file exists (a product removal deletes it; an unreadable one still counts,
|
||||
// the R-1220 rule). An install's hold of an app without one is not reported.
|
||||
holdReporter.SetInstalled(func(stack string) bool {
|
||||
_, err := os.Stat(filepath.Join(cfg.Paths.StacksDir, stack, "app.yaml"))
|
||||
return err == nil
|
||||
})
|
||||
sched.Every("hold-report", time.Minute, func(ctx context.Context) error {
|
||||
holdReporter.Tick(time.Now())
|
||||
return nil
|
||||
})
|
||||
sched.Every("deadapp-check", 30*time.Second, func(ctx context.Context) error {
|
||||
if bootGrace.Within(time.Now()) {
|
||||
return nil // still inside the startup settle window (90 s, or ~15 min after a crash boot)
|
||||
@@ -2562,22 +2577,7 @@ type driveStartGate struct {
|
||||
// stop read as „held after a failed restore whose rollback also failed" for 11 days on demo-hp, so the operator could
|
||||
// not tell a crash loop from a broken restore). Pinned by TestR1286_EveryHoldReasonSaysWhatItIs.
|
||||
func holdWhy(h settings.RestoreHold) string {
|
||||
switch h.Reason {
|
||||
case settings.HoldReasonUpdateFailed:
|
||||
return "held after a failed update (" + h.At + ") — restore it from its backup to start it"
|
||||
case settings.HoldReasonUnhealthyStop:
|
||||
kind := h.UnhealthyKind
|
||||
if kind == "" {
|
||||
kind = "unhealthy"
|
||||
}
|
||||
return "stopped by the box for a " + strings.ReplaceAll(kind, "_", " ") + " (" + h.At + ") — the household's Start gives it one more try"
|
||||
case settings.HoldReasonRestoreMixed:
|
||||
return "held after an off-site restore failed its database replay (" + h.At + ") — clear the hold to start it"
|
||||
case settings.HoldReasonPutBackPending:
|
||||
return "held until the pre-restore copy is poured back (" + h.At + ") — the put-back starts it"
|
||||
default:
|
||||
return "held after a failed restore whose rollback also failed (" + h.At + ") — clear the hold to start it"
|
||||
}
|
||||
return h.Why() // one sentence per reason, shared with the hub's „Held apps" list (D53)
|
||||
}
|
||||
|
||||
func (g driveStartGate) MayStart(stackName string) (bool, string) {
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
module gitea.dooplex.hu/admin/felhom-controller
|
||||
|
||||
go 1.27.2
|
||||
go 1.27.3
|
||||
|
||||
require (
|
||||
github.com/emersion/go-sasl v0.0.0-20241020182733-b788ff22d5a6
|
||||
|
||||
@@ -42,6 +42,9 @@ func dbTypeForImage(image string) (DBType, bool) {
|
||||
return DBType(""), false
|
||||
}
|
||||
|
||||
// DBTypeForImage is dbTypeForImage for other packages (R-1187 g: the .fab import classifies with the SAME predicate).
|
||||
func DBTypeForImage(image string) (DBType, bool) { return dbTypeForImage(image) }
|
||||
|
||||
// composeServicesDoc is the minimal view of a compose file needed here: the `services:` MAP and each
|
||||
// service's `image:`. Deliberately a real YAML parse and not a line scan — a top-level `volumes:`
|
||||
// block (immich's `immich_ml_cache:`) has exactly the shape a naive scan misreads as a service, and
|
||||
|
||||
@@ -728,10 +728,11 @@ func (e *Exporter) exportHDDData(req ExportRequest, dataDir string, manifest *Ma
|
||||
e.debugf("tarring HDD mount: %s → %s (%d exclude(s))", mount, tarPath, len(excludes))
|
||||
tarStart := time.Now()
|
||||
if err := tarDirectoryExcluding(mount, tarPath, excludes); err != nil {
|
||||
// v0.125.0: claim the subdir ONLY on success — a claimed-but-absent tar would trip
|
||||
// the packaging assertion; an honestly-skipped mount stays out of the manifest.
|
||||
e.logger.Printf("[WARN] Export: failed to tar %s (excluded from the bundle): %v", mount, err)
|
||||
// R-1187 (a): a folder that cannot be packed FAILS the export. It used to be left out with a WARN while the
|
||||
// export ended „done" — a bundle silently missing a whole drive folder. Pinned by TestR1187a_*.
|
||||
e.logger.Printf("[WARN] Export: failed to pack %s — the export FAILS rather than leave it out (R-1187 a): %v", mount, err)
|
||||
os.Remove(tarPath)
|
||||
return err
|
||||
} else {
|
||||
manifest.HDDSubdirs = append(manifest.HDDSubdirs, subdir)
|
||||
if info, _ := os.Stat(tarPath); info != nil {
|
||||
|
||||
@@ -2,6 +2,7 @@ package appexport
|
||||
|
||||
import (
|
||||
"archive/tar"
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/util"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
@@ -289,7 +290,20 @@ func tarDirectoryExcluding(sourceDir, outputPath string, excludeRels []string) e
|
||||
return nil
|
||||
}
|
||||
}
|
||||
header, err := tar.FileInfoHeader(info, "")
|
||||
// R-1187 (a): a symlink is carried AS A LINK (its target's bytes never — the walk used to copy them into the link
|
||||
// header, the tar failed, and the whole folder was left out); a FIFO, socket or device stops the export with a
|
||||
// sentence naming the path (os.Open on a FIFO used to HANG the export). Pinned by TestR1187a_*.
|
||||
mode := info.Mode()
|
||||
if mode&(os.ModeNamedPipe|os.ModeSocket|os.ModeDevice|os.ModeCharDevice|os.ModeIrregular) != 0 {
|
||||
return util.MsgError("err.appexport.kulonleges_fajl", path)
|
||||
}
|
||||
link := ""
|
||||
if mode&os.ModeSymlink != 0 {
|
||||
if link, err = os.Readlink(path); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
header, err := tar.FileInfoHeader(info, link)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -297,7 +311,7 @@ func tarDirectoryExcluding(sourceDir, outputPath string, excludeRels []string) e
|
||||
if err := tw.WriteHeader(header); err != nil {
|
||||
return err
|
||||
}
|
||||
if info.IsDir() {
|
||||
if info.IsDir() || link != "" {
|
||||
return nil
|
||||
}
|
||||
f, err := os.Open(path)
|
||||
|
||||
@@ -0,0 +1,274 @@
|
||||
package appexport
|
||||
|
||||
import (
|
||||
"archive/tar"
|
||||
"io"
|
||||
"log"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
"syscall"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// R-1187 (a), (b), (d), (f), (g) — the open parts of the `.fab` data-loss row. Each test asserts the CONSEQUENCE for
|
||||
// the household's files. Red-proved 2026-10-26 night against the code before the fix (function shapes added first with
|
||||
// the old behaviour, so every failure below was a behaviour failure, not a compile error).
|
||||
|
||||
func r1187TarEntries(t *testing.T, path string) map[string]*tar.Header {
|
||||
t.Helper()
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer f.Close()
|
||||
out := map[string]*tar.Header{}
|
||||
tr := tar.NewReader(f)
|
||||
for {
|
||||
h, err := tr.Next()
|
||||
if err == io.EOF {
|
||||
return out
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatalf("reading %s: %v", path, err)
|
||||
}
|
||||
out[h.Name] = h
|
||||
}
|
||||
}
|
||||
|
||||
// (a) A symlink in a drive folder is carried AS A LINK — before, the walk wrote a link header and then copied the
|
||||
// target's bytes into it, the tar failed, and the whole folder was left out with only a WARN (the export ended „done").
|
||||
func TestR1187a_ASymlinkIsCarriedAsALink(t *testing.T) {
|
||||
src := t.TempDir()
|
||||
os.WriteFile(filepath.Join(src, "photo.jpg"), []byte("jpeg"), 0o644)
|
||||
os.Symlink("photo.jpg", filepath.Join(src, "latest.jpg"))
|
||||
os.Symlink("/etc/hostname", filepath.Join(src, "outside"))
|
||||
out := filepath.Join(t.TempDir(), "x.tar")
|
||||
if err := tarDirectoryExcluding(src, out, nil); err != nil {
|
||||
t.Fatalf("a folder with symlinks could not be packed: %v", err)
|
||||
}
|
||||
e := r1187TarEntries(t, out)
|
||||
if h := e["latest.jpg"]; h == nil || h.Typeflag != tar.TypeSymlink || h.Linkname != "photo.jpg" {
|
||||
t.Fatalf("the inside link is not carried as a link: %+v", h)
|
||||
}
|
||||
if h := e["outside"]; h == nil || h.Typeflag != tar.TypeSymlink || h.Size != 0 {
|
||||
t.Fatalf("the outside link must be carried as a bare link (never its target's bytes): %+v", h)
|
||||
}
|
||||
if h := e["photo.jpg"]; h == nil || h.Typeflag != tar.TypeReg {
|
||||
t.Fatalf("the regular file is missing: %+v", h)
|
||||
}
|
||||
}
|
||||
|
||||
// (a) A FIFO stops the export with a sentence naming the path — never a hang on os.Open, never a silent drop. The
|
||||
// export of the folder FAILS as a whole (exportHDDData returns the error; the job then fails).
|
||||
func TestR1187a_AFifoFailsTheExportNamingThePath(t *testing.T) {
|
||||
mount := filepath.Join(t.TempDir(), "appdata-x")
|
||||
os.MkdirAll(mount, 0o755)
|
||||
os.WriteFile(filepath.Join(mount, "keep.txt"), []byte("k"), 0o644)
|
||||
fifo := filepath.Join(mount, "pipe")
|
||||
if err := syscall.Mkfifo(fifo, 0o644); err != nil {
|
||||
t.Skipf("no FIFO here: %v", err)
|
||||
}
|
||||
prov := &r1187HDDProvider{rtProvider: rtProvider{stackDir: t.TempDir()}, mounts: []string{mount}}
|
||||
e := NewExporter(prov, log.New(io.Discard, "", 0), "test")
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- e.exportHDDData(ExportRequest{StackName: "x"}, t.TempDir(), &Manifest{}) }()
|
||||
select {
|
||||
case err := <-done:
|
||||
if err == nil || !strings.Contains(err.Error(), fifo) {
|
||||
t.Fatalf("a FIFO must fail the export naming %s, got %v", fifo, err)
|
||||
}
|
||||
case <-time.After(10 * time.Second):
|
||||
t.Fatal("the export HUNG on a FIFO")
|
||||
}
|
||||
}
|
||||
|
||||
type r1187HDDProvider struct {
|
||||
rtProvider
|
||||
mounts []string
|
||||
}
|
||||
|
||||
func (p *r1187HDDProvider) GetStackHDDMounts(string) []string { return p.mounts }
|
||||
|
||||
// (b) The import keeps owners and modes, restores a link that stays inside the app's folder, and refuses (and lists) a
|
||||
// link that points outside it. Before: only folders and regular files were written — owners lost, every link dropped.
|
||||
func TestR1187b_TheImportKeepsOwnersModesAndInsideLinks(t *testing.T) {
|
||||
tarPath := filepath.Join(t.TempDir(), "d.tar")
|
||||
f, _ := os.Create(tarPath)
|
||||
tw := tar.NewWriter(f)
|
||||
tw.WriteHeader(&tar.Header{Name: "sub/", Typeflag: tar.TypeDir, Mode: 0o750, Uid: 1000, Gid: 1000})
|
||||
tw.WriteHeader(&tar.Header{Name: "sub/a.txt", Typeflag: tar.TypeReg, Mode: 0o640, Size: 2, Uid: 1000, Gid: 1001})
|
||||
tw.Write([]byte("hi"))
|
||||
tw.WriteHeader(&tar.Header{Name: "link-in", Typeflag: tar.TypeSymlink, Linkname: "sub/a.txt", Uid: 1000, Gid: 1000})
|
||||
tw.WriteHeader(&tar.Header{Name: "link-out", Typeflag: tar.TypeSymlink, Linkname: "../../etc/passwd"})
|
||||
tw.WriteHeader(&tar.Header{Name: "link-abs", Typeflag: tar.TypeSymlink, Linkname: "/etc/passwd"})
|
||||
tw.Close()
|
||||
f.Close()
|
||||
|
||||
var chowned []string
|
||||
old := lchownFn
|
||||
lchownFn = func(_ *os.Root, p string, uid, gid int) error {
|
||||
chowned = append(chowned, filepath.Base(p)+":"+strconv.Itoa(uid)+":"+strconv.Itoa(gid))
|
||||
return nil
|
||||
}
|
||||
defer func() { lchownFn = old }()
|
||||
|
||||
dest := t.TempDir()
|
||||
refused, err := extractTar(tarPath, dest)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if fi, err := os.Stat(filepath.Join(dest, "sub/a.txt")); err != nil || fi.Mode().Perm() != 0o640 {
|
||||
t.Fatalf("mode not kept: %v %v", fi, err)
|
||||
}
|
||||
if l, err := os.Readlink(filepath.Join(dest, "link-in")); err != nil || l != "sub/a.txt" {
|
||||
t.Fatalf("the inside link was not restored: %q %v", l, err)
|
||||
}
|
||||
for _, bad := range []string{"link-out", "link-abs"} {
|
||||
if _, err := os.Lstat(filepath.Join(dest, bad)); err == nil {
|
||||
t.Fatalf("a link pointing OUTSIDE the app's folder was restored: %s", bad)
|
||||
}
|
||||
}
|
||||
if strings.Join(refused, ",") != "link-out,link-abs" {
|
||||
t.Fatalf("the refused links must be listed, got %v", refused)
|
||||
}
|
||||
got := strings.Join(chowned, ",")
|
||||
for _, want := range []string{"sub:1000:1000", "a.txt:1000:1001", "link-in:1000:1000"} {
|
||||
if !strings.Contains(got, want) {
|
||||
t.Fatalf("owner %s not restored (got %s)", want, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// (d) An import is refused while another operation holds the app-stop marker (a restore, a put-back, a stop-first
|
||||
// export) — before, StartImport checked only its own job slot.
|
||||
func TestR1187d_AnImportWaitsForARestore(t *testing.T) {
|
||||
e := NewExporter(&rtProvider{stackDir: t.TempDir()}, log.New(io.Discard, "", 0), "test")
|
||||
e.SetStopGuard(&busyGuard{})
|
||||
fab := filepath.Join(t.TempDir(), "x.fab")
|
||||
os.WriteFile(fab, []byte("x"), 0o644)
|
||||
err := e.StartImport(ImportRequest{FABPath: fab})
|
||||
if err == nil || !strings.Contains(err.Error(), "unit-restore:bookstack") {
|
||||
t.Fatalf("an import started while a restore holds the box: %v", err)
|
||||
}
|
||||
if e.activeJob != nil {
|
||||
t.Fatal("a refused import created a job")
|
||||
}
|
||||
}
|
||||
|
||||
// (f) A bundle whose app.yaml carries an empty or relative HDD_PATH is refused BEFORE anything is touched — before, an
|
||||
// empty one made the drive tars extract into a RELATIVE path (the controller's working directory).
|
||||
func TestR1187f_AnEmptyOrRelativeHDDPathIsRefused(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
hdd string
|
||||
ok bool
|
||||
}{{"", false}, {"data", false}, {"/mnt/drive/../../etc", false}, {"/mnt/sys_drive", true}} {
|
||||
tmp := t.TempDir()
|
||||
os.MkdirAll(filepath.Join(tmp, "config"), 0o755)
|
||||
os.WriteFile(filepath.Join(tmp, "config", "app.yaml"), []byte("env:\n HDD_PATH: \""+tc.hdd+"\"\n"), 0o644)
|
||||
os.MkdirAll(filepath.Join(tmp, "data", "hdd"), 0o755)
|
||||
os.WriteFile(filepath.Join(tmp, "data", "hdd", "appdata.tar"), []byte("not empty"), 0o644)
|
||||
err := validateBundleData(tmp, &Manifest{HDDSubdirs: []string{"appdata"}, HasHDDData: true})
|
||||
if tc.ok != (err == nil) {
|
||||
t.Errorf("HDD_PATH %q: refused=%v, want refused=%v (%v)", tc.hdd, err != nil, !tc.ok, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// (g) PostGIS / pgvector / timescaledb images are PostgreSQL for the import too (the backup side's predicate, R-484),
|
||||
// and psql stops on the first error (the dump is --clean --if-exists, so ON_ERROR_STOP is safe).
|
||||
func TestR1187g_PostgisIsPostgresAndPsqlStopsOnError(t *testing.T) {
|
||||
for _, img := range []string{"postgis/postgis:16-3.5-alpine", "pgvector/pgvector:pg16", "timescale/timescaledb:2.14.2-pg16", "postgres:16"} {
|
||||
if info := classifyDBImage("db", img, "c"); info == nil || info.DBType != "postgres" {
|
||||
t.Errorf("%s is not classified as PostgreSQL: %+v", img, info)
|
||||
}
|
||||
}
|
||||
// The load itself goes through appbackup.ImportDump (psql -v ON_ERROR_STOP=1 --single-transaction) since the
|
||||
// import's own client calls were removed — TestR1187_TheImportLoadsTheDumpThroughTheBackupSide.
|
||||
}
|
||||
|
||||
// (b), the security review's case: a link already in the destination folder (left by the old app, or made by an earlier
|
||||
// entry) that points OUTSIDE must never be written through — a tar entry „evil/passwd" would otherwise create a file
|
||||
// wherever „evil" points. Every write is confined to the app's folder (os.Root). RED-PROOF: with plain os.Create the
|
||||
// outside file is created.
|
||||
func TestR1187b_NoWriteThroughALinkThatLeavesTheFolder(t *testing.T) {
|
||||
outside := t.TempDir()
|
||||
dest := t.TempDir()
|
||||
os.Symlink(outside, filepath.Join(dest, "evil")) // pre-existing, pointing out
|
||||
tarPath := filepath.Join(t.TempDir(), "d.tar")
|
||||
f, _ := os.Create(tarPath)
|
||||
tw := tar.NewWriter(f)
|
||||
tw.WriteHeader(&tar.Header{Name: "evil/passwd", Typeflag: tar.TypeReg, Mode: 0o644, Size: 3})
|
||||
tw.Write([]byte("pwn"))
|
||||
tw.WriteHeader(&tar.Header{Name: "ok.txt", Typeflag: tar.TypeReg, Mode: 0o644, Size: 2})
|
||||
tw.Write([]byte("ok"))
|
||||
tw.Close()
|
||||
f.Close()
|
||||
old := lchownFn
|
||||
lchownFn = func(*os.Root, string, int, int) error { return nil }
|
||||
defer func() { lchownFn = old }()
|
||||
extractTar(tarPath, dest)
|
||||
if _, err := os.Stat(filepath.Join(outside, "passwd")); err == nil {
|
||||
t.Fatal("a file was written OUTSIDE the app's folder through a pre-existing link")
|
||||
}
|
||||
}
|
||||
|
||||
// (b), the security review's second case — a parser differential: link „a" → „." makes „a/b" really „b", so a later
|
||||
// link „a/b/l" → „../.." passes a path-TEXT check (root/a/b/../.. = root) while it really points one level ABOVE the
|
||||
// app's folder. A link entry whose parent path passes through a link is refused and listed. RED-PROOF: with the text
|
||||
// check only, „a/b/l" is created and points outside.
|
||||
func TestR1187b_NoLinkThroughALinkedParent(t *testing.T) {
|
||||
dest := t.TempDir()
|
||||
tarPath := filepath.Join(t.TempDir(), "d.tar")
|
||||
f, _ := os.Create(tarPath)
|
||||
tw := tar.NewWriter(f)
|
||||
tw.WriteHeader(&tar.Header{Name: "a", Typeflag: tar.TypeSymlink, Linkname: "."})
|
||||
tw.WriteHeader(&tar.Header{Name: "a/b/", Typeflag: tar.TypeDir, Mode: 0o755})
|
||||
tw.WriteHeader(&tar.Header{Name: "a/b/l", Typeflag: tar.TypeSymlink, Linkname: "../.."})
|
||||
tw.Close()
|
||||
f.Close()
|
||||
old := lchownFn
|
||||
lchownFn = func(*os.Root, string, int, int) error { return nil }
|
||||
defer func() { lchownFn = old }()
|
||||
refused, err := extractTar(tarPath, dest)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, p := range []string{filepath.Join(dest, "b", "l"), filepath.Join(dest, "a", "b", "l")} {
|
||||
if _, err := os.Lstat(p); err == nil {
|
||||
real, _ := filepath.EvalSymlinks(p)
|
||||
t.Fatalf("a link through a linked parent was created (%s → %s)", p, real)
|
||||
}
|
||||
}
|
||||
if !strings.Contains(strings.Join(refused, ","), "a/b/l") {
|
||||
t.Fatalf("the refused link must be listed, got %v", refused)
|
||||
}
|
||||
}
|
||||
|
||||
// (b), the security review's third case: a link TARGET that passes through a link already in the folder („evil" → an
|
||||
// outside folder, then „x" → „evil/y") reads as inside by its text and really points outside. Restored links may only
|
||||
// point DOWNWARD to real folders: no absolute target, no „..", no link on the way. RED-PROOF: the text check created „x".
|
||||
func TestR1187b_NoLinkTargetThroughALink(t *testing.T) {
|
||||
outside := t.TempDir()
|
||||
dest := t.TempDir()
|
||||
os.Symlink(outside, filepath.Join(dest, "evil"))
|
||||
tarPath := filepath.Join(t.TempDir(), "d.tar")
|
||||
f, _ := os.Create(tarPath)
|
||||
tw := tar.NewWriter(f)
|
||||
tw.WriteHeader(&tar.Header{Name: "x", Typeflag: tar.TypeSymlink, Linkname: "evil/y"})
|
||||
tw.WriteHeader(&tar.Header{Name: "up", Typeflag: tar.TypeSymlink, Linkname: "sub/../sub"})
|
||||
tw.Close()
|
||||
f.Close()
|
||||
old := lchownFn
|
||||
lchownFn = func(*os.Root, string, int, int) error { return nil }
|
||||
defer func() { lchownFn = old }()
|
||||
refused, _ := extractTar(tarPath, dest)
|
||||
if _, err := os.Lstat(filepath.Join(dest, "x")); err == nil {
|
||||
t.Fatal("a link whose target passes through an outside-pointing link was created")
|
||||
}
|
||||
if strings.Join(refused, ",") != "x,up" {
|
||||
t.Fatalf("both links must be refused and listed, got %v", refused)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,47 @@
|
||||
package appexport
|
||||
|
||||
import (
|
||||
"context"
|
||||
"io"
|
||||
"log"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/appbackup"
|
||||
)
|
||||
|
||||
// R-1187, found live on 9202 2026-10-26 night: the `.fab` import waited with `mysqladmin` and loaded with `mysql` — names
|
||||
// the MariaDB 11+ images do not ship (only mariadb-admin / mariadb; measured in mariadb:11.4) — and read the password
|
||||
// from the app's env under two fixed names (grimmory's is DB_ROOT_PASSWORD). Every import of a MariaDB 11+ app timed
|
||||
// out and left the app down. The import now uses the BACKUP side's discovery (the live container's own credentials)
|
||||
// and appbackup.ImportDump (mariadb / mariadb-admin, psql ON_ERROR_STOP + --single-transaction). The seams keep this
|
||||
// test off the real Docker (R-650). RED-PROOF: the old body (its own waitForDB + importDBDump) never reaches the seams.
|
||||
func TestR1187_TheImportLoadsTheDumpThroughTheBackupSide(t *testing.T) {
|
||||
oldD, oldI := discoverDBsFn, importDumpFn
|
||||
defer func() { discoverDBsFn, importDumpFn = oldD, oldI }()
|
||||
discoverDBsFn = func(ctx context.Context, l *log.Logger, debug bool, known []string) ([]appbackup.DiscoveredDB, error) {
|
||||
return []appbackup.DiscoveredDB{
|
||||
{ContainerName: "paperless-postgres", ContainerID: "1b47ee5f58d2", DBType: appbackup.DBTypePostgres, StackName: "paperless-ngx"},
|
||||
{ContainerName: "grimmory-db", ContainerID: "2310c379bd74", DBType: appbackup.DBTypeMariaDB, DBName: "grimmory", StackName: "grimmory"},
|
||||
}, nil
|
||||
}
|
||||
var got appbackup.DiscoveredDB
|
||||
var gotPath string
|
||||
importDumpFn = func(ctx context.Context, db appbackup.DiscoveredDB, dumpPath string, l *log.Logger, debug bool) error {
|
||||
got, gotPath = db, dumpPath
|
||||
return nil
|
||||
}
|
||||
e := NewExporter(&rtProvider{}, log.New(io.Discard, "", 0), "test")
|
||||
err := e.importIntoContainer("grimmory", "2310c379bd742a9bbb3a78ceb60d1872edb2dfe0ddc15b134edb9142b048e558", "mariadb", "/tmp/x/grimmory-mariadb.sql.gz", nil)
|
||||
if err != nil {
|
||||
t.Fatalf("import: %v", err)
|
||||
}
|
||||
if got.ContainerName != "grimmory-db" || gotPath != "/tmp/x/grimmory-mariadb.sql.gz" {
|
||||
t.Fatalf("the dump did not go through the backup side's ImportDump for the started container: %+v %q", got, gotPath)
|
||||
}
|
||||
// A container the backup side cannot see is a clear refusal, never a guess.
|
||||
err = e.importIntoContainer("grimmory", "ffffffffffff", "mariadb", "/tmp/x/d.sql.gz", nil)
|
||||
if err == nil || !strings.Contains(err.Error(), "ffffffffffff") {
|
||||
t.Fatalf("an undiscovered container must be refused naming it, got %v", err)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,40 @@
|
||||
package appexport
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/appbackup"
|
||||
)
|
||||
|
||||
// R-1187, found live on 9202 2026-10-26 night (grimmory): the import ran `compose` WITHOUT the two path variables the
|
||||
// controller adds at every install and start (USERDATA_PATH, IMPORT_PATH — stacks.policyEnv). Compose then printed
|
||||
// „The USERDATA_PATH variable is not set" on its combined output, the DB step read that WARNING as the container id,
|
||||
// waited 60 s for a container that does not exist, and left the app removed and not started. Both halves pinned here.
|
||||
// RED-PROOF: with the old helpers (the first output line; the env unchanged) both tests fail.
|
||||
func TestR1187_TheContainerIDIsNeverAComposeWarning(t *testing.T) {
|
||||
out := []byte("time=\"2026-10-26T20:18:06Z\" level=warning msg=\"The \\\"USERDATA_PATH\\\" variable is not set. Defaulting to a blank string.\"\n" +
|
||||
"1047c698c778ab12cd34ef56ab78cd90ef12ab34cd56ef78ab90cd12ef34ab56\n")
|
||||
if got := containerIDFrom(out); got != "1047c698c778ab12cd34ef56ab78cd90ef12ab34cd56ef78ab90cd12ef34ab56" {
|
||||
t.Fatalf("container id = %q", got)
|
||||
}
|
||||
if got := containerIDFrom([]byte("time=x level=warning msg=y\n")); got != "" {
|
||||
t.Fatalf("a warning alone must give no id, got %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestR1187_TheImportEnvCarriesTheControllersPathVars(t *testing.T) {
|
||||
env := map[string]string{"HDD_PATH": "/mnt/felhom-drives/d", "USERDATA_PATH": "/mnt/felhom-drives/d/appdata/other", "X": "1"}
|
||||
got := withImportPathVars(env, "/mnt/felhom-drives/d", "/mnt/sys_drive/felhom-data/userdata/import")
|
||||
if got["USERDATA_PATH"] != appbackup.UserdataDir("/mnt/felhom-drives/d") {
|
||||
t.Fatalf("USERDATA_PATH = %q — must be the controller's own derivation, never the bundle's", got["USERDATA_PATH"])
|
||||
}
|
||||
if got["IMPORT_PATH"] != "/mnt/sys_drive/felhom-data/userdata/import" || got["X"] != "1" || got["HDD_PATH"] != "/mnt/felhom-drives/d" {
|
||||
t.Fatalf("env = %v", got)
|
||||
}
|
||||
if env["USERDATA_PATH"] != "/mnt/felhom-drives/d/appdata/other" {
|
||||
t.Fatal("the caller's map was changed")
|
||||
}
|
||||
if g := withImportPathVars(map[string]string{"IMPORT_PATH": "/x"}, "", ""); g["USERDATA_PATH"] != "" || g["IMPORT_PATH"] != "" {
|
||||
t.Fatalf("no namespace / no import root must leave neither variable: %v", g)
|
||||
}
|
||||
}
|
||||
@@ -5,11 +5,11 @@ import (
|
||||
"compress/gzip"
|
||||
"context"
|
||||
"fmt"
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/appbackup"
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/dockerexec"
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/util"
|
||||
"io"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
@@ -184,6 +184,16 @@ func (e *Exporter) StartImport(req ImportRequest) error {
|
||||
e.debugf("StartImport rejected: another job is already running")
|
||||
return fmt.Errorf("export or import already in progress")
|
||||
}
|
||||
// R-1187 (d): another operation holds the app-stop marker (a restore, a put-back, a stop-first export) — an import
|
||||
// now would stop and rewrite an app that operation is moving. Refused before anything is read. Pinned by
|
||||
// TestR1187d_AnImportWaitsForARestore.
|
||||
if e.stopGuard != nil {
|
||||
if busy := e.stopGuard.BusyWith(); busy != "" {
|
||||
e.mu.Unlock()
|
||||
e.logger.Printf("[WARN] Import: refused — another operation holds the box (%s) (R-1187 d)", busy)
|
||||
return util.MsgError("err.appexport.import_foglalt", busy)
|
||||
}
|
||||
}
|
||||
|
||||
// Validate file exists
|
||||
info, err := os.Stat(req.FABPath)
|
||||
@@ -431,6 +441,8 @@ func (e *Exporter) executeImport(req ImportRequest, job *Job) {
|
||||
e.failJob(job, step, fmt.Sprintf("Konfiguráció visszaállítása sikertelen: %v", err))
|
||||
return
|
||||
}
|
||||
// R-1187 (found live): compose runs below see the controller's path variables, as at every install and start.
|
||||
env = withImportPathVars(env, e.provider.GetStackNamespaceRoot(manifest.AppName), e.provider.GetImportRoot())
|
||||
e.debugf("config restored: %d env vars", len(env))
|
||||
if e.debug {
|
||||
for k := range env {
|
||||
@@ -674,9 +686,14 @@ func (e *Exporter) restoreHDDData(tmpDir string, manifest *Manifest, composePath
|
||||
|
||||
e.logger.Printf("[INFO] Import: extracting HDD data %s → %s", subdir, destPath)
|
||||
extractStart := time.Now()
|
||||
if err := extractTar(tarPath, destPath); err != nil {
|
||||
refused, err := extractTar(tarPath, destPath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("extracting %s: %w", subdir, err)
|
||||
}
|
||||
if len(refused) > 0 {
|
||||
e.logger.Printf("[WARN] Import: %s — %d link(s) pointing outside the app's folder were NOT restored (R-1187 b): %s",
|
||||
subdir, len(refused), strings.Join(refused, ", "))
|
||||
}
|
||||
e.debugf("restoreHDDData: extracted %s in %v", subdir, time.Since(extractStart))
|
||||
}
|
||||
|
||||
@@ -726,6 +743,16 @@ func resolveHDDMounts(composePath string, env map[string]string) []string {
|
||||
// validateBundleData asserts every manifest-claimed data tar exists non-empty in the extracted
|
||||
// bundle (v0.125.0, scenario C). Pure read — called before ANY destructive import step.
|
||||
func validateBundleData(tmpDir string, manifest *Manifest) error {
|
||||
// R-1187 (f): drive data needs an ABSOLUTE, clean HDD_PATH from the bundle's app.yaml — an empty one made the drive
|
||||
// tars extract into a relative path (the controller's working directory). Refused before anything is touched.
|
||||
// Pinned by TestR1187f_AnEmptyOrRelativeHDDPathIsRefused.
|
||||
if len(manifest.HDDSubdirs) > 0 {
|
||||
env, _ := readPlaintextAppYaml(filepath.Join(tmpDir, "config", "app.yaml"))
|
||||
hdd := strings.TrimSpace(env["HDD_PATH"])
|
||||
if hdd == "" || !filepath.IsAbs(hdd) || filepath.Clean(hdd) != hdd || strings.Contains(hdd, "..") {
|
||||
return util.MsgError("err.appexport.hdd_path_hibas", hdd)
|
||||
}
|
||||
}
|
||||
for _, v := range manifest.VolumeNames {
|
||||
if err := ValidateSegment("volume_name", v); err != nil {
|
||||
return err
|
||||
@@ -856,29 +883,53 @@ func (e *Exporter) restoreDatabase(manifest *Manifest, tmpDir, composePath, stac
|
||||
}
|
||||
e.debugf("restoreDatabase: container ID=%s", containerID)
|
||||
|
||||
// Wait for DB readiness
|
||||
e.logger.Printf("[INFO] Import: waiting for DB readiness (container %s)", containerID[:12])
|
||||
e.debugf("restoreDatabase: waiting for DB readiness...")
|
||||
waitStart := time.Now()
|
||||
if err := waitForDB(containerID, dbSvc.DBType, env); err != nil {
|
||||
e.debugf("restoreDatabase: DB not ready after %v: %v", time.Since(waitStart), err)
|
||||
return fmt.Errorf("waiting for DB: %w", err)
|
||||
}
|
||||
e.debugf("restoreDatabase: DB ready in %v", time.Since(waitStart))
|
||||
|
||||
// Import the dump
|
||||
e.logger.Printf("[INFO] Import: importing database dump")
|
||||
e.debugf("restoreDatabase: importing dump into container %s", containerID[:12])
|
||||
// Wait for readiness and import the dump.
|
||||
importStart := time.Now()
|
||||
if err := importDBDump(containerID, dumpGzPath, dbSvc.DBType, env); err != nil {
|
||||
if err := e.importIntoContainer(manifest.AppName, containerID, dbSvc.DBType, dumpGzPath, env); err != nil {
|
||||
e.debugf("restoreDatabase: import failed after %v: %v", time.Since(importStart), err)
|
||||
return fmt.Errorf("importing dump: %w", err)
|
||||
return err
|
||||
}
|
||||
e.debugf("restoreDatabase: dump imported in %v", time.Since(importStart))
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// Seams for importIntoContainer (a test replaces them; production is the backup side's own code).
|
||||
var (
|
||||
discoverDBsFn = appbackup.DiscoverDatabases
|
||||
importDumpFn = appbackup.ImportDump
|
||||
)
|
||||
|
||||
// importIntoContainer waits for the database and loads the dump — through the BACKUP side (R-1187, found live on 9202):
|
||||
// the container's own credentials from appbackup's discovery, and appbackup.ImportDump (mariadb / mariadb-admin, which
|
||||
// the MariaDB 11+ images ship; psql ON_ERROR_STOP + --single-transaction). The import's own waitForDB/importDBDump used
|
||||
// mysqladmin/mysql (absent from mariadb:11.4) and two fixed env names for the password, so every MariaDB 11+ import
|
||||
// timed out. Pinned by TestR1187_TheImportLoadsTheDumpThroughTheBackupSide.
|
||||
func (e *Exporter) importIntoContainer(app, containerID, dbType, dumpPath string, env map[string]string) error {
|
||||
ctx := context.Background()
|
||||
dbs, err := discoverDBsFn(ctx, e.logger, e.debug, []string{app})
|
||||
if err != nil {
|
||||
return fmt.Errorf("finding the database container: %w", err)
|
||||
}
|
||||
for _, db := range dbs {
|
||||
if db.ContainerID == "" || !(strings.HasPrefix(containerID, db.ContainerID) || strings.HasPrefix(db.ContainerID, containerID)) {
|
||||
continue
|
||||
}
|
||||
e.logger.Printf("[INFO] Import: loading the database dump into %s (%s)", db.ContainerName, db.DBType)
|
||||
if err := importDumpFn(ctx, db, dumpPath, e.logger, e.debug); err != nil {
|
||||
return fmt.Errorf("importing dump: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
return fmt.Errorf("the started database container %s is not among the databases the box can see", shortID(containerID))
|
||||
}
|
||||
|
||||
func shortID(id string) string {
|
||||
if len(id) > 12 {
|
||||
return id[:12]
|
||||
}
|
||||
return id
|
||||
}
|
||||
|
||||
// findDumpFile finds the gzipped SQL dump in the database directory.
|
||||
func findDumpFile(dbDir string) (string, error) {
|
||||
entries, err := os.ReadDir(dbDir)
|
||||
@@ -967,11 +1018,12 @@ func findDBServiceInCompose(composePath string) *dbServiceInfo {
|
||||
}
|
||||
|
||||
func classifyDBImage(service, image, container string) *dbServiceInfo {
|
||||
img := strings.ToLower(image)
|
||||
if strings.Contains(img, "postgres") {
|
||||
// R-1187 (g): the backup side's predicate (appbackup.DBTypeForImage, R-484) — postgis / pgvector / timescaledb are
|
||||
// PostgreSQL too. The import used its own narrower list and skipped adventurelog's and dawarich's database.
|
||||
switch t, ok := appbackup.DBTypeForImage(image); {
|
||||
case ok && t == appbackup.DBTypePostgres:
|
||||
return &dbServiceInfo{ServiceName: service, DBType: "postgres", ContainerName: container}
|
||||
}
|
||||
if strings.Contains(img, "mariadb") || strings.Contains(img, "mysql") {
|
||||
case ok && t == appbackup.DBTypeMariaDB:
|
||||
return &dbServiceInfo{ServiceName: service, DBType: "mariadb", ContainerName: container}
|
||||
}
|
||||
return nil
|
||||
@@ -1002,118 +1054,43 @@ func getComposeContainerID(stackDir string, env map[string]string, service strin
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("compose ps: %w", err)
|
||||
}
|
||||
id := strings.TrimSpace(string(out))
|
||||
id := containerIDFrom(out)
|
||||
if id == "" {
|
||||
return "", fmt.Errorf("no container found for service %s", service)
|
||||
}
|
||||
// Take the first line in case of multiple
|
||||
if idx := strings.Index(id, "\n"); idx > 0 {
|
||||
id = id[:idx]
|
||||
}
|
||||
return id, nil
|
||||
}
|
||||
|
||||
// waitForDB polls until the database is ready to accept connections.
|
||||
func waitForDB(containerID, dbType string, env map[string]string) error {
|
||||
timeout := 60 * time.Second
|
||||
start := time.Now()
|
||||
|
||||
for {
|
||||
if time.Since(start) > timeout {
|
||||
return fmt.Errorf("timeout waiting for %s to become ready", dbType)
|
||||
// containerIDFrom picks the container id out of `compose ps -q` output: the first line that IS an id (12–64 hex). The
|
||||
// output is combined, so a compose WARNING can come first — it was read as the id once (R-1187, found live on 9202).
|
||||
func containerIDFrom(out []byte) string {
|
||||
for _, line := range strings.Split(string(out), "\n") {
|
||||
line = strings.TrimSpace(line)
|
||||
if n := len(line); n >= 12 && n <= 64 && strings.Trim(line, "0123456789abcdef") == "" {
|
||||
return line
|
||||
}
|
||||
|
||||
var cmd *exec.Cmd
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
|
||||
switch dbType {
|
||||
case "postgres":
|
||||
user := env["POSTGRES_USER"]
|
||||
if user == "" {
|
||||
user = "postgres"
|
||||
}
|
||||
cmd = dockerexec.CommandContext(ctx, "docker", "exec", containerID,
|
||||
"pg_isready", "-U", user)
|
||||
case "mariadb":
|
||||
password := env["MYSQL_ROOT_PASSWORD"]
|
||||
if password == "" {
|
||||
password = env["MARIADB_ROOT_PASSWORD"]
|
||||
}
|
||||
cmd = dockerexec.CommandContext(ctx, "docker", "exec", containerID,
|
||||
"mysqladmin", "ping", "-u", "root", "-p"+password)
|
||||
default:
|
||||
cancel()
|
||||
return fmt.Errorf("unknown DB type: %s", dbType)
|
||||
}
|
||||
|
||||
err := cmd.Run()
|
||||
cancel()
|
||||
if err == nil {
|
||||
return nil
|
||||
}
|
||||
|
||||
time.Sleep(2 * time.Second)
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// importDBDump imports a (possibly gzipped) SQL dump into the running database container.
|
||||
func importDBDump(containerID, dumpPath, dbType string, env map[string]string) error {
|
||||
// Open the dump file
|
||||
f, err := os.Open(dumpPath)
|
||||
if err != nil {
|
||||
return err
|
||||
// withImportPathVars is the import's env as `compose` must see it: the bundle's values plus the two path variables the
|
||||
// controller itself adds at every install and start (stacks.policyEnv) — USERDATA_PATH always the controller's own
|
||||
// derivation from the namespace root (never a bundle's value), IMPORT_PATH the drop-zone root. Without them compose
|
||||
// printed a warning the DB step read as the container id (R-1187, live on 9202). Returns a new map.
|
||||
func withImportPathVars(env map[string]string, nsRoot, importRoot string) map[string]string {
|
||||
out := make(map[string]string, len(env)+2)
|
||||
for k, v := range env {
|
||||
out[k] = v
|
||||
}
|
||||
defer f.Close()
|
||||
|
||||
// If gzipped, wrap in gzip reader
|
||||
var reader io.Reader = f
|
||||
if strings.HasSuffix(dumpPath, ".gz") {
|
||||
gr, err := gzip.NewReader(f)
|
||||
if err != nil {
|
||||
return fmt.Errorf("opening gzip: %w", err)
|
||||
}
|
||||
defer gr.Close()
|
||||
reader = gr
|
||||
delete(out, "USERDATA_PATH")
|
||||
delete(out, "IMPORT_PATH")
|
||||
if nsRoot != "" {
|
||||
out["USERDATA_PATH"] = appbackup.UserdataDir(nsRoot)
|
||||
}
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Minute)
|
||||
defer cancel()
|
||||
|
||||
var cmd *exec.Cmd
|
||||
switch dbType {
|
||||
case "postgres":
|
||||
user := env["POSTGRES_USER"]
|
||||
if user == "" {
|
||||
user = "postgres"
|
||||
}
|
||||
dbName := env["POSTGRES_DB"]
|
||||
if dbName == "" {
|
||||
dbName = user
|
||||
}
|
||||
cmd = dockerexec.CommandContext(ctx, "docker", "exec", "-i", containerID,
|
||||
"psql", "-U", user, "-d", dbName)
|
||||
case "mariadb":
|
||||
password := env["MYSQL_ROOT_PASSWORD"]
|
||||
if password == "" {
|
||||
password = env["MARIADB_ROOT_PASSWORD"]
|
||||
}
|
||||
dbName := env["MYSQL_DATABASE"]
|
||||
if dbName == "" {
|
||||
dbName = env["MARIADB_DATABASE"]
|
||||
}
|
||||
cmd = dockerexec.CommandContext(ctx, "docker", "exec", "-i", containerID,
|
||||
"mysql", "-u", "root", "-p"+password, dbName)
|
||||
default:
|
||||
return fmt.Errorf("unknown DB type: %s", dbType)
|
||||
if importRoot != "" {
|
||||
out["IMPORT_PATH"] = importRoot
|
||||
}
|
||||
|
||||
cmd.Stdin = reader
|
||||
out, err := cmd.CombinedOutput()
|
||||
if err != nil {
|
||||
return fmt.Errorf("%s import failed: %s — %w", dbType, strings.TrimSpace(string(out)), err)
|
||||
}
|
||||
|
||||
return nil
|
||||
return out
|
||||
}
|
||||
|
||||
// extractTarGz extracts a tar.gz archive to a directory.
|
||||
@@ -1170,50 +1147,142 @@ func extractTarGz(tgzPath, destDir string) error {
|
||||
}
|
||||
}
|
||||
|
||||
// extractTar extracts a plain tar archive to a directory.
|
||||
func extractTar(tarPath, destDir string) error {
|
||||
// safeDownwardLink is the rule for a restored link: a relative target with no „.." component, and no symbolic link on
|
||||
// the way — not among the folders of the link's own path, not among the existing components of its target.
|
||||
func safeDownwardLink(rt *os.Root, name, target string) bool {
|
||||
if target == "" || filepath.IsAbs(target) {
|
||||
return false
|
||||
}
|
||||
for _, part := range strings.Split(filepath.ToSlash(target), "/") {
|
||||
if part == ".." {
|
||||
return false
|
||||
}
|
||||
}
|
||||
if parentThroughLink(rt, name) {
|
||||
return false
|
||||
}
|
||||
acc := filepath.Dir(name)
|
||||
for _, part := range strings.Split(filepath.Clean(target), string(os.PathSeparator)) {
|
||||
acc = filepath.Join(acc, part)
|
||||
fi, err := rt.Lstat(acc)
|
||||
if err != nil {
|
||||
break // not there (yet): nothing further down can be a link that exists now
|
||||
}
|
||||
if fi.Mode()&os.ModeSymlink != 0 {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// parentThroughLink is true when any folder on name's path (inside the root) is a symbolic link.
|
||||
func parentThroughLink(rt *os.Root, name string) bool {
|
||||
dir := filepath.Dir(name)
|
||||
if dir == "." {
|
||||
return false
|
||||
}
|
||||
acc := ""
|
||||
for _, part := range strings.Split(dir, string(os.PathSeparator)) {
|
||||
acc = filepath.Join(acc, part)
|
||||
fi, err := rt.Lstat(acc)
|
||||
if err != nil {
|
||||
return false // not there yet: MkdirAll creates real folders
|
||||
}
|
||||
if fi.Mode()&os.ModeSymlink != 0 {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// lchownFn is the owner seam (a test records it; production is os.Lchown).
|
||||
var lchownFn = func(r *os.Root, name string, uid, gid int) error { return r.Lchown(name, uid, gid) }
|
||||
|
||||
// extractTar extracts a plain tar archive (one drive folder) into destDir and returns the entries it REFUSED.
|
||||
//
|
||||
// R-1187 (b): owners and modes are kept (Lchown — the controller runs as root on a box), and a symlink is restored when
|
||||
// it stays inside destDir; a link pointing outside (absolute, or climbing out with ..) is refused and listed — never
|
||||
// followed, never created. Only folders, regular files and links exist in a drive tar (the export refuses the rest).
|
||||
// Pinned by TestR1187b_TheImportKeepsOwnersModesAndInsideLinks.
|
||||
func extractTar(tarPath, destDir string) ([]string, error) {
|
||||
f, err := os.Open(tarPath)
|
||||
if err != nil {
|
||||
return err
|
||||
return nil, err
|
||||
}
|
||||
defer f.Close()
|
||||
|
||||
root := filepath.Clean(destDir)
|
||||
// Every write goes through os.Root: no entry can create or write a file outside destDir, not even through a link
|
||||
// already in the folder (left by the old app, or made by an earlier entry) that points out — the security review's
|
||||
// case, pinned by TestR1187b_NoWriteThroughALinkThatLeavesTheFolder.
|
||||
rt, err := os.OpenRoot(root)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rt.Close()
|
||||
var refused []string
|
||||
tr := tar.NewReader(f)
|
||||
for {
|
||||
hdr, err := tr.Next()
|
||||
if err == io.EOF {
|
||||
return nil
|
||||
return refused, nil
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
return refused, err
|
||||
}
|
||||
|
||||
target := filepath.Join(destDir, hdr.Name)
|
||||
|
||||
// Security: prevent path traversal
|
||||
if !strings.HasPrefix(filepath.Clean(target), filepath.Clean(destDir)+string(os.PathSeparator)) {
|
||||
continue
|
||||
name := filepath.Clean(hdr.Name)
|
||||
if name == "." || filepath.IsAbs(name) || name == ".." || strings.HasPrefix(name, ".."+string(os.PathSeparator)) {
|
||||
continue // path traversal
|
||||
}
|
||||
|
||||
switch hdr.Typeflag {
|
||||
case tar.TypeDir:
|
||||
if err := os.MkdirAll(target, os.FileMode(hdr.Mode)); err != nil {
|
||||
return err
|
||||
if err := rt.MkdirAll(name, os.FileMode(hdr.Mode).Perm()); err != nil {
|
||||
return refused, err
|
||||
}
|
||||
rt.Chmod(name, os.FileMode(hdr.Mode).Perm())
|
||||
case tar.TypeReg:
|
||||
if err := os.MkdirAll(filepath.Dir(target), 0755); err != nil {
|
||||
return err
|
||||
if dir := filepath.Dir(name); dir != "." {
|
||||
if err := rt.MkdirAll(dir, 0755); err != nil {
|
||||
return refused, err
|
||||
}
|
||||
}
|
||||
outFile, err := os.Create(target)
|
||||
out, err := rt.OpenFile(name, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, os.FileMode(hdr.Mode).Perm())
|
||||
if err != nil {
|
||||
return err
|
||||
refused = append(refused, hdr.Name) // e.g. its path goes through a link that leaves the folder
|
||||
continue
|
||||
}
|
||||
if _, err := io.Copy(outFile, tr); err != nil {
|
||||
outFile.Close()
|
||||
return err
|
||||
if _, err := io.Copy(out, tr); err != nil {
|
||||
out.Close()
|
||||
return refused, err
|
||||
}
|
||||
outFile.Close()
|
||||
os.Chmod(target, os.FileMode(hdr.Mode))
|
||||
out.Close()
|
||||
rt.Chmod(name, os.FileMode(hdr.Mode).Perm())
|
||||
case tar.TypeSymlink:
|
||||
// A restored link may only point DOWNWARD inside the folder, through real folders: no absolute target, no
|
||||
// „..", and no link on the way — neither on the link's own path nor in its target. Such a link can never
|
||||
// resolve outside the app's folder, whatever else the folder holds. Everything else is refused and listed.
|
||||
// (Three rounds of the automated security review: a link written through, a linked parent, a target through
|
||||
// a link. Pinned by TestR1187b_*.)
|
||||
if !safeDownwardLink(rt, name, hdr.Linkname) {
|
||||
refused = append(refused, hdr.Name)
|
||||
continue
|
||||
}
|
||||
dst := hdr.Linkname
|
||||
if dir := filepath.Dir(name); dir != "." {
|
||||
if err := rt.MkdirAll(dir, 0755); err != nil {
|
||||
return refused, err
|
||||
}
|
||||
}
|
||||
rt.Remove(name)
|
||||
if err := rt.Symlink(dst, name); err != nil {
|
||||
refused = append(refused, hdr.Name)
|
||||
continue
|
||||
}
|
||||
default:
|
||||
refused = append(refused, hdr.Name)
|
||||
continue
|
||||
}
|
||||
if err := lchownFn(rt, name, hdr.Uid, hdr.Gid); err != nil && !os.IsPermission(err) {
|
||||
return refused, fmt.Errorf("setting the owner of %s: %w", hdr.Name, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -714,6 +714,14 @@ func (m *Manager) OffboxConfigured() bool {
|
||||
return true
|
||||
}
|
||||
|
||||
// OffboxRepoPasswordOnBox reports whether the off-site repository password FILE is on the box — whatever the target's
|
||||
// state. R-1218: ClearOffboxTarget keeps that file on purpose, and a switched-off target keeps it too, so a recovery
|
||||
// ceremony run then must still seal it; OffboxConfigured() is the wrong question for „is there a key to escrow".
|
||||
func (m *Manager) OffboxRepoPasswordOnBox() bool {
|
||||
_, err := os.Stat(m.offboxPwPath())
|
||||
return err == nil
|
||||
}
|
||||
|
||||
// offboxRepoPwPattern matches a valid restic repo password (generateOffboxPassword = 32 rand bytes → 64 hex).
|
||||
var offboxRepoPwPattern = regexp.MustCompile(`^[0-9a-fA-F]{64}$`)
|
||||
|
||||
|
||||
@@ -0,0 +1,171 @@
|
||||
// Package holdreport tells the hub which apps are held (D53, felhom.eu `08` §10).
|
||||
//
|
||||
// A held app is stopped on purpose and stays stopped until a person acts. Until D53 the hub knew nothing about it:
|
||||
// the crash-loop stop sent one event, the other holds sent none, and demo-hp's Kimai sat held for 11 days unseen
|
||||
// (R-1291). The legacy report is frozen, so the event channel carries the holds:
|
||||
//
|
||||
// - app_held when a hold appears or changes (stack, reason, since, the hold's sentence);
|
||||
// - app_hold_cleared when it is gone — also when it went while the controller was down, because the set last
|
||||
// reported is kept in settings (ReportedHolds), not in memory;
|
||||
// - every hold still in place is sent again once a day and at every controller start, because an event can be
|
||||
// lost. The hub treats a repeat as idempotent.
|
||||
//
|
||||
// ONE OBSERVER, NOT A HOOK PER WRITER. Seven code paths write a hold and four clear one. A send in each of them is
|
||||
// the shape where the one path nobody found is the next silent hold; this package reads the hold store itself, so a
|
||||
// new writer is covered the day it is added. Both events are severity info: the hub mails nothing on receipt. Its
|
||||
// reminder after the grace per reason is the operator's mail, and it never goes to the household (D53).
|
||||
//
|
||||
// Pinned by holdreport_test.go.
|
||||
package holdreport
|
||||
|
||||
import (
|
||||
"log"
|
||||
"sort"
|
||||
"time"
|
||||
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/settings"
|
||||
)
|
||||
|
||||
// Event types — the hub's allowedEventTypes carries both (the coupling rule).
|
||||
const (
|
||||
EventHeld = "app_held"
|
||||
EventCleared = "app_hold_cleared"
|
||||
)
|
||||
|
||||
// ResendEvery is how often every hold still in place is sent again.
|
||||
const ResendEvery = 24 * time.Hour
|
||||
|
||||
// Store is what the reporter needs of the settings.
|
||||
type Store interface {
|
||||
ListRestoreHolds() []settings.RestoreHold
|
||||
GetReportedHolds() map[string]string
|
||||
SetReportedHolds(map[string]string) error
|
||||
}
|
||||
|
||||
// Push sends one event to the hub (notify.Notifier.PushEvent).
|
||||
type Push func(eventType, severity, message string, details interface{})
|
||||
|
||||
// Details is the payload of both events.
|
||||
type Details struct {
|
||||
StackName string `json:"stack_name"`
|
||||
Reason string `json:"reason,omitempty"`
|
||||
Since string `json:"since,omitempty"`
|
||||
Why string `json:"why,omitempty"`
|
||||
Repeat bool `json:"repeat,omitempty"` // a daily or start-up re-send, not a new hold
|
||||
}
|
||||
|
||||
// Reporter compares the hold store with what was last reported. Tick is called by the scheduler.
|
||||
type Reporter struct {
|
||||
store Store
|
||||
push Push
|
||||
logger *log.Logger
|
||||
lastFull time.Time
|
||||
// installed (R-1298) — nil = every app counts as installed.
|
||||
installed func(stack string) bool
|
||||
skipLogged map[string]bool
|
||||
}
|
||||
|
||||
// New builds a reporter. The first Tick re-sends every hold (a controller start).
|
||||
func New(store Store, push Push, logger *log.Logger) *Reporter {
|
||||
return &Reporter{store: store, push: push, logger: logger}
|
||||
}
|
||||
|
||||
func key(h settings.RestoreHold) string { return h.WireReason() + "|" + h.At }
|
||||
|
||||
// SetInstalled wires the „is this app installed?" check (R-1298). nil = every app counts as installed.
|
||||
func (r *Reporter) SetInstalled(f func(stack string) bool) { r.installed = f }
|
||||
|
||||
// installHold is a hold that belongs to an app's INSTALL (R-704): a removal ends it. A restore hold (restore_failed,
|
||||
// restore_mixed) does not — a person clears it, so it stays reported whatever the app's state (R-379).
|
||||
func installHold(h settings.RestoreHold) bool {
|
||||
switch h.Reason {
|
||||
case settings.HoldReasonUnhealthyStop, settings.HoldReasonUpdateFailed, settings.HoldReasonPutBackPending:
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// Tick sends what changed since the last tick, and every hold again when a day has passed.
|
||||
func (r *Reporter) Tick(now time.Time) {
|
||||
if r.store == nil || r.push == nil {
|
||||
return
|
||||
}
|
||||
all := r.store.ListRestoreHolds()
|
||||
sort.Slice(all, func(i, j int) bool { return all[i].Stack < all[j].Stack })
|
||||
// R-1298: an install's hold of an app that is not installed any more (removed outside the product) is not a held
|
||||
// app — it is skipped, and reported cleared if it was reported before. Logged once per stack per process.
|
||||
holds := all[:0:0]
|
||||
for _, h := range all {
|
||||
if r.installed != nil && installHold(h) && !r.installed(h.Stack) {
|
||||
if r.skipLogged == nil {
|
||||
r.skipLogged = map[string]bool{}
|
||||
}
|
||||
if !r.skipLogged[h.Stack] {
|
||||
r.skipLogged[h.Stack] = true
|
||||
r.logf("[INFO] [holdreport] %s: a %s hold of an app that is not installed — not reported (R-1298)", h.Stack, h.WireReason())
|
||||
}
|
||||
continue
|
||||
}
|
||||
holds = append(holds, h)
|
||||
}
|
||||
prev := r.store.GetReportedHolds()
|
||||
full := r.lastFull.IsZero() || now.Sub(r.lastFull) >= ResendEvery
|
||||
|
||||
cur := make(map[string]string, len(holds))
|
||||
sent, cleared := 0, 0
|
||||
for _, h := range holds {
|
||||
k := key(h)
|
||||
cur[h.Stack] = k
|
||||
known, seen := prev[h.Stack]
|
||||
if seen && known == k && !full {
|
||||
continue
|
||||
}
|
||||
repeat := seen && known == k
|
||||
r.push(EventHeld, "info", "App "+h.Stack+" is held: "+h.Why(),
|
||||
Details{StackName: h.Stack, Reason: h.WireReason(), Since: h.At, Why: h.Why(), Repeat: repeat})
|
||||
sent++
|
||||
if !repeat {
|
||||
r.logf("[INFO] [holdreport] %s held (%s since %s) — reported to the hub", h.Stack, h.WireReason(), h.At)
|
||||
}
|
||||
}
|
||||
gone := make([]string, 0)
|
||||
for stack := range prev {
|
||||
if _, ok := cur[stack]; !ok {
|
||||
gone = append(gone, stack)
|
||||
}
|
||||
}
|
||||
sort.Strings(gone)
|
||||
for _, stack := range gone {
|
||||
r.push(EventCleared, "info", "App "+stack+" is no longer held", Details{StackName: stack})
|
||||
cleared++
|
||||
r.logf("[INFO] [holdreport] %s hold cleared — reported to the hub", stack)
|
||||
}
|
||||
if full {
|
||||
r.lastFull = now
|
||||
r.logf("[DEBUG] [holdreport] full re-send: %d hold(s) sent, %d cleared", sent, cleared)
|
||||
}
|
||||
if !sameMap(prev, cur) {
|
||||
if err := r.store.SetReportedHolds(cur); err != nil {
|
||||
// Not fatal: the next tick sends the same difference again, which the hub reads as idempotent.
|
||||
r.logf("[WARN] [holdreport] could not save the reported holds: %v — the next tick repeats the difference", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func sameMap(a, b map[string]string) bool {
|
||||
if len(a) != len(b) {
|
||||
return false
|
||||
}
|
||||
for k, v := range a {
|
||||
if b[k] != v {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func (r *Reporter) logf(format string, args ...interface{}) {
|
||||
if r.logger != nil {
|
||||
r.logger.Printf(format, args...)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,211 @@
|
||||
package holdreport
|
||||
|
||||
import (
|
||||
"io"
|
||||
"log"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/settings"
|
||||
)
|
||||
|
||||
type fakeStore struct {
|
||||
holds map[string]settings.RestoreHold
|
||||
reported map[string]string
|
||||
}
|
||||
|
||||
func (f *fakeStore) ListRestoreHolds() []settings.RestoreHold {
|
||||
out := []settings.RestoreHold{}
|
||||
for _, h := range f.holds {
|
||||
out = append(out, h)
|
||||
}
|
||||
return out
|
||||
}
|
||||
func (f *fakeStore) GetReportedHolds() map[string]string {
|
||||
out := map[string]string{}
|
||||
for k, v := range f.reported {
|
||||
out[k] = v
|
||||
}
|
||||
return out
|
||||
}
|
||||
func (f *fakeStore) SetReportedHolds(m map[string]string) error { f.reported = m; return nil }
|
||||
|
||||
type sent struct {
|
||||
typ, sev string
|
||||
d Details
|
||||
}
|
||||
|
||||
func rec(out *[]sent) Push {
|
||||
return func(t, sev, msg string, d interface{}) { *out = append(*out, sent{t, sev, d.(Details)}) }
|
||||
}
|
||||
|
||||
var t0 = time.Date(2026, 10, 26, 18, 0, 0, 0, time.UTC)
|
||||
|
||||
// The report field (D53): a new hold is sent once as app_held with its stack, reason and since, severity info (the hub
|
||||
// mails nothing on receipt; its reminder after the grace is the mail).
|
||||
func TestD53_ANewHoldIsReportedWithStackReasonAndSince(t *testing.T) {
|
||||
st := &fakeStore{holds: map[string]settings.RestoreHold{
|
||||
"kimai": {Stack: "kimai", At: "2026-10-14T08:00:00Z", Reason: settings.HoldReasonUnhealthyStop, UnhealthyKind: "crash_loop"},
|
||||
}}
|
||||
var out []sent
|
||||
r := New(st, rec(&out), nil)
|
||||
r.Tick(t0)
|
||||
if len(out) != 1 || out[0].typ != EventHeld || out[0].sev != "info" {
|
||||
t.Fatalf("want one info app_held, got %+v", out)
|
||||
}
|
||||
d := out[0].d
|
||||
if d.StackName != "kimai" || d.Reason != "unhealthy_stop" || d.Since != "2026-10-14T08:00:00Z" || d.Why == "" {
|
||||
t.Fatalf("payload: %+v", d)
|
||||
}
|
||||
if st.reported["kimai"] != "unhealthy_stop|2026-10-14T08:00:00Z" {
|
||||
t.Fatalf("reported record: %v", st.reported)
|
||||
}
|
||||
// The next tick inside a day sends nothing.
|
||||
out = nil
|
||||
r.Tick(t0.Add(time.Minute))
|
||||
if len(out) != 0 {
|
||||
t.Fatalf("an unchanged hold was sent again inside a day: %+v", out)
|
||||
}
|
||||
}
|
||||
|
||||
// An event can be lost: every hold still in place is sent again after a day, marked as a repeat.
|
||||
func TestD53_AHoldIsSentAgainEveryDay(t *testing.T) {
|
||||
st := &fakeStore{holds: map[string]settings.RestoreHold{"a": {Stack: "a", At: "2026-10-20T00:00:00Z", Reason: settings.HoldReasonUpdateFailed}}}
|
||||
var out []sent
|
||||
r := New(st, rec(&out), nil)
|
||||
r.Tick(t0)
|
||||
out = nil
|
||||
r.Tick(t0.Add(ResendEvery))
|
||||
if len(out) != 1 || out[0].typ != EventHeld || !out[0].d.Repeat {
|
||||
t.Fatalf("want one repeat app_held after a day, got %+v", out)
|
||||
}
|
||||
}
|
||||
|
||||
// A hold cleared by a person ends it: one app_hold_cleared, then nothing.
|
||||
func TestD53_AClearedHoldSendsClearedOnce(t *testing.T) {
|
||||
st := &fakeStore{holds: map[string]settings.RestoreHold{"a": {Stack: "a", At: "2026-10-20T00:00:00Z", Reason: settings.HoldReasonRestoreMixed}}}
|
||||
var out []sent
|
||||
r := New(st, rec(&out), nil)
|
||||
r.Tick(t0)
|
||||
delete(st.holds, "a")
|
||||
out = nil
|
||||
r.Tick(t0.Add(time.Minute))
|
||||
if len(out) != 1 || out[0].typ != EventCleared || out[0].d.StackName != "a" || out[0].sev != "info" {
|
||||
t.Fatalf("want one info app_hold_cleared for a, got %+v", out)
|
||||
}
|
||||
if len(st.reported) != 0 {
|
||||
t.Fatalf("the reported record still names the cleared hold: %v", st.reported)
|
||||
}
|
||||
out = nil
|
||||
r.Tick(t0.Add(2 * time.Minute))
|
||||
r.Tick(t0.Add(ResendEvery + time.Hour))
|
||||
if len(out) != 0 {
|
||||
t.Fatalf("a cleared hold was reported again: %+v", out)
|
||||
}
|
||||
}
|
||||
|
||||
// A hold cleared while the controller was down is still reported cleared: the record lives in settings.
|
||||
func TestD53_AHoldClearedWhileDownIsReportedAtStart(t *testing.T) {
|
||||
st := &fakeStore{holds: map[string]settings.RestoreHold{}, reported: map[string]string{"gone": "update_failed|2026-10-01T00:00:00Z"}}
|
||||
var out []sent
|
||||
New(st, rec(&out), nil).Tick(t0)
|
||||
if len(out) != 1 || out[0].typ != EventCleared || out[0].d.StackName != "gone" {
|
||||
t.Fatalf("want app_hold_cleared for gone at start, got %+v", out)
|
||||
}
|
||||
}
|
||||
|
||||
// A hold whose reason changes (a put-back that failed becomes a mixed restore) is a new hold for the hub.
|
||||
func TestD53_AChangedReasonIsReportedAgain(t *testing.T) {
|
||||
st := &fakeStore{holds: map[string]settings.RestoreHold{"a": {Stack: "a", At: "2026-10-20T00:00:00Z", Reason: settings.HoldReasonPutBackPending}}}
|
||||
var out []sent
|
||||
r := New(st, rec(&out), nil)
|
||||
r.Tick(t0)
|
||||
st.holds["a"] = settings.RestoreHold{Stack: "a", At: "2026-10-20T01:00:00Z", Reason: settings.HoldReasonRestoreMixed}
|
||||
out = nil
|
||||
r.Tick(t0.Add(time.Minute))
|
||||
if len(out) != 1 || out[0].typ != EventHeld || out[0].d.Reason != "restore_mixed" || out[0].d.Repeat {
|
||||
t.Fatalf("want a new (not repeat) app_held restore_mixed, got %+v", out)
|
||||
}
|
||||
}
|
||||
|
||||
// The legacy empty reason is a restore that failed with its rollback: the hub reads it as restore_failed.
|
||||
func TestD53_TheEmptyReasonIsRestoreFailedOnTheWire(t *testing.T) {
|
||||
st := &fakeStore{holds: map[string]settings.RestoreHold{"db": {Stack: "db", At: "2026-10-20T00:00:00Z"}}}
|
||||
var out []sent
|
||||
New(st, rec(&out), nil).Tick(t0)
|
||||
if len(out) != 1 || out[0].d.Reason != "restore_failed" {
|
||||
t.Fatalf("want reason restore_failed, got %+v", out)
|
||||
}
|
||||
}
|
||||
|
||||
// The seam is wired: main builds the reporter on the real settings and notifier and ticks it from the scheduler. A
|
||||
// reporter nobody ticks is the inert-seam shape (four instances in this project). A commented-out line does not count.
|
||||
func TestD53_TheReporterIsWiredInMain(t *testing.T) {
|
||||
raw, err := os.ReadFile(filepath.Join("..", "..", "cmd", "controller", "main.go"))
|
||||
if err != nil {
|
||||
t.Fatalf("cannot read main.go: %v", err)
|
||||
}
|
||||
live := []string{}
|
||||
for _, l := range strings.Split(string(raw), "\n") {
|
||||
if !strings.HasPrefix(strings.TrimSpace(l), "//") {
|
||||
live = append(live, l)
|
||||
}
|
||||
}
|
||||
src := strings.Join(live, "\n")
|
||||
for _, want := range []string{"holdreport.New(sett, notifier.PushEvent, logger)", `sched.Every("hold-report"`, "holdReporter.Tick(", "holdReporter.SetInstalled("} {
|
||||
if !strings.Contains(src, want) {
|
||||
t.Errorf("main.go lacks %q — the hold report is not wired", want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The reported record survives a restart: it is what lets a hold cleared while the controller was down be reported.
|
||||
func TestD53_TheReportedRecordSurvivesAReload(t *testing.T) {
|
||||
p := filepath.Join(t.TempDir(), "settings.json")
|
||||
s, err := settings.Load(p, log.New(io.Discard, "", 0))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := s.SetReportedHolds(map[string]string{"kimai": "unhealthy_stop|2026-10-14T08:00:00Z"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
s2, err := settings.Load(p, log.New(io.Discard, "", 0))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := s2.GetReportedHolds()["kimai"]; got != "unhealthy_stop|2026-10-14T08:00:00Z" {
|
||||
t.Fatalf("after a reload the record reads %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
// R-1298: an install's hold (crash-loop stop, failed update, waiting put-back) of an app that is NOT installed any more is
|
||||
// not reported — on 9202 four such holds from a hand removal would have mailed the operator weekly about apps that do not
|
||||
// exist. A restore hold stays reported whatever the app's state (R-379: a person clears it). A hold reported earlier and
|
||||
// now skipped is reported cleared. COMPANION RED-PROOF: without the installed check all three holds are reported.
|
||||
func TestR1298_OnlyHoldsOfInstalledAppsAreReported(t *testing.T) {
|
||||
st := &fakeStore{holds: map[string]settings.RestoreHold{
|
||||
"gone": {Stack: "gone", At: "2026-09-24T12:16:24Z", Reason: settings.HoldReasonUnhealthyStop},
|
||||
"gonedb": {Stack: "gonedb", At: "2026-09-24T12:16:24Z", Reason: settings.HoldReasonRestoreFailed},
|
||||
"present": {Stack: "present", At: "2026-10-26T17:37:52Z", Reason: settings.HoldReasonUpdateFailed},
|
||||
}, reported: map[string]string{"gone": "unhealthy_stop|2026-09-24T12:16:24Z"}}
|
||||
var out []sent
|
||||
r := New(st, rec(&out), nil)
|
||||
r.SetInstalled(func(stack string) bool { return stack == "present" })
|
||||
r.Tick(t0)
|
||||
got := map[string]string{}
|
||||
for _, s := range out {
|
||||
got[s.d.StackName] = s.typ
|
||||
}
|
||||
if got["gone"] != EventCleared {
|
||||
t.Errorf("the hold of an app that is not installed must be reported cleared, got %q", got["gone"])
|
||||
}
|
||||
if got["gonedb"] != EventHeld {
|
||||
t.Errorf("a restore hold must stay reported even when the app is gone, got %q", got["gonedb"])
|
||||
}
|
||||
if got["present"] != EventHeld {
|
||||
t.Errorf("an installed app's hold must be reported, got %q", got["present"])
|
||||
}
|
||||
}
|
||||
@@ -1258,6 +1258,9 @@
|
||||
"err.appexport.a_z_kotet_adata_hianyzik_a": "the data of volume %q is missing from the package",
|
||||
"err.appexport.jelszo_hibas_vagy_a_fajl_serult": "wrong password, or the file is damaged",
|
||||
"err.appexport.ket_adatkonyvtar_azonos_nevvel_vegzodik_es": "two data folders end in the same name (%q: %s and %s) — the package could not tell them apart",
|
||||
"err.appexport.kulonleges_fajl": "%s is a special file (a pipe, socket or device) that cannot be saved — the export stopped; nothing was left out silently",
|
||||
"err.appexport.import_foglalt": "a package cannot be loaded right now: another operation is working on the box (%s). Wait until it finishes and try again.",
|
||||
"err.appexport.hdd_path_hibas": "the data-drive path in the package is not valid (“%s”) — the import did not start, nothing changed",
|
||||
"err.appexport.visszafejtes_sikertelen": "decryption failed: %s",
|
||||
"err.backup.whole_restore_space": "There is not enough room on the drive to bring the files back: %.1f GB needed, %.1f GB free. Nothing was changed.",
|
||||
"err.backup.whole_restore_no_proven_copy": "The second drive holds no checked copy that can bring the app back together with its files. Nothing was changed.",
|
||||
@@ -2663,6 +2666,7 @@
|
||||
"api.escrow.too_many_attempts": "Too many failed attempts, try again in 1 minute.",
|
||||
"api.escrow.stage_failed": "Preparing the remote backup password for escrow failed — the process did not start. Try again.",
|
||||
"api.escrow.already_running": "A code is already being created — wait until it finishes.",
|
||||
"api.escrow.recovery_offer_open": "You cannot create a new recovery code right now: first open your old off-site backups with your existing code (Backups page). A new code would replace the key of the old backups.",
|
||||
"api.escrow.code_not_shown": "The code was created but not shown — for security it cannot be fetched again. Start the process again: the new code replaces the old one.",
|
||||
"api.escrow.no_active_job": "There is no active recovery process — start creating the code first.",
|
||||
"api.escrow.confirm_expired": "The time to type the new code back has run out — your previous code still works. Start the process again.",
|
||||
|
||||
@@ -1253,6 +1253,9 @@
|
||||
"err.appexport.a_z_kotet_adata_hianyzik_a": "a(z) %q kötet adata hiányzik a csomagból",
|
||||
"err.appexport.jelszo_hibas_vagy_a_fajl_serult": "jelszó hibás vagy a fájl sérült",
|
||||
"err.appexport.ket_adatkonyvtar_azonos_nevvel_vegzodik_es": "két adatkönyvtár azonos névvel végződik (%q: %s és %s) — a csomag nem tudná megkülönböztetni őket",
|
||||
"err.appexport.kulonleges_fajl": "a(z) %s különleges fájl (cső, foglalat vagy eszköz), ezt nem lehet menteni — az exportálás leállt, semmi nem maradt ki csendben",
|
||||
"err.appexport.import_foglalt": "most nem tölthetsz vissza csomagot: egy másik művelet dolgozik a dobozon (%s). Várd meg, míg befejeződik, és próbáld újra.",
|
||||
"err.appexport.hdd_path_hibas": "a csomag adatmeghajtó-útvonala hibás („%s”) — a betöltés el sem indult, semmi nem változott",
|
||||
"err.appexport.visszafejtes_sikertelen": "visszafejtés sikertelen: %s",
|
||||
"err.backup.whole_restore_space": "Nincs elég hely a meghajtón a fájlok visszaállításához: %.1f GB kellene, %.1f GB szabad. Semmi nem változott.",
|
||||
"err.backup.whole_restore_no_proven_copy": "A második meghajtón nincs olyan ellenőrzött másolat, amely az alkalmazást a fájljaival együtt vissza tudná hozni. Semmi nem változott.",
|
||||
@@ -2651,6 +2654,7 @@
|
||||
"api.escrow.too_many_attempts": "Túl sok sikertelen próbálkozás, próbáld újra 1 perc múlva.",
|
||||
"api.escrow.stage_failed": "A távoli mentés jelszavának letéti előkészítése nem sikerült — a folyamat nem indult el. Próbáld újra.",
|
||||
"api.escrow.already_running": "Egy kódkészítés már folyamatban van — várd meg, míg befejeződik.",
|
||||
"api.escrow.recovery_offer_open": "Most nem készíthetsz új helyreállítási kódot: előbb nyisd meg a régi, házon kívüli mentéseidet a meglévő kódoddal (Mentések oldal). Egy új kód lecserélné a régi mentések kulcsát.",
|
||||
"api.escrow.code_not_shown": "A kód létrejött, de nem lett megjelenítve — biztonsági okból újra nem kérhető le. Indítsd újra a folyamatot: az új kód a régit érvényteleníti.",
|
||||
"api.escrow.no_active_job": "Nincs aktív helyreállítási folyamat — előbb indítsd el a kódkészítést.",
|
||||
"api.escrow.confirm_expired": "Lejárt az idő az új kód visszaírására — a régi kódod továbbra is érvényes. Indítsd újra a folyamatot.",
|
||||
|
||||
@@ -20,10 +20,12 @@ import (
|
||||
)
|
||||
|
||||
// Pinned image tags — NEVER ":latest" (a floating tag breaks reproducible golden bakes and lets the
|
||||
// deployed version drift). Verified to resolve on Docker Hub before baking.
|
||||
// deployed version drift). Verified to resolve on Docker Hub before baking. cloudflared follows Cloudflare's security
|
||||
// notices (felhom.eu `01` „Tunnel placement"): 2026.10.3 since the notice of 2026-10-20 — before 2026.10.2 the image
|
||||
// trusts eight eMudhra (emSign) intermediates with weak keys.
|
||||
const (
|
||||
TraefikImage = "traefik:v3.7.13"
|
||||
CloudflaredImage = "cloudflare/cloudflared:2026.9.3"
|
||||
CloudflaredImage = "cloudflare/cloudflared:2026.10.3"
|
||||
FileBrowserImage = "gtstef/filebrowser:1.5.6-stable"
|
||||
// FileBrowserImportMount is the in-container mount point NAME for the canonical drop-zone
|
||||
// (R-75): the bind lands at /srv/<this>. ASCII and space-free on purpose — it appears in a
|
||||
|
||||
@@ -232,6 +232,10 @@ type Settings struct {
|
||||
// out-of-memory storm (RFC3339). Survives the hold being lifted by Start, so a second stop within
|
||||
// 24 h is known to be a second one.
|
||||
UnhealthyStops map[string]string `json:"unhealthy_stops,omitempty"`
|
||||
// ReportedHolds (D53, `08` §10) — the holds last told to the hub as app_held: stack → „reason|at". The hold
|
||||
// reporter (internal/holdreport) compares it with RestoreHolds to send app_held for a new or changed hold and
|
||||
// app_hold_cleared for one that is gone — also one cleared while the controller was down, because it is kept here.
|
||||
ReportedHolds map[string]string `json:"reported_holds,omitempty"`
|
||||
|
||||
// Cross-drive restic repo password (auto-generated on first use)
|
||||
CrossDriveResticPassword string `json:"cross_drive_restic_password,omitempty"`
|
||||
@@ -1998,6 +2002,35 @@ const (
|
||||
HoldReasonPutBackPending = "putback_pending"
|
||||
)
|
||||
|
||||
// WireReason is the hold's reason as the hub reads it (D53): the empty legacy value is named restore_failed.
|
||||
func (h RestoreHold) WireReason() string {
|
||||
if h.Reason == HoldReasonRestoreFailed {
|
||||
return "restore_failed"
|
||||
}
|
||||
return h.Reason
|
||||
}
|
||||
|
||||
// Why says which operation put a hold in place and what lifts it — one sentence per reason (R-1286). English,
|
||||
// operator-facing: the controller's own -list-restore-holds output and the hub's „Held apps" list (D53) both use it.
|
||||
func (h RestoreHold) Why() string {
|
||||
switch h.Reason {
|
||||
case HoldReasonUpdateFailed:
|
||||
return "held after a failed update (" + h.At + ") — restore it from its backup to start it"
|
||||
case HoldReasonUnhealthyStop:
|
||||
kind := h.UnhealthyKind
|
||||
if kind == "" {
|
||||
kind = "unhealthy"
|
||||
}
|
||||
return "stopped by the box for a " + strings.ReplaceAll(kind, "_", " ") + " (" + h.At + ") — the household's Start gives it one more try"
|
||||
case HoldReasonRestoreMixed:
|
||||
return "held after an off-site restore failed its database replay (" + h.At + ") — clear the hold to start it"
|
||||
case HoldReasonPutBackPending:
|
||||
return "held until the pre-restore copy is poured back (" + h.At + ") — the put-back starts it"
|
||||
default:
|
||||
return "held after a failed restore whose rollback also failed (" + h.At + ") — clear the hold to start it"
|
||||
}
|
||||
}
|
||||
|
||||
// SetRestoreHold records a hold. Modelled on SetDisconnected: a condition, plus what it is holding.
|
||||
func (s *Settings) SetRestoreHold(h RestoreHold) error {
|
||||
s.mu.Lock()
|
||||
@@ -2031,6 +2064,32 @@ func (s *Settings) ListRestoreHolds() []RestoreHold {
|
||||
return out
|
||||
}
|
||||
|
||||
// GetReportedHolds returns a copy of the holds last reported to the hub (D53): stack → „reason|at".
|
||||
func (s *Settings) GetReportedHolds() map[string]string {
|
||||
s.mu.RLock()
|
||||
defer s.mu.RUnlock()
|
||||
out := make(map[string]string, len(s.ReportedHolds))
|
||||
for k, v := range s.ReportedHolds {
|
||||
out[k] = v
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// SetReportedHolds replaces the record of the holds reported to the hub (D53). An empty map removes the field.
|
||||
func (s *Settings) SetReportedHolds(m map[string]string) error {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
if len(m) == 0 {
|
||||
s.ReportedHolds = nil
|
||||
} else {
|
||||
s.ReportedHolds = make(map[string]string, len(m))
|
||||
for k, v := range m {
|
||||
s.ReportedHolds[k] = v
|
||||
}
|
||||
}
|
||||
return s.save()
|
||||
}
|
||||
|
||||
// ClearRestoreHold removes a hold. Returns false when there was none, so a caller can tell "cleared"
|
||||
// from "there was nothing to clear" rather than reporting success either way.
|
||||
func (s *Settings) ClearRestoreHold(stack string) (bool, error) {
|
||||
|
||||
@@ -326,3 +326,33 @@ func TestEnsureForwardedHeaders_R1033_RewritesAnOldFile(t *testing.T) {
|
||||
t.Fatal("the rewritten file must carry the https-only redirect")
|
||||
}
|
||||
}
|
||||
|
||||
// A controller update that changes only cloudflared's IMAGE recreates the running container on the new image (the
|
||||
// 2026.10.3 security bump rides this: no other step moves a household's tunnel). The control: the same render again
|
||||
// is a no-op.
|
||||
func TestEnsureCloudflared_ANewImageRecreatesTheTunnel(t *testing.T) {
|
||||
state := stubDocker(t)
|
||||
touch(t, filepath.Join(state, "running-cloudflared"), "")
|
||||
m, calls := tunnelTestManager(t, "")
|
||||
dir := t.TempDir()
|
||||
cur, _ := infra.RenderCloudflared(infra.CloudflaredData{CFTunnelToken: "tok-test", Tunnel: true})
|
||||
old := strings.Replace(cur["docker-compose.yml"].Content, "image: "+infra.CloudflaredImage, "image: cloudflare/cloudflared:2026.9.3", 1)
|
||||
if old == cur["docker-compose.yml"].Content {
|
||||
t.Fatal("the render has no `image:` line to age — the test would prove nothing")
|
||||
}
|
||||
touch(t, filepath.Join(dir, "docker-compose.yml"), old)
|
||||
|
||||
if err := m.ensureCloudflared(dir, true); err != nil {
|
||||
t.Fatalf("ensureCloudflared: %v", err)
|
||||
}
|
||||
cmp, _ := os.ReadFile(filepath.Join(dir, "docker-compose.yml"))
|
||||
if !strings.Contains(string(cmp), "image: "+infra.CloudflaredImage) {
|
||||
t.Fatalf("the compose must carry %s:\n%s", infra.CloudflaredImage, cmp)
|
||||
}
|
||||
if len(*calls) != 1 || (*calls)[0].args != "up -d" {
|
||||
t.Fatalf("want one `up -d` (compose recreates on the new image), got %+v", *calls)
|
||||
}
|
||||
if err := m.ensureCloudflared(dir, true); err != nil || len(*calls) != 1 {
|
||||
t.Fatalf("an unchanged image must not recreate the tunnel again; err %v calls %+v", err, *calls)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -217,10 +217,31 @@ func readKeptMarker(dir string) *KeptMarker {
|
||||
}
|
||||
|
||||
// liveDriveBinds is every drive folder an INSTALLED app binds — never kept data.
|
||||
func (m *Manager) liveDriveBinds() []string {
|
||||
//
|
||||
// R-1220 — FAIL CLOSED ON AN UNREADABLE app.yaml. A parse error gives a nil AppConfig, so ScanStacks reads the app as
|
||||
// „not installed"; before this, its live folder was then listed as kept data and offered for deletion. An app whose
|
||||
// app.yaml EXISTS but cannot be read counts as installed: its HDD_PATH is unknown, so every folder its compose would bind
|
||||
// on ANY of the given drives is live. One WARN per app per listing names it. Pinned by
|
||||
// TestR1220_AnUnreadableAppYamlKeepsTheFolderLive.
|
||||
func (m *Manager) liveDriveBinds(drives []string) []string {
|
||||
var out []string
|
||||
for _, st := range m.GetStacks() {
|
||||
if st.ComposePath == "" {
|
||||
continue
|
||||
}
|
||||
if !st.Deployed || st.AppConfig == nil {
|
||||
if !st.Deployed && !appYAMLUnreadable(filepath.Dir(st.ComposePath)) {
|
||||
continue // not installed: no app.yaml, or one that reads as not deployed
|
||||
}
|
||||
m.logger.Printf("[WARN] [stacks] kept: %s has an app.yaml that cannot be read — it counts as INSTALLED, so its drive folders are never listed as kept data (R-1220)", st.Name)
|
||||
for _, d := range drives {
|
||||
if d = strings.TrimSpace(d); d == "" {
|
||||
continue
|
||||
}
|
||||
for _, p := range ParseComposeHDDMounts(st.ComposePath, filepath.Clean(d)) {
|
||||
out = append(out, filepath.Clean(p))
|
||||
}
|
||||
}
|
||||
continue
|
||||
}
|
||||
hdd := strings.TrimSpace(st.AppConfig.Env["HDD_PATH"])
|
||||
@@ -234,6 +255,15 @@ func (m *Manager) liveDriveBinds() []string {
|
||||
return out
|
||||
}
|
||||
|
||||
// appYAMLUnreadable is true when the stack folder holds an app.yaml that LoadAppConfig cannot turn into a config — the
|
||||
// R-1220 case. A missing app.yaml is false (a template that was never installed, or a removed app).
|
||||
func appYAMLUnreadable(stackDir string) bool {
|
||||
if _, err := os.Stat(filepath.Join(stackDir, "app.yaml")); err != nil {
|
||||
return false
|
||||
}
|
||||
return LoadAppConfig(stackDir) == nil
|
||||
}
|
||||
|
||||
func overlaps(a, b string) bool {
|
||||
sep := string(filepath.Separator)
|
||||
return a == b || strings.HasPrefix(a, b+sep) || strings.HasPrefix(b, a+sep)
|
||||
@@ -284,7 +314,7 @@ var sizeFn = getDirSizeBytes
|
||||
// ListKept lists every kept item on the given drive roots: the dated folders under <drive>/kept/<app>/,
|
||||
// and each <drive>/appdata/<x> that holds something and that no installed app binds. Sorted newest first.
|
||||
func (m *Manager) ListKept(drives []string) []KeptItem {
|
||||
live := m.liveDriveBinds()
|
||||
live := m.liveDriveBinds(drives)
|
||||
var out []KeptItem
|
||||
seen := map[string]bool{}
|
||||
for _, d := range drives {
|
||||
|
||||
@@ -0,0 +1,36 @@
|
||||
package stacks
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// R-1220: an installed app whose app.yaml cannot be read must count as INSTALLED (fail closed). Before the fix a parse
|
||||
// error gave a nil AppConfig, so the app looked „not installed", its live drive folder was listed as kept data — and
|
||||
// the household's Delete was offered on it. Red-proved against the code before the fix (the folder was listed).
|
||||
func TestR1220_AnUnreadableAppYamlKeepsTheFolderLive(t *testing.T) {
|
||||
m, drive := keptManager(t)
|
||||
app := filepath.Join(filepath.Dir(m.stacks["cloudapp"].ComposePath))
|
||||
// An app.yaml that exists but does not parse (a half-written file, a disk error).
|
||||
must(t, os.WriteFile(filepath.Join(app, "app.yaml"), []byte("deployed: true\nenv: [this is: not: yaml\n"), 0o600))
|
||||
must(t, m.ScanStacks())
|
||||
items := m.ListKept([]string{drive})
|
||||
for _, it := range items {
|
||||
if it.App == "cloudapp" || it.Path == filepath.Join(drive, "appdata", "cloudapp") {
|
||||
t.Fatalf("an app with an unreadable app.yaml had its live folder listed as kept: %+v", it)
|
||||
}
|
||||
}
|
||||
if _, err := m.DeleteKept([]string{drive}, filepath.Join(drive, "appdata", "cloudapp")); err == nil {
|
||||
t.Fatal("Delete was allowed on the folder of an app whose app.yaml cannot be read")
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(drive, "appdata/cloudapp/user1/file.txt")); err != nil {
|
||||
t.Fatalf("the live folder's file is gone: %v", err)
|
||||
}
|
||||
// The control: with NO app.yaml at all (a real leftover), the folder IS listed.
|
||||
must(t, os.Remove(filepath.Join(app, "app.yaml")))
|
||||
must(t, m.ScanStacks())
|
||||
if items := m.ListKept([]string{drive}); len(items) != 1 || items[0].App != "cloudapp" {
|
||||
t.Fatalf("a real leftover (no app.yaml) is not listed: %+v", items)
|
||||
}
|
||||
}
|
||||
@@ -53,6 +53,11 @@ func (s *Server) escrowBannerVisible(r *http.Request) bool {
|
||||
if r == nil || !s.hasAdminSession(r) || !s.escrowPaused() {
|
||||
return false
|
||||
}
|
||||
// R-1219: during a recovery offer a NEW code would replace the key that opens the old off-site copy (R-199) — the
|
||||
// backups page hides the ceremony then, and so does this bar. Pinned by TestR1219_NoCeremonyDuringARecoveryOffer.
|
||||
if s.recoveryOffer() {
|
||||
return false
|
||||
}
|
||||
// R-546: while the agent says the ceremony cannot run yet, do not urge the household into it.
|
||||
// Unknown readiness keeps the bar (fail loud).
|
||||
if ready, known := s.escrowReadiness(r.Context(), false); known && !ready {
|
||||
|
||||
@@ -213,8 +213,20 @@ func (s *Server) escrowStartAPIHandler(w http.ResponseWriter, r *http.Request) {
|
||||
return
|
||||
}
|
||||
|
||||
// (2) Re-stage-first (only when offsite is configured — Scenario C boxes skip it).
|
||||
if s.backupMgr != nil && s.backupMgr.OffboxConfigured() {
|
||||
// (1c) R-1219: no ceremony during a recovery offer — a new code would replace the key that opens the old off-site
|
||||
// copy (R-199). The page hides the form; a direct POST (or the old top-bar link) is refused here, before anything is
|
||||
// staged or started. Pinned by TestR1219_NoCeremonyDuringARecoveryOffer.
|
||||
if s.recoveryOffer() {
|
||||
s.logger.Printf("[INFO] [web] escrow start refused: a recovery offer is open (R-1219)")
|
||||
escrowJSON(w, http.StatusConflict, nil, s.msg(r, "api.escrow.recovery_offer_open"))
|
||||
return
|
||||
}
|
||||
|
||||
// (2) Re-stage-first whenever the repository password FILE is on the box (R-1218) — not only when off-site is
|
||||
// configured: a cleared or switched-off target keeps the file, and a ceremony that seals no password makes the hub
|
||||
// demote the household's good package. A box with no password (Scenario C, R-241's awaiting key) skips it.
|
||||
// Pinned by TestR1218_TheCeremonyStagesWhenThePasswordFileExists.
|
||||
if s.backupMgr != nil && s.backupMgr.OffboxRepoPasswordOnBox() {
|
||||
if err := s.escrowStage(r.Context()); err != nil {
|
||||
s.logger.Printf("[WARN] [web] escrow start: re-stage failed (ceremony NOT started): %v", err) // err carries no secret
|
||||
escrowJSON(w, http.StatusBadGateway, nil, s.msg(r, "api.escrow.stage_failed"))
|
||||
|
||||
@@ -0,0 +1,56 @@
|
||||
package web
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// R-1218: a recovery ceremony run while off-site is switched OFF (or its target cleared) must still stage the off-site
|
||||
// repository password when the password FILE is on the box — ClearOffboxTarget keeps that file on purpose. Before the
|
||||
// fix the stage ran only when OffboxConfigured() was true, the agent sealed no password, uploaded sha "", and the hub
|
||||
// demoted the household's good package: the family's code no longer opened its off-site copy. Red-proved against the
|
||||
// OffboxConfigured() gate (order „preflight,start", no stage).
|
||||
func TestR1218_TheCeremonyStagesWhenThePasswordFileExists(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
off func(h *escrowWizardHarness)
|
||||
}{
|
||||
{"off-site switched off", func(h *escrowWizardHarness) {
|
||||
tg := h.sett.GetOffboxTarget()
|
||||
tg.Enabled = false
|
||||
if err := h.sett.SetOffboxTarget(tg); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}},
|
||||
{"off-site target cleared", func(h *escrowWizardHarness) {
|
||||
if err := h.sett.SetOffboxTarget(nil); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
h := newEscrowWizardHarness(t)
|
||||
h.configureOffbox(t, "escrowed")
|
||||
tc.off(h)
|
||||
if h.m.OffboxConfigured() {
|
||||
t.Fatal("setup: off-site should read as not configured")
|
||||
}
|
||||
w := postStart(t, h.s, wizardPassword)
|
||||
if w.Code != http.StatusOK {
|
||||
t.Fatalf("start: %d %s", w.Code, w.Body.String())
|
||||
}
|
||||
if got := strings.Join(h.order, ","); got != "preflight,stage,start" {
|
||||
t.Fatalf("call order = %q — the repository password on the box was NOT staged, so the new package would cover no key", got)
|
||||
}
|
||||
})
|
||||
}
|
||||
// The control: no password file at all (a box that never had off-site, R-241's awaiting-key shape) → nothing staged.
|
||||
h := newEscrowWizardHarness(t)
|
||||
if w := postStart(t, h.s, wizardPassword); w.Code != http.StatusOK {
|
||||
t.Fatalf("start without off-site: %d %s", w.Code, w.Body.String())
|
||||
}
|
||||
if got := strings.Join(h.order, ","); got != "preflight,start" {
|
||||
t.Fatalf("a box with no repository password staged something: %q", got)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,69 @@
|
||||
package web
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// offerShapeC puts the harness box in the recovery offer, shape (c): the hub holds a package for a DIFFERENT repository
|
||||
// key than the one the box uses — the old off-site copy opens only with the household's EXISTING recovery code.
|
||||
func offerShapeC(t *testing.T, h *escrowWizardHarness) {
|
||||
t.Helper()
|
||||
h.configureOffbox(t, "pending")
|
||||
if err := h.sett.SetHubEscrowIdentityPresent(true); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := h.sett.SetHubEscrowKeySHA256("0000000000000000000000000000000000000000000000000000000000000000", "2026-10-26T20:00:00Z"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !h.m.OffsiteRecoveryOffer() {
|
||||
t.Fatal("setup: the box should be in the recovery offer")
|
||||
}
|
||||
}
|
||||
|
||||
// R-1219: during a recovery offer a NEW recovery code would replace the key that opens the old off-site copy (R-199).
|
||||
// The backups page already hid the ceremony; the global top bar still linked to it and the start handler ran it. Now
|
||||
// the bar is not shown and the start handler answers 409 with a sentence in the household's language, BEFORE anything
|
||||
// is staged or started. COMPANION RED-PROOF: before the fix the bar was visible and the start answered 200
|
||||
// („preflight,stage,start").
|
||||
func TestR1219_NoCeremonyDuringARecoveryOffer(t *testing.T) {
|
||||
h := newEscrowWizardHarness(t)
|
||||
offerShapeC(t, h)
|
||||
if !h.s.escrowPaused() {
|
||||
t.Fatal("setup: the tier should read as paused (pending) — the bar would otherwise be hidden for another reason")
|
||||
}
|
||||
r := httptest.NewRequest("GET", "/", nil)
|
||||
r.AddCookie(&http.Cookie{Name: sessionCookieName, Value: newTestSession(h.s)})
|
||||
if h.s.escrowBannerVisible(r) {
|
||||
t.Error("the top bar urges the recovery-code ceremony during a recovery offer")
|
||||
}
|
||||
w := postStart(t, h.s, wizardPassword)
|
||||
if w.Code != http.StatusConflict {
|
||||
t.Fatalf("start during an offer = %d (%s), want 409", w.Code, w.Body.String())
|
||||
}
|
||||
if !strings.Contains(w.Body.String(), "helyre") { // ASCII fragment of the Hungarian sentence (ui-hungarian rule)
|
||||
t.Errorf("the refusal does not say why in Hungarian: %s", w.Body.String())
|
||||
}
|
||||
for _, step := range h.order {
|
||||
if step == "stage" || step == "start" {
|
||||
t.Fatalf("the agent was reached during an offer: %v", h.order)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The control for the bar half: the SAME paused box with no recovery offer, the same logged-in request — the bar IS
|
||||
// shown. Without this, a bar hidden for any other reason (no session, readiness) would pass the test above.
|
||||
func TestR1219_TheBarStillShowsWithoutAnOffer(t *testing.T) {
|
||||
h := newEscrowWizardHarness(t)
|
||||
h.configureOffbox(t, "pending")
|
||||
if h.m.OffsiteRecoveryOffer() {
|
||||
t.Fatal("setup: no offer expected")
|
||||
}
|
||||
r := httptest.NewRequest("GET", "/", nil)
|
||||
r.AddCookie(&http.Cookie{Name: sessionCookieName, Value: newTestSession(h.s)})
|
||||
if !h.s.escrowBannerVisible(r) {
|
||||
t.Fatal("the bar is hidden on a paused box with no offer — the offer test above would prove nothing")
|
||||
}
|
||||
}
|
||||
@@ -328,7 +328,11 @@
|
||||
"err.backup.restore_image_gone": "BORN AS A KEY (D48, operator ruling 2026-10-25; 09 decision 257) -- a restore downloads first: the refusal when an image of the backup's version cannot be downloaded (deleted by its maker, or the registry busy or unreachable); the app untouched. Pinned by internal/backup/restore_pull_test.go TestD48_*.",
|
||||
"err.backup.restore_image_unreachable": "BORN AS A KEY (D48, operator ruling 2026-10-25; 09 decision 257) -- a restore downloads first: the refusal when an image of the backup's version cannot be downloaded (deleted by its maker, or the registry busy or unreachable); the app untouched. Pinned by internal/backup/restore_pull_test.go TestD48_*.",
|
||||
"kept.backup.future_note": "BORN AS A KEY (R-1271, brief 2026-10-25 Part 4) -- a copy dated in the future is never the newest; when it is the only copy the kept-data question and the kept list say its date is in the future. Pinned by internal/backup/kept_pick_test.go and internal/api/r1271_future_copy_test.go.",
|
||||
"kept.choice.use.future": "BORN AS A KEY (R-1271, brief 2026-10-25 Part 4) -- a copy dated in the future is never the newest; when it is the only copy the kept-data question and the kept list say its date is in the future. Pinned by internal/backup/kept_pick_test.go and internal/api/r1271_future_copy_test.go."
|
||||
"kept.choice.use.future": "BORN AS A KEY (R-1271, brief 2026-10-25 Part 4) -- a copy dated in the future is never the newest; when it is the only copy the kept-data question and the kept list say its date is in the future. Pinned by internal/backup/kept_pick_test.go and internal/api/r1271_future_copy_test.go.",
|
||||
"api.escrow.recovery_offer_open": "BORN AS A KEY (R-1219, brief 2026-10-26 night Part 3) -- the escrow start handler refuses during a recovery offer. Pinned by TestR1219_NoCeremonyDuringARecoveryOffer.",
|
||||
"err.appexport.kulonleges_fajl": "BORN AS A KEY (R-1187 a, brief 2026-10-26 night Part 4) -- the export refuses a FIFO, socket or device, naming the path. Pinned by TestR1187a_AFifoFailsTheExportNamingThePath.",
|
||||
"err.appexport.import_foglalt": "BORN AS A KEY (R-1187 d, brief 2026-10-26 night Part 4) -- an import is refused while another operation holds the box. Pinned by TestR1187d_AnImportWaitsForARestore.",
|
||||
"err.appexport.hdd_path_hibas": "BORN AS A KEY (R-1187 f, brief 2026-10-26 night Part 4) -- a bundle with an empty or relative HDD_PATH is refused before anything is touched. Pinned by TestR1187f_AnEmptyOrRelativeHDDPathIsRefused."
|
||||
},
|
||||
"flash.share.already_on": "A megosztás már be van kapcsolva.",
|
||||
"flash.share.enable_failed": "A megosztás bekapcsolása nem sikerült.",
|
||||
|
||||
@@ -141,6 +141,28 @@ run_build_in absent "$CLEAN_REPO" 0.999.0 --push
|
||||
check "ahead of origin/main -> refuses" 1 "not at origin/main" "FAKE-DOCKER-REACHED"
|
||||
rm -rf "$CLEAN_REPO"
|
||||
|
||||
# ── OR AT THE PUSHED TAG v<VERSION> (v0.345.1, 2026-10-26) ───────────────────────────────────────
|
||||
# A patch release off a released commit while main holds unreleased work. The tag must be on the SERVER, name THIS version,
|
||||
# and point at HEAD. RED-PROOF: with the tag branch accepting any tag the three refusals fail; with it removed the
|
||||
# first case fails.
|
||||
TAG_REPO="$(scratch_repo)"; TAG_ORIGIN="$(mktemp -d)"
|
||||
git init -q --bare "$TAG_ORIGIN" 2>/dev/null; git -C "$TAG_REPO" remote add origin "$TAG_ORIGIN"
|
||||
echo y > "$TAG_REPO/controller/f2"
|
||||
git -C "$TAG_REPO" add -A >/dev/null; git -C "$TAG_REPO" commit -qm fix >/dev/null
|
||||
git -C "$TAG_REPO" tag -a v0.999.1 -m t; git -C "$TAG_REPO" push -q origin v0.999.1 2>/dev/null
|
||||
run_build_in absent "$TAG_REPO" 0.999.1 --push
|
||||
check "at the pushed tag v<VERSION> -> proceeds" "$rc" "at the pushed tag v0.999.1" "refusing to push"
|
||||
run_build_in absent "$TAG_REPO" 0.999.2 --push
|
||||
check "a pushed tag of ANOTHER version -> refuses" 1 "nor at a pushed tag v0.999.2" "FAKE-DOCKER-REACHED"
|
||||
git -C "$TAG_REPO" tag v0.999.3
|
||||
run_build_in absent "$TAG_REPO" 0.999.3 --push
|
||||
check "a tag only on this clone -> refuses" 1 "nor at a pushed tag v0.999.3" "FAKE-DOCKER-REACHED"
|
||||
echo z > "$TAG_REPO/controller/f3"
|
||||
git -C "$TAG_REPO" add -A >/dev/null; git -C "$TAG_REPO" commit -qm later >/dev/null
|
||||
run_build_in absent "$TAG_REPO" 0.999.1 --push
|
||||
check "the pushed tag on another commit -> refuses" 1 "nor at a pushed tag v0.999.1" "FAKE-DOCKER-REACHED"
|
||||
rm -rf "$TAG_REPO" "$TAG_ORIGIN"
|
||||
|
||||
NOT_A_REPO="$(mktemp -d)"; mkdir -p "$NOT_A_REPO/controller"
|
||||
run_build_in absent "$NOT_A_REPO" 0.999.0 --push
|
||||
check "not a git checkout -> refuses" 1 "not a git checkout" "FAKE-DOCKER-REACHED"
|
||||
|
||||
Reference in New Issue
Block a user