Compare commits
2 Commits
main
..
f35b382c6d
| Author | SHA1 | Date | |
|---|---|---|---|
 admin
|
f35b382c6d | ||
|
admin
|
2b46619e15 |
@@ -1,43 +0,0 @@
|
||||
---
|
||||
paths: ["controller/internal/agentapi/**"]
|
||||
---
|
||||
|
||||
# Coupling to the host agent — felhom-controller
|
||||
|
||||
`internal/agentapi` is **the disk seam**: the pinned-TLS client to the host agent's per-guest local
|
||||
API. The controller holds no Proxmox credentials; everything disk/host/Proxmox goes through here.
|
||||
|
||||
## Declaring a coupled feature
|
||||
|
||||
Controller behaviour that depends on a specific agent version needs **all three**, or it ships broken
|
||||
on an older box:
|
||||
|
||||
1. a `featureProbes` table row in `internal/agentapi/features.go`
|
||||
2. a `Supports` gate call **at the feature's entry point** — not somewhere on the path to it
|
||||
3. `MinAgent: X.Y.Z` in the CHANGELOG entry header
|
||||
|
||||
Rules: `felhom.eu/documentation/runbooks/publish-train-rules.md`.
|
||||
|
||||
## Never push a controller past the agent it depends on
|
||||
|
||||
The R-216 guard compared the box's agent against the **golden's** MinAgent while serving a **floor**
|
||||
that could point elsewhere. Raise a floor above the vouched golden — which the day-0 runbook
|
||||
recommends and a per-customer override makes trivial — and the guard checks a version it is not
|
||||
serving. A box then landed on a controller needing a newer agent, and its customer was told a correct
|
||||
recovery code was wrong.
|
||||
|
||||
**A floor above the vouched golden is HELD, with its own reason** (hub v0.97.0).
|
||||
|
||||
## Distinguish "could not reach" from "wrong answer"
|
||||
|
||||
A failed bundle FETCH must not be reported to a customer as a bad recovery code. Classify by **value**
|
||||
(`ErrBundleFetch` → HTTP 502), never by error string — a string is not something a caller can branch
|
||||
on. Unknown class → neutral message, never the typing message.
|
||||
|
||||
<!--
|
||||
R-224, measured live 2026-08-05 (CAMPAIGN-11 F3/F4) with a correct current code: 0.0556 s with the
|
||||
hub firewalled off and 0.0299 s with the agent stopped, against ~1.0 s for a genuine unseal — the
|
||||
machine accused the customer of something it had not attempted. A green test named this exact
|
||||
consequence since v0.125.0 and did not prevent it, because it asserted this package's error STRING
|
||||
one layer below where the merge happened. Fixed agent v0.126.0 + controller v0.202.0.
|
||||
-->
|
||||
@@ -1,41 +0,0 @@
|
||||
---
|
||||
paths: ["controller/internal/backup/**", "controller/internal/appbackup/**", "controller/internal/recovery/**", "controller/internal/appexport/**", "controller/internal/quiesce/**"]
|
||||
---
|
||||
|
||||
# Backup, recovery units and export — felhom-controller
|
||||
|
||||
## Assert the consequence across the whole run, not the mechanism inside one function
|
||||
|
||||
The R-181 recovery-unit refusal claimed *"the previous unit is untouched and NOTHING was deleted"*.
|
||||
*Nothing deleted* held; **untouched was measured false** — the floor was checked ONLY in
|
||||
`captureAllRecoveryUnits`, while the two dump legs wrote the bulk into the same tree first and
|
||||
unguarded, so a 182,272 B tar became 2,147,666,432 B under a manifest that had not moved. A full
|
||||
green suite plus three of its own red-proofs missed it, because every one asserted the mechanism
|
||||
inside `captureAllRecoveryUnits`.
|
||||
|
||||
**The test that catches this class: fingerprint the tree before and after the whole backup run, and
|
||||
compare.** Full doctrine and the other eight instances: the `felhom-testing` skill.
|
||||
|
||||
## Presence is not success
|
||||
|
||||
A timestamp recording an **attempt** must never be read as evidence of a **result**. Where a status
|
||||
field travels alongside a timestamp, the verdict consults both — or the timestamp records only
|
||||
successes. Ask of any timestamp: *what exactly must have happened for this to be set?* If the answer
|
||||
is "we tried", it cannot answer "did it work".
|
||||
|
||||
**Corollary:** when a verdict changes which field it counts from, the alarm text has to change with
|
||||
it. `last run 8h ago` while alarming on a six-day-old success turns a true alarm into one the
|
||||
operator dismisses.
|
||||
|
||||
<!--
|
||||
Two instances. F-CRIT-2: a phantom snapshot's ctime set tier freshness — an aborted 1-byte upload
|
||||
made the tier look backed up. R-100: LastRun is written on failure, so a nightly-failing offsite
|
||||
tier kept the staleness clock fresh forever.
|
||||
-->
|
||||
|
||||
## Storage keys and paths
|
||||
|
||||
- Never guess a persisted key — it is `offbox`, not `offbox_target` (R-7b).
|
||||
- `.fab` export/import uses strict segment validation; bundles from controller ≤0.124.0 are hollow.
|
||||
- Recovery-unit restore and tier-2 copies share `appbackup`'s path primitives — change them there,
|
||||
once, not per caller.
|
||||
@@ -1,54 +0,0 @@
|
||||
---
|
||||
paths: ["controller/**/*.go", "controller/**/*.html", "controller/**/*.css", "controller/scripts/**"]
|
||||
---
|
||||
|
||||
# Gates and logging — felhom-controller
|
||||
|
||||
## The ONE entry point
|
||||
|
||||
**Run `python3 controller/scripts/controller_gates.py` (from `controller/`) after ANY change in this
|
||||
repo.** It runs all seven local gates — `template_id_gate`, `emoji_gate`, `native_confirm_gate`,
|
||||
`offbox_rename_gate`, `app_row_dedup_gate`, `mojibake_gate`, `docker_run_volume_path_gate` — plus
|
||||
`reuse_refs_check` and `instructions_gate` on the repo root, streaming each gate's own output and
|
||||
exiting non-zero if any fails.
|
||||
|
||||
- `--fast` selects the gates that touch no network and no container runtime; today that is all of them.
|
||||
- **A missing gate script is a FAILURE, never a skip.**
|
||||
- **The shared `reuse_refs_check.py` and `instructions_gate.py` live in `felhom.eu/scripts/` and are
|
||||
never copied here** — a copy would recreate the drift they detect; an absent sibling clone FAILS.
|
||||
- **The pre-push hook** (`.githooks/pre-push`) runs it with `--fast` and refuses a failing push. It is
|
||||
per-clone — switch it on once with `git config core.hooksPath .githooks`, and a manual run WARNS
|
||||
when this clone is unarmed. `git push --no-verify` bypasses it deliberately; **say so in the session
|
||||
report when you use it** — CI re-runs the same entry point on every push and **emails the operator
|
||||
on failure**, so a bypass is noticed even though it is not blocked (R-168, CLOSED 2026-08-02).
|
||||
|
||||
<!--
|
||||
WHY A RUNNER AND NOT SEVEN INVOCATIONS (2026-08-02, R-29) — rationale, not a directive.
|
||||
A census of all thirteen gates across the four repos found that every check a CLAUDE.md named was
|
||||
passing, and two of the four nobody is told to run were failing. This repo's CLAUDE.md used to name
|
||||
two of the seven; the other five were reachable only through a line in REUSE.md, and
|
||||
docker_run_volume_path_gate.py was RED. The single-entry-point shape is the only one that
|
||||
demonstrably gets run. app-catalog-felhom.eu/scripts/catalog_gates.py is the canonical version of
|
||||
the runner (R-161); repo_gates.py copies it. site_gates.py is a *gate*, not a runner — do not model
|
||||
new work on it.
|
||||
-->
|
||||
|
||||
## Logging
|
||||
|
||||
New leveled lines use `internal/logx` — DEBUG always reaches the debug ring; stdout respects
|
||||
`logging.level`. English, keys-never-values, durations on outcomes. Full rules:
|
||||
`felhom.eu/documentation/runbooks/logging-conventions.md`.
|
||||
|
||||
## Health checks issue no block I/O
|
||||
|
||||
A probe that touches a wedged device enters uninterruptible sleep, survives `SIGKILL`, and cannot be
|
||||
recovered until the device returns or the host reboots — so `systemctl restart` hangs too. A timeout
|
||||
protects the caller's control flow and nothing else: the blocked thread remains. Liveness is decided
|
||||
from `/proc` and kernel state, never by reading or writing the filesystem.
|
||||
|
||||
<!--
|
||||
Measured, R-117 spike §6.3 (felhom.eu/documentation/audits/SPIKE-r117-bind-liveness-2026-07-30.md):
|
||||
a probe stayed in D state 3m50s after kill -9; a buffered write with no fsync blocked too (O_CREAT
|
||||
needs journal access); and statfs/getdents returned HEALTHY on a namespace that EIOs every byte —
|
||||
fast, and wrong.
|
||||
-->
|
||||
@@ -1,27 +0,0 @@
|
||||
---
|
||||
paths: ["controller/internal/web/templates/**", "controller/internal/web/**/*.go", "**/*.css", "**/*.html"]
|
||||
---
|
||||
|
||||
# UI and Hungarian copy — felhom-controller
|
||||
|
||||
- **All UI text is Hungarian**, Budapest timezone.
|
||||
- Design tokens, badge/colour rules, the 2px/no-shadow/no-emoji/BOM hard rules and the mechanical
|
||||
gate to run after each surface: **use the `felhom-ui-design` skill.**
|
||||
- Template methods need **value receivers** — pointer receivers compile, pass `go vet`, pass the
|
||||
suite, and then 500 at render time.
|
||||
|
||||
## Grep fetched pages with ASCII-only substrings
|
||||
|
||||
Accented Hungarian patterns get mangled through the `ssh → pct exec → bash -c` chain and return a
|
||||
false `0` — which reads exactly like the banner or string being gone. Use `kezel`, `Utols`,
|
||||
`Biztons`. **Never let an accented pattern gate a conclusion.**
|
||||
|
||||
<!--
|
||||
From the 2026-07-20 remediation: an accented grep nearly produced a wrong "banner cleared" claim.
|
||||
This is the "an absent line is not evidence" rule aimed at a UTF-8 transport, not at a log.
|
||||
-->
|
||||
|
||||
## Credentials containing `!` or `'` break in heredoc-built helper scripts
|
||||
|
||||
History expansion eats `!!`. Use the proven inline `-d "password=$PW"` form for authed curl, and
|
||||
delete any credential-bearing helper from `/tmp` (host AND guest) when done.
|
||||
@@ -1,104 +0,0 @@
|
||||
# gates — re-run this repo's gate entry point on every push, on a machine that does not care who
|
||||
# pushed or what they typed.
|
||||
#
|
||||
# *** THIS REPORTS. IT CANNOT REFUSE. ***
|
||||
#
|
||||
# felhom repos push straight to `main` with no pull request, so there is no merge for a status
|
||||
# check to stand at. The refusing half is `.githooks/pre-push`, which is local to a clone and which
|
||||
# `git push --no-verify` skips; this half is what notices when that happened. Neither half is the
|
||||
# whole thing, and both are named in felhom.eu documentation/backlog/OPEN-ITEMS.md R-168.
|
||||
#
|
||||
# NO `uses:` STEP ANYWHERE, deliberately: JavaScript actions need a node runtime in the runner, and
|
||||
# the runner is a host-mode container with python3 and git and nothing else (see
|
||||
# homelab-manifests/gitea-system/act-runner.yaml for why it is not privileged). Probe P3 measured
|
||||
# that a plain `git fetch` of the pushed SHA from the in-cluster Gitea service is enough.
|
||||
#
|
||||
# A failing run must reach a person — a detector nobody hears is the defect R-29 filed, rebuilt one
|
||||
# layer up. That is the last step, and it runs ONLY on failure.
|
||||
name: gates
|
||||
on: [push]
|
||||
|
||||
jobs:
|
||||
gates:
|
||||
runs-on: felhom-gates
|
||||
steps:
|
||||
- name: Fetch the pushed commit and the sibling clone it needs
|
||||
# This repo's entry point invokes a SHARED checker that lives in the felhom.eu clone next
|
||||
# door and is deliberately never copied here — so CI has to reproduce the workspace's
|
||||
# sibling layout or the gate fails closed with "gate is MISSING". The sibling is also
|
||||
# needed for CONTENT: this repo's REUSE.md cites a path that lives in the hub.
|
||||
run: |
|
||||
# Shallow, and pinned to the exact SHA that was pushed — not to the branch tip,
|
||||
# which can move under us if two pushes race.
|
||||
mkdir -p ws/felhom-controller
|
||||
cd ws/felhom-controller
|
||||
git init -q .
|
||||
git remote add origin http://gitea.gitea-system.svc.cluster.local:3000/admin/felhom-controller.git
|
||||
git fetch -q --depth 1 origin "$GITHUB_SHA"
|
||||
git checkout -q FETCH_HEAD
|
||||
echo "checked out $(git rev-parse HEAD)"
|
||||
cd .. && git clone -q --depth 1 http://gitea.gitea-system.svc.cluster.local:3000/admin/felhom.eu.git felhom.eu
|
||||
echo "sibling felhom.eu present at $(cd felhom.eu && git rev-parse --short HEAD)"
|
||||
|
||||
- name: Run the gate entry point
|
||||
# The ONLY thing CI runs. No go build, no go test, no linting, no deploy. The
|
||||
# exit code IS the result: no `|| true`, no pipe that could swallow it.
|
||||
run: cd ws/felhom-controller/controller && python3 scripts/controller_gates.py --fast
|
||||
|
||||
- name: Alarm on failure
|
||||
# THE POINT OF THE WHOLE THING. Probe P5 measured that a failed run produces NO mail, NO
|
||||
# notification row and NO log line from Gitea itself — a red tick in a web UI nobody watches
|
||||
# is exactly the shape R-29 filed against. So the run sends its own alarm, on the project's
|
||||
# existing transactional path (Resend, the same one the hub uses), and prints the provider's
|
||||
# accepted id so "a message left the machine" is an observable, not an assumption.
|
||||
#
|
||||
# Pure python3 and urllib, NOT curl: the runner image carries python3 and git and nothing
|
||||
# else on purpose, and the first version of this step died on `curl: command not found`.
|
||||
# Reaching for a bigger image to send one HTTP request would have been the wrong trade.
|
||||
if: failure()
|
||||
env:
|
||||
RESEND_API_KEY: ${{ secrets.RESEND_API_KEY }}
|
||||
run: |
|
||||
python3 - <<'PY'
|
||||
import json, os, sys, urllib.request, urllib.error
|
||||
|
||||
key = os.environ.get("RESEND_API_KEY", "")
|
||||
if not key:
|
||||
sys.exit("ALARM FAILED: RESEND_API_KEY is empty — the alarm cannot be sent, and a "
|
||||
"silent alarm is worse than none. Set the user-level Actions secret.")
|
||||
|
||||
repo = os.environ.get("GITHUB_REPOSITORY", "?")
|
||||
sha = os.environ.get("GITHUB_SHA", "?")
|
||||
run = os.environ.get("GITHUB_RUN_NUMBER", "?")
|
||||
srv = os.environ.get("GITHUB_SERVER_URL", "https://gitea.dooplex.hu")
|
||||
|
||||
body = json.dumps({
|
||||
"from": "Felhom CI <monitoring@felhom.eu>",
|
||||
"to": ["admin@felhom.eu"],
|
||||
"subject": "[felhom CI] gates FAILED in %s" % repo,
|
||||
"text": (
|
||||
"The gate entry point exited non-zero.\n\n"
|
||||
"Repository : %s\n"
|
||||
"Commit : %s\n"
|
||||
"Run : %s/%s/actions/runs/%s\n\n"
|
||||
"The failing gate names itself in the run log.\n\n"
|
||||
"If the local pre-push hook was GREEN for this commit, then CI and the hook\n"
|
||||
"disagree - that is a finding about the gates themselves, not about CI, and it\n"
|
||||
"outranks whatever the push was for.\n"
|
||||
) % (repo, sha, srv, repo, run),
|
||||
}).encode()
|
||||
|
||||
req = urllib.request.Request(
|
||||
"https://api.resend.com/emails", data=body, method="POST",
|
||||
headers={"Authorization": "Bearer %s" % key,
|
||||
"Content-Type": "application/json",
|
||||
# Cloudflare fronts api.resend.com and BLOCKS the default
|
||||
# "Python-urllib/3.x" agent with its own 403 (error 1010) — which looks
|
||||
# exactly like an auth failure and is not one. Measured 2026-08-02.
|
||||
"User-Agent": "felhom-ci/1.0"})
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=30) as r:
|
||||
print("RESEND-ACCEPTED id=%s" % json.load(r)["id"])
|
||||
except urllib.error.HTTPError as e:
|
||||
sys.exit("ALARM FAILED: Resend returned HTTP %s: %s" % (e.code, e.read().decode()[:300]))
|
||||
PY
|
||||
@@ -1,82 +0,0 @@
|
||||
#!/bin/sh
|
||||
# pre-push — refuse a push that carries a broken gate. (2026-08-02, R-29 leg (b) first half.)
|
||||
#
|
||||
# Runs this repo's ONE gate entry point in --fast mode: only checks that touch no network and no
|
||||
# container runtime, so a push stays a push and never pulls images or starts containers. The slow
|
||||
# gates stay deliberate periodic runs; a hook that takes minutes gets bypassed within a week and
|
||||
# the bypass becomes the habit.
|
||||
#
|
||||
# BOTH LINES BELOW ARE DELIBERATE. An absent log line is not evidence a hook ran — a silent pass is
|
||||
# equally consistent with "gates green" and "hook never fired", so a passing push says so out loud.
|
||||
#
|
||||
# HONEST LIMITS, stated so this is not mistaken for enforcement it cannot provide:
|
||||
# * per-clone — core.hooksPath is local config and a clone does not carry it. Arm a clone once:
|
||||
# git config core.hooksPath .githooks
|
||||
# Any manual entry-point run WARNS when the clone is unarmed.
|
||||
# * skippable — `git push --no-verify` bypasses this entirely. That is on purpose: an escape
|
||||
# hatch that cannot be reached is one that gets removed the first time it is
|
||||
# inconvenient. USING IT MUST BE STATED IN THE SESSION REPORT.
|
||||
# The half that is neither per-clone nor skippable is CI — felhom.eu OPEN-ITEMS.md R-168.
|
||||
#
|
||||
# Measured 2026-08-02 (git 2.47.3): a relative core.hooksPath resolves correctly and the hook's cwd
|
||||
# is the repo root whether `git push` is issued from the root or from any subdirectory. The
|
||||
# explicit rev-parse below does not depend on that.
|
||||
set -u
|
||||
|
||||
root=$(git rev-parse --show-toplevel 2>/dev/null) || {
|
||||
echo "pre-push: FAIL - cannot resolve the repo root (git rev-parse --show-toplevel)." >&2
|
||||
exit 1
|
||||
}
|
||||
cd "$root" || exit 1
|
||||
|
||||
# ── WORKSPACE-ROOT ASSERTION (2026-08-05, R-204 rider) ───────────────────────────────────────────
|
||||
# Refuse a push from a clone outside the felhom workspace.
|
||||
#
|
||||
# WHY THIS IS A HOOK AND NOT A LINE IN A DOCUMENT: the workspace root is ALREADY written down, in
|
||||
# documentation/runbooks/workspace-CLAUDE.md and in the workspace-root CLAUDE.md ("stay inside it"),
|
||||
# and work drifted into a home directory anyway. A rule that has failed once as a reminder is not
|
||||
# fixed by writing it down again — it has to be asserted where it can bite.
|
||||
#
|
||||
# A PUSH IS THE RIGHT TRIGGER, deliberately: throwaway clones under /tmp for probes and red-proofs
|
||||
# never push, so nothing legitimate breaks. Reads and builds elsewhere stay unaffected.
|
||||
#
|
||||
# Symlinks are resolved on BOTH sides before comparison, so a symlinked path neither falsely passes
|
||||
# nor falsely fails. If the workspace root does not exist on this machine the check is SKIPPED, not
|
||||
# failed — this hook must not brick a legitimate clone on a different host.
|
||||
#
|
||||
# The only bypass is the documented `git push --no-verify`, whose use is already reportable.
|
||||
FELHOM_WORKSPACE_ROOT=/mnt/5_hdd/felhom.eu
|
||||
if [ -d "$FELHOM_WORKSPACE_ROOT" ]; then
|
||||
ws_real=$(cd "$FELHOM_WORKSPACE_ROOT" 2>/dev/null && pwd -P) || ws_real=""
|
||||
root_real=$(pwd -P) || root_real=""
|
||||
if [ -n "$ws_real" ] && [ -n "$root_real" ]; then
|
||||
case "$root_real/" in
|
||||
"$ws_real"/*) : ;; # inside the workspace — proceed
|
||||
*)
|
||||
echo "pre-push: PUSH REFUSED - this clone is OUTSIDE the felhom workspace." >&2
|
||||
echo " clone: $root_real" >&2
|
||||
echo " expected: under $ws_real (repos live in $ws_real/git/<repo>)" >&2
|
||||
echo " Work in the workspace clone, or bypass with 'git push --no-verify'" >&2
|
||||
echo " and state that you did in the session report." >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
fi
|
||||
fi
|
||||
|
||||
if ! command -v python3 >/dev/null 2>&1; then
|
||||
echo "pre-push: FAIL - python3 not found, so the gates CANNOT run. This is a failure, never a" >&2
|
||||
echo " pass by default. Install python3, or push with --no-verify and say so." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "pre-push [felhom-controller]: running controller/scripts/controller_gates.py --fast ..."
|
||||
python3 "controller/scripts/controller_gates.py" --fast
|
||||
rc=$?
|
||||
if [ "$rc" -ne 0 ]; then
|
||||
echo "pre-push [felhom-controller]: PUSH REFUSED - gates exited $rc. Fix the finding above, or bypass with" >&2
|
||||
echo " 'git push --no-verify' and state that you did in the session report." >&2
|
||||
else
|
||||
echo "pre-push [felhom-controller]: gates OK - push proceeding."
|
||||
fi
|
||||
exit $rc
|
||||
@@ -0,0 +1,77 @@
|
||||
# AUDIT — felhom-controller deep sweep — 2026-06-13
|
||||
|
||||
**Branch:** `audit/2026-06-13-deep-sweep`
|
||||
**Auditor:** Claude Code (unattended overnight session)
|
||||
|
||||
| Repo | HEAD commit | Note |
|
||||
|---|---|---|
|
||||
| felhom-controller | `76a570da3284a742829cfeabe588255e3c74095f` | v0.51.0 (2026-06-12) — all findings cite this commit |
|
||||
| felhom-agent (reference) | `716cbcd70500602f80a3e71869308a171396f1b6` | v0.28.0 |
|
||||
| felhom.eu / hub (reference) | `d59691dd826a901da39562aeaa5d989b8ea1a7ee` | hub v0.11.0 |
|
||||
|
||||
**Tooling:** go1.26.0 windows/amd64; staticcheck (latest, installed this session); go vet.
|
||||
**Prior art:** `BUGHUNT.md` (2026-02-25, v0.30.3) read in full — findings there are NOT re-reported unless regressed; note that many BUGHUNT items refer to packages deleted in slice 8C.
|
||||
|
||||
## Progress log
|
||||
|
||||
- 2026-06-13 ~00:05 — Phase 0 start: repos pulled, branch created, baseline run.
|
||||
- (in progress)
|
||||
|
||||
## Baseline (Phase 0)
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| `go build ./...` | PASS |
|
||||
| `go vet ./...` | PASS (clean) |
|
||||
| `gofmt -l .` | ~75 files flagged — **all CRLF noise** from `core.autocrlf=true` on this Windows checkout; `gofmt -d` shows whitespace-only diffs. Not a code finding; see Info section (missing `.gitattributes`). |
|
||||
| `go test ./...` | **1 pre-existing FAIL**: `TestBackupCopiesOnPath` (internal/web/storage_handlers_test.go:295) — see findings. |
|
||||
| staticcheck | 17 reports — triaged in Phase 1. |
|
||||
|
||||
## Executive summary
|
||||
|
||||
(to be written in Phase 6)
|
||||
|
||||
## Top-10 action list
|
||||
|
||||
(to be written in Phase 6)
|
||||
|
||||
## Findings — Critical / High
|
||||
|
||||
(pending)
|
||||
|
||||
## Findings — Medium / Low
|
||||
|
||||
(pending)
|
||||
|
||||
## Findings — Info
|
||||
|
||||
(pending)
|
||||
|
||||
## Contract checks (controller↔agent, controller↔hub)
|
||||
|
||||
(pending)
|
||||
|
||||
## Invariant checklist results
|
||||
|
||||
(pending)
|
||||
|
||||
## Refactor & shared-code opportunities
|
||||
|
||||
(pending)
|
||||
|
||||
## Test-gap analysis
|
||||
|
||||
(pending)
|
||||
|
||||
## Dead code inventory
|
||||
|
||||
(pending)
|
||||
|
||||
## Session notes, assumptions, open questions
|
||||
|
||||
- Session is unattended; conservative assumptions recorded inline.
|
||||
- gofmt noise: repo is checked out with CRLF (`core.autocrlf=true`); `gofmt -l` flags nearly every file. Treated as environment artifact.
|
||||
|
||||
## What was NOT covered
|
||||
|
||||
(to be written honestly in Phase 6)
|
||||
-14
@@ -1,19 +1,5 @@
|
||||
# Bug Hunt Report — Comprehensive Controller Audit
|
||||
|
||||
> **RECONCILED 2026-06-13 (controller now v0.59.0).** This report is from v0.30.3 and predates the
|
||||
> Proxmox re-platform / slice-8C de-privileging — much of the audited code has since been rewritten,
|
||||
> moved to `felhom-agent`, or deleted. A full reconciliation against current code lives at
|
||||
> **`felhom.eu/documentation/audits/bughunt-reconcile-2026-06-13.md`** (authoritative). Summary:
|
||||
> - **H1–H4, H12** (concurrency Highs): already FIXED (confirmed in the deep-sweep audit).
|
||||
> - **H10** (plaintext secret on encrypt failure): the old `// H10 fix` only logged a WARN and still
|
||||
> persisted plaintext — now **FIXED fail-closed** in v0.59.0 (`5a80739`).
|
||||
> - **C2, H5, H6, H7, H8**: verified FIXED. **C3** (SSD-only DB DR loss): **MOOT-by-architecture**
|
||||
> (DR is now whole-LXC PBS). **C1, H9, H11** + several file-gone Lows: **MOOT** (gone, not migrated-with-bug).
|
||||
> - Remaining Mediums/Lows: mostly CODE-GONE; a small `[backlog]` set (M4/M5/M6, M18/M19, M25) survived
|
||||
> but is **not yet deep-verified** — see the reconciliation for the list.
|
||||
>
|
||||
> The original v0.30.3 content is preserved unchanged below for history.
|
||||
|
||||
**Date:** 2026-02-25
|
||||
**Scope:** Full controller codebase (`controller/` — all ~80 Go files, all packages)
|
||||
**Method:** Systematic code review of every file + live testing on demo node (192.168.0.162)
|
||||
|
||||
-7131
File diff suppressed because it is too large
Load Diff
@@ -1,110 +1,316 @@
|
||||
# CLAUDE.md — `felhom-controller`
|
||||
# CLAUDE.md — Project Instructions for Claude Code
|
||||
|
||||
> Stable orientation only — **current state lives in `CONTEXT.md` and the top of `CHANGELOG.md`**,
|
||||
> never here. Cross-repo conventions (clean-tree gate, secrets, trunk-based, artifact taxonomy):
|
||||
> workspace-root `/mnt/5_hdd/felhom.eu/git/CLAUDE.md`. Path-scoped detail: `.claude/rules/`.
|
||||
> This file is read automatically by Claude Code at the start of every session. It replaces the "Instructions" panel from the claude.ai Project. Keep it updated as the project evolves.
|
||||
|
||||
## What this repo is
|
||||
!!! IMPORTANT !!!
|
||||
- Always update CHANGELOG.md whenever you modified the code, and pushed to git!!
|
||||
- IF controller feature changed (new/modify/remove) always update the relevant part of controller/README.md with the architectural change!!
|
||||
|
||||
The **in-guest controller** — one per customer LXC, Docker-only, **holds NO Proxmox credentials**. It
|
||||
owns the app domain: stack/deploy management, the Hungarian web UI, app-data backup, metrics,
|
||||
integrations, git-sync, notifications. Disk/host/Proxmox concerns are delegated to the host agent via
|
||||
`internal/agentapi`. Whole-guest backup (PBS vzdump) is the agent's, not ours.
|
||||
## Project overview
|
||||
|
||||
**Don't confuse the two ex-"controllers":** `felhom-agent` (host, operator-tier, was
|
||||
`proxmox-controller`) vs this repo (in-guest, was `deploy-felhom-compose`).
|
||||
Creating a business (Felhom) for home-server deployment for Hungarian customers. This repository (`felhom-controller`) contains the felhom-controller — a Go application that manages Docker Compose stacks on customer hardware via a Hungarian-language web dashboard.
|
||||
|
||||
## Doing X → read Y
|
||||
See `controller/README.md` for full architecture and status (update after each session, keep track of how different functions/features operate, like backup, monitoring, storage handling, app management, user settings, update workflow, notification system, etc-etc...).
|
||||
See `CHANGELOG.md` for recent work (update after each session — see "Working with CHANGELOG.md" below).
|
||||
See `CONTEXT.md` for current project state, decisions and roadmap (update after each session).
|
||||
See `TASK.md` for the current task to implement (if it exists).
|
||||
|
||||
| Doing | Read |
|
||||
|---|---|
|
||||
| writing any new code | `REUSE.md` — canonical helpers, patterns, traps, seams |
|
||||
| needing current state / roadmap | `CONTEXT.md` |
|
||||
| needing a feature or architecture reference | `controller/README.md` |
|
||||
| build, deploy, publish, verify a version | the **`felhom-build-deploy`** skill |
|
||||
| writing or reviewing a test, fixing a bug | the **`felhom-testing`** skill |
|
||||
| UI, tokens, badges, Hungarian copy | the **`felhom-ui-design`** skill |
|
||||
| which box may I break | `felhom.eu/documentation/runbooks/target-selection.md` |
|
||||
| host addresses, break-glass, node facts | `felhom.eu/documentation/operations/nodes.md` |
|
||||
| what version is live anywhere | ask the hub (`/hosts`, `/configs`) or the box — **never a doc** |
|
||||
| the authoritative design | `felhom.eu/documentation/architecture/01/02/03-*.md` |
|
||||
Claude in Chrome extension is available — can be used to test web UI on demo-felhom.eu or verify dashboard deployments in browser.
|
||||
|
||||
## Session-critical invariants
|
||||
## System context — the Proxmox re-platform (READ THIS FIRST)
|
||||
|
||||
The rest live in `REUSE.md`. These cost incidents to learn:
|
||||
The project has **re-platformed onto Proxmox**, with a locked **three-component model**:
|
||||
- **Hub** (`felhom.eu/hub/`) — operator backend on k3s.
|
||||
- **Host agent** (`felhom-agent/`, formerly `proxmox-controller`) — one per Proxmox host; operator-tier; owns ALL Proxmox interaction.
|
||||
- **In-guest controller** (THIS repo) — one per customer LXC; **Docker-only; holds NO Proxmox credentials**.
|
||||
|
||||
- `docker compose restart` does NOT pick up new images/env — always `up -d` (`RedeployFromEnv`).
|
||||
- Docker's `.State` says "running" even for unhealthy containers — the `.Status` parse is the truth.
|
||||
- In-memory `Deployed` is set BEFORE `compose up -d` (slow-pull race); reverted on failure.
|
||||
- `compose up -d` exits 0 on crash-loops — the post-start status check is the detection.
|
||||
- Env var KEYS are logged, never values. Protected stacks (traefik, cloudflared, felhom-controller)
|
||||
cannot be stopped from the UI.
|
||||
- Verify a container image HAS the healthcheck tool before using it (BusyBox wget / python3 / curl —
|
||||
the catalog `REUSE.md` maps the families).
|
||||
- `IsRunning()` is CONCURRENCY, false during a verification restore — display MUST use
|
||||
`RestoreStatus()`.
|
||||
**This repo is being de-privileged.** In the target model, host/disk/Proxmox/Cloudflare responsibilities move OUT of the controller into the **host agent**: System info, Storage (disk scan/format/mount/migrate), the disk-tier Backup (restic, cross-drive, drive-restore, infra-backup), and the Cloudflare-API geo enforcement. The controller keeps the **app domain**: stack/deploy management, the Hungarian web UI, app-data backup (DB dumps + Docker-volume tars), metrics/telemetry, integrations, git-sync, notifications.
|
||||
|
||||
## Live validation — the fence
|
||||
> **Authoritative map:** `felhom.eu/documentation/architecture/02-controller-module-map.md` — the per-package **KEEP / PORT / DELETE(→agent) / DELETE(obsolete) / MODIFY** classification. Read it before touching `backup/`, `storage/`, `cloudflare/`, `system/`, or `config/`. Also doc 01 (topology/trust) and doc 03 (the host agent).
|
||||
|
||||
Exercise the SERVER-SIDE PIPELINE a real user triggers, end-to-end (connect → enroll → deploy). **The
|
||||
forbidden shortcut is BYPASSING that pipeline** — the F9 episode was a raw agent guest-attach with
|
||||
hand-set state, and it proved nothing.
|
||||
**⚠️ Status — do NOT assume the target state is implemented.** The de-privileging has only *started*: the recent `internal/appbackup/` extraction split the keep-side app-data-backup primitives from the delete-side disk/host code (groundwork, no behaviour change). The **bulk strip has NOT happened** — the current code STILL contains the full privileged storage / restic / cross-drive / disk / Cloudflare stack. The strip + the agent-local-API client land at **~slice 8**. So the code you see is the **pre-strip, still-privileged** controller; match the code, not the target, unless a TASK says otherwise.
|
||||
|
||||
`claude-in-chrome` is NOT available on DooPlex. The standard method is endpoint-level: invoke the
|
||||
exact endpoint the UI invokes (no server logic is skipped, only rendering) and **say which method was
|
||||
used**. Strict end-to-end UI coverage is a manual click-through by the operator.
|
||||
**Don't confuse the two ex-"controllers":** `felhom-agent` (host, operator-tier, was `proxmox-controller`) vs this `felhom-controller` (in-guest, was `deploy-felhom-compose`).
|
||||
|
||||
Two traps in that method live in `.claude/rules/ui-hungarian.md` (ASCII-only greps; `!` in
|
||||
credentials) — they load when you touch a template or stylesheet.
|
||||
## Cross-repo & artifacts
|
||||
|
||||
## Commands — one per surface
|
||||
- Workspace orientation (the felhom system, shared conventions, access) lives in the workspace-root `e:\git\CLAUDE.md`. Sibling per-repo files: `felhom-agent/CLAUDE.md`, `felhom.eu/CLAUDE.md`.
|
||||
- **Artifact taxonomy:** `TASK.md` / `TASK-*.md` = a spec for YOU to implement (then push + update CHANGELOG + CONTEXT + README).
|
||||
- **`RUNBOOK-*.md`** — an operational procedure. CC executes the steps it has access and capability for, including live validation on the demo nodes and the demo Proxmox host (CC has root@felhom-pve SSH + the felhom-agent token). A step is human-only only when it genuinely needs physical presence, a real-world decision, or credentials CC truly lacks — mark those steps HUMAN. Do not decline a whole procedure because it touches a live host or a privileged token. (Judgment still applies: confirm before irreversible ops on real customer data — but demo scratch guests are fair game.)
|
||||
|
||||
| Surface | Command |
|
||||
|---|---|
|
||||
| Gates (after ANY change) | `python3 controller/scripts/controller_gates.py` — from `controller/` |
|
||||
| Green gate | `go build ./... && go vet ./... && go test ./...` |
|
||||
| Build + deploy | the **`felhom-build-deploy`** skill — do not hand-roll it |
|
||||
> **In every repository where you make a change, update both files in that repo:**
|
||||
> - **`CHANGELOG.md`** — a cumulative log of **all** changes; newest entry on top.
|
||||
> - **`REPORT.md`** — **overwrite** with a summary of the **most recent** implementation (or significant validation/operational run) only; not cumulative.
|
||||
>
|
||||
> **Never write secrets** — tokens, passwords, private keys, API keys — into `CHANGELOG.md`, `REPORT.md`, or any committed file. Reference them as "stored out-of-band" instead.
|
||||
|
||||
Guest 9201 is **bootstrap-managed — there is no compose file**;
|
||||
`felhom-controller-bootstrap.service` runs the tag written in `/etc/felhom-controller-image`. Catalog
|
||||
changes (`app-catalog-felhom.eu`) are picked up by controller sync ≤15 min, or via the "Sablonok
|
||||
frissítése" button.
|
||||
## Code quality rules
|
||||
|
||||
- Always double-check generated code for bugs, logic issues, syntax errors
|
||||
- Handle edge cases without overcomplicating the script/program
|
||||
- Add debug capabilities (logging, verbose output) for easier troubleshooting
|
||||
- If you need more input or troubleshooting command output, ask first — don't guess
|
||||
|
||||
## Environment
|
||||
|
||||
| Machine | OS | IP | Purpose |
|
||||
|---------|----|----|---------|
|
||||
| **Local (this machine)** | Windows 11 | — | Development, Claude Code runs here. Repos in `E:\git\` |
|
||||
| **Build server (k3s, infra)** | Debian 13 | 192.168.0.180 | Build + push container images, k3s cluster |
|
||||
| **Demo node** | Debian 13 | 192.168.0.162 | Test deployment (demo-felhom.eu) |
|
||||
| **Demo node 2** | Debian 13 | router.abonet.hu:33022 | Remote test deployment |
|
||||
|
||||
## Workspace layout
|
||||
|
||||
Claude Code runs on Windows 11. The working directory is `E:\git\` (mapped as `/e/git/` in Git Bash). This repo is at:
|
||||
|
||||
```
|
||||
E:\git\felhom-controller\ (or /e/git/felhom-controller/ in Git Bash)
|
||||
├── controller/ # Go application (main codebase)
|
||||
│ ├── cmd/controller/ # Entry point (main.go)
|
||||
│ ├── internal/
|
||||
│ │ ├── config/ # YAML config loading
|
||||
│ │ ├── settings/ # settings.json persistence (password hash, DB cache)
|
||||
│ │ ├── stacks/ # Docker Compose operations, deploy flow
|
||||
│ │ ├── sync/ # Git sync — periodic pull of app catalog repo
|
||||
│ │ ├── api/ # REST API endpoints
|
||||
│ │ ├── system/ # System info (memory, disk)
|
||||
│ │ └── web/ # Dashboard UI
|
||||
│ │ ├── server.go # Server struct, routing, static serving
|
||||
│ │ ├── auth.go # Session auth, login/logout handlers
|
||||
│ │ ├── handlers.go # Page handlers (dashboard, stacks, deploy, etc.)
|
||||
│ │ ├── funcmap.go # Template function map
|
||||
│ │ ├── embed.go # go:embed directive for templates
|
||||
│ │ ├── templates.go # Felhom logo SVG constant
|
||||
│ │ └── templates/ # go:embed HTML/CSS files (Hungarian UI)
|
||||
│ ├── Dockerfile
|
||||
│ ├── Makefile
|
||||
│ └── go.mod
|
||||
├── scripts/ # Setup scripts for customer nodes
|
||||
├── CLAUDE.md # This file
|
||||
├── CHANGELOG.md # Changelog
|
||||
├── CONTEXT.md # Project memory / state / architectural state/decisions/roadmap
|
||||
└── TASK.md # Current task (if exists)
|
||||
```
|
||||
|
||||
Related repos (same parent directory):
|
||||
```
|
||||
E:\git\app-catalog-felhom.eu\ # Docker Compose templates + .felhom.yml metadata per app
|
||||
E:\git\felhom.eu\ # Website (htmls) + k3s manifests
|
||||
E:\git\homelab-manifests\ # k3s cluster manifests (dooplex.hu services)
|
||||
E:\git\misc-scripts\ # Helper scripts
|
||||
```
|
||||
|
||||
All repos hosted at `gitea.dooplex.hu/admin/`. Git credentials are stored (`git config credential.helper store`).
|
||||
|
||||
## SSH access
|
||||
|
||||
SSH key-based authentication is configured and working. No password prompts.
|
||||
|
||||
**IMPORTANT — SSH binary:** Claude Code runs in Git Bash, which has its own SSH at `/usr/bin/ssh` (= `C:\Program Files\Git\usr\bin\ssh.exe`). This binary does NOT have access to the Windows SSH agent and will fail silently (exit 0/141 with no output). Always use the Windows native OpenSSH binary with the full path:
|
||||
|
||||
```
|
||||
SSH=/c/Windows/System32/OpenSSH/ssh.exe
|
||||
```
|
||||
|
||||
All SSH commands in this file use `$SSH` — set it at the start of your session or substitute the full path manually.
|
||||
|
||||
| Host | OS | IP | User | Role |
|
||||
|------|----|----|------|------|
|
||||
| Build server | Debian 13 | 192.168.0.180 | kisfenyo | Build + push container images |
|
||||
| Demo Proxmox host | 192.168.0.162 | root@pam (SSH alias felhom-pve, root, no sudo) | pveum/pct + live Proxmox validation — available to CC |
|
||||
|
||||
## Test environments
|
||||
|
||||
| Node | OS | Hardware | Domain | IP | Notes |
|
||||
|------|-----|----------|--------|----|-------|
|
||||
| demo-felhom | Debian 13 | Acemagic N100, 16G RAM, 512G SSD + 1TB HDD | demo-felhom.eu | 192.168.0.162 | Primary test node, Cloudflare Tunnel |
|
||||
| felhotest | Debian 13 | Proxmox VM (4-16G RAM, 8 vCPU, 200G + 100G SCSI) | — | router.abonet.hu:33022 | Remote test node |
|
||||
| pi-customer-1 | Debian 13 | Raspberry Pi 3B+, 1G RAM, 32G SD | pi-customer-1.local | 192.168.0.161 | Secondary test, not yet active |
|
||||
|
||||
- Pi-hole DNS on local network forwards `*.demo-felhom.eu` → 192.168.0.162
|
||||
- External access via Cloudflare Tunnel → Traefik reverse proxy
|
||||
|
||||
> **⚠️ Re-platform note:** per the host-agent work, `192.168.0.162` is now a **Proxmox host** (`demo-felhom`, PVE 9.2.2) — the demo-node tables above predate that. Confirm how/where the controller is currently deployed and tested post-re-platform before relying on the bare-metal `docker compose` deploy steps below; on the re-platformed node the controller may now run inside an LXC guest rather than directly on the host.
|
||||
|
||||
## Build & deploy workflow — MANDATORY
|
||||
|
||||
After making code changes to the controller, you **MUST** build, push, and deploy the new image. Do NOT leave code changes uncommitted or undeployed. The full cycle is:
|
||||
|
||||
### Step 1: Commit and push changes
|
||||
|
||||
```bash
|
||||
cd /e/git/felhom-controller
|
||||
git add -A && git commit -m "<descriptive message>" && git push
|
||||
```
|
||||
|
||||
### Step 2: Build + push the container image on the build server
|
||||
|
||||
The build server (192.168.0.180) has the build toolchain. The version tag should be incremented from the current running version.
|
||||
|
||||
!! Important: use "kisfenyo" user for SSH, as written below
|
||||
|
||||
First, set the SSH variable (required for every session — Git Bash's built-in ssh does NOT work):
|
||||
```bash
|
||||
SSH=/c/Windows/System32/OpenSSH/ssh.exe
|
||||
```
|
||||
|
||||
Check the current running version:
|
||||
```bash
|
||||
$SSH kisfenyo@192.168.0.162 "docker ps --filter name=felhom-controller --format '{{.Image}}'"
|
||||
```
|
||||
|
||||
Then build with the next version (e.g., if current is 0.2.10, use 0.2.11): IMPORTANT!: Build directory is: ~/build/felhom-controller
|
||||
```bash
|
||||
$SSH kisfenyo@192.168.0.180 "cd ~/build/felhom-controller && git -C ~/git/felhom-controller pull && ./build.sh <NEW_VERSION> --push"
|
||||
```
|
||||
|
||||
The build script:
|
||||
- Pulls latest code from Gitea
|
||||
- Builds a multi-arch Docker image (amd64 + arm64) if `--multiarch`, or current arch if `--push`
|
||||
- Pushes to `gitea.dooplex.hu/admin/felhom-controller:<VERSION>`
|
||||
- Expects the version as first argument (e.g., `0.2.11`)
|
||||
|
||||
### Step 3: Deploy on demo nodes
|
||||
|
||||
```bash
|
||||
# Demo node 1 (local)
|
||||
$SSH kisfenyo@192.168.0.162 "cd /opt/docker/felhom-controller && sudo docker pull gitea.dooplex.hu/admin/felhom-controller:<NEW_VERSION> && sudo sed -i 's|image: gitea.dooplex.hu/admin/felhom-controller:.*|image: gitea.dooplex.hu/admin/felhom-controller:<NEW_VERSION>|' docker-compose.yml && sudo docker compose up -d"
|
||||
|
||||
# Demo node 2 (remote)
|
||||
$SSH -p 33022 kisfenyo@router.abonet.hu "cd /opt/docker/felhom-controller && sudo docker pull gitea.dooplex.hu/admin/felhom-controller:<NEW_VERSION> && sudo sed -i 's|image: gitea.dooplex.hu/admin/felhom-controller:.*|image: gitea.dooplex.hu/admin/felhom-controller:<NEW_VERSION>|' docker-compose.yml && sudo docker compose up -d"
|
||||
```
|
||||
|
||||
### Step 4: Verify the deployment
|
||||
|
||||
```bash
|
||||
$SSH kisfenyo@192.168.0.162 "docker ps --filter name=felhom-controller --format '{{.Image}} {{.Status}}'"
|
||||
$SSH -p 33022 kisfenyo@router.abonet.hu "docker ps --filter name=felhom-controller --format '{{.Image}} {{.Status}}'"
|
||||
```
|
||||
|
||||
Should show the new version and "Up" status. Also check logs for startup errors:
|
||||
```bash
|
||||
$SSH kisfenyo@192.168.0.162 "docker logs felhom-controller --tail 20"
|
||||
$SSH -p 33022 kisfenyo@router.abonet.hu "docker logs felhom-controller --tail 20"
|
||||
```
|
||||
|
||||
### Build workflow summary
|
||||
|
||||
| Step | Command | Where |
|
||||
|------|---------|-------|
|
||||
| 0. Set SSH var | `SSH=/c/Windows/System32/OpenSSH/ssh.exe` | Local (once per session) |
|
||||
| 1. Commit + push | `git add -A && git commit -m "..." && git push` | Local (this repo) |
|
||||
| 2. Build + push image | `$SSH kisfenyo@192.168.0.180 "cd ~/build/felhom-controller... ./build.sh <VER> --push"` | Build server |
|
||||
| 3. Deploy (node 1) | `$SSH kisfenyo@192.168.0.162 "... docker compose up -d"` | Demo node |
|
||||
| 3b. Deploy (node 2) | `$SSH -p 33022 kisfenyo@router.abonet.hu "... docker compose up -d"` | Demo node 2 |
|
||||
| 4. Verify | `$SSH kisfenyo@192.168.0.162 "docker ps ..."` + same for router.abonet.hu | Both nodes |
|
||||
|
||||
### Build & deploy workflow — Hub (felhom-hub)
|
||||
|
||||
The central hub (`hub.felhom.eu`) is a separate Go app in the `E:\git\felhom.eu\hub\` repo. The controller pushes periodic reports to it (when `hub.enabled: true` in `controller.yaml`).
|
||||
|
||||
| Step | Command | Where |
|
||||
|------|---------|-------|
|
||||
| 1. Commit + push | `cd /e/git/felhom.eu && git add -A && git commit && git push` | Local |
|
||||
| 2. Build + push image | `$SSH kisfenyo@192.168.0.180 "cd ~/build/felhom-hub && ./build.sh <VER> --push"` | Build server |
|
||||
| 3. Deploy to k3s | `$SSH kisfenyo@192.168.0.180 "sudo kubectl set image -n felhom-system deploy/hub hub=gitea.dooplex.hu/admin/felhom-hub:<VER>"` | Build server |
|
||||
| 4. Verify | `$SSH kisfenyo@192.168.0.180 "sudo kubectl get pods -n felhom-system -l app=hub && sudo kubectl logs -n felhom-system -l app=hub --tail 10"` | Build server |
|
||||
|
||||
See `E:\git\felhom.eu\CLAUDE.md` for full hub details.
|
||||
|
||||
**IMPORTANT:** If you make changes to the app-catalog-felhom.eu repo, commit and push those too:
|
||||
```bash
|
||||
cd /e/git/app-catalog-felhom.eu
|
||||
git add -A && git commit -m "<message>" && git push
|
||||
```
|
||||
The controller's git sync will pick up catalog changes within 15 minutes, or you can trigger it manually via the dashboard "Sablonok frissítése" button.
|
||||
|
||||
## Tech stack
|
||||
|
||||
- **Language:** Go 1.22+
|
||||
- **Web framework:** stdlib `net/http` + `html/template` (no frameworks)
|
||||
- **Templates:** go:embed HTML files in `internal/web/templates/` (Hungarian UI)
|
||||
- **CSS:** go:embed CSS file in `internal/web/templates/style.css`
|
||||
- **Auth:** bcrypt password hash + session cookies
|
||||
- **Container orchestration:** Docker Compose via CLI (`docker compose up -d`)
|
||||
- **Reverse proxy:** Traefik (separate stack, managed by controller)
|
||||
- **Tunnel:** Cloudflare Tunnel (cloudflared, separate stack)
|
||||
|
||||
## Key patterns
|
||||
|
||||
- All UI text is in Hungarian (Budapest timezone, Hungarian locale)
|
||||
- Templates use Go template functions: `stateColor`, `stateLabel`, `stateIcon`, `stateStr`, `isOperational`, `logoURL`, `logoPNGURL`, `appPageURL`
|
||||
- Container states: `running`, `starting`, `unhealthy`, `stopped`, `exited`, `restarting`, `paused`, `not_deployed`
|
||||
- Docker `.State` field is combined with `.Status` field to detect health substatus
|
||||
- Stacks are sorted alphabetically by DisplayName
|
||||
- Protected stacks (traefik, cloudflared, felhom-controller) can't be stopped from UI
|
||||
- `app.yaml` persists deploy config; `deployed: true` flag controls UI state
|
||||
- In-memory `Deployed` flag is set BEFORE `docker compose up -d` (avoids race condition with slow image pulls); reverted on failure
|
||||
- Password fields require explicit user input or generation (no silent auto-fill)
|
||||
- App cards on dashboard and stacks pages are clickable via `data-href` attribute (skip protected stacks)
|
||||
- Logs page uses AJAX polling (`?raw=1` query param returns plain text) with auto-scroll and pause/resume
|
||||
- Memory bar on deploy page uses two-segment stacked bar (committed = solid green, new = translucent green)
|
||||
- Deploy flow shows 3-step progress panel (config → containers → health), polls `GET /api/stacks/{name}` every 3s until running/unhealthy/timeout(120s)
|
||||
- Telepítés buttons have `checkBeforeDeploy()` onclick guard — fetches live state from API before navigating to deploy page
|
||||
- App info pages at `/apps/{slug}` — detail view with use cases, setup guide, screenshots, optional config
|
||||
- Optional config saves to `app.yaml` and restarts deployed apps via `docker compose up -d`
|
||||
- `optional_config` fields in `.felhom.yml` define post-deploy configurable env vars (e.g., API keys)
|
||||
- `app_info` in `.felhom.yml` provides tagline, use_cases, first_steps, prerequisites, default_creds, docs_url
|
||||
|
||||
## Git sync module (internal/sync)
|
||||
|
||||
- Uses `os/exec` to call `git` CLI — no Go git library dependency
|
||||
- On startup: clones repo to `{data_dir}/catalog-cache/` (shallow clone, `--depth 1`)
|
||||
- Periodically: `git fetch --depth 1` + `git reset --hard origin/{branch}`
|
||||
- Copies only `docker-compose.yml` and `.felhom.yml` to stacks dir
|
||||
- **Never overwrites** `app.yaml` — this contains deployed secrets
|
||||
- Content-hash comparison (SHA-256) — only writes if file actually changed
|
||||
- After sync, triggers `ScanStacks()` rescan for dashboard update
|
||||
- `POST /api/sync` triggers immediate sync (30s debounce)
|
||||
- "Sablonok frissítése" button on Alkalmazások page
|
||||
- Sync status exposed in `/api/system/info` response
|
||||
|
||||
## Debug logging
|
||||
|
||||
The controller has two-tier logging controlled by `logging.level` in `controller.yaml` (or `FELHOM_LOGGING_LEVEL` env var):
|
||||
|
||||
- **`info`** (default): Operation success/failure with elapsed time, post-start container states, scan counts
|
||||
- **`debug`**: All of above plus env var keys per compose command, local image availability checks, compose command completion times, log fetch byte counts
|
||||
|
||||
Key patterns used in `internal/stacks/`:
|
||||
- `time.Since(start)` for operation timing — always logged at INFO level
|
||||
- `m.isDebug()` gates verbose output (env var keys, image checks)
|
||||
- `truncateStr(s, 500)` caps stdout/stderr in error logs
|
||||
- `logPostStartStatus()` runs async (goroutine + 3s sleep) after start/restart/update/deploy — never blocks or fails the operation
|
||||
- `checkLocalImages()` parses compose YAML for `image:` lines, runs `docker image inspect` per image
|
||||
- Env var **keys** are logged, never values (secrets safety)
|
||||
|
||||
## Important lessons learned
|
||||
|
||||
1. `PAPERLESS_OCR_LANGUAGES` (plural, with S) **installs** tesseract packs; `PAPERLESS_OCR_LANGUAGE` (singular) **selects** which to use
|
||||
2. `docker compose restart` does NOT pick up new images — always use `docker compose up -d`
|
||||
3. Go map iteration order is random — always sort before displaying in UI
|
||||
4. Docker's `.State` field says "running" even for unhealthy containers — must parse `.Status` for health info
|
||||
5. In-memory `Deployed` flag must be set BEFORE `docker compose up -d` (not after) — compose can take 30-60s for image pulls; revert both in-memory and disk on failure
|
||||
6. `docker compose up -d` returns exit 0 even when containers crash-loop — post-start status check is essential for detecting failures
|
||||
7. Mealie image has no wget/curl — use Python TCP socket check for healthcheck; set `start_period: 60s` for DB migration time
|
||||
8. Always verify container images have the healthcheck tool (`wget`, `curl`, etc.) before using it — Alpine has BusyBox wget, Python images have `python3`
|
||||
|
||||
## Working with CHANGELOG.md
|
||||
|
||||
**DO NOT read the full file** — it is large and will waste context.
|
||||
**DO NOT read the full file** — it is large (29K+ tokens) and will waste context or fail.
|
||||
|
||||
- Session start: `CONTEXT.md` + `controller/README.md` for current state.
|
||||
- Adding an entry: Read only the top ~30 lines for format, then Edit-insert after line 1.
|
||||
- History: Grep for topics instead of reading.
|
||||
- **At session start:** Do NOT read CHANGELOG.md. Use `CONTEXT.md` and `controller/README.md` for current state.
|
||||
- **To add a new entry:** Read only the top ~30 lines (`limit: 30`) to see the format and insertion point, then use Edit to insert the new entry after line 1 (`## Changelog`).
|
||||
- **To check history:** Use Grep to search for specific topics instead of reading the file.
|
||||
|
||||
## End-of-session checklist
|
||||
|
||||
1. **Commit and push** all code changes (explicit paths; no `git add -A`).
|
||||
2. **Build, push, and deploy** the new controller image, if controller code changed.
|
||||
3. **`CHANGELOG.md`** — always, whenever code changed and was pushed.
|
||||
4. **`CONTEXT.md`** — decisions made, state, what is next.
|
||||
5. **`controller/README.md`** — whenever a feature was added, modified or removed.
|
||||
6. **`REPORT.md`** — overwrite with this run's summary only.
|
||||
7. **`REUSE.md`** — if a shared helper or pattern was added/changed/deprecated (same commit).
|
||||
8. **Verify** the deployment (`docker ps` + logs).
|
||||
Before ending a session, always:
|
||||
|
||||
<!--
|
||||
WHY THIS FILE IS SHORT (2026-08-06, instruction-trim task).
|
||||
Removed from here and rehomed, not lost:
|
||||
- the `## Layout (verified against the tree)` block -> derivable by `ls internal/`; REUSE.md
|
||||
carries the per-package seams and traps that the annotations were really for.
|
||||
- the `!!! IMPORTANT !!!` header -> its two requirements are checklist items 3 and 5. One voice,
|
||||
one place; a rule stated twice in one file is a rule that gets edited in one of them.
|
||||
- the host/access table -> documentation/operations/nodes.md is the single home. The copy here
|
||||
had drifted: it gave demo-felhom as plain root@192.168.0.162 (the LAN fallback, not the route),
|
||||
pinned "agent 0.93.0" against the project's own no-versions-in-docs rule, and claimed no drill
|
||||
VM was provisioned on demo-hp. Measured 2026-08-06: `qm list` on demo-hp shows VM 300
|
||||
`drill-r50` present. felhom-agent/CLAUDE.md was right; this file was wrong.
|
||||
- the "felhom-pve is back on the home LAN" block -> it was bookkeeping about a retired block; the
|
||||
record is in documentation/audits/AUDIT-vacation-remote-ops-2026-07-20.md.
|
||||
- the "Legacy: Windows workstation" block -> the workspace-root CLAUDE.md carries the full version.
|
||||
- the gates/logging/coupling/UI paragraphs -> .claude/rules/*.md, which load when a matching file
|
||||
is read instead of in every session.
|
||||
Full per-block accounting: felhom.eu/documentation/audits/LEDGER-instruction-trim-2026-08-06.md
|
||||
-->
|
||||
1. **Commit and push** all code changes
|
||||
2. **Build, push, and deploy** the new controller image (if controller code changed)
|
||||
3. **Update CHANGELOG.md** with what was done
|
||||
4. **Update CONTEXT.md** with decisions made, update architectural state and what's next
|
||||
5. **Update controller/README.md** if architecture or features changed
|
||||
6. **Verify** the deployment is working (check `docker ps` and logs)
|
||||
+4
-2090
File diff suppressed because it is too large
Load Diff
@@ -1,441 +1,43 @@
|
||||
# REPORT — controller v0.215.0 → v0.216.0: disk-health severity ladder, escalation, and the alert that never sent
|
||||
|
||||
**Date:** 2026-08-14 · **Task class:** Implementation · **Repos touched:** `felhom-controller` (code),
|
||||
`felhom.eu` (documentation only — no hub code, no manifest bump, no ArgoCD sync)
|
||||
|
||||
---
|
||||
|
||||
## 1. Confirmed baselines used (as read at the start of the run)
|
||||
|
||||
| Repo | `main` @ start | Version | → Shipped |
|
||||
|------|----------------|---------|-----------|
|
||||
| felhom-controller | `3e3ee94b7bbe6b66663c468e22aa86616365a45a` | v0.214.0 | **v0.215.0**, then **v0.216.0** (a defect found live in v0.215.0 — §14) |
|
||||
| felhom.eu | `e0b56c976f8e4a7352309d78754fec448dd55f99` | n/a (docs only) | n/a |
|
||||
|
||||
Both trees verified clean (`git status --porcelain` empty, `HEAD == origin/main`) before any build.
|
||||
The controller hash matched the spec's stated baseline exactly. `MinAgent` stays **0.129.0** — no
|
||||
agent change; every field read here has been on the wire since agent v0.94.0/v0.95.0.
|
||||
|
||||
---
|
||||
|
||||
## 2. Files created / modified
|
||||
|
||||
**felhom-controller**
|
||||
- `controller/internal/agentapi/diskverdict.go` — modified (14-row ladder, `DiskPrior`, `UncorrectableSectors`, `TemperatureFailC`)
|
||||
- `controller/internal/agentapi/diskverdict_test.go` — modified
|
||||
- `controller/internal/agentapi/diskverdict_ladder_test.go` — **created**
|
||||
- `controller/internal/notify/notifier.go` — modified (severity, `DiskAlert`, `DiskAlertKind`, `Severity()`, 5 message shapes)
|
||||
- `controller/internal/notify/disk_health_test.go` — rewritten
|
||||
- `controller/internal/web/disk_health_state.go` — **created** (persistence + decision)
|
||||
- `controller/internal/web/disk_health.go` — modified
|
||||
- `controller/internal/web/disk_health_test.go` — rewritten
|
||||
- `controller/internal/web/server.go` — modified (seam signature)
|
||||
- `controller/cmd/controller/main.go` — modified (6h → 1h)
|
||||
- *(v0.216.0)* `controller/internal/web/disk_health.go` + `disk_health_test.go` — the R-335 dedup fix and its test
|
||||
- `CHANGELOG.md`, `CONTEXT.md`, `REUSE.md`, `controller/README.md`, `REPORT.md`
|
||||
|
||||
**felhom.eu** (documentation only)
|
||||
- `documentation/audits/DIAG-smart-passed-trap-2026-08-14.md` — **created**
|
||||
- `documentation/audits/fixtures/smart-ST3000VX010-failing-2026-08-14.json` — **created** (raw `smartctl -a -j`, verbatim)
|
||||
- `documentation/audits/fixtures/smartd-history-sdg-2026-08-14.txt` — **created** (406 `smartd` journal lines)
|
||||
- `documentation/architecture/00-capability-map.md`, `documentation/backlog/ROADMAP.md`, `documentation/backlog/OPEN-ITEMS.md` — modified
|
||||
|
||||
---
|
||||
|
||||
## 3. Commits pushed to `main`
|
||||
|
||||
| Repo | Hash | What |
|
||||
|------|------|------|
|
||||
| felhom.eu | `848de81` | Part 0 — fixtures + findings doc |
|
||||
| felhom-controller | `bb50e12` | Parts 1–3 — ladder, severity, persisted state + tests |
|
||||
| felhom-controller | `c24f192` | Group L strengthened to two post-restart checks |
|
||||
| felhom-controller | `34d83f5` | Part 4 — cadence 6h → 1h (measured) |
|
||||
| felhom-controller | `8144a70` | Part 5 — CHANGELOG / CONTEXT / README / REUSE |
|
||||
| felhom.eu | `767960b` | Part 5 — capability map, ROADMAP, register rows R-328…R-334 |
|
||||
| felhom-controller | `90f2545` | **v0.216.0** — R-335, one physical disk evaluated once per run |
|
||||
| felhom.eu | `fa4748d` | R-335 register row |
|
||||
|
||||
---
|
||||
|
||||
## 4. Per-test results — all twelve groups
|
||||
|
||||
| Group | Scenario | Test | Result |
|
||||
|-------|----------|------|--------|
|
||||
| A | real drive, 2nd observation | `TestDiskCheck_RealDrive_HibaAndOneCriticalEvent` + `TestLadder_RealDrive_ReachesHiba` | **PASS** |
|
||||
| B | the transient that cleared | `TestDiskCheck_FirstSightingIsWarnOnly` | **PASS** |
|
||||
| C | sustained → Hiba | `TestDiskLadder_SustainDrivesTheEscalation` + `TestLadder_SustainIsWhatFires` | **PASS** |
|
||||
| D | recovered, silent | `TestDiskCheck_RecoveryIsSilentAndClearsState` | **PASS** |
|
||||
| E | flap damping | `TestDiskCheck_FlapDamping` | **PASS** |
|
||||
| F | escalation beats damping | `TestDiskCheck_EscalationBeatsDamping` | **PASS** |
|
||||
| G | still getting worse | `TestDiskCheck_RealertWhenStillWorsening` | **PASS** |
|
||||
| H | below both bars | `TestDiskCheck_NoRealertBelowBothBars` | **PASS** |
|
||||
| I | heat | `TestLadder_Temperature` + `TestDiskCheck_TemperatureShape` | **PASS** |
|
||||
| J | no data never alarms | `TestDiskCheck_UnknownNeverAlarmsNorErasesPrior` + `TestLadder_UnknownNeverAlarms` | **PASS** |
|
||||
| K | severity routes | `TestNotifyDiskHealthDegraded_SeverityRoutes` | **PASS** |
|
||||
| L | state survives restart (seam) | `TestDiskCheck_StateSurvivesRestart_ProductionPath` | **PASS** |
|
||||
|
||||
Supporting: `TestLadder_CountBackstopBoundary`, `TestLadder_ZeroPriorIsFailSafe`,
|
||||
`TestUncorrectableSectors`, `TestDegradedAttributes_NamesFailCounters`,
|
||||
`TestNotifyDiskHealthDegraded_{WarnShape,FailShapes,CopyDiscipline}`,
|
||||
`TestDiskAlertDecision_Table`, `TestDiskState_CorruptFileFallsBackToNoPrior`,
|
||||
`TestDiskCheck_DisappearedDiskIsForgotten`, `TestDiskCheck_UnreachableAgentIsInert` — all PASS.
|
||||
|
||||
---
|
||||
|
||||
## 5. Red-proof outcomes — all twelve, individually
|
||||
|
||||
Each mutation was applied by script, **asserted present in the source before the run** (the harness
|
||||
aborts with `MUTATION-NOT-APPLIED` if the target text is absent), the named test run, and the file
|
||||
reverted with `git checkout --`. The tree was confirmed clean after the sweep.
|
||||
|
||||
| # | Mutation applied | Target test | Outcome |
|
||||
|---|------------------|-------------|---------|
|
||||
| A | remove truth-table row 6 (the sustain rule) | `TestDiskCheck_RealDrive_HibaAndOneCriticalEvent` | **RED-PROOF PASSED — FINDING, see below** |
|
||||
| B | make row 9 return `Fail` | `TestDiskCheck_FirstSightingIsWarnOnly` | failed as required |
|
||||
| C | pass a zero `DiskPrior` in `RunDiskHealthCheck` | `TestDiskLadder_SustainDrivesTheEscalation` | failed as required |
|
||||
| D | let Rendben fall through the silence guard | `TestDiskCheck_RecoveryIsSilentAndClearsState` | failed as required |
|
||||
| E | compare against last **observed** verdict, not last **alerted** | `TestDiskCheck_FlapDamping` | failed as required |
|
||||
| F | let damping cover escalations | `TestDiskCheck_EscalationBeatsDamping` | failed as required |
|
||||
| G | remove the re-alert branch | `TestDiskCheck_RealertWhenStillWorsening` | failed as required |
|
||||
| H | make cooldown/doubling an **OR** instead of an AND | `TestDiskCheck_NoRealertBelowBothBars` | failed as required |
|
||||
| I | remove truth-table rows 3 **and** 13 | `TestLadder_Temperature`, `TestDiskCheck_TemperatureShape` | failed as required |
|
||||
| J | let UNKNOWN delete the prior record | `TestDiskCheck_UnknownNeverAlarmsNorErasesPrior` | failed as required |
|
||||
| K | restore `severity := "warn"` | `TestNotifyDiskHealthDegraded_SeverityRoutes` | failed as required |
|
||||
| L | skip loading the persisted state | `TestDiskCheck_StateSurvivesRestart_ProductionPath` | failed as required |
|
||||
|
||||
### A thirteenth red-proof, added after the deploy (R-335)
|
||||
|
||||
| # | Mutation applied | Target test | Outcome |
|
||||
|---|------------------|-------------|---------|
|
||||
| M | delete the `if seen[key] { continue }` dedup guard | `TestDiskCheck_SameDiskTwiceIsEvaluatedOnce` | failed as required |
|
||||
|
||||
Observed failure: `first sighting of an aliased disk must be silent, got 1: [{Label:felhom-backup … Kind:2 Sectors:8}]`
|
||||
— i.e. `Kind:2` is `DiskAlertFailSectors`, a **Hiba on a first sighting of 8 sectors**. Reverted.
|
||||
|
||||
### FINDING — red-proof A passed, and it is the spec's mutation that is at fault, not the code
|
||||
|
||||
The task specified group A's red-proof as *"remove truth-table row 6 → verdict is Warn"*. **That
|
||||
mutation cannot fail a test built on the real drive's values**: the real drive carries **352**
|
||||
unreadable sectors, so with row 6 deleted it still reaches Hiba via **row 8** (count ≥ 64). The test
|
||||
correctly stayed green, so the mutation proves nothing about row 6.
|
||||
|
||||
This was anticipated while writing the tests and is documented in the test's own comment rather than
|
||||
discovered afterwards. **Row 6 is genuinely pinned**, by two tests that hold the counters at **8**
|
||||
(far below the 64 backstop) and vary *only* the prior:
|
||||
|
||||
- `TestLadder_SustainIsWhatFires` (agentapi) — same `SmartSummary`, `DiskPrior{}` → Warn,
|
||||
`DiskPrior{SawUncorrectable:true}` → Fail.
|
||||
- `TestDiskLadder_SustainDrivesTheEscalation` (web) — the event-level twin.
|
||||
|
||||
Both were run under the row-6-deleted mutation and **both failed**, as recorded:
|
||||
`SAME 8 sectors, now sustained = 2 (Figyelmeztetés), want Fail/Hiba` and
|
||||
`severity = "warning", want critical` / `chip = "Figyelmeztetés", want Hiba`. So the invariant is
|
||||
covered; only the spec's chosen mutation was invalid.
|
||||
|
||||
### A second finding, from building red-proof L
|
||||
|
||||
The first version of the Group L seam test ran **one** check after the restart and **passed under the
|
||||
mutation** — because a controller that has forgotten its state is also silent on its first check. The
|
||||
test was strengthened to run **two** checks (commit `c24f192`), after which the mutation fails. This
|
||||
is the exact shape §10 warns about, caught by running the red-proof rather than assuming it.
|
||||
|
||||
---
|
||||
|
||||
## 6. Test count and suite state
|
||||
|
||||
- **Before:** 1391 test functions (at `3e3ee94`) · **After:** 1414 (+23)
|
||||
- `go build ./... && go vet ./... && go test ./...` — **all green**, no failures, no skips introduced.
|
||||
- `python3 controller/scripts/controller_gates.py` — **all 11 gates OK.**
|
||||
|
||||
---
|
||||
|
||||
## 7. Cadence measurement (Part 4)
|
||||
|
||||
Measured on **demo-hp** (Tier 0, disposable), through `fetchDisks`' real path — the agent local API
|
||||
`GET /disks`, not the 60 s card cache. Ten consecutive calls, all **HTTP 200**:
|
||||
|
||||
```
|
||||
0.840558 0.817000 0.815783 0.831992 0.809066
|
||||
0.832899 0.824788 0.804886 0.812539 0.833547 (seconds)
|
||||
```
|
||||
|
||||
| min | median | max | disk count |
|
||||
|-----|--------|-----|------------|
|
||||
| **0.804886 s** | **0.820894 s** | **0.840558 s** | **3 physical rows** across 2 devices (SanDisk X600 M.2 SATA SSD; Toshiba KXG50PNV1T02 NVMe, counted twice as `c11-scratch` + `felhom-backup`) |
|
||||
|
||||
**Branch taken: median < 5 s → `6*time.Hour` → `1*time.Hour`.** The median is ~6× under the bar. The
|
||||
detection argument is the real one: the observed benign excursion lasted about **one hour**, so a
|
||||
6-hourly sampler can land either side of it and then catch the terminal run half a day late.
|
||||
|
||||
No spin-up signature appeared in the timings (uniform ~0.82 s; demo-hp is all-flash), so the
|
||||
measurement did not suggest the spun-down-drive concern. That question is recorded as an Observation
|
||||
below and deliberately **not acted on**.
|
||||
|
||||
---
|
||||
|
||||
## 8. Live validation
|
||||
|
||||
Deployed to **demo-hp guest 9201** via the bootstrap path (`docker pull` →
|
||||
`/etc/felhom-controller-image` → `systemctl restart felhom-controller-bootstrap.service`).
|
||||
|
||||
```
|
||||
gitea.dooplex.hu/admin/felhom-controller:0.215.0 Up 19 seconds (healthy) # 06:23Z
|
||||
gitea.dooplex.hu/admin/felhom-controller:0.216.0 Up 6 seconds (healthy) # after the R-335 fix
|
||||
```
|
||||
|
||||
### Leg 1 — no over-correction (the load-bearing check)
|
||||
|
||||
Method: **endpoint-level** — authenticated `GET /dashboard` on the real controller
|
||||
(`https://felhom.enkisfelhom.hu/dashboard`, HTTP 200, 44 036 bytes), i.e. the exact endpoint the UI
|
||||
invokes; only rendering is skipped. No browser is available on DooPlex.
|
||||
|
||||
Card contents, parsed from the response body:
|
||||
|
||||
| Disk | Chip | Class | Temp |
|
||||
|------|------|-------|------|
|
||||
| KXG50PNV1T02 NVMe TOSHIBA 1024GB | **Rendben** | `state-text-run` | 53 °C |
|
||||
| KXG50PNV1T02 NVMe TOSHIBA 1024GB | **Rendben** | `state-text-run` | 53 °C |
|
||||
| SanDisk X600 M.2 2280 SATA 128GB | **Rendben** | `state-text-run` | 44 °C |
|
||||
|
||||
`Figyelmeztetés` = 0, `Hiba` = 0, `Nincs adat` = 0, `state-text-warn` = 0, `state-text-crit` = 0.
|
||||
**No healthy disk was over-corrected.**
|
||||
|
||||
**Positive observable, at deploy:**
|
||||
`[INFO] [scheduler] Registered periodic job: disk-health-check (every 1h0m0s)` — the new cadence is
|
||||
in force, not merely compiled.
|
||||
|
||||
**Positive observable, per cycle** — two full hourly cycles observed after the deploy, from the
|
||||
container log:
|
||||
|
||||
```
|
||||
2026/08/14 06:23:13 [INFO] [scheduler] Registered periodic job: disk-health-check (every 1h0m0s)
|
||||
2026/08/14 07:23:13 [INFO] [scheduler] Running job: disk-health-check
|
||||
2026/08/14 07:23:14 [INFO] [web] disk-health check complete: 3 disk(s) evaluated, 0 alert(s)
|
||||
2026/08/14 07:23:14 [INFO] [scheduler] Job disk-health-check completed (took 849ms)
|
||||
2026/08/14 08:23:13 [INFO] [scheduler] Running job: disk-health-check
|
||||
2026/08/14 08:23:14 [INFO] [web] disk-health check complete: 3 disk(s) evaluated, 0 alert(s)
|
||||
2026/08/14 08:23:14 [INFO] [scheduler] Job disk-health-check completed (took 843ms)
|
||||
```
|
||||
|
||||
`grep -c disk_health_degraded` over the whole container log: **0**. Both cycles ran (849 ms / 843 ms,
|
||||
matching the §7 measurement), evaluated every disk, and emitted nothing. **Zero alerts from a check
|
||||
that demonstrably ran** — not silence.
|
||||
|
||||
Persisted state written by the first cycle (`/opt/docker/felhom-controller/data/disk-health-state.json`,
|
||||
428 bytes, on the `felhom-controller-data` docker volume, so it survives container recreation):
|
||||
|
||||
```json
|
||||
{"version": 1, "disks": {
|
||||
"path:/var/lib/vz": {"verdict": 1, "saw_uncorrectable": false, ...},
|
||||
"uuid:91d2dc2d-2d28-4929-9bdd-3e11fa2f41ae": {"verdict": 1, "saw_uncorrectable": false, ...}}}
|
||||
```
|
||||
|
||||
`verdict: 1` is `DiskVerdictOK` for both, `saw_uncorrectable: false`, never alerted.
|
||||
|
||||
**Reading those two artefacts against each other is what exposed R-335** — see §14.
|
||||
|
||||
### Leg 2 — the severity fix arrives (the point of the task)
|
||||
|
||||
Two synthetic `disk_health_degraded` events pushed for customer `demo-hp` **through the real hub
|
||||
event endpoint** (`POST https://hub.felhom.eu/api/v1/event`), from the guest's own controller using
|
||||
its own hub credentials — the genuine controller→hub path, not a hand-crafted operator call. Both
|
||||
returned `HTTP 200 {"ok":true}`. The hub DB was read with its `-wal` and `-shm` copied alongside
|
||||
`hub.db` (a `hub.db`-only read is stale).
|
||||
|
||||
**As STORED by the hub (`events`):**
|
||||
|
||||
| id | severity pushed | severity STORED |
|
||||
|----|-----------------|-----------------|
|
||||
| 2964 | `warning` | **`warning`** |
|
||||
| 2965 | `warn` | **`info`** ← coerced |
|
||||
|
||||
**`notification_log` rows for those two events:**
|
||||
|
||||
| id | event_type | severity | channel | status | error |
|
||||
|----|-----------|----------|---------|--------|-------|
|
||||
| 689 | `disk_health_degraded` | `warning` | `operator` | **`sent`** | *(none)* |
|
||||
| — | *(the `"warn"` push)* | — | — | **NO ROW EXISTS** | — |
|
||||
|
||||
**That pair is the proof.** The identical event, differing only in one word of the severity string,
|
||||
is the difference between *delivered to the operator* and *stored as an informational notice and
|
||||
delivered to nobody*. This is the first time this leg has been observed end to end.
|
||||
|
||||
Only the operator leg fired because **demo-hp has no `customer_notifications` row at all** (no
|
||||
customer email, no `enabled_events`), so no customer row was possible for either push — verified
|
||||
directly, not assumed. **One real email was sent to the operator**, as the task anticipated.
|
||||
|
||||
---
|
||||
|
||||
## 9. NOT yet live-validated — stated explicitly
|
||||
|
||||
**The Fail-from-counters path has never fired on real hardware.** Everything in §4/§5 exercises it
|
||||
against the committed fixture's values in unit tests only. The live legs above prove the *negative*
|
||||
(no false alert on three healthy disks) and the *severity wire* (end to end, through the hub) — they
|
||||
do **not** prove a live disk reaching Hiba. The fixture tests must not be read as a live proof.
|
||||
|
||||
Tracked as **R-332 (WATCHING)**. Closing condition: a live disk reaching Hiba from counters, or a
|
||||
deliberate injection through the real pipeline (agent `/disks` → controller check → hub event) — not
|
||||
a hand-set verdict.
|
||||
|
||||
**One item originally listed here has since been proven live** and is no longer part of this gap: the
|
||||
**persisted state surviving a controller restart**. The v0.215.0 → v0.216.0 redeploy destroyed and
|
||||
rebuilt the container, and the new one read back a `changed_at` written by the previous version rather
|
||||
than re-baselining — see §14. What remains unproven is the stronger half: an already-**alerted** disk
|
||||
not re-alerting after a restart, which needs a disk that has actually alerted. The drive that produced the fixture lives in DooPlex, which is Tier 2 and never a
|
||||
drill target; the demo boxes are all-flash and healthy.
|
||||
|
||||
---
|
||||
|
||||
## 10. Teardown
|
||||
|
||||
**This run provisioned nothing** — no VM, no guest, no hub customer record, no storage. Nothing was
|
||||
formatted, mounted, unmounted, repaired or written on any monitored disk; the only write is the
|
||||
controller's own `disk-health-state.json` inside its data volume.
|
||||
|
||||
Disposition of what the run did create:
|
||||
|
||||
- **Two synthetic hub events (`events` id 2964, 2965) and one `notification_log` row (id 689)** on the
|
||||
live hub. **Left in place deliberately.** Both messages are self-labelling
|
||||
(`"R-328 severity probe (…) - synthetic, no real disk fault"`), and deleting rows from the
|
||||
production hub DB is a riskier act than leaving two clearly-marked probe rows. Named here so they
|
||||
are not mistaken later for a real disk fault on demo-hp.
|
||||
- **One real operator email** resulting from row 689.
|
||||
- A local copy of `hub.db`/`-wal`/`-shm` in the session scratchpad only (not committed, not exported).
|
||||
|
||||
---
|
||||
|
||||
## 11. Register rows
|
||||
|
||||
| Row | State | Owner |
|
||||
|-----|-------|-------|
|
||||
| **R-328** — the severity drop: `"warn"` coerced to `info`, emailed to nobody | **CLOSED** (controller v0.215.0), proven live side by side | CC |
|
||||
| **R-329** — `app_start_failed` carries the identical defect | READY — **not fixed here**; needs a decision on whether it should notify at all | Viktor |
|
||||
| **R-330** — Phase 2: collect SMART attrs 187/199/188 + persist samples | READY — a declared wire change, hub models it in the same session under G-1 | CC |
|
||||
| **R-331** — Phase 3: growth-rate detection; revisit the static 64 | READY, blocked on R-330 | CC |
|
||||
| **R-332** — the Fail path has never fired on real hardware | **WATCHING** | CC |
|
||||
| **R-333** — NVMe temperature bands; agent `smartctl` has no `-n standby` | READY (S each) | Viktor decides (a); CC does (b) |
|
||||
| **R-334** — released with no golden carrying it (gate waiver) | READY — now applies to **v0.216.0** | CC bakes; **Viktor vouches** |
|
||||
| **R-335** — one physical disk walked twice per run, sustaining against itself | **CLOSED** (controller v0.216.0) | CC |
|
||||
|
||||
`smartd`-on-DooPlex-alerts-nobody is recorded in `DIAG-smart-passed-trap-2026-08-14.md` §8 as the
|
||||
same shape one layer out.
|
||||
|
||||
---
|
||||
|
||||
## 12. Observations — noticed, NOT acted on
|
||||
|
||||
1. **`app_start_failed` has the identical severity defect** (`notifier.go` ~L546, `"warn"`). Left
|
||||
untouched per scope. It needs a prior decision — should a stopped app email the customer at all? —
|
||||
because flipping the string alone converts a silent event into a mail flood on a crash-looping box.
|
||||
**R-329.**
|
||||
|
||||
2. **The 55/60 °C bands are spinning-disk bands being applied to NVMe, and this is close to biting.**
|
||||
Adopted unchanged from the operator's Prometheus config by explicit decision — but demo-hp's
|
||||
**healthy** Toshiba NVMe idles at **53 °C**, i.e. **2 °C below Figyelmeztetés and 7 °C below Hiba**,
|
||||
and NVMe routinely passes 60 °C under sustained write with no fault. As shipped, a healthy customer
|
||||
NVMe under load can be reported as **Hiba** — the single worst outcome this feature can produce, and
|
||||
the one leg 1 exists to guard. Not changed here because the threshold is a stated, settled operator
|
||||
decision; flagged rather than overridden. **R-333(a) — recommend splitting the bands by device
|
||||
class, or dropping them for NVMe and relying on `critical_warning`.**
|
||||
|
||||
3. **The agent runs bare `smartctl -a -j` with no `-n standby`**
|
||||
(`felhom-agent/internal/storage/hostops.go:368`), so every poll wakes a spun-down drive, and 6h → 1h
|
||||
multiplies that by six. Recorded, not acted on, per the task's instruction. demo-hp is all-flash so
|
||||
the measurement could not reveal it. Mitigating datum from the fixture: the failing drive logged
|
||||
only **3375 load cycles in 60505 power-on hours** (~one per 18 h), so this duty cycle barely spins
|
||||
down at all. **R-333(b).**
|
||||
|
||||
4. **`source ~/.config/credentials` prints two recovery codes to the terminal.** The file contains
|
||||
hyphenated keys (`R_DEMO-FELHOM`, `R_DEMO-HP`) that bash cannot assign, so sourcing it emits
|
||||
`command not found` errors **containing the secret values**. Anything that sources that file leaks
|
||||
them into logs, scrollback and transcripts. Not a code defect and out of scope; worth quoting
|
||||
values from it by other means, or renaming the keys.
|
||||
|
||||
5. **`golden_currency_gate.py` has no waiver parser.** Its own failure text says *"record a waiver in
|
||||
`OPEN-ITEMS.md` — never a bypass"*, but nothing reads such a waiver, so the only way past it is the
|
||||
bypass it warns against. See §13.
|
||||
|
||||
---
|
||||
|
||||
## 13. Deviations, stated plainly
|
||||
|
||||
- **`git push --no-verify` was used once**, on the `felhom.eu` docs push (`767960b`), and only there.
|
||||
Cause: `golden_currency_gate.py` correctly convicts the fact that controller **v0.215.0 is released
|
||||
and no golden carries it** (newest bake 0.214.0), so a *newly installed* machine would receive
|
||||
0.214.0 — without the severity fix. A golden bake was out of the task's scope, and its second half
|
||||
(vouching in the hub's day-0 artifact manifest) is operator-password-gated, so CC cannot complete it;
|
||||
a baked-but-unvouched golden is worse than none. Recorded as **R-334** with the bake+vouch owners
|
||||
named. CI re-runs the same entry point and will mail the operator. The running fleet is unaffected.
|
||||
- **One pre-existing test changed meaning by design:** `TestDiskVerdictFor`'s
|
||||
`critical_warning>0 → warn` case is now `→ fail` (truth-table row 4 — NVMe's own critical flag is a
|
||||
device declaration, not a drifting counter). `TestDiskHealthCheck_DegradationOnce` and its siblings
|
||||
were rewritten into the scenario groups because they encoded the pre-v0.215.0 single-alert behaviour
|
||||
the task deliberately replaces (Scenario C).
|
||||
|
||||
---
|
||||
|
||||
## 14. R-335 — a defect in v0.215.0, found live, fixed as v0.216.0
|
||||
|
||||
**How it was found.** Not by a test and not by review: by reading the release's own **positive
|
||||
observable** against the release's own **persisted artefact**. The hourly check logged *"3 disk(s)
|
||||
evaluated"*; `disk-health-state.json` held **two** records. Two artefacts that should have agreed did
|
||||
not.
|
||||
|
||||
**Cause.** demo-hp's `c11-scratch` and `felhom-backup` are the same physical NVMe (`/dev/nvme0n1`) and
|
||||
resolve to the same `diskKey`, so one disk was walked twice in a single run.
|
||||
|
||||
**Why it mattered.** `RunDiskHealthCheck` writes a disk's new record before the next entry reads it, so
|
||||
the **second** copy of an aliased disk consumed the **first** copy's write as its prior. The disk
|
||||
therefore **sustained against itself and reached Hiba on a first sighting** — defeating truth-table
|
||||
row 6, the single rule separating a one-hour benign excursion from a false critical alert — and would
|
||||
have emitted **two identical events** for one drive.
|
||||
|
||||
**Severity in practice: latent, not active.** Nothing fired on demo-hp because all three entries are
|
||||
healthy with zero counters. But any aliased disk developing one pending sector would have gone
|
||||
straight to Hiba, which is precisely the outcome §8 leg 1 exists to prevent. Aliasing is not exotic —
|
||||
it is the *normal* shape whenever a box has two PVE storage entries on one physical device.
|
||||
|
||||
**Fix (v0.216.0, `90f2545`).** Each `diskKey` is evaluated once per run. Both entries stay marked
|
||||
`seen`, so neither is mistaken for a disappeared disk, and the card still renders **both** storage
|
||||
rows — the dedup is about state and alerts, not display. Pinned by
|
||||
`TestDiskCheck_SameDiskTwiceIsEvaluatedOnce`, red-proof run and reverted (§5).
|
||||
|
||||
**Deployed:** `gitea.dooplex.hu/admin/felhom-controller:0.216.0 Up 6 seconds (healthy)`.
|
||||
|
||||
**Confirming cycle on v0.216.0 — CONFIRMED LIVE, 09:31:35Z:**
|
||||
|
||||
```
|
||||
live image: gitea.dooplex.hu/admin/felhom-controller:0.216.0 Up About an hour (healthy)
|
||||
2026/08/14 09:31:35 [INFO] [web] disk-health check complete: 2 disk(s) evaluated, 0 alert(s)
|
||||
grep -c disk_health_degraded: 0
|
||||
```
|
||||
|
||||
**`2 disk(s) evaluated` now matches the 2 persisted records.** The count and the artefact agree, which
|
||||
is the disagreement that exposed R-335 in the first place. Still zero alerts, still both card rows.
|
||||
|
||||
### The redeploy also proved persistence live — a gap §9 had listed as unproven
|
||||
|
||||
The 0.215.0 → 0.216.0 redeploy **replaced the container**, and the state file came back intact:
|
||||
|
||||
```json
|
||||
"path:/var/lib/vz": {"verdict": 1, "changed_at": "2026-08-14T07:23:14.640216851Z", ...}
|
||||
"uuid:91d2dc2d-…": {"verdict": 1, "changed_at": "2026-08-14T07:23:14.640216851Z", ...}
|
||||
```
|
||||
|
||||
That `changed_at` was written by **v0.215.0's first cycle at 07:23Z**, before the container was
|
||||
destroyed and rebuilt. The v0.216.0 container read it back and preserved it rather than stamping a
|
||||
fresh time — so the new container **loaded the pre-restart record instead of silently re-baselining**.
|
||||
That is Scenario L observed on real hardware, not just through the production-path unit test, and it
|
||||
is exactly the behaviour that was impossible before v0.215.0 (the baseline was in-memory).
|
||||
|
||||
It also incidentally confirms the unchanged-verdict path: `changed_at` is preserved across four checks
|
||||
and two controller versions because the verdict never changed, rather than being churned every cycle.
|
||||
|
||||
**What this still does NOT prove:** these disks are healthy and were never alerted, so the stronger
|
||||
half — *an already-ALERTED disk not re-alerting after a restart* — remains unit-tested only. R-332
|
||||
stands.
|
||||
|
||||
**Process note, recorded because it nearly cost the fix.** The red-proof harness reverts with
|
||||
`git checkout --`, which restores to `HEAD`. Running a red-proof against an **uncommitted** fix
|
||||
therefore *deletes the fix* along with the mutation — which happened here and was caught only by
|
||||
re-grepping the source afterwards. Commit the fix before red-proofing it, or snapshot outside git.
|
||||
# REPORT — felhom-controller v0.51.0
|
||||
|
||||
Offsite-backup UI (felhom-pbs = real DR) + Model-A double-nest fix. Pairs with felhom-agent v0.28.0
|
||||
(whole-guest backup re-targeted to the offsite PBS tier). Live-deployed in guest 9201 on demo-felhom.
|
||||
|
||||
## Backups page — whole-guest backup shown as real DR
|
||||
- `backupTargetLabel` returns **"Biztonsági szerver – külön hardver (PBS)"** for a PBS-stored backup
|
||||
(detected via `backupIsPBS` on the target id / archive volid), so the customer sees the backup
|
||||
survives a host hardware failure.
|
||||
- The app-data section's **"Távoli mentés"** card stops reading "nincs beállítva": new
|
||||
`guestBackupView.Offsite` flag drives it to **"külön hardveren (PBS)"** with a ✓ when the whole-guest
|
||||
backup landed on PBS.
|
||||
- The restore-test "Visszaállítás ellenőrizve" trust signal is unchanged (already wired).
|
||||
- Live: agent `/backup/status` reports `target_id=felhom-pbs`; `/restore-test/status` reports
|
||||
`pass:true, verified:"boot+running", source_tier:"pbs"` → the page renders the PBS label, the offsite
|
||||
card, and verified-restorable.
|
||||
|
||||
## Model-A double-nest fix
|
||||
- Under slice-10 Model A the host agent binds `<drive>/felhom-data` onto the guest mountpoint, so an
|
||||
enrolled drive's in-guest mount IS the felhom-data namespace root (basename need not be `felhom-data`,
|
||||
e.g. `/mnt/felhom-usb`). The backup path helpers were re-prepending `felhom-data`, producing
|
||||
`.../felhom-data/felhom-data/...` on the host (confirmed live: `/mnt/felhom-usb/felhom-data/felhom-data/...`).
|
||||
- `appbackup` path helpers now take a **namespace ROOT** (no internal `felhom-data` join) plus a new
|
||||
`NamespaceRoot(drivePath, inGuestDrive)`. `backup.Manager.namespaceRoot`/`AppNamespaceRoot` resolve
|
||||
provenance (`drivePath != systemDataPath` ⟺ a registered in-guest drive → namespace root as-is; the
|
||||
SSD-only `systemDataPath` fallback appends `felhom-data`).
|
||||
- All parallel constructions updated coherently so writes, deletion (`GetStackBackupData`,
|
||||
`RemoveStack` backups-base + `ProtectedHDDPaths` — legacy double-nest dirs KEPT protected), the
|
||||
wipe-warning secondary scan, and export all agree. `api.router` passes the namespace root across the
|
||||
package boundary. Result: a drive-resident app's DB-dump lands single-nested at `<drive>/backups/...`
|
||||
in-guest = `<drive>/felhom-data/backups/...` on the host.
|
||||
- New `appbackup` test asserts no doubled `felhom-data` segment for an in-guest drive and exactly one
|
||||
for the system fallback. Full `go build ./...` + tests green.
|
||||
|
||||
## Decommission (P3) — NO controller change
|
||||
- Permanent decommission is operator-signature-gated (never customer-confirmable), so it is wired
|
||||
entirely agent-side (hub jobs-queue → signed-jobs runner). The controller deliberately exposes no
|
||||
decommission UI. (felhom-agent v0.28.0.)
|
||||
|
||||
## Live deploy
|
||||
- `gitea.dooplex.hu/admin/felhom-controller:0.51.0` running + healthy in guest 9201 (bootstrap-launched
|
||||
via `/etc/felhom-controller-image`; prior 0.50.0). Startup clean (catalog sync, health ok,
|
||||
FileBrowser mounts synced).
|
||||
|
||||
@@ -1,328 +0,0 @@
|
||||
# REUSE.md — felhom-controller
|
||||
|
||||
> Before writing new code, check here. Canonical helpers, patterns to copy, traps to avoid.
|
||||
> Maintenance: update in the SAME commit that adds/changes/deprecates a shared helper.
|
||||
> Entries cite file + symbol. Line numbers are landmarks only — reconfirm before editing.
|
||||
|
||||
## 1. Canonical helpers (MUST reuse — do not reinvent)
|
||||
|
||||
### Paths & namespaces (felhom-data layout)
|
||||
|
||||
| Symbol | File | Short signature | Use for | Gotchas |
|
||||
|---|---|---|---|---|
|
||||
| `NamespaceRoot` | controller/internal/appbackup/paths.go | `(drivePath string, inGuestDrive bool) string` | Resolve felhom-data root for a drive | `inGuestDrive=true` returns path AS-IS (Model A: guest mount IS the ns root); false appends `felhom-data`. Never double-nest |
|
||||
| `PrimaryBackupPath` / `RecoveryUnitPath` / `RecoveryUnitComposePath` / `RecoveryUnitManifestPath` | controller/internal/appbackup/paths.go | `(nsRoot[, stackName]) string` | All backup dir layout | Take the NAMESPACE ROOT, not a bare drive path |
|
||||
| `AppDBDumpPath` / `AppVolumeDumpPath` / `AppDataDir` | controller/internal/appbackup/paths.go | `(nsRoot, stackName) string` | Per-app dump/data dirs | Same nsRoot contract. `AppDataDir`'s final segment is the app's real appdata dir NAME — NOT always the stack name (paperless-ngx → `paperless`); resolve via `AppDataDirNames` first (F-S2/F-S3) |
|
||||
| `AppDataDirNames` / `AppDataBindsPresent` | controller/internal/appbackup/paths.go | `(hddPath, stackName string, hddMounts []string) []string` / `(hddPath, hddMounts) bool` | Resolve the real `appdata/<name>` dir(s) from compose `${HDD_PATH}` binds (F-S2/F-S3) | `hddMounts` = ParseComposeHDDMounts shape. Deduped+sorted; falls back to `[stackName]` when no appdata bind. Tier-2 (`backup.Manager.tier2AppDataName`) refuses N>1; migrate (`stacks.Manager.ResolveAppDataDirNames`) loops N. `BindsPresent` drives the WARN-on-missing-declared-dir |
|
||||
| `UserdataDir` / `ImportDir` / `EnsureUserdataSkeleton` / `EnsureDirOwned` | controller/internal/appbackup/userdata.go | `(nsRoot)` / `(nsRoot)` / `(nsRoot, dirs []string)` / `(path, gid int)` | userdata/ tree w/ 2775 setgid gid-1000 convention. **R-75:** `ImportDir` is the CANONICAL drop-zone (`<nsRoot>/userdata/import`) and callers MUST resolve it against the SYSTEM namespace, never an app's HDD_PATH — use `stacks.Manager.GetImportRoot()`. `EnsureUserdataSkeleton` now takes the dir set: build it with `BuildUserdataSkeleton(DeriveUserdataDirs(stacksDir))`, or via `Manager.EnsureUserdataSkeleton` / `web.Server.ensureUserdataSkeleton`. | Linux-only chown via build-tag twin userdata_linux.go. **The set MUST stay sorted** — `fbNeedsRecreate` force-recreates FileBrowser on any byte diff and the naive map-order derivation measured 20/20 distinct (SPIKE P6). `UserdataSkeletonCarry()` is the old hardcoded list, retained forever so derivation can only ADD (zero removals). |
|
||||
| `BuildUserdataSkeleton` / `UserdataSkeletonCarry` / `DeriveUserdataDirs` | appbackup/userdata.go, stacks/skeleton_derive.go | `([]string)` / `()` / `(stacksDir)` | catalog-derived userdata skeleton (R-75) | Derives `${USERDATA_PATH}` binds only — `${IMPORT_PATH}` is NOT part of a drive skeleton (one root, system drive, `Manager.EnsureImportRoot`). Do NOT wire the catalog sync to `SyncFileBrowserMounts`. |
|
||||
| `appbackup.ValidateRelPath` / `ValidRoot` | controller/internal/appbackup/classify.go | `(root, path)` / `(root)` | THE single path-safety refusal set for every `${VAR}`-relative catalog path | Shared by `backup:` and `data_paths:`. **Do not write a second path validator.** |
|
||||
| `stacks.ValidateDataPaths` | controller/internal/stacks/datapaths.go | `(entries, binds, appName, logger)` | `data_paths:` annotation validation | ASYMMETRIC on purpose (Fork-3): malformed PATH ⇒ whole-block reject (data handling, `backup:` precedent); unknown ROLE ⇒ fails OPEN, one WARN (presentation, `Lifecycle` precedent). |
|
||||
| `web.fileBrowserLink` / `importFolderLink` | controller/internal/web/filebrowser_link.go | `(domain, sourceName, relPath)` | FileBrowser Quantum deep link | Template read out of the shipped router (SPIKE P2). **`url.PathEscape` per segment — NEVER `QueryEscape`** (space→`+` is a literal plus in a path). Let `html/template` do the attribute escaping; do not pre-escape. |
|
||||
| `HumanizeBytes` | controller/internal/appbackup/appdata.go | `(b int64) string` | Human byte sizes | Exported canonical; private clones exist (§6) |
|
||||
| `stablePathForName` / `agentWhere` | controller/internal/web/intermediary.go | `(name/registeredPath) string` | Map registry stable path `/mnt/felhom-drives/<n>` ↔ raw agent mount | Registry stores STABLE path; agent ops take the RAW mount — always convert |
|
||||
| `offsiteRestoreRootFor` | controller/internal/backup/offbox_verify_copies.go | `(drivePath string) string` | THE only place `backups/offsite-restore` is spelled | `offboxRestoreScratchDir` builds on it — the listing/delete surface MUST resolve byte-identical paths to what the restore wrote. Do not re-hardcode the segments (they were open-coded in 3 places before v0.147.0) |
|
||||
| `ProtectedHDDPaths` | controller/internal/stacks/delete.go | `(hddPath string) map[string]bool` | Never-delete set (root, appdata, backups, media, legacy felhom-data) | Consult before ANY recursive delete under a drive |
|
||||
|
||||
### Subprocess + timeout + exit-code discipline
|
||||
|
||||
| Symbol | File | Short signature | Use for | Gotchas |
|
||||
|---|---|---|---|---|
|
||||
| `Manager.composeExec` / `composeExecCustomEnv` | controller/internal/stacks/manager.go | `(dir string, [env,] args...) (string, error)` | ALL docker-compose invocations | Logs env KEYS only (secrets safety), truncates output to 500, extracts exit code; `up` triggers userdata pre-create belt. NO timeout — see §3 |
|
||||
| `rsyncCopy` | controller/internal/stacks/migrate.go | `(ctx, src, dst, onBytes)` | Additive copy (migration/moves) | `-a --checksum`, NEVER `--delete`; progress2 byte callback; ctx timeout |
|
||||
| `rsyncVerify` | controller/internal/stacks/migrate.go | `(ctx, src, dst) error` | Post-copy verification | Dry-run `-ani`; fails on any pending content transfer; attr-only lines ignored |
|
||||
| `walkMerge` | controller/internal/stacks/migrate.go | `(lg, srcNS, dstNS, skip, assertOnly, onBytes)` | Collision-safe userdata merge | Renames to lowest-free sibling on content mismatch; additive |
|
||||
| `runCommand` / `runCommandStdin` | controller/internal/selfupdate/updater.go | `(name, args...) (string, error)` | docker CLI in updater | stdin variant for `docker login --password-stdin` (no secret in argv); package VARS since v0.112.0 — override in tests (fakeRunner in registry_anon_test.go) |
|
||||
| `parseWWWAuthenticate` + `fetchAnonymousToken` | controller/internal/selfupdate/updater.go | Bearer-challenge parse + anonymous Docker v2 token | Any credential-free registry API access | realm comes FROM THE HEADER (never hardcode a token URL); denial = errAnonymousDenied, never "credentials missing" |
|
||||
| `Syncer.runGit` / `runGitInDir` | controller/internal/sync/sync.go | `(args...) error` | git CLI ops | Credentials masked in logs via `maskRepoURL` |
|
||||
|
||||
### HTTP/JSON envelopes + flash messages
|
||||
|
||||
| Symbol | File | Short signature | Use for | Gotchas |
|
||||
|---|---|---|---|---|
|
||||
| `writeJSON` | controller/internal/api/router.go | `(w, status, v)` | REST API (`/api/*` router) responses | Pair with `apiResponse{OK,Data,Error}` envelope |
|
||||
| `writeDiskJSON` | controller/internal/web/agent_disk_handlers.go | `(w, status, ok, errMsg, data)` | Storage/disk web-API responses | The `{ok,error,data}` envelope the storage JS expects |
|
||||
| `jsonResponse` / `jsonError` | controller/internal/web/handler_export.go | `(w, v)` / `(w, msg, code)` | Export/import API | Third envelope shape — keep within export surface |
|
||||
| `limitBody` | controller/internal/api/router.go | `(w, req)` | Bound request bodies (1MB) | Apply before decode on any new POST |
|
||||
| `offboxRedirect` | controller/internal/web/offbox_handlers.go | `(w, r, msg string, isErr bool)` | Flash-message redirects | Flash = `?flash=` / `?flash_error=` query params, read by page handlers |
|
||||
| `offboxRedirectTo` | controller/internal/web/offbox_handlers.go | `(w, r, page, msg string, isErr bool)` | Same, to an EXPLICIT page | **TRAP (fixed v0.154.0): the separator is chosen, not `"?"`.** Targets may already carry a query — the R-48 wizard is `/backups/restore/app?name=<app>` — and a hardcoded `"?"` buries the flash inside the previous parameter's value |
|
||||
| `restoreOpInFlight` + `hasRecentRestoreResult` | controller/internal/web/restore_wizard.go | `(backup.RestoreOpStatus) bool` / `(st, app, now) bool` | THE "is a restore running / did one just finish" display reads | **TRAP (v0.154.0 shipped this bug): `Manager` has TWO running flags.** `IsRunning()` reads the CONCURRENCY flag, acquired inside the goroutine — and `RestoreOffboxScratch` never acquires it, so it is false for the whole verification restore. Display must read `RestoreStatus().Running` (set synchronously by `BeginRestoreOp`). Read the status ONCE per render or the strip and the suppression can disagree. `hasRecentRestoreResult` is app-bound and window-bounded — a process-wide result must not light another app's „Eredmény" |
|
||||
| `restoreWizardPath` / `deriveWizardStep` / `resolveWizardApp` | controller/internal/web/restore_wizard.go | `(app) string` / `(restoreWizardInput) restoreWizardView` / `([]OffboxAppRow, name) *OffboxAppRow` | R-48 offsite restore wizard: URL builder + the PURE step/unlock derivation + the app-resolution refusals | The step is **never** taken from the request. Precedence is load-bearing: op-running outranks a stale `?full_prep=`, else a commit button reappears mid-restore. Truth table + red-proof: `restore_wizard_test.go`. Adding a form here that posts anywhere new breaks `TestRestoreWizard_NoNewMutationEndpoints` **by design** — R-48 adds no mutation surface |
|
||||
| `redirectTier2` | controller/internal/web/tier2_config_handler.go | `(w, r, name, flash, flashErr)` | Tier2 page flash redirects | Same convention |
|
||||
| `validStackName` | controller/internal/web/validate.go | `(name string) bool` | Any stack name from a request | Single-segment, no `/ \ ..` — blocks path traversal into stacks/userdata |
|
||||
| `ValidateSegment` | controller/internal/appexport/validate.go | `(kind, s string) error` | Any attacker-controlled path segment (.fab manifest fields) | CTRL-001 guard; deliberately NOT for dotfile ConfigFiles |
|
||||
| `validateSubdomain` / `SubdomainInUse` | controller/internal/stacks/deploy.go | `(s)` / `(subdomain, excludeStack)` | Subdomain fields on deploy | — |
|
||||
|
||||
### Crash-safe journal / atomic writes
|
||||
|
||||
| Symbol | File | Short signature | Use for | Gotchas |
|
||||
|---|---|---|---|---|
|
||||
| `backup.ErrOffboxSealedPackageHeld` + `IsOffboxSealedPackageHeld` + `sealedPackageHeld` + `OffboxAwaitingRecoveryKey` (R-241, v0.206.0) | controller/internal/backup/offbox.go | sentinel; `(error) bool`; `() bool`; `() bool` | **THE MINT GUARD** — a box never creates a repository key while the hub holds a sealed package for it | **The guard is a CONJUNCTION** (package held AND no key present). Widening it to "never mint" leaves a first-time box unable to start, waiting for a package that will never exist — pinned by `TestR241_ScenarioB_FirstTimeBoxStillMints`. **The refusal is a HOLDING state, not a failure:** `ApplyOffsiteTarget` catches the sentinel and still writes the transport, so `/recovery`'s synchronous tier-up (R-219) can bring the tier up the instant the key arrives; returning the error instead leaves `needsOffsiteCredential` true and the hub re-staging a consumed credential for ever. `OffboxAwaitingRecoveryKey` is **DERIVED, never stored** — and **`t.Enabled` is load-bearing in it**: a customer who switched off-site OFF is not awaiting anything (the Scenario-E carve-out `needsOffsiteCredential` makes two functions above; the first draft omitted it and an existing test caught it). A nil settings store reads as "no package held" — a transient read failure must never become a permanently-held tier |
|
||||
| `settings.HubEscrowKeySHA256` + `SetHubEscrowKeySHA256` / `GetHubEscrowKeySHA256`, and `OffsiteRecoveryOffer` **shape (c)** (R-241, v0.206.0) | controller/internal/settings/settings.go, controller/internal/backup/offbox.go | `(sha, checkedAt string) error` / `() (string, string)` | **THE DISCRIMINATOR the recovery screen asks** — does the hub hold a package for a key other than the one we use? | **The comparison was ALREADY computed on every ACK since SLICE 3 and persisted nowhere** — that is R-241's second half. Wire the recorder in `main.go`'s `EscrowAutoConfirmer` literal or shape (c) reads an empty hash for ever and the fix ships INERT (pinned by `TestMainWiresRecordEscrowKeyHash`). **§7.2 staleness, decided:** a KNOWN DIFFERENCE offers **however old the reading** — age is deliberately NOT gated on, because gating makes a box offline from the hub silently stop offering; an **ABSENT hash falls back to (a)/(b)** and does NOT offer, because `""` is the hub positively saying its package seals no key (legacy hash-less escrow), not an unknown. `CheckedAt` is for diagnosis, never a gate |
|
||||
| `backup.AbandonStatus` / `AbandonSweep` / `CancelAbandon` / `ClearAbandonPurgeIfConfirmed` / `ExtendAbandon` / `StopAbandon` + `AbandonGraceDays` (R-241, v0.206.0) | controller/internal/backup/offbox_abandon.go | see file | **The 14-day abandonment countdown** — the ONLY thing in the product that deletes a customer's off-site history | **BOTH HALVES OR NEITHER.** The set-aside store and the sealed package that protects it are two halves of one thing; removing only one leaves a package that opens nothing, or ciphertext nobody can decrypt. Not atomic across two machines, so it is a **two-phase commit**: delete the store, set `AbandonPurgeRequested`, and keep declaring it until the hub's ACK stops reporting a superseded package — the confirmation rides the SAME ACK as the request. **The countdown starts in `ResetOrphanedRepo`, NOT in the shared `resetOrphanedRepo`** — the helper is also the UNCLAIMED auto-reset, where nobody decided anything. **The recovery offer stays reachable for the whole grace** (a grace in which recovery is impossible is decorative). **Drive it with `SetOffboxClock`, never a shortened live timer** (§7.4). A transport failure leaves the countdown DUE so tomorrow retries; the operator levers REFUSE rather than no-op when nothing is running or the store is already gone |
|
||||
| `settings.SyncRecoveryOfferEpoch` / `PostponeRecoveryNoticeForEpoch` / `OptOutRecoveryRemindersForEpoch` + `web.recoveryBannerCookie` (R-241, v0.206.0) | controller/internal/settings/settings.go, controller/internal/web/recovery_handlers.go | `(offered bool, now) (RecoveryOfferView, error)` | **The offer EPOCH** — "once per entry into the offered state", not once ever | **Sync the epoch FIRST and UNCONDITIONALLY in `recoveryInterrupts`.** The first draft returned early when the offer was false, so the FALLING edge was never recorded, `RecoveryOfferActive` stayed true through a settled period, and the next entry counted as a continuation — **the exact defect the epoch exists to fix, reintroduced inside the fix**. Dismissals are recorded against the epoch they were made in, so a fresh entry resets them **by arithmetic**, with nothing to clear. **Three levers, three scopes, and NONE removes the entry point on `/backups/remote`:** the banner cookie is a browser SESSION cookie (no MaxAge — cleared on login) and persists nothing; the reminder opt-out is durable but silences the BANNER ONLY; "most nem" suppresses the full page only |
|
||||
| `atomicWrite` | controller/internal/backup/recovery_unit.go | `(path, data, perm) error` | Atomic file writes (backup pkg) | tmp+rename; no dir creation, no fallback |
|
||||
| `writeFileAtomic` | controller/internal/bootstrap/bootstrap.go | `(path, b) error` | controller.yaml writes from bootstrap | Always 0600 (holds local-api token + hub key) |
|
||||
| `writeConfig0600` | controller/internal/api/router.go | `(path, body) error` | config writes via API | ALWAYS chmods 0600 even pre-existing (F8); direct-write fallback on bind-mount EBUSY (non-atomic!) |
|
||||
| `atomicWriteFile` | controller/internal/setup/handlers.go | `(path, data, perm) error` | setup-wizard writes | Same bind-mount fallback caveat |
|
||||
| `Settings.save` (unexported) | controller/internal/settings/settings.go | via mutator methods only | ALL settings.json persistence | tmp+rename, then `.bak` last-known-good AFTER rename succeeds. Never write settings.json by hand |
|
||||
| `settings.Load` | controller/internal/settings/settings.go | `(path, logger) (*Settings, error)` | Startup load | Corruption recovery: `.bak` restore → else preserve `.corrupt-<ts>` + safe defaults; never crash-loops |
|
||||
| `Manager.writeJournal` / `loadJournal` | controller/internal/stacks/migrate.go | `(j *MigrationJob)` | Migration crash journal | Enables `RecoverMigration` at startup |
|
||||
| `backup.SharesPseudoStack` / `DisplayStackName` | controller/internal/backup/shares_payload.go | `"_shares"` / `(key) string` | THE reserved key for the shares source (restic tag, `backups/secondary/_shares`, CrossDriveBackup record) + its display mapping | NEVER let the raw key reach a Hungarian surface — map at the notification/prose boundary ONLY; the persisted `EnlargedBlocked` set and the templates index by the RAW key |
|
||||
| `Manager.buildSharesPayload` / `classifiedShares` | controller/internal/backup/shares_payload.go | `() (dir, passdbOK, error)` / `() []classifiedShare` | the definitions+credential payload and the availability-filtered share set both tiers read | payload is SECRET-BEARING (0600 passdb.tar) — never log its bytes/name at INFO. `classifiedShares` is the single place a dead mount is dropped, so both jobs agree |
|
||||
| `Manager.selectTier2TargetFrom` | controller/internal/backup/tier2.go | `(stack, sourceDrive, fullSize, stateOnlySize) (*Tier2Target, error)` | tier-2 target choice with the source drive supplied EXPLICITLY | the seam the shares job reuses — NEVER fork the headroom math; `selectTier2Target` is now a thin wrapper over it |
|
||||
| `Manager.tier2ReconcileRoots` | controller/internal/backup/tier2.go | `(destBase, roots, legRels)` | staleness pruning with explicit dest roots | pure extraction from `tier2Reconcile` (which now calls it with `hdd`/`userdata`); reuse it rather than writing a second pruner |
|
||||
| `Manager.liveShareRootOK` / `scratchJoin` | controller/internal/backup/shares_restore.go | `(dst) bool` / `(scratch, abs) string` | THE place guard for shares restore + scratch path reconstruction | a snapshot is UNTRUSTED layout input: require a STRICT descendant of a live registered root, refuse `..` and the drive root itself. `scratchJoin` strips the volume name — plain `filepath.Join` splices a drive letter mid-path |
|
||||
| `infra.SambaContainerName` / `SambaPassdbVolume` / `SambaPassdbMount` | controller/internal/infra/samba.go | consts | single source of truth for the samba container identity | the compose renderer interpolates them; stacks/backup/monitor read them. The CONTAINER name (`felhom-samba`) is NOT the stack name (`samba`) — `EffectiveProtected` needs the container one |
|
||||
| `sambaWriteAtomic` | controller/internal/stacks/samba.go | `(path, data, mode) error` | samba smb.conf/compose writes | tmp+**fsync**+rename (the only one of these that fsyncs). Fourth atomic-write helper in the tree — see §6 |
|
||||
| `Loop.writeMarker` / `Recover` | controller/internal/quiesce/quiesce.go | `(m Marker)` / `()` | Quiesce crash-safety | Marker written BEFORE stopping stacks; Recover restarts stranded stacks at boot |
|
||||
| `quiesce.TieredBackend` + `Loop.resolveDueTiers` / `quiesceAndPollTiers` | controller/internal/quiesce/tiers.go, quiesce.go | `Tiers/DueFor/StartBackupFor/BackupStatusFor`; `resolveDueTiers(ctx) ([]dueTier,bool,error)` | THE R-82 multi-tier backup schedule — several whole-guest tiers (local daily + PBS weekly) reconciled into ONE quiesce window | **Both tiers due ⇒ ONE stop/start pair**, never two (two = two app outages for one night). Tiers run SEQUENTIALLY (vzdump holds a guest lock) and the app stays down until the LAST tier snapshots — resuming earlier loses app-consistency on the DR tier. Order is fast-first (agent advertises primary first) or downtime blows up. `ErrTiersUnsupported` (route 404) ⇒ pre-R-82 agent ⇒ degrade to the untargeted path and **STILL BACK UP** — never read it as "nothing due". |
|
||||
| `quiesce.failureBreaker` + `Loop.dropBackedOffTiers` / `noteTierFailure` / `noteTierSuccess` | controller/internal/quiesce/breaker.go, quiesce.go | `blocked/recordFailure/recordSuccess(target, now)`; `backoffFor(n) time.Duration` | **R-88** — a tier whose backups keep failing stops re-quiescing. Backoff 15m→30m→1h→2h→4h (cap), reset on success | **It gates the QUIESCE, not the backup** — the harm was never the failing backup, it was the app outage taken to attempt it, so backed-off tiers are dropped from the due set BEFORE any stack is stopped. **Per TARGET** — a broken offsite tier must never suppress a healthy local one (`TestBreaker_OneFailingTierDoesNotSuppressAHealthyOne`). **Never permanent** — the cap bounds the retry INTERVAL, it never stops retrying; a latched breaker is a silent backup outage, worse than the loop it replaces. **`TriggerNow` is never gated** (it already bypasses due-ness and the window gate), though a manual run still RECORDS its outcome. **`stillRunning` is NOT a failure** — a first full offsite snapshot legitimately runs for hours. State is **in-memory on purpose**: a restart forgets the backoff and re-attempts, which is the cheap direction to fail. Log the deferral ONCE when armed, never per tick. |
|
||||
| `quiesce.TierNotifier` + `Loop.SetTierNotifier` / `noteTierFailure` / `noteTierSuccess` | controller/internal/quiesce/breaker.go, quiesce.go | `BackupFailed(tier,msg,err)` / `BackupRecovered(tier,msg)`; `SetTierNotifier(n)` INIT-ONLY | **R-97a** — the whole-guest backup tier reports its outcome to the hub | A **seam, not an import** — quiesce keeps no dependency on `internal/notify` (same reason `windowStartFn` is injected). Wired by a setter because main.go builds the notifier AFTER the loop; `nil` = unprovisioned guest, not an error. **Edge-triggered:** failure fires only when the breaker ARMS (`n == 1`), never per retry — the cadence is 15m/30m/1h/2h/4h and an event per attempt is an inbox nobody reads. Recovery rides `recordSuccess`'s existing bool. **Event types are OPERATOR-ONLY** (`whole_guest_backup_failed`/`_recovered`, hub >= v0.78.0) — NOT `backup_failed`, which has a customerMessages entry AND sits in live `enabled_events`, so it would email the CUSTOMER about a backup they cannot act on. `WholeGuestBackupDetails.Tier` is load-bearing: the hub keys its per-tier cooldown on it. |
|
||||
| `quiesce.Loop.SuppressedStacks` + `markQuiesced` / `markUnquiesced` | controller/internal/quiesce/suppress.go | `() map[string]bool` (nil-safe on a nil *Loop) | **R-97b** — an app THIS controller stopped for a backup is not a fault | Consumed at the SINGLE derivation point `classifyRunStates` (which computes both the banner dead-list and the notifier Down-set — keep it one place). **Cycle-keyed, not state-based:** v0.164.0's `!= StateStopped` filter cannot see an app caught MID-RESTART (`starting`/`unhealthy`), which is how BookStack alarmed on 2026-07-27. The window (`quiesceAlarmGrace` = 180 s, derived from the deploy flow's 120 s health timeout and Mealie's 60 s start_period) **EXPIRES** — permanent suppression turns a loud false alarm into a silent real one. Open-ended while the cycle runs (a first offsite snapshot legitimately takes hours). |
|
||||
| `agentapi.BackupTiers` / `BackupDueFor` / `StartBackupFor` / `BackupStatusFor` | controller/internal/agentapi/backup_tiers.go | `(ctx[, target]) (…, error)` | The per-tier agent surface (agent >= v0.97.0) | `targetQuery("")` returns an EMPTY suffix so an untargeted call hits the pre-R-82 route byte-for-byte. `BackupTiers` maps a 404 to `ErrTiersUnsupported` — the documented ROUTE-PROBE capability signal, NOT a `featureProbes` row (the loop needs the tier LIST, not a yes/no). |
|
||||
|
||||
### Compose ops / stack lifecycle
|
||||
|
||||
| Symbol | File | Short signature | Use for | Gotchas |
|
||||
|---|---|---|---|---|
|
||||
| `Manager.DeployStack` | controller/internal/stacks/deploy.go | `(req DeployRequest) (string, error)` | Full deploy flow | Sets in-memory `Deployed` BEFORE compose up (slow-pull race), reverts on failure |
|
||||
| `Manager.RedeployFromEnv` | controller/internal/stacks/deploy.go | `(name, env map[string]string) error` | Re-up with changed env (migration flip, config edits) | `compose up -d`, never `restart` (restart won't pick up images/env) |
|
||||
| `Manager.PersistUnitRedeployConfig` (R-47, v0.153.0) | controller/internal/stacks/deploy.go | `(name, env map[string]string) error` | the PERSIST half of `RedeployFromEnv` — app.yaml + locked fields + in-memory flags, **starts nothing** | **TRAP: the restore paths must use THIS, never `RedeployFromEnv`.** RedeployFromEnv ends in a full `up -d`, which before the replay IS the H4 race. RedeployFromEnv is now literally this + the unchanged up-and-report tail |
|
||||
| `Manager.StartStackServices` (R-47, v0.153.0) | controller/internal/stacks/manager.go | `(name string, services []string) error` | scoped `compose up -d <svc>...` — the DB-only window a dump is replayed in | **REFUSES an empty list** (argument-less `up -d` is a FULL start — the one silent fall-through that would reintroduce the race). No `logPostStartStatus`: the app containers are absent on purpose. Never `RestartStack` here — it is a full up in disguise |
|
||||
| `appbackup.DBServiceNames` / `dbTypeForImage` (R-47, v0.153.0) | controller/internal/appbackup/dbservices.go | `(composePath string) ([]string, error)` | naming the compose SERVICE(s) holding a database, sorted | yaml.v3 `services:` MAP parse — **never a line scan** (immich's top-level `immich_ml_cache:` / `immich_postgres_data:` volume keys look exactly like services). `dbTypeForImage` is shared with `DiscoverDatabases`, which is what makes "a dump exists ⇒ a service can be named" hold. An error means CANNOT-TELL, never "no database" — callers refuse when a dump exists |
|
||||
| `Manager.StartStack/StopStack/RestartStack/UpdateStack` | controller/internal/stacks/manager.go | `(name string) error` | Lifecycle | Protected stacks refuse stop; all funnel through composeExec. **NOT writers of desired state (R-166)** — 14 call sites, only 2 are the customer; recording intent here would make a nightly backup indistinguishable from the customer pressing Stop. Use `SetDesiredState` at the intent point instead |
|
||||
| `Manager.SetDesiredState` / `DesiredStateOf` / `BackfillDesiredState` (R-166, v0.189.0) | controller/internal/stacks/desiredstate.go | `(name, desired string) error` / `(Stack) string` / `() int` | THE customer-intent record — `app.yaml` `desired_state`, tri-state `""`/`running`/`stopped` | **ONE OWNER: the customer's action.** Writers are the API action switch, `DeployStack`, `UpdateOptionalConfig`'s redeploy branch, and the `.fab` restore adapter — nothing else, ever. **`""` (absent) means UNKNOWN, never "running"**: every pre-v0.189.0 app.yaml reads absent, so treating it as running would start every deliberately-stopped app on upgrade. Write intent BEFORE the act and REFUSE the act if it fails (§8.2). Backfill is **running-only** — never infer `stopped` from zero containers, that inference IS the defect |
|
||||
| `Manager.DriveLive` (R-171, v0.190.0) | controller/internal/stacks/deploy.go | `(hddPath string) bool` | is an app's data drive a live mountpoint RIGHT NOW | Wraps the **same** `isMountPoint` seam the userdata belt uses (`manager.go`) — never write a second liveness check, the two would drift invisibly. The system/local path is legitimately not a mountpoint and returns true |
|
||||
| `bootrecon.StartGate` (R-171, v0.190.0) | controller/internal/bootrecon/bootrecon.go | `MayStart(stack) (bool, reason)` | THE one question the boot sweep asks before starting anything | **Fail-safe: cannot determine ⇒ return FALSE.** One seam for all three holders (absent drive · quiesce · an in-flight app-data operation) because they differ only in the reason string. Implemented in `main.go` (`bootDriveGate`) reusing `quiesce.SuppressedStacks()`, `AppStopGuard.HeldStacks()` and `Manager.DriveLive` — never re-derive any of them. Held apps go to `Result.HeldByDrive`, **never** `StillDown` (that is the dead-app alarm's bucket) |
|
||||
| the boot settle window (R-157 A, v0.190.0) | controller/cmd/controller/main.go | `bootReconcileSample` / `StableFor` / `Budget` | sample the fleet until it stops changing, then sweep ONCE | **settle + budget + one `DefaultRetryDelay` must stay under `deadAppBootGrace`** — pinned by `TestBootWindow_CommonCaseFitsInsideTheDeadAppGrace`, which is why the budget is 50 s and not 60 s. Sampling is READ-ONLY; sweeping per sample would never see a settled fleet (the sweep's own StartStack changes it). A late recovery is REPORTED (`recordLateRecovery`), never hidden by widening the grace |
|
||||
| `backup.AppStopGuard` (`Begin`/`End`/`Recover`) (R-166, v0.189.0) | controller/internal/backup/appstop_marker.go | `(opID, reason, stacks) error` / `()` / `() *AppStopRecovery` | THE crash marker for stop→work→start windows (volume dump, offbox reconstitute, `.fab` export) | Its **own** file (`appstop-state.json`), never quiesce's — one file, one writer. **A `defer` is NOT the mechanism** (Campaign 8 fault 10: SIGKILL runs no defer); the marker is. Written BEFORE the stop, cleared ONLY after a restart that succeeded; a FAILED restart deliberately KEEPS it. `Recover` RETURNS its outcome rather than notifying, because it must complete before the boot reconciler while the notifier does not exist yet |
|
||||
| `backup.ErrStartRefused` + `AppStopRecovery.Refused`/`Alarming()` (R-174, v0.191.0) | controller/internal/backup/appstop_marker.go | `errors.Is(err, ErrStartRefused)` / `() bool` | THE refusal-vs-failure split in the app-stop crash recovery | **A gated starter's refusal is NOT a restart failure.** `Recover`'s starter MUST be the gated `gatedAppStopStarter` (cmd/controller/main.go), never the raw `stacks.Manager` — that was the v0.189.0 defect, which started apps onto ABSENT drives at boot (R-171 one path over). A refusal goes to `Refused` (marker KEPT, silent), a real error to `Failed` (marker kept, ALARMS). Collapsing them routes a deliberate hold into `NotifyBackupFailed`, a customer-enabled type — the R-171 false alarm again. `main.go` must guard the notify with `Alarming()`, not `!= nil` |
|
||||
| `Manager.DeleteStack` / `RemoveStack` | controller/internal/stacks/delete.go | `(name, removeHDDData[, backupPaths])` | THE guarded removal paths | Orphan/protected/deploying/running checks + ProtectedHDDPaths filter before any RemoveAll |
|
||||
| `resolveContainerState` / `aggregateState` | controller/internal/stacks/manager.go | `(dockerState, dockerStatus)` / `([]ContainerInfo)` | State classification | `.State` says "running" even when unhealthy — `.Status` parse is the fix |
|
||||
| `Manager.logPostStartStatus` | controller/internal/stacks/manager.go | `(name, stackDir, env)` | Async post-start verification | compose up exits 0 on crash-loops; this is the detection. Goroutine + 3s, never blocks |
|
||||
| `Manager.EnsureBaseStack` | controller/internal/stacks/infra.go | `() error` | Traefik/cloudflared/FileBrowser infra convergence | Renders from `internal/infra` templates |
|
||||
| `appbackup.ClassifyBinds` / `ValidateBackupSpec` | controller/internal/appbackup/classify.go | `(spec, binds) ([]ClassifiedBind, bool)` / `(spec, binds) error` | Backup-classification (Task 2, referential coupling) — pure | Two-level default: explicit wins over `:ro`; unlisted writable→mandatory, unlisted `:ro`→excluded; nil spec→legacy/false. Validate REJECTS the WHOLE block on any defect (whole-block semantics). INERT — no tier consumes it yet |
|
||||
| `ParseComposeClassifiableBinds` | controller/internal/stacks/classify_binds.go | `(composePath) []appbackup.ComposeBind` | `${VAR}`-relative binds + `:ro` for classification | Do NOT use `ParseComposeHDDMounts`/`ExportDataMounts` as classifier input (§traps) — they resolve absolutes, drop `:ro`, or union the userdata ROOT. Short-syntax only |
|
||||
| `Metadata.EffectiveLifecycle` / `CanInstall` / `IsAbandoned` + `web.lifecycleBadge` / `web.visibleCatalogStacks` | controller/internal/stacks/metadata.go, controller/internal/web/metabadge.go, controller/internal/web/handlers.go | `meta.CanInstall() bool` | app lifecycle: `available` / `hidden` / `abandoned` (v0.158.0) | THE single interpretation of `.felhom.yml` `lifecycle:` — every surface must go through these, never compare the raw string. Listing drops `!Deployed && !Protected && !CanInstall()`; `api.deployStack` refuses server-side BEFORE any mutation (hiding a button is not a gate), `stacks.DeployStack` repeats it for non-API callers. **Unknown value fails OPEN** (→ available + one WARN) — opposite to the gate on purpose: a typo must never pull a working app out of every catalog. **NEVER let lifecycle reach orphan detection** (`getCatalogTemplateSlugs`) — a withdrawn template stays in the tree, or every deployed instance reads as `Elavult` and gets a Törlés button. Badges: `MetaBadge` + `meta_badge` partial, built generic for R-56 difficulty labels |
|
||||
| `Manager.ClassifiedBinds` + `StackDataProvider.GetStackClassifiedBinds` | controller/internal/stacks/metadata.go, appbackup/appdata.go | `(name) ([]appbackup.ClassifiedBind, bool)` | Per-stack classification through the REAL LoadMetadata validate path | The wired seam Task 3 consumes; LoadMetadata is the SINGLE validation choke point (bad block → nil + one ERROR → legacy) |
|
||||
| `backup.Manager.DumpAppVolumesSafe` | controller/internal/backup/backup.go | `(stackName) error` | Volume tar of a live app | Stops → dumps → restarts; surfaces BOTH errors (app may be left stopped). Check `GetDockerVolumes()!=0` + `IsProtectedStack` BEFORE calling — it stops the stack before its own volume check (see `runVolumeDumps`) |
|
||||
| `backup.Manager.ListRestorePoints` | controller/internal/backup/restore_points.go | `(stackName) ([]RestorePoint, bool)` | Restorable keep-side backups (the /api/backup/snapshots payload) | ONE point per app (the current unit); tier always 1 — never list Tier-2 (not restorable via /backup/restore) |
|
||||
| `backup.Manager.RestoreTier2Files` | controller/internal/backup/tier2_restore.go | `(stackName) (filesRestored int, err error)` | In-place ADDITIVE-ONLY class-C file restore from the recorded Tier-2 copy (`POST /backup/tier2/restore`) | Never overwrites/deletes live files; refusals (Hungarian) before any stop; source = recorded `DestinationPath`, never re-selected. **C9-F1 (v0.183.0): reads `hdd/` + `userdata/` ONLY — never `recovery-unit/`.** For 43 of 53 catalog apps that is a guaranteed no-op, so it now refuses with `ErrTier2NoRestorableData` BEFORE stopping the app. Ask `Tier2RestoreCoverage` first |
|
||||
| `backup.Manager.Tier2RestoreCoverage` | controller/internal/backup/tier2_restore.go | `(stackName) (Tier2Coverage{Legs, HasUnit}, error)` | Answers what a Tier-2 restore CAN and CANNOT return for an app, from the RECORDED copy on disk | **C9-F1.** `Legs` = subtrees the restore reads; `HasUnit` = the copy also holds DB dumps + volume tarballs it will NEVER read. Use it to refuse up front and to decide whether the success message must disclose uncovered data. Judged from the copy, not the catalog, so a retemplated app is judged by what it actually has |
|
||||
| `Manager.acquireRunning`/`releaseRunning`, `acquireMigrating` | controller/internal/backup/backup.go, controller/internal/stacks/migrate.go | `() error` | Single-flight for long ops | Copy this mutex-flag pattern for any new long-running manager op |
|
||||
|
||||
### Secrets hygiene
|
||||
|
||||
| Symbol | File | Short signature | Use for | Gotchas |
|
||||
|---|---|---|---|---|
|
||||
| `crypto.Encrypt/Decrypt/IsEncrypted/DecryptMap` | controller/internal/crypto/crypto.go | AES-256-GCM, `ENC:` prefix | app.yaml sensitive values | `Decrypt` errors on non-ENC input — use `DecryptMap` for whole env maps (passes through + warns) |
|
||||
| `crypto.LoadOrCreateKey` | controller/internal/crypto/crypto.go | `(path) ([]byte, error)` | The 32-byte key file (0600) | — |
|
||||
| `SaveAppConfig` / `LoadAppConfigDecrypted` | controller/internal/stacks/deploy.go | `(stackDir, cfg, encKey, sensitiveVars)` | app.yaml persistence | Encrypts only `SensitiveEnvVars(meta)`; never write app.yaml directly |
|
||||
| `generateValue` / `randomAlphanumeric` | controller/internal/stacks/deploy.go | `(spec "password:N\|hex:N\|base64key:N\|static:v")` | Auto-generated secrets | crypto/rand-backed; reuse the spec grammar |
|
||||
| `Manager.GenerateSecretForField` | controller/internal/stacks/deploy.go | `(stackName, envVar) (string, bool)` | Replacement value for a RESETTABLE secret from its catalog `generate` spec (O4 restore path via `backup.SetSecretGenerator`) | REFUSES `data_key` fields, spec-less and non-secret fields; never log the value |
|
||||
| `reconcileRestoreSecrets` | controller/internal/backup/restore_unit.go | `(nonSecretEnv, unitSecrets, guestSecrets, secretNames, dataKeyNames)` | Recovery-unit restore env merge | **Precedence: UNIT WINS over guest** (the unit's secrets match the data being restored; the guest's are merely newest). Pure — new sources arrive as ARGUMENTS. Fail-closed data-key gate lives here |
|
||||
| `stacks.PortableSecretEnvVars` | controller/internal/stacks/deploy.go | `(meta) []string` | **THE D5 secret boundary**: which secrets may travel on a customer drive | `type: secret` travels, `type: password` NEVER, minus the `nonPortableSecrets` code register. Withholding the password class is what licenses plaintext — do not relax one without the other |
|
||||
| `buildUnitAppYaml` / `readUnitEnv` | controller/internal/backup/{recovery_unit,restore_unit}.go | `(info) []byte` / `(path, portableNames)` | The ONE place the unit's app.yaml is written / split back | Split is driven by the MANIFEST's portable names, never guessed from key names; write 0600; empty `portableNames` = schema-1 unit ⇒ everything is plain config |
|
||||
| `EncryptFile` / `DecryptFile` / `IsEncryptedFAB` | controller/internal/appexport/crypto.go | password-based file crypto | .fab export bundles | scrypt-derived AES+HMAC keys |
|
||||
| `maskRepoURL` | controller/internal/sync/sync.go | `(url) string` | Logging git URLs | Strips embedded credentials |
|
||||
| `metrics.RedactLine` | controller/internal/metrics/redact.go | `(s string) string` | ANY log line shipped off-box (issue context, log tails) | Masks password/passwd/secret/token/api-key/authorization/bearer values + 64-hex; apply BEFORE the line leaves the box — controller-side redaction is authoritative |
|
||||
| `settingsRetrievalPasswordRevealHandler` | controller/internal/web/handlers.go | `POST /settings/retrieval-password/reveal` | **THE PATTERN for showing a secret in the UI** — an XHR that returns only the value | **Never template a secret into a page and hide it with CSS.** `display:none` / `hidden` / `type="password"` stop a browser DRAWING the value; the plaintext is still in the response body, so a `curl` of the page returns it, and it reaches caches, history and any screen-share of the source. R-249 shipped exactly that for two months and was found by it landing in a transcript. The page carries a **boolean** (`HasRetrievalPassword`); the value comes from a POST (CSRF-covered, uncacheable) and the reveal is **logged as an act**. `escrow_handlers.go` states the same rule for R. **Test on the RESPONSE BODY** — a test asserting what the customer *sees* cannot see this class at all. **Both R-254 sites are now FIXED the same way** — `POST /apps/<slug>/initial-credentials/reveal` (re-reads the container, never a cached copy) and `POST /stacks/<name>/auto-field/reveal` (authorised on the field being a `type: secret` auto-field of that stack). **Per-secret, never one generic reveal-any-named-secret endpoint.** The PRE-DEPLOY hidden input is deliberate and untouched — a form must carry what it submits (README §318). Enforced by `scripts/secret_in_markup_gate.py`, whose measured blind spot (a secret under a neutral page-data key) is in its docstring; runtime body-assertion covers 4 of 27 pages — R-255. |
|
||||
|
||||
### Storage registry + mount detection
|
||||
|
||||
| Symbol | File | Short signature | Use for | Gotchas |
|
||||
|---|---|---|---|---|
|
||||
| `Settings.AddStoragePath/RemoveStoragePath/RepointStoragePath` | controller/internal/settings/settings.go | registry CRUD | ALL drive registration | `AddStoragePath` dedupes (double-register is clean no-op); `AutoDiscoverStoragePaths` never re-adds a known-in-any-state path |
|
||||
| `Settings.SetDisconnected/ClearDisconnected/SetDecommissioned` | controller/internal/settings/settings.go | state flags + stopped-stacks memo | Drive lifecycle state | Records `stoppedStacks` so reconnect restarts exactly those |
|
||||
| `registerStoragePath` | controller/internal/web/storage_handlers.go | `(where, label, setDefault) error` | Post-enroll registration | The single funnel used by init/attach/manual-add |
|
||||
| `system.IsMountPoint` / `IsWritable` / `PathsOverlap` | controller/internal/system/mounts_linux.go | `(path) bool` | Mount checks | `_other.go` stubs return permissive values — Linux behavior is the real one |
|
||||
| `system.CheckBackupDestination` | controller/internal/system/mounts_linux.go | `(path) DestinationHealth` | Tier2/offbox target vetting | Detects same-physical-device (`SamePhysicalDevice`) |
|
||||
| `system.ProbeStoragePath` | controller/internal/system/mounts_linux.go | `(path) ProbeResult` | Disconnect detection | — |
|
||||
| `appexport.DiskFree` | controller/internal/appexport/estimate.go | `(path) int64` | Free bytes for space gates (df-based, 0 on any error) | Exported v0.128.0 for the browser-upload gate; test seam = `web.uploadDiskFree` package var |
|
||||
| `stacks.ExportDataMounts` | controller/internal/stacks/delete.go | `(composePath, hddPath) []string` | THE .fab-export mount discovery (v0.130.0 C6B-F1) | Unions `${HDD_PATH}` binds + the `${USERDATA_PATH}` ROOT (single `userdata` entry — basename must round-trip the import's `<HDD_PATH>/<subdir>` mapping; NEVER return per-bind userdata subpaths). Containment-deduped. Backup-side `stackAdapter` deliberately does NOT use it |
|
||||
| `Server.deployedAppsOnPath` | controller/internal/web/netstorage_handlers.go | `(base) []string` | Deployed stacks whose HDD_PATH is base or a subpath | The C6B-F2 share-removal guard; nil-safe on stackMgr |
|
||||
| `planDriveGates` / `Server.ReconcileDriveGates` | controller/internal/web/intermediary.go | pure plan + executor | Drive appear/disappear reactions | `planDriveGates` is PURE (unit-testable); loop at `driveGateLoop` |
|
||||
| `Server.runStorageInit` / `runStorageAttach` | controller/internal/web/storage_handlers.go | wizard pipelines | New-drive enroll / re-attach | Format goes through the agent's two-step confirm (below) |
|
||||
|
||||
| `Server.sharingResolvePath` / `sharingResolveStorageRoot` | controller/internal/web/sharing_handlers.go | `(raw) (string, error)` | THE guard for every customer-supplied SMB share path | resolvePath validates a share TARGET (refuses the drive root); resolveStorageRoot validates the new-folder PARENT (accepts exactly a registered live root). Refusals are UNIFORM (no filesystem oracle). Never add a second deny-list — `stacks.SharingDeniedRoots` derives from `ProtectedHDDPaths` |
|
||||
|
||||
### Agent local-API client (cross-repo edge)
|
||||
|
||||
| Symbol | File | Short signature | Use for | Gotchas |
|
||||
|---|---|---|---|---|
|
||||
| `agentapi.New` | controller/internal/agentapi/client.go | `(endpoint, token, fingerprintHex) (*Client, error)` | Pinned-TLS client to felhom-agent | Leaf-DER SHA-256 pin replaces chain verify; fails closed. Bounded idle pool (leak fix) |
|
||||
| `Server.agentClient` | controller/internal/web/agent_disk_handlers.go | `() (*agentapi.Client, error)` | THE memoized client accessor | Always use this, never a fresh `agentapi.New` per request (§3) |
|
||||
| `Server.ProbeAgentChannel` | controller/internal/web/agent_disk_handlers.go | `(ctx) (constructionErr bool, err error)` | Channel health probe | Probes via the PRODUCTION client on purpose (self-heals, mirrors UI) |
|
||||
| `Client.FormatDisk` | controller/internal/agentapi/client.go | `(ctx, device, fstype, confirmed, durableID)` | ONLY format/wipe entry | Sentinels: `ErrNeedsConfirmation` (user-data, resubmit confirmed+durableID) / `ErrFormatRefused` (system/backup — operator opsign only). Agent re-checks role server-side |
|
||||
| `Client.EjectDisk` / `Decommission` / `AssignDisk` / `GuestAttach` / `ListCandidates` | controller/internal/agentapi/client.go | disk lifecycle | Delegate ALL disk ops to agent | Controller holds no Proxmox creds — never shell out to disk tools in-guest |
|
||||
| `Client.AddNetStorage/ListNetStorage/RemoveNetStorage` | controller/internal/agentapi/client.go | NAS mounts (A1) | Network storage | Password passes through to agent's 0600 cred file; controller NEVER persists it |
|
||||
| `agentapi.StatusError` | controller/internal/agentapi/client.go | `{Path, Code}` typed non-2xx GET error | Distinguishing HTTP statuses from transport errors (`errors.As`) | NEVER string-match agent error text — the capability probe keys on `Code==404` |
|
||||
| `SupportCache.Supports` / `Client.Supports` | controller/internal/agentapi/features.go | `(ctx, prober, Feature) SupportState` | Agent-capability gate for COUPLED features (route probe, TTL 5m) | 404 ⇒ No; transport/5xx ⇒ Unknown (NEVER refuse on Unknown). New coupled feature = new `featureProbes` row + gate call at the entry point + `MinAgent:` in the CHANGELOG header (publish-train-rules.md). Web layer: `Server.netFeatures` through the `netAgent` seam |
|
||||
| `agentapi.DiskVerdictFor` / `DiskVerdict.Label` / `DegradedAttributes` / `UncorrectableSectors` / `DiskPrior` / `TemperatureFailC` | controller/internal/agentapi/diskverdict.go | `(*SmartSummary, DiskPrior) DiskVerdict` | THE shared disk-health verdict (card chip + hourly check) — v0.169.0, 14-row ladder v0.215.0 | Pure — no clock, no I/O; history arrives as `DiskPrior`. nil/UNKNOWN → `DiskVerdictUnknown` (Nincs adat, NEVER alarms, row 1 is first for that reason). **Never trust `smart_status.passed`**: attrs 187/197/198 carry `thresh: 0`, so it cannot fail on unreadable sectors. A zero `DiskPrior` is the fail-safe (first sighting can only reach Figyelmeztetés). **Four labels, no fifth** — predicted failure is „Hiba". Do NOT recompute the verdict inline anywhere, and do NOT re-literal 60 °C — use `TemperatureFailC` |
|
||||
| `Server.resolveBackupTargetState` / `backupTargetView` | controller/internal/web/backup_target_offer.go | `(ctx)` → state / `*BackupTargetView` (nil = render nothing) | The whole-system backup-target answer: healthy · degraded-never-configured · **TargetAbsent** (configured, drive gone) · unknown | Test seams `Server.tiersFn` + `Server.disksFn` (nil → the real client). **`degradedMessageFor` is the ONE place that decides customer copy** — add a state there, never in a template. `backupTargetView` returns **nil** for healthy AND unknown so a template typo cannot decorate a working box. R-112: this state had NO consumer for two releases; the render is server-side on `backups.html`, and the seam test drives `backupsHandler` and asserts rendered HTML |
|
||||
| `Server.cachedDisks` / `RunDiskHealthCheck` | controller/internal/web/disk_health.go | `(ctx)` | Card fetch (60s TTL) / the hourly degradation check | Card uses the 60s TTL cache (anti-smartctl-storm); the CHECK fetches FRESH (`fetchDisks`). Test seams: `Server.disksFn` (source) + `Server.diskNotifyFn(notify.DiskAlert)` (sink). State is PERSISTED (v0.215.0) — a restart no longer re-baselines |
|
||||
| `diskAlertDecision` / `diskAlertKindFor` / `Server.priorFor` / `Server.cardPriorFor` | controller/internal/web/disk_health_state.go | pure + `(key) agentapi.DiskPrior` | Whether an observation emits, and which message shape | Compares against the **last ALERTED** verdict, not the last observed — that is what collapses a flap to one alert. Re-alert needs doubling **AND** 24h (an AND). **`priorFor` is for the CHECK, `cardPriorFor` for the CARD** — they differ by one observation and mixing them makes the chip read one level more severe than the email |
|
||||
| `diskRecord` / `writeDiskState` / `Server.loadDiskStateLocked` | controller/internal/web/disk_health_state.go | `disk-health-state.json` in `cfg.Paths.DataDir` | Persisted per-disk observation + alert history | Atomic tmp+rename (the `selfupdate.SaveState` shape, copied not imported). Missing file = normal; corrupt = LOG and fall back to no-prior, **never fatal**. Written ONCE per check run. Keyed by `diskKey`. **One record per disk, NOT a sample series** — history is Phase 2/3 in `metrics.MetricsStore` |
|
||||
|
||||
### Notifications / hub sync
|
||||
|
||||
| Symbol | File | Short signature | Use for | Gotchas |
|
||||
|---|---|---|---|---|
|
||||
| `Notifier.PushEvent` | controller/internal/notify/notifier.go | `(eventType, severity, message, details)` | Hub events | Async goroutine, 3 attempts/3s backoff. NEW event types MUST be added to hub `allowedEventTypes` or POST /event 400s. **SEVERITY IS AN EXACT WIRE CONTRACT: `{"info","warning","error","critical"}` and nothing else.** The hub silently COERCES any other string to `"info"` (`hub/internal/api/handler.go`, the ingest severity switch) and `severityNotifies` (`hub/internal/notify/dispatcher.go`) emails only warning/error/critical — so a typo'd severity is stored and delivered to NOBODY, with no error anywhere. **`"warn"` is not a severity.** It shipped on `disk_health_degraded` (fixed v0.215.0, R-328) and is STILL live on `app_start_failed` (R-329) |
|
||||
| `notify.DiskAlert` / `DiskAlertKind` / `DiskAlertKind.Severity()` | controller/internal/notify/notifier.go | `NotifyDiskHealthDegraded(DiskAlert)` | The disk-health alert payload + its five Hungarian message shapes | The notifier owns customer copy — pass a `DiskAlert`, never a pre-formatted string, or Hungarian scatters across packages. `Severity()` is the ONE mapping kind→hub severity and is exported so any package can assert the contract instead of duplicating the literal |
|
||||
| `Notifier.Notify*` convenience methods | controller/internal/notify/notifier.go | typed wrappers (backup/DB/storage/channel/DR…) | Standard events | Add a typed wrapper rather than raw PushEvent calls |
|
||||
| `report.BuildReport` / `Pusher.Push` | controller/internal/report/builder.go + pusher.go | periodic hub report | Box→hub reporting | ACK carries `config_version` → `ConfigRefresher.Reconcile` |
|
||||
| `report.Trigger` (`NewTrigger`/`Fire`/`Run`) | controller/internal/report/trigger.go | `Fire()` after a hub-relevant user action | THE out-of-cycle report push (v0.139.0) — fire via `api.Router.reportPushNow` / `web.Server.reportTriggerNow`, both nil-safe | Coalesce-and-eventually-fire (trailing edge; quiet 2s, min spacing 15s). NEVER add retries (Pusher owns them); NEVER reuse the `internal/sync` REFUSE-debounce for hub pushes (a refused fire loses the update until the next cycle). Fire only AFTER a successful local commit |
|
||||
| `report.SetPendingLogTails` + `buildLogTailsSection` | controller/internal/report/logtail.go | ACK `log_tail_requests` → next report `log_tails` | THE pull-based ACK-flag pattern (hub asks, controller pushes next cycle) — copy for any new hub→box request | Consume-once drain at BuildReport; failed push re-arms from the hub's still-pending request; NEVER add a hub→controller push channel |
|
||||
| `metrics.FetchContainerLogTail` | controller/internal/metrics/logscanner.go | `(name, tailLines) (string, error)` | Raw per-container `docker logs --tail=N` | 15s timeout; caller caps/redacts (capTailLines) |
|
||||
| `ConfigRefresher.Reconcile` | controller/internal/report/config_refresh.go | `(ackVersion int)` | Pull-based config refresh | Re-pulls controller.yaml (re-merging local_api), then graceful self-restart; first-run = baseline, no restart |
|
||||
| `offsiteapply.SettleProvider` / `SettleFunc` / `Bridge.AwaitSettle` / `ReconcileWhenSettled` (R-71a, v0.162.0) | controller/internal/offsiteapply/offsiteapply.go + seams.go | `SettleState() (version, floor string, updateRunning, floorKnown bool)` | THE settle-gate: defers the offsite one-time-password consume past a managed day-0 floor-update (the F10 race). Wire the `SettleFunc` adapter over `updater.GetFloor()`/`IsUpdateRunning()` — **the updater's knowledge is the ONE floor source; never fetch the floor a second way**. Gate ONLY the bridge goroutine, and only when an updater exists (nil `Settle` = reconcile immediately). Bounds `settlePoll`/`settleFloorSubBound`/`settleOverallBound`; the floor is in-memory (report-ACK-derived, ~5–10 s), NOT persisted → unknown until the first ACK on any restart. Inject `Now`/`Sleep` in tests (no real sleeps). B′: at/above-floor GOes on the first poll, zero wait. Do NOT touch the consume/persist order or the 404 contract — ordering only |
|
||||
| `bootstrap.MaybeIngest` / `RefreshConfig` | controller/internal/bootstrap/bootstrap.go | bootstrap.json → controller.yaml | Day-0 + refresh | Overwrites controller.yaml, NEVER settings.json |
|
||||
| `api.GracefulSelfRestart` | controller/internal/api/selfrestart.go | `(logger)` | Controller self-restart | Detached exit; bootstrap unit re-runs the image |
|
||||
| `Settings.AddPendingEvent/DrainPendingEvents` | controller/internal/settings/settings.go | offline event queue | Events while hub unreachable | — |
|
||||
| `Manager.SetUnitNotify` + `UnitSpace` (R-158/R-167, v0.191.0) | controller/internal/backup/recovery_unit.go | `(func(stack string, err error, *UnitSpace))` | THE per-app Tier-1 recovery-unit capture failure alert — fires PER APP from `captureAllRecoveryUnits`, loop continues | **OPERATOR-TIER** (`recovery_unit_capture_failed`, in the hub's `operatorOnlyEvents`). **NEVER route it to `backup_failed`** — that type is in `DefaultEnabledEvents` and carries Hungarian copy, so it emails the CUSTOMER about a failure they cannot act on (D-c; R-158's own proposal said `backup_failed` and D-c overrides it). `UnitSpace` is **nil when the target filesystem is unreadable** and renders as *"unavailable"*, never as zeros — "0 GB free" and "we could not look" are opposite diagnoses. No controller-side cooldown: the hub owns it |
|
||||
| `Manager.beginRunSummary` / `noteFailure` / `noteAttempted` / `emitRunSummary` / `SetRunSummaryNotify` (R-182, v0.194.0) | controller/internal/backup/runsummary.go | `(kind, runID) func()` / `(app, leg, reason)` / `(RunSummary)` | **THE per-run operator digest.** One `backup_run_failures` event at the end of a run listing every failed app, its leg and its reason — emitted ONLY when something failed | **The RECORD and the NOTIFICATION are different things and must stay so.** The per-app `recovery_unit_capture_failed` event is the record (hub routes it *record-only*, stored + logged every time); this digest is the notification. Before R-182 one event was both, and did neither: nine arrived, two were mailed, seven vanished before `LogNotification`. **Lifetime is `admissionSet`'s exactly** — absent collector means "no run in flight", never a stale answer. **A refusal is noted ONCE, inside `admitApp` where the verdict is taken**, not at the three legs that consult it: R-181's one-verdict-covers-all-three contract makes per-leg noting produce "2 of 1 apps failed". **Deliberate skips (disconnected / decommissioned) must NEVER be noted** — they have their own alert and a nightly digest about an unplugged drive is an ignored digest. **A clean run emits NOTHING**; silence is safe only because the hub's deadline check (`monitor/deadline.go:396,417`) raises a missed backup from report freshness independently — if that is ever weakened this design loses its footing. **`run_id` is unique per real run** (so the hub's 1-h cooldown cannot collapse a manual run into the nightly one) and **deliberately EMPTY on the periodic refresh sweep**, which must stay under that cooldown or a polled status page becomes a mail flood |
|
||||
| `Manager.admitApp` / `beginAdmissionRun` / `decideAdmission` / `estimatedWriteBytes` (R-181, v0.193.0) | controller/internal/backup/admission.go | `(stackName) bool` / `() func()` | **THE reserve gate. Call it before ANY per-app backup write** — one verdict per app per run, covering the DB dump, the volume dump and the unit capture (all three write under one per-app root) | **Decided LAZILY at the app's first write, never once at run start** — app A's dump can put app B under the reserve, so a run-start verdict reads a disk that no longer exists. **Never re-decided between an app's own legs**: that is exactly the split R-181 closed (bulk written, capture refused). **Reset per run** via the closer `beginAdmissionRun` returns. **Must sit ahead of `DumpAppVolumesSafe`**, which stops the stack as its first act — a refusal decided inside it has already bounced the app. Fires **exactly one** `unitNotify` per refused app per run. Nil admission set (periodic status refresh) → decides fresh, which is still once per app per sweep. Wiring pinned by an **AST walk** in `TestAdmission_IsWiredIntoEveryProductionWriteLeg`, not `strings.Contains` |
|
||||
| `Manager.floorVerdict` + `FloorUsedPercent`/`FloorFreeGiB` / `ErrCaptureFloor` / `floorReason` (R-165 B2 v0.192.0, size term R-181 v0.193.0) | controller/internal/backup/recovery_unit.go | `(*UnitSpace, estGiB float64) (*UnitSpace, floorReason)` | The pure two-question predicate behind `admitApp`: is the filesystem already below the reserve (`floorHeadroom`), and would THIS app's write take it below (`floorSize`)? | **Headroom is about the FILESYSTEM, never a per-unit cap** — a size cap is R-163 rebuilt inside one volume; the size term bounds the *delta*, not the unit. **REFUSES, never deletes:** nothing here is generational (a unit is one fixed path per app, a DB dump one fixed name), so pruning could only destroy a DIFFERENT app's only local copy — **never repurpose `pruneStalePrimaryDirs`**, which removes ORPHANED dirs from an app that moved drives and has no notion of age. Two terms (97% / 1 GiB) in `fillwatch`'s shape, deliberately BEYOND its critical band (95% / 2 GiB) so the customer is always warned first — pinned by `TestFloorSitsBelowTheCriticalWarningBand`. **`estGiB == 0` degrades to headroom-only on purpose** — refusing an app with no history makes the FIRST backup the one that can never happen. A nil reading neither refuses nor warns (§8.4). Inject `unitSpaceFn` in tests rather than manufacturing occupancy on a real disk |
|
||||
| `fillwatch.Watcher` (`New`/`SetNotify`/`Check`) (R-167, v0.191.0) | controller/internal/fillwatch/fillwatch.go | `(statePath, logger, targetsFn, usageFn)` → `Check() error` | THE customer fill warning — warns BEFORE a filesystem fills, per FILESYSTEM (never per app: one full disk holding ten apps would fire ten times) | Emits the **pre-existing** `disk_warning`/`disk_critical` pair, which was allowlisted + copy'd + default-enabled with **no producer in any repo** until now — do NOT mint a new type beside it. **Two threshold terms, whichever trips first** (85% / 5 GiB; critical 95% / 2 GiB) because a percentage alone lies at both ends of this fleet's size range. **Edge-triggered on ESCALATION ONLY**, state persisted; de-escalation is silent and re-arms. Hysteresis dead zone between clear (75% / 7 GiB) and warn — pinned by `TestThresholdsKeepTheirHysteresisGap`. **A nil usage read is NEVER a warning** (§8.4). The hub has **no `customerMessages` entry** for either type on purpose — an entry would override the dynamic message and discard the drive label + free space |
|
||||
|
||||
### Scheduler / time / UI
|
||||
|
||||
| Symbol | File | Short signature | Use for | Gotchas |
|
||||
|---|---|---|---|---|
|
||||
| `Scheduler.Every` / `Daily` | controller/internal/scheduler/scheduler.go | `(name, interval/"HH:MM", fn)` | ALL background jobs | Daily is Europe/Budapest, DST-safe (`nextDailyRun` avoids Add(24h)); register in main.go block (§5) |
|
||||
| `Scheduler.UpdateDaily` | controller/internal/scheduler/scheduler.go | `(name, "HH:MM") bool` | Retime a daily job at runtime (no restart) | Per-job buffered `resched` chan + select case in `runDailyJob`; false (WARN) on invalid time / unknown-or-non-daily name; read `Schedule` under the mutex in the loop |
|
||||
| `backupwindow.*` (LegTimes / GateWindow / EffectiveWindow / ParseHHMM / FmtHHMM / Valid) | controller/internal/backupwindow/backupwindow.go | pure `string`↔`int` | Backup-window arithmetic (v0.168.0) | Offsets (W+60m/W+105m, gate W+2h..W+6h) are CONSTANTS — derived, never stored; wrap-safe modulo 1440; `EffectiveWindow(settings, yaml)` = settings>yaml>"02:30" |
|
||||
| `getBudapestLocation` | controller/internal/scheduler/scheduler.go | `() *time.Location` | Local-time math | web has its own `getTimezone` (§6); quiesce has its own `budapestLocation` (window gate) — 3rd copy, see §6 |
|
||||
| `Server.templateFuncMap` | controller/internal/web/funcmap.go | template.FuncMap | ALL template functions | `stateColor` outputs v2 suffixes `run/progress/warn/neutral/off`; stopped = NEUTRAL not red (operator-approved); `stateLabel` copy is frozen byte-identical (unit-tested) |
|
||||
| `timeAgoStr` | controller/internal/web/funcmap.go | `(s RFC3339 string) string` | Ago-format for STRING timestamps | Exists because `timeAgo(time.Time)` 500'd on strings (v0.93 bug) |
|
||||
| `Server.baseData` / `executeTemplate` | controller/internal/web/handlers.go + server.go | page-data plumbing | New pages | baseData injects nav/alerts/version; templates must pass `controller/scripts/template_id_gate.py` + `controller/scripts/emoji_gate.py` |
|
||||
| `Server.RequireAuth` / `CsrfProtect` / `csrfField` | controller/internal/web/auth.go + csrf.go | middleware | Any new authed route/form | csrfField emits the hidden input; setup wizard has its OWN csrf (§6) |
|
||||
| `LogBuffer` + `Lines(maxBytes)` | controller/internal/web/logbuffer.go | ring buffer io.Writer | In-memory log capture for the debug UI + the report `controller_log_tail` source | v0.116.0: ALWAYS constructed (any logging.level) — the logger is `MultiWriter(LevelFilterWriter(stdout, level), ring)`; `Lines` drops OLDEST to honor the byte budget |
|
||||
| `logx.Debugf/Infof/Warnf/Errorf` | controller/internal/logx/logx.go | `(l *log.Logger, format, args…)` | ALL NEW leveled log lines (the v0.116.0 sweep standard) | routing is the WRITER's job — Debugf always reaches the ring, stdout filters; nil logger = no-op; caller-attributed (Output calldepth 3) |
|
||||
| `web.LevelFilterWriter` | controller/internal/web/levelfilter.go | `NewLevelFilterWriter(w, minLevel)` | stdout leveling under the always-on ring | untagged lines parse INFO; always reports full length written |
|
||||
| `monitor.RunHealthCheck` / `EffectiveProtected` | controller/internal/monitor/healthcheck.go | system health report | Health + protected-container list | — |
|
||||
| `util.TruncateStr` | controller/internal/util/strings.go | `(s, maxLen) string` | Rune-safe truncation | The intended shared helper; stacks still uses its byte-based twin (§6) |
|
||||
|
||||
## 2. Canonical patterns (copy structure from THE named file)
|
||||
|
||||
| Pattern | Canonical file | Key traits |
|
||||
|---|---|---|
|
||||
| Agent-proxy web handler | controller/internal/web/agent_disk_handlers.go | memoized `s.agentClient()` → typed client call → `writeDiskJSON` envelope, Hungarian error strings, 502/503 mapping |
|
||||
| Two-step confirmed destructive op | controller/internal/web/storage_handlers.go `handleStorageWipe` | server-side type-to-confirm + probe(unconfirmed) → sentinel error → resubmit bound to agent durable-id; agent re-checks role regardless |
|
||||
| Crash-safe long job (journal + recover) | controller/internal/stacks/migrate.go | state machine + `writeJournal` per transition + `RecoverMigration` at startup + single-flight acquire/release + done-hook |
|
||||
| Quiesce/marker loop | controller/internal/quiesce/quiesce.go | marker BEFORE side effects, guaranteed undo (defer + max bound), `Recover()` once at startup, `TriggerNow` 409 single-flight |
|
||||
| Settings mutator | controller/internal/settings/settings.go (any Set*/Add*) | Lock → mutate → `s.save()`; getters return copies; never expose internal slices |
|
||||
| Channel-health checker w/ born-down alerting | controller/internal/channelhealth/checker.go | classify → debounce N≥2 → `alerted` flag re-armed on reason change (F2) |
|
||||
| Platform split | controller/internal/system/mounts_linux.go + mounts_other.go | `_linux.go`/`_other.go` twins; other = permissive no-op stubs for dev on Windows |
|
||||
| Debounced trigger + status (REFUSE-style — a too-soon fire is refused/lost) | controller/internal/sync/sync.go | `TriggerSync` 30s debounce, `Status()` snapshot struct, post-sync hook fan-out |
|
||||
| Coalescing trigger (trailing edge — a burst collapses but the LAST state always fires) | controller/internal/report/trigger.go | buffered-1 chan + non-blocking `Fire()` + single worker (quiet window → drain → min-interval → fire once); shape from hub `wgsync/reconciler.go` |
|
||||
| Detached job + status poll (single-flight, phase strings) | controller/internal/web/storage_init_job.go | acquire/release/set/**deep-copied** snapshot; phases mapped to Hungarian in the template; 1–3 s poll; terminal state **PROBED, not inferred**. Clones: `netstorage_job.go`, `samba_ensure_job.go` (v0.147.0). **Five of these now exist and agree on nothing — R-45 will unify them; prefer extending an existing one over a sixth** |
|
||||
| Streaming subprocess progress | controller/internal/backup/offbox_progress.go | `offboxStreamRunner` seam (stdout scanned line-by-line, stderr buffered, output tail-bounded) + a PURE line parser + a mutex-guarded published snapshot. Traps it encodes: a source reporting nothing is **normal** (restic sends 0 bytes for a whole incremental run) and the progress source may only update on unit completion — degrade bytes → files → current item + elapsed, never fake a percentage |
|
||||
| Post-start async verification | controller/internal/stacks/manager.go `logPostStartStatus` | goroutine + sleep, INFO log, never blocks/fails the operation |
|
||||
| Startup wiring order | controller/cmd/controller/main.go | init-only setters (`SetStackProvider` M2 contract: exactly once, before scheduler/HTTP), scheduler registration block |
|
||||
|
||||
## 3. Dangerous lookalikes — do NOT reuse
|
||||
|
||||
| Trap | Why it bites | Use instead |
|
||||
|---|---|---|
|
||||
| `rsyncMirror` (controller/internal/backup/tier2.go) | `rsync -a --delete` — DESTROYS anything extra at dst; correct only for tier-2 mirror dirs (backup DIRECTION). In the tier2→live restore direction it would erase every live file created since the last copy | `rsyncCopy` + `rsyncVerify` (controller/internal/stacks/migrate.go) for any move/copy; `rsyncRestoreMissing` (controller/internal/backup/tier2_restore.go, `-a --ignore-existing`) for the additive-only restore direction |
|
||||
| raw `os.RemoveAll` on drive/HDD paths | Bypasses the protected-set; wipes appdata/backups/media | `Manager.DeleteStack`/`RemoveStack` (controller/internal/stacks/delete.go) — gated by `ProtectedHDDPaths` + orphan/protected/running checks |
|
||||
| fresh `agentapi.New` per request | Idle-conn leak → EADDRNOTAVAIL, port exhaustion (live incident, fixed ctrl v0.74.0) | `Server.agentClient()` memoized accessor |
|
||||
| `timeAgo` on an RFC3339 string field | Template 500 (OffboxTarget.LastRun bug, fixed v0.96.0) | `timeAgoStr` |
|
||||
| `backup.Manager.DumpAppVolumes` on a running DB app | Inconsistent tar of live DB volume | `DumpAppVolumesSafe` (stop → dump → restart, both errors surfaced) |
|
||||
| `stacks.Manager.execCommand` / `composeExecCustomEnv` for NEW long-running calls | No context/timeout — a hung docker CLI blocks forever | `exec.CommandContext` + explicit timeout (copy `rsyncCopy` or appexport `composeExecEnv`) |
|
||||
| `config.LoadPermissive` | Skips validation — setup-mode only (customer.id/domain may be unset) | `config.Load` everywhere else |
|
||||
| `ExportDataMounts` / `ParseComposeHDDMounts` as **backup-classification** input | `ExportDataMounts` unions the `${USERDATA_PATH}` ROOT (export-capture logic, not per-bind); `ParseComposeHDDMounts` resolves absolutes AND drops the `:ro` flag — classification needs `${VAR}`-relative paths + read-only awareness | `ParseComposeClassifiableBinds` (controller/internal/stacks/classify_binds.go) |
|
||||
| `docker compose restart` (any wrapper) | Does not pick up new images or env | `RedeployFromEnv` / composeExec `up -d` |
|
||||
|
||||
## 4. Seams & interfaces (testing + cross-repo)
|
||||
|
||||
| Interface | Defined in | Implemented by | Fakes/tests at |
|
||||
|---|---|---|---|
|
||||
| `diskAgent` | controller/internal/web/storage_handlers.go | `*agentapi.Client` | `mockAgent` in controller/internal/web/storage_handlers_test.go |
|
||||
| `netAgent` + `Server.netAgentFn/netProbeFn/netListFn` | controller/internal/web/netstorage_job.go (+ server.go fields) | `*agentapi.Client` / `runNetProbe` (linux re-exec) / `agent.ListNetStorage` | `fakeNetAgent` + fn injections in controller/internal/web/netstorage_job_test.go — the NAS add orchestration never shells/TLS-dials in tests |
|
||||
| `Server.agentLogsFn` (func seam) | controller/internal/web/server.go | nil → `agentClient().DebugLogs` (agent GET /debug/logs) | injected in controller/internal/web/observability_test.go (incl. the pre-0.83 typed-404 notice path) |
|
||||
| `escrowAgent` + `Server.escrowAgentFn/escrowStageFn/escrowStaleFn` | controller/internal/web/escrow_handlers.go (+ server.go fields) | `*agentapi.Client` / `PushOffboxPasswordForEscrow` / `report.EscrowAutoConfirmer.StaleBlob` (SetEscrowStale) | `fakeEscrowAgent` + fn injections in escrow_wizard_test.go — call-ORDER assertions (stage BEFORE trigger) + agent-never-called gates. The claim leg is the ONLY surface R crosses: no-store, never logged, never templated |
|
||||
| `offboxCeremonyWaitState` + `escrowCeremonyGraceWindow` | controller/internal/web/handlers.go | pure pick: (awaiting, timedOut) from `OffboxTarget.{EscrowState,CeremonyCompletedAt}` — the v0.138.0 "megerősítésre vár" card. Stamp SET on claim (escrow_handlers.go), CLEARED on the flip (main.go Flip + offbox_handlers.go manual confirm) | escrow_wait_state_test.go truth table (escrowed/unstamped/unparseable → plain CTA; boundary via `>=`) |
|
||||
| `Manager.sambaUpFn` / `sambaPasswdFn` / `sambaRunFn` / `sambaAddrFn` (func seams) | controller/internal/stacks/manager.go (fields) + samba.go | nil → `composeUp` / `docker exec smbpasswd` (STDIN) / `containerRunning("felhom-samba")` / `docker exec felhom-samba ip -4 -o addr show eth0` | injected in controller/internal/stacks/samba_test.go — the idempotency test asserts the up-seam is called **zero** times when config is unchanged; the passwd seam means no unit test ever handles a real secret or touches docker. **`sambaRunFn` has an EXPORTED setter (`SetSambaRunProbe`)** — internal/web's status-contract tests need a live-container world from another package. `sambaAddrFn` backs `SambaLANAddress()` (v0.151.0); its parse is separately pinned in samba_lanaddr_test.go and it returns "" on any failure — the page omits a line rather than printing a wrong address |
|
||||
| `Manager.SambaLANAddress()` | controller/internal/stacks/samba.go | `() string` — the guest's LAN IPv4 for the Megosztás connect card (v0.151.0, S-2) | Read from the SAMBA container's netns (`network_mode: host`), never `net.InterfaceAddrs()` — the controller is on a docker BRIDGE and would answer 172.x (the same trap `setup.DetectLocalIPs` needs `HOST_IP` for). **NEVER cache/persist it** — the guest holds it by DHCP (S-5); callers re-derive per render. `""` = omit the line |
|
||||
| `Server.sambaAddrFn` (func seam) | controller/internal/web/server.go (field) + sharing_handlers.go `sambaLANAddress()` | nil → `stackMgr.SambaLANAddress()` | The web-side half of the connect card. Tests inject a COUNTED fn — the fresh-per-render assertion is what stops anyone memoizing a DHCP lease |
|
||||
| `Manager.guestNetExecFn` (func seam) + `GuestGateway()` / `GuestNetSnapshot()` | controller/internal/stacks/manager.go (field) + guestnet.go | nil → `docker exec felhom-samba <args>` — ONE seam for all R-66 guest-netns reads (route/link/addr/resolv.conf); tests script canned outputs per argv | guestnet_test.go. **The netns door rule:** the controller's OWN netns is the docker bridge, so any in-process read (`net.Interfaces`, `/proc/net/route`, its own `/etc/resolv.conf` = 127.0.0.11) is the S-2 wrong answer — guest-net reads MUST go through the samba (`network_mode: host`) exec door. Megosztás off ⇒ door closed ⇒ "" / per-item error strings; NEVER substitute an in-process value. Same S-5 law as SambaLANAddress: live per render, never cached/persisted. Parsers (`parseDefaultRoute`, `parseGuestInterfaces`, `parseResolvConf`) are pure + separately pinned |
|
||||
| `buildFileBrowserPaths` + `fbPathDeps` (R-67, v0.160.0) | controller/internal/web/handlers.go | pure assembly of one FileBrowser sync pass: (mount lines, config source paths) from the registry, with per-kind gates | filebrowser_network_test.go. **Two storage classes, two DIFFERENT gates:** drives keep the drive-absent gate + userdata scoping + skeleton (byte-identical to pre-R-67 — tested); network shares bind the share ROOT `:rslave` with the STUB gate instead (`classifyFSPath`; stub ⇒ excluded from mounts AND sources — an exposed stub swallows uploads the real mount later shadows; idle autofs / unknown ⇒ include, fail open). NEVER call `EnsureUserdataSkeleton` toward a network path (red-proven); never force-wake an idle trigger in the sync (doctrine) |
|
||||
| `Settings.RefuseAsAppNamespace` (R-108, v0.187.0) | controller/internal/settings/settings.go | `(path) (refuse bool, hungarianReason string)` — may an app's DATA NAMESPACE live here? | **THE single predicate for every placement surface** (deploy POST `api/router.go`, per-app migrate list + `handleStorageMigrateApp`, `handleStorageDecommission` mode=migrate TARGET). **Network storage is refused** because an app's namespace root IS its backup root (`namespaceRoot` returns a non-system drive path as-is → `<HDD_PATH>/backups/primary/<stack>/`), and on a share that lands inside FileBrowser's share-ROOT `download:true` bind — which CANNOT be narrowed (R-67 `:rslave` = automount wake; and apps on a share store at `<share>/<app>`, so there is no `userdata/` to scope to and creating one would write Felhom convention onto a customer's NAS). **DISTINCT from `refuseNetworkLifecycle`** — that asks "may a DRIVE lifecycle op run on this path" and is applied to the op's SUBJECT; this asks "may an app live here" and is applied to a placement TARGET. Migrate needs BOTH. **FAILS CLOSED:** `/mnt/felhom-drives` holds both kinds, so a path prefix cannot classify — `Kind` exists only on a REGISTERED path, therefore an unregistered path under that root is un-classifiable and REFUSES. Empty path = SSD-resident = allowed; nil receiver refuses. network_app_namespace_test.go, 4 red-proofs |
|
||||
| `Server.guestGatewayFn` / `guestNetFn` (func seams) | controller/internal/web/server.go (fields) + sharing_handlers.go accessors | nil → `stackMgr.GuestGateway` / `stackMgr.GuestNetSnapshot` | network_card_test.go — the counted-fn freshness test (2 renders ⇒ 2 resolves) is what stops anyone memoizing a DHCP lease; the Hálózati név row is gated on `smb.Enabled` (red-proven: gate dropped ⇒ \\FELHOM rendered while samba is down) |
|
||||
| `sambaEnsureState.consumeIfRunning()` | controller/internal/web/samba_ensure_job.go | serve-once `snapshot()` for terminal `running` only | `/sharing/status` carries a job EDGE (`phase`) and a service LEVEL (`running`) in one envelope — never let a level reach the phase channel, and never re-serve a consumed edge: the client answers `phase=="running"` with `location.reload()`, so both mistakes produce an infinite page reload (S-1/S-4, DIAG-sharing-2026-07-20.md). `failed`/`needs_password`/in-flight are NOT consumed |
|
||||
| `infra.SambaHostInterface` | controller/internal/infra/samba.go | the guest LAN nic name (`eth0`) | Single source for smb.conf's `interfaces =`, the container's `FELHOM_IFACE`, and the LAN-address read — if they name different nics, the service and the address the page prints drift apart |
|
||||
| `Manager.sambaImgFn` (func seam) | controller/internal/stacks/manager.go (field) + samba.go | nil → `docker image inspect <infra.SambaImage>` | drives the 4b card's pulling-vs-starting decision, which MUST be taken before `compose up` (afterwards the image is always present) |
|
||||
| `Manager.offboxStreamRunner` + `SetOffboxStreamRunner` | controller/internal/backup/offbox_progress.go | nil → `defaultOffboxStreamRunner` (real `restic`, stdout scanned live) | streaming sibling of `offboxRunner`; fakes emit canned `--json` status lines in offbox_progress_test.go, so the whole progress path runs with no restic, network or repo |
|
||||
| `Manager.offsitePreDumpFn` + `SetOffsitePreDumpFn` (R-44, v0.148.0) | controller/internal/backup/offbox_reconstitute.go (seam) + offbox.go (call site) | nil → `runDBDumpsInternal` under the SAME running flag | THE dumps-before-capture ordering seam. Extracted so the order is observable without Docker/restic — an ordering guarantee no test can see is one refactor from silently reverting to the DIAG-immich-restore-2026-07-19 behaviour. Red-proof: moving the capture first yields `[capture dump]` |
|
||||
| `Manager.offboxFullPlaceCopier` + `SetOffboxFullPlaceCopier` (R-43) | controller/internal/backup/offbox_reconstitute.go | nil → `rsyncRestoreOverwrite` (`-a --itemize-changes`; **no** `--ignore-existing`, **no** `--delete`) | **TRAP: do NOT reuse `offboxPlaceCopier` here.** The two copiers have OPPOSITE semantics for an existing file — `--ignore-existing` is exactly what a full restore must not do, and conflating them is how a missing-only merge came to be labelled a restore. Never `rsyncMirror` (`--delete`) in any restore direction |
|
||||
| `Manager.safetyDumpFn` + `SetSafetyDumpFn` (R-43) | controller/internal/backup/offbox_reconstitute.go | nil → `DumpOne` | the pre-restore undo. Invariant: the `pre-restore-`-prefixed dump must be verified ON DISK before anything is stopped/overwritten/replayed; failure ⇒ refuse with zero changes. Red-proof requires removing BOTH guards (the `err != nil` return and the `os.Stat`) — removing one leaves the other holding |
|
||||
| `reimportDBDumpsFrom(ctx, stack, dumpDir)` | controller/internal/backup/restore_db.go | explicit-dir sibling of `reimportDBDumps` (which passes `AppDBDumpPath`) | offsite reconstitution replays from the SCRATCH unit: the live unit is deliberately never overwritten, so replaying from it would replay the current DB over itself and restore nothing |
|
||||
| The DB-only replay window (R-47, v0.153.0) | controller/internal/backup/{offbox_reconstitute,restore_unit}.go | both restore paths: stop → place/volumes → `StartStackServices(dbServices)` → replay → `StartStack` (full) | **THE ordering invariant.** Replaying while the whole stack is up lets the app's own schema management race the dump — measured at 2 s on 2026-07-19 (H4), replay aborted `already exists`. Fail-closed: a dump with NO identifiable DB service refuses BEFORE the first mutation. Every exit from the window (replay error, DB-only start error) MUST still do a best-effort full start, or a failed restore becomes an outage. `hasReplayableDump` excludes `pre-restore-` safety dumps — counting them would arm the window for an app with nothing to replay |
|
||||
| `Manager.OffsiteScratchPair` / `OffsitePairInfo` | controller/internal/backup/offbox_reconstitute.go | reads the restored scratch unit's manifest (`offsite_run_id` / `dumps_at`) + the R-44 sniff | the confirm-dialog honesty surface. All warn-level: a pre-v0.148 (unstamped) pair and an empty-looking dump are SURFACED, never blocked — a false positive that refused a legitimate restore would be worse than the skew |
|
||||
| `appbackup.DumpValidation.LooksEmpty` (R-44 sniff) | controller/internal/appbackup/dbdump.go | computed in ValidateDump's existing single pass; `userTableNames` is EXACT-match | size and table count are both useless as emptiness heuristics (the 2026-07-19 dump: 52MB, 60+ tables, zero users — all geodata). **TRAP: never widen to a substring match on "user"** — it would flag `user_metadata` / `album_user` / `user_audit` on every healthy single-user box. A row wider than the read buffer still counts as a row |
|
||||
| `Manager.execFn` (func seam) + `restartPolicyLookup` / `inspectRestartPolicyFn` (R-51, v0.156.0) | controller/internal/stacks/manager.go | nil → real `exec.Command` / `docker inspect -f {{.HostConfig.RestartPolicy.Name}}` | `scriptedDocker` in controller/internal/stacks/degraded_test.go drives the WHOLE production path (docker ps → aggregateState → docker inspect) — an aggregateState-only test proves the function, not the caller. Policy answers are cached per container+state and pruned to the live `docker ps` set; a FAILED inspect is deliberately never cached (a hiccup must not pin a container to "unknown") and reads as SUPERVISED, i.e. fail-closed — the opposite of `IsDownState`'s fail-open, because there the state is ambiguous while here a member is known dead |
|
||||
| `bootrecon.StackProvider` (R-52, v0.156.0) | controller/internal/bootrecon/bootrecon.go | `*stacks.Manager` (GetStacks/StartStack/RefreshStatus) | `fakeStacks` counts StartStack per app; the load-bearing assertion is the NEGATIVE — a zero-container stack (a UI Stop = `compose down` = containers removed) must record **0** starts, while a boot orphan (containers present, Exited) records exactly 1. `Reconciler.sleep` is injected so the 30 s gap costs nothing |
|
||||
| `bootReconcileFn` + `runBootReconcile` (package-main seam, v0.156.0) | controller/cmd/controller/main.go | `bootrecon.New(mgr, logger).Run` | controller/cmd/controller/bootrecon_wiring_test.go. **The wiring itself is asserted by an AST walk** over `func main()`, not a `strings.Contains` — the substring version passed its own red-proof because a commented-out call still contains the string. Comments are not callers |
|
||||
| `classifyRunStates` (pure fix-3 derivation, v0.164.0) | controller/cmd/controller/main.go | `([]stacks.Stack, quiesced, failedRestart map[string]bool, now time.Time)` → `(dead []web.DeadApp, states []notify.AppRunState)` | classify_runstates_test.go. **THE single fix-3 rule: down = `(IsDownState(st.State) || st.CrashLooping(now)) && !userStopped && !quiesced`.** C9-F2 (v0.183.0) added the crash-loop term: `restarting` is NOT in `IsDownState` and must not be — adding it alarms on every deploy and update fleet-wide — so a SUSTAINED restarting run (`stacks.crashLoopAfter` = 5 m, above the 120 s deploy timeout, Mealie's 60 s start_period AND R-97b's 180 s grace) becomes down instead. `now` is injected so the threshold is a testable contract. A deliberate UI stop (`compose down` → zero containers → StateStopped, I1) must not alarm — banner OR email — while faults (Exited/Degraded) alarm byte-identically; I2 (P2 census: all catalog services `unless-stopped`) is why a crash never rests at stopped. **Do NOT touch `IsDownState`** (other callers rely on stopped=down) and do NOT filter in `buildDeadAppAlerts`/`NotifyAppStartFailures` — one derivation point. If I1 or I2 changes, revisit the suppression |
|
||||
| `report.SetPendingControllerLog` / `SetControllerLogSource` | controller/internal/report/selftail.go | ACK-armed consume-once self-log pull (the logtail.go shape) | selftail_test.go; source = `logBuffer.Lines`, wired once in main.go |
|
||||
| `util.ParseVersion` / `util.Version.Compare` | controller/internal/util/version.go | THE one semver comparator (house rule: never a second) — selfupdate aliases it; agentapi's MinAgent comparison uses it | rejects pre-release/dev/latest (callers fall back, never trust); numeric compare (0.100 > 0.81) |
|
||||
| `agentapi.AgentVersionReporter` + `featureMinAgent` | controller/internal/agentapi/features.go | version-first Supports (v0.82.0 header channel); probe = fallback for header-less agents | a coupled feature adds BOTH a featureProbes row AND a featureMinAgent row; v0.116.0: `SupportsWithSource` also reports HOW the verdict was reached (version/probe-cache/probe) for the gate log line |
|
||||
| `netProbeReadBack` (package var) | controller/internal/web/netprobe.go | `os.ReadFile` | overridden in TestNetProbeChild (nonce-tamper + cleanup-fail rows); package var because the child is a RE-EXEC'd process in production |
|
||||
| `system.ClassifyPathFS(Timeout)` + `netProbeFSClass` / `Server.classifyFSPath` / `Router.classifyFSPath` | controller/internal/system/fsclass*.go (+ web/netprobe.go, web/server.go, api/router.go seams) | statfs f_type → network/autofs/stub/unknown in THIS namespace (RCA fix 2) | idle autofs = HEALTHY, never force-mount; unknown = fail OPEN; seams injected in netprobe_stub_test.go / networkstub_test.go / deploygate_test.go |
|
||||
| `quiesce.Backend` / `quiesce.Stacks` | controller/internal/quiesce/quiesce.go | adapter over `*agentapi.Client` / `*stacks.Manager` | `fakeBackend`/`fakeStacks` in controller/internal/quiesce/quiesce_test.go |
|
||||
| `channelhealth.Probe` (func) + `Sink` | controller/internal/channelhealth/checker.go | `Server.ProbeAgentChannel` / notifier adapter | `fakeSink` in controller/internal/channelhealth/checker_test.go |
|
||||
| `appbackup.StackDataProvider` | controller/internal/appbackup/appdata.go | `*stacks.Manager` (via `backup.SetStackProvider`) | `fakeRecoveryProvider` in controller/internal/backup/recovery_unit_test.go |
|
||||
| `appexport.ExportStackProvider` | controller/internal/appexport/provider.go | `*stacks.Manager` | exercised in appexport tests |
|
||||
| `selfupdate.AgentSwapper` | controller/internal/selfupdate/updater.go | `*agentapi.Client` (SwapController/SwapStatus) | `fakeAgent` in controller/internal/selfupdate/updater_test.go |
|
||||
| `mailrelay.Forwarder` | controller/internal/mailrelay/forward.go | `HubForwarder` (hub relay endpoint) | `fakeForwarder` in controller/internal/mailrelay/mailrelay_test.go |
|
||||
| `integrations.Handler` + `StackProvider` | controller/internal/integrations/integrations.go + manager.go | OnlyOffice handlers | table-driven tests in package |
|
||||
| `bootstrap.PullFunc` | controller/internal/bootstrap/bootstrap.go | `report.PullConfig` | injected in bootstrap tests |
|
||||
| `offboxRunner` (func) | controller/internal/backup/offbox.go | `defaultOffboxRunner` (restic exec) | `SetOffboxRunner` injection point |
|
||||
| `dumpVolumesSafe` (func seam) | controller/internal/backup/backup.go | nil → real `DumpAppVolumesSafe` | injected in controller/internal/backup/volume_dumps_test.go (gating tests without Docker) |
|
||||
| `generateSecret` (func seam) | controller/internal/backup/backup.go | `stacks.Manager.GenerateSecretForField` via `SetSecretGenerator` (main.go) | injected in controller/internal/backup/restore_secrets_gen_test.go |
|
||||
| `restoreFilesCopier` (func seam) | controller/internal/backup/backup.go | nil → real `rsyncRestoreMissing` | injected in controller/internal/backup/tier2_restore_test.go (orchestration without rsync) |
|
||||
| `tier2Mirror` (func seam) | controller/internal/backup/backup.go | nil → real `rsyncMirror` | both RunTier2 rsync legs; injected in controller/internal/backup/tier2_test.go (resolve→mirror without rsync) |
|
||||
| `migSeams.resolveNames` (func seam) | controller/internal/stacks/migrate.go | nil → real `ResolveAppDataDirNames` (compose-derived) | injected in controller/internal/stacks/migrate_fs3_test.go (F-S3 appdata dir-name resolution) |
|
||||
|
||||
Cross-repo edges:
|
||||
- `controller/internal/agentapi/client.go` ↔ **felhom-agent** local API (`/storage`, `/disks*`, `/backup*`, `/netstorage*`, `/guest/*`): pinned leaf SHA-256 + per-guest bearer token from bootstrap.json.
|
||||
- `controller/internal/report/pusher.go` → **hub** `/api/v1/report` ingest; ACK `config_version` drives config_refresh.go; `controller/internal/notify/notifier.go` → hub `/api/v1/event` (hub-side `allowedEventTypes` allowlist must include new types).
|
||||
- `controller/internal/sync/sync.go` ← **app-catalog-felhom.eu**: copies ONLY `docker-compose.yml` + `.felhom.yml` per app (SHA-256 change detection); NEVER overwrites `app.yaml` (deployed secrets).
|
||||
|
||||
## 5. Extension points (where new features plug in)
|
||||
|
||||
- **New storage web endpoint**: switch in `ServeStorageAPI` (controller/internal/web/storage_handlers.go); disk ops in `ServeDiskAPI` (controller/internal/web/agent_disk_handlers.go); backup in `ServeBackupAPI`; export in `ServeExportAPI`; debug in `handleDebugAPI` (debug-mode gated).
|
||||
- **New REST endpoint**: path dispatch in `Router.ServeHTTP` (controller/internal/api/router.go); use `writeJSON` + `limitBody`.
|
||||
- **New background job**: `sched.Every`/`sched.Daily` registration block in controller/cmd/controller/main.go.
|
||||
- **New template function**: `Server.templateFuncMap` (controller/internal/web/funcmap.go) — obey v2 state-suffix vocabulary.
|
||||
- **New page/nav item**: `baseData` + sidebar in controller/internal/web/templates/ (nested sub-links pattern `.nav-links-nested`); must pass `controller/scripts/template_id_gate.py` + `controller/scripts/emoji_gate.py` + `controller/scripts/native_confirm_gate.py` + `controller/scripts/offbox_rename_gate.py` + `controller/scripts/app_row_dedup_gate.py` + `controller/scripts/mojibake_gate.py`.
|
||||
- **Docker volume tar streaming (v0.125.0)**: `appexport.dockerExec` (seam, package var) + `withVolumeHelper`/`exportVolumeTar`/`importVolumeTar` — stream volume content via `docker cp` through a stopped helper container. NEVER `docker run -v <controller-local path>` — the daemon resolves `-v` host-side and strands the data when the controller is containerized (the v0.124.0 HIGH finding); `controller/scripts/docker_run_volume_path_gate.py` enforces (every `"-v"` allowlisted with its WHY).
|
||||
- **Guarded file download (v0.124.0)**: `handler_export_download.go` — the canonical shape for streaming a server-side file to the browser: accept a BASENAME only (shape regexp + no separators/`..`), `filepath.Join` then assert `filepath.Dir(path) == dir`, `io.Copy` (never ReadAll), `Content-Disposition: attachment`, remove after a successful stream, TTL sweep (`sweepFabDownloads(dir, now, maxAge, logger)` — now injected for tests). Red-proof the guard by loosening to prefix-matching (the `..` case must fail).
|
||||
- **Backups sub-page data**: `backupsCommonData(page, title, r)` + `backupsOffboxData(data)` (handlers.go) — the ONLY builders for the four `/backups*` pages; a new backups section extends these, never re-derives in a page handler. (The one-shot v0.124.0 move gate `backups_split_move_check.py` was retired in v0.126.0.)
|
||||
- **App-list row (v0.126.0)**: `app_list_row`/`app_list_row_end` in `controller/internal/web/templates/app_row.html` is THE canonical list pattern — icon+name(+secondary) left, caller action block right; open with `dict "Slug" ... "Name" ...` (optional `Secondary`/`RowClass`/`Href`/`FallbackIcon`), close with `app_list_row_end`. Do NOT hand-roll app rows — `controller/scripts/app_row_dedup_gate.py` enforces single-sourcing (the backups_apps expander header is the one allowlisted aligned copy). Infra display identity: `inframeta.go` map + `infraMeta` func (filebrowser is the only Linked stack).
|
||||
- **Consequential-action confirm (LIGHT)**: `felhomConfirm(el, question, onYes)` in layout.html (v0.123.0) — the trigger swaps in place to "kérdés + Igen/Mégse"; form buttons opt in with `data-confirm="…"` (delegated listener, `requestSubmit` keeps formaction/name-value). NEVER native `confirm()`/`prompt()` (OS-modals freeze browser automation — drill F-11; `native_confirm_gate.py` enforces). Heavy destructive flows keep the `.confirm-overlay` `openDialog` pattern.
|
||||
- **New hub event**: typed `Notify*` wrapper on Notifier + hub allowlist entry (cross-repo).
|
||||
- **New app integration**: `integrations.Manager.RegisterHandler` with `IntegrationKey(provider, target)`.
|
||||
- **New startup self-check**: append check fn in `selftest.Run` (controller/internal/selftest/selftest.go).
|
||||
- **New settings field**: struct + accessor pair in controller/internal/settings/settings.go following the Lock→mutate→save pattern.
|
||||
|
||||
## 6. Known duplication (observed — NOT fixed)
|
||||
|
||||
| Duplication | Locations |
|
||||
|---|---|
|
||||
| Atomic write ×4 (+inline in settings.save) | controller/internal/backup/recovery_unit.go `atomicWrite`; controller/internal/bootstrap/bootstrap.go `writeFileAtomic`; controller/internal/setup/handlers.go `atomicWriteFile`; controller/internal/api/router.go `writeConfig0600` |
|
||||
| String truncation ×3 | controller/internal/util/strings.go `TruncateStr` (rune-safe, 1 caller); controller/internal/stacks/manager.go `truncateStr` (byte-based, widely used); controller/internal/backup/offbox.go `truncate` |
|
||||
| humanizeBytes ×3 | controller/internal/appbackup/appdata.go `HumanizeBytes` (canonical) + private twin; controller/internal/appexport/estimate.go `humanizeBytes`; controller/internal/backup/appbackup_bridge.go wrapper (deliberate bridge) |
|
||||
| copyFile ×2 | controller/internal/stacks/migrate.go (returns bytes) vs controller/internal/appexport/export.go |
|
||||
| dir-size ×6 | controller/internal/stacks/delete.go `getDirSizeBytes`/`getDirSizeHuman`; controller/internal/backup/tier2.go `dirSizeBytes` (du -sb); controller/internal/appexport/estimate.go `dirSize`+`duBytes`; controller/internal/appexport/export.go `calcDirSize`; controller/internal/web/handlers.go `dirSizeHuman` |
|
||||
| timeAgo switch body ×2 | controller/internal/web/funcmap.go `timeAgo` vs `timeAgoStr` (identical formatting logic) |
|
||||
| CSRF ×2 | controller/internal/web/csrf.go (session HMAC) vs controller/internal/setup/csrf.go (cookie double-submit) — intentional (pre-auth wizard) but unlabeled |
|
||||
| Budapest timezone loader ×3 | controller/internal/scheduler/scheduler.go `getBudapestLocation` vs controller/internal/web/funcmap.go `getTimezone` vs controller/internal/quiesce/quiesce.go `budapestLocation` (v0.168.0 window gate — Budapest wall-clock, kept local to avoid a scheduler↔quiesce import edge) |
|
||||
| JSON writers ×5, 3 envelope shapes | api `writeJSON`; web `writeDiskJSON`, `jsonResponse`/`jsonError`, `writeDebugJSON` |
|
||||
| Safe-name validators ×4 | controller/internal/web/validate.go `validStackName`; controller/internal/api/router.go `validStackParam` (same body — api↔web import cycle); controller/internal/backup/offbox.go `isSafeStackName`; controller/internal/appexport/validate.go `ValidateSegment` (strictest) |
|
||||
| DB wait/import ×2 | controller/internal/appbackup/dbdump.go `waitDBReady`/`ImportDump` vs controller/internal/appexport/restore.go `waitForDB`/`importDBDump` |
|
||||
| compose exec ×2 | controller/internal/stacks/manager.go `composeExecCustomEnv` vs controller/internal/appexport/restore.go `composeExecEnv` (the latter has ctx+timeout; the former has the userdata belt) |
|
||||
@@ -1,134 +0,0 @@
|
||||
# RUNBOOK — End-to-end live drive (SUPERVISED) — felhom-controller
|
||||
|
||||
**Purpose:** a hands-on end-to-end exercise of the demo stack that deliberately stresses the fixes
|
||||
shipped 2026-06-13 — **CTRL-T2-1** (crash-safe deploy state), **CTRL-001** (import path traversal),
|
||||
**AGENT-001** (anti-retarget wipe), plus backup/restore — and surfaces UX/latency/error-handling
|
||||
friction. **Execute SUPERVISED (operator present).** Do the non-destructive sections (1–6) first; the
|
||||
destructive section (7) is last and gated.
|
||||
|
||||
**Status:** NOT executed yet — written 2026-06-13 for the supervised session.
|
||||
|
||||
## Environment & conventions
|
||||
|
||||
- **Demo controller:** guest **9201** (`demo-felhom`) on Proxmox host `felhom-pve` (192.168.0.162),
|
||||
bootstrap-managed. Public dashboard/API: **https://felhom.demo-felhom.eu** (no dashboard password →
|
||||
the API is open; drive it via the PUBLIC URL, not the container IP).
|
||||
- **Versions at writing:** controller **v0.60.0**, agent **v0.30.0**, hub v0.11.0.
|
||||
- Run from DooPlex (192.168.0.180); host root via SSH alias `felhom-pve` — plain `ssh felhom-pve`.
|
||||
(Legacy Windows workstation: needed `SSH=/c/Windows/System32/OpenSSH/ssh.exe` and
|
||||
`export MSYS_NO_PATHCONV=1` for `pct exec`.)
|
||||
- **Findings log:** record every observation (✓/✗ + notes on UX friction, latency, confusing labels,
|
||||
error handling) in a new `REPORT-e2e-live-drive-<date>.md`. Each step says what "good" looks like and
|
||||
what to watch for.
|
||||
- **Markers:** **[DESTRUCTIVE — operator confirm]** = needs operator eyes + explicit go-ahead.
|
||||
**[HUMAN]** = cannot be done by the agent (physical / real-decision).
|
||||
|
||||
---
|
||||
|
||||
## 1. Baseline (read-only)
|
||||
|
||||
1. Controller healthy + version:
|
||||
- `curl -s https://felhom.demo-felhom.eu/api/health` → `{"ok":true,...}`.
|
||||
- `ssh felhom-pve "pct exec 9201 -- docker ps --filter name=felhom-controller --format '{{.Image}} {{.Status}}'"` → `:0.60.0 Up ... (healthy)`.
|
||||
- Dashboard loads (Hungarian UI), no error banners.
|
||||
2. Agent healthy + version: `ssh felhom-pve "systemctl is-active felhom-agent; /usr/local/bin/felhom-agent --version"` → `active`, `0.30.0`.
|
||||
3. Headroom (deploys pull images — **bound to ≤3 small apps**):
|
||||
- Docker-data volume free: dashboard storage bars, or `ssh felhom-pve "pct exec 9201 -- df -h /var/lib/docker /"`. Need comfortably above the v0.58 reserve (`max(5GB,10%)`) or deploys will be gated **507**.
|
||||
- RAM: `ssh felhom-pve "pct exec 9201 -- free -h"`.
|
||||
- Disk list sane: `curl -s https://felhom.demo-felhom.eu/api/disks` → felhom-usb (user-data, data_bearing), local/local-lvm (system), felhom-pbs (backup).
|
||||
4. Record current deployed apps (so cleanup is unambiguous): dashboard "Alkalmazások", or `curl -s https://felhom.demo-felhom.eu/api/stacks/rescan` then the stacks list. (actualbudget is expected already deployed.)
|
||||
- **Good:** all green/healthy; free space well above reserve. **Watch for:** any app stuck "Telepítés alatt" (deploying) from a prior run — note and resolve before starting.
|
||||
|
||||
---
|
||||
|
||||
## 2. Deploy flow — happy path (stresses CTRL-T2-1, normal case)
|
||||
|
||||
Pick **two small apps** not currently deployed (suggest: `vikunja`, `mealie` — small images; avoid immich/nextcloud/paperless which are large pulls). UI: **Alkalmazások → <app> → Telepítés**, or API.
|
||||
|
||||
1. Deploy app #1 via UI (fill any required fields, generate passwords where prompted), click **Telepítés**.
|
||||
- Watch the **3-step progress panel** (config → containers → health) and the live polling
|
||||
(`GET /api/stacks/<app>` every 3s). API equivalent: `curl -s -X POST https://felhom.demo-felhom.eu/api/stacks/<app>/deploy -H 'Content-Type: application/json' -d '{"values":{}}'` then poll `GET /api/stacks/<app>`.
|
||||
- **Good:** progresses config→containers→health; ends `running`/healthy within ~120s; the card flips to deployed; no "Telepítés" button reappears mid-pull (in-memory Deployed=true during pull).
|
||||
- **Watch for:** stuck at a step, health-probe never going green (check the app's healthcheck tool exists), confusing Hungarian labels, the deploy gate returning **507** (insufficient Docker-data headroom — expected if low on space; note the banner wording).
|
||||
2. Deploy app #2; same checks.
|
||||
3. Confirm on disk the durable record is correct (CTRL-T2-1, happy case): `ssh felhom-pve "pct exec 9201 -- docker exec felhom-controller cat /opt/docker/stacks/<app>/app.yaml | grep deployed"` → `deployed: true` (only after success).
|
||||
- **Good:** `deployed: true` on disk after a successful deploy. **Watch for:** secrets appearing in plaintext in app.yaml (they must be `enc:`-prefixed — H10/encryption check).
|
||||
|
||||
---
|
||||
|
||||
## 3. Deploy crash-window probe (THE CTRL-T2-1 test)
|
||||
|
||||
Goal: prove a crash during the image-pull window leaves the stack **NOT-deployed and redeployable**, not ghost-stuck.
|
||||
|
||||
1. Pick a **third app with a non-trivial image pull** (so the pull window is a few seconds — e.g. `paperless-ngx` if space allows, else `mealie`). Start the deploy (UI Telepítés or API POST), and **immediately** — while it is still pulling (status `deploying`, before `running`) — kill the controller:
|
||||
- `ssh felhom-pve "pct exec 9201 -- docker kill felhom-controller"` **[operator: time this during the pull]**
|
||||
- The bootstrap service (`felhom-controller-bootstrap.service`) restarts it within seconds. Confirm back up: `curl -s https://felhom.demo-felhom.eu/api/health`.
|
||||
2. After restart, check the stack state:
|
||||
- On disk: `ssh felhom-pve "pct exec 9201 -- docker exec felhom-controller cat /opt/docker/stacks/<app>/app.yaml | grep deployed"` → **`deployed: false`** (transitional — the fix).
|
||||
- UI/API: `GET /api/stacks/<app>` → state `not_deployed` (the card shows **Telepítés**, not a ghost "deployed").
|
||||
3. **Redeploy** the same app — it must be **allowed** (no "already deployed; use update instead" refusal) and complete normally.
|
||||
- **Good:** post-crash the app reads not-deployed and redeploys cleanly. **PRE-FIX behaviour (must NOT occur):** app.yaml `deployed: true` with no containers, and redeploy refused — that's the ghost-stuck regression the fix removes.
|
||||
- **Watch for:** any orphaned containers from the killed pull (`docker ps -a`); the half-pulled image is fine (compose re-pulls).
|
||||
|
||||
---
|
||||
|
||||
## 4. Import flow — round-trip + negative (THE CTRL-001 test)
|
||||
|
||||
1. **Export** a deployed app to a `.fab` bundle. UI: app page → **Exportálás** (or API `POST /api/export/start` with `{"stack_name":"<app>","drive":"<drive>"}`, poll `GET /api/export/status`). The `.fab` lands under the chosen drive's `exports/`.
|
||||
- **Good:** export completes; a `.fab` appears; manifest readable via `POST /api/export/manifest`.
|
||||
2. **Re-import** the same `.fab`: UI import flow (or `POST /api/export/import` with the file path), poll `GET /api/export/import/status`.
|
||||
- **Good:** imports, recreates the stack, data restored; fail-closed data-key gate honored if the app has a data-encrypting key.
|
||||
3. **Negative — path traversal (CTRL-001):** craft a hostile `.fab` and confirm it is **rejected at parse**, not written.
|
||||
- Build a minimal bundle whose `manifest.json` has `"app_name":"../evil"` (and/or an `hdd_subdirs` / `volume_names` entry with `../`). Place it under a registered `exports/` dir on the host:
|
||||
`ssh felhom-pve "pct exec 9201 -- docker exec felhom-controller sh -c 'ls /mnt/felhom-usb/exports/'"` to find the dir.
|
||||
- Attempt import of the hostile bundle.
|
||||
- **Good (the fix):** import **fails immediately** with a manifest/validation error; **no directory is created outside the stacks dir** (verify: `ssh felhom-pve "pct exec 9201 -- docker exec felhom-controller ls -la /opt/docker/evil /etc/evil 2>/dev/null"` → nothing). **PRE-FIX (must NOT occur):** a dir/file written outside `/opt/docker/stacks/`.
|
||||
- **Watch for:** the error message clarity (does the UI explain why it was rejected?).
|
||||
|
||||
---
|
||||
|
||||
## 5. Backup — per-app + full (3-2-1 tiers)
|
||||
|
||||
1. **Per-app / on-demand DB-dump + recovery-unit refresh:** trigger `POST /api/backup/run` (or the dashboard "Biztonsági mentés" action). Watch `GET /api/backup/status`.
|
||||
- **Good:** completes; DB dumps written under `<drive>/backups/primary/<app>/db-dumps/`; recovery unit refreshed (`backups/primary/<app>/` has compose + manifest.json).
|
||||
2. **Tier-2 off-drive copy:** `POST /api/backup/tier2`. (Single-drive demo → Tier-2 goes to the internal SSD for SMALL units only, DB/config-only, with the rootfs-headroom guard.)
|
||||
- **Good:** the per-app "2. mentés" card in the UI populates (success → target drive, or an honest "needs 2nd HDD" no-target reason). **Watch for:** the rootfs-headroom guard correctly **refusing** rather than filling rootfs when the unit is too big for the SSD.
|
||||
3. **Whole-guest (PBS) tier** is the agent's: confirm via the monitoring page that the PBS/offsite tier shows recent snapshots (the controller surfaces agent/PBS status; it does not run vzdump itself).
|
||||
- **Good:** 3-2-1 visible — primary (on-drive recovery unit) + secondary (Tier-2 off-drive) + offsite (PBS). **Watch for:** any tier showing stale/never — note it.
|
||||
4. Verify completion is reflected, not just started: re-check `GET /api/backup/status` returns `done` and the UI timestamps update.
|
||||
|
||||
---
|
||||
|
||||
## 6. Wipe gate re-check (AGENT-001 — NON-destructive)
|
||||
|
||||
Re-confirm the two refusals proven on 2026-06-13. **Do NOT send a matching confirmation for a real data drive.** Target the real data-bearing device `/dev/sdb1` (felhom-usb) — both calls REFUSE, nothing is formatted.
|
||||
|
||||
1. **Refusal A — no confirmation:** `curl -s -w '\n%{http_code}\n' -X POST https://felhom.demo-felhom.eu/api/disks/format -H 'Content-Type: application/json' -d '{"device":"/dev/sdb1","fstype":"ext4"}'`
|
||||
- **Good:** `formatted:false`, `needs_confirmation:true`, **HTTP 409**; no mkfs.
|
||||
2. **Refusal B — wrong durable_id:** same call with `"confirmed":true,"durable_id":"byid:wwn-0xDEADBEEF-DOES-NOT-EXIST"`.
|
||||
- **Good:** `formatted:false`, refused **409**; a non-matching confirmation does not authorize a wipe.
|
||||
3. **Data-safety assertion:** `ssh felhom-pve "findmnt /mnt/felhom-usb -o TARGET,SOURCE,FSTYPE; pct exec 9201 -- docker exec felhom-controller sh -c 'df -h /mnt/felhom-usb'"` → still mounted, used space unchanged.
|
||||
4. **Happy-path destructive wipe** = **[HUMAN]** — never wipe a real/customer drive to test; covered by the agent unit test `retarget-mismatch-refused`. Only on a genuinely disposable blank device, supervised. **[DESTRUCTIVE — operator confirm]**
|
||||
|
||||
---
|
||||
|
||||
## 7. Restore — DESTRUCTIVE **[DESTRUCTIVE — operator confirm]**
|
||||
|
||||
Do this LAST, on a NON-critical app (e.g. one deployed in §2, not customer RomM data). Restore overwrites the app's current data.
|
||||
|
||||
1. Pick an app with a backup/snapshot from §5. Note its current data state (log in, note an item) so you can confirm the restore round-trip.
|
||||
2. **[operator confirm]** Restore: `POST /backup/restore` (web form: `stack_name=<app>&snapshot_id=<id>`), or the dashboard restore UI.
|
||||
- **Good:** the app stops, restores volume tars / recovery unit, restarts; data returns to the snapshot state; the fail-closed data-key gate refuses (with a clear message) if a data-encrypting key can't be recovered — in which case STOP and do NOT force.
|
||||
- **Watch for:** data loss vs the snapshot, secret-regeneration (must recover from the guest's own app.yaml, regenerate nothing), confusing progress/labels.
|
||||
3. **[operator]** Any **delete/remove** of a stack to clean up the §2/§3 test apps is **[DESTRUCTIVE — operator confirm]** — protected stacks (traefik/cloudflared/filebrowser/controller) must remain unstoppable server-side; confirm a delete attempt on one of those is refused.
|
||||
|
||||
---
|
||||
|
||||
## 8. Wrap-up
|
||||
|
||||
- Tear down the test apps deployed in §2/§3 (operator-confirmed deletes), leaving the demo as found (actualbudget + RomM intact).
|
||||
- Write `REPORT-e2e-live-drive-<date>.md`: per-section ✓/✗, every UX/latency/error-handling finding, and any regression (especially: did §3 leave a ghost-deployed stack? did §4 write outside the stacks dir? did any wipe call format?). File new bugs against the audit record.
|
||||
- Confirm final: controller `:0.60.0` healthy, agent `0.30.0` active, felhom-usb intact, no orphaned containers.
|
||||
|
||||
### Order summary (non-destructive first)
|
||||
1 Baseline → 2 Deploy happy → 3 Crash-window probe → 4 Import + traversal-negative → 5 Backup → 6 Wipe-gate refusals → **7 Restore [DESTRUCTIVE]** → 8 Wrap-up.
|
||||
-104
@@ -1,104 +0,0 @@
|
||||
# TEST-REPORT — Test campaign #3: NO MERCY (N100 / guest 9201)
|
||||
|
||||
**Run start:** 2026-06-22 (CC, unattended). Brutal chaos/edge — push to unintended states, verify
|
||||
handles-or-fails-safe, then recovers. Every break carries a timed auto-revert; Phase-0 backup is the floor.
|
||||
**Legend:** PASS / FAIL / SKIP + raw evidence. Each chaos line: break→detect→recover→verify.
|
||||
(Campaigns #1/#2 + diagnoses in git history + `felhom.eu/documentation/tests/`.)
|
||||
|
||||
## Phase 0 — Baseline + floor — **PASS (gate OPEN)**
|
||||
- ctrl **v0.75.0** healthy; agent v0.39.0; **25 containers, 0 unhealthy**; rootfs `/` 4%, `/var/lib/docker` 8%, drives 1%; mem **8.8 Gi available**.
|
||||
- Floor: PBS `felhom-pbs:backup/ct/9201/2026-06-22T19:01:56Z` (success, crash-consistent, verified).
|
||||
|
||||
## Phase 1 — Resource starvation — **PASS (fail-safe held)**
|
||||
|
||||
| # | Break (timed-revert) | Result | Evidence |
|
||||
|---|---|---|---|
|
||||
| R1 | python hog ~8 GB (~91% RAM), 75s | **PASS** | mid-stress: 10Gi used / 1.1Gi avail — controller **healthy**, 25/0 unhealthy; post: **dockerd NRestarts=0, no OOM** (alloc fit via cache eviction), recovered to 8.9Gi. Backstop pkill guard armed. |
|
||||
| R2 | settings save on full disk | **PASS (code-verified)** | `settings.save()` = atomic write-`.tmp`-then-`os.Rename` (`settings.go:236-261`) → a disk-full fails at WriteFile, original untouched. Live disk-full not reproducible: settings.json is on the 252 GB docker vol. |
|
||||
| R3 | fill rootfs (holds `/mnt/sys_drive` backups) to 94% (timed rm @150s) | **PASS** | controller stayed healthy (it's on the separate 252 GB vol); DB-dump + Tier-2 (8 apps, immich 155MB) **succeeded**; **settings.json stayed valid JSON** (no corruption). Reverted → rootfs 4%. (Literal-full refusal not pushed — near-0 rootfs risks wedging the guest OS unattended; the 2 GB margin had room so the gate didn't need to refuse.) |
|
||||
| R4 | memory gate hard-block | **PASS (code-verified)** | `deploy.go:186` is a **hard block** (returns an error on `committed+new > usable`), using **committed-memory** accounting (sum of deployed `mem_request`), reserved 384 MB, usable 11904 MB. Live committed ≈ 4.4 GB → ~7.5 GB headroom, so tripping needs mass-deploying ~7.5 GB of requests (impractical/risky unattended); the branch + math are verified. |
|
||||
## Phase 2 — State corruption [HIGH SEV] — **mostly PASS; 1 medium finding (S1)**
|
||||
|
||||
| # | Break (backed up first) | Result | Evidence |
|
||||
|---|---|---|---|
|
||||
| S1 | truncate settings.json → restart | **⚠ FINDING (medium)** | controller **crash-loops**: `[FATAL] Failed to load settings … unexpected end of JSON input`, RestartCount=7, restarting. **No safe-defaults fallback** — a corrupt settings.json takes the management plane down. *Not silent* (FATAL logged — better than the worst case). Restore → running/healthy, 3 storage_paths back. (Docker restart-manager cycling it re-confirms Finding #1.) |
|
||||
| S2 | garbage in uptime-kuma `app.yaml` → rescan | **PASS** | controller stays **healthy** (25/0), logs `[WARN] LoadAppConfig: yaml: … did not find expected key` (not silent, not fatal), other apps unaffected. Restore → `running/deployed`. **Contrast with S1:** per-stack app.yaml corruption is graceful; settings.json is fatal. |
|
||||
| S3 | corrupt quiesce marker (bad JSON) → restart | **PASS (minor finding)** | no panic/crash-loop, controller healthy, **stacks not stranded** (restarts=0). But the unparseable marker is **silently ignored** (no log) **and left in place** (not cleared/quarantined) — a real corrupted-mid-quiesce marker would skip recovery with no signal. |
|
||||
| S4 | symlink `→ /etc` in userdata/appdata → backup | **PASS (security holds)** 🔒 | Tier-2 rsync **preserved the symlink** (`EVIL_ETC -> /etc`), did **not** follow it; **/etc contents did NOT leak** into the backup (no passwd/shadow). No path-escape/exfil. |
|
||||
## Phase 3 — Concurrency storms — **PASS**
|
||||
|
||||
| # | Break | Result | Evidence |
|
||||
|---|---|---|---|
|
||||
| C1 | deploy + backup + restore + git-sync simultaneously | **PASS** | backup/run + tier2 both single-flighted ("Mentés már folyamatban"); restore mutex'd (302); sync ran independently ("nincs változás"); after settle **no stuck flag** (`running:false`), no deadlock, 25/0 healthy |
|
||||
| C2 | rapid felhom-usb flap ×5 in ~10s | **PASS** | converged → felhom-usb **MOUNTED**, disconnected mark `None` (not stuck); **0 `permission denied`** during the flap; the v0.75 mountpoint-gate fired **5 clean skips** ("not mounted") in the disconnected windows; 25/0 |
|
||||
| C3 | kill `felhom-agent` mid-quiesce-backup | **PASS** | quiesce stopped 14 stacks → agent killed → controller `[quiesce] unquiescing (backup start failed): restarting 14 stack(s)` (clean error + defer-unwind, **no stack stranded**); agent restarted → `/api/disks` OK; **idle sockets→8443 = 1** (v0.74 bound holds); recovered 25/0 |
|
||||
## Phase 4 — Time chaos — **SKIPPED (not safely isolable)**
|
||||
|
||||
T1 (+25h) / T2 (−2h): the guest is an **unprivileged LXC** — it **cannot set its own clock**
|
||||
(`date -s` → `Operation not permitted`, no CAP_SYS_TIME) and shares the host's `CLOCK_REALTIME` (time
|
||||
namespaces don't isolate the wall clock). The only way to jump the guest's wall-clock is to change the
|
||||
**host** (felhom-pve) clock, which would risk: cloudflared tunnel cert time-validation (demo down),
|
||||
the agent's leaf-cert pin to PVE/PBS, and **agent→DooPlex hub/PBS TLS auth** (§0.1 forbids degrading
|
||||
DooPlex). NTP is active on the host. **Not safely isolable to the guest unattended → SKIP.** Proper
|
||||
venue: a dedicated throwaway VM with its own clock, or an injectable-clock unit test. **Deferred to
|
||||
supervised / implementation.**
|
||||
## Phase 5 — Network partitions (guest/host-side timed rules) — **PASS**
|
||||
|
||||
| # | Break (timed-revert) | Result | Evidence |
|
||||
|---|---|---|---|
|
||||
| N1 | block cloudflared egress (iptables DOCKER-USER, 130s) | **PASS** | tunnel partitioned: cloudflared `timeout: no recent network activity` + `Retrying connection` (no crash); **apps serve locally** (romm → HTTP 200); controller healthy 25/0. Unblock → `Registered tunnel connection`; no leftover rule. |
|
||||
| N2 | block agent→PBS `192.168.0.180:8007` mid-quiesce-backup (N100-side OUTPUT rule per §0.1, 150s) | **PASS** | quiesce stopped 14 stacks → `[quiesce] backup job failed` (vzdump phase=failed, clean error) → `[quiesce] unquiescing (backup failed): restarting 14 stack(s)` (**fails safe, no stack stranded**). Unblock → next PBS backup `success:true`; 25/0; no leftover rule. (Rule was N100-side; DooPlex untouched.) |
|
||||
## Phase 6 — Input/security fuzzing [HIGH SEV] — **PASS (no escape) + 1 medium defense-in-depth finding**
|
||||
|
||||
| # | Break | Result | Evidence |
|
||||
|---|---|---|---|
|
||||
| F1 | malformed/oversized/unknown-field JSON | **PASS** | not-json → **400**, empty → 400, 1MB → 400, nested-object-where → 400; unknown fields safely ignored (502 from a downstream non-existent mount, **no partial mutation**); controller **healthy** (no panic) |
|
||||
| F2 | path traversal (storage `where`, restore `stack_name`) | **PASS (no escape) 🔒 + FINDING** | **storage:** all 5 traversals (`/etc`, `../../etc`, `/mnt/../etc`, `/mnt/felhom-drives/../../../etc`, `/etc/shadow`) **rejected** by `gateWhere` (`path.Clean`+`HasPrefix("/mnt/")`). **restore:** the boundary HELD — `/etc/passwd` **intact**, no `/opt/etc`/`/etc/passwd/` artifacts, no `/etc` writes — but **the restore handler does NOT reject a traversal `stack_name` upfront**: `RestoreFromRecoveryUnit("../../../etc")` proceeded (`GetAppDrivePath` → default `/mnt/sys_drive`), saved only by downstream **map-based** `StopStack`/`StartStack` (`stack "../../../etc" not found`) + no recovery-unit/volumes (no-op). **Defense-in-depth gap — recommend explicit `stack_name` validation at the restore handler.** (My own batched test looped these restores ×90s each, briefly holding the mutex — a test artifact that cleared when stopped, NOT a stuck-flag bug.) |
|
||||
| F3 | hostile compose on a scratch app | **N/A (good property)** | the standard deploy renders only from the git-synced **catalog** — there is **no customer-facing arbitrary-compose injection vector**. (`.fab` import is the only path; not fuzzed — lower priority.) |
|
||||
| F4 | weird userdata filenames (unicode, spaces, `(N)`, 200-char) | **PASS** | files created + Tier-2 backup ran clean ("Tier 2 run complete: 8 apps"), no crash/panic, 25/0. (`(N)` merge/dedup is migration-specific — `migrate.go` unit-tested, not live-run here.) |
|
||||
## Phase 7 — Brutal recovery — **PASS (B1-B3); B4 skipped**
|
||||
|
||||
| # | Break | Result | Evidence |
|
||||
|---|---|---|---|
|
||||
| B1 | restart dockerd in 9201 | **PASS** | boot-restore brought all **25 containers + controller** back in <8s (drives stay mounted on a daemon-only restart → no boot-ordering issue); controller+cloudflared healthy |
|
||||
| B2 | `kill -9` controller PID (real crash) | **PASS** | `unless-stopped` auto-recovered: RestartCount 0→1, **healthy in 8s** — re-confirms Finding #1 (docker kill ≠ crash) live |
|
||||
| B3 | reboot 9201 with **felhom-flash detached** | **PASS** 🔑 | booted; flash **DETACHED** (disconnect intent persisted across reboot); 8 HDD apps **held** (absent, not crash-looping); **NO rootfs shadow dirs** (`…/felhom-flash/userdata` doesn't exist). Docker boot-restore made 2 bind-source mkdir attempts → **denied by the unprivileged-LXC mapping** → no escape (the v0.75 gate covers the controller's post-boot belt/FileBrowser; the unprivileged mapping covers docker's boot-restore). Reconnect → flash remounted, 8 apps recovered → 25/0. |
|
||||
| B4 | host reboot of felhom-pve | **SKIPPED** | unattended risk — no physical recovery if the N100 doesn't POST/return. Deferred to supervised (consistent with #1/#2). |
|
||||
|
||||
---
|
||||
## Findings (ranked by real-world severity)
|
||||
|
||||
1. **[MEDIUM] Corrupt `settings.json` → controller FATAL crash-loop (S1).** No safe-defaults fallback —
|
||||
a truncated/invalid settings.json takes the management plane down (RestartCount climbs via docker's
|
||||
restart-manager). *Not silent* (FATAL is logged — strictly better than the worst case). Apps/tunnel
|
||||
keep running (control/data separation). **Fix:** on parse failure, log a WARN + load safe defaults
|
||||
(or boot read-only) instead of `[FATAL]`. (Contrast: per-stack `app.yaml` corruption (S2) is handled
|
||||
gracefully with a WARN — settings.json should match that.)
|
||||
2. **[MEDIUM, defense-in-depth] Restore `stack_name` not validated against path traversal (F2).** The
|
||||
`/backup/restore` handler proceeds with a traversal name (`../../../etc`) into
|
||||
`RestoreFromRecoveryUnit`/`RestoreApp`; **no escape occurred** (`/etc` intact, no artifacts) only
|
||||
because downstream `StopStack`/`StartStack` are map-based ("stack not found") + there's no
|
||||
recovery-unit/volumes for a bogus name. A future code path that built a filesystem path from the raw
|
||||
name could escape. **Fix:** reject `stack_name` containing `/`, `..`, NUL, etc. at the handler.
|
||||
3. **[LOW] Corrupt quiesce marker silently ignored (S3).** `Recover()` doesn't panic (good) but an
|
||||
unparseable marker is **neither logged nor cleared/quarantined** — a real corrupted-mid-quiesce
|
||||
marker would skip stack-recovery with no signal. **Fix:** log + quarantine a bad marker.
|
||||
4. **[INFO] Boot-ordering (re-confirmed, B3).** Docker's boot-restore attempts drive-backed bind-source
|
||||
`mkdir` before mounts converge; the **unprivileged-LXC mapping denies it** (no shadow dirs), and the
|
||||
v0.75 gate covers the controller side — benign but noisy. (Tracked from the v0.75 task.)
|
||||
|
||||
**Everything else fail-safe held:** R1-R4, S2, S4, C1-C3, N1-N2, F1, F3, F4, B1-B3 all PASS. No
|
||||
silent-corruption and **no path-escape/exfil** found (the two highest-severity classes the runbook
|
||||
prioritized). **No code changes shipped** (no broken apps surfaced; the findings are behaviours logged
|
||||
for supervised fix). Phase 4 (time chaos) and B4 (host reboot) SKIPPED for architectural/unattended-risk
|
||||
reasons (documented).
|
||||
|
||||
## Cleanup confirmation
|
||||
- **9201 running, 25 containers, 0 unhealthy**; controller + cloudflared healthy; agent active.
|
||||
- Drives mounted (flash sdc1, usb sdb1); rootfs back to **4% / 29 G free** (R3 fill reverted).
|
||||
- **No leftover rules:** host `OUTPUT` 8007-DROP = 0 (N2), guest `DOCKER-USER` cloudflared-DROP = 0 (N1).
|
||||
- **No leftover guards** (the 2 residual `sleep` procs are host system monitors, parents 747/700843 — not mine).
|
||||
- Corrupted files restored (settings.json valid; app.yaml restored; quiesce marker removed; symlinks removed); F4 test files removed.
|
||||
- **No 9300 / scratch guest created** this campaign; no loopbacks (Phase 4 N/A, destructive disk used live drives only reversibly).
|
||||
- Pre-existing (NOT this run): 9001 (spike-lxc) + 9999 (felhom-selftest-scratch) stopped.
|
||||
- Net: demo restored to baseline (= Phase-0 state).
|
||||
@@ -4,13 +4,8 @@ bin/
|
||||
*.dll
|
||||
*.so
|
||||
*.dylib
|
||||
# ANCHORED (leading slash) on purpose: a bare `controller` also matches the DIRECTORY
|
||||
# cmd/controller/, so ripgrep silently skipped main.go and new files there needed `git add -f`.
|
||||
# Both directions produce inert-seam mistakes — a grep for a setter finds no caller and reads as
|
||||
# "this is unused", and a genuinely-new file never gets committed. Only the built binary at the
|
||||
# module root should be ignored here.
|
||||
/controller
|
||||
/controller.exe
|
||||
controller
|
||||
controller.exe
|
||||
|
||||
# Test artifacts
|
||||
coverage.out
|
||||
|
||||
@@ -39,6 +39,7 @@ FROM debian:bookworm-slim
|
||||
# Install runtime dependencies:
|
||||
# - docker-cli: for "docker compose" commands
|
||||
# - ca-certificates: for HTTPS (healthchecks pings, git)
|
||||
# - restic: for backup operations
|
||||
# - postgresql-client: for pg_dump
|
||||
# - default-mysql-client: for mysqldump
|
||||
# - sqlite3: for SQLite backup
|
||||
@@ -49,15 +50,12 @@ FROM debian:bookworm-slim
|
||||
# - e2fsprogs: mkfs.ext4 (filesystem formatting)
|
||||
# - rsync: for data migration between storage paths
|
||||
# - parted: partprobe (partition table re-read after sfdisk)
|
||||
# - restic: encrypted off-box (NAS) backup over SFTP (Part B; version pinned by the Debian release)
|
||||
# - openssh-client: restic's sftp backend shells out to `ssh` (the SFTP transport); also ssh-copy-id/sftp
|
||||
# for the SLICE-2 offsite apply-bridge key install
|
||||
# - sshpass: feeds the one-time storage-box password to ssh-copy-id -s -f (offsite apply-bridge)
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
ca-certificates \
|
||||
curl \
|
||||
gnupg \
|
||||
git \
|
||||
restic \
|
||||
postgresql-client \
|
||||
default-mysql-client \
|
||||
sqlite3 \
|
||||
@@ -66,9 +64,6 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
e2fsprogs \
|
||||
rsync \
|
||||
parted \
|
||||
restic \
|
||||
openssh-client \
|
||||
sshpass \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Install docker-cli (without daemon)
|
||||
|
||||
+51
-1733
File diff suppressed because it is too large
Load Diff
+3
-3
@@ -2,7 +2,7 @@
|
||||
# =============================================================================
|
||||
# felhom-controller — Docker image build script
|
||||
# =============================================================================
|
||||
# Location: /mnt/5_hdd/felhom.eu/build/felhom-controller/build.sh (moved off the DooPlex SSD 2026-07-18)
|
||||
# Location: /home/kisfenyo/build/felhom-controller/build.sh
|
||||
#
|
||||
# Copies source from the git repo, syncs app assets, and builds the image.
|
||||
# Build artifacts stay here — the git repo stays clean.
|
||||
@@ -16,9 +16,9 @@
|
||||
set -euo pipefail
|
||||
|
||||
# --- Configuration (edit these if your paths differ) ---
|
||||
REPO_DIR="/mnt/5_hdd/felhom.eu/git/felhom-controller"
|
||||
REPO_DIR="/home/kisfenyo/git/felhom-controller"
|
||||
CONTROLLER_SRC="${REPO_DIR}/controller"
|
||||
WEBSITE_ASSETS_DIR="/mnt/5_hdd/felhom.eu/git/felhom.eu/website/assets"
|
||||
WEBSITE_ASSETS_DIR="/home/kisfenyo/git/felhom.eu/website/assets"
|
||||
REGISTRY="gitea.dooplex.hu/admin"
|
||||
IMAGE="${REGISTRY}/felhom-controller"
|
||||
|
||||
|
||||
@@ -1,495 +0,0 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"go/ast"
|
||||
"go/parser"
|
||||
"go/token"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// R-166 §10 seam discipline — the recovery and the backfill are seams, and a seam that is never
|
||||
// called is the defect class this project has shipped four times: a correct component, green unit
|
||||
// tests that inject it directly, and no production caller.
|
||||
//
|
||||
// These walk main.go's AST. NOT strings.Contains — the sibling bootrecon test records the reason at
|
||||
// first hand: a commented-out call still satisfies a substring match, so the text version passed the
|
||||
// very red-proof it existed to fail. Comments are not code.
|
||||
|
||||
// mainBody returns func main()'s body from main.go, parsed.
|
||||
func mainBody(t *testing.T) *ast.BlockStmt {
|
||||
t.Helper()
|
||||
fset := token.NewFileSet()
|
||||
f, err := parser.ParseFile(fset, "main.go", nil, 0)
|
||||
if err != nil {
|
||||
t.Fatalf("parse main.go: %v", err)
|
||||
}
|
||||
for _, decl := range f.Decls {
|
||||
if fn, ok := decl.(*ast.FuncDecl); ok && fn.Name.Name == "main" && fn.Body != nil {
|
||||
return fn.Body
|
||||
}
|
||||
}
|
||||
t.Fatal("func main() not found in main.go")
|
||||
return nil
|
||||
}
|
||||
|
||||
// callsInMain returns, in source order, the names of every call in func main() whose function
|
||||
// expression is `x.Sel(...)` or `Sel(...)` — enough to identify the wiring calls by name.
|
||||
func callsInMain(t *testing.T, body *ast.BlockStmt) []string {
|
||||
t.Helper()
|
||||
var names []string
|
||||
ast.Inspect(body, func(n ast.Node) bool {
|
||||
call, ok := n.(*ast.CallExpr)
|
||||
if !ok {
|
||||
return true
|
||||
}
|
||||
switch fun := call.Fun.(type) {
|
||||
case *ast.SelectorExpr:
|
||||
names = append(names, fun.Sel.Name)
|
||||
case *ast.Ident:
|
||||
names = append(names, fun.Name)
|
||||
}
|
||||
return true
|
||||
})
|
||||
return names
|
||||
}
|
||||
|
||||
func indexOfCall(names []string, want string) int {
|
||||
for i, n := range names {
|
||||
if n == want {
|
||||
return i
|
||||
}
|
||||
}
|
||||
return -1
|
||||
}
|
||||
|
||||
// TestMainWiresAppStopRecovery is the Group-I seam test. Comment out the `appStopGuard.Recover()`
|
||||
// line in main.go and this fails, where every behavioural test in internal/backup still passes.
|
||||
func TestMainWiresAppStopRecovery(t *testing.T) {
|
||||
names := callsInMain(t, mainBody(t))
|
||||
|
||||
if indexOfCall(names, "NewAppStopGuard") < 0 {
|
||||
t.Fatal("func main() no longer builds the R-166 app-stop guard — nothing writes or reads the marker")
|
||||
}
|
||||
if indexOfCall(names, "SetStarter") < 0 {
|
||||
t.Fatal("func main() no longer calls SetStarter on the app-stop guard — Recover would find the " +
|
||||
"marker and be unable to start anything, leaving every interrupted app down")
|
||||
}
|
||||
if indexOfCall(names, "Recover") < 0 {
|
||||
t.Fatal("func main() no longer calls Recover() on the app-stop guard — apps left stopped by an " +
|
||||
"interrupted backup stay down forever (the R-166 defect, un-fixed)")
|
||||
}
|
||||
if indexOfCall(names, "SetAppStopGuard") < 0 {
|
||||
t.Fatal("func main() no longer hands the recovered guard to the backup manager — the manager " +
|
||||
"would build a SECOND guard over the same file, i.e. one file with two owners")
|
||||
}
|
||||
if indexOfCall(names, "SetStopGuard") < 0 {
|
||||
t.Fatal("func main() no longer wires the exporter's stop guard — the .fab export path would be " +
|
||||
"the one uncovered stop-and-restart site, which is how a reader concludes the class is handled")
|
||||
}
|
||||
}
|
||||
|
||||
// TestMainWiresDesiredStateBackfill pins the Part-1.5 call.
|
||||
func TestMainWiresDesiredStateBackfill(t *testing.T) {
|
||||
if indexOfCall(callsInMain(t, mainBody(t)), "BackfillDesiredState") < 0 {
|
||||
t.Fatal("func main() no longer calls BackfillDesiredState — every existing app would stay on " +
|
||||
"legacy inference until someone pressed a button on it")
|
||||
}
|
||||
}
|
||||
|
||||
// TestAppStopRecoveryPrecedesTheBootReconciler is §8.4's ORDERING requirement, and it is the reason
|
||||
// the recovery returns its result instead of pushing it through a notifier seam.
|
||||
//
|
||||
// The recovery must COMPLETE — not merely be reached — before `go runBootReconcile(...)` is
|
||||
// launched. If the boot reconciler ran first it would see an app the marker already explains, list
|
||||
// it as an unexplained boot orphan, and one fault would be reported as two.
|
||||
func TestAppStopRecoveryPrecedesTheBootReconciler(t *testing.T) {
|
||||
names := callsInMain(t, mainBody(t))
|
||||
|
||||
recover := indexOfCall(names, "Recover")
|
||||
bootrecon := indexOfCall(names, "runBootReconcile")
|
||||
backfill := indexOfCall(names, "BackfillDesiredState")
|
||||
|
||||
if recover < 0 || bootrecon < 0 || backfill < 0 {
|
||||
t.Fatalf("missing a call: Recover=%d runBootReconcile=%d BackfillDesiredState=%d", recover, bootrecon, backfill)
|
||||
}
|
||||
if recover >= bootrecon {
|
||||
t.Fatal("the app-stop Recover no longer runs BEFORE the boot reconciler is launched — an app " +
|
||||
"the marker explains would also be reported as an unexplained boot orphan (§8.4)")
|
||||
}
|
||||
if backfill >= bootrecon {
|
||||
t.Fatal("the desired-state backfill no longer runs BEFORE the boot reconciler — the reconciler " +
|
||||
"would decide from intent the backfill had not yet written")
|
||||
}
|
||||
if recover >= backfill {
|
||||
t.Fatal("the backfill no longer runs AFTER the app-stop recovery — an app the recovery just " +
|
||||
"restarted would still read as down and be left unrecorded")
|
||||
}
|
||||
}
|
||||
|
||||
// TestMainReportsTheInterruptedOperation pins §2.4: the recovery's outcome reaches the operator.
|
||||
//
|
||||
// The reporting call is deliberately far from the recovery (the notifier does not exist yet at
|
||||
// recovery time), which is exactly the distance across which a wiring gets dropped.
|
||||
func TestMainReportsTheInterruptedOperation(t *testing.T) {
|
||||
body := mainBody(t)
|
||||
names := callsInMain(t, body)
|
||||
|
||||
if indexOfCall(names, "NotifyBackupFailed") < 0 {
|
||||
t.Fatal("func main() no longer reports an interrupted app-data operation to the operator — the " +
|
||||
"controller died mid-backup and nobody is told (§2.4)")
|
||||
}
|
||||
// It must be guarded, not unconditional: a box with nothing to recover must not email an operator
|
||||
// on every single boot.
|
||||
//
|
||||
// R-174 STRENGTHENED THIS. `!= nil` alone is no longer sufficient, because Recover now returns a
|
||||
// non-nil result for a recovery that merely REFUSED starts (an absent data drive) — the drive
|
||||
// gate working as designed. `NotifyBackupFailed` sends `backup_failed`, which is customer-enabled
|
||||
// by default (settings.DefaultEnabledEvents), so a nil-only guard would email the customer
|
||||
// "A biztonsági mentés sikertelen!" about an app nothing is wrong with. The guard must consult
|
||||
// Alarming().
|
||||
guardedByNil, guardedByAlarming := false, false
|
||||
ast.Inspect(body, func(n ast.Node) bool {
|
||||
ifst, ok := n.(*ast.IfStmt)
|
||||
if !ok || ifst.Cond == nil {
|
||||
return true
|
||||
}
|
||||
carries := false
|
||||
for _, name := range callsInMain(t, ifst.Body) {
|
||||
if name == "NotifyBackupFailed" {
|
||||
carries = true
|
||||
}
|
||||
}
|
||||
if !carries {
|
||||
return true
|
||||
}
|
||||
// Walk the whole condition: it may be `a != nil && a.Alarming()`.
|
||||
ast.Inspect(ifst.Cond, func(c ast.Node) bool {
|
||||
switch e := c.(type) {
|
||||
case *ast.BinaryExpr:
|
||||
if x, ok := e.X.(*ast.Ident); ok && x.Name == "appStopRecovery" && e.Op == token.NEQ {
|
||||
guardedByNil = true
|
||||
}
|
||||
case *ast.CallExpr:
|
||||
if sel, ok := e.Fun.(*ast.SelectorExpr); ok && sel.Sel.Name == "Alarming" {
|
||||
if x, ok := sel.X.(*ast.Ident); ok && x.Name == "appStopRecovery" {
|
||||
guardedByAlarming = true
|
||||
}
|
||||
}
|
||||
}
|
||||
return true
|
||||
})
|
||||
return true
|
||||
})
|
||||
if !guardedByNil {
|
||||
t.Fatal("the interrupted-operation alert is not guarded by `appStopRecovery != nil` — every " +
|
||||
"healthy boot would page the operator about a backup that was never interrupted")
|
||||
}
|
||||
if !guardedByAlarming {
|
||||
t.Fatal("the interrupted-operation alert is not guarded by appStopRecovery.Alarming() — a " +
|
||||
"recovery that only REFUSED starts (drive absent) would be reported through " +
|
||||
"NotifyBackupFailed, a customer-enabled event type, telling the customer their backup " +
|
||||
"failed when the drive gate was simply doing its job (R-174)")
|
||||
}
|
||||
}
|
||||
|
||||
// --- R-171 seam: the boot drive gate must be WIRED in production -------------------------------
|
||||
|
||||
// TestMainWiresBootDriveGate is the Group-H seam test. An unwired drive gate is not a crash — it is
|
||||
// SILENTLY the pre-v0.190.0 behaviour, which started apps onto absent drives (observed live,
|
||||
// audits/DIAG-bootrecon-drive-absent-2026-08-02.md). Every behavioural test in internal/bootrecon
|
||||
// still passes with the wiring gone, which is exactly the hole this walks the AST to close.
|
||||
//
|
||||
// AST, not strings.Contains: a commented-out call still contains the string — the distinction that
|
||||
// made a previous version of this project's own seam test pass its red-proof (2026-07-21).
|
||||
func TestMainWiresBootDriveGate(t *testing.T) {
|
||||
fset := token.NewFileSet()
|
||||
f, err := parser.ParseFile(fset, "main.go", nil, 0)
|
||||
if err != nil {
|
||||
t.Fatalf("parse main.go: %v", err)
|
||||
}
|
||||
|
||||
// (a) the settings handle the gate reads is assigned somewhere in main().
|
||||
assigned := false
|
||||
for _, name := range assignedIdentsIn(mainBody(t)) {
|
||||
if name == "bootDriveSettings" {
|
||||
assigned = true
|
||||
}
|
||||
}
|
||||
if !assigned {
|
||||
t.Fatal("func main() no longer assigns bootDriveSettings — the boot drive gate would read a " +
|
||||
"nil settings handle and could not see a disconnected drive")
|
||||
}
|
||||
|
||||
// (b) SetDriveGate is actually called where the reconciler is constructed.
|
||||
called := false
|
||||
ast.Inspect(f, func(n ast.Node) bool {
|
||||
call, ok := n.(*ast.CallExpr)
|
||||
if !ok {
|
||||
return true
|
||||
}
|
||||
if sel, ok := call.Fun.(*ast.SelectorExpr); ok && sel.Sel.Name == "SetDriveGate" {
|
||||
called = true
|
||||
}
|
||||
return true
|
||||
})
|
||||
if !called {
|
||||
t.Fatal("main.go no longer calls SetDriveGate on the boot reconciler — the sweep would start " +
|
||||
"apps whose data drive is absent (R-171, a regression observed live on 2026-08-02)")
|
||||
}
|
||||
}
|
||||
|
||||
// --- R-174 seam: the app-stop guard's starter must be GATED in production -----------------------
|
||||
|
||||
// TestMainWiresGatedAppStopStarter pins Part 0's production wiring. `SetStarter(stackMgr)` — the raw
|
||||
// manager, which is what shipped in v0.189.0 — compiles, passes every behavioural test in
|
||||
// internal/backup (they inject their own gating starter), and silently starts apps onto absent
|
||||
// drives at boot. The ONLY thing that distinguishes the fixed wiring from the broken one is the
|
||||
// argument at the call site, so that is what this reads.
|
||||
//
|
||||
// AST, not strings.Contains: a commented-out call still contains the string.
|
||||
func TestMainWiresGatedAppStopStarter(t *testing.T) {
|
||||
body := mainBody(t)
|
||||
|
||||
var arg ast.Expr
|
||||
found := false
|
||||
ast.Inspect(body, func(n ast.Node) bool {
|
||||
call, ok := n.(*ast.CallExpr)
|
||||
if !ok {
|
||||
return true
|
||||
}
|
||||
sel, ok := call.Fun.(*ast.SelectorExpr)
|
||||
if !ok || sel.Sel.Name != "SetStarter" || len(call.Args) != 1 {
|
||||
return true
|
||||
}
|
||||
// Only the app-stop guard's SetStarter, not some other type's.
|
||||
if x, ok := sel.X.(*ast.Ident); !ok || x.Name != "appStopGuard" {
|
||||
return true
|
||||
}
|
||||
arg, found = call.Args[0], true
|
||||
return false
|
||||
})
|
||||
if !found {
|
||||
t.Fatal("func main() no longer calls appStopGuard.SetStarter — Recover would find the marker " +
|
||||
"and be unable to start anything")
|
||||
}
|
||||
|
||||
// The argument must be a gatedAppStopStarter composite literal. A bare identifier (`stackMgr`)
|
||||
// is precisely the v0.189.0 defect.
|
||||
lit, ok := arg.(*ast.CompositeLit)
|
||||
if !ok {
|
||||
t.Fatalf("appStopGuard.SetStarter is wired with %T, not a gatedAppStopStarter literal — an "+
|
||||
"un-gated starter restarts apps onto MISSING drives at boot (R-174, the R-171 defect one "+
|
||||
"path over)", arg)
|
||||
}
|
||||
id, ok := lit.Type.(*ast.Ident)
|
||||
if !ok || id.Name != "gatedAppStopStarter" {
|
||||
t.Fatalf("appStopGuard.SetStarter is wired with a %v literal, want gatedAppStopStarter", lit.Type)
|
||||
}
|
||||
|
||||
// And that gate must be a driveStartGate — the SAME predicate the boot sweep uses, so the two
|
||||
// cannot disagree about whether an app's drive is available.
|
||||
gated := false
|
||||
for _, el := range lit.Elts {
|
||||
kv, ok := el.(*ast.KeyValueExpr)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
k, ok := kv.Key.(*ast.Ident)
|
||||
if !ok || k.Name != "gate" {
|
||||
continue
|
||||
}
|
||||
if gl, ok := kv.Value.(*ast.CompositeLit); ok {
|
||||
if gid, ok := gl.Type.(*ast.Ident); ok && gid.Name == "driveStartGate" {
|
||||
gated = true
|
||||
}
|
||||
}
|
||||
}
|
||||
if !gated {
|
||||
t.Fatal("the app-stop starter's gate is not a driveStartGate — the crash recovery and the " +
|
||||
"boot sweep would answer \"may this app start?\" from two different implementations, " +
|
||||
"which is the drift the extraction exists to prevent")
|
||||
}
|
||||
}
|
||||
|
||||
// TestBootDriveGateAndAppStopShareTheDrivePredicate pins the OTHER half of the same claim: the boot
|
||||
// sweep must keep delegating to driveStartGate rather than growing its own copy of the drive checks.
|
||||
//
|
||||
// This is the "a comment asserting an invariant needs a test pinning it" rule. The claim — that the
|
||||
// two gates cannot disagree — is true only while both call the same code.
|
||||
func TestBootDriveGateAndAppStopShareTheDrivePredicate(t *testing.T) {
|
||||
fset := token.NewFileSet()
|
||||
f, err := parser.ParseFile(fset, "main.go", nil, 0)
|
||||
if err != nil {
|
||||
t.Fatalf("parse main.go: %v", err)
|
||||
}
|
||||
|
||||
var mayStart *ast.FuncDecl
|
||||
for _, decl := range f.Decls {
|
||||
fn, ok := decl.(*ast.FuncDecl)
|
||||
if !ok || fn.Name.Name != "MayStart" || fn.Recv == nil || len(fn.Recv.List) != 1 {
|
||||
continue
|
||||
}
|
||||
if id, ok := fn.Recv.List[0].Type.(*ast.Ident); ok && id.Name == "bootDriveGate" {
|
||||
mayStart = fn
|
||||
}
|
||||
}
|
||||
if mayStart == nil {
|
||||
t.Fatal("bootDriveGate.MayStart not found in main.go")
|
||||
}
|
||||
|
||||
// It must call through to the shared predicate.
|
||||
delegates := false
|
||||
ast.Inspect(mayStart.Body, func(n ast.Node) bool {
|
||||
call, ok := n.(*ast.CallExpr)
|
||||
if !ok {
|
||||
return true
|
||||
}
|
||||
sel, ok := call.Fun.(*ast.SelectorExpr)
|
||||
if !ok || sel.Sel.Name != "MayStart" {
|
||||
return true
|
||||
}
|
||||
if x, ok := sel.X.(*ast.SelectorExpr); ok && x.Sel.Name == "drive" {
|
||||
delegates = true
|
||||
}
|
||||
return true
|
||||
})
|
||||
if !delegates {
|
||||
t.Fatal("bootDriveGate.MayStart no longer delegates to the shared driveStartGate — the boot " +
|
||||
"sweep and the app-stop crash recovery would each carry their own drive logic, and the " +
|
||||
"two can then disagree about whether an app may start (R-174)")
|
||||
}
|
||||
}
|
||||
|
||||
// --- R-158 / R-167 seams: both new alerts must be WIRED in production ---------------------------
|
||||
|
||||
// TestMainWiresTheUnitCaptureAlert pins Part 1's seam. `SetUnitNotify` is nil-safe by design, so an
|
||||
// unwired seam is not a crash — it is SILENTLY the pre-v0.191.0 behaviour, in which a per-app Tier-1
|
||||
// capture failure is a `[WARN]` line and reaches no hub channel at all. Every behavioural test in
|
||||
// internal/backup injects its own callback and passes with the production wiring gone, which is
|
||||
// exactly the hole this closes. THIS PROJECT'S COUNT OF "BUILT BUT NEVER WIRED" REACHES FIVE WITH
|
||||
// R-158 — the defect being fixed here IS an instance of it.
|
||||
func TestMainWiresTheUnitCaptureAlert(t *testing.T) {
|
||||
names := callsInMain(t, mainBody(t))
|
||||
|
||||
if indexOfCall(names, "SetUnitNotify") < 0 {
|
||||
t.Fatal("func main() no longer calls backupMgr.SetUnitNotify — a per-app recovery-unit " +
|
||||
"capture failure would reach no hub channel, which is R-158 un-fixed (the seam built " +
|
||||
"and left disconnected, for the fifth time in this project)")
|
||||
}
|
||||
if indexOfCall(names, "NotifyRecoveryUnitCaptureFailed") < 0 {
|
||||
t.Fatal("main.go no longer calls NotifyRecoveryUnitCaptureFailed — the seam is wired to " +
|
||||
"something that pushes no event, which looks identical to a working alert from inside " +
|
||||
"internal/backup")
|
||||
}
|
||||
}
|
||||
|
||||
// TestMainWiresTheFillWatcher pins Part 2's seam. Three separate things can be dropped and each one
|
||||
// silently reverts the customer to "nothing warns before a disk fills": the watcher can go
|
||||
// unconstructed, its notify can go unwired (the Watcher is nil-safe), or it can never be scheduled.
|
||||
func TestMainWiresTheFillWatcher(t *testing.T) {
|
||||
body := mainBody(t)
|
||||
names := callsInMain(t, body)
|
||||
|
||||
if indexOfCall(names, "New") < 0 || !assignsIdent(body, "fillWatcher") {
|
||||
t.Fatal("func main() no longer constructs the fill watcher — nothing warns the customer " +
|
||||
"before a filesystem fills (R-167, decision D-c's customer half)")
|
||||
}
|
||||
if indexOfCall(names, "SetNotify") < 0 {
|
||||
t.Fatal("func main() no longer calls SetNotify on the fill watcher — the Watcher is nil-safe, " +
|
||||
"so it would run the checks, update its state, log, and tell the CUSTOMER nothing")
|
||||
}
|
||||
|
||||
// It must actually be scheduled: a watcher nobody calls is a watcher that never fires.
|
||||
scheduled := false
|
||||
ast.Inspect(body, func(n ast.Node) bool {
|
||||
call, ok := n.(*ast.CallExpr)
|
||||
if !ok || len(call.Args) == 0 {
|
||||
return true
|
||||
}
|
||||
sel, ok := call.Fun.(*ast.SelectorExpr)
|
||||
if !ok || (sel.Sel.Name != "Daily" && sel.Sel.Name != "Every") {
|
||||
return true
|
||||
}
|
||||
lit, ok := call.Args[0].(*ast.BasicLit)
|
||||
if ok && strings.Contains(lit.Value, "fill-watch") {
|
||||
scheduled = true
|
||||
}
|
||||
return true
|
||||
})
|
||||
if !scheduled {
|
||||
t.Fatal("the fill watcher is never registered on the scheduler — it would be constructed, " +
|
||||
"wired, and never run, which is indistinguishable from a filesystem that never fills")
|
||||
}
|
||||
|
||||
// It must ALSO run once at startup. Neither `Every` nor `Daily` fires on registration (both wait
|
||||
// for their first tick), so a schedule-only wiring means a box that BOOTS with a filesystem
|
||||
// already over the line stays silent for up to 24 hours — a real fault visible only after a
|
||||
// deadline elapses, which is the R-100 shape. The hub's own checkers leave already-breached keys
|
||||
// unseeded at init for exactly this reason.
|
||||
if indexOfCall(names, "After") < 0 {
|
||||
t.Fatal("nothing delays a startup fill check — see fillWatchStartupDelay")
|
||||
}
|
||||
startupRun := false
|
||||
ast.Inspect(body, func(n ast.Node) bool {
|
||||
g, ok := n.(*ast.GoStmt)
|
||||
if !ok || g.Call == nil {
|
||||
return true
|
||||
}
|
||||
lit, ok := g.Call.Fun.(*ast.FuncLit)
|
||||
if !ok {
|
||||
return true
|
||||
}
|
||||
var sawDelay, sawCheck bool
|
||||
ast.Inspect(lit.Body, func(m ast.Node) bool {
|
||||
if id, ok := m.(*ast.Ident); ok && id.Name == "fillWatchStartupDelay" {
|
||||
sawDelay = true
|
||||
}
|
||||
if call, ok := m.(*ast.CallExpr); ok {
|
||||
if sel, ok := call.Fun.(*ast.SelectorExpr); ok && sel.Sel.Name == "Check" {
|
||||
if x, ok := sel.X.(*ast.Ident); ok && x.Name == "fillWatcher" {
|
||||
sawCheck = true
|
||||
}
|
||||
}
|
||||
}
|
||||
return true
|
||||
})
|
||||
if sawDelay && sawCheck {
|
||||
startupRun = true
|
||||
}
|
||||
return true
|
||||
})
|
||||
if !startupRun {
|
||||
t.Fatal("the fill watcher never runs at STARTUP — Daily/Every both wait for their first " +
|
||||
"tick, so a box that boots with a full disk would not warn for up to 24 hours (the " +
|
||||
"R-100 shape: a real fault visible only after a deadline elapses)")
|
||||
}
|
||||
}
|
||||
|
||||
// assignsIdent reports whether a block assigns to the named identifier.
|
||||
func assignsIdent(body *ast.BlockStmt, want string) bool {
|
||||
for _, n := range assignedIdentsIn(body) {
|
||||
if n == want {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// assignedIdentsIn returns the names assigned to in a block (plain `=` and `:=`).
|
||||
func assignedIdentsIn(body *ast.BlockStmt) []string {
|
||||
var names []string
|
||||
ast.Inspect(body, func(n ast.Node) bool {
|
||||
as, ok := n.(*ast.AssignStmt)
|
||||
if !ok {
|
||||
return true
|
||||
}
|
||||
for _, lhs := range as.Lhs {
|
||||
if id, ok := lhs.(*ast.Ident); ok {
|
||||
names = append(names, id.Name)
|
||||
}
|
||||
}
|
||||
return true
|
||||
})
|
||||
return names
|
||||
}
|
||||
@@ -1,122 +0,0 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"go/ast"
|
||||
"go/parser"
|
||||
"go/token"
|
||||
"io"
|
||||
"log"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/bootrecon"
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/stacks"
|
||||
)
|
||||
|
||||
// §9 rule 6 — the seam-discipline test. Two inert-seam defects shipped in the two days before this
|
||||
// task (controller v0.154.0 and agent v0.91.0), both the same shape: the component was correct, its
|
||||
// unit tests injected the seam directly, and the PRODUCTION CALLER was never made. Everything was
|
||||
// green and the feature did nothing. So R-52 gets its wiring asserted from package main, not only
|
||||
// from internal/bootrecon.
|
||||
|
||||
// TestRunBootReconcile_InvokesTheSweep pins the function main() actually calls: after the settle
|
||||
// window it runs the sweep exactly once, with the manager it was handed.
|
||||
func TestRunBootReconcile_InvokesTheSweep(t *testing.T) {
|
||||
orig := bootReconcileFn
|
||||
t.Cleanup(func() { bootReconcileFn = orig })
|
||||
origSettle := bootReconcileSettle
|
||||
t.Cleanup(func() { bootReconcileSettle = origSettle })
|
||||
bootReconcileSettle = time.Millisecond
|
||||
|
||||
calls := 0
|
||||
var gotMgr bootrecon.StackProvider
|
||||
bootReconcileFn = func(_ context.Context, mgr bootrecon.StackProvider, _ *log.Logger) bootrecon.Result {
|
||||
calls++
|
||||
gotMgr = mgr
|
||||
return bootrecon.Result{}
|
||||
}
|
||||
|
||||
fake := &wiringStacks{}
|
||||
runBootReconcile(context.Background(), fake, log.New(io.Discard, "", 0))
|
||||
|
||||
if calls != 1 {
|
||||
t.Fatalf("the boot sweep ran %d times, want exactly 1 (start-once, never a loop)", calls)
|
||||
}
|
||||
if gotMgr != bootrecon.StackProvider(fake) {
|
||||
t.Fatalf("the sweep was handed %v, want the stack manager main() owns", gotMgr)
|
||||
}
|
||||
}
|
||||
|
||||
// A controller shutting down during its own settle window must not start anything.
|
||||
func TestRunBootReconcile_CancelledDuringSettleDoesNothing(t *testing.T) {
|
||||
orig := bootReconcileFn
|
||||
t.Cleanup(func() { bootReconcileFn = orig })
|
||||
|
||||
calls := 0
|
||||
bootReconcileFn = func(context.Context, bootrecon.StackProvider, *log.Logger) bootrecon.Result {
|
||||
calls++
|
||||
return bootrecon.Result{}
|
||||
}
|
||||
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
cancel()
|
||||
runBootReconcile(ctx, &wiringStacks{}, log.New(io.Discard, "", 0))
|
||||
|
||||
if calls != 0 {
|
||||
t.Fatalf("the sweep ran %d times on a cancelled context, want 0", calls)
|
||||
}
|
||||
}
|
||||
|
||||
// The call site itself. A function-variable test can only prove the function is correct — it cannot
|
||||
// prove main() calls it, which is exactly the hole both inert-seam defects fell through. This walks
|
||||
// main.go's AST for a `go runBootReconcile(...)` inside func main(); delete or comment out that line
|
||||
// and this fails, where every behavioural test above would still pass.
|
||||
//
|
||||
// It is an AST walk and not a strings.Contains for a reason found while red-proofing it: a
|
||||
// commented-out call still satisfies a substring match, so the text version passed the very
|
||||
// red-proof it existed to fail. Comments are not code.
|
||||
func TestMainWiresBootReconcile(t *testing.T) {
|
||||
fset := token.NewFileSet()
|
||||
f, err := parser.ParseFile(fset, "main.go", nil, 0)
|
||||
if err != nil {
|
||||
t.Fatalf("parse main.go: %v", err)
|
||||
}
|
||||
|
||||
found := false
|
||||
for _, decl := range f.Decls {
|
||||
fn, ok := decl.(*ast.FuncDecl)
|
||||
if !ok || fn.Name.Name != "main" || fn.Body == nil {
|
||||
continue
|
||||
}
|
||||
ast.Inspect(fn.Body, func(n ast.Node) bool {
|
||||
gostmt, ok := n.(*ast.GoStmt)
|
||||
if !ok {
|
||||
return true
|
||||
}
|
||||
if ident, ok := gostmt.Call.Fun.(*ast.Ident); ok && ident.Name == "runBootReconcile" {
|
||||
found = true
|
||||
}
|
||||
return true
|
||||
})
|
||||
}
|
||||
if !found {
|
||||
t.Fatal("func main() no longer starts the R-52 boot reconciliation with `go runBootReconcile(...)` " +
|
||||
"— the sweep is inert (the v0.154.0 / v0.91.0 defect class: a correct component nobody calls)")
|
||||
}
|
||||
}
|
||||
|
||||
// The settle window must stay inside the dead-app boot grace, or a successful recovery would alert.
|
||||
func TestBootReconcileFitsInsideTheBootGrace(t *testing.T) {
|
||||
worst := bootReconcileSettle + time.Duration(bootrecon.DefaultAttempts-1)*bootrecon.DefaultRetryDelay
|
||||
if worst >= deadAppBootGrace {
|
||||
t.Fatalf("worst-case sweep %s does not fit inside the %s boot grace — a successful "+
|
||||
"recovery would fire app_start_failed", worst, deadAppBootGrace)
|
||||
}
|
||||
}
|
||||
|
||||
type wiringStacks struct{}
|
||||
|
||||
func (w *wiringStacks) GetStacks() []stacks.Stack { return nil }
|
||||
func (w *wiringStacks) StartStack(string) error { return nil }
|
||||
func (w *wiringStacks) RefreshStatus() error { return nil }
|
||||
@@ -1,366 +0,0 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"io"
|
||||
"log"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/bootrecon"
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/stacks"
|
||||
)
|
||||
|
||||
// R-157 mechanism A — the sweep that looked once.
|
||||
//
|
||||
// TIMING IS NOT TESTED BY SLEEPING (§10). The window's constants are package vars, so each test
|
||||
// shrinks them to sub-millisecond values: the CONTRACT under test is "how many samples, and what
|
||||
// ends the window", not "how long a second is". A test that waited real seconds would be slow,
|
||||
// flaky, and would still not prove the contract.
|
||||
|
||||
// windowStacks is a StackProvider whose fleet CHANGES over successive GetStacks() calls — which is
|
||||
// the whole point: the pre-v0.190.0 sweep sampled once and could not see a late settler.
|
||||
type windowStacks struct {
|
||||
// frames is the fleet as seen on each successive GetStacks() call; the last frame repeats.
|
||||
frames [][]stacks.Stack
|
||||
calls int
|
||||
starts map[string]int
|
||||
onStart func(*windowStacks, string)
|
||||
refreshes int
|
||||
refreshErr error
|
||||
// cycle makes the fleet NEVER settle: frames repeat forever instead of the last one sticking.
|
||||
// Required by the budget test — with frames that eventually stop changing, the window terminates
|
||||
// by SETTLING even with the budget removed, so the red-proof would not reach the hang it exists
|
||||
// to demonstrate.
|
||||
cycle bool
|
||||
}
|
||||
|
||||
func (w *windowStacks) GetStacks() []stacks.Stack {
|
||||
i := w.calls
|
||||
w.calls++
|
||||
if i >= len(w.frames) {
|
||||
if w.cycle {
|
||||
i = i % len(w.frames)
|
||||
} else {
|
||||
i = len(w.frames) - 1
|
||||
}
|
||||
}
|
||||
return w.frames[i]
|
||||
}
|
||||
|
||||
func (w *windowStacks) RefreshStatus() error {
|
||||
w.refreshes++
|
||||
if w.refreshErr != nil {
|
||||
return w.refreshErr
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (w *windowStacks) StartStack(name string) error {
|
||||
if w.starts == nil {
|
||||
w.starts = map[string]int{}
|
||||
}
|
||||
w.starts[name]++
|
||||
if w.onStart != nil {
|
||||
w.onStart(w, name)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// shrinkWindow makes the window fast and deterministic, and restores the shipped values after.
|
||||
func shrinkWindow(t *testing.T, sample time.Duration, stableFor int, budget time.Duration) {
|
||||
t.Helper()
|
||||
os, ost, ob, osettle := bootReconcileSample, bootReconcileStableFor, bootReconcileBudget, bootReconcileSettle
|
||||
t.Cleanup(func() {
|
||||
bootReconcileSample, bootReconcileStableFor, bootReconcileBudget, bootReconcileSettle = os, ost, ob, osettle
|
||||
})
|
||||
bootReconcileSample, bootReconcileStableFor, bootReconcileBudget = sample, stableFor, budget
|
||||
bootReconcileSettle = time.Millisecond
|
||||
}
|
||||
|
||||
// captureSweep replaces the sweep with a recorder and returns the fleet it was handed.
|
||||
func captureSweep(t *testing.T) *[][]stacks.Stack {
|
||||
t.Helper()
|
||||
orig := bootReconcileFn
|
||||
t.Cleanup(func() { bootReconcileFn = orig })
|
||||
var seen [][]stacks.Stack
|
||||
bootReconcileFn = func(_ context.Context, mgr bootrecon.StackProvider, _ *log.Logger) bootrecon.Result {
|
||||
seen = append(seen, mgr.GetStacks())
|
||||
return bootrecon.Result{}
|
||||
}
|
||||
return &seen
|
||||
}
|
||||
|
||||
func upStack(name string) stacks.Stack {
|
||||
return stacks.Stack{
|
||||
Name: name, Deployed: true, State: stacks.StateRunning,
|
||||
Containers: []stacks.ContainerInfo{{Name: name, State: stacks.StateRunning}},
|
||||
AppConfig: &stacks.AppConfig{Deployed: true, DesiredState: stacks.DesiredStateRunning},
|
||||
}
|
||||
}
|
||||
|
||||
// settlingLate is the R-157-A shape: at T+5s the app is still `starting` with its containers coming
|
||||
// up, and it only comes to rest in a DOWN state later.
|
||||
func settlingLate(name string) stacks.Stack {
|
||||
return stacks.Stack{
|
||||
Name: name, Deployed: true, State: stacks.StateStarting,
|
||||
Containers: []stacks.ContainerInfo{{Name: name, State: stacks.StateStarting}},
|
||||
AppConfig: &stacks.AppConfig{Deployed: true, DesiredState: stacks.DesiredStateRunning},
|
||||
}
|
||||
}
|
||||
|
||||
func settledDown(name string) stacks.Stack {
|
||||
return stacks.Stack{
|
||||
Name: name, Deployed: true, State: stacks.StateExited,
|
||||
Containers: []stacks.ContainerInfo{{Name: name, State: stacks.StateExited}},
|
||||
AppConfig: &stacks.AppConfig{Deployed: true, DesiredState: stacks.DesiredStateRunning},
|
||||
}
|
||||
}
|
||||
|
||||
// --- Group A / Scenario B — a late settler IS swept -----------------------------------------------
|
||||
|
||||
func TestBootWindow_LateSettlerIsSweptOnASettledFleet(t *testing.T) {
|
||||
// The fleet is still moving for the first frames and settles only later. The sweep must run
|
||||
// AFTER it settles and must be handed the SETTLED fleet — because the pre-v0.190.0 defect was a
|
||||
// candidate set derived from a fleet that had not finished moving.
|
||||
//
|
||||
// RED-PROOF: restore the single-sweep shape (delete the sampling loop so runBootReconcile calls
|
||||
// bootReconcileFn straight after the settle delay) and this test fails — the sweep is handed the
|
||||
// `starting` frame, in which the app is not a down-state candidate at all.
|
||||
// Demonstrated in REPORT.md §4.
|
||||
shrinkWindow(t, time.Millisecond, 2, 500*time.Millisecond)
|
||||
seen := captureSweep(t)
|
||||
|
||||
w := &windowStacks{frames: [][]stacks.Stack{
|
||||
{settlingLate("immich")}, // T+5s: still coming up
|
||||
{settlingLate("immich")},
|
||||
{settledDown("immich")}, // settles into a down state only now
|
||||
{settledDown("immich")},
|
||||
{settledDown("immich")},
|
||||
}}
|
||||
|
||||
runBootReconcile(context.Background(), w, log.New(io.Discard, "", 0))
|
||||
|
||||
if len(*seen) != 1 {
|
||||
t.Fatalf("the sweep ran %d times, want exactly 1 — the window samples, it does not sweep per sample", len(*seen))
|
||||
}
|
||||
got := (*seen)[0]
|
||||
if len(got) != 1 || got[0].State != stacks.StateExited {
|
||||
t.Fatalf("the sweep was handed state=%v, want the SETTLED (exited) fleet — a candidate set "+
|
||||
"derived from a still-moving fleet is exactly the R-157 mechanism-A defect", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBootWindow_SweepRunsExactlyOnceEvenOnAQuietBoot(t *testing.T) {
|
||||
shrinkWindow(t, time.Millisecond, 2, 500*time.Millisecond)
|
||||
seen := captureSweep(t)
|
||||
w := &windowStacks{frames: [][]stacks.Stack{{upStack("bookstack")}}}
|
||||
|
||||
runBootReconcile(context.Background(), w, log.New(io.Discard, "", 0))
|
||||
|
||||
if len(*seen) != 1 {
|
||||
t.Fatalf("sweeps=%d, want exactly 1 on a quiet boot", len(*seen))
|
||||
}
|
||||
}
|
||||
|
||||
// --- Group B / Scenario C — the window TERMINATES -------------------------------------------------
|
||||
|
||||
func TestBootWindow_BudgetEndsAForeverChangingFleet(t *testing.T) {
|
||||
// A fleet that never stops changing must not sample forever. The budget ends it, the sweep runs
|
||||
// once anyway (a churning box is exactly the box that needs it), and the log SAYS the budget
|
||||
// ended it — "settled and found nothing" and "ran out of time" are different facts.
|
||||
//
|
||||
// RED-PROOF: remove the `time.Since(started) < bootReconcileBudget` loop condition and this test
|
||||
// hangs — the unbounded-loop shape §5 bans. Demonstrated in REPORT.md §4 (observed as a timeout).
|
||||
shrinkWindow(t, time.Millisecond, 3, 30*time.Millisecond)
|
||||
seen := captureSweep(t)
|
||||
var buf strings.Builder
|
||||
|
||||
// Every frame differs, so `stable` can never reach stableFor.
|
||||
frames := make([][]stacks.Stack, 0, 200)
|
||||
for i := 0; i < 200; i++ {
|
||||
s := upStack("immich")
|
||||
s.Containers = make([]stacks.ContainerInfo, i%7) // container count changes every sample
|
||||
frames = append(frames, []stacks.Stack{s})
|
||||
}
|
||||
w := &windowStacks{frames: frames, cycle: true}
|
||||
|
||||
done := make(chan struct{})
|
||||
go func() {
|
||||
runBootReconcile(context.Background(), w, log.New(&buf, "", 0))
|
||||
close(done)
|
||||
}()
|
||||
select {
|
||||
case <-done:
|
||||
case <-time.After(5 * time.Second):
|
||||
t.Fatal("runBootReconcile did not terminate on a forever-changing fleet — this is the " +
|
||||
"unbounded restart-loop shape the package's own boundary forbids")
|
||||
}
|
||||
|
||||
if len(*seen) != 1 {
|
||||
t.Fatalf("sweeps=%d, want exactly 1 after the budget expired", len(*seen))
|
||||
}
|
||||
if out := buf.String(); !strings.Contains(out, "budget") {
|
||||
t.Fatalf("the log does not say the BUDGET ended the window, so a churning boot reads like a "+
|
||||
"quiet one:\n%s", out)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBootWindow_SettledPathSaysSettled(t *testing.T) {
|
||||
shrinkWindow(t, time.Millisecond, 2, 500*time.Millisecond)
|
||||
captureSweep(t)
|
||||
var buf strings.Builder
|
||||
w := &windowStacks{frames: [][]stacks.Stack{{upStack("docmost")}}}
|
||||
|
||||
runBootReconcile(context.Background(), w, log.New(&buf, "", 0))
|
||||
|
||||
out := buf.String()
|
||||
if !strings.Contains(out, "settled") {
|
||||
t.Fatalf("a settled window must say so — otherwise it is indistinguishable from a budget "+
|
||||
"expiry:\n%s", out)
|
||||
}
|
||||
if strings.Contains(out, "budget") {
|
||||
t.Fatalf("a settled window must NOT claim the budget ended it:\n%s", out)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBootWindow_CancelledContextStopsImmediately(t *testing.T) {
|
||||
shrinkWindow(t, time.Millisecond, 3, time.Second)
|
||||
seen := captureSweep(t)
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
cancel()
|
||||
runBootReconcile(ctx, &windowStacks{frames: [][]stacks.Stack{{upStack("x")}}}, log.New(io.Discard, "", 0))
|
||||
if len(*seen) != 0 {
|
||||
t.Fatalf("the sweep ran %d times on a cancelled context, want 0", len(*seen))
|
||||
}
|
||||
}
|
||||
|
||||
// --- Group C / Scenario D — a customer's Stop survives the WIDENED window -------------------------
|
||||
|
||||
func TestBootWindow_CustomerStoppedAppSurvivesEveryPass(t *testing.T) {
|
||||
// THE REGRESSION THIS TASK COULD INTRODUCE. A longer window means more chances to resurrect an
|
||||
// app the customer deliberately stopped. It must survive the whole window — this drives the REAL
|
||||
// bootrecon sweep (not the captured stub), so the desired-state check is genuinely exercised.
|
||||
//
|
||||
// RED-PROOF: drop the DesiredStateStopped branch from isBootOrphan (make it fall through to the
|
||||
// running case) and this test fails with a start count of 1. Demonstrated in REPORT.md §4.
|
||||
shrinkWindow(t, time.Millisecond, 2, 200*time.Millisecond)
|
||||
|
||||
stopped := stacks.Stack{
|
||||
Name: "nextcloud", Deployed: true, State: stacks.StateStopped, Containers: nil,
|
||||
AppConfig: &stacks.AppConfig{Deployed: true, DesiredState: stacks.DesiredStateStopped},
|
||||
}
|
||||
// The fleet churns around it, so the window runs many passes before settling.
|
||||
frames := [][]stacks.Stack{
|
||||
{stopped, settlingLate("immich")},
|
||||
{stopped, settlingLate("immich")},
|
||||
{stopped, settledDown("immich")},
|
||||
{stopped, upStack("immich")},
|
||||
{stopped, upStack("immich")},
|
||||
{stopped, upStack("immich")},
|
||||
}
|
||||
w := &windowStacks{frames: frames, onStart: func(w *windowStacks, _ string) {}}
|
||||
|
||||
runBootReconcile(context.Background(), w, log.New(io.Discard, "", 0))
|
||||
|
||||
if n := w.starts["nextcloud"]; n != 0 {
|
||||
t.Fatalf("the customer-stopped app was started %d time(s) by the widened window — this is the "+
|
||||
"regression a longer window makes possible and it is the worst outcome available here", n)
|
||||
}
|
||||
}
|
||||
|
||||
// --- §8.3 — a late recovery is REPORTED, never hidden ---------------------------------------------
|
||||
|
||||
func TestRecordLateRecovery_WarnsWhenTheGraceHasAlreadyExpired(t *testing.T) {
|
||||
var buf strings.Builder
|
||||
lg := log.New(&buf, "", 0)
|
||||
// started far enough back that settle + elapsed exceeds the 90 s grace
|
||||
recordLateRecovery(lg, time.Now().Add(-(deadAppBootGrace + 10*time.Second)), bootrecon.Result{Recovered: []string{"immich"}})
|
||||
out := buf.String()
|
||||
if !strings.Contains(out, "LATE RECOVERY") || !strings.Contains(out, "immich") {
|
||||
t.Fatalf("a recovery past the dead-app grace must be reported by name — otherwise a stale "+
|
||||
"alarm stands with no counter-evidence (§8.3):\n%s", out)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRecordLateRecovery_SilentInsideTheGrace(t *testing.T) {
|
||||
var buf strings.Builder
|
||||
recordLateRecovery(log.New(&buf, "", 0), time.Now(), bootrecon.Result{Recovered: []string{"immich"}})
|
||||
if buf.Len() != 0 {
|
||||
t.Fatalf("a recovery INSIDE the grace must stay silent — that is what makes a successful "+
|
||||
"recovery invisible to the customer:\n%s", buf.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestRecordLateRecovery_SilentWhenNothingRecovered(t *testing.T) {
|
||||
var buf strings.Builder
|
||||
recordLateRecovery(log.New(&buf, "", 0), time.Now().Add(-time.Hour), bootrecon.Result{})
|
||||
if buf.Len() != 0 {
|
||||
t.Fatalf("nothing was recovered, so there is nothing late to report:\n%s", buf.String())
|
||||
}
|
||||
}
|
||||
|
||||
// --- The window's constants must fit the grace they are justified against -------------------------
|
||||
|
||||
func TestBootWindow_CommonCaseFitsInsideTheDeadAppGrace(t *testing.T) {
|
||||
// The comment on the window constants justifies them against deadAppBootGrace. A comment
|
||||
// asserting an invariant needs a test pinning it, or it is a wish.
|
||||
common := bootReconcileSettle + bootReconcileBudget + bootrecon.DefaultRetryDelay
|
||||
if common > deadAppBootGrace {
|
||||
t.Fatalf("settle(%s) + budget(%s) + one retry(%s) = %s exceeds the %s dead-app grace — the "+
|
||||
"COMMON case must stay silent, or every slow boot alerts",
|
||||
bootReconcileSettle, bootReconcileBudget, bootrecon.DefaultRetryDelay, common, deadAppBootGrace)
|
||||
}
|
||||
if bootReconcileSample <= 0 || bootReconcileStableFor < 2 {
|
||||
t.Fatalf("sample=%s stableFor=%d — one sample cannot distinguish 'settled' from 'sampled "+
|
||||
"between two docker events'", bootReconcileSample, bootReconcileStableFor)
|
||||
}
|
||||
}
|
||||
|
||||
// --- the sample must observe REALITY, not the Manager's cache ------------------------------------
|
||||
|
||||
func TestBootWindow_EverySampleRefreshesTheStatus(t *testing.T) {
|
||||
// FOUND BY LIVE VALIDATION, not review. GetStacks() returns the Manager's in-memory map, which
|
||||
// the scheduler refreshes on its own 10 s cadence. Sampling every 5 s WITHOUT refreshing means two
|
||||
// consecutive samples can be identical because the cache did not update — so the window declares
|
||||
// "settled" on stale data and sweeps on a picture of the box from up to 10 s ago. On 9201 a
|
||||
// container removed ~5 s before the window closed was still in the sampled fleet, and the sweep
|
||||
// logged "no boot-orphaned apps" for an app that had none.
|
||||
//
|
||||
// RED-PROOF: delete the `_ = mgr.RefreshStatus()` line from sampleBootFleet and this test fails
|
||||
// with refreshes=0. Demonstrated in REPORT.md §4.
|
||||
shrinkWindow(t, time.Millisecond, 3, 500*time.Millisecond)
|
||||
captureSweep(t)
|
||||
w := &windowStacks{frames: [][]stacks.Stack{{upStack("immich")}}}
|
||||
|
||||
runBootReconcile(context.Background(), w, log.New(io.Discard, "", 0))
|
||||
|
||||
if w.refreshes < 3 {
|
||||
t.Fatalf("the window refreshed %d time(s) for %d samples — every sample must observe reality, "+
|
||||
"or 'settled' can mean 'the cache did not update'", w.refreshes, w.calls)
|
||||
}
|
||||
// calls includes ONE extra GetStacks from the captured sweep itself, which does not sample.
|
||||
if w.refreshes != w.calls-1 {
|
||||
t.Fatalf("refreshes=%d but samples=%d — each sample must refresh exactly once before reading",
|
||||
w.refreshes, w.calls-1)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBootWindow_RefreshErrorDoesNotStopTheWindow(t *testing.T) {
|
||||
// A boot window that cannot reach docker is exactly when a stale verdict is most dangerous, but
|
||||
// giving up entirely would leave the sweep un-run. Degrade, do not abort.
|
||||
shrinkWindow(t, time.Millisecond, 2, 200*time.Millisecond)
|
||||
seen := captureSweep(t)
|
||||
w := &windowStacks{frames: [][]stacks.Stack{{upStack("immich")}}, refreshErr: errRefresh{}}
|
||||
|
||||
runBootReconcile(context.Background(), w, log.New(io.Discard, "", 0))
|
||||
|
||||
if len(*seen) != 1 {
|
||||
t.Fatalf("sweeps=%d, want 1 — a refresh error must not abort the window", len(*seen))
|
||||
}
|
||||
}
|
||||
|
||||
type errRefresh struct{}
|
||||
|
||||
func (errRefresh) Error() string { return "docker unreachable" }
|
||||
@@ -1,125 +0,0 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"time"
|
||||
|
||||
"testing"
|
||||
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/notify"
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/stacks"
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/web"
|
||||
)
|
||||
|
||||
// v0.164.0: classifyRunStates is the single fix-3 derivation point. A deliberate user stop
|
||||
// (StateStopped) must NOT alarm — it is excluded from both the banner dead-list and the notifier
|
||||
// Down-set — while every genuine fault (StateExited / StateDegraded) keeps alerting byte-identically.
|
||||
// Invariants behind the suppression are documented at classifyRunStates (I1: compose down ⇒ zero
|
||||
// containers ⇒ StateStopped; I2: P2 census — all catalog services unless-stopped ⇒ faults never rest
|
||||
// at stopped).
|
||||
|
||||
func stack(name string, st stacks.ContainerState, deployed, deploying bool) stacks.Stack {
|
||||
return stacks.Stack{
|
||||
Name: name,
|
||||
Meta: stacks.Metadata{DisplayName: name},
|
||||
State: st,
|
||||
Deployed: deployed,
|
||||
Deploying: deploying,
|
||||
}
|
||||
}
|
||||
|
||||
func downByName(states []notify.AppRunState) map[string]bool {
|
||||
m := map[string]bool{}
|
||||
for _, s := range states {
|
||||
m[s.Name] = s.Down
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
func deadNames(dead []web.DeadApp) map[string]bool {
|
||||
m := map[string]bool{}
|
||||
for _, d := range dead {
|
||||
m[d.Name] = true
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
// Group A (Scenario A) — suppression. Over a [running, stopped, exited, degraded] fixture, the dead
|
||||
// list is EXACTLY {exited, degraded} and the Down flags are {false, false, true, true}: the stopped
|
||||
// app is silent, the two faults still alarm.
|
||||
//
|
||||
// COMPANION red-proof: revert the filter to bare `stacks.IsDownState(st.State)` (drop the
|
||||
// `&& st.State != stacks.StateStopped` guard) → stopped reports Down=true and enters the dead list →
|
||||
// both the dead-set and the Down-flag assertions below fail. (Verified by hand-editing the seam.)
|
||||
func TestClassifyRunStates_StoppedIsSuppressed(t *testing.T) {
|
||||
sts := []stacks.Stack{
|
||||
stack("radarr", stacks.StateRunning, true, false),
|
||||
stack("cwa", stacks.StateStopped, true, false),
|
||||
stack("immich", stacks.StateExited, true, false),
|
||||
stack("nextcloud", stacks.StateDegraded, true, false),
|
||||
}
|
||||
|
||||
dead, states := classifyRunStates(sts, nil, nil, time.Now())
|
||||
|
||||
gotDead := deadNames(dead)
|
||||
if len(gotDead) != 2 || !gotDead["immich"] || !gotDead["nextcloud"] {
|
||||
t.Fatalf("dead list must be exactly {immich(exited), nextcloud(degraded)}, got %+v", dead)
|
||||
}
|
||||
if gotDead["cwa"] {
|
||||
t.Errorf("a deliberately stopped app must NOT be in the dead list (no banner)")
|
||||
}
|
||||
if gotDead["radarr"] {
|
||||
t.Errorf("a running app must never be in the dead list")
|
||||
}
|
||||
|
||||
down := downByName(states)
|
||||
want := map[string]bool{"radarr": false, "cwa": false, "immich": true, "nextcloud": true}
|
||||
if len(down) != len(want) {
|
||||
t.Fatalf("every deployed app must have a run state, got %+v", down)
|
||||
}
|
||||
for name, w := range want {
|
||||
if down[name] != w {
|
||||
t.Errorf("Down[%s] = %v, want %v (stopped ⇒ false ⇒ no app_start_failed event)", name, down[name], w)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Group B (Scenario B) — fault parity. With only exited + degraded present, BOTH surface in the dead
|
||||
// list AND both report Down=true — byte-identical to v0.163.1 for every non-stopped down state. The
|
||||
// suppression touches stopped and nothing else.
|
||||
func TestClassifyRunStates_FaultParity(t *testing.T) {
|
||||
sts := []stacks.Stack{
|
||||
stack("immich", stacks.StateExited, true, false),
|
||||
stack("nextcloud", stacks.StateDegraded, true, false),
|
||||
}
|
||||
|
||||
dead, states := classifyRunStates(sts, nil, nil, time.Now())
|
||||
|
||||
gotDead := deadNames(dead)
|
||||
if len(gotDead) != 2 || !gotDead["immich"] || !gotDead["nextcloud"] {
|
||||
t.Fatalf("both faults must appear in the dead list, got %+v", dead)
|
||||
}
|
||||
down := downByName(states)
|
||||
if !down["immich"] || !down["nextcloud"] {
|
||||
t.Fatalf("both faults must report Down=true, got %+v", down)
|
||||
}
|
||||
// State strings must ride through to the banner unchanged (banner shows "(exited)"/"(degraded)").
|
||||
byName := map[string]string{}
|
||||
for _, d := range dead {
|
||||
byName[d.Name] = d.State
|
||||
}
|
||||
if byName["immich"] != string(stacks.StateExited) || byName["nextcloud"] != string(stacks.StateDegraded) {
|
||||
t.Errorf("dead-app State must carry the raw aggregate state, got %+v", byName)
|
||||
}
|
||||
}
|
||||
|
||||
// Deploying and undeployed stacks are skipped entirely (unchanged fix-3 behavior).
|
||||
func TestClassifyRunStates_SkipsDeployingAndUndeployed(t *testing.T) {
|
||||
sts := []stacks.Stack{
|
||||
stack("mid", stacks.StateDeploying, true, true), // mid-deploy → skipped
|
||||
stack("gone", stacks.StateExited, false, false), // not deployed → skipped
|
||||
}
|
||||
dead, states := classifyRunStates(sts, nil, nil, time.Now())
|
||||
if len(dead) != 0 || len(states) != 0 {
|
||||
t.Fatalf("deploying and undeployed stacks must be skipped, got dead=%+v states=%+v", dead, states)
|
||||
}
|
||||
}
|
||||
@@ -1,139 +0,0 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/stacks"
|
||||
)
|
||||
|
||||
// C9-F2 — a SUSTAINED `restarting` is a crash loop and must alarm; a BRIEF one must not.
|
||||
//
|
||||
// The defect: `IsDownState` excludes `restarting` as "self-recovering", but for the catalog's
|
||||
// standard `restart: unless-stopped` Docker retries forever, so a crash loop sat in `restarting`
|
||||
// indefinitely and was counted as working. Campaign 9 watched docmost loop for nine minutes
|
||||
// (restartcount 18) while the F-OBS heartbeat printed "4 deployed app(s) evaluated, 0 currently down".
|
||||
//
|
||||
// The whole design tension is that B must keep passing while A does: an alarm that fires on every
|
||||
// deploy is one the operator learns to ignore.
|
||||
|
||||
// restartingSince builds a deployed stack that has been restarting since `since`.
|
||||
func restartingSince(name string, since time.Time) stacks.Stack {
|
||||
s := stack(name, stacks.StateRestarting, true, false)
|
||||
s.RestartingSince = since
|
||||
return s
|
||||
}
|
||||
|
||||
// SCENARIO A — a crash loop alarms. A stack restarting for longer than the threshold enters BOTH the
|
||||
// banner dead-list and the notifier Down-set, so app_start_failed can fire.
|
||||
//
|
||||
// RED-PROOF (observed): drop `|| crashLooping` from the `down` expression in classifyRunStates →
|
||||
//
|
||||
// crashloop_classify_test.go:52: docmost is NOT in the Down-set — a crash loop is silent (this is C9-F2)
|
||||
// crashloop_classify_test.go:55: docmost is NOT in the banner dead-list
|
||||
func TestClassifyRunStates_SustainedRestartingAlarms(t *testing.T) {
|
||||
now := time.Now()
|
||||
sts := []stacks.Stack{
|
||||
stack("paperless-ngx", stacks.StateRunning, true, false),
|
||||
restartingSince("docmost", now.Add(-9*time.Minute)), // the Campaign 9 observation, exactly
|
||||
}
|
||||
|
||||
dead, states := classifyRunStates(sts, nil, nil, now)
|
||||
|
||||
if !downByName(states)["docmost"] {
|
||||
t.Errorf("docmost is NOT in the Down-set — a crash loop is silent (this is C9-F2)")
|
||||
}
|
||||
if !deadNames(dead)["docmost"] {
|
||||
t.Errorf("docmost is NOT in the banner dead-list")
|
||||
}
|
||||
if downByName(states)["paperless-ngx"] {
|
||||
t.Errorf("a healthy app was dragged down with it")
|
||||
}
|
||||
}
|
||||
|
||||
// SCENARIO B — a normal deploy or update does NOT alarm. `docker compose up -d` passes through
|
||||
// restarting; alarming there would page the operator on every routine operation, fleet-wide.
|
||||
//
|
||||
// This is the test that must fail against the naive fix. RED-PROOF (observed): add StateRestarting
|
||||
// to IsDownState instead of using the threshold →
|
||||
//
|
||||
// crashloop_classify_test.go:78: a BRIEFLY restarting app alarms — every deploy and update would page the operator
|
||||
func TestClassifyRunStates_BriefRestartingIsSilent(t *testing.T) {
|
||||
now := time.Now()
|
||||
sts := []stacks.Stack{
|
||||
restartingSince("mealie", now.Add(-30*time.Second)), // mid-deploy
|
||||
restartingSince("ghost", now.Add(-2*time.Minute)), // slow image pull, still normal
|
||||
}
|
||||
|
||||
dead, states := classifyRunStates(sts, nil, nil, now)
|
||||
|
||||
for _, name := range []string{"mealie", "ghost"} {
|
||||
if downByName(states)[name] {
|
||||
t.Errorf("a BRIEFLY restarting app alarms (%s) — every deploy and update would page the operator", name)
|
||||
}
|
||||
}
|
||||
if len(dead) != 0 {
|
||||
t.Errorf("banner dead-list should be empty during normal restarts, got %v", deadNames(dead))
|
||||
}
|
||||
}
|
||||
|
||||
// The boundary itself, asserted from both sides so the threshold cannot drift silently.
|
||||
func TestCrashLooping_ThresholdBoundary(t *testing.T) {
|
||||
now := time.Now()
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
age time.Duration
|
||||
want bool
|
||||
}{
|
||||
{"just under the threshold", 4*time.Minute + 59*time.Second, false},
|
||||
{"exactly at the threshold", 5 * time.Minute, true},
|
||||
{"well past it", 30 * time.Minute, true},
|
||||
} {
|
||||
s := restartingSince("app", now.Add(-tc.age))
|
||||
if got := s.CrashLooping(now); got != tc.want {
|
||||
t.Errorf("%s: CrashLooping(age=%s) = %v, want %v", tc.name, tc.age, got, tc.want)
|
||||
}
|
||||
}
|
||||
|
||||
// A stack that is not restarting is never a crash loop, however old the stamp.
|
||||
s := stack("app", stacks.StateRunning, true, false)
|
||||
s.RestartingSince = now.Add(-time.Hour)
|
||||
if s.CrashLooping(now) {
|
||||
t.Error("a RUNNING stack reported as crash-looping — the state test is missing")
|
||||
}
|
||||
|
||||
// A zero stamp is "not yet observed restarting", never a crash loop — this is what makes the
|
||||
// first scan after a controller restart silent instead of alarming on everything at once.
|
||||
z := stack("app", stacks.StateRestarting, true, false)
|
||||
if z.CrashLooping(now) {
|
||||
t.Error("a zero RestartingSince reported as crash-looping — a controller restart would alarm fleet-wide")
|
||||
}
|
||||
}
|
||||
|
||||
// SCENARIO C — R-97b's quiesce suppression still wins inside its window. A stack the backup stopped
|
||||
// and is restarting must stay silent while suppressed, even if its restarting run is old enough to
|
||||
// qualify. The window EXPIRES, so a genuinely dead app still alarms afterwards — proven by the
|
||||
// second half of this test.
|
||||
//
|
||||
// RED-PROOF (observed): drop `&& !quiesced[st.Name]` from the `down` expression →
|
||||
//
|
||||
// crashloop_classify_test.go:129: a quiesced stack alarms — every backup would page the customer
|
||||
func TestClassifyRunStates_QuiesceSuppressionBeatsCrashLoop(t *testing.T) {
|
||||
now := time.Now()
|
||||
sts := []stacks.Stack{restartingSince("docmost", now.Add(-9*time.Minute))}
|
||||
|
||||
// Inside the R-97b window.
|
||||
_, states := classifyRunStates(sts, map[string]bool{"docmost": true}, nil, now)
|
||||
if downByName(states)["docmost"] {
|
||||
t.Errorf("a quiesced stack alarms — every backup would page the customer")
|
||||
}
|
||||
|
||||
// Window expired (the stack is no longer reported as suppressed): the same stack must now alarm.
|
||||
dead, states := classifyRunStates(sts, nil, nil, now)
|
||||
if !downByName(states)["docmost"] {
|
||||
t.Errorf("suppression outlived its window — a genuinely dead app stayed silent (R-97b's own warning)")
|
||||
}
|
||||
if !deadNames(dead)["docmost"] {
|
||||
t.Errorf("suppression outlived its window for the banner too")
|
||||
}
|
||||
}
|
||||
@@ -1,97 +0,0 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"log"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// F-OBS (Campaign 8): on a default `info`-level box there was NO positive observable that
|
||||
// `deadapp-check` had run. Its per-cycle scheduler line goes through Scheduler.dbg(), which is gated
|
||||
// on logging.level==debug and therefore never PRODUCED on a default box — so it could not even reach
|
||||
// the always-DEBUG ring — and a 30 s interval also puts the job on the scheduler's quiet path.
|
||||
//
|
||||
// "No alarms" was therefore indistinguishable from "the detector never ran", which is exactly the
|
||||
// fallacy this project now has a standing rule against, and it undermines confidence in the
|
||||
// F-CRIT-1 fix in the field.
|
||||
//
|
||||
// Scenario F — the observable must appear AT INFO LEVEL. These tests assert the emitted LINE, not
|
||||
// merely that a function was called; asserting the call would reproduce the original mistake.
|
||||
|
||||
// RED-PROOF: delete the logger.Printf in noteDeadAppScan (or drop the whole call from the job
|
||||
// closure) → every case below sees an empty buffer and this fails with
|
||||
// "no observable emitted at scan 20 — silence is indistinguishable from not running".
|
||||
func TestNoteDeadAppScan_EmitsAtInfoLevel(t *testing.T) {
|
||||
var buf bytes.Buffer
|
||||
lg := log.New(&buf, "", 0)
|
||||
|
||||
noteDeadAppScan(lg, deadAppHeartbeatEvery, 7, 2)
|
||||
|
||||
out := buf.String()
|
||||
if out == "" {
|
||||
t.Fatalf("no observable emitted at scan %d — silence is indistinguishable from not running", deadAppHeartbeatEvery)
|
||||
}
|
||||
if !strings.Contains(out, "[INFO]") {
|
||||
t.Errorf("the observable is not at INFO level, so a default `logging.level: info` box would never see it:\n%s", out)
|
||||
}
|
||||
if !strings.Contains(out, "[deadapp]") {
|
||||
t.Errorf("the observable does not identify the check that produced it:\n%s", out)
|
||||
}
|
||||
// it must carry WHAT IT SAW, not just "I ran" — an operator needs to distinguish
|
||||
// "running and everything is up" from "running and 2 apps are down".
|
||||
for _, want := range []string{"scans since boot", "evaluated", "currently down"} {
|
||||
if !strings.Contains(out, want) {
|
||||
t.Errorf("the observable omits %q — it proves the check ran but not what it found:\n%s", want, out)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// It must NOT be a line per run. At a 30 s cadence that is 2880 lines/day, which is precisely why
|
||||
// the original author chose silence — so a fix that floods is not a fix.
|
||||
//
|
||||
// RED-PROOF: change the guard to `scans%1 != 0` (i.e. emit every run) → this fails with
|
||||
// "emitted 60 lines across 60 scans — that is the flood that made silence attractive".
|
||||
func TestNoteDeadAppScan_IsASummaryNotAFlood(t *testing.T) {
|
||||
var buf bytes.Buffer
|
||||
lg := log.New(&buf, "", 0)
|
||||
|
||||
const scans = 60
|
||||
for i := 1; i <= scans; i++ {
|
||||
noteDeadAppScan(lg, i, 3, 0)
|
||||
}
|
||||
|
||||
got := strings.Count(buf.String(), "[deadapp] check alive")
|
||||
want := scans / deadAppHeartbeatEvery
|
||||
if got == scans {
|
||||
t.Fatalf("emitted %d lines across %d scans — that is the flood that made silence attractive", got, scans)
|
||||
}
|
||||
if got != want {
|
||||
t.Errorf("emitted %d heartbeat lines across %d scans, want %d (one per %d)", got, scans, want, deadAppHeartbeatEvery)
|
||||
}
|
||||
}
|
||||
|
||||
// The cadence must be frequent enough that a STALLED detector is obvious well inside the 180 s alarm
|
||||
// grace this check feeds. 20 scans x 30 s = 10 min; if someone widens it to hours the observable
|
||||
// stops being useful as a liveness signal, and this is the tripwire.
|
||||
func TestDeadAppHeartbeatEvery_StaysUsefulAsALivenessSignal(t *testing.T) {
|
||||
const scanInterval = 30 // seconds, matching sched.Every("deadapp-check", 30*time.Second, ...)
|
||||
periodSec := deadAppHeartbeatEvery * scanInterval
|
||||
if periodSec > 15*60 {
|
||||
t.Errorf("heartbeat period is %ds (>15min) — too sparse to notice a stalled detector", periodSec)
|
||||
}
|
||||
if deadAppHeartbeatEvery < 2 {
|
||||
t.Errorf("heartbeat every %d scans is a per-run flood", deadAppHeartbeatEvery)
|
||||
}
|
||||
}
|
||||
|
||||
// Off-cadence scans stay quiet, and a nil logger is tolerated (the job closure must never panic).
|
||||
func TestNoteDeadAppScan_QuietOffCadenceAndNilSafe(t *testing.T) {
|
||||
var buf bytes.Buffer
|
||||
lg := log.New(&buf, "", 0)
|
||||
noteDeadAppScan(lg, deadAppHeartbeatEvery-1, 1, 0)
|
||||
if buf.Len() != 0 {
|
||||
t.Errorf("emitted off-cadence:\n%s", buf.String())
|
||||
}
|
||||
noteDeadAppScan(nil, deadAppHeartbeatEvery, 1, 0) // must not panic
|
||||
}
|
||||
@@ -1,124 +0,0 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"time"
|
||||
|
||||
"testing"
|
||||
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/stacks"
|
||||
)
|
||||
|
||||
// F-CRIT-1 cause 2 (Campaign 8): `classifyRunStates` whitelisted StateStopped on invariant I1
|
||||
// ("StateStopped means the USER stopped it"). The quiesce loop broke I1 by stopping stacks via the
|
||||
// same `docker compose down` path, so a stack quiesce stopped and then FAILED to restart was also
|
||||
// StateStopped — and was whitelisted into total silence. Live evidence: a customer app dead
|
||||
// indefinitely, no banner, no event, no email, while the dead-app scanner ran 11 times over it.
|
||||
//
|
||||
// The two cases are byte-identical on the Docker side. The ONLY thing that separates them is that
|
||||
// the quiesce loop knows it tried to restart and could not — `failedRestart` is that knowledge.
|
||||
|
||||
// Scenario A (cause 2) — a stack quiesce failed to restart MUST alarm, despite being StateStopped.
|
||||
//
|
||||
// RED-PROOF: restore the unconditional whitelist (`down := IsDownState(st.State) &&
|
||||
// st.State != stacks.StateStopped && !quiesced[st.Name]`) → immich reports Down=false and stays out
|
||||
// of the dead list, and this fails with "a stack that FAILED to restart is silent".
|
||||
func TestClassifyRunStates_FailedRestartAlarmsDespiteStateStopped(t *testing.T) {
|
||||
sts := []stacks.Stack{
|
||||
stack("bookstack", stacks.StateRunning, true, false),
|
||||
stack("immich", stacks.StateStopped, true, false), // quiesce stopped it; restart FAILED
|
||||
}
|
||||
failed := map[string]bool{"immich": true}
|
||||
|
||||
dead, states := classifyRunStates(sts, nil, failed, time.Now())
|
||||
|
||||
if !downByName(states)["immich"] {
|
||||
t.Error("a stack that FAILED to restart is silent (Down=false) — this is F-CRIT-1")
|
||||
}
|
||||
if !deadNames(dead)["immich"] {
|
||||
t.Error("a stack that FAILED to restart is absent from the dashboard dead-list — this is F-CRIT-1")
|
||||
}
|
||||
if downByName(states)["bookstack"] {
|
||||
t.Error("a healthy running stack was marked down")
|
||||
}
|
||||
}
|
||||
|
||||
// Scenario B — a DELIBERATE user stop must still be silent. This pins v0.164.0 and is what stops
|
||||
// the fix above from becoming a regression.
|
||||
//
|
||||
// RED-PROOF: make the whitelist unconditional in the other direction (drop the `&& !failedRestart`
|
||||
// term, i.e. treat every StateStopped as a failed restart) → cwa alarms and this fails with
|
||||
// "a deliberate user stop alarmed".
|
||||
func TestClassifyRunStates_UserStopStillSilent(t *testing.T) {
|
||||
sts := []stacks.Stack{
|
||||
stack("cwa", stacks.StateStopped, true, false), // the user stopped this from the UI
|
||||
stack("immich", stacks.StateStopped, true, false),
|
||||
}
|
||||
// only immich failed to restart; cwa was never touched by a quiesce
|
||||
failed := map[string]bool{"immich": true}
|
||||
|
||||
dead, states := classifyRunStates(sts, nil, failed, time.Now())
|
||||
down := downByName(states)
|
||||
|
||||
if down["cwa"] || deadNames(dead)["cwa"] {
|
||||
t.Error("a deliberate user stop alarmed — that is the v0.164.0 regression this must not reintroduce")
|
||||
}
|
||||
if !down["immich"] {
|
||||
t.Error("the failed restart went silent")
|
||||
}
|
||||
}
|
||||
|
||||
// Scenario B, stronger form — with NO failed restarts at all, behaviour is byte-identical to
|
||||
// v0.164.0: every StateStopped is silent.
|
||||
func TestClassifyRunStates_NoFailedRestartsIsV0164Behaviour(t *testing.T) {
|
||||
sts := []stacks.Stack{
|
||||
stack("radarr", stacks.StateRunning, true, false),
|
||||
stack("cwa", stacks.StateStopped, true, false),
|
||||
stack("immich", stacks.StateExited, true, false),
|
||||
stack("nextcloud", stacks.StateDegraded, true, false),
|
||||
}
|
||||
|
||||
dead, states := classifyRunStates(sts, nil, nil, time.Now())
|
||||
down := downByName(states)
|
||||
|
||||
if down["cwa"] {
|
||||
t.Error("stopped alarmed with no failed restarts — v0.164.0 behaviour broken")
|
||||
}
|
||||
if !down["immich"] || !down["nextcloud"] {
|
||||
t.Error("a genuine fault (exited/degraded) stopped alarming")
|
||||
}
|
||||
if got := len(deadNames(dead)); got != 2 {
|
||||
t.Errorf("dead list has %d entries, want exactly {immich, nextcloud}", got)
|
||||
}
|
||||
}
|
||||
|
||||
// Scenario C — during the R-97b grace window the stack is suppressed even if its restart failed.
|
||||
// The grace exists so a slow-starting app is not called dead; it EXPIRES, and the alarm follows.
|
||||
//
|
||||
// RED-PROOF: drop the `&& !quiesced[st.Name]` term → the app alarms mid-restart on every normal
|
||||
// backup, which is the false-alarm R-97b was built to remove.
|
||||
func TestClassifyRunStates_GraceWindowStillSuppresses(t *testing.T) {
|
||||
sts := []stacks.Stack{stack("immich", stacks.StateStopped, true, false)}
|
||||
quiesced := map[string]bool{"immich": true} // still inside quiesceAlarmGrace
|
||||
failed := map[string]bool{"immich": true} // and we already know the restart failed
|
||||
|
||||
dead, states := classifyRunStates(sts, quiesced, failed, time.Now())
|
||||
|
||||
if downByName(states)["immich"] {
|
||||
t.Error("alarmed while still inside the grace window — R-97b Scenario E broken")
|
||||
}
|
||||
if len(dead) != 0 {
|
||||
t.Errorf("dead list not empty during grace: %v", deadNames(dead))
|
||||
}
|
||||
}
|
||||
|
||||
// An undeployed or mid-deploy stack is never classified, failed restart or not.
|
||||
func TestClassifyRunStates_UndeployedIgnored(t *testing.T) {
|
||||
sts := []stacks.Stack{
|
||||
stack("ghost", stacks.StateStopped, false, false),
|
||||
stack("deploying", stacks.StateStopped, true, true),
|
||||
}
|
||||
dead, states := classifyRunStates(sts, nil, map[string]bool{"ghost": true, "deploying": true}, time.Now())
|
||||
if len(dead) != 0 || len(states) != 0 {
|
||||
t.Errorf("undeployed/deploying stacks were classified: dead=%v states=%v", deadNames(dead), states)
|
||||
}
|
||||
}
|
||||
+30
-1655
File diff suppressed because it is too large
Load Diff
@@ -1,30 +0,0 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/quiesce"
|
||||
)
|
||||
|
||||
// R-97a — REACHABILITY, not behaviour.
|
||||
//
|
||||
// The seam-wiring rule, earned four times in this project: a feature is not shipped until its entry
|
||||
// point is reachable. `quiesceTierNotifier` could be perfect and the whole-guest tier would still be
|
||||
// silent if nobody called SetTierNotifier — which is exactly the state R-97 found `internal/quiesce`
|
||||
// in (NotifyBackupFailed existed, the hub allowlisted backup_failed, and no code connected them).
|
||||
//
|
||||
// This asserts the adapter SATISFIES the interface the loop requires. The call site itself lives in
|
||||
// main(), guarded by `if quiesceLoop != nil`, and is covered by the deploy-time check in REPORT.md.
|
||||
func TestQuiesceTierNotifierIsWired(t *testing.T) {
|
||||
var _ quiesce.TierNotifier = quiesceTierNotifier{}
|
||||
|
||||
// And it must not panic on a nil notifier — main() constructs it with a real one, but a future
|
||||
// refactor that reorders startup must fail loudly here rather than at 03:00 on a customer box.
|
||||
defer func() {
|
||||
if r := recover(); r != nil {
|
||||
t.Fatalf("the adapter panicked with a nil notifier: %v", r)
|
||||
}
|
||||
}()
|
||||
var n quiesceTierNotifier
|
||||
_ = n
|
||||
}
|
||||
@@ -1,22 +0,0 @@
|
||||
package main
|
||||
|
||||
import "gitea.dooplex.hu/admin/felhom-controller/internal/quiesce"
|
||||
|
||||
// COMPILE-TIME WITNESSES for OPTIONAL interfaces satisfied by a RUNTIME type assertion.
|
||||
//
|
||||
// Moved here from a _test.go file (R-88 Part 2) on purpose: a witness in a test fires on `go test`
|
||||
// and `go vet`, but NOT on `go build` alone. The failure it guards against — a signature change that
|
||||
// silently breaks an interface nobody checks at compile time — is exactly the kind that gets pushed
|
||||
// by a build-only step.
|
||||
//
|
||||
// THE INCIDENT THIS PREVENTS, which already happened once: when `TieredBackend.DueFor` gained a
|
||||
// return value during R-88 Part 2, `quiesceBackend` stopped satisfying the interface and the whole
|
||||
// repo still BUILT AND VETTED CLEAN, because `resolveDueTiers` only ever asserts it at runtime
|
||||
// (`l.backend.(TieredBackend)`). A failed assertion silently falls back to the untargeted
|
||||
// single-tier path — so every box would have quietly lost R-82's multi-tier backups, with no error
|
||||
// anywhere. It was caught by accident, not by the toolchain.
|
||||
//
|
||||
// THIS DOES NOT MAKE THE INTERFACE REQUIRED. Optionality is deliberate: it is what lets a new
|
||||
// controller meet an old agent, and what `resolveDueTiers` degrades through on purpose. The witness
|
||||
// pins the IMPLEMENTATION, not the CONTRACT.
|
||||
var _ quiesce.TieredBackend = quiesceBackend{}
|
||||
@@ -95,7 +95,7 @@ monitoring:
|
||||
hub:
|
||||
enabled: true # Enable central reporting
|
||||
url: "https://hub.felhom.eu" # Hub API endpoint
|
||||
api_key: "<hub-issued-per-customer-key>" # From the hub-generated config; never commit a real key
|
||||
api_key: "094091de545ce28795c47ac2158fc30750db5c24a621c49329b001ee8db57fb8" # Shared secret for authentication
|
||||
push_interval: "15m" # How often to push reports
|
||||
|
||||
# --- Self-update ---
|
||||
|
||||
@@ -3,9 +3,6 @@ module gitea.dooplex.hu/admin/felhom-controller
|
||||
go 1.24.0
|
||||
|
||||
require (
|
||||
github.com/emersion/go-sasl v0.0.0-20241020182733-b788ff22d5a6
|
||||
github.com/emersion/go-smtp v0.24.0
|
||||
github.com/skip2/go-qrcode v0.0.0-20200617195104-da1b6568686e
|
||||
golang.org/x/crypto v0.31.0
|
||||
gopkg.in/yaml.v3 v3.0.1
|
||||
modernc.org/sqlite v1.45.0
|
||||
|
||||
@@ -1,9 +1,5 @@
|
||||
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
|
||||
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
|
||||
github.com/emersion/go-sasl v0.0.0-20241020182733-b788ff22d5a6 h1:oP4q0fw+fOSWn3DfFi4EXdT+B+gTtzx8GC9xsc26Znk=
|
||||
github.com/emersion/go-sasl v0.0.0-20241020182733-b788ff22d5a6/go.mod h1:iL2twTeMvZnrg54ZoPDNfJaJaqy0xIQFuBdrLsmspwQ=
|
||||
github.com/emersion/go-smtp v0.24.0 h1:g6AfoF140mvW0vLNPD/LuCBLEAdlxOjIXqbIkJIS6Wk=
|
||||
github.com/emersion/go-smtp v0.24.0/go.mod h1:ZtRRkbTyp2XTHCA+BmyTFTrj8xY4I+b4McvHxCU2gsQ=
|
||||
github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e h1:ijClszYn+mADRFY17kjQEVQ1XRhq2/JR1M3sGqeJoxs=
|
||||
github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e/go.mod h1:boTsfXsheKC2y+lKOCMpSfarhxDeIzfZG1jqGcPl3cA=
|
||||
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
|
||||
@@ -16,8 +12,6 @@ github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOF
|
||||
github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
|
||||
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
|
||||
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
|
||||
github.com/skip2/go-qrcode v0.0.0-20200617195104-da1b6568686e h1:MRM5ITcdelLK2j1vwZ3Je0FKVCfqOLp5zO6trqMLYs0=
|
||||
github.com/skip2/go-qrcode v0.0.0-20200617195104-da1b6568686e/go.mod h1:XV66xRDqSt+GTGFMVlhk3ULuV0y9ZmzeVGR4mloJI3M=
|
||||
golang.org/x/crypto v0.31.0 h1:ihbySMvVjLAeSH1IbfcRTkD/iNscyz8rGzjF/E5hV6U=
|
||||
golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk=
|
||||
golang.org/x/exp v0.0.0-20251023183803-a4bb9ffd2546 h1:mgKeJMpvi0yx/sU5GsxQ7p6s2wtOnGAHZWCHUM4KGzY=
|
||||
@@ -29,8 +23,6 @@ golang.org/x/sync v0.17.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
|
||||
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.37.0 h1:fdNQudmxPjkdUTPnLn5mdQv7Zwvbvpaxqs831goi9kQ=
|
||||
golang.org/x/sys v0.37.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
|
||||
golang.org/x/term v0.27.0 h1:WP60Sv1nlK1T6SupCHbXzSaN0b9wUmsPoRS9b61A23Q=
|
||||
golang.org/x/term v0.27.0/go.mod h1:iMsnZpn0cago0GOrHO2+Y7u7JPn5AylBrcoWkElMTSM=
|
||||
golang.org/x/tools v0.38.0 h1:Hx2Xv8hISq8Lm16jvBZ2VQf+RLmbd7wVUsALibYI/IQ=
|
||||
golang.org/x/tools v0.38.0/go.mod h1:yEsQ/d/YK8cjh0L6rZlY8tgtlKiBNTL14pGDJPJpYQs=
|
||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
|
||||
|
||||
@@ -1,42 +0,0 @@
|
||||
# felhom-samba — the LAN SMB-sharing infra image for felhom-controller (R-7 slice 1).
|
||||
#
|
||||
# DUMB BY DESIGN: /etc/samba/smb.conf is bind-mounted READ-ONLY by the controller, which
|
||||
# owns all rendering. This image templates nothing and bakes NO share name and NO password.
|
||||
# The three-daemon discovery stack is the spike verdict
|
||||
# (felhom.eu/documentation/audits/SPIKE-lan-discovery-2026-07-18.md, S4/S4b):
|
||||
# - smbd : the SMB/CIFS server (445)
|
||||
# - nmbd : NetBIOS name service — REQUIRED alongside wsdd. wsdd-only makes the box visible
|
||||
# in Explorer but the double-click fails 0x80070035 (no flat-name resolution);
|
||||
# nmbd is what makes \\<NAME> resolve + mount (S4b, proven live).
|
||||
# - wsdd : WS-Discovery, so the box appears in Windows Explorer's Network view.
|
||||
# - avahi : mDNS/Bonjour (v1.1.0) — THE macOS path. Windows and macOS do not share a
|
||||
# discovery mechanism, and nmbd does not cover the Mac: captured live on
|
||||
# 2026-07-20, macOS broadcasts a correct NBNS query for FELHOM<20>, the box
|
||||
# answers correctly in 140us (flags 0x8580, RCODE=0, the right address), and
|
||||
# macOS REFUSES TO ACT ON IT — no TCP follows. NetBIOS feeds legacy browsing
|
||||
# there, not smb:// URL resolution. With mDNS, `smb://<NAME>.local` connects
|
||||
# immediately — PROVEN live from a Mac on 2026-07-20.
|
||||
# NOT proven: automatic appearance in the Finder sidebar. The _smb._tcp record
|
||||
# is published and answers browse queries on the wire, but the test Mac's
|
||||
# sidebar stayed empty (it had no Network/Bonjour section shown at all, which
|
||||
# is a Finder Settings -> Sidebar toggle). Treat sidebar discovery as an OPEN
|
||||
# question, not a shipped feature.
|
||||
# Evidence: felhom.eu/documentation/audits/DIAG-sharing-2026-07-20.md.
|
||||
FROM alpine:3.21@sha256:48b0309ca019d89d40f670aa1bc06e426dc0931948452e8491e3d65087abc07d
|
||||
|
||||
# samba = smbd + nmbd + smbpasswd/testparm (meta-package proven installable in the spike);
|
||||
# wsdd = WS-Discovery daemon; tini = a proper PID1 to reap nmbd/wsdd/avahi and forward signals;
|
||||
# avahi + dbus = mDNS/Bonjour (avahi-daemon talks to the system bus, so dbus is not optional).
|
||||
RUN apk add --no-cache samba wsdd tini avahi dbus \
|
||||
&& rm -rf /var/cache/apk/* \
|
||||
&& rm -f /etc/samba/smb.conf \
|
||||
&& rm -f /etc/avahi/services/*.service
|
||||
|
||||
# passdb on a named volume → the household SMB password survives container recreation
|
||||
# (share add/remove re-renders + `compose up -d`, which recreates the container).
|
||||
VOLUME ["/var/lib/samba"]
|
||||
|
||||
COPY entrypoint.sh /entrypoint.sh
|
||||
RUN chmod +x /entrypoint.sh
|
||||
|
||||
ENTRYPOINT ["/sbin/tini", "--", "/entrypoint.sh"]
|
||||
@@ -1,83 +0,0 @@
|
||||
#!/bin/sh
|
||||
# felhom-samba entrypoint (R-7 slice 1). A dumb supervisor: smb.conf is bind-mounted
|
||||
# READ-ONLY by the controller, so nothing here templates config or bakes a secret. It
|
||||
# only ensures the household unix user exists (uid:gid 1000) and launches the three
|
||||
# discovery daemons. Verdict source: SPIKE-lan-discovery-2026-07-18 (S4/S4b).
|
||||
set -e
|
||||
|
||||
FELHOM_UID="${FELHOM_UID:-1000}"
|
||||
FELHOM_GID="${FELHOM_GID:-1000}"
|
||||
SERVER_NAME="${FELHOM_SERVER_NAME:-FELHOM}"
|
||||
IFACE="${FELHOM_IFACE:-eth0}"
|
||||
|
||||
# Household group/user at uid:gid 1000 — files written over SMB then match the app +
|
||||
# backup ownership convention (smb.conf sets `force user = felhom` per share).
|
||||
if ! getent group "$FELHOM_GID" >/dev/null 2>&1; then
|
||||
addgroup -g "$FELHOM_GID" felhom 2>/dev/null || true
|
||||
fi
|
||||
GRP_NAME="$(getent group "$FELHOM_GID" 2>/dev/null | cut -d: -f1)"
|
||||
[ -z "$GRP_NAME" ] && GRP_NAME=felhom
|
||||
if ! getent passwd "$FELHOM_UID" >/dev/null 2>&1; then
|
||||
adduser -D -H -u "$FELHOM_UID" -G "$GRP_NAME" -s /sbin/nologin felhom 2>/dev/null || true
|
||||
fi
|
||||
|
||||
mkdir -p /var/lib/samba/private /run/samba
|
||||
|
||||
# --- mDNS / Bonjour (v1.1.0) -------------------------------------------------------------
|
||||
# THE macOS path. Templated from SERVER_NAME rather than baked, so renaming the server in the
|
||||
# UI re-advertises under the new name on the next container recreate — a baked name would
|
||||
# leave the box answering to something the customer no longer sees anywhere.
|
||||
#
|
||||
# A STATIC service file, deliberately, rather than smbd's own `multicast dns register`: it
|
||||
# needs no line in smb.conf (which is bind-mounted READ-ONLY and owned by the controller's
|
||||
# renderer) and it lets us publish _device-info._tcp so the Finder shows a sensible icon
|
||||
# instead of a generic globe.
|
||||
mkdir -p /etc/avahi/services /run/dbus
|
||||
cat > /etc/avahi/avahi-daemon.conf <<CONF
|
||||
[server]
|
||||
host-name=${SERVER_NAME}
|
||||
use-ipv4=yes
|
||||
use-ipv6=no
|
||||
allow-interfaces=${IFACE}
|
||||
ratelimit-interval-usec=1000000
|
||||
ratelimit-burst=1000
|
||||
|
||||
[wide-area]
|
||||
enable-wide-area=no
|
||||
|
||||
[publish]
|
||||
publish-addresses=yes
|
||||
publish-hinfo=no
|
||||
publish-workstation=no
|
||||
CONF
|
||||
|
||||
cat > /etc/avahi/services/smb.service <<CONF
|
||||
<?xml version="1.0" standalone='no'?><!DOCTYPE service-group SYSTEM "avahi-service.dtd">
|
||||
<service-group>
|
||||
<name replace-wildcards="yes">%h</name>
|
||||
<service>
|
||||
<type>_smb._tcp</type>
|
||||
<port>445</port>
|
||||
</service>
|
||||
<service>
|
||||
<type>_device-info._tcp</type>
|
||||
<port>0</port>
|
||||
<txt-record>model=RackMac</txt-record>
|
||||
</service>
|
||||
</service-group>
|
||||
CONF
|
||||
|
||||
echo "[felhom-samba] launching nmbd + wsdd + avahi + smbd (server=${SERVER_NAME} iface=${IFACE} uid=${FELHOM_UID})"
|
||||
|
||||
# nmbd: NetBIOS flat-name resolution so \\<NAME> resolves and mounts on WINDOWS (the S4b fix).
|
||||
# It does NOT serve macOS — see the Dockerfile header for the captured proof.
|
||||
nmbd --daemon --no-process-group
|
||||
# wsdd: WS-Discovery so the box appears in Windows Explorer's Network view.
|
||||
wsdd -i "$IFACE" -4 -H 4 -s -n "$SERVER_NAME" -w WORKGROUP &
|
||||
# dbus + avahi: mDNS, so `smb://<NAME>.local` resolves and the box appears in the Finder sidebar.
|
||||
# Non-fatal on failure: sharing over an address still works, and refusing to start smbd because
|
||||
# a discovery daemon did not come up would turn a convenience gap into an outage.
|
||||
dbus-daemon --system --fork 2>/dev/null || echo "[felhom-samba] WARN: dbus failed to start — mDNS disabled"
|
||||
avahi-daemon --daemonize --no-drop-root 2>/dev/null || echo "[felhom-samba] WARN: avahi failed to start — mDNS disabled"
|
||||
# smbd in the foreground = the container's main process.
|
||||
exec smbd --foreground --no-process-group
|
||||
@@ -1,130 +0,0 @@
|
||||
package agentapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/url"
|
||||
)
|
||||
|
||||
// R-82 Slice B — the per-tier backup surface (agent >= v0.97.0).
|
||||
//
|
||||
// Every method here is ADDITIVE. The untargeted BackupDue/StartBackup/BackupStatus keep their exact
|
||||
// pre-R-82 meaning and are still the single-tier path used against an older agent.
|
||||
|
||||
// ErrTiersUnsupported reports that this agent does not serve GET /backup/tiers — it predates R-82.
|
||||
// It is the DESIGNED capability probe (the route 404s), not a fault. The caller MUST degrade to the
|
||||
// untargeted single-tier path and still take a backup; concluding "nothing to do" from it would
|
||||
// silently stop backups during a fleet rollout.
|
||||
var ErrTiersUnsupported = errors.New("agentapi: agent does not serve /backup/tiers (pre-R-82)")
|
||||
|
||||
// BackupTierInfo is one advertised tier.
|
||||
type BackupTierInfo struct {
|
||||
Target string `json:"target"`
|
||||
CadenceSeconds int64 `json:"cadence_seconds"`
|
||||
Primary bool `json:"primary"`
|
||||
}
|
||||
|
||||
// TiersResponse mirrors the agent's GET /backup/tiers payload.
|
||||
type TiersResponse struct {
|
||||
VMID int `json:"vmid"`
|
||||
Tiers []BackupTierInfo `json:"tiers"`
|
||||
}
|
||||
|
||||
// BackupTiers lists the agent's backup tiers, primary first.
|
||||
// Returns ErrTiersUnsupported (wrapped) on a pre-R-82 agent — key on it with errors.Is.
|
||||
func (c *Client) BackupTiers(ctx context.Context) (TiersResponse, error) {
|
||||
var out TiersResponse
|
||||
body, err := c.get(ctx, "/backup/tiers")
|
||||
if err != nil {
|
||||
var se *StatusError
|
||||
if errors.As(err, &se) && se.Code == http.StatusNotFound {
|
||||
return out, ErrTiersUnsupported
|
||||
}
|
||||
return out, err
|
||||
}
|
||||
if err := json.Unmarshal(body, &out); err != nil {
|
||||
return out, fmt.Errorf("agentapi: decode /backup/tiers: %w", err)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// targetQuery renders the ?target= suffix. An EMPTY target yields an empty string, so the caller
|
||||
// hits the untargeted route byte-for-byte — that is what keeps the pre-R-82 contract intact when
|
||||
// this client talks to an older agent.
|
||||
func targetQuery(target string) string {
|
||||
if target == "" {
|
||||
return ""
|
||||
}
|
||||
return "?target=" + url.QueryEscape(target)
|
||||
}
|
||||
|
||||
// BackupDueFor reports whether THIS TIER is due. A fresh backup on another tier must not satisfy it
|
||||
// — that filtering happens agent-side (latestSuccessfulBackupForTarget); this just asks per tier.
|
||||
func (c *Client) BackupDueFor(ctx context.Context, target string) (DueResponse, error) {
|
||||
var out DueResponse
|
||||
body, err := c.get(ctx, "/backup/due"+targetQuery(target))
|
||||
if err != nil {
|
||||
return out, err
|
||||
}
|
||||
if err := json.Unmarshal(body, &out); err != nil {
|
||||
return out, fmt.Errorf("agentapi: decode /backup/due (target %q): %w", target, err)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// StartBackupFor enqueues a backup of this guest ON THE GIVEN TIER.
|
||||
func (c *Client) StartBackupFor(ctx context.Context, target string) (BackupResponse, error) {
|
||||
var out BackupResponse
|
||||
body, err := c.post(ctx, "/backup"+targetQuery(target), struct{}{})
|
||||
if err != nil {
|
||||
return out, err
|
||||
}
|
||||
if err := json.Unmarshal(body, &out); err != nil {
|
||||
return out, fmt.Errorf("agentapi: decode POST /backup (target %q): %w", target, err)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// BackupStatusFor reports THIS TIER's current/last job phase. Jobs are keyed per tier agent-side,
|
||||
// so polling the wrong target would report a different tier's progress.
|
||||
func (c *Client) BackupStatusFor(ctx context.Context, target string) (StatusResponse, error) {
|
||||
var out StatusResponse
|
||||
body, err := c.get(ctx, "/backup/status"+targetQuery(target))
|
||||
if err != nil {
|
||||
return out, err
|
||||
}
|
||||
if err := json.Unmarshal(body, &out); err != nil {
|
||||
return out, fmt.Errorf("agentapi: decode /backup/status (target %q): %w", target, err)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// SetBackupTargetResponse mirrors POST /backup/target (agent >= v0.113.0).
|
||||
type SetBackupTargetResponse struct {
|
||||
Target string `json:"target"`
|
||||
Where string `json:"where"`
|
||||
// RestartRequired is always true on success: the agent builds its tiers once at daemon start, so
|
||||
// the move needs a restart. The agent deliberately does NOT restart itself — restarting with a
|
||||
// backup in flight cancels the wait and records a spurious tier failure for a backup that actually
|
||||
// succeeded. The RESTART IS THE OPERATOR'S, behind an immediate in-flight check.
|
||||
RestartRequired bool `json:"restart_required"`
|
||||
}
|
||||
|
||||
// SetBackupTarget moves the primary whole-guest backup tier onto the drive at raw host mount `where`.
|
||||
// Creates the storage and grants the agent access as one ordered operation.
|
||||
func (c *Client) SetBackupTarget(ctx context.Context, where string) (SetBackupTargetResponse, error) {
|
||||
var out SetBackupTargetResponse
|
||||
// vmid is deliberately omitted: the agent derives the guest from the token and scopedFromBody
|
||||
// treats an absent vmid as "use the token's" — the same shape as AssignDisk/GuestAttach.
|
||||
body, err := c.post(ctx, "/backup/target", map[string]string{"where": where})
|
||||
if err != nil {
|
||||
return out, err
|
||||
}
|
||||
if err := json.Unmarshal(body, &out); err != nil {
|
||||
return out, fmt.Errorf("agentapi: decode /backup/target: %w", err)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
@@ -16,14 +16,9 @@ import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"log"
|
||||
"net/http"
|
||||
"regexp"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/logx"
|
||||
)
|
||||
|
||||
// Client talks to one agent local-API endpoint with a pinned leaf + bearer token.
|
||||
@@ -31,49 +26,6 @@ type Client struct {
|
||||
baseURL string
|
||||
token string
|
||||
hc *http.Client
|
||||
// features caches capability-probe verdicts for Supports (features.go).
|
||||
features SupportCache
|
||||
// verMu guards lastAgentVersion — the most recent STRICTLY-VALIDATED X-Felhom-Agent-Version
|
||||
// seen on any agent response (v0.82.0 version channel). "" = never seen (pre-0.82 agent) →
|
||||
// Supports falls back to the route probe.
|
||||
verMu sync.Mutex
|
||||
lastAgentVersion string
|
||||
// logger is the optional per-call DEBUG trace sink (v0.116.0 observability — the
|
||||
// capture ring holds these even at logging.level=info). nil = silent (unchanged).
|
||||
logger *log.Logger
|
||||
}
|
||||
|
||||
// SetLogger wires the optional per-call DEBUG trace logger (method, path, status,
|
||||
// duration + agent-version changes — never bodies or tokens).
|
||||
func (c *Client) SetLogger(l *log.Logger) { c.logger = l }
|
||||
|
||||
// reAgentVersion is the bare-semver shape the publish pipeline enforces (publish-agent.sh) — the
|
||||
// ONLY header values trusted for capability comparison. Anything else (garbage, "dev", suffixes)
|
||||
// is ignored and the probe fallback stays in charge.
|
||||
var reAgentVersion = regexp.MustCompile(`^[0-9]+\.[0-9]+\.[0-9]+$`)
|
||||
|
||||
// noteAgentVersion records a response's version header (passive capture — called on EVERY response
|
||||
// path). Invalid/absent headers never overwrite a previously-seen valid version.
|
||||
func (c *Client) noteAgentVersion(resp *http.Response) {
|
||||
v := strings.TrimSpace(resp.Header.Get("X-Felhom-Agent-Version"))
|
||||
if v == "" || !reAgentVersion.MatchString(v) {
|
||||
return
|
||||
}
|
||||
c.verMu.Lock()
|
||||
prev := c.lastAgentVersion
|
||||
c.lastAgentVersion = v
|
||||
c.verMu.Unlock()
|
||||
if prev != v {
|
||||
logx.Debugf(c.logger, "[agentapi] agent version seen: %s (was %q)", v, prev)
|
||||
}
|
||||
}
|
||||
|
||||
// AgentVersion returns the last strictly-validated agent version seen on this client's traffic
|
||||
// ("" = unknown — header-less agent or no traffic yet). This is the Supports comparison source.
|
||||
func (c *Client) AgentVersion() string {
|
||||
c.verMu.Lock()
|
||||
defer c.verMu.Unlock()
|
||||
return c.lastAgentVersion
|
||||
}
|
||||
|
||||
// MountInfo mirrors the agent's GET /storage mount entry (doc 03 §6).
|
||||
@@ -132,26 +84,12 @@ func New(endpoint, token, fingerprintHex string) (*Client, error) {
|
||||
baseURL: "https://" + endpoint,
|
||||
token: token,
|
||||
hc: &http.Client{
|
||||
Timeout: 15 * time.Second,
|
||||
// Bound + expire the idle-conn pool. With the controller reusing one Client (so the pool
|
||||
// stays ~2), IdleConnTimeout also lets idle conns to a RESTARTED agent drain instead of
|
||||
// lingering as stale ESTABLISHED entries, and caps any future per-call misuse. (The earlier
|
||||
// bare Transport had IdleConnTimeout:0 = idle keep-alives never expire → the leak.)
|
||||
Transport: &http.Transport{
|
||||
TLSClientConfig: tlsCfg,
|
||||
MaxIdleConns: 4,
|
||||
MaxIdleConnsPerHost: 2,
|
||||
IdleConnTimeout: 90 * time.Second,
|
||||
},
|
||||
Timeout: 15 * time.Second,
|
||||
Transport: &http.Transport{TLSClientConfig: tlsCfg},
|
||||
},
|
||||
}, nil
|
||||
}
|
||||
|
||||
// Close releases the client's idle keep-alive connections. Optional hygiene for any caller that builds
|
||||
// a short-lived client; the controller reuses one long-lived client, so it relies on the bounded,
|
||||
// expiring idle pool (above) rather than calling this.
|
||||
func (c *Client) Close() { c.hc.CloseIdleConnections() }
|
||||
|
||||
// Storage calls GET /storage and returns this guest's mounts (connectivity + placement view).
|
||||
func (c *Client) Storage(ctx context.Context) (StorageResponse, error) {
|
||||
var out StorageResponse
|
||||
@@ -174,14 +112,6 @@ type DueResponse struct {
|
||||
Due bool `json:"due"`
|
||||
Reason string `json:"reason"`
|
||||
AgeSecs *int64 `json:"age_seconds"`
|
||||
// AgeState (R-88 Part 2, agent >= v0.105.0) says WHY AgeSecs is nil: "absent" (a positive
|
||||
// determination that no backup has ever landed) or "unknown" (the agent could not tell —
|
||||
// unreadable storage, unparseable timestamp). "known" accompanies a real age.
|
||||
//
|
||||
// EMPTY MEANS LEGACY — an agent older than v0.105.0 simply omits the field. It does NOT mean
|
||||
// "unknown", and the distinction is load-bearing: see quiesce.ageStateFromWire. Never
|
||||
// discriminate on Reason instead; those strings are operator copy and will drift.
|
||||
AgeState string `json:"age_state"`
|
||||
}
|
||||
|
||||
// BackupResponse mirrors the agent's POST /backup payload.
|
||||
@@ -204,10 +134,10 @@ type StatusResponse struct {
|
||||
// BackupRecord mirrors the agent's hub.Backup — one whole-guest vzdump/PBS backup result. The
|
||||
// controller renders it read-only (it does NOT own whole-guest backup; the agent does).
|
||||
type BackupRecord struct {
|
||||
TargetID string `json:"target_id"` // backup storage name (e.g. "local", "felhom-pbs")
|
||||
TargetID string `json:"target_id"` // backup storage name (e.g. "local", "felhom-pbs")
|
||||
VMID int `json:"vmid"`
|
||||
Archive string `json:"archive"` // produced vzdump volid (e.g. "local:backup/vzdump-lxc-…")
|
||||
Mode string `json:"mode"` // snapshot | stop
|
||||
Archive string `json:"archive"` // produced vzdump volid (e.g. "local:backup/vzdump-lxc-…")
|
||||
Mode string `json:"mode"` // snapshot | stop
|
||||
CrashConsistent bool `json:"crash_consistent"`
|
||||
SizeBytes int64 `json:"size_bytes"`
|
||||
Success bool `json:"success"`
|
||||
@@ -315,31 +245,6 @@ type DiskInfo struct {
|
||||
// fs UUID (strip the "uuid:" prefix) is the key the controller passes to AssignDisk — it's the
|
||||
// only way the de-privileged controller learns a mount key it cannot read off the device itself.
|
||||
DurableID string `json:"durable_id"`
|
||||
// WipeDurableID is the device's wipe-binding id in the gate's scheme (byid:<wwn>/byuuid:<uuid>) —
|
||||
// the id a customer-confirmed data-bearing wipe must carry (F20-BUG2). DISTINCT from DurableID
|
||||
// (uuid:, used for assign): confirming a wipe with DurableID was rejected (binding_mismatch).
|
||||
WipeDurableID string `json:"wipe_durable_id,omitempty"`
|
||||
// GuestAttached reports whether the drive is actually bound into THIS guest (usable in-guest), as
|
||||
// opposed to merely present on the host (F9) — the signal whose absence let the HDD look available
|
||||
// when it wasn't attached. LEGACY (per-drive mp model); the intermediary model uses BoundUnderParent.
|
||||
GuestAttached bool `json:"guest_attached"`
|
||||
// BackupTarget (E-2, agent >= v0.112.0) reports that this drive backs the PRIMARY whole-guest
|
||||
// backup tier. The agent is the only component that can answer: our own
|
||||
// settings.StoragePath.BackupTarget is customer INTENT, and on a box migrated by hand (E-1) that
|
||||
// intent was never recorded while the drive really IS the target. Absent on an older agent →
|
||||
// false, which degrades to the pre-E-2 behaviour (a generic disconnect alarm, never a wrong one).
|
||||
BackupTarget bool `json:"backup_target,omitempty"`
|
||||
// GuestPath is the drive's STABLE in-guest path in the intermediary-mount model
|
||||
// (/mnt/felhom-drives/<name>) — what the controller registers + repoints HDD_PATH to. Distinct from
|
||||
// MountPath (the raw /mnt/<name> host PVE mount the agent ops on). "" for non-user-data drives.
|
||||
GuestPath string `json:"guest_path,omitempty"`
|
||||
// BoundUnderParent reports whether the drive's felhom-data is currently bound under the shared parent
|
||||
// (live + usable in the guest). The controller's drive-absent gate keys on this + State.
|
||||
BoundUnderParent bool `json:"bound_under_parent"`
|
||||
// Smart is the per-disk SMART health (agent v0.94.0+), nil when the device exposes no SMART or the
|
||||
// agent predates the field — the disk-health card + 6h degradation check feature-detect on this and
|
||||
// render "Nincs adat" (never alarm) when nil. See DiskVerdictFor.
|
||||
Smart *SmartSummary `json:"smart,omitempty"`
|
||||
}
|
||||
|
||||
// FSUUID returns the raw filesystem UUID from a "uuid:<…>" DurableID, or "" if this disk's identity
|
||||
@@ -355,9 +260,6 @@ func (d DiskInfo) FSUUID() string {
|
||||
type DisksResponse struct {
|
||||
VMID int `json:"vmid"`
|
||||
Disks []DiskInfo `json:"disks"`
|
||||
// GuestBootID changes on every guest boot (host or guest reboot) but is stable across a
|
||||
// controller-only restart — the deterministic signal the controller recreates drive-backed apps on.
|
||||
GuestBootID string `json:"guest_boot_id,omitempty"`
|
||||
}
|
||||
|
||||
// FormatResult mirrors POST /disks/format (the success/refusal payload).
|
||||
@@ -413,45 +315,6 @@ func (c *Client) Disks(ctx context.Context) (DisksResponse, error) {
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// DiskCandidate mirrors one entry from the agent's GET /disks/candidates (Impl-2a candidates.go) —
|
||||
// a host disk the agent's unclaimed-disk filter proved is FREE for Felhom to enroll.
|
||||
type DiskCandidate struct {
|
||||
Device string `json:"device"`
|
||||
SizeBytes int64 `json:"size_bytes"`
|
||||
Model string `json:"model,omitempty"`
|
||||
FSType string `json:"fstype,omitempty"`
|
||||
DataBearing bool `json:"data_bearing"`
|
||||
Mountable bool `json:"mountable"`
|
||||
MountSource string `json:"mount_source,omitempty"`
|
||||
DurableID string `json:"durable_id,omitempty"`
|
||||
// AlreadyMounted marks a candidate the CONTROLLER contributed from its own mount table (R-280),
|
||||
// not one the agent scanned. The agent NEVER sets it. Its action is REGISTER the existing
|
||||
// mountpoint — sending it down the device-attach path would try to mount an in-guest path as if
|
||||
// it were a raw device. See web/attach_sources.go for why the agent's scan cannot supply these.
|
||||
AlreadyMounted bool `json:"already_mounted,omitempty"`
|
||||
}
|
||||
|
||||
// CandidatesResult mirrors GET /disks/candidates: disks free to enroll, split into initialize (all
|
||||
// unclaimed) and attach (the mountable-FS subset).
|
||||
type CandidatesResult struct {
|
||||
VMID int `json:"vmid"`
|
||||
Initialize []DiskCandidate `json:"initialize"`
|
||||
Attach []DiskCandidate `json:"attach"`
|
||||
}
|
||||
|
||||
// ListCandidates fetches the host disks free for Felhom to enroll (Impl-2b wizard source).
|
||||
func (c *Client) ListCandidates(ctx context.Context) (CandidatesResult, error) {
|
||||
var out CandidatesResult
|
||||
body, err := c.get(ctx, "/disks/candidates")
|
||||
if err != nil {
|
||||
return out, err
|
||||
}
|
||||
if err := json.Unmarshal(body, &out); err != nil {
|
||||
return out, fmt.Errorf("agentapi: decode /disks/candidates: %w", err)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// AssignDisk attaches a drive (by fs-UUID) as a host mount (benign, self-serve).
|
||||
func (c *Client) AssignDisk(ctx context.Context, uuid, where, fstype, options string) error {
|
||||
_, err := c.post(ctx, "/disks/assign", map[string]string{
|
||||
@@ -477,148 +340,6 @@ func (c *Client) GuestReboot(ctx context.Context) error {
|
||||
return err
|
||||
}
|
||||
|
||||
// ---- v0.143.0: guest RAM resize (R-24, agent ≥ 0.90.0) ------------------------------------
|
||||
|
||||
// GuestMemoryInfo mirrors the agent's GET /guest/memory (every field MB, agent-computed). The
|
||||
// min/max/floor are the CURRENTLY-enforced bounds — the UI renders them but the agent re-checks fresh.
|
||||
type GuestMemoryInfo struct {
|
||||
VMID int `json:"vmid"`
|
||||
AllocatedMB int64 `json:"allocated_mb"`
|
||||
UsageMB int64 `json:"usage_mb"`
|
||||
HostTotalMB int64 `json:"host_total_mb"`
|
||||
MinMB int64 `json:"min_mb"`
|
||||
MaxMB int64 `json:"max_mb"`
|
||||
FloorMB int64 `json:"floor_mb"`
|
||||
Running bool `json:"running"`
|
||||
}
|
||||
|
||||
// MemoryResizeResult mirrors the agent's POST /guest/memory success body.
|
||||
type MemoryResizeResult struct {
|
||||
VMID int `json:"vmid"`
|
||||
OldMB int64 `json:"old_mb"`
|
||||
NewMB int64 `json:"new_mb"`
|
||||
Unchanged bool `json:"unchanged"`
|
||||
}
|
||||
|
||||
// MemoryRefusedError carries the agent's machine refusal code (below_min | above_max |
|
||||
// below_usage_floor) plus the fresh bounds, so the web layer maps it to a Hungarian message and
|
||||
// re-renders the range honestly — the agent's English message is never shown raw.
|
||||
type MemoryRefusedError struct {
|
||||
Code string
|
||||
Bounds GuestMemoryInfo
|
||||
Msg string
|
||||
}
|
||||
|
||||
func (e *MemoryRefusedError) Error() string {
|
||||
return "agentapi: memory resize refused (" + e.Code + "): " + e.Msg
|
||||
}
|
||||
|
||||
// GuestMemory reads the guest's current allocation, live usage, and the enforced bounds. A pre-0.90
|
||||
// agent has no such route → the get helper returns *StatusError{404} (the capability probe signal
|
||||
// and the UI's "needs an update" path).
|
||||
func (c *Client) GuestMemory(ctx context.Context) (GuestMemoryInfo, error) {
|
||||
var out GuestMemoryInfo
|
||||
data, err := c.get(ctx, "/guest/memory")
|
||||
if err != nil {
|
||||
return out, err
|
||||
}
|
||||
if err := json.Unmarshal(data, &out); err != nil {
|
||||
return out, fmt.Errorf("agentapi: decode /guest/memory: %w", err)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// ResizeMemory requests a bounded resize. The agent enforces every bound; a ruled refusal (412)
|
||||
// returns *MemoryRefusedError carrying the code + fresh bounds; a non-coded failure (e.g. the 502
|
||||
// verify-after-apply) returns a plain error; success returns old→new.
|
||||
func (c *Client) ResizeMemory(ctx context.Context, memoryMB int64) (MemoryResizeResult, error) {
|
||||
var out MemoryResizeResult
|
||||
env, status, err := c.postWithStatus(ctx, "/guest/memory", map[string]int64{"memory_mb": memoryMB})
|
||||
if err != nil {
|
||||
return out, err
|
||||
}
|
||||
if status == http.StatusOK && env.OK {
|
||||
if len(env.Data) > 0 {
|
||||
_ = json.Unmarshal(env.Data, &out)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
// Refusal — the data carries {code, ...fresh bounds}.
|
||||
var ref struct {
|
||||
Code string `json:"code"`
|
||||
AllocatedMB int64 `json:"allocated_mb"`
|
||||
UsageMB int64 `json:"usage_mb"`
|
||||
HostTotalMB int64 `json:"host_total_mb"`
|
||||
MinMB int64 `json:"min_mb"`
|
||||
MaxMB int64 `json:"max_mb"`
|
||||
FloorMB int64 `json:"floor_mb"`
|
||||
}
|
||||
if len(env.Data) > 0 {
|
||||
_ = json.Unmarshal(env.Data, &ref)
|
||||
}
|
||||
if ref.Code != "" {
|
||||
return out, &MemoryRefusedError{
|
||||
Code: ref.Code,
|
||||
Bounds: GuestMemoryInfo{
|
||||
AllocatedMB: ref.AllocatedMB, UsageMB: ref.UsageMB, HostTotalMB: ref.HostTotalMB,
|
||||
MinMB: ref.MinMB, MaxMB: ref.MaxMB, FloorMB: ref.FloorMB,
|
||||
},
|
||||
Msg: truncateErr(env.Error, 300),
|
||||
}
|
||||
}
|
||||
if rerr := refusalError("/guest/memory", status, env); rerr != nil {
|
||||
return out, rerr
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// SwapResult mirrors the agent's 202 from POST /controller/swap (agentic controller update, Phase 1).
|
||||
type SwapResult struct {
|
||||
Status string `json:"status"` // "swapping"
|
||||
PreviousImage string `json:"previous_image"` // image before the swap (for the UI/log)
|
||||
TargetImage string `json:"target_image"`
|
||||
}
|
||||
|
||||
// SwapController asks the agent to swap the in-guest controller to `image` (which the controller has
|
||||
// already pulled into the guest's docker storage). The agent responds 202 and performs the swap+verify
|
||||
// +rollback asynchronously, EXTERNALLY to this controller container (so it survives this process being
|
||||
// killed by the swap). Latest-only is enforced by the caller (queryRegistry); the agent re-validates
|
||||
// the ref shape.
|
||||
func (c *Client) SwapController(ctx context.Context, image string) (SwapResult, error) {
|
||||
var out SwapResult
|
||||
body, err := c.post(ctx, "/controller/swap", map[string]string{"image": image})
|
||||
if err != nil {
|
||||
return out, err
|
||||
}
|
||||
if err := json.Unmarshal(body, &out); err != nil {
|
||||
return out, fmt.Errorf("agentapi: decode /controller/swap: %w", err)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// SwapStatus mirrors the agent's GET /controller/swap/status (observability for the post-restart UI).
|
||||
type SwapStatus struct {
|
||||
State string `json:"state"` // none | swapping | done | failed
|
||||
InFlight bool `json:"in_flight"`
|
||||
Current string `json:"current"`
|
||||
Previous string `json:"previous"`
|
||||
Target string `json:"target"`
|
||||
Error string `json:"error"`
|
||||
}
|
||||
|
||||
// SwapStatus reads the last/in-flight swap outcome for this guest.
|
||||
func (c *Client) SwapStatus(ctx context.Context) (SwapStatus, error) {
|
||||
var out SwapStatus
|
||||
body, err := c.get(ctx, "/controller/swap/status")
|
||||
if err != nil {
|
||||
return out, err
|
||||
}
|
||||
if err := json.Unmarshal(body, &out); err != nil {
|
||||
return out, fmt.Errorf("agentapi: decode /controller/swap/status: %w", err)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// EjectResult mirrors POST /disks/eject (the dependent-guest warning).
|
||||
type EjectResult struct {
|
||||
VMID int `json:"vmid"`
|
||||
@@ -627,110 +348,18 @@ type EjectResult struct {
|
||||
}
|
||||
|
||||
// EjectDisk safe-unmounts a host mount (data preserved) and returns the dependent guests.
|
||||
// Status-aware POST (campaign F2 evidence gap): the agent's refusal body carries the reason
|
||||
// (e.g. "…eject refused (role: system)") — surface it instead of a bare "HTTP 403".
|
||||
// StageEscrowSecret pushes the offsite restic repo password to the agent (fork-4), which stages it
|
||||
// transiently for the escrow-create ceremony to wrap under the customer recovery code R. The value is
|
||||
// sent over the authenticated pinned local-API channel; the CALLER must never log it.
|
||||
func (c *Client) StageEscrowSecret(ctx context.Context, resticRepoPassword string) error {
|
||||
env, status, err := c.postWithStatus(ctx, "/escrow/stage-secret", map[string]string{"restic_repo_password": resticRepoPassword})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return refusalError("/escrow/stage-secret", status, env)
|
||||
}
|
||||
|
||||
// WipeStagedEscrowSecret removes the agent-staged offsite repo password (fork-4 hygiene) — called whenever
|
||||
// EscrowState flips to escrowed, so the transient 0600 staging file doesn't outlive its purpose. Idempotent
|
||||
// on the agent side (an absent file is a clean 200). Requires agent >= v0.78.0 (older agents 404 — the
|
||||
// caller logs loudly and moves on).
|
||||
func (c *Client) WipeStagedEscrowSecret(ctx context.Context) error {
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodDelete, c.baseURL+"/escrow/stage-secret", nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
req.Header.Set("Authorization", "Bearer "+c.token)
|
||||
resp, err := c.hc.Do(req)
|
||||
if err != nil {
|
||||
return fmt.Errorf("agentapi: DELETE /escrow/stage-secret: %w", err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
c.noteAgentVersion(resp) // v0.82.0 version channel: passive capture on EVERY response
|
||||
raw, _ := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
|
||||
var env apiResponse
|
||||
if err := json.Unmarshal(raw, &env); err != nil {
|
||||
return fmt.Errorf("agentapi: DELETE /escrow/stage-secret: HTTP %d, bad envelope: %w", resp.StatusCode, err)
|
||||
}
|
||||
return refusalError("/escrow/stage-secret", resp.StatusCode, env)
|
||||
}
|
||||
|
||||
func (c *Client) EjectDisk(ctx context.Context, where string) (EjectResult, error) {
|
||||
var out EjectResult
|
||||
env, status, err := c.postWithStatus(ctx, "/disks/eject", map[string]string{"where": where})
|
||||
body, err := c.post(ctx, "/disks/eject", map[string]string{"where": where})
|
||||
if err != nil {
|
||||
return out, err
|
||||
}
|
||||
if err := refusalError("/disks/eject", status, env); err != nil {
|
||||
return out, err
|
||||
}
|
||||
if err := json.Unmarshal(env.Data, &out); err != nil {
|
||||
if err := json.Unmarshal(body, &out); err != nil {
|
||||
return out, fmt.Errorf("agentapi: decode /disks/eject: %w", err)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// DecommissionResult mirrors POST /disks/decommission.
|
||||
type DecommissionResult struct {
|
||||
VMID int `json:"vmid"`
|
||||
Decommissioned string `json:"decommissioned"`
|
||||
DependentGuests []int `json:"dependent_guests"`
|
||||
}
|
||||
|
||||
// Decommission permanently removes a user-data drive (self-serve, non-destructive — the agent records
|
||||
// IntentDecommissioned, prunes the bind record, and unmounts; it NEVER formats). Data stays on the
|
||||
// drive. The agent role-gates to user-data and refuses a system/backup mount regardless.
|
||||
// Status-aware POST (campaign F2 evidence gap): the agent's refusal body carries the reason
|
||||
// (e.g. "…decommission refused (role: system)") — surface it instead of a bare "HTTP 403".
|
||||
func (c *Client) Decommission(ctx context.Context, where string) (DecommissionResult, error) {
|
||||
var out DecommissionResult
|
||||
env, status, err := c.postWithStatus(ctx, "/disks/decommission", map[string]string{"where": where})
|
||||
if err != nil {
|
||||
return out, err
|
||||
}
|
||||
if err := refusalError("/disks/decommission", status, env); err != nil {
|
||||
return out, err
|
||||
}
|
||||
if err := json.Unmarshal(env.Data, &out); err != nil {
|
||||
return out, fmt.Errorf("agentapi: decode /disks/decommission: %w", err)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// refusalError converts a non-2xx status or an ok:false envelope into an error that CARRIES the
|
||||
// agent's reason (truncated; never request bodies or secrets). nil on an accepted 200/202+ok=true.
|
||||
func refusalError(path string, status int, env apiResponse) error {
|
||||
accepted := status == http.StatusOK || status == http.StatusAccepted
|
||||
if accepted && env.OK {
|
||||
return nil
|
||||
}
|
||||
reason := truncateErr(env.Error, 300)
|
||||
if reason == "" {
|
||||
reason = "(no reason in agent response)"
|
||||
}
|
||||
if accepted { // 2xx but ok:false — business refusal without an HTTP error code
|
||||
return fmt.Errorf("agentapi: POST %s: %s", path, reason)
|
||||
}
|
||||
return fmt.Errorf("agentapi: POST %s: HTTP %d: %s", path, status, reason)
|
||||
}
|
||||
|
||||
// truncateErr mirrors stacks.truncateStr for agent refusal reasons.
|
||||
func truncateErr(s string, maxLen int) string {
|
||||
if len(s) <= maxLen {
|
||||
return s
|
||||
}
|
||||
return s[:maxLen] + "..."
|
||||
}
|
||||
|
||||
// FormatDisk asks the agent to format a device. The AGENT inspects the device and tiers it by ROLE
|
||||
// (its own classification, never the controller's claim):
|
||||
// - blank device → formatted.
|
||||
@@ -744,15 +373,15 @@ func (c *Client) FormatDisk(ctx context.Context, device, fstype string, confirme
|
||||
var out FormatResult
|
||||
// Status-aware POST: the agent returns the FULL FormatResponse (incl. pending_op / durable_id)
|
||||
// even on the 403 refusal, so we must read the body on non-2xx rather than discarding it.
|
||||
env, status, err := c.postWithStatus(ctx, "/disks/format", map[string]any{
|
||||
data, status, err := c.postWithStatus(ctx, "/disks/format", map[string]any{
|
||||
"device": device, "fstype": fstype, "confirmed": confirmed, "durable_id": durableID,
|
||||
})
|
||||
if err != nil {
|
||||
return out, err
|
||||
}
|
||||
// env.Data is the envelope's {data:…} payload (present on both success and the 403 refusal).
|
||||
if len(env.Data) > 0 {
|
||||
_ = json.Unmarshal(env.Data, &out) // best-effort; fields default on a missing/partial body
|
||||
// data is the envelope's {data:…} payload (present on both success and the 403 refusal).
|
||||
if len(data) > 0 {
|
||||
_ = json.Unmarshal(data, &out) // best-effort; fields default on a missing/partial body
|
||||
}
|
||||
if out.Formatted {
|
||||
return out, nil
|
||||
@@ -765,241 +394,34 @@ func (c *Client) FormatDisk(ctx context.Context, device, fstype string, confirme
|
||||
out.DataBearing = true
|
||||
return out, ErrFormatRefused // system/backup: surface the opsign command
|
||||
}
|
||||
// F20-BUG1: a non-2xx response (or ok:false) that is NOT one of the recognized refusals above is a
|
||||
// real failure (e.g. the agent's 502 on a mkfs error: "device is mounted"). Returning the zero-value
|
||||
// result with a nil error here made a failed destructive format read as a silent SUCCESS in the web
|
||||
// layer. Surface it as an error so the caller (and the dashboard) report the failure.
|
||||
if status < 200 || status >= 300 || !env.OK {
|
||||
msg := strings.TrimSpace(env.Error)
|
||||
if msg == "" {
|
||||
msg = "format failed"
|
||||
}
|
||||
return out, fmt.Errorf("agentapi: format: HTTP %d: %s", status, msg)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// FormatStatusResult mirrors GET /disks/format/status (F20-BUG3): the most-recent / in-flight format
|
||||
// job on the host. Phase ∈ idle | running | done | failed. The drive-init flow polls this to follow a
|
||||
// mkfs that outran the 15 s client timeout — the agent runs the mkfs DETACHED and keeps the record, so
|
||||
// the client can learn the real outcome instead of assuming failure (F6).
|
||||
type FormatStatusResult struct {
|
||||
Phase string `json:"phase"`
|
||||
Device string `json:"device"`
|
||||
FSType string `json:"fstype"`
|
||||
Error string `json:"error"`
|
||||
}
|
||||
|
||||
// FormatStatus fetches the agent's most-recent format-job record.
|
||||
func (c *Client) FormatStatus(ctx context.Context) (FormatStatusResult, error) {
|
||||
var out FormatStatusResult
|
||||
body, err := c.get(ctx, "/disks/format/status")
|
||||
if err != nil {
|
||||
return out, err
|
||||
}
|
||||
if err := json.Unmarshal(body, &out); err != nil {
|
||||
return out, fmt.Errorf("agentapi: decode /disks/format/status: %w", err)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// ---- NAS network storage (Part A2 → agent A1 /netstorage/*) ------------------------------
|
||||
//
|
||||
// A NAS share is a DISTINCT storage class from a drive: the controller proxies add/list/remove to the
|
||||
// agent (which owns the host-side automount), holds NO mount authority, and persists NO SMB password
|
||||
// (it passes the credentials straight through to the agent's add request — the agent writes the 0600
|
||||
// creds file). There is NO eject/decommission/migrate/wipe/SMART here — those are drive-only.
|
||||
|
||||
// NetworkMountStatus mirrors the agent's GET /netstorage entry (A1). health ∈ {ok, idle, unreachable}:
|
||||
// `idle` (reachable + automount idle-unmounted) is BENIGN, not a fault; only `unreachable` is degraded.
|
||||
type NetworkMountStatus struct {
|
||||
Name string `json:"name"`
|
||||
Protocol string `json:"protocol"`
|
||||
Server string `json:"server"`
|
||||
Export string `json:"export"`
|
||||
Where string `json:"where"`
|
||||
Configured bool `json:"configured"`
|
||||
Mounted bool `json:"mounted"`
|
||||
Reachable bool `json:"reachable"`
|
||||
Health string `json:"health"` // ok | idle | unreachable
|
||||
}
|
||||
|
||||
// Unreachable reports the degraded state (NAS not reachable). `idle` is explicitly NOT unreachable — an
|
||||
// idle-unmounted automount is the normal steady state, never a warning.
|
||||
func (n NetworkMountStatus) Unreachable() bool { return n.Health == "unreachable" }
|
||||
|
||||
// AddNetStorageRequest is the controller→agent POST /netstorage/add body (A1). Username/Password are SMB
|
||||
// only and flow STRAIGHT THROUGH to the agent (which writes the 0600 creds file) — the controller never
|
||||
// stores the password at rest.
|
||||
type AddNetStorageRequest struct {
|
||||
Name string `json:"name"`
|
||||
Protocol string `json:"protocol"` // nfs | smb
|
||||
Server string `json:"server"`
|
||||
Export string `json:"export"`
|
||||
MappedUID int `json:"mapped_uid"`
|
||||
MappedGID int `json:"mapped_gid"`
|
||||
IdleTimeoutSec int `json:"idle_timeout_sec,omitempty"`
|
||||
Username string `json:"username,omitempty"` // SMB secret — pass-through, never persisted
|
||||
Password string `json:"password,omitempty"` // SMB secret — pass-through, never persisted
|
||||
}
|
||||
|
||||
// NetStorageAddResult mirrors the agent's add response (the in-guest path the media app's data dir
|
||||
// points at). Since agent v0.81.0 (verify-before-commit) a successful add means "units installed,
|
||||
// verify STARTED" — Verify/JobID carry the detached verify job the caller polls via NetVerifyStatus.
|
||||
type NetStorageAddResult struct {
|
||||
Name string `json:"name"`
|
||||
Protocol string `json:"protocol"`
|
||||
Where string `json:"where"`
|
||||
GuestPath string `json:"guest_path"`
|
||||
HostUID int `json:"host_uid"`
|
||||
HostGID int `json:"host_gid"`
|
||||
Verify string `json:"verify"` // "started" on a verify-before-commit agent (v0.81.0+)
|
||||
JobID string `json:"job_id"`
|
||||
Code string `json:"code"` // set on a categorized SYNC refusal (e.g. "unreachable", "busy")
|
||||
}
|
||||
|
||||
// NetAddRefusedError is the agent's CATEGORIZED sync refusal of a netstorage add (the 2 s TCP
|
||||
// pre-probe "unreachable", or "busy" single-flight). Code is the verify-category vocabulary the UI
|
||||
// maps to Hungarian; nothing was installed agent-side.
|
||||
type NetAddRefusedError struct {
|
||||
Code string
|
||||
Msg string
|
||||
}
|
||||
|
||||
func (e *NetAddRefusedError) Error() string {
|
||||
return "agentapi: netstorage add refused (" + e.Code + "): " + e.Msg
|
||||
}
|
||||
|
||||
// NetVerifyStatus mirrors the agent's GET /netstorage/verify-status: the single verify slot.
|
||||
// Phase "none" is a REAL signal — after an agent restart the in-memory job is gone; the caller
|
||||
// treats none-after-install as verify-lost and rolls the add back (Scenario F).
|
||||
type NetVerifyStatus struct {
|
||||
Phase string `json:"phase"` // none | running | done | failed
|
||||
Name string `json:"name"`
|
||||
Where string `json:"where"`
|
||||
Protocol string `json:"protocol"`
|
||||
Code string `json:"code"` // failure category (the agent's classifier vocabulary)
|
||||
Detail string `json:"detail"` // operator hint + raw journal fragment
|
||||
JobID string `json:"job_id"`
|
||||
}
|
||||
|
||||
// AddNetStorage installs a NAS share host-side and starts the agent's detached verify job (agent
|
||||
// v0.81.0 verify-before-commit). A categorized sync refusal returns *NetAddRefusedError carrying
|
||||
// the category code (the result body also carries it); other failures are plain errors.
|
||||
func (c *Client) AddNetStorage(ctx context.Context, req AddNetStorageRequest) (NetStorageAddResult, error) {
|
||||
var out NetStorageAddResult
|
||||
env, status, err := c.postWithStatus(ctx, "/netstorage/add", req)
|
||||
if err != nil {
|
||||
return out, err
|
||||
}
|
||||
if len(env.Data) > 0 {
|
||||
_ = json.Unmarshal(env.Data, &out) // best-effort; the refusal body carries {code}
|
||||
}
|
||||
if rerr := refusalError("/netstorage/add", status, env); rerr != nil {
|
||||
if out.Code != "" {
|
||||
return out, &NetAddRefusedError{Code: out.Code, Msg: truncateErr(env.Error, 300)}
|
||||
}
|
||||
return out, rerr
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// NetVerifyStatus polls the agent's verify slot (short GET — fits the client's global 15 s timeout;
|
||||
// the LONG wait lives in the caller's poll loop, never in one HTTP call).
|
||||
func (c *Client) NetVerifyStatus(ctx context.Context) (NetVerifyStatus, error) {
|
||||
var out NetVerifyStatus
|
||||
body, err := c.get(ctx, "/netstorage/verify-status")
|
||||
if err != nil {
|
||||
return out, err
|
||||
}
|
||||
if err := json.Unmarshal(body, &out); err != nil {
|
||||
return out, fmt.Errorf("agentapi: decode /netstorage/verify-status: %w", err)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// ListNetStorage returns the configured NAS shares + per-share liveness.
|
||||
func (c *Client) ListNetStorage(ctx context.Context) ([]NetworkMountStatus, error) {
|
||||
body, err := c.get(ctx, "/netstorage")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var wrap struct {
|
||||
NetworkMounts []NetworkMountStatus `json:"network_mounts"`
|
||||
}
|
||||
if err := json.Unmarshal(body, &wrap); err != nil {
|
||||
return nil, fmt.Errorf("agentapi: decode /netstorage: %w", err)
|
||||
}
|
||||
return wrap.NetworkMounts, nil
|
||||
}
|
||||
|
||||
// RemoveNetStorage unmounts + removes a NAS share (the agent drops the mount + creds file). This is NOT a
|
||||
// drive decommission/migrate — a NAS has no device lifecycle.
|
||||
func (c *Client) RemoveNetStorage(ctx context.Context, name string) error {
|
||||
_, err := c.post(ctx, "/netstorage/remove", map[string]string{"name": name})
|
||||
return err
|
||||
}
|
||||
|
||||
// postWithStatus issues an authenticated JSON POST and returns the envelope's data payload + the HTTP
|
||||
// status, even on a non-2xx (so callers like FormatDisk can read a 403 refusal body). A transport or
|
||||
// envelope-parse failure is still an error; an `ok:false` business refusal is NOT (the data carries it).
|
||||
func (c *Client) postWithStatus(ctx context.Context, path string, body any) (apiResponse, int, error) {
|
||||
var env apiResponse
|
||||
func (c *Client) postWithStatus(ctx context.Context, path string, body any) (json.RawMessage, int, error) {
|
||||
buf, err := json.Marshal(body)
|
||||
if err != nil {
|
||||
return env, 0, err
|
||||
return nil, 0, err
|
||||
}
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPost, c.baseURL+path, bytes.NewReader(buf))
|
||||
if err != nil {
|
||||
return env, 0, err
|
||||
return nil, 0, err
|
||||
}
|
||||
req.Header.Set("Authorization", "Bearer "+c.token)
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
start := time.Now()
|
||||
resp, err := c.hc.Do(req)
|
||||
if err != nil {
|
||||
logx.Debugf(c.logger, "[agentapi] POST %s failed after %dms: %v", path, time.Since(start).Milliseconds(), err)
|
||||
return env, 0, fmt.Errorf("agentapi: POST %s: %w", path, err)
|
||||
return nil, 0, fmt.Errorf("agentapi: POST %s: %w", path, err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
c.noteAgentVersion(resp) // v0.82.0 version channel: passive capture on EVERY response
|
||||
logx.Debugf(c.logger, "[agentapi] POST %s -> %d (%dms)", path, resp.StatusCode, time.Since(start).Milliseconds())
|
||||
raw, _ := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
|
||||
var env apiResponse
|
||||
if err := json.Unmarshal(raw, &env); err != nil {
|
||||
return env, resp.StatusCode, fmt.Errorf("agentapi: POST %s: HTTP %d, bad envelope: %w", path, resp.StatusCode, err)
|
||||
return nil, resp.StatusCode, fmt.Errorf("agentapi: POST %s: HTTP %d, bad envelope: %w", path, resp.StatusCode, err)
|
||||
}
|
||||
return env, resp.StatusCode, nil
|
||||
}
|
||||
|
||||
// ---- v0.116.0: agent debug-log ring (the Debug page agent tab) ----------------------------
|
||||
|
||||
// AgentLogEntry mirrors the agent's GET /debug/logs entry (agent ≥ 0.83.0).
|
||||
type AgentLogEntry struct {
|
||||
Timestamp time.Time `json:"timestamp"`
|
||||
Level string `json:"level"`
|
||||
Message string `json:"message"`
|
||||
}
|
||||
|
||||
// AgentLogsResponse mirrors the agent's GET /debug/logs data payload.
|
||||
type AgentLogsResponse struct {
|
||||
VMID int `json:"vmid"`
|
||||
Entries []AgentLogEntry `json:"entries"`
|
||||
Total int `json:"total"`
|
||||
}
|
||||
|
||||
// DebugLogs fetches the agent's always-DEBUG capture ring. Against a pre-0.83
|
||||
// agent the route is absent → a typed *StatusError with Code 404 (the caller
|
||||
// renders the "available after the agent's next update" notice — S6).
|
||||
func (c *Client) DebugLogs(ctx context.Context) (AgentLogsResponse, error) {
|
||||
var out AgentLogsResponse
|
||||
data, err := c.get(ctx, "/debug/logs")
|
||||
if err != nil {
|
||||
return out, err
|
||||
}
|
||||
if err := json.Unmarshal(data, &out); err != nil {
|
||||
return out, fmt.Errorf("agentapi: parsing debug logs: %w", err)
|
||||
}
|
||||
return out, nil
|
||||
return env.Data, resp.StatusCode, nil
|
||||
}
|
||||
|
||||
// ---- slice 9: host metrics (the customer host-health view) -------------------------------
|
||||
@@ -1026,32 +448,14 @@ type ThinPoolFill struct {
|
||||
MetadataUsedFraction *float64 `json:"metadata_used_fraction"`
|
||||
}
|
||||
|
||||
// SmartSummary mirrors the agent's per-disk SMART health. Pointers are null when the device type
|
||||
// does not expose that attribute (a null is "unknown / not-applicable", distinct from a real zero).
|
||||
// The SATA set (reallocated/pending/offline-uncorrectable) and the NVMe set
|
||||
// (critical_warning/media_errors/percentage_used) are both carried; a device populates only its own.
|
||||
// SmartSummary mirrors the agent's per-disk SMART health (only the fields the UI renders). Pointers
|
||||
// are null when the device type does not expose that attribute.
|
||||
type SmartSummary struct {
|
||||
Health string `json:"health"` // PASSED | FAILING | UNKNOWN
|
||||
ModelName *string `json:"model_name,omitempty"` // smartctl device model (agent v0.95.0+); nil on older agents
|
||||
TemperatureC *int `json:"temperature_c"`
|
||||
PowerOnHours *int `json:"power_on_hours"`
|
||||
// SATA attributes.
|
||||
ReallocatedSectors *int `json:"reallocated_sectors"`
|
||||
PendingSectors *int `json:"pending_sectors"`
|
||||
OfflineUncorrectable *int `json:"offline_uncorrectable"`
|
||||
// NVMe attributes.
|
||||
CriticalWarning *int `json:"critical_warning"`
|
||||
MediaErrors *int `json:"media_errors"`
|
||||
PercentageUsed *int `json:"percentage_used"` // NVMe wear (%); null for SATA/USB
|
||||
Health string `json:"health"` // PASSED | FAILING | UNKNOWN
|
||||
TemperatureC *int `json:"temperature_c"`
|
||||
PercentageUsed *int `json:"percentage_used"` // NVMe wear (%); null for SATA/USB
|
||||
}
|
||||
|
||||
// SMART health vocabulary (mirrors the agent's).
|
||||
const (
|
||||
SmartPassed = "PASSED"
|
||||
SmartFailing = "FAILING"
|
||||
SmartUnknown = "UNKNOWN"
|
||||
)
|
||||
|
||||
// StorageTarget mirrors the agent's GET /host/metrics storage_targets entry (the per-storage
|
||||
// capacity + health the monitoring view renders). It is a SUBSET of the agent's wire shape — only
|
||||
// the fields the UI reads; unknown JSON keys are ignored.
|
||||
@@ -1069,11 +473,6 @@ type StorageTarget struct {
|
||||
ClassHint string `json:"class_hint"`
|
||||
ThinPool *ThinPoolFill `json:"thin_pool,omitempty"`
|
||||
Smart SmartSummary `json:"smart"`
|
||||
// Label and Purpose are controller-side display enrichment (NOT from the agent): a friendly
|
||||
// Hungarian name + one-line purpose so the customer understands what each storage holds. The
|
||||
// raw PVE storage id stays in Name (display-only labels — we never rename the actual storage).
|
||||
Label string `json:"label,omitempty"`
|
||||
Purpose string `json:"purpose,omitempty"`
|
||||
}
|
||||
|
||||
// HostMetricsResponse mirrors the agent's GET /host/metrics payload (host-wide health + per-storage
|
||||
@@ -1097,33 +496,6 @@ func (c *Client) HostMetrics(ctx context.Context) (HostMetricsResponse, error) {
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// StatusError is a non-2xx agent HTTP status surfaced as a TYPED error (same text the old
|
||||
// fmt.Errorf produced). errors.As-able — the capability probe (features.go) keys on Code 404 to
|
||||
// distinguish "this agent predates the route" from every other failure. Never match the string.
|
||||
// StatusError is a non-2xx response from the agent, carrying the STATUS CODE so callers can react
|
||||
// to specific ones rather than string-matching an error message.
|
||||
//
|
||||
// F-A1: this exists on the POST path because HTTP 409 from `POST /backup` is not a failure — it is
|
||||
// the agent's R-85 single-flight gate correctly refusing while a restore-test holds it. Treating
|
||||
// that refusal as a tier failure armed the breaker and emailed the operator about a backup that was
|
||||
// never actually broken. The controller now needs to tell 409 apart from a real error, and a typed
|
||||
// code is the only honest way to do that.
|
||||
type StatusError struct {
|
||||
// Method is the HTTP method. Empty means GET, so the message stays byte-identical for the
|
||||
// pre-existing GET call sites.
|
||||
Method string
|
||||
Path string
|
||||
Code int
|
||||
}
|
||||
|
||||
func (e *StatusError) Error() string {
|
||||
m := e.Method
|
||||
if m == "" {
|
||||
m = http.MethodGet
|
||||
}
|
||||
return fmt.Sprintf("agentapi: %s %s: HTTP %d", m, e.Path, e.Code)
|
||||
}
|
||||
|
||||
// get issues an authenticated GET and unwraps the {ok,data,error} envelope.
|
||||
func (c *Client) get(ctx context.Context, path string) (json.RawMessage, error) {
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, c.baseURL+path, nil)
|
||||
@@ -1131,18 +503,14 @@ func (c *Client) get(ctx context.Context, path string) (json.RawMessage, error)
|
||||
return nil, err
|
||||
}
|
||||
req.Header.Set("Authorization", "Bearer "+c.token)
|
||||
start := time.Now()
|
||||
resp, err := c.hc.Do(req)
|
||||
if err != nil {
|
||||
logx.Debugf(c.logger, "[agentapi] GET %s failed after %dms: %v", path, time.Since(start).Milliseconds(), err)
|
||||
return nil, fmt.Errorf("agentapi: GET %s: %w", path, err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
c.noteAgentVersion(resp) // v0.82.0 version channel: passive capture on EVERY response
|
||||
logx.Debugf(c.logger, "[agentapi] GET %s -> %d (%dms)", path, resp.StatusCode, time.Since(start).Milliseconds())
|
||||
raw, _ := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
return nil, &StatusError{Path: path, Code: resp.StatusCode}
|
||||
return nil, fmt.Errorf("agentapi: GET %s: HTTP %d", path, resp.StatusCode)
|
||||
}
|
||||
var env apiResponse
|
||||
if err := json.Unmarshal(raw, &env); err != nil {
|
||||
@@ -1167,20 +535,14 @@ func (c *Client) post(ctx context.Context, path string, body any) (json.RawMessa
|
||||
}
|
||||
req.Header.Set("Authorization", "Bearer "+c.token)
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
start := time.Now()
|
||||
resp, err := c.hc.Do(req)
|
||||
if err != nil {
|
||||
logx.Debugf(c.logger, "[agentapi] POST %s failed after %dms: %v", path, time.Since(start).Milliseconds(), err)
|
||||
return nil, fmt.Errorf("agentapi: POST %s: %w", path, err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
c.noteAgentVersion(resp) // v0.82.0 version channel: passive capture on EVERY response
|
||||
logx.Debugf(c.logger, "[agentapi] POST %s -> %d (%dms)", path, resp.StatusCode, time.Since(start).Milliseconds())
|
||||
raw, _ := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
|
||||
if resp.StatusCode != http.StatusOK && resp.StatusCode != http.StatusAccepted {
|
||||
// Typed, not fmt.Errorf: callers must be able to distinguish 409 (the agent's single-flight
|
||||
// gate refusing — contention, not failure) from a genuine 5xx. See StatusError.
|
||||
return nil, &StatusError{Method: http.MethodPost, Path: path, Code: resp.StatusCode}
|
||||
return nil, fmt.Errorf("agentapi: POST %s: HTTP %d", path, resp.StatusCode)
|
||||
}
|
||||
var env apiResponse
|
||||
if err := json.Unmarshal(raw, &env); err != nil {
|
||||
|
||||
@@ -1,30 +0,0 @@
|
||||
package agentapi
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// T2: New must build a Transport with a BOUNDED, EXPIRING idle-conn pool. The earlier bare
|
||||
// &http.Transport{TLSClientConfig:…} had IdleConnTimeout==0 (idle keep-alives never expire) — the
|
||||
// leak. White-box (package agentapi) so we can read the unexported hc.
|
||||
//
|
||||
// Companion red-proof: the bare Transport gives IdleConnTimeout==0 → this test fails (demonstrated,
|
||||
// then reverted, during implementation).
|
||||
func TestNew_TransportIdlePoolBounded(t *testing.T) {
|
||||
c, err := New("127.0.0.1:8443", "tok", strings.Repeat("a", 64))
|
||||
if err != nil {
|
||||
t.Fatalf("New: %v", err)
|
||||
}
|
||||
tr, ok := c.hc.Transport.(*http.Transport)
|
||||
if !ok {
|
||||
t.Fatalf("unexpected transport type %T", c.hc.Transport)
|
||||
}
|
||||
if tr.IdleConnTimeout <= 0 {
|
||||
t.Fatalf("IdleConnTimeout must be > 0 (idle keep-alives must expire); got %v", tr.IdleConnTimeout)
|
||||
}
|
||||
if tr.MaxIdleConnsPerHost <= 0 {
|
||||
t.Fatalf("MaxIdleConnsPerHost must be > 0 (bounded pool); got %d", tr.MaxIdleConnsPerHost)
|
||||
}
|
||||
}
|
||||
@@ -15,12 +15,6 @@ func diskStub(t *testing.T) (*httptest.Server, string) {
|
||||
mux.HandleFunc("GET /disks", func(w http.ResponseWriter, r *http.Request) {
|
||||
_, _ = w.Write([]byte(`{"ok":true,"data":{"vmid":8200,"disks":[{"name":"bulk","data_bearing":true,"data_reason":"has ext4"}]}}`))
|
||||
})
|
||||
mux.HandleFunc("GET /disks/candidates", func(w http.ResponseWriter, r *http.Request) {
|
||||
_, _ = w.Write([]byte(`{"ok":true,"data":{"vmid":8200,` +
|
||||
`"initialize":[{"device":"/dev/sdd","size_bytes":64000000000,"model":"USB","fstype":"ext4","data_bearing":true,"mountable":true,"mount_source":"/dev/sdd","durable_id":"uuid:abc"},` +
|
||||
`{"device":"/dev/sde","size_bytes":1000,"data_bearing":false,"mountable":false}],` +
|
||||
`"attach":[{"device":"/dev/sdd","fstype":"ext4","mountable":true,"mount_source":"/dev/sdd","durable_id":"uuid:abc"}]}}`))
|
||||
})
|
||||
mux.HandleFunc("POST /disks/assign", func(w http.ResponseWriter, r *http.Request) {
|
||||
_, _ = w.Write([]byte(`{"ok":true,"data":{"assigned":"/mnt/data"}}`))
|
||||
})
|
||||
@@ -40,9 +34,6 @@ func diskStub(t *testing.T) (*httptest.Server, string) {
|
||||
case strings.Contains(body.Device, "data") && !body.Confirmed: // user-data, not yet confirmed
|
||||
w.WriteHeader(http.StatusForbidden)
|
||||
_, _ = w.Write([]byte(`{"ok":false,"data":{"device":"` + body.Device + `","data_bearing":true,"role":"user-data","needs_confirmation":true,"durable_id":"byid:wwn-1"}}`))
|
||||
case strings.Contains(body.Device, "mounted"): // mkfs failed (e.g. device mounted) → agent 502, data:null
|
||||
w.WriteHeader(http.StatusBadGateway)
|
||||
_, _ = w.Write([]byte(`{"ok":false,"error":"format failed: /dev/sdb1 is mounted; will not make a filesystem here!","data":null}`))
|
||||
default: // blank, or user-data confirmed
|
||||
_, _ = w.Write([]byte(`{"ok":true,"data":{"device":"` + body.Device + `","formatted":true,"role":"user-data"}}`))
|
||||
}
|
||||
@@ -77,40 +68,6 @@ func TestDisks_List(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestListCandidates(t *testing.T) {
|
||||
s, ep := diskStub(t)
|
||||
defer s.Close()
|
||||
c := clientFor(t, s, ep)
|
||||
res, err := c.ListCandidates(context.Background())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(res.Initialize) != 2 {
|
||||
t.Fatalf("want 2 initialize candidates, got %+v", res.Initialize)
|
||||
}
|
||||
if len(res.Attach) != 1 || res.Attach[0].Device != "/dev/sdd" || res.Attach[0].FSType != "ext4" {
|
||||
t.Fatalf("attach candidate wrong: %+v", res.Attach)
|
||||
}
|
||||
if res.Initialize[0].DurableID != "uuid:abc" || !res.Initialize[0].DataBearing {
|
||||
t.Fatalf("initialize[0] fields wrong: %+v", res.Initialize[0])
|
||||
}
|
||||
}
|
||||
|
||||
func TestListCandidates_Error(t *testing.T) {
|
||||
// A non-2xx / malformed agent response surfaces as an error, not a silent empty list.
|
||||
mux := http.NewServeMux()
|
||||
mux.HandleFunc("GET /disks/candidates", func(w http.ResponseWriter, r *http.Request) {
|
||||
w.WriteHeader(http.StatusBadGateway)
|
||||
_, _ = w.Write([]byte(`{"ok":false,"error":"agent unreachable"}`))
|
||||
})
|
||||
s := httptest.NewTLSServer(mux)
|
||||
defer s.Close()
|
||||
c := clientFor(t, s, strings.TrimPrefix(s.URL, "https://"))
|
||||
if _, err := c.ListCandidates(context.Background()); err == nil {
|
||||
t.Fatal("expected an error from a 502 candidates response")
|
||||
}
|
||||
}
|
||||
|
||||
func TestFormat_BlankOK(t *testing.T) {
|
||||
s, ep := diskStub(t)
|
||||
defer s.Close()
|
||||
@@ -160,28 +117,6 @@ func TestFormat_UserDataConfirmed(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// F20-BUG1: a real mkfs failure (agent 502, ok:false, data:null) must surface as a non-nil error —
|
||||
// NOT a zero-value FormatResult with nil err (which read as a silent SUCCESS in the web layer).
|
||||
func TestFormat_MountedFailureSurfacesError(t *testing.T) {
|
||||
s, ep := diskStub(t)
|
||||
defer s.Close()
|
||||
c := clientFor(t, s, ep)
|
||||
res, err := c.FormatDisk(context.Background(), "/dev/sdb1-mounted", "ext4", true, "byid:wwn-1")
|
||||
if err == nil {
|
||||
t.Fatalf("expected a non-nil error for a failed format, got (res=%+v, err=nil) — silent success regression", res)
|
||||
}
|
||||
if res.Formatted {
|
||||
t.Fatalf("Formatted must be false on a failed format: %+v", res)
|
||||
}
|
||||
if !strings.Contains(err.Error(), "502") || !strings.Contains(err.Error(), "mounted") {
|
||||
t.Fatalf("error should carry the HTTP status + agent message, got: %v", err)
|
||||
}
|
||||
// Must not be misclassified as one of the gated refusals.
|
||||
if errors.Is(err, ErrNeedsConfirmation) || errors.Is(err, ErrFormatRefused) {
|
||||
t.Fatalf("a 502 mkfs failure must not be reported as a refusal: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestEject_Dependents(t *testing.T) {
|
||||
s, ep := diskStub(t)
|
||||
defer s.Close()
|
||||
|
||||
@@ -1,201 +0,0 @@
|
||||
package agentapi
|
||||
|
||||
// DiskVerdict is the customer-facing disk-health verdict derived from a SmartSummary (v0.169.0).
|
||||
// It is the SHARED source of truth for both the "Lemezek állapota" dashboard card and the periodic
|
||||
// degradation check — one pure function so the chip and the alert can never disagree.
|
||||
type DiskVerdict int
|
||||
|
||||
const (
|
||||
// DiskVerdictUnknown — no SMART data (nil / UNKNOWN / old agent). Renders "Nincs adat"; NEVER
|
||||
// alarms and NEVER participates in degradation transitions (excluded both directions).
|
||||
DiskVerdictUnknown DiskVerdict = iota
|
||||
DiskVerdictOK // "Rendben" — clean
|
||||
DiskVerdictWarn // "Figyelmeztetés" — a wear/relocation counter is non-zero, below the Hiba bar
|
||||
DiskVerdictFail // "Hiba" — FAILING, or failing-but-not-self-reported (v0.215.0)
|
||||
)
|
||||
|
||||
// Thresholds. A number without a reason becomes permanent by default, so each carries its provenance.
|
||||
// The evidence is committed at felhom.eu/documentation/audits/DIAG-smart-passed-trap-2026-08-14.md
|
||||
// and its two fixtures (ST3000VX010 S/N Z6A07P2G, /dev/sdg on DooPlex, 11-13 Aug 2026).
|
||||
const (
|
||||
// percentageUsedWarn / percentageUsedFail — NVMe wear (%). 100 means the vendor's rated endurance
|
||||
// is spent; that is a declaration, not a trend, so it is Hiba.
|
||||
percentageUsedWarn = 90
|
||||
percentageUsedFail = 100
|
||||
|
||||
// uncorrectableFailCount — unreadable sectors too numerous to be a blip.
|
||||
//
|
||||
// PROVENANCE: on the one real failing drive observed, the benign excursion peaked at 16 and
|
||||
// cleared COMPLETELY within an hour (11 Aug 12:28 -> 13:28); the terminal run passed 64 at
|
||||
// 13 Aug 11:28 and never came back below it. 64 sits above the one observed transient and below
|
||||
// the observed terminal run. This is a judgement from ONE drive: it is a static BACKSTOP behind
|
||||
// the sustain rule, not the primary signal, and Phase 3 is expected to replace it with
|
||||
// growth-rate detection once the box keeps history.
|
||||
uncorrectableFailCount = 64
|
||||
|
||||
// temperatureWarnC / TemperatureFailC — adopted UNCHANGED from the operator's existing Prometheus
|
||||
// bands on DooPlex, so the two systems cannot disagree about the same drive.
|
||||
temperatureWarnC = 55
|
||||
// TemperatureFailC is exported because the alert-copy layer must pick the "overheated" message
|
||||
// shape from the SAME number the verdict fired on. A second literal elsewhere would be free to
|
||||
// drift, and the drift would show up as a customer told the wrong reason.
|
||||
TemperatureFailC = 60
|
||||
)
|
||||
|
||||
// DiskPrior is what the previous check observed for THIS SAME disk. It is the only history the
|
||||
// verdict consults, and it is passed in rather than read so the function stays pure — the caller
|
||||
// (internal/web) owns loading it from the persisted per-disk state.
|
||||
//
|
||||
// Plain value type: no methods, no I/O. A zero DiskPrior means "nothing known", which is the correct
|
||||
// fail-safe — a first-ever observation can only reach Figyelmeztetés from counters, never Hiba.
|
||||
type DiskPrior struct {
|
||||
// SawUncorrectable reports whether unreadable sectors (pending OR offline-uncorrectable) were
|
||||
// present at the previous check. It is what turns a one-off excursion into a sustained fault.
|
||||
SawUncorrectable bool
|
||||
}
|
||||
|
||||
// DiskVerdictFor maps a SmartSummary plus the previous observation to a verdict. Rules are evaluated
|
||||
// TOP-DOWN and the FIRST match wins (v0.215.0):
|
||||
//
|
||||
// 1. nil / "" / UNKNOWN -> Nincs adat
|
||||
// 2. Health == FAILING -> Hiba (drive self-reports)
|
||||
// 3. temperature_c >= 60 -> Hiba
|
||||
// 4. critical_warning > 0 (NVMe's own flag: a declaration) -> Hiba
|
||||
// 5. percentage_used >= 100 -> Hiba
|
||||
// 6. unreadable > 0 AND prior.SawUncorrectable -> Hiba (SUSTAINED)
|
||||
// 7. unreadable > 0 AND reallocated > 0 -> Hiba (accumulating + remapping)
|
||||
// 8. unreadable >= 64 -> Hiba (too large to be a blip)
|
||||
// 9. unreadable > 0 -> Figyelmeztetés (first sighting)
|
||||
// 10. reallocated > 0 -> Figyelmeztetés
|
||||
// 11. media_errors > 0 -> Figyelmeztetés
|
||||
// 12. percentage_used >= 90 -> Figyelmeztetés
|
||||
// 13. temperature_c >= 55 -> Figyelmeztetés
|
||||
// 14. otherwise -> Rendben
|
||||
//
|
||||
// WHY rows 2-8 exist at all: smart_status.passed CANNOT fail on unreadable sectors. Attributes 187,
|
||||
// 197 and 198 all carry thresh 0, and a normalized SMART value floors at 1, so it can never drop to
|
||||
// or below the threshold. The real drive stayed PASSED at 352 pending sectors with 1001 reported
|
||||
// uncorrectable reads. A verdict built on the drive's own self-assessment is blind to this whole
|
||||
// class of failure, which is why rows 3-8 read the raw counters instead.
|
||||
//
|
||||
// WHY row 6 sits ABOVE row 8: sustain is the PRIMARY rule and the count is the backstop. On the real
|
||||
// drive sustain fires a full day earlier (12 Aug) than the count threshold (13 Aug). Row 8 exists for
|
||||
// a box that was powered off or restarted across the sustain window and so has no prior.
|
||||
//
|
||||
// Pure: no clock, no I/O, no logging. Everything it needs arrives as an argument.
|
||||
func DiskVerdictFor(s *SmartSummary, prior DiskPrior) DiskVerdict {
|
||||
// 1 — no data. Never alarms.
|
||||
if s == nil || s.Health == "" || s.Health == SmartUnknown {
|
||||
return DiskVerdictUnknown
|
||||
}
|
||||
// 2 — the drive admits failure.
|
||||
if s.Health == SmartFailing {
|
||||
return DiskVerdictFail
|
||||
}
|
||||
// Health == PASSED (or any non-empty non-FAILING value we treat as passing): inspect the counters,
|
||||
// because the overall verdict is structurally unable to report this class of fault.
|
||||
switch {
|
||||
case atLeast(s.TemperatureC, TemperatureFailC): // 3
|
||||
return DiskVerdictFail
|
||||
case positive(s.CriticalWarning): // 4
|
||||
return DiskVerdictFail
|
||||
case atLeast(s.PercentageUsed, percentageUsedFail): // 5
|
||||
return DiskVerdictFail
|
||||
}
|
||||
unreadable := UncorrectableSectors(s)
|
||||
switch {
|
||||
case unreadable > 0 && prior.SawUncorrectable: // 6 — sustained across two consecutive checks
|
||||
return DiskVerdictFail
|
||||
case unreadable > 0 && positive(s.ReallocatedSectors): // 7 — accumulating and remapping together
|
||||
return DiskVerdictFail
|
||||
case unreadable >= uncorrectableFailCount: // 8 — too large to be a blip
|
||||
return DiskVerdictFail
|
||||
case unreadable > 0: // 9 — first sighting, below the bar
|
||||
return DiskVerdictWarn
|
||||
case positive(s.ReallocatedSectors): // 10
|
||||
return DiskVerdictWarn
|
||||
case positive(s.MediaErrors): // 11
|
||||
return DiskVerdictWarn
|
||||
case atLeast(s.PercentageUsed, percentageUsedWarn): // 12
|
||||
return DiskVerdictWarn
|
||||
case atLeast(s.TemperatureC, temperatureWarnC): // 13
|
||||
return DiskVerdictWarn
|
||||
}
|
||||
return DiskVerdictOK // 14
|
||||
}
|
||||
|
||||
// UncorrectableSectors is the disk's unreadable-sector count: max(pending, offline_uncorrectable).
|
||||
// The two attributes track the same physical defect and on the real drive moved in lockstep, so the
|
||||
// larger is the honest figure. 0 when neither is reported (an old agent or a device without them).
|
||||
// Exported because the alert copy quotes this number and the persisted state remembers it.
|
||||
func UncorrectableSectors(s *SmartSummary) int {
|
||||
if s == nil {
|
||||
return 0
|
||||
}
|
||||
n := 0
|
||||
if s.PendingSectors != nil && *s.PendingSectors > n {
|
||||
n = *s.PendingSectors
|
||||
}
|
||||
if s.OfflineUncorrectable != nil && *s.OfflineUncorrectable > n {
|
||||
n = *s.OfflineUncorrectable
|
||||
}
|
||||
return n
|
||||
}
|
||||
|
||||
// Label is the exact Hungarian customer copy for the verdict (shared by the card chip and the email).
|
||||
//
|
||||
// There are FOUR labels and there will not be a fifth: a predicted failure is "Hiba", the same word a
|
||||
// self-reported failure gets. A fourth word sharing a root with "Figyelmeztetés" would make the MORE
|
||||
// severe state read as the milder one (settled operator decision, v0.215.0).
|
||||
func (v DiskVerdict) Label() string {
|
||||
switch v {
|
||||
case DiskVerdictOK:
|
||||
return "Rendben"
|
||||
case DiskVerdictWarn:
|
||||
return "Figyelmeztetés"
|
||||
case DiskVerdictFail:
|
||||
return "Hiba"
|
||||
default:
|
||||
return "Nincs adat"
|
||||
}
|
||||
}
|
||||
|
||||
// DegradedAttributes returns the human-readable Hungarian names of the attribute(s) behind a
|
||||
// degraded verdict, for the alert body.
|
||||
//
|
||||
// v0.215.0: this now also names the attributes behind a Hiba REACHED FROM COUNTERS (truth-table rows
|
||||
// 3 and 6-8), not only a Figyelmeztetés — the alert message needs to say what is wrong, and those
|
||||
// rows do have a triggering counter. It returns nil ONLY for row 2 (the drive self-reports FAILING,
|
||||
// a whole-disk verdict with no single triggering counter) and, naturally, for Nincs adat / Rendben.
|
||||
func DegradedAttributes(s *SmartSummary) []string {
|
||||
if s == nil || s.Health == "" || s.Health == SmartUnknown || s.Health == SmartFailing {
|
||||
return nil
|
||||
}
|
||||
var out []string
|
||||
if positive(s.ReallocatedSectors) {
|
||||
out = append(out, "áthelyezett szektorok")
|
||||
}
|
||||
if positive(s.PendingSectors) {
|
||||
out = append(out, "függőben lévő szektorok")
|
||||
}
|
||||
if positive(s.OfflineUncorrectable) {
|
||||
out = append(out, "javíthatatlan szektorok")
|
||||
}
|
||||
if positive(s.CriticalWarning) {
|
||||
out = append(out, "kritikus figyelmeztetés")
|
||||
}
|
||||
if positive(s.MediaErrors) {
|
||||
out = append(out, "adathordozó-hibák")
|
||||
}
|
||||
if atLeast(s.PercentageUsed, percentageUsedWarn) {
|
||||
out = append(out, "elhasználódás")
|
||||
}
|
||||
// Newly able to trigger a verdict on its own (rows 3 and 13), so it must be nameable.
|
||||
if atLeast(s.TemperatureC, temperatureWarnC) {
|
||||
out = append(out, "hőmérséklet")
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func positive(p *int) bool { return p != nil && *p > 0 }
|
||||
func atLeast(p *int, n int) bool { return p != nil && *p >= n }
|
||||
@@ -1,202 +0,0 @@
|
||||
package agentapi
|
||||
|
||||
import "testing"
|
||||
|
||||
// The v0.215.0 severity ladder, verdict half. The event half (emission, damping, cooldown,
|
||||
// persistence) lives in internal/web — this file pins ONLY what the pure function decides.
|
||||
//
|
||||
// Every value used here is taken from the committed evidence:
|
||||
// felhom.eu/documentation/audits/fixtures/smart-ST3000VX010-failing-2026-08-14.json
|
||||
// (ST3000VX010-2E3166, S/N Z6A07P2G, /dev/sdg on DooPlex).
|
||||
|
||||
// realDrive is the failing drive AS CAPTURED on 2026-08-14: PASSED, 352 pending, 352 offline
|
||||
// uncorrectable, 0 reallocated, 40 °C. The whole point of the fixture is that Health is PASSED.
|
||||
func realDrive() *SmartSummary {
|
||||
return &SmartSummary{
|
||||
Health: SmartPassed,
|
||||
PendingSectors: ip(352),
|
||||
OfflineUncorrectable: ip(352),
|
||||
ReallocatedSectors: ip(0),
|
||||
TemperatureC: ip(40),
|
||||
}
|
||||
}
|
||||
|
||||
// Group A (verdict half) — Scenario A. The real drive on its SECOND observation reaches Hiba, and
|
||||
// the chip label is exactly "Hiba".
|
||||
//
|
||||
// Red-proof: delete truth-table row 6 (the `prior.SawUncorrectable` case) from DiskVerdictFor →
|
||||
// the drive still reaches Fail via row 8 (352 >= 64), so this test alone does NOT prove row 6.
|
||||
// TestLadder_SustainIsWhatFires below is the one that isolates it.
|
||||
func TestLadder_RealDrive_ReachesHiba(t *testing.T) {
|
||||
got := DiskVerdictFor(realDrive(), DiskPrior{SawUncorrectable: true})
|
||||
if got != DiskVerdictFail {
|
||||
t.Fatalf("real failing drive verdict = %d (%s), want Fail/Hiba", got, got.Label())
|
||||
}
|
||||
if got.Label() != "Hiba" {
|
||||
t.Errorf("label = %q, want %q", got.Label(), "Hiba")
|
||||
}
|
||||
// The trap this whole change exists for: the drive's own verdict says everything is fine.
|
||||
if realDrive().Health != SmartPassed {
|
||||
t.Fatal("fixture drift: the real drive's Health must be PASSED — that IS the defect")
|
||||
}
|
||||
}
|
||||
|
||||
// Groups B + C (verdict half) — Scenarios B and C. The SAME SmartSummary yields Figyelmeztetés on a
|
||||
// first sighting and Hiba once it is sustained. This is the pair that isolates row 6: the counters
|
||||
// are identical and only `prior` differs, so nothing else in the table can be producing the change.
|
||||
//
|
||||
// The values are the 11 August excursion (8 sectors), which cleared completely within an hour — a
|
||||
// count deliberately far below the 64 backstop so row 8 cannot mask row 6.
|
||||
//
|
||||
// Red-proof: remove the `prior.SawUncorrectable` clause from row 6 → the sustained case stays Warn.
|
||||
func TestLadder_SustainIsWhatFires(t *testing.T) {
|
||||
excursion := func() *SmartSummary {
|
||||
return &SmartSummary{Health: SmartPassed, PendingSectors: ip(8), OfflineUncorrectable: ip(8), ReallocatedSectors: ip(0)}
|
||||
}
|
||||
if got := DiskVerdictFor(excursion(), DiskPrior{}); got != DiskVerdictWarn {
|
||||
t.Errorf("first sighting of 8 sectors = %d (%s), want Warn/Figyelmeztetés — a single "+
|
||||
"excursion that clears by itself is normal and must NOT reach Hiba", got, got.Label())
|
||||
}
|
||||
if got := DiskVerdictFor(excursion(), DiskPrior{SawUncorrectable: true}); got != DiskVerdictFail {
|
||||
t.Errorf("SAME 8 sectors, now sustained = %d (%s), want Fail/Hiba", got, got.Label())
|
||||
}
|
||||
if got := DiskVerdictFor(excursion(), DiskPrior{}).Label(); got != "Figyelmeztetés" {
|
||||
t.Errorf("first-sighting label = %q, want Figyelmeztetés", got)
|
||||
}
|
||||
}
|
||||
|
||||
// Row 8, the backstop — for a box that was powered off or restarted across the sustain window and so
|
||||
// has NO prior. 63 stays Warn, 64 reaches Hiba. The boundary is inclusive, which is what
|
||||
// `uncorrectableFailCount` claims and what the real drive did at 13 Aug 11:28 (exactly 64).
|
||||
//
|
||||
// Red-proof: change `>=` to `>` in row 8 → the "exactly 64" case reads Warn.
|
||||
func TestLadder_CountBackstopBoundary(t *testing.T) {
|
||||
cases := []struct {
|
||||
pending int
|
||||
want DiskVerdict
|
||||
}{
|
||||
{63, DiskVerdictWarn},
|
||||
{64, DiskVerdictFail},
|
||||
{352, DiskVerdictFail},
|
||||
}
|
||||
for _, c := range cases {
|
||||
s := &SmartSummary{Health: SmartPassed, PendingSectors: ip(c.pending)}
|
||||
if got := DiskVerdictFor(s, DiskPrior{}); got != c.want {
|
||||
t.Errorf("%d pending sectors, no prior = %d (%s), want %d", c.pending, got, got.Label(), c.want)
|
||||
}
|
||||
}
|
||||
// Row 7 — unreadable AND remapping together is Hiba even at a low count with no prior.
|
||||
s := &SmartSummary{Health: SmartPassed, PendingSectors: ip(8), ReallocatedSectors: ip(1)}
|
||||
if got := DiskVerdictFor(s, DiskPrior{}); got != DiskVerdictFail {
|
||||
t.Errorf("row 7 (unreadable + reallocated) = %d, want Fail", got)
|
||||
}
|
||||
}
|
||||
|
||||
// Group I — Scenario I, heat. 61 → Hiba, 56 → Figyelmeztetés, 54 → Rendben, with all counters clean.
|
||||
//
|
||||
// Red-proof: remove rows 3 and 13 → all three read Rendben.
|
||||
func TestLadder_Temperature(t *testing.T) {
|
||||
cases := []struct {
|
||||
temp int
|
||||
want DiskVerdict
|
||||
}{
|
||||
{54, DiskVerdictOK},
|
||||
{55, DiskVerdictWarn}, // inclusive boundary
|
||||
{56, DiskVerdictWarn},
|
||||
{59, DiskVerdictWarn},
|
||||
{60, DiskVerdictFail}, // inclusive boundary
|
||||
{61, DiskVerdictFail},
|
||||
}
|
||||
for _, c := range cases {
|
||||
s := &SmartSummary{Health: SmartPassed, TemperatureC: ip(c.temp), PendingSectors: ip(0), ReallocatedSectors: ip(0)}
|
||||
if got := DiskVerdictFor(s, DiskPrior{}); got != c.want {
|
||||
t.Errorf("%d °C = %d (%s), want %d", c.temp, got, got.Label(), c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Group J (verdict half) — Scenario J. No data never alarms, and a prior must not manufacture one:
|
||||
// a nil/UNKNOWN SmartSummary reads Nincs adat EVEN WITH SawUncorrectable set. Row 1 is first in the
|
||||
// table for exactly this reason.
|
||||
//
|
||||
// Red-proof: move row 1 below row 6 → the UNKNOWN-with-prior case reads Hiba, i.e. a disk whose
|
||||
// SMART briefly became unreadable would be reported as failing.
|
||||
func TestLadder_UnknownNeverAlarms(t *testing.T) {
|
||||
for _, s := range []*SmartSummary{nil, {Health: ""}, {Health: SmartUnknown}} {
|
||||
if got := DiskVerdictFor(s, DiskPrior{SawUncorrectable: true}); got != DiskVerdictUnknown {
|
||||
t.Errorf("no-data disk with a prior = %d (%s), want Unknown/Nincs adat", got, got.Label())
|
||||
}
|
||||
}
|
||||
if got := DiskVerdictFor(&SmartSummary{Health: SmartUnknown}, DiskPrior{}).Label(); got != "Nincs adat" {
|
||||
t.Errorf("label = %q, want Nincs adat", got)
|
||||
}
|
||||
}
|
||||
|
||||
// The zero DiskPrior must be the SAFE default: a caller that forgets to load history can only
|
||||
// under-report (Figyelmeztetés), never over-report (Hiba) on a first sighting. This pins the
|
||||
// fail-safe direction the persisted-state loader relies on when its file is missing or corrupt.
|
||||
func TestLadder_ZeroPriorIsFailSafe(t *testing.T) {
|
||||
s := &SmartSummary{Health: SmartPassed, PendingSectors: ip(8)}
|
||||
if got := DiskVerdictFor(s, DiskPrior{}); got != DiskVerdictWarn {
|
||||
t.Fatalf("zero prior must degrade to Warn, not Fail; got %d (%s)", got, got.Label())
|
||||
}
|
||||
}
|
||||
|
||||
// UncorrectableSectors is max(pending, offline) — the number the alert copy quotes and the persisted
|
||||
// state remembers. A wrong answer here puts a wrong count in a customer's email.
|
||||
func TestUncorrectableSectors(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
in *SmartSummary
|
||||
want int
|
||||
}{
|
||||
{"nil summary", nil, 0},
|
||||
{"neither reported (old agent)", &SmartSummary{Health: SmartPassed}, 0},
|
||||
{"both zero", &SmartSummary{PendingSectors: ip(0), OfflineUncorrectable: ip(0)}, 0},
|
||||
{"pending only", &SmartSummary{PendingSectors: ip(8)}, 8},
|
||||
{"offline only", &SmartSummary{OfflineUncorrectable: ip(24)}, 24},
|
||||
{"pending larger", &SmartSummary{PendingSectors: ip(40), OfflineUncorrectable: ip(24)}, 40},
|
||||
{"offline larger", &SmartSummary{PendingSectors: ip(24), OfflineUncorrectable: ip(40)}, 40},
|
||||
{"the real drive", realDrive(), 352},
|
||||
}
|
||||
for _, c := range cases {
|
||||
if got := UncorrectableSectors(c.in); got != c.want {
|
||||
t.Errorf("%s: UncorrectableSectors = %d, want %d", c.name, got, c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// DegradedAttributes must NAME the counters behind a Hiba reached from counters (v0.215.0) — the
|
||||
// alert body is built from this and an empty list produces a message that says nothing is wrong.
|
||||
// It still returns nil for row 2 (drive-reported FAILING), which has no single triggering counter.
|
||||
//
|
||||
// Red-proof: restore the pre-v0.215.0 body (nil for anything at Fail) → the real-drive case returns
|
||||
// an empty list.
|
||||
func TestDegradedAttributes_NamesFailCounters(t *testing.T) {
|
||||
got := DegradedAttributes(realDrive())
|
||||
if len(got) == 0 {
|
||||
t.Fatal("a Hiba reached from counters must name its attributes, got none")
|
||||
}
|
||||
found := map[string]bool{}
|
||||
for _, a := range got {
|
||||
found[a] = true
|
||||
}
|
||||
for _, want := range []string{"függőben lévő szektorok", "javíthatatlan szektorok"} {
|
||||
if !found[want] {
|
||||
t.Errorf("missing attribute %q in %v", want, got)
|
||||
}
|
||||
}
|
||||
// Row 2 — the drive self-reports FAILING: no single triggering counter, so nil.
|
||||
if a := DegradedAttributes(&SmartSummary{Health: SmartFailing, PendingSectors: ip(5)}); a != nil {
|
||||
t.Errorf("FAILING (row 2) must return nil attributes, got %v", a)
|
||||
}
|
||||
// Nincs adat must never produce attribute names either.
|
||||
if a := DegradedAttributes(&SmartSummary{Health: SmartUnknown}); a != nil {
|
||||
t.Errorf("UNKNOWN must return nil attributes, got %v", a)
|
||||
}
|
||||
// Temperature is newly able to trigger on its own, so it must be nameable.
|
||||
hot := DegradedAttributes(&SmartSummary{Health: SmartPassed, TemperatureC: ip(61)})
|
||||
if len(hot) != 1 || hot[0] != "hőmérséklet" {
|
||||
t.Errorf("hot disk attributes = %v, want [hőmérséklet]", hot)
|
||||
}
|
||||
}
|
||||
@@ -1,71 +0,0 @@
|
||||
package agentapi
|
||||
|
||||
import "testing"
|
||||
|
||||
func ip(v int) *int { return &v }
|
||||
|
||||
// Verdict table (Part 2, extended v0.215.0). Red-proof: change the PercentageUsed boundary from
|
||||
// `>= 90` to `> 90` in DiskVerdictFor → the "NVMe percentage_used exactly 90 → Figyelmeztetés" case
|
||||
// fails.
|
||||
//
|
||||
// v0.215.0 moved ONE pre-existing case deliberately: critical_warning>0 was Figyelmeztetés and is
|
||||
// now Hiba (truth-table row 4). It is NVMe's own critical flag — a declaration by the device, not a
|
||||
// counter that might drift back — so it belongs with the self-reported failures, not below them.
|
||||
func TestDiskVerdictFor(t *testing.T) {
|
||||
noPrior := DiskPrior{}
|
||||
cases := []struct {
|
||||
name string
|
||||
in *SmartSummary
|
||||
prior DiskPrior
|
||||
want DiskVerdict
|
||||
}{
|
||||
{"nil → unknown", nil, noPrior, DiskVerdictUnknown},
|
||||
{"empty health → unknown", &SmartSummary{Health: ""}, noPrior, DiskVerdictUnknown},
|
||||
{"UNKNOWN → unknown", &SmartSummary{Health: SmartUnknown}, noPrior, DiskVerdictUnknown},
|
||||
{"FAILING → fail", &SmartSummary{Health: SmartFailing}, noPrior, DiskVerdictFail},
|
||||
{"FAILING beats counters", &SmartSummary{Health: SmartFailing, ReallocatedSectors: ip(0)}, noPrior, DiskVerdictFail},
|
||||
{"PASSED clean → ok", &SmartSummary{Health: SmartPassed, ReallocatedSectors: ip(0), PendingSectors: ip(0), TemperatureC: ip(30)}, noPrior, DiskVerdictOK},
|
||||
{"PASSED nil counters → ok", &SmartSummary{Health: SmartPassed}, noPrior, DiskVerdictOK},
|
||||
{"reallocated>0 alone → warn", &SmartSummary{Health: SmartPassed, ReallocatedSectors: ip(1)}, noPrior, DiskVerdictWarn},
|
||||
{"pending>0 first sighting → warn", &SmartSummary{Health: SmartPassed, PendingSectors: ip(5)}, noPrior, DiskVerdictWarn},
|
||||
{"offline_unc>0 first sighting → warn", &SmartSummary{Health: SmartPassed, OfflineUncorrectable: ip(2)}, noPrior, DiskVerdictWarn},
|
||||
{"critical_warning>0 → fail (row 4)", &SmartSummary{Health: SmartPassed, CriticalWarning: ip(1)}, noPrior, DiskVerdictFail},
|
||||
{"media_errors>0 → warn", &SmartSummary{Health: SmartPassed, MediaErrors: ip(3)}, noPrior, DiskVerdictWarn},
|
||||
{"percentage_used 89 → ok", &SmartSummary{Health: SmartPassed, PercentageUsed: ip(89)}, noPrior, DiskVerdictOK},
|
||||
{"percentage_used exactly 90 → warn", &SmartSummary{Health: SmartPassed, PercentageUsed: ip(90)}, noPrior, DiskVerdictWarn},
|
||||
{"percentage_used 95 → warn", &SmartSummary{Health: SmartPassed, PercentageUsed: ip(95)}, noPrior, DiskVerdictWarn},
|
||||
{"percentage_used exactly 100 → fail (row 5)", &SmartSummary{Health: SmartPassed, PercentageUsed: ip(100)}, noPrior, DiskVerdictFail},
|
||||
}
|
||||
for _, c := range cases {
|
||||
if got := DiskVerdictFor(c.in, c.prior); got != c.want {
|
||||
t.Errorf("%s: DiskVerdictFor = %d, want %d", c.name, got, c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestDiskVerdict_Label(t *testing.T) {
|
||||
want := map[DiskVerdict]string{
|
||||
DiskVerdictUnknown: "Nincs adat",
|
||||
DiskVerdictOK: "Rendben",
|
||||
DiskVerdictWarn: "Figyelmeztetés",
|
||||
DiskVerdictFail: "Hiba",
|
||||
}
|
||||
for v, w := range want {
|
||||
if got := v.Label(); got != w {
|
||||
t.Errorf("verdict %d Label = %q, want %q", v, got, w)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A warn lists every triggering attribute at once (Scenario "multiple attributes degrade" → ONE event).
|
||||
func TestDegradedAttributes_ListsAll(t *testing.T) {
|
||||
s := &SmartSummary{Health: SmartPassed, PendingSectors: ip(5), ReallocatedSectors: ip(2), PercentageUsed: ip(91)}
|
||||
got := DegradedAttributes(s)
|
||||
if len(got) != 3 {
|
||||
t.Fatalf("want 3 attributes, got %d: %v", len(got), got)
|
||||
}
|
||||
// clean disk → none
|
||||
if a := DegradedAttributes(&SmartSummary{Health: SmartPassed}); len(a) != 0 {
|
||||
t.Errorf("clean disk should list no attributes, got %v", a)
|
||||
}
|
||||
}
|
||||
@@ -1,282 +0,0 @@
|
||||
package agentapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
)
|
||||
|
||||
// Controller-driven escrow ceremony client methods (v0.127.0, agent ≥ v0.88.0). The claim call
|
||||
// is the ONLY place the recovery code R crosses this client — its response body must never be
|
||||
// logged (the shared helpers log path/status/duration only, never bodies) and the caller hands
|
||||
// R straight to the wizard's claim XHR, nowhere else.
|
||||
|
||||
// EscrowPreflightItem mirrors one agent preflight checklist row.
|
||||
type EscrowPreflightItem struct {
|
||||
ID string `json:"id"`
|
||||
OK bool `json:"ok"`
|
||||
Detail string `json:"detail"`
|
||||
}
|
||||
|
||||
// EscrowPreflightResponse mirrors GET /escrow/preflight.
|
||||
type EscrowPreflightResponse struct {
|
||||
VMID int `json:"vmid"`
|
||||
OK bool `json:"ok"`
|
||||
Items []EscrowPreflightItem `json:"items"`
|
||||
}
|
||||
|
||||
// EscrowPreflight fetches the agent's ceremony prerequisite checklist.
|
||||
func (c *Client) EscrowPreflight(ctx context.Context) (EscrowPreflightResponse, error) {
|
||||
var out EscrowPreflightResponse
|
||||
body, err := c.get(ctx, "/escrow/preflight")
|
||||
if err != nil {
|
||||
return out, err
|
||||
}
|
||||
if err := json.Unmarshal(body, &out); err != nil {
|
||||
return out, fmt.Errorf("agentapi: decode /escrow/preflight: %w", err)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// EscrowCeremonyStartResponse mirrors the POST /escrow/ceremony 202 payload.
|
||||
type EscrowCeremonyStartResponse struct {
|
||||
JobID string `json:"job_id"`
|
||||
Phase string `json:"phase"`
|
||||
}
|
||||
|
||||
// EscrowCeremonyStart triggers the agent's detached root ceremony job. Status-aware: the HTTP
|
||||
// status is returned so the caller can map the agent's 409 (a ceremony already running) to its
|
||||
// own house-style refusal.
|
||||
func (c *Client) EscrowCeremonyStart(ctx context.Context) (EscrowCeremonyStartResponse, int, error) {
|
||||
var out EscrowCeremonyStartResponse
|
||||
env, status, err := c.postWithStatus(ctx, "/escrow/ceremony", struct{}{})
|
||||
if err != nil {
|
||||
return out, status, err
|
||||
}
|
||||
if err := refusalError("/escrow/ceremony", status, env); err != nil {
|
||||
return out, status, err
|
||||
}
|
||||
if err := json.Unmarshal(env.Data, &out); err != nil {
|
||||
return out, status, fmt.Errorf("agentapi: decode /escrow/ceremony: %w", err)
|
||||
}
|
||||
return out, status, nil
|
||||
}
|
||||
|
||||
// EscrowCeremonyStatusResponse mirrors GET /escrow/ceremony/status — the NON-SECRET job view
|
||||
// (R is structurally absent from the agent's payload).
|
||||
type EscrowCeremonyStatusResponse struct {
|
||||
Phase string `json:"phase"` // none | running | done | failed | unclaimed_void
|
||||
JobID string `json:"job_id"`
|
||||
KeyFingerprint string `json:"key_fingerprint"`
|
||||
EntropyBits float64 `json:"entropy_bits"`
|
||||
ResticPwSealed bool `json:"restic_pw_sealed"`
|
||||
Uploaded bool `json:"uploaded"`
|
||||
Claimable bool `json:"claimable"`
|
||||
Claimed bool `json:"claimed"`
|
||||
ClaimExpiresInSec int `json:"claim_expires_in_sec"`
|
||||
Detail string `json:"detail"`
|
||||
}
|
||||
|
||||
// EscrowCeremonyStatus polls the ceremony job.
|
||||
func (c *Client) EscrowCeremonyStatus(ctx context.Context) (EscrowCeremonyStatusResponse, error) {
|
||||
var out EscrowCeremonyStatusResponse
|
||||
body, err := c.get(ctx, "/escrow/ceremony/status")
|
||||
if err != nil {
|
||||
return out, err
|
||||
}
|
||||
if err := json.Unmarshal(body, &out); err != nil {
|
||||
return out, fmt.Errorf("agentapi: decode /escrow/ceremony/status: %w", err)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// EscrowCeremonyClaim performs the ONE-SHOT R claim. Returns the recovery code + the agent's
|
||||
// HTTP status (410 = already claimed / expired — the wizard's void state). The code must never
|
||||
// be logged, persisted, or placed anywhere but the claim XHR response; the error path carries
|
||||
// the agent's reason text, never the code.
|
||||
func (c *Client) EscrowCeremonyClaim(ctx context.Context) (string, int, error) {
|
||||
env, status, err := c.postWithStatus(ctx, "/escrow/ceremony/claim", struct{}{})
|
||||
if err != nil {
|
||||
return "", status, err
|
||||
}
|
||||
if status == http.StatusGone {
|
||||
return "", status, fmt.Errorf("agentapi: POST /escrow/ceremony/claim: gone (claimed or expired)")
|
||||
}
|
||||
if err := refusalError("/escrow/ceremony/claim", status, env); err != nil {
|
||||
return "", status, err
|
||||
}
|
||||
var out struct {
|
||||
RecoveryCode string `json:"recovery_code"`
|
||||
}
|
||||
if err := json.Unmarshal(env.Data, &out); err != nil {
|
||||
return "", status, fmt.Errorf("agentapi: decode /escrow/ceremony/claim: %w", err)
|
||||
}
|
||||
if out.RecoveryCode == "" {
|
||||
return "", status, fmt.Errorf("agentapi: /escrow/ceremony/claim returned no code")
|
||||
}
|
||||
return out.RecoveryCode, status, nil
|
||||
}
|
||||
|
||||
// RecoverOffsiteRepoPassword asks the agent to open this host's hub-held sealed bundle with the
|
||||
// customer's recovery code and return ONLY the offsite restic repository password, plus its sha256
|
||||
// (R-199, agent >= v0.125.0).
|
||||
//
|
||||
// R CROSSES HERE, AND NOWHERE ELSE IN THIS DIRECTION. It travels in the request body over the pinned
|
||||
// local-API channel (the operator's 2026-08-04 acceptance) and is not retained by this client. The
|
||||
// shared POST helper logs path/status/duration and never bodies — do not add a body log, on either
|
||||
// the request or the response side: the request carries R and the response carries the password.
|
||||
func (c *Client) RecoverOffsiteRepoPassword(ctx context.Context, recoveryCode string) (password, sha256hex string, err error) {
|
||||
env, status, perr := c.postWithStatus(ctx, "/escrow/recover-offsite-password",
|
||||
map[string]string{"recovery_code": recoveryCode})
|
||||
if perr != nil {
|
||||
return "", "", perr
|
||||
}
|
||||
// R-224: this route's refusal keeps its STATUS as a value. `refusalError` flattens status into a
|
||||
// sentence, and a sentence is not something a caller can branch on — which is exactly how a failed
|
||||
// fetch and a wrong recovery code came to produce one customer-facing message.
|
||||
if status < 200 || status > 299 || !env.OK {
|
||||
return "", "", &RecoveryRefusal{Status: status, Reason: truncateErr(env.Error, 300)}
|
||||
}
|
||||
var out struct {
|
||||
ResticRepoPassword string `json:"restic_repo_password"`
|
||||
ResticPwSHA256 string `json:"restic_pw_sha256"`
|
||||
}
|
||||
if uerr := json.Unmarshal(env.Data, &out); uerr != nil {
|
||||
return "", "", fmt.Errorf("agentapi: decode /escrow/recover-offsite-password: %w", uerr)
|
||||
}
|
||||
if out.ResticRepoPassword == "" || out.ResticPwSHA256 == "" {
|
||||
return "", "", fmt.Errorf("agentapi: the agent returned an empty recovery result")
|
||||
}
|
||||
return out.ResticRepoPassword, out.ResticPwSHA256, nil
|
||||
}
|
||||
|
||||
// ── R-224 — CLASSIFYING A FAILED UNLOCK ─────────────────────────────────────────────────────────
|
||||
//
|
||||
// CAMPAIGN-11 measured what happens without this. On 2026-08-05, with a CORRECT current recovery
|
||||
// code: the hub firewalled off returned the customer "this code does not open your package" in
|
||||
// 0.0556 s, and this agent stopped returned the same in 0.0299 s — against ~1.0 s for a genuine
|
||||
// unseal. Neither attempted one. The failure path had exactly two branches, both of them statements
|
||||
// about the customer's code, and `rerr` was never inspected.
|
||||
//
|
||||
// The rule this type exists to enforce: **the customer is blamed only after a real attempt refused
|
||||
// their code.** Everything else — including anything we cannot classify — says something else.
|
||||
|
||||
// RecoveryRefusal is the agent's refusal of an unlock, carrying the STATUS as a value so callers
|
||||
// classify on it rather than on the sentence. The message keeps `refusalError`'s shape so operator
|
||||
// logs read as they did.
|
||||
type RecoveryRefusal struct {
|
||||
Status int
|
||||
Reason string
|
||||
}
|
||||
|
||||
func (e *RecoveryRefusal) Error() string {
|
||||
reason := e.Reason
|
||||
if reason == "" {
|
||||
reason = "(no reason in agent response)"
|
||||
}
|
||||
return fmt.Sprintf("agentapi: POST /escrow/recover-offsite-password: HTTP %d: %s", e.Status, reason)
|
||||
}
|
||||
|
||||
// RecoveryFailure is what went wrong, as far as it can be known.
|
||||
type RecoveryFailure int
|
||||
|
||||
const (
|
||||
// RecoveryUnknown — the cause could not be determined. **The safe default**, and deliberately the
|
||||
// zero value: a new status, a transport shape nobody anticipated, or an agent too old to
|
||||
// distinguish fetch from refusal all land here, and none of them may blame the customer.
|
||||
RecoveryUnknown RecoveryFailure = iota
|
||||
// RecoveryHubUnreachable — the agent answered, and it could not FETCH the sealed package: the hub
|
||||
// refused, was unreachable, or recovery is not configured on this agent. **The code was not used.**
|
||||
RecoveryHubUnreachable
|
||||
// RecoveryAskedAndRefused — the bundle was fetched and the code did not open it. The ONLY class
|
||||
// from which the customer may be told to check their typing.
|
||||
RecoveryAskedAndRefused
|
||||
// RecoveryNoBundle — the hub holds no sealed package for this host at all.
|
||||
RecoveryNoBundle
|
||||
// RecoveryBundleTooOld — the bundle opened but predates the repository-password field.
|
||||
RecoveryBundleTooOld
|
||||
// RecoveryAgentUnreachable — the machine's own in-house service never answered, so there is no
|
||||
// agent verdict at all. **The code was not used.** Distinct from RecoveryHubUnreachable because
|
||||
// it is a different fault, with different words and a different remedy.
|
||||
RecoveryAgentUnreachable
|
||||
// RecoveryCodeOpensRetained — the code was used, it WORKED, and it opened a RETAINED earlier
|
||||
// package rather than the one currently held (R-311, agent >= v0.129.0).
|
||||
//
|
||||
// **The customer is not at fault here and must not be told they might be.** This class exists
|
||||
// because until 2026-08-12 this situation and a mistype were indistinguishable: both fail closed
|
||||
// against the current package, and nothing ever tried the retained ones. The screen said as much
|
||||
// out loud — a true sentence about our own incuriosity that a customer reads as a statement about
|
||||
// their code.
|
||||
RecoveryCodeOpensRetained
|
||||
)
|
||||
|
||||
// ClassifyRecoveryFailure maps an unlock error to its class, from the VALUE and never the text.
|
||||
//
|
||||
// ⚠ `trustRefusal` is the agent-version gate and it is not optional. An agent older than v0.126.0
|
||||
// answers **400 for BOTH** a fetch failure and a wrong code, so a 400 from one cannot be read as
|
||||
// "the code was refused" — it means "one of two things, and we cannot tell which". Pass false there
|
||||
// and the 400 degrades to RecoveryUnknown, which is neutral. That degradation is the point: it is
|
||||
// safe, it is silent, and it heals itself when the agent updates.
|
||||
// ⚠ `trustRetained` is the R-311 twin of `trustRefusal` and is separate on purpose: the two gates
|
||||
// name different agent versions (v0.126.0 and v0.129.0) and a box can sit between them. Passing
|
||||
// `trustRefusal` for both would let a v0.126–128 agent's unexpected 422 be read as a verdict it
|
||||
// cannot produce.
|
||||
func ClassifyRecoveryFailure(err error, trustRefusal, trustRetained bool) RecoveryFailure {
|
||||
if err == nil {
|
||||
return RecoveryUnknown
|
||||
}
|
||||
var ref *RecoveryRefusal
|
||||
if !errors.As(err, &ref) {
|
||||
// Not a refusal at all — the request never produced an agent verdict (dial failure, TLS,
|
||||
// timeout, or the channel could not be built). The machine could not even ASK its own service,
|
||||
// which is a different sentence from "the hub was unreachable" and a different thing to fix.
|
||||
return RecoveryAgentUnreachable
|
||||
}
|
||||
switch ref.Status {
|
||||
case http.StatusBadGateway, http.StatusServiceUnavailable, http.StatusGatewayTimeout:
|
||||
// 502 is agent >= v0.126.0's "the sealed bundle could not be fetched". 503 is its
|
||||
// "recovery is not configured on this agent (no hub client)". Neither used the code.
|
||||
return RecoveryHubUnreachable
|
||||
case http.StatusNotFound:
|
||||
return RecoveryNoBundle
|
||||
case http.StatusConflict:
|
||||
return RecoveryBundleTooOld
|
||||
case http.StatusUnprocessableEntity:
|
||||
// R-311. Gated on the SAME trust flag as 400, and for the mirror-image reason: an agent that
|
||||
// predates the retained lookup cannot emit 422 at all, so a 422 from anywhere else is a shape
|
||||
// we did not design and must not be read as a statement about the customer's code.
|
||||
if trustRetained {
|
||||
return RecoveryCodeOpensRetained
|
||||
}
|
||||
return RecoveryUnknown
|
||||
case http.StatusBadRequest:
|
||||
if trustRefusal {
|
||||
return RecoveryAskedAndRefused
|
||||
}
|
||||
return RecoveryUnknown
|
||||
default:
|
||||
return RecoveryUnknown
|
||||
}
|
||||
}
|
||||
|
||||
// String names the class for the operator log. The customer never sees these words.
|
||||
func (f RecoveryFailure) String() string {
|
||||
switch f {
|
||||
case RecoveryHubUnreachable:
|
||||
return "hub-unreachable"
|
||||
case RecoveryAgentUnreachable:
|
||||
return "agent-unreachable"
|
||||
case RecoveryAskedAndRefused:
|
||||
return "asked-and-refused"
|
||||
case RecoveryNoBundle:
|
||||
return "no-bundle"
|
||||
case RecoveryBundleTooOld:
|
||||
return "bundle-too-old"
|
||||
case RecoveryCodeOpensRetained:
|
||||
return "code-opens-retained"
|
||||
default:
|
||||
return "unknown"
|
||||
}
|
||||
}
|
||||
@@ -1,328 +0,0 @@
|
||||
package agentapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"net/http"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/logx"
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/util"
|
||||
)
|
||||
|
||||
// Agent-capability probing (the publish-train backstop). A controller release that depends on
|
||||
// coupled agent behavior must not fail mid-pipeline against an older agent — it detects support up
|
||||
// front and refuses honestly. Detection is a ROUTE PROBE: a route that shipped together with the
|
||||
// coupled semantics either answers (2xx ⇒ supported) or 404s (older agent). Transport errors and
|
||||
// 5xx are INDETERMINATE — an agent problem is never claimed as "too old".
|
||||
|
||||
// Feature names one coupled controller↔agent capability.
|
||||
type Feature string
|
||||
|
||||
// FeatureNetstorageVerify is the NAS verify-before-commit add semantics (agent v0.81.0): the
|
||||
// coupled add behavior shipped together with GET /netstorage/verify-status, so that route IS the
|
||||
// capability signal.
|
||||
const FeatureNetstorageVerify Feature = "netstorage_verify"
|
||||
|
||||
// FeatureGuestMemoryResize is the guest RAM resize (agent v0.90.0, R-24): the resize endpoints
|
||||
// (GET/POST /guest/memory) shipped together, so GET /guest/memory IS the capability signal.
|
||||
const FeatureGuestMemoryResize Feature = "guest_memory_resize"
|
||||
|
||||
// FeatureBackupAgeState is R-88 Part 2 (agent v0.105.0): GET /backup/due carries `age_state`,
|
||||
// distinguishing "never backed up" (absent) from "could not tell" (unknown). There is no route
|
||||
// probe for it — the signal is a FIELD on an existing route, so the version floor is the gate and
|
||||
// an empty field means legacy.
|
||||
const FeatureBackupAgeState Feature = "backup_age_state"
|
||||
|
||||
// FeatureOffsiteKeyRecovery is the customer-facing off-site key recovery (agent v0.125.0, R-199
|
||||
// links 7–8): POST /escrow/recover-offsite-password fetches this host's sealed bundle, unseals it
|
||||
// with R and returns the single repository-password field.
|
||||
//
|
||||
// ⚠ THIS GATE FAILS CLOSED, and it is the ONLY feature in this table that does. Read §7.1 of the
|
||||
// R-216 fix before "correcting" it back to the package default.
|
||||
//
|
||||
// The package default is fail-OPEN: SupportUnknown proceeds, because for every other coupled feature
|
||||
// a wrong "unsupported" would block something harmless while a down agent already speaks through the
|
||||
// normal error paths. **That default is what produced R-216.** Measured live on 2026-08-05
|
||||
// (CAMPAIGN-11 Phase 1): an agent 0.120.0 answered the recovery route with 404, the unlock attempt
|
||||
// went ahead anyway, and the customer was told — in Hungarian, on the one screen whose whole purpose
|
||||
// is to be believed about backups — that their perfectly correct recovery code was not accepted and
|
||||
// they should check their typing. A correct code, refused in 0.134 s, blamed on the customer.
|
||||
//
|
||||
// So here: anything other than SupportYes means the screen says THE MACHINE cannot ask yet. The
|
||||
// unlock is never attempted when it cannot complete, because the failure of an attempt that could
|
||||
// never have worked is attributed to the code.
|
||||
const FeatureOffsiteKeyRecovery Feature = "offsite_key_recovery"
|
||||
|
||||
// FeatureRecoveryFailureClass is agent v0.126.0's SPLIT of a failed unlock into distinguishable
|
||||
// statuses (R-224): 502 the sealed bundle could not be FETCHED · 400 it was fetched and the code was
|
||||
// refused · 404 no bundle · 409 the bundle predates the repository-password field.
|
||||
//
|
||||
// ⚠ WHAT THIS GATE ACTUALLY GUARDS is the meaning of **400**, and nothing else. An agent older than
|
||||
// v0.126.0 answers 400 for BOTH a fetch failure and a wrong code — one status, one sentence, two
|
||||
// situations — so on such an agent a 400 cannot be read as "the code was refused". It means "one of
|
||||
// two things and we cannot tell which", which is `RecoveryUnknown`, which is neutral.
|
||||
//
|
||||
// So this gate does not block anything and has no fail-closed behaviour to get wrong: the unlock is
|
||||
// attempted either way (FeatureOffsiteKeyRecovery already decides THAT). It only decides whether the
|
||||
// customer may be told to check their typing. Unknown → they may not. **That is the safe direction,
|
||||
// and it heals itself the moment the agent updates.**
|
||||
const FeatureRecoveryFailureClass Feature = "recovery_failure_class"
|
||||
|
||||
// FeatureRetainedRecoveryClass is agent v0.129.0's FIFTH status on a failed unlock (R-311): 422, the
|
||||
// code is correct and opens a RETAINED earlier package rather than the current one.
|
||||
//
|
||||
// ⚠ WHAT THIS GATE GUARDS is whether the screen may say WHICH of the two causes it is. Before
|
||||
// v0.129.0 nothing ever tried the retained packages, so a correct-but-earlier code and a mistype were
|
||||
// genuinely indistinguishable and the screen said so. That sentence was HONEST then and becomes a
|
||||
// falsehood the moment the agent can tell them apart — so the gate decides which of two true
|
||||
// sentences to print, never whether to attempt the unlock.
|
||||
//
|
||||
// Unknown → the older, hedged sentence. That is the safe direction: it claims less, it was correct
|
||||
// for two months, and it heals itself when the agent updates.
|
||||
const FeatureRetainedRecoveryClass Feature = "retained_recovery_class"
|
||||
|
||||
// SupportState is a probe verdict. The zero value is SupportUnknown (fail-open: unknown never
|
||||
// refuses — the existing agent-error paths speak honestly when the agent is down).
|
||||
type SupportState int
|
||||
|
||||
const (
|
||||
// SupportUnknown — the probe could not decide (transport error, timeout, auth, 5xx).
|
||||
SupportUnknown SupportState = iota
|
||||
// SupportYes — the agent answered 2xx on the feature's probe route.
|
||||
SupportYes
|
||||
// SupportNo — the agent answered 404: it predates the route, and with it the coupled semantics.
|
||||
SupportNo
|
||||
)
|
||||
|
||||
// String returns the state's wire/template vocabulary: "yes" | "no" | "unknown".
|
||||
func (s SupportState) String() string {
|
||||
switch s {
|
||||
case SupportYes:
|
||||
return "yes"
|
||||
case SupportNo:
|
||||
return "no"
|
||||
default:
|
||||
return "unknown"
|
||||
}
|
||||
}
|
||||
|
||||
// SupportProber is the minimal agent surface a probe needs. *Client satisfies it, and so does the
|
||||
// web layer's netAgent seam — tests inject fakes there.
|
||||
type SupportProber interface {
|
||||
NetVerifyStatus(ctx context.Context) (NetVerifyStatus, error)
|
||||
}
|
||||
|
||||
// featureProbes maps each coupled feature to its route probe (the FALLBACK path for agents that
|
||||
// predate the v0.82.0 version header).
|
||||
//
|
||||
// CONVENTION (publish-train rules doc, felhom.eu/documentation/runbooks/publish-train-rules.md):
|
||||
// every future coupled feature adds a row here AND a featureMinAgent row, plus a Supports gate
|
||||
// call at its entry point, and declares MinAgent in its CHANGELOG header.
|
||||
var featureProbes = map[Feature]func(ctx context.Context, p SupportProber) error{
|
||||
FeatureNetstorageVerify: func(ctx context.Context, p SupportProber) error {
|
||||
_, err := p.NetVerifyStatus(ctx)
|
||||
return err
|
||||
},
|
||||
// The memory-resize prober needs GET /guest/memory, not NetVerifyStatus. Rather than couple the
|
||||
// shared SupportProber (and every unrelated prober/fake) to the memory surface, the probe
|
||||
// type-asserts the ONE method it needs — the memory feature is only ever probed with a
|
||||
// GuestMemory-capable prober (the web memAgent seam / *Client). A prober without it → a non-404
|
||||
// error → SupportUnknown (fail-open), never a false "supported".
|
||||
FeatureGuestMemoryResize: func(ctx context.Context, p SupportProber) error {
|
||||
gm, ok := p.(interface {
|
||||
GuestMemory(ctx context.Context) (GuestMemoryInfo, error)
|
||||
})
|
||||
if !ok {
|
||||
return errNoMemoryProbe
|
||||
}
|
||||
_, err := gm.GuestMemory(ctx)
|
||||
return err
|
||||
},
|
||||
// The recovery route is a POST that performs work and consumes a recovery code — it cannot be
|
||||
// probed. Like the memory prober's negative case this returns a sentinel that classifies to
|
||||
// SupportUnknown, so the decision falls to the VERSION path above.
|
||||
//
|
||||
// The row must exist even though it cannot probe: SupportsWithSource looks up featureProbes
|
||||
// FIRST and returns "unregistered"/SupportUnknown on a table gap, before the version path runs.
|
||||
// A featureMinAgent row without a featureProbes row is therefore never consulted at all.
|
||||
FeatureOffsiteKeyRecovery: func(ctx context.Context, p SupportProber) error {
|
||||
return errNoRecoveryProbe
|
||||
},
|
||||
// Same POST route, same reason it cannot be probed — the decision falls to the VERSION path.
|
||||
FeatureRecoveryFailureClass: func(ctx context.Context, p SupportProber) error {
|
||||
return errNoRecoveryProbe
|
||||
},
|
||||
// R-311, same route and same reason. The row must exist or SupportsWithSource returns
|
||||
// "unregistered"/SupportUnknown on the table gap and the version row is never consulted.
|
||||
FeatureRetainedRecoveryClass: func(ctx context.Context, p SupportProber) error {
|
||||
return errNoRecoveryProbe
|
||||
},
|
||||
}
|
||||
|
||||
// errNoMemoryProbe classifies to SupportUnknown (not a *StatusError 404), so a prober that cannot be
|
||||
// asked never reads as "unsupported".
|
||||
var errNoMemoryProbe = errors.New("agentapi: prober does not support the guest-memory probe")
|
||||
|
||||
// errNoRecoveryProbe classifies to SupportUnknown: the off-site key recovery route is a POST that
|
||||
// consumes a recovery code and so cannot be probed, leaving the VERSION path to decide. Its caller
|
||||
// fails CLOSED on Unknown — see FeatureOffsiteKeyRecovery.
|
||||
var errNoRecoveryProbe = errors.New("agentapi: the offsite key recovery route cannot be probed")
|
||||
|
||||
// featureMinAgent maps each coupled feature to the MINIMUM agent version that carries its coupled
|
||||
// semantics (the CHANGELOG `MinAgent:` header value). Used by Supports when the agent's version is
|
||||
// KNOWN (the v0.82.0 X-Felhom-Agent-Version channel) — a direct comparison, no probe traffic. A
|
||||
// feature missing here (or an unparseable table value) falls back to the probe.
|
||||
var featureMinAgent = map[Feature]string{
|
||||
FeatureNetstorageVerify: "0.81.0",
|
||||
FeatureGuestMemoryResize: "0.90.0",
|
||||
// R-88 Part 2: /backup/due carries age_state, distinguishing "never backed up" from "cannot tell".
|
||||
FeatureBackupAgeState: "0.105.0",
|
||||
// R-199 links 7–8: POST /escrow/recover-offsite-password. R-216 — this row is the whole reason a
|
||||
// correct recovery code can no longer be reported as wrong on an agent that cannot answer.
|
||||
FeatureOffsiteKeyRecovery: "0.125.0",
|
||||
|
||||
// R-224 — the four-way status split of a failed unlock.
|
||||
FeatureRecoveryFailureClass: "0.126.0",
|
||||
|
||||
// R-311 — the FIFTH status: 422, "your code is correct, it opens an EARLIER package". Before
|
||||
// v0.129.0 the agent never looked at retained packages, so this situation was indistinguishable
|
||||
// from a mistype and arrived as 400. An older agent therefore cannot produce a 422 at all, and the
|
||||
// screen must keep saying it cannot tell the two apart — which was true, and is what this gate
|
||||
// preserves for boxes that have not updated yet.
|
||||
FeatureRetainedRecoveryClass: "0.129.0",
|
||||
}
|
||||
|
||||
// MinAgentFor returns the declared minimum agent version for a feature ("" when the feature has no
|
||||
// row). Read-only accessor over featureMinAgent so a refusal can NAME the version it needs instead of
|
||||
// hard-coding the number a second time at the call site.
|
||||
func MinAgentFor(f Feature) string { return featureMinAgent[f] }
|
||||
|
||||
// AgentVersionReporter is optionally implemented by a SupportProber (*Client is one): it reports
|
||||
// the last strictly-validated agent version seen on its traffic ("" = unknown → probe fallback).
|
||||
type AgentVersionReporter interface {
|
||||
AgentVersion() string
|
||||
}
|
||||
|
||||
// supportTTL bounds how long a probe verdict (either polarity) is trusted. An agent updated
|
||||
// mid-window flips within this — no invalidation plumbing by design.
|
||||
const supportTTL = 5 * time.Minute
|
||||
|
||||
type supportEntry struct {
|
||||
state SupportState
|
||||
at time.Time
|
||||
}
|
||||
|
||||
// SupportCache caches per-feature probe verdicts. Yes and No are cached for supportTTL; Unknown is
|
||||
// NEVER cached (a down agent re-probes on the next call, so recovery is immediate). The zero value
|
||||
// is ready to use.
|
||||
type SupportCache struct {
|
||||
mu sync.Mutex
|
||||
now func() time.Time // test seam; nil → time.Now
|
||||
entries map[Feature]supportEntry
|
||||
}
|
||||
|
||||
// Supports reports whether the agent behind p provides feature f.
|
||||
//
|
||||
// Order (v0.115.0): (1) the agent's VERSION is known (the v0.82.0 header channel, strictly
|
||||
// validated at capture) AND the feature has a MinAgent row → pure semver comparison, NO probe
|
||||
// traffic, no cache involvement (the cache stays probe-only); (2) otherwise — pre-0.82 agent, no
|
||||
// traffic yet, or a table gap — the v0.114.0 probe path, byte-identical (cache inside the TTL
|
||||
// window, probe on miss). The probe runs OUTSIDE the lock — concurrent misses may double-probe
|
||||
// (harmless: the probe is one cheap GET).
|
||||
func (sc *SupportCache) Supports(ctx context.Context, p SupportProber, f Feature) SupportState {
|
||||
state, _ := sc.SupportsWithSource(ctx, p, f)
|
||||
return state
|
||||
}
|
||||
|
||||
// SupportsWithSource is Supports plus the DECISION SOURCE ("version" | "probe-cache" |
|
||||
// "probe" | "unregistered") — the v0.116.0 observability extension so the gate line can
|
||||
// say HOW the verdict was reached. Behavior is byte-identical to v0.115.0 Supports.
|
||||
func (sc *SupportCache) SupportsWithSource(ctx context.Context, p SupportProber, f Feature) (SupportState, string) {
|
||||
probe, ok := featureProbes[f]
|
||||
if !ok {
|
||||
return SupportUnknown, "unregistered" // unregistered feature — never refuse on a table gap
|
||||
}
|
||||
if vr, hasVer := p.(AgentVersionReporter); hasVer {
|
||||
if state, decided := supportsByVersion(vr.AgentVersion(), f); decided {
|
||||
return state, "version"
|
||||
}
|
||||
}
|
||||
sc.mu.Lock()
|
||||
nowFn := sc.now
|
||||
if nowFn == nil {
|
||||
nowFn = time.Now
|
||||
}
|
||||
if e, hit := sc.entries[f]; hit && nowFn().Sub(e.at) < supportTTL {
|
||||
sc.mu.Unlock()
|
||||
return e.state, "probe-cache"
|
||||
}
|
||||
sc.mu.Unlock()
|
||||
|
||||
state := classifySupportErr(probe(ctx, p))
|
||||
if state != SupportUnknown {
|
||||
sc.mu.Lock()
|
||||
if sc.entries == nil {
|
||||
sc.entries = map[Feature]supportEntry{}
|
||||
}
|
||||
sc.entries[f] = supportEntry{state: state, at: nowFn()}
|
||||
sc.mu.Unlock()
|
||||
}
|
||||
return state, "probe"
|
||||
}
|
||||
|
||||
// Supports probes (cached, TTL 5m, both polarities) whether the connected agent provides the
|
||||
// feature. 2xx ⇒ Yes. 404 ⇒ No. Anything else ⇒ Unknown (never "too old"). The web layer drives
|
||||
// the same machinery through its netAgent seam (Server.netFeatures) so tests can fake the probe.
|
||||
func (c *Client) Supports(ctx context.Context, f Feature) SupportState {
|
||||
state, source := c.features.SupportsWithSource(ctx, c, f)
|
||||
logx.Debugf(c.logger, "[agentapi] Supports(%s) = %s (source=%s agent_version=%q)",
|
||||
f, state, source, c.AgentVersion())
|
||||
return state
|
||||
}
|
||||
|
||||
// supportsByVersion decides a feature by version comparison alone. decided=false (unknown/garbage
|
||||
// version, missing MinAgent row, unparseable table value) sends the caller to the probe fallback —
|
||||
// a bad version string must never be trusted in EITHER direction.
|
||||
func supportsByVersion(agentVer string, f Feature) (state SupportState, decided bool) {
|
||||
if agentVer == "" {
|
||||
return SupportUnknown, false
|
||||
}
|
||||
minStr, ok := featureMinAgent[f]
|
||||
if !ok {
|
||||
return SupportUnknown, false
|
||||
}
|
||||
av, err := util.ParseVersion(agentVer)
|
||||
if err != nil {
|
||||
return SupportUnknown, false // capture validates the shape, but stay defensive
|
||||
}
|
||||
mv, err := util.ParseVersion(minStr)
|
||||
if err != nil {
|
||||
return SupportUnknown, false // a broken table row falls back to probing, never refuses
|
||||
}
|
||||
if av.Compare(mv) >= 0 {
|
||||
return SupportYes, true
|
||||
}
|
||||
return SupportNo, true
|
||||
}
|
||||
|
||||
// classifySupportErr maps a probe outcome to a SupportState. ONLY a typed HTTP 404 means
|
||||
// "unsupported" — every other error (transport, timeout, 401, 5xx, envelope problems) is Unknown,
|
||||
// so a merely-down agent is never reported as outdated. Typed errors only; never string-match.
|
||||
func classifySupportErr(err error) SupportState {
|
||||
if err == nil {
|
||||
return SupportYes
|
||||
}
|
||||
var se *StatusError
|
||||
if errors.As(err, &se) && se.Code == http.StatusNotFound {
|
||||
return SupportNo
|
||||
}
|
||||
return SupportUnknown
|
||||
}
|
||||
|
||||
// AgentVersionReporter witness. Asserted at SupportsWithSource (`p.(AgentVersionReporter)`); a failed
|
||||
// assertion falls back from the version gate to the live probe. That degrade is benign — both paths
|
||||
// decide correctly — but *Client is the production prober and losing the version path would silently
|
||||
// turn every MinAgent floor into a probe round-trip, which is a behaviour change nobody would see.
|
||||
var _ AgentVersionReporter = (*Client)(nil)
|
||||
@@ -1,164 +0,0 @@
|
||||
package agentapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// fakeProber scripts the capability probe (the SupportProber seam) and counts calls.
|
||||
type fakeProber struct {
|
||||
err error
|
||||
calls int
|
||||
}
|
||||
|
||||
func (f *fakeProber) NetVerifyStatus(context.Context) (NetVerifyStatus, error) {
|
||||
f.calls++
|
||||
return NetVerifyStatus{Phase: "none"}, f.err
|
||||
}
|
||||
|
||||
// --- T4: probe classification — ONLY a typed 404 means "too old" -------------------------------
|
||||
// Companion red-proof (the classification trap): mutate classifySupportErr to string-match
|
||||
// "HTTP 404" → the "plain error with 404 text" case classifies No → FAIL. A second mutant treating
|
||||
// ANY error as SupportNo fails every Unknown case here (and T3 in web).
|
||||
func TestSupports_Classification(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
err error
|
||||
want SupportState
|
||||
}{
|
||||
{"2xx (nil error)", nil, SupportYes},
|
||||
{"typed 404", &StatusError{Path: "/netstorage/verify-status", Code: http.StatusNotFound}, SupportNo},
|
||||
{"typed 404 wrapped", fmt.Errorf("probe: %w", &StatusError{Path: "/x", Code: 404}), SupportNo},
|
||||
{"typed 401", &StatusError{Path: "/x", Code: http.StatusUnauthorized}, SupportUnknown},
|
||||
{"typed 500", &StatusError{Path: "/x", Code: http.StatusInternalServerError}, SupportUnknown},
|
||||
{"typed 502", &StatusError{Path: "/x", Code: http.StatusBadGateway}, SupportUnknown},
|
||||
{"connection refused", errors.New("dial tcp 10.0.0.9:8443: connect: connection refused"), SupportUnknown},
|
||||
{"timeout", context.DeadlineExceeded, SupportUnknown},
|
||||
// The string-matching trap: the OLD untyped error text carries "HTTP 404" but is NOT a
|
||||
// *StatusError — a classifier that matches the message would wrongly say "too old".
|
||||
{"plain error with 404 text", errors.New("agentapi: GET /netstorage/verify-status: HTTP 404"), SupportUnknown},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
sc := &SupportCache{}
|
||||
got := sc.Supports(context.Background(), &fakeProber{err: tc.err}, FeatureNetstorageVerify)
|
||||
if got != tc.want {
|
||||
t.Errorf("Supports(%v) = %v, want %v", tc.err, got, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Unknown feature names must never refuse (a table gap fails open).
|
||||
func TestSupports_UnknownFeature(t *testing.T) {
|
||||
sc := &SupportCache{}
|
||||
p := &fakeProber{}
|
||||
if got := sc.Supports(context.Background(), p, Feature("no_such_feature")); got != SupportUnknown {
|
||||
t.Errorf("unknown feature = %v, want SupportUnknown", got)
|
||||
}
|
||||
if p.calls != 0 {
|
||||
t.Errorf("unknown feature must not probe (calls=%d)", p.calls)
|
||||
}
|
||||
}
|
||||
|
||||
// --- T6: cache TTL — both polarities cached; Unknown NEVER cached ------------------------------
|
||||
// Companion red-proof: drop the TTL expiry check (treat every entry as fresh) → the
|
||||
// "re-probes after TTL" assertion fails. Dropping the cache entirely → the warm-cache negative
|
||||
// assertion (calls stays 1) fails (T2's red-proof shape).
|
||||
func TestSupports_CacheTTL(t *testing.T) {
|
||||
t.Run("positive cached, re-probes after TTL", func(t *testing.T) {
|
||||
now := time.Date(2026, 7, 11, 12, 0, 0, 0, time.UTC)
|
||||
sc := &SupportCache{now: func() time.Time { return now }}
|
||||
p := &fakeProber{} // nil err → Yes
|
||||
if got := sc.Supports(context.Background(), p, FeatureNetstorageVerify); got != SupportYes {
|
||||
t.Fatalf("first = %v, want Yes", got)
|
||||
}
|
||||
if got := sc.Supports(context.Background(), p, FeatureNetstorageVerify); got != SupportYes {
|
||||
t.Fatalf("warm = %v, want Yes", got)
|
||||
}
|
||||
if p.calls != 1 { // the NEGATIVE assertion: a warm cache must NOT re-probe
|
||||
t.Errorf("probe calls on a warm cache = %d, want 1", p.calls)
|
||||
}
|
||||
now = now.Add(supportTTL + time.Second)
|
||||
if got := sc.Supports(context.Background(), p, FeatureNetstorageVerify); got != SupportYes {
|
||||
t.Fatalf("post-TTL = %v, want Yes", got)
|
||||
}
|
||||
if p.calls != 2 {
|
||||
t.Errorf("probe calls after TTL expiry = %d, want 2 (must re-fire)", p.calls)
|
||||
}
|
||||
})
|
||||
t.Run("negative cached too", func(t *testing.T) {
|
||||
now := time.Date(2026, 7, 11, 12, 0, 0, 0, time.UTC)
|
||||
sc := &SupportCache{now: func() time.Time { return now }}
|
||||
p := &fakeProber{err: &StatusError{Path: "/x", Code: 404}}
|
||||
for i := 0; i < 2; i++ {
|
||||
if got := sc.Supports(context.Background(), p, FeatureNetstorageVerify); got != SupportNo {
|
||||
t.Fatalf("call %d = %v, want No", i, got)
|
||||
}
|
||||
}
|
||||
if p.calls != 1 {
|
||||
t.Errorf("negative verdict not cached (calls=%d, want 1)", p.calls)
|
||||
}
|
||||
})
|
||||
t.Run("unknown never cached", func(t *testing.T) {
|
||||
sc := &SupportCache{}
|
||||
p := &fakeProber{err: errors.New("connection refused")}
|
||||
for i := 0; i < 2; i++ {
|
||||
if got := sc.Supports(context.Background(), p, FeatureNetstorageVerify); got != SupportUnknown {
|
||||
t.Fatalf("call %d = %v, want Unknown", i, got)
|
||||
}
|
||||
}
|
||||
if p.calls != 2 {
|
||||
t.Errorf("Unknown must re-probe every call (calls=%d, want 2)", p.calls)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// --- 1.1 wire-level: a real HTTP 404 through Client.get IS the typed StatusError ----------------
|
||||
// This pins the verify-first finding: an agent without the route (≤0.80's plain mux 404) reaches
|
||||
// classifySupportErr as *StatusError{404}, end-to-end through the pinned-TLS client.
|
||||
func TestClient_404IsTypedAndSupportsSaysNo(t *testing.T) {
|
||||
mux := http.NewServeMux() // NO /netstorage/verify-status route — the ≤0.80 shape
|
||||
srv := httptest.NewTLSServer(mux)
|
||||
defer srv.Close()
|
||||
fp := sha256.Sum256(srv.Certificate().Raw)
|
||||
c, err := New(strings.TrimPrefix(srv.URL, "https://"), "test-token", hex.EncodeToString(fp[:]))
|
||||
if err != nil {
|
||||
t.Fatalf("New: %v", err)
|
||||
}
|
||||
_, verr := c.NetVerifyStatus(context.Background())
|
||||
var se *StatusError
|
||||
if !errors.As(verr, &se) || se.Code != http.StatusNotFound {
|
||||
t.Fatalf("404 must surface as *StatusError{404}, got %T: %v", verr, verr)
|
||||
}
|
||||
if got := c.Supports(context.Background(), FeatureNetstorageVerify); got != SupportNo {
|
||||
t.Errorf("Supports on a routeless agent = %v, want SupportNo", got)
|
||||
}
|
||||
}
|
||||
|
||||
// The supported shape: the route answers the envelope → Supports says Yes.
|
||||
func TestClient_SupportsYesOnLiveRoute(t *testing.T) {
|
||||
mux := http.NewServeMux()
|
||||
mux.HandleFunc("/netstorage/verify-status", func(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
_, _ = w.Write([]byte(`{"ok":true,"data":{"phase":"none"}}`)) // the no-job envelope
|
||||
})
|
||||
srv := httptest.NewTLSServer(mux)
|
||||
defer srv.Close()
|
||||
fp := sha256.Sum256(srv.Certificate().Raw)
|
||||
c, err := New(strings.TrimPrefix(srv.URL, "https://"), "test-token", hex.EncodeToString(fp[:]))
|
||||
if err != nil {
|
||||
t.Fatalf("New: %v", err)
|
||||
}
|
||||
if got := c.Supports(context.Background(), FeatureNetstorageVerify); got != SupportYes {
|
||||
t.Errorf("Supports on a live route = %v, want SupportYes", got)
|
||||
}
|
||||
}
|
||||
@@ -1,179 +0,0 @@
|
||||
package agentapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/util"
|
||||
)
|
||||
|
||||
// Version-aware Supports (v0.115.0, pairs with agent v0.82.0's X-Felhom-Agent-Version): a KNOWN
|
||||
// version decides by comparison with ZERO probe traffic; unknown/garbage stays on the v0.114.0
|
||||
// probe path byte-identically.
|
||||
|
||||
// verProber implements SupportProber + AgentVersionReporter with a probe counter.
|
||||
type verProber struct {
|
||||
ver string
|
||||
probes int
|
||||
}
|
||||
|
||||
func (p *verProber) NetVerifyStatus(context.Context) (NetVerifyStatus, error) {
|
||||
p.probes++
|
||||
return NetVerifyStatus{Phase: "none"}, nil // a live route (probe verdict would be Yes)
|
||||
}
|
||||
func (p *verProber) AgentVersion() string { return p.ver }
|
||||
|
||||
// --- B3.1: version known → compare path, probe count 0 -----------------------------------------
|
||||
// Companion red-proof: drop the supportsByVersion short-circuit in Supports → probes becomes 1.
|
||||
func TestSupports_VersionKnown_ComparesWithoutProbe(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
ver string
|
||||
want SupportState
|
||||
}{
|
||||
{"0.82.0", SupportYes},
|
||||
{"0.81.0", SupportYes}, // boundary: MinAgent itself qualifies
|
||||
{"1.0.0", SupportYes}, // numeric compare across majors
|
||||
{"0.100.0", SupportYes}, // numeric, NOT lexicographic (0.100 > 0.81)
|
||||
{"0.79.0", SupportNo}, // below MinAgent → No, still without probing
|
||||
{"0.80.9", SupportNo},
|
||||
} {
|
||||
t.Run(tc.ver, func(t *testing.T) {
|
||||
p := &verProber{ver: tc.ver}
|
||||
var sc SupportCache
|
||||
if got := sc.Supports(context.Background(), p, FeatureNetstorageVerify); got != tc.want {
|
||||
t.Errorf("Supports(ver=%s) = %v, want %v", tc.ver, got, tc.want)
|
||||
}
|
||||
if p.probes != 0 {
|
||||
t.Errorf("version-known path must NOT probe (ver=%s, probes=%d)", tc.ver, p.probes)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// --- B3.2: garbage/absent version → the probe fallback, byte-identical --------------------------
|
||||
// Companion red-proof: trust the unvalidated header text (compare without ParseVersion error
|
||||
// handling) → the garbage rows would refuse or panic instead of probing.
|
||||
func TestSupports_GarbageOrNoVersion_ProbeFallback(t *testing.T) {
|
||||
for _, ver := range []string{"", "dev", "0.82", "0.82.0-rc1", "v0.82.0-beta", "evil;rm -rf", "9999999999999999999999.0.0"} {
|
||||
t.Run("ver="+ver, func(t *testing.T) {
|
||||
p := &verProber{ver: ver}
|
||||
var sc SupportCache
|
||||
got := sc.Supports(context.Background(), p, FeatureNetstorageVerify)
|
||||
if p.probes != 1 {
|
||||
t.Fatalf("unusable version %q must fall back to EXACTLY one probe, got %d", ver, p.probes)
|
||||
}
|
||||
if got != SupportYes { // the fake's route answers → the probe decides Yes
|
||||
t.Errorf("probe fallback verdict = %v, want SupportYes", got)
|
||||
}
|
||||
// Second call: the probe verdict is cached (the v0.114.0 behavior, unchanged).
|
||||
_ = sc.Supports(context.Background(), p, FeatureNetstorageVerify)
|
||||
if p.probes != 1 {
|
||||
t.Errorf("cached probe verdict must not re-probe (probes=%d)", p.probes)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// A prober that does NOT implement AgentVersionReporter (the web fakes' shape) keeps the pure
|
||||
// v0.114.0 behavior — the interface assertion must not change anything for it.
|
||||
func TestSupports_NonReporterProber_Unchanged(t *testing.T) {
|
||||
p := &fakeProber{} // the existing v0.114.0 test fake (features_test.go)
|
||||
var sc SupportCache
|
||||
if got := sc.Supports(context.Background(), p, FeatureNetstorageVerify); got != SupportYes {
|
||||
t.Fatalf("non-reporter prober = %v, want SupportYes via probe", got)
|
||||
}
|
||||
if p.calls != 1 {
|
||||
t.Errorf("non-reporter prober must probe exactly once, got %d", p.calls)
|
||||
}
|
||||
}
|
||||
|
||||
// --- B3.4: the ONE comparator, table-driven (incl. pre-release suffixes) ------------------------
|
||||
func TestVersionComparator_Table(t *testing.T) {
|
||||
lt := func(a, b string) {
|
||||
t.Helper()
|
||||
av, err1 := util.ParseVersion(a)
|
||||
bv, err2 := util.ParseVersion(b)
|
||||
if err1 != nil || err2 != nil {
|
||||
t.Fatalf("parse %q/%q: %v %v", a, b, err1, err2)
|
||||
}
|
||||
if av.Compare(bv) != -1 || bv.Compare(av) != 1 {
|
||||
t.Errorf("want %s < %s", a, b)
|
||||
}
|
||||
}
|
||||
lt("0.81.0", "0.82.0")
|
||||
lt("0.81.0", "0.100.0") // numeric minor, not lexicographic
|
||||
lt("0.99.9", "1.0.0")
|
||||
lt("1.2.3", "1.2.10")
|
||||
if v, err := util.ParseVersion("v0.82.0"); err != nil || v.Raw != "0.82.0" {
|
||||
t.Errorf("v-prefix must parse: %v %v", v, err)
|
||||
}
|
||||
if eq, _ := util.ParseVersion("0.81.0"); eq.Compare(eq) != 0 {
|
||||
t.Error("equal versions must compare 0")
|
||||
}
|
||||
// Pre-release suffixes are REJECTED by the house comparator — in Supports they mean "fall back
|
||||
// to the probe", never a trusted comparison.
|
||||
for _, bad := range []string{"1.2.3-rc1", "dev", "latest", "", "1.2", "a.b.c"} {
|
||||
if _, err := util.ParseVersion(bad); err == nil {
|
||||
t.Errorf("ParseVersion(%q) must error", bad)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// --- B3.5: wire-level — the header decides over the probe through the REAL pinned client --------
|
||||
// A ROUTELESS agent (probe would say No) that sends X-Felhom-Agent-Version 9.9.9 must be judged by
|
||||
// the VERSION (Yes): proof the channel takes precedence end-to-end, and that capture happens on an
|
||||
// ordinary (even failing) call.
|
||||
func TestClient_VersionHeaderWins_WireLevel(t *testing.T) {
|
||||
mux := http.NewServeMux() // NO /netstorage/verify-status (the pre-0.81 route shape)
|
||||
mux.HandleFunc("/storage", func(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("X-Felhom-Agent-Version", "9.9.9")
|
||||
_, _ = w.Write([]byte(`{"ok":true,"data":{"vmid":1,"mounts":[]}}`))
|
||||
})
|
||||
wrapped := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("X-Felhom-Agent-Version", "9.9.9") // every response, like the agent's wrap
|
||||
mux.ServeHTTP(w, r)
|
||||
})
|
||||
srv := httptest.NewTLSServer(wrapped)
|
||||
defer srv.Close()
|
||||
fp := sha256.Sum256(srv.Certificate().Raw)
|
||||
c, err := New(strings.TrimPrefix(srv.URL, "https://"), "test-token", hex.EncodeToString(fp[:]))
|
||||
if err != nil {
|
||||
t.Fatalf("New: %v", err)
|
||||
}
|
||||
if _, err := c.Storage(context.Background()); err != nil {
|
||||
t.Fatalf("storage: %v", err)
|
||||
}
|
||||
if got := c.AgentVersion(); got != "9.9.9" {
|
||||
t.Fatalf("AgentVersion = %q, want 9.9.9 (passive capture)", got)
|
||||
}
|
||||
if got := c.Supports(context.Background(), FeatureNetstorageVerify); got != SupportYes {
|
||||
t.Errorf("Supports = %v, want SupportYes via version despite the missing probe route", got)
|
||||
}
|
||||
}
|
||||
|
||||
// A GARBAGE header must never be captured: the routeless agent stays on the probe → SupportNo.
|
||||
func TestClient_GarbageHeaderIgnored_WireLevel(t *testing.T) {
|
||||
wrapped := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("X-Felhom-Agent-Version", "not-a-version;x")
|
||||
http.NotFound(w, r)
|
||||
})
|
||||
srv := httptest.NewTLSServer(wrapped)
|
||||
defer srv.Close()
|
||||
fp := sha256.Sum256(srv.Certificate().Raw)
|
||||
c, err := New(strings.TrimPrefix(srv.URL, "https://"), "test-token", hex.EncodeToString(fp[:]))
|
||||
if err != nil {
|
||||
t.Fatalf("New: %v", err)
|
||||
}
|
||||
_, _ = c.Storage(context.Background()) // 404s; the garbage header must be dropped at capture
|
||||
if got := c.AgentVersion(); got != "" {
|
||||
t.Fatalf("AgentVersion = %q, want empty (strict validation at capture)", got)
|
||||
}
|
||||
if got := c.Supports(context.Background(), FeatureNetstorageVerify); got != SupportNo {
|
||||
t.Errorf("Supports = %v, want SupportNo via the probe fallback", got)
|
||||
}
|
||||
}
|
||||
@@ -1,132 +0,0 @@
|
||||
package agentapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// memProber satisfies the SupportProber contract (NetVerifyStatus) PLUS the memory probe's
|
||||
// type-asserted GuestMemory + the AgentVersion fast-path — the shape the web memAgent seam has.
|
||||
type memProber struct {
|
||||
memErr error
|
||||
ver string
|
||||
}
|
||||
|
||||
func (p *memProber) NetVerifyStatus(context.Context) (NetVerifyStatus, error) {
|
||||
return NetVerifyStatus{Phase: "none"}, nil
|
||||
}
|
||||
func (p *memProber) GuestMemory(context.Context) (GuestMemoryInfo, error) {
|
||||
return GuestMemoryInfo{AllocatedMB: 8192}, p.memErr
|
||||
}
|
||||
func (p *memProber) AgentVersion() string { return p.ver }
|
||||
|
||||
// The capability table: v0.90.0 → Yes, v0.89.0 → No via the version fast-path; the probe (no
|
||||
// version) classifies 404 → No, nil → Yes.
|
||||
func TestGuestMemory_Capability(t *testing.T) {
|
||||
t.Run("version >= 0.90 → Yes", func(t *testing.T) {
|
||||
var sc SupportCache
|
||||
if got := sc.Supports(context.Background(), &memProber{ver: "0.90.0"}, FeatureGuestMemoryResize); got != SupportYes {
|
||||
t.Errorf("v0.90.0 = %v, want SupportYes", got)
|
||||
}
|
||||
})
|
||||
t.Run("version < 0.90 → No", func(t *testing.T) {
|
||||
var sc SupportCache
|
||||
if got := sc.Supports(context.Background(), &memProber{ver: "0.89.0"}, FeatureGuestMemoryResize); got != SupportNo {
|
||||
t.Errorf("v0.89.0 = %v, want SupportNo", got)
|
||||
}
|
||||
})
|
||||
t.Run("no version, probe 404 → No", func(t *testing.T) {
|
||||
var sc SupportCache
|
||||
p := &memProber{memErr: &StatusError{Path: "/guest/memory", Code: http.StatusNotFound}}
|
||||
if got := sc.Supports(context.Background(), p, FeatureGuestMemoryResize); got != SupportNo {
|
||||
t.Errorf("probe 404 = %v, want SupportNo", got)
|
||||
}
|
||||
})
|
||||
t.Run("no version, probe ok → Yes", func(t *testing.T) {
|
||||
var sc SupportCache
|
||||
if got := sc.Supports(context.Background(), &memProber{}, FeatureGuestMemoryResize); got != SupportYes {
|
||||
t.Errorf("probe ok = %v, want SupportYes", got)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// GuestMemory decodes the agent's payload; a 404 (pre-0.90) is the typed *StatusError.
|
||||
func TestClient_GuestMemory(t *testing.T) {
|
||||
mux := http.NewServeMux()
|
||||
mux.HandleFunc("/guest/memory", func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method != http.MethodGet {
|
||||
w.WriteHeader(405)
|
||||
return
|
||||
}
|
||||
_, _ = w.Write([]byte(`{"ok":true,"data":{"vmid":9201,"allocated_mb":8192,"usage_mb":3000,"host_total_mb":16384,"min_mb":2048,"max_mb":14336,"floor_mb":3512,"running":true}}`))
|
||||
})
|
||||
c := newMemTestClient(t, mux)
|
||||
info, err := c.GuestMemory(context.Background())
|
||||
if err != nil {
|
||||
t.Fatalf("GuestMemory: %v", err)
|
||||
}
|
||||
if info.AllocatedMB != 8192 || info.UsageMB != 3000 || info.MaxMB != 14336 || info.FloorMB != 3512 || !info.Running {
|
||||
t.Errorf("decoded wrong: %+v", info)
|
||||
}
|
||||
}
|
||||
|
||||
func TestClient_GuestMemory_404(t *testing.T) {
|
||||
c := newMemTestClient(t, http.NewServeMux()) // no route → 404
|
||||
_, err := c.GuestMemory(context.Background())
|
||||
var se *StatusError
|
||||
if !errors.As(err, &se) || se.Code != http.StatusNotFound {
|
||||
t.Fatalf("pre-0.90 agent must 404 as *StatusError, got %T: %v", err, err)
|
||||
}
|
||||
}
|
||||
|
||||
// ResizeMemory: success returns old→new; a 412 refusal surfaces *MemoryRefusedError with the code.
|
||||
func TestClient_ResizeMemory(t *testing.T) {
|
||||
t.Run("success", func(t *testing.T) {
|
||||
mux := http.NewServeMux()
|
||||
mux.HandleFunc("/guest/memory", func(w http.ResponseWriter, r *http.Request) {
|
||||
_, _ = w.Write([]byte(`{"ok":true,"data":{"vmid":9201,"old_mb":8192,"new_mb":12288,"unchanged":false}}`))
|
||||
})
|
||||
c := newMemTestClient(t, mux)
|
||||
res, err := c.ResizeMemory(context.Background(), 12288)
|
||||
if err != nil {
|
||||
t.Fatalf("ResizeMemory: %v", err)
|
||||
}
|
||||
if res.OldMB != 8192 || res.NewMB != 12288 {
|
||||
t.Errorf("result = %+v", res)
|
||||
}
|
||||
})
|
||||
t.Run("refusal carries the code", func(t *testing.T) {
|
||||
mux := http.NewServeMux()
|
||||
mux.HandleFunc("/guest/memory", func(w http.ResponseWriter, r *http.Request) {
|
||||
w.WriteHeader(http.StatusPreconditionFailed)
|
||||
_, _ = w.Write([]byte(`{"ok":false,"error":"requested 3300 MB is too close to current usage 3000 MB (floor 3512 MB)","data":{"code":"below_usage_floor","usage_mb":3000,"floor_mb":3512,"min_mb":2048,"max_mb":14336}}`))
|
||||
})
|
||||
c := newMemTestClient(t, mux)
|
||||
_, err := c.ResizeMemory(context.Background(), 3300)
|
||||
var refused *MemoryRefusedError
|
||||
if !errors.As(err, &refused) {
|
||||
t.Fatalf("want *MemoryRefusedError, got %T: %v", err, err)
|
||||
}
|
||||
if refused.Code != "below_usage_floor" || refused.Bounds.UsageMB != 3000 || refused.Bounds.FloorMB != 3512 {
|
||||
t.Errorf("refusal = %+v", refused)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func newMemTestClient(t *testing.T, mux *http.ServeMux) *Client {
|
||||
t.Helper()
|
||||
srv := httptest.NewTLSServer(mux)
|
||||
t.Cleanup(srv.Close)
|
||||
fp := sha256.Sum256(srv.Certificate().Raw)
|
||||
c, err := New(strings.TrimPrefix(srv.URL, "https://"), "test-token", hex.EncodeToString(fp[:]))
|
||||
if err != nil {
|
||||
t.Fatalf("New: %v", err)
|
||||
}
|
||||
return c
|
||||
}
|
||||
@@ -1,106 +0,0 @@
|
||||
package agentapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func netStub(t *testing.T) (*httptest.Server, string, *struct {
|
||||
addBody AddNetStorageRequest
|
||||
removed string
|
||||
}) {
|
||||
captured := &struct {
|
||||
addBody AddNetStorageRequest
|
||||
removed string
|
||||
}{}
|
||||
mux := http.NewServeMux()
|
||||
mux.HandleFunc("POST /netstorage/add", func(w http.ResponseWriter, r *http.Request) {
|
||||
_ = decodeJSON(r, &captured.addBody)
|
||||
_, _ = w.Write([]byte(`{"ok":true,"data":{"name":"media","protocol":"nfs","where":"/mnt/felhom-drives/media","guest_path":"/mnt/felhom-drives/media","host_uid":101000,"host_gid":101000}}`))
|
||||
})
|
||||
mux.HandleFunc("GET /netstorage", func(w http.ResponseWriter, r *http.Request) {
|
||||
_, _ = w.Write([]byte(`{"ok":true,"data":{"vmid":8200,"network_mounts":[
|
||||
{"name":"media","protocol":"nfs","server":"10.0.0.5","export":"/srv/media","where":"/mnt/felhom-drives/media","configured":true,"mounted":true,"reachable":true,"health":"ok"},
|
||||
{"name":"photos","protocol":"smb","server":"10.0.0.5","export":"photos","where":"/mnt/felhom-drives/photos","configured":true,"mounted":false,"reachable":true,"health":"idle"},
|
||||
{"name":"vids","protocol":"nfs","server":"10.0.0.6","export":"/srv/vids","where":"/mnt/felhom-drives/vids","configured":true,"mounted":true,"reachable":false,"health":"unreachable"}
|
||||
]}}`))
|
||||
})
|
||||
mux.HandleFunc("POST /netstorage/remove", func(w http.ResponseWriter, r *http.Request) {
|
||||
var body struct {
|
||||
Name string `json:"name"`
|
||||
}
|
||||
_ = decodeJSON(r, &body)
|
||||
captured.removed = body.Name
|
||||
_, _ = w.Write([]byte(`{"ok":true,"data":{"name":"` + body.Name + `","removed":true}}`))
|
||||
})
|
||||
s := httptest.NewTLSServer(mux)
|
||||
return s, strings.TrimPrefix(s.URL, "https://"), captured
|
||||
}
|
||||
|
||||
func TestNetStorage_Add_ForwardsCredsAndMapping(t *testing.T) {
|
||||
s, ep, cap := netStub(t)
|
||||
defer s.Close()
|
||||
c := clientFor(t, s, ep)
|
||||
|
||||
res, err := c.AddNetStorage(context.Background(), AddNetStorageRequest{
|
||||
Name: "media", Protocol: "smb", Server: "10.0.0.5", Export: "media",
|
||||
MappedUID: 1000, MappedGID: 1000, Username: "u", Password: "p",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if res.GuestPath != "/mnt/felhom-drives/media" || res.HostUID != 101000 {
|
||||
t.Fatalf("add result mismatch: %+v", res)
|
||||
}
|
||||
// The SMB creds + mapping must be forwarded to the agent verbatim (the agent writes the 0600 file).
|
||||
if cap.addBody.Username != "u" || cap.addBody.Password != "p" {
|
||||
t.Fatalf("creds not forwarded to the agent: %+v", cap.addBody)
|
||||
}
|
||||
if cap.addBody.MappedUID != 1000 || cap.addBody.Protocol != "smb" {
|
||||
t.Fatalf("mapping/protocol not forwarded: %+v", cap.addBody)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNetStorage_List_HealthStates(t *testing.T) {
|
||||
s, ep, _ := netStub(t)
|
||||
defer s.Close()
|
||||
c := clientFor(t, s, ep)
|
||||
|
||||
mounts, err := c.ListNetStorage(context.Background())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(mounts) != 3 {
|
||||
t.Fatalf("want 3 mounts, got %d", len(mounts))
|
||||
}
|
||||
byName := map[string]NetworkMountStatus{}
|
||||
for _, m := range mounts {
|
||||
byName[m.Name] = m
|
||||
}
|
||||
// ok + idle are NOT degraded; only unreachable is.
|
||||
if byName["media"].Unreachable() || byName["photos"].Unreachable() {
|
||||
t.Fatalf("ok/idle must not be unreachable: %+v", byName)
|
||||
}
|
||||
if byName["photos"].Health != "idle" {
|
||||
t.Fatalf("photos should be idle (benign), got %q", byName["photos"].Health)
|
||||
}
|
||||
if !byName["vids"].Unreachable() {
|
||||
t.Fatalf("vids should be unreachable (degraded), got %q", byName["vids"].Health)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNetStorage_Remove(t *testing.T) {
|
||||
s, ep, cap := netStub(t)
|
||||
defer s.Close()
|
||||
c := clientFor(t, s, ep)
|
||||
|
||||
if err := c.RemoveNetStorage(context.Background(), "media"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if cap.removed != "media" {
|
||||
t.Fatalf("remove did not forward the name, got %q", cap.removed)
|
||||
}
|
||||
}
|
||||
@@ -1,100 +0,0 @@
|
||||
package agentapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// Campaign F2 evidence gap: the agent's 403 refusal body carries the informative reason
|
||||
// (agent disks.go handleDiskDecommission — "…decommission refused (role: X)"), but the old
|
||||
// c.post discarded any non-2xx body, so operators saw a bare
|
||||
// "agentapi: POST /disks/decommission: HTTP 403". These tests pin the reason surfacing.
|
||||
|
||||
func refusalStub(t *testing.T) (*httptest.Server, string) {
|
||||
t.Helper()
|
||||
mux := http.NewServeMux()
|
||||
mux.HandleFunc("POST /disks/decommission", func(w http.ResponseWriter, r *http.Request) {
|
||||
w.WriteHeader(http.StatusForbidden)
|
||||
_, _ = w.Write([]byte(`{"ok":false,"error":"mount is system/backup-protected — decommission refused (role: system)"}`))
|
||||
})
|
||||
mux.HandleFunc("POST /disks/eject", func(w http.ResponseWriter, r *http.Request) {
|
||||
w.WriteHeader(http.StatusForbidden)
|
||||
_, _ = w.Write([]byte(`{"ok":false,"error":"mount is system/backup-protected — eject refused (role: backup)"}`))
|
||||
})
|
||||
s := httptest.NewTLSServer(mux)
|
||||
return s, strings.TrimPrefix(s.URL, "https://")
|
||||
}
|
||||
|
||||
// T-D1: Decommission surfaces the agent's refusal reason, not a bare HTTP code.
|
||||
// RED-PROOF: on the pre-fix c.post shape the error is exactly
|
||||
// "agentapi: POST /disks/decommission: HTTP 403" → the Contains assertion FAILS.
|
||||
func TestDecommission_RefusalReasonSurfaced(t *testing.T) {
|
||||
s, ep := refusalStub(t)
|
||||
defer s.Close()
|
||||
c := clientFor(t, s, ep)
|
||||
|
||||
_, err := c.Decommission(context.Background(), "/mnt/sys_drive")
|
||||
if err == nil {
|
||||
t.Fatal("expected the 403 refusal to be an error")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "(role: system)") {
|
||||
t.Fatalf("agent refusal reason discarded — operator sees only: %v", err)
|
||||
}
|
||||
if !strings.Contains(err.Error(), "HTTP 403") {
|
||||
t.Fatalf("HTTP status lost from the error: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// T-D2: same surfacing for EjectDisk.
|
||||
func TestEjectDisk_RefusalReasonSurfaced(t *testing.T) {
|
||||
s, ep := refusalStub(t)
|
||||
defer s.Close()
|
||||
c := clientFor(t, s, ep)
|
||||
|
||||
_, err := c.EjectDisk(context.Background(), "/mnt/felhom-flash")
|
||||
if err == nil {
|
||||
t.Fatal("expected the 403 refusal to be an error")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "(role: backup)") {
|
||||
t.Fatalf("agent refusal reason discarded — operator sees only: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// T-D3: the success path is unchanged — a 200 envelope still decodes into the result struct.
|
||||
// (EjectDisk success is covered by TestEject_Dependents; this pins Decommission.)
|
||||
func TestDecommission_SuccessUnchanged(t *testing.T) {
|
||||
mux := http.NewServeMux()
|
||||
mux.HandleFunc("POST /disks/decommission", func(w http.ResponseWriter, r *http.Request) {
|
||||
_, _ = w.Write([]byte(`{"ok":true,"data":{"vmid":8200,"decommissioned":"/mnt/bulk","dependent_guests":[8200]}}`))
|
||||
})
|
||||
s := httptest.NewTLSServer(mux)
|
||||
defer s.Close()
|
||||
c := clientFor(t, s, strings.TrimPrefix(s.URL, "https://"))
|
||||
|
||||
out, err := c.Decommission(context.Background(), "/mnt/bulk")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if out.Decommissioned != "/mnt/bulk" || len(out.DependentGuests) != 1 {
|
||||
t.Fatalf("success payload mis-decoded: %+v", out)
|
||||
}
|
||||
}
|
||||
|
||||
// A 2xx envelope with ok:false (business refusal without an HTTP error) must also carry the reason.
|
||||
func TestDecommission_OkFalseBusinessRefusal(t *testing.T) {
|
||||
mux := http.NewServeMux()
|
||||
mux.HandleFunc("POST /disks/decommission", func(w http.ResponseWriter, r *http.Request) {
|
||||
_, _ = w.Write([]byte(`{"ok":false,"error":"drive is busy: unmount blocked by open files"}`))
|
||||
})
|
||||
s := httptest.NewTLSServer(mux)
|
||||
defer s.Close()
|
||||
c := clientFor(t, s, strings.TrimPrefix(s.URL, "https://"))
|
||||
|
||||
_, err := c.Decommission(context.Background(), "/mnt/bulk")
|
||||
if err == nil || !strings.Contains(err.Error(), "unmount blocked by open files") {
|
||||
t.Fatalf("ok:false reason not surfaced: %v", err)
|
||||
}
|
||||
}
|
||||
@@ -1,149 +0,0 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"io"
|
||||
"log"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/backup"
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/config"
|
||||
)
|
||||
|
||||
// snapshotsStubProvider implements backup.StackDataProvider for the F1 endpoint tests: one known
|
||||
// stack ("app") living on `hdd`. Everything else is inert.
|
||||
type snapshotsStubProvider struct{ hdd string }
|
||||
|
||||
func (p *snapshotsStubProvider) GetStackComposePath(name string) (string, bool) {
|
||||
if name == "app" {
|
||||
return filepath.Join(p.hdd, "compose", "docker-compose.yml"), true
|
||||
}
|
||||
return "", false
|
||||
}
|
||||
func (p *snapshotsStubProvider) ListDeployedStacks() []backup.StackSummary { return nil }
|
||||
func (p *snapshotsStubProvider) GetStackHDDMounts(string) []string { return nil }
|
||||
func (p *snapshotsStubProvider) GetStackHDDPath(string) string { return p.hdd }
|
||||
func (p *snapshotsStubProvider) GetImportRoot() string { return "" } // R-75: no import binds in this fixture
|
||||
func (p *snapshotsStubProvider) GetDockerVolumes(string) []string { return nil }
|
||||
func (p *snapshotsStubProvider) StopStack(string) error { return nil }
|
||||
func (p *snapshotsStubProvider) StartStack(string) error { return nil }
|
||||
func (p *snapshotsStubProvider) RefreshAndIsRunning(string) bool { return true }
|
||||
func (p *snapshotsStubProvider) GetStackRecoveryInfo(string) (backup.RecoveryInfo, bool) {
|
||||
return backup.RecoveryInfo{}, false
|
||||
}
|
||||
func (p *snapshotsStubProvider) GetStackClassifiedBinds(string) ([]backup.ClassifiedBind, bool) {
|
||||
return nil, false
|
||||
}
|
||||
func (p *snapshotsStubProvider) RecoverStackSecrets(string, []string) map[string]string { return nil }
|
||||
func (p *snapshotsStubProvider) RecreateStackDefinitionFromUnit(string, string, map[string]string) error {
|
||||
return nil
|
||||
}
|
||||
func (p *snapshotsStubProvider) StartStackServices(string, []string) error { return nil }
|
||||
|
||||
// newSnapshotsRouter wires a Router with a real backup.Manager over a tempdir drive.
|
||||
func newSnapshotsRouter(t *testing.T) (*Router, string) {
|
||||
t.Helper()
|
||||
drive := filepath.Join(t.TempDir(), "drive")
|
||||
cfg := &config.Config{}
|
||||
cfg.Paths.SystemDataPath = filepath.Join(t.TempDir(), "sys")
|
||||
mgr := backup.NewManager(cfg, nil, log.New(io.Discard, "", 0))
|
||||
mgr.SetStackProvider(&snapshotsStubProvider{hdd: drive})
|
||||
return &Router{cfg: cfg, backupMgr: mgr, logger: log.New(io.Discard, "", 0)}, drive
|
||||
}
|
||||
|
||||
// getSnapshots dispatches through Router.ServeHTTP so the tests also prove the ROUTE is
|
||||
// registered — the F1 bug was precisely a fetch to a route that did not exist.
|
||||
func getSnapshots(t *testing.T, r *Router, rawQuery string) *httptest.ResponseRecorder {
|
||||
t.Helper()
|
||||
req := httptest.NewRequest(http.MethodGet, "/api/backup/snapshots?"+rawQuery, nil)
|
||||
rec := httptest.NewRecorder()
|
||||
r.ServeHTTP(rec, req)
|
||||
return rec
|
||||
}
|
||||
|
||||
type snapshotsResp struct {
|
||||
OK bool `json:"ok"`
|
||||
Error string `json:"error"`
|
||||
Data []struct {
|
||||
Time string `json:"time"`
|
||||
ShortID string `json:"short_id"`
|
||||
Tier int `json:"tier"`
|
||||
DriveLabel string `json:"drive_label"`
|
||||
} `json:"data"`
|
||||
}
|
||||
|
||||
func decodeSnapshots(t *testing.T, rec *httptest.ResponseRecorder) snapshotsResp {
|
||||
t.Helper()
|
||||
var v snapshotsResp
|
||||
if err := json.Unmarshal(rec.Body.Bytes(), &v); err != nil {
|
||||
t.Fatalf("decode %q: %v", rec.Body.String(), err)
|
||||
}
|
||||
return v
|
||||
}
|
||||
|
||||
// Scenario A (data half): a recovery unit on disk → 200 with EXACTLY ONE tier-1 "helyi" entry the
|
||||
// restore panel JS can render and POST back.
|
||||
func TestBackupSnapshots_UnitOnDisk(t *testing.T) {
|
||||
r, drive := newSnapshotsRouter(t)
|
||||
manifest := backup.RecoveryUnitManifestPath(drive, "app")
|
||||
if err := os.MkdirAll(filepath.Dir(manifest), 0755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(manifest, []byte("{}"), 0644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
rec := getSnapshots(t, r, "stack=app")
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d, want 200; body=%s", rec.Code, rec.Body.String())
|
||||
}
|
||||
v := decodeSnapshots(t, rec)
|
||||
if !v.OK || len(v.Data) != 1 {
|
||||
t.Fatalf("want ok with exactly 1 entry, got %+v", v)
|
||||
}
|
||||
if v.Data[0].Tier != 1 || v.Data[0].ShortID != "helyi" || v.Data[0].Time == "" {
|
||||
t.Errorf("entry = %+v (want tier 1, short_id helyi, non-empty time)", v.Data[0])
|
||||
}
|
||||
}
|
||||
|
||||
// Scenario B: known stack, no recovery unit yet → ok:true with an EMPTY data list (the JS shows
|
||||
// "Nincs elérhető mentés" and keeps the button disabled — reached honestly, not via a 404).
|
||||
func TestBackupSnapshots_NoBackupYet(t *testing.T) {
|
||||
r, _ := newSnapshotsRouter(t)
|
||||
rec := getSnapshots(t, r, "stack=app")
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d, want 200; body=%s", rec.Code, rec.Body.String())
|
||||
}
|
||||
v := decodeSnapshots(t, rec)
|
||||
if !v.OK || len(v.Data) != 0 {
|
||||
t.Errorf("want ok with empty list, got %+v", v)
|
||||
}
|
||||
}
|
||||
|
||||
// Scenario C: guards — traversal and empty names are 400 BEFORE any filesystem work; an unknown
|
||||
// stack is 404; a Router without a backup manager is 400.
|
||||
func TestBackupSnapshots_Guards(t *testing.T) {
|
||||
r, _ := newSnapshotsRouter(t)
|
||||
|
||||
for _, q := range []string{"stack=../../etc", "stack=", "stack=a/b", "stack=.."} {
|
||||
rec := getSnapshots(t, r, q)
|
||||
if rec.Code != http.StatusBadRequest {
|
||||
t.Errorf("%q: status = %d, want 400; body=%s", q, rec.Code, rec.Body.String())
|
||||
}
|
||||
}
|
||||
|
||||
rec := getSnapshots(t, r, "stack=ghost")
|
||||
if rec.Code != http.StatusNotFound {
|
||||
t.Errorf("unknown stack: status = %d, want 404; body=%s", rec.Code, rec.Body.String())
|
||||
}
|
||||
|
||||
noMgr := &Router{cfg: &config.Config{}, logger: log.New(io.Discard, "", 0)}
|
||||
rec = getSnapshots(t, noMgr, "stack=app")
|
||||
if rec.Code != http.StatusBadRequest {
|
||||
t.Errorf("nil backupMgr: status = %d, want 400", rec.Code)
|
||||
}
|
||||
}
|
||||
@@ -1,39 +0,0 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"runtime"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// TestWriteConfig0600 asserts F8: controller.yaml is persisted 0600 (it holds infra secrets), even when
|
||||
// the target file already existed with looser (0644) permissions. POSIX modes only — skipped on Windows.
|
||||
func TestWriteConfig0600(t *testing.T) {
|
||||
if runtime.GOOS == "windows" {
|
||||
t.Skip("POSIX file modes not represented on Windows")
|
||||
}
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "controller.yaml")
|
||||
|
||||
// Pre-create with world-readable 0644 to prove the helper tightens an existing file.
|
||||
if err := os.WriteFile(path, []byte("old: true\n"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
if err := writeConfig0600(path, []byte("hub:\n api_key: redacted\n")); err != nil {
|
||||
t.Fatalf("writeConfig0600: %v", err)
|
||||
}
|
||||
|
||||
fi, err := os.Stat(path)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if mode := fi.Mode().Perm(); mode != 0o600 {
|
||||
t.Fatalf("config mode = %o, want 0600", mode)
|
||||
}
|
||||
// No leftover temp file.
|
||||
if _, err := os.Stat(path + ".tmp"); !os.IsNotExist(err) {
|
||||
t.Fatalf("temp file not cleaned up")
|
||||
}
|
||||
}
|
||||
@@ -1,68 +0,0 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"io"
|
||||
"log"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/settings"
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/system"
|
||||
)
|
||||
|
||||
func newDeployGateRouter(t *testing.T, class string) *Router {
|
||||
t.Helper()
|
||||
lg := log.New(io.Discard, "", 0)
|
||||
sett, err := settings.Load(filepath.Join(t.TempDir(), "settings.json"), lg)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := sett.AddStoragePath(settings.StoragePath{
|
||||
Path: "/mnt/felhom-drives/nas-media", Label: "NAS", Schedulable: true, Kind: settings.StorageKindNetwork,
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := sett.AddStoragePath(settings.StoragePath{
|
||||
Path: "/mnt/felhom-drives/felhom-usb", Label: "USB", Schedulable: true,
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
r := &Router{sett: sett, logger: lg}
|
||||
r.classifyFSPath = func(string) string { return class }
|
||||
return r
|
||||
}
|
||||
|
||||
// The deploy-time stub gate (RCA fix 2): a registered network HDD_PATH that classifies as a STUB
|
||||
// in this namespace refuses with the §2.3 Hungarian message; the healthy idle autofs trigger, a
|
||||
// live mount, an unknown verdict (fail open), local paths and empty paths all proceed.
|
||||
// Companion red-proofs:
|
||||
// - remove the gate call from deployStack → the refusal test fails (deploy proceeds onto a stub);
|
||||
// - an impl requiring MOUNTED-only (refusing autofs) → the idle-autofs row fails (it would
|
||||
// wrongly block deploying onto a healthy idle share).
|
||||
func TestRefuseNetworkStubDeploy_Table(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
class string
|
||||
hdd string
|
||||
refuse bool
|
||||
}{
|
||||
{"stub on network path → REFUSE", system.FSClassStub, "/mnt/felhom-drives/nas-media", true},
|
||||
{"idle autofs on network path → proceed (healthy)", system.FSClassAutofs, "/mnt/felhom-drives/nas-media", false},
|
||||
{"live network fs → proceed", system.FSClassNetwork, "/mnt/felhom-drives/nas-media", false},
|
||||
{"unknown (timeout) → proceed (fail open)", system.FSClassUnknown, "/mnt/felhom-drives/nas-media", false},
|
||||
{"local path → proceed even when classifier says stub", system.FSClassStub, "/mnt/felhom-drives/felhom-usb", false},
|
||||
{"unregistered path → proceed", system.FSClassStub, "/mnt/elsewhere", false},
|
||||
{"empty HDD_PATH (SSD app) → proceed", system.FSClassStub, "", false},
|
||||
}
|
||||
for _, c := range cases {
|
||||
r := newDeployGateRouter(t, c.class)
|
||||
msg := r.refuseNetworkStubDeploy(c.hdd)
|
||||
if c.refuse && !strings.Contains(msg, "a telepítés nem indítható") {
|
||||
t.Errorf("%s: want the refusal message, got %q", c.name, msg)
|
||||
}
|
||||
if !c.refuse && msg != "" {
|
||||
t.Errorf("%s: must proceed, got refusal %q", c.name, msg)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,149 +0,0 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"go/ast"
|
||||
"go/parser"
|
||||
"go/token"
|
||||
"testing"
|
||||
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/stacks"
|
||||
)
|
||||
|
||||
// R-166 Part 1.3 — THE CUSTOMER-INTENT POINT.
|
||||
//
|
||||
// `stackMgr` is a concrete *stacks.Manager, so actionStack cannot be driven with a fake without
|
||||
// Docker. The two properties that actually carry the correctness are therefore pinned the only way
|
||||
// they can be: the mapping is a pure function with its own table test, and the ORDER (§8.2) is
|
||||
// asserted structurally over actionStack's AST. Both fail if someone reverses the write and the act,
|
||||
// which is the mistake that would undo a customer's Stop at the next boot.
|
||||
|
||||
func TestDesiredStateForAction_MapsEveryAction(t *testing.T) {
|
||||
cases := []struct {
|
||||
action string
|
||||
want string
|
||||
ok bool
|
||||
}{
|
||||
{"start", stacks.DesiredStateRunning, true},
|
||||
// restart and update both END in `compose up -d`, so a customer who presses either is asking
|
||||
// for the app to be up afterwards.
|
||||
{"restart", stacks.DesiredStateRunning, true},
|
||||
{"update", stacks.DesiredStateRunning, true},
|
||||
{"stop", stacks.DesiredStateStopped, true},
|
||||
// Anything unrecognised records NOTHING rather than guessing — a future action must not
|
||||
// silently acquire an intent it was never meant to carry.
|
||||
{"", "", false},
|
||||
{"delete", "", false},
|
||||
{"pause", "", false},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
got, ok := desiredStateForAction(tc.action)
|
||||
if got != tc.want || ok != tc.ok {
|
||||
t.Fatalf("desiredStateForAction(%q) = (%q, %v), want (%q, %v)", tc.action, got, ok, tc.want, tc.ok)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestDesiredStateForAction_NeverRecordsStoppedForANonStop(t *testing.T) {
|
||||
// The asymmetry that matters: writing "stopped" for anything other than a Stop would permanently
|
||||
// disable auto-recovery for an app nobody stopped.
|
||||
for _, a := range []string{"start", "restart", "update", "deploy", "delete", ""} {
|
||||
if got, _ := desiredStateForAction(a); got == stacks.DesiredStateStopped {
|
||||
t.Fatalf("action %q maps to desired_state=stopped", a)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestActionStack_RecordsIntentBeforeActing is §8.2, asserted structurally.
|
||||
//
|
||||
// If the SetDesiredState call moved BELOW the action switch, a stop could remove every container
|
||||
// while app.yaml still recorded `running` — and the boot reconciler would then start an app the
|
||||
// customer had just deliberately stopped. That is the single worst outcome available in Part 1, and
|
||||
// no behavioural test in this package can reach it without a Docker daemon.
|
||||
func TestActionStack_RecordsIntentBeforeActing(t *testing.T) {
|
||||
body := funcBody(t, "actionStack")
|
||||
|
||||
setPos, switchPos := -1, -1
|
||||
ast.Inspect(body, func(n ast.Node) bool {
|
||||
switch node := n.(type) {
|
||||
case *ast.CallExpr:
|
||||
if sel, ok := node.Fun.(*ast.SelectorExpr); ok && sel.Sel.Name == "SetDesiredState" && setPos < 0 {
|
||||
setPos = int(node.Pos())
|
||||
}
|
||||
case *ast.SwitchStmt:
|
||||
// The action switch is the one whose tag is the `action` identifier.
|
||||
if id, ok := node.Tag.(*ast.Ident); ok && id.Name == "action" && switchPos < 0 {
|
||||
switchPos = int(node.Pos())
|
||||
}
|
||||
}
|
||||
return true
|
||||
})
|
||||
|
||||
if setPos < 0 {
|
||||
t.Fatal("actionStack no longer calls SetDesiredState — the customer's start/stop decision is " +
|
||||
"recorded nowhere, which is the R-166 defect un-fixed")
|
||||
}
|
||||
if switchPos < 0 {
|
||||
t.Fatal("actionStack no longer has a `switch action` — this test needs updating")
|
||||
}
|
||||
if setPos >= switchPos {
|
||||
t.Fatal("actionStack records the desired state AFTER performing the action (§8.2 violated): a " +
|
||||
"stop whose intent write fails or lands late leaves zero containers with `running` " +
|
||||
"recorded, and the boot reconciler would restart an app the customer just stopped")
|
||||
}
|
||||
}
|
||||
|
||||
// TestActionStack_RefusesTheActionWhenIntentCannotBeRecorded pins the other half of §8.2: a failed
|
||||
// write REFUSES the act. Proceeding anyway would perform a stop that nothing records — exactly the
|
||||
// ambiguity this release removes.
|
||||
func TestActionStack_RefusesTheActionWhenIntentCannotBeRecorded(t *testing.T) {
|
||||
body := funcBody(t, "actionStack")
|
||||
|
||||
refuses := false
|
||||
ast.Inspect(body, func(n ast.Node) bool {
|
||||
ifst, ok := n.(*ast.IfStmt)
|
||||
if !ok || ifst.Init == nil {
|
||||
return true
|
||||
}
|
||||
// Look for `if derr := ...SetDesiredState(...); derr != nil { ... return }`
|
||||
assign, ok := ifst.Init.(*ast.AssignStmt)
|
||||
if !ok || len(assign.Rhs) != 1 {
|
||||
return true
|
||||
}
|
||||
call, ok := assign.Rhs[0].(*ast.CallExpr)
|
||||
if !ok {
|
||||
return true
|
||||
}
|
||||
sel, ok := call.Fun.(*ast.SelectorExpr)
|
||||
if !ok || sel.Sel.Name != "SetDesiredState" {
|
||||
return true
|
||||
}
|
||||
for _, stmt := range ifst.Body.List {
|
||||
if _, isReturn := stmt.(*ast.ReturnStmt); isReturn {
|
||||
refuses = true
|
||||
}
|
||||
}
|
||||
return true
|
||||
})
|
||||
|
||||
if !refuses {
|
||||
t.Fatal("actionStack does not RETURN when SetDesiredState fails — it would go on to stop or " +
|
||||
"start an app whose intent could not be recorded (§8.2)")
|
||||
}
|
||||
}
|
||||
|
||||
// funcBody parses router.go and returns the named method's body.
|
||||
func funcBody(t *testing.T, name string) *ast.BlockStmt {
|
||||
t.Helper()
|
||||
fset := token.NewFileSet()
|
||||
f, err := parser.ParseFile(fset, "router.go", nil, 0)
|
||||
if err != nil {
|
||||
t.Fatalf("parse router.go: %v", err)
|
||||
}
|
||||
for _, decl := range f.Decls {
|
||||
if fn, ok := decl.(*ast.FuncDecl); ok && fn.Name.Name == name && fn.Body != nil {
|
||||
return fn.Body
|
||||
}
|
||||
}
|
||||
t.Fatalf("func %s not found in router.go", name)
|
||||
return nil
|
||||
}
|
||||
@@ -82,10 +82,6 @@ func (r *Router) geoUpdateSettings(w http.ResponseWriter, req *http.Request) {
|
||||
}()
|
||||
}
|
||||
|
||||
// Push a fresh hub report out-of-band so the hub reflects the new geo state right
|
||||
// away instead of after the next ~15-min cycle.
|
||||
r.reportPushNow()
|
||||
|
||||
writeJSON(w, http.StatusOK, apiResponse{OK: true, Message: "Geo-korlátozás beállítva"})
|
||||
}
|
||||
|
||||
@@ -98,11 +94,7 @@ func (r *Router) geoTriggerSync(w http.ResponseWriter, _ *http.Request) {
|
||||
go func() {
|
||||
if err := r.geoSync.Sync(context.Background()); err != nil {
|
||||
r.logger.Printf("[ERROR] [api] Manual geo sync failed: %v", err)
|
||||
return
|
||||
}
|
||||
// On success the sync clears any stale last_sync_error — push a report so the hub
|
||||
// reflects the cleared state immediately rather than after the next cycle.
|
||||
r.reportPushNow()
|
||||
}()
|
||||
|
||||
writeJSON(w, http.StatusOK, apiResponse{OK: true, Message: "Szinkronizálás elindítva"})
|
||||
|
||||
@@ -1,57 +0,0 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"io"
|
||||
"log"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/config"
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/settings"
|
||||
)
|
||||
|
||||
func newGeoTestRouter(t *testing.T) (*Router, *int) {
|
||||
t.Helper()
|
||||
sett, err := settings.Load(filepath.Join(t.TempDir(), "settings.json"), log.New(io.Discard, "", 0))
|
||||
if err != nil {
|
||||
t.Fatalf("settings.Load: %v", err)
|
||||
}
|
||||
calls := 0
|
||||
r := &Router{cfg: &config.Config{}, sett: sett, logger: log.New(io.Discard, "", 0)}
|
||||
r.triggerReportPush = func() { calls++ }
|
||||
return r, &calls
|
||||
}
|
||||
|
||||
func postGeoSettings(t *testing.T, r *Router, body string) *httptest.ResponseRecorder {
|
||||
t.Helper()
|
||||
req := httptest.NewRequest(http.MethodPost, "/api/geo/settings", bytes.NewReader([]byte(body)))
|
||||
rec := httptest.NewRecorder()
|
||||
r.geoUpdateSettings(rec, req)
|
||||
return rec
|
||||
}
|
||||
|
||||
func TestGeoUpdateSettings_Success_PushesReport(t *testing.T) {
|
||||
r, calls := newGeoTestRouter(t)
|
||||
rec := postGeoSettings(t, r, `{"enabled":true,"allowed_countries":["HU"]}`)
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d, want 200; body=%s", rec.Code, rec.Body.String())
|
||||
}
|
||||
if *calls != 1 {
|
||||
t.Fatalf("report push called %d times, want 1 (successful save)", *calls)
|
||||
}
|
||||
}
|
||||
|
||||
// COMPANION: a validation-failed save (invalid country code) must NOT push a report.
|
||||
func TestGeoUpdateSettings_InvalidCountry_NoPush(t *testing.T) {
|
||||
r, calls := newGeoTestRouter(t)
|
||||
rec := postGeoSettings(t, r, `{"enabled":true,"allowed_countries":["XX"]}`)
|
||||
if rec.Code != http.StatusBadRequest {
|
||||
t.Fatalf("status = %d, want 400 (invalid country code)", rec.Code)
|
||||
}
|
||||
if *calls != 0 {
|
||||
t.Fatalf("report push called %d times, want 0 (validation failed before save)", *calls)
|
||||
}
|
||||
}
|
||||
@@ -1,97 +0,0 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"go/ast"
|
||||
"go/parser"
|
||||
"go/token"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// TestDeployStackWiresTheLifecycleGate — the seam-discipline test (§9 rule 6).
|
||||
//
|
||||
// The lifecycle predicate is unit-tested in internal/stacks, and a test there passes whether or not
|
||||
// deployStack ever calls it. Three inert-seam defects shipped fully-green in three days (controller
|
||||
// v0.154.0, agent v0.91.0, agent v0.92.0's missing sudoers grant), all this exact shape: correct
|
||||
// component, absent caller. So the CALLER is asserted here, from source.
|
||||
//
|
||||
// It walks the AST rather than doing strings.Contains on the file, because a commented-out call
|
||||
// still contains the string — the lesson recorded in PROMPT-TEMPLATE §10.
|
||||
//
|
||||
// It also asserts ORDER: the gate must precede the DeployStack call, or it is not fail-closed.
|
||||
func TestDeployStackWiresTheLifecycleGate(t *testing.T) {
|
||||
fset := token.NewFileSet()
|
||||
f, err := parser.ParseFile(fset, "router.go", nil, 0) // comments dropped: a commented call is not a call
|
||||
if err != nil {
|
||||
t.Fatalf("parse router.go: %v", err)
|
||||
}
|
||||
|
||||
var fn *ast.FuncDecl
|
||||
ast.Inspect(f, func(n ast.Node) bool {
|
||||
if d, ok := n.(*ast.FuncDecl); ok && d.Name.Name == "deployStack" {
|
||||
fn = d
|
||||
return false
|
||||
}
|
||||
return true
|
||||
})
|
||||
if fn == nil {
|
||||
t.Fatal("deployStack not found in router.go — did it move? the gate's wiring is now unasserted")
|
||||
}
|
||||
|
||||
canInstallPos, deployPos := -1, -1
|
||||
ast.Inspect(fn, func(n ast.Node) bool {
|
||||
call, ok := n.(*ast.CallExpr)
|
||||
if !ok {
|
||||
return true
|
||||
}
|
||||
sel, ok := call.Fun.(*ast.SelectorExpr)
|
||||
if !ok {
|
||||
return true
|
||||
}
|
||||
off := fset.Position(call.Pos()).Offset
|
||||
switch sel.Sel.Name {
|
||||
case "CanInstall":
|
||||
if canInstallPos == -1 {
|
||||
canInstallPos = off
|
||||
}
|
||||
case "DeployStack":
|
||||
if deployPos == -1 {
|
||||
deployPos = off
|
||||
}
|
||||
}
|
||||
return true
|
||||
})
|
||||
|
||||
if canInstallPos == -1 {
|
||||
t.Fatal("deployStack never calls Meta.CanInstall() — the lifecycle gate is INERT: " +
|
||||
"a withdrawn app is hidden from the catalog page but still installable by direct POST")
|
||||
}
|
||||
if deployPos == -1 {
|
||||
t.Fatal("deployStack no longer calls DeployStack — this test's ordering assertion is meaningless")
|
||||
}
|
||||
if canInstallPos > deployPos {
|
||||
t.Fatalf("the lifecycle gate (offset %d) runs AFTER DeployStack (offset %d) — a gate that "+
|
||||
"fires after the mutation is not fail-closed", canInstallPos, deployPos)
|
||||
}
|
||||
}
|
||||
|
||||
// TestLifecycleRefusalMessageIsCustomerFacingHungarian: the refusal text reaches the customer via
|
||||
// showAlert(), so it must be the sentence the spec ruled, not a Go error string.
|
||||
func TestLifecycleRefusalMessageIsCustomerFacingHungarian(t *testing.T) {
|
||||
fset := token.NewFileSet()
|
||||
f, err := parser.ParseFile(fset, "router.go", nil, 0)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
const want = "Ez az alkalmazás jelenleg nem telepíthető."
|
||||
found := false
|
||||
ast.Inspect(f, func(n ast.Node) bool {
|
||||
if lit, ok := n.(*ast.BasicLit); ok && lit.Kind == token.STRING && strings.Contains(lit.Value, want) {
|
||||
found = true
|
||||
}
|
||||
return true
|
||||
})
|
||||
if !found {
|
||||
t.Fatalf("the ruled refusal message %q is not present in router.go", want)
|
||||
}
|
||||
}
|
||||
@@ -1,11 +0,0 @@
|
||||
package api
|
||||
|
||||
import "testing"
|
||||
|
||||
// Group D (Scenario D, v0.139.0): with the seam unset (hub reporting disabled → main.go
|
||||
// never calls SetReportPushTrigger), reportPushNow is a strict nil-safe no-op — the
|
||||
// deploy/remove/geo handlers calling it must never panic.
|
||||
func TestReportPushNow_NilSeamIsNoOp(t *testing.T) {
|
||||
r := &Router{}
|
||||
r.reportPushNow() // must not panic with triggerReportPush == nil
|
||||
}
|
||||
@@ -1,7 +1,6 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
@@ -42,18 +41,8 @@ type Router struct {
|
||||
notifier *notify.Notifier
|
||||
logger *log.Logger
|
||||
|
||||
// restart triggers a graceful self-restart (config-apply + the manual restart button).
|
||||
// Defaults to a real exit-after-flush so Docker's restart:unless-stopped brings the
|
||||
// process back with fresh config; tests inject a recorder via SetRestarter.
|
||||
restart func()
|
||||
|
||||
// classifyFSPath classifies a path's filesystem in this process's namespace (the deploy-time
|
||||
// stub gate, RCA fix 2). Defaults to system.ClassifyPathFSTimeout; tests inject fake classes.
|
||||
classifyFSPath func(path string) string
|
||||
|
||||
// triggerReportPush fires an out-of-band, non-blocking hub report push (e.g. after a
|
||||
// geo settings change so the hub reflects the new state immediately). Nil = no-op.
|
||||
triggerReportPush func()
|
||||
// OnConfigApplied is called after a successful config apply (e.g., to push infra backup).
|
||||
OnConfigApplied func()
|
||||
|
||||
// OnGeoRelevantChange is called after deploy/remove to re-sync geo rules.
|
||||
OnGeoRelevantChange func()
|
||||
@@ -97,41 +86,7 @@ func (r *Router) SetIntegrationManager(im *integrations.Manager) {
|
||||
}
|
||||
|
||||
func NewRouter(cfg *config.Config, configPath string, sett *settings.Settings, stackMgr *stacks.Manager, syncer *catalogsync.Syncer, cpuCollector *system.CPUCollector, backupMgr *backup.Manager, metricsStore *metrics.MetricsStore, updater *selfupdate.Updater, notif *notify.Notifier, logger *log.Logger) *Router {
|
||||
r := &Router{cfg: cfg, configPath: configPath, sett: sett, stackMgr: stackMgr, syncer: syncer, cpuCollector: cpuCollector, backupMgr: backupMgr, metricsStore: metricsStore, updater: updater, notifier: notif, logger: logger}
|
||||
r.restart = func() { gracefulSelfRestart(r.logger) }
|
||||
r.classifyFSPath = system.ClassifyPathFSTimeout
|
||||
return r
|
||||
}
|
||||
|
||||
// SetRestarter overrides the graceful-restart action. Tests inject a recorder so the
|
||||
// process is not actually killed.
|
||||
func (r *Router) SetRestarter(fn func()) { r.restart = fn }
|
||||
|
||||
// refuseNetworkStubDeploy is the deploy-time stub gate (RCA fix 2). Non-empty return = the
|
||||
// Hungarian refusal for a registered NETWORK HDD_PATH whose filesystem in THIS namespace is a
|
||||
// local stub. Everything else proceeds: idle autofs is HEALTHY (first app access mounts it);
|
||||
// classification timeout/unknown fails OPEN (a wedged share is the unreachable badge's business);
|
||||
// local (non-network) and empty paths keep today's behavior exactly.
|
||||
func (r *Router) refuseNetworkStubDeploy(hdd string) string {
|
||||
if hdd == "" || r.sett == nil || !r.sett.IsNetworkStoragePath(hdd) {
|
||||
return ""
|
||||
}
|
||||
if r.classifyFSPath(hdd) != system.FSClassStub {
|
||||
return ""
|
||||
}
|
||||
return "A kiválasztott hálózati tárhely jelenleg nem érhető el az alkalmazások környezetéből — a telepítés nem indítható. Próbálja újra pár perc múlva, vagy jelezze az üzemeltetőnek."
|
||||
}
|
||||
|
||||
// SetReportPushTrigger wires the out-of-band hub report push used after geo changes.
|
||||
// The provided func MUST be non-blocking (it is called from request handlers).
|
||||
func (r *Router) SetReportPushTrigger(fn func()) { r.triggerReportPush = fn }
|
||||
|
||||
// reportPushNow fires the report-push trigger if wired. Called after a state change the
|
||||
// hub should reflect immediately (geo settings/sync) instead of waiting for the next cycle.
|
||||
func (r *Router) reportPushNow() {
|
||||
if r.triggerReportPush != nil {
|
||||
r.triggerReportPush()
|
||||
}
|
||||
return &Router{cfg: cfg, configPath: configPath, sett: sett, stackMgr: stackMgr, syncer: syncer, cpuCollector: cpuCollector, backupMgr: backupMgr, metricsStore: metricsStore, updater: updater, notifier: notif, logger: logger}
|
||||
}
|
||||
|
||||
type apiResponse struct {
|
||||
@@ -157,12 +112,6 @@ func (r *Router) ServeHTTP(w http.ResponseWriter, req *http.Request) {
|
||||
case path == "/stacks/rescan" && req.Method == http.MethodPost:
|
||||
r.rescanStacks(w, req)
|
||||
|
||||
// F4: /api/stacks/rescan with a non-POST method must be a clear 405, not fall through to the
|
||||
// GET /stacks/{name} lookup below (which returned the misleading "stack not found: rescan").
|
||||
case path == "/stacks/rescan":
|
||||
w.Header().Set("Allow", http.MethodPost)
|
||||
writeJSON(w, http.StatusMethodNotAllowed, apiResponse{OK: false, Error: "method not allowed: use POST /api/stacks/rescan"})
|
||||
|
||||
// GET /api/stacks/{name}
|
||||
case strings.HasPrefix(path, "/stacks/") && req.Method == http.MethodGet && !hasSubpath(path, "/stacks/"):
|
||||
r.getStack(w, req, trimSegment(path, "/stacks/"))
|
||||
@@ -191,10 +140,6 @@ func (r *Router) ServeHTTP(w http.ResponseWriter, req *http.Request) {
|
||||
case path == "/config" && req.Method == http.MethodGet:
|
||||
r.configContent(w, req)
|
||||
|
||||
// POST /api/selfrestart — customer-facing graceful self-restart (auth + CSRF via /api/ mount)
|
||||
case path == "/selfrestart" && req.Method == http.MethodPost:
|
||||
r.selfRestart(w, req)
|
||||
|
||||
// --- Integration routes (must be before hasSuffix-based stack cases) ---
|
||||
|
||||
// GET /api/integrations/{provider} — list integrations for a provider
|
||||
@@ -272,23 +217,10 @@ func (r *Router) ServeHTTP(w http.ResponseWriter, req *http.Request) {
|
||||
case path == "/backup/status" && req.Method == http.MethodGet:
|
||||
r.backupStatus(w, req)
|
||||
|
||||
// GET /api/backup/restore-status — async restore progress for the backups-page banner (Part B).
|
||||
// Distinct from /backup/status (which proxies the agent's PBS whole-guest status).
|
||||
case path == "/backup/restore-status" && req.Method == http.MethodGet:
|
||||
r.backupRestoreStatus(w, req)
|
||||
|
||||
// GET /api/backup/snapshots?stack=<name> — restorable keep-side backups for the restore panel
|
||||
case path == "/backup/snapshots" && req.Method == http.MethodGet:
|
||||
r.backupSnapshots(w, req)
|
||||
|
||||
// POST /api/backup/run
|
||||
case path == "/backup/run" && req.Method == http.MethodPost:
|
||||
r.triggerBackup(w, req)
|
||||
|
||||
// POST /api/backup/tier2 — run off-drive Tier 2 copies for all HDD apps
|
||||
case path == "/backup/tier2" && req.Method == http.MethodPost:
|
||||
r.triggerTier2(w, req)
|
||||
|
||||
// GET /api/metrics/system
|
||||
case path == "/metrics/system" && req.Method == http.MethodGet:
|
||||
r.metricsSystem(w, req)
|
||||
@@ -410,58 +342,6 @@ func (r *Router) deployStack(w http.ResponseWriter, req *http.Request, name stri
|
||||
return
|
||||
}
|
||||
|
||||
// Lifecycle gate: an app withdrawn from the catalog (`lifecycle: hidden` / `abandoned`) is not
|
||||
// installable. FAIL-CLOSED and server-side on purpose — the catalog page already omits these, so
|
||||
// anything reaching here is a stale link, a bookmarked deploy form, or a direct POST, and a gate
|
||||
// that only hides the button is not a gate. Deliberately BEFORE every mutation.
|
||||
//
|
||||
// This does NOT touch an already-deployed instance: it is on the deploy path only, and the
|
||||
// manager refuses a redeploy of an existing stack through its own "already deployed" check.
|
||||
if st, ok := r.stackMgr.GetStack(name); ok && !st.Meta.CanInstall() {
|
||||
r.logger.Printf("[WARN] [api] Deploy refused for %s: lifecycle=%s (not offered for new installs)",
|
||||
name, st.Meta.EffectiveLifecycle())
|
||||
writeJSON(w, http.StatusConflict, apiResponse{OK: false,
|
||||
Error: "Ez az alkalmazás jelenleg nem telepíthető."})
|
||||
return
|
||||
}
|
||||
|
||||
// Prevention layer (storage-split): refuse a deploy when the Docker-data volume is at/under its
|
||||
// reserved buffer, so customer apps can't fill the volume the infra containers (controller,
|
||||
// traefik, cloudflared, filebrowser) depend on. Fail-OPEN on a measurement error — the buffer is
|
||||
// a safety net, not a security control, so a transient statfs failure must not block all deploys.
|
||||
if hr := system.GetDockerVolumeHeadroom(); hr.OK && hr.BelowReserve {
|
||||
r.logger.Printf("[WARN] [api] Deploy refused for %s: Docker volume below reserved buffer (%.1fG free, reserve %.1fG of %.0fG)",
|
||||
name, hr.AvailGB, hr.ReserveGB, hr.TotalGB)
|
||||
writeJSON(w, http.StatusInsufficientStorage, apiResponse{OK: false, Error: fmt.Sprintf(
|
||||
"Nincs elég szabad tárhely a telepítéshez: csak %.0f GB szabad, és a rendszer %.0f GB tartalékot tart fenn az alapszolgáltatások (vezérlő, proxy) védelmében. Szabadítson fel helyet, vagy bővítse a tárhelyet.",
|
||||
hr.AvailGB, hr.ReserveGB)})
|
||||
return
|
||||
}
|
||||
|
||||
// RCA fix 2 (AUDIT-nas-cwa-rca-2026-07-11): a deploy targeting a registered NETWORK storage path
|
||||
// must see a network filesystem (or its healthy idle autofs trigger) in THIS namespace — the one
|
||||
// the app will consume the path in. A stub (plain local dir after a guest reboot) would silently
|
||||
// send the app's data to the system drive.
|
||||
if msg := r.refuseNetworkStubDeploy(body.Values["HDD_PATH"]); msg != "" {
|
||||
r.logger.Printf("[WARN] [api] Deploy refused for %s: network HDD_PATH %s is a stub in the controller namespace", name, body.Values["HDD_PATH"])
|
||||
writeJSON(w, http.StatusConflict, apiResponse{OK: false, Error: msg})
|
||||
return
|
||||
}
|
||||
|
||||
// R-108: an app's data namespace may NOT live on network storage — its backups would land at
|
||||
// `<share>/backups/primary/<stack>/`, inside the share-ROOT bind FileBrowser serves with
|
||||
// download:true (and that bind cannot be narrowed — see settings.RefuseAsAppNamespace).
|
||||
//
|
||||
// THIS is the boundary, not the deploy dropdown. The dropdown is a UI list; this endpoint accepts
|
||||
// whatever HDD_PATH a caller supplies and `DeployStack` validates only that it EXISTS on the
|
||||
// filesystem (os.Stat, internal/stacks/deploy.go). A filter on the list alone would have left the
|
||||
// surface wide open — the R-108 row's "no IsNetwork() filter on the dropdown" understates it.
|
||||
if refuse, why := r.sett.RefuseAsAppNamespace(body.Values["HDD_PATH"]); refuse {
|
||||
r.logger.Printf("[WARN] [api] Deploy refused for %s: HDD_PATH is not usable as an app namespace (R-108)", name)
|
||||
writeJSON(w, http.StatusConflict, apiResponse{OK: false, Error: why})
|
||||
return
|
||||
}
|
||||
|
||||
deployReq := stacks.DeployRequest{
|
||||
StackName: name,
|
||||
Values: body.Values,
|
||||
@@ -481,15 +361,11 @@ func (r *Router) deployStack(w http.ResponseWriter, req *http.Request, name stri
|
||||
return
|
||||
}
|
||||
|
||||
// F6: the deploy runs asynchronously (compose pull/up + health happen after this returns; the UI
|
||||
// polls GET /api/stacks/{name}). The old "Stack X deployed" message asserted completion before it
|
||||
// was true — misleading for API/script consumers. Report that the deploy STARTED, not that it
|
||||
// finished. 202 Accepted reflects "accepted, processing"; ok:true is preserved for the UI.
|
||||
resp := apiResponse{OK: true, Message: "Telepítés elindítva – az állapot a kártyán követhető"}
|
||||
resp := apiResponse{OK: true, Message: "Stack " + name + " deployed"}
|
||||
if warning != "" {
|
||||
resp.Data = map[string]string{"warning": warning}
|
||||
}
|
||||
writeJSON(w, http.StatusAccepted, resp)
|
||||
writeJSON(w, http.StatusOK, resp)
|
||||
|
||||
// Push app deployed event to Hub
|
||||
if r.notifier != nil {
|
||||
@@ -505,53 +381,12 @@ func (r *Router) deployStack(w http.ResponseWriter, req *http.Request, name stri
|
||||
go r.OnGeoRelevantChange()
|
||||
}
|
||||
|
||||
// v0.139.0: the hub sees the deploy in seconds (debounced trigger, not per-request)
|
||||
r.reportPushNow()
|
||||
|
||||
// Re-apply integrations that target this newly deployed stack
|
||||
if r.integrationMgr != nil {
|
||||
go r.integrationMgr.OnStackStart(context.Background(), name)
|
||||
}
|
||||
}
|
||||
|
||||
// startGatedByMissingDrive reports whether starting `name` must be BLOCKED because the drive its
|
||||
// HDD_PATH points at is currently disconnected or decommissioned. Returns the storage path for the
|
||||
// message. SSD-resident apps (no HDD_PATH) are never gated.
|
||||
func (r *Router) startGatedByMissingDrive(name string) (bool, string) {
|
||||
cfg := r.stackMgr.LoadAppConfigByName(name)
|
||||
if cfg == nil {
|
||||
return false, ""
|
||||
}
|
||||
hdd := cfg.Env["HDD_PATH"]
|
||||
if hdd == "" {
|
||||
return false, ""
|
||||
}
|
||||
for _, sp := range r.sett.GetStoragePaths() {
|
||||
if sp.Path == hdd && (sp.Disconnected || sp.Decommissioned) {
|
||||
return true, hdd
|
||||
}
|
||||
}
|
||||
return false, ""
|
||||
}
|
||||
|
||||
// desiredStateForAction maps a stack action to the customer intent it expresses, or (_, false) for
|
||||
// an action that expresses none. Pure, so the §8.1/§1.3 mapping is testable without a Manager.
|
||||
//
|
||||
// `restart` and `update` both mean running: a customer who updates or restarts an app is asking for
|
||||
// it to be up afterwards, and both end in `compose up -d`. Anything not listed here — an unknown
|
||||
// action string — records nothing rather than guessing, so a future action cannot silently acquire
|
||||
// an intent it was never meant to carry.
|
||||
func desiredStateForAction(action string) (string, bool) {
|
||||
switch action {
|
||||
case "start", "restart", "update":
|
||||
return stacks.DesiredStateRunning, true
|
||||
case "stop":
|
||||
return stacks.DesiredStateStopped, true
|
||||
default:
|
||||
return "", false
|
||||
}
|
||||
}
|
||||
|
||||
func (r *Router) actionStack(w http.ResponseWriter, action, name string) {
|
||||
r.logger.Printf("[INFO] [api] %s requested for stack: %s", action, name)
|
||||
r.dbg("actionStack: action=%s name=%s", action, name)
|
||||
@@ -562,19 +397,6 @@ func (r *Router) actionStack(w http.ResponseWriter, action, name string) {
|
||||
return
|
||||
}
|
||||
|
||||
// Drive-absent gate: refuse to start an app whose data drive is currently disconnected/decommissioned
|
||||
// (the intermediary-mount gate). Starting it would let it write to the empty fail-closed stable path
|
||||
// or just crash-loop; block with a clear message until the drive returns (then the gate auto-restarts).
|
||||
if action == "start" {
|
||||
if gated, hdd := r.startGatedByMissingDrive(name); gated {
|
||||
writeJSON(w, http.StatusConflict, apiResponse{
|
||||
OK: false,
|
||||
Error: fmt.Sprintf("A(z) %s tárhely jelenleg nem elérhető — az alkalmazás nem indítható, amíg a meghajtó vissza nem csatlakozik.", hdd),
|
||||
})
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
// Memory check before starting a stopped app
|
||||
if action == "start" {
|
||||
stackMemMB := r.stackMgr.StackMemoryMB(name)
|
||||
@@ -597,26 +419,6 @@ func (r *Router) actionStack(w http.ResponseWriter, action, name string) {
|
||||
}
|
||||
}
|
||||
|
||||
// R-166: THE CUSTOMER-INTENT POINT. This switch is where a human's decision about whether their
|
||||
// app should be running enters the system, and until v0.189.0 that decision was recorded nowhere
|
||||
// — so the box had to infer it from container counts, and inferred wrong for a power cut and for
|
||||
// an interrupted backup alike.
|
||||
//
|
||||
// Written BEFORE the act (§8.2) and a failed write REFUSES the act: performing a stop whose
|
||||
// intent could not be recorded would recreate exactly the ambiguity this closes. Both gates that
|
||||
// can legitimately refuse an action (protected-stack, drive-absent, memory) have already run
|
||||
// above, so nothing is recorded for an action that was never going to happen.
|
||||
if desired, ok := desiredStateForAction(action); ok {
|
||||
if derr := r.stackMgr.SetDesiredState(name, desired); derr != nil {
|
||||
r.logger.Printf("[ERROR] [api] %s for %s refused: could not record desired state: %v", action, name, derr)
|
||||
writeJSON(w, http.StatusInternalServerError, apiResponse{
|
||||
OK: false,
|
||||
Error: "A művelet nem hajtható végre: az alkalmazás beállításai nem menthetők.",
|
||||
})
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
var err error
|
||||
switch action {
|
||||
case "start":
|
||||
@@ -844,9 +646,6 @@ func (r *Router) removeStack(w http.ResponseWriter, req *http.Request, name stri
|
||||
if r.OnGeoRelevantChange != nil {
|
||||
go r.OnGeoRelevantChange()
|
||||
}
|
||||
|
||||
// v0.139.0: the hub sees the removal in seconds (debounced trigger, not per-request)
|
||||
r.reportPushNow()
|
||||
}
|
||||
|
||||
func (r *Router) deleteStack(w http.ResponseWriter, req *http.Request, name string) {
|
||||
@@ -878,9 +677,6 @@ func (r *Router) deleteStack(w http.ResponseWriter, req *http.Request, name stri
|
||||
}
|
||||
|
||||
writeJSON(w, http.StatusOK, apiResponse{OK: true, Data: resp, Message: "Stack " + name + " deleted"})
|
||||
|
||||
// v0.139.0: the hub sees the delete in seconds (debounced trigger, not per-request)
|
||||
r.reportPushNow()
|
||||
}
|
||||
|
||||
func (r *Router) triggerSync(w http.ResponseWriter, _ *http.Request) {
|
||||
@@ -896,20 +692,6 @@ func (r *Router) triggerSync(w http.ResponseWriter, _ *http.Request) {
|
||||
|
||||
func (r *Router) systemInfo(w http.ResponseWriter, _ *http.Request) {
|
||||
info := system.GetInfo(r.cfg.Paths.HDDPath, r.cpuCollector)
|
||||
// F1: GetInfo now reports the guest RAM cap (from the Docker daemon) as TotalMemMB, but the guest-wide
|
||||
// "used" is not observable from the container. Report the controller's accurate committed-app memory
|
||||
// (sum of running apps' mem requests) as used — a meaningful "allocated of cap" figure for the UI.
|
||||
if r.stackMgr != nil && info.TotalMemMB > 0 {
|
||||
if reqMB, _ := r.stackMgr.CommittedMemory(); reqMB >= 0 {
|
||||
used := uint64(reqMB)
|
||||
if used > info.TotalMemMB {
|
||||
used = info.TotalMemMB
|
||||
}
|
||||
info.UsedMemMB = used
|
||||
info.AvailMemMB = info.TotalMemMB - used
|
||||
info.MemPercent = float64(used) / float64(info.TotalMemMB) * 100
|
||||
}
|
||||
}
|
||||
syncStatus := r.syncer.Status()
|
||||
data := map[string]interface{}{
|
||||
"system": info,
|
||||
@@ -946,54 +728,6 @@ func (r *Router) backupStatus(w http.ResponseWriter, _ *http.Request) {
|
||||
writeJSON(w, http.StatusOK, apiResponse{OK: true, Data: data})
|
||||
}
|
||||
|
||||
// backupRestoreStatus (Part B) surfaces the async restore-op progress the backups page polls to drive
|
||||
// its banner (running: <op>/<stack> → terminal last{ok,message}). Display-only; empty when idle.
|
||||
func (r *Router) backupRestoreStatus(w http.ResponseWriter, _ *http.Request) {
|
||||
if r.backupMgr == nil {
|
||||
writeJSON(w, http.StatusOK, apiResponse{OK: true, Data: map[string]interface{}{"running": false}})
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, apiResponse{OK: true, Data: r.backupMgr.RestoreStatus()})
|
||||
}
|
||||
|
||||
// validStackParam reports whether a stack name from a request is a safe single path segment
|
||||
// (same semantics as web.validStackName — see internal/web/validate.go; duplicated here because
|
||||
// api ↔ web would be a circular import). Rejects traversal/escape so the name can never become
|
||||
// a filesystem path outside the app's namespace root.
|
||||
func validStackParam(name string) bool {
|
||||
if name == "" || name == "." || name == ".." {
|
||||
return false
|
||||
}
|
||||
if strings.ContainsAny(name, "/\\\x00") {
|
||||
return false
|
||||
}
|
||||
return name == filepath.Clean(name)
|
||||
}
|
||||
|
||||
// backupSnapshots lists the restorable keep-side backups for one app — the data source of the
|
||||
// /backups restore panel's snapshot dropdown (F1: this route was fetched by the template but never
|
||||
// existed, so the dropdown could never populate and the restore button never enabled).
|
||||
func (r *Router) backupSnapshots(w http.ResponseWriter, req *http.Request) {
|
||||
stack := req.URL.Query().Get("stack")
|
||||
if !validStackParam(stack) {
|
||||
r.dbg("backupSnapshots: invalid stack param %q", stack)
|
||||
writeJSON(w, http.StatusBadRequest, apiResponse{OK: false, Error: "invalid stack name"})
|
||||
return
|
||||
}
|
||||
if r.backupMgr == nil {
|
||||
writeJSON(w, http.StatusBadRequest, apiResponse{OK: false, Error: "Backup not configured"})
|
||||
return
|
||||
}
|
||||
|
||||
points, found := r.backupMgr.ListRestorePoints(stack)
|
||||
if !found {
|
||||
writeJSON(w, http.StatusNotFound, apiResponse{OK: false, Error: "stack not found: " + stack})
|
||||
return
|
||||
}
|
||||
r.dbg("backupSnapshots: stack=%s points=%d", stack, len(points))
|
||||
writeJSON(w, http.StatusOK, apiResponse{OK: true, Data: points})
|
||||
}
|
||||
|
||||
// triggerBackup runs the app-data database dumps. Disk-tier (restic) backup has
|
||||
// moved to the host agent (slice 8C).
|
||||
func (r *Router) triggerBackup(w http.ResponseWriter, _ *http.Request) {
|
||||
@@ -1010,28 +744,11 @@ func (r *Router) triggerBackup(w http.ResponseWriter, _ *http.Request) {
|
||||
}
|
||||
|
||||
r.logger.Println("[INFO] [api] Manual app-data backup (DB dump) triggered")
|
||||
r.backupMgr.MarkManualRun() // R-182: operator-triggered — its digest must not be collapsed into the nightly one
|
||||
go r.backupMgr.RunDBDumps(context.Background())
|
||||
|
||||
writeJSON(w, http.StatusOK, apiResponse{OK: true, Message: "Mentés elindítva"})
|
||||
}
|
||||
|
||||
// triggerTier2 runs the off-drive Tier 2 copies for all HDD apps (recovery unit + userdata to a
|
||||
// different physical disk). Auto-targets and applies the rootfs-headroom guard internally.
|
||||
func (r *Router) triggerTier2(w http.ResponseWriter, _ *http.Request) {
|
||||
if r.backupMgr == nil {
|
||||
writeJSON(w, http.StatusBadRequest, apiResponse{OK: false, Error: "Backup not configured"})
|
||||
return
|
||||
}
|
||||
if r.backupMgr.IsRunning() {
|
||||
writeJSON(w, http.StatusConflict, apiResponse{OK: false, Error: "Mentés már folyamatban"})
|
||||
return
|
||||
}
|
||||
r.logger.Println("[INFO] [api] Manual Tier 2 (off-drive) backup triggered")
|
||||
go r.backupMgr.RunAllTier2()
|
||||
writeJSON(w, http.StatusOK, apiResponse{OK: true, Message: "2. mentés elindítva"})
|
||||
}
|
||||
|
||||
// --- Metrics handlers ---
|
||||
|
||||
func (r *Router) metricsSystem(w http.ResponseWriter, req *http.Request) {
|
||||
@@ -1251,54 +968,33 @@ func (r *Router) configApply(w http.ResponseWriter, req *http.Request) {
|
||||
return
|
||||
}
|
||||
|
||||
// No-op guard: if the pushed config is byte-identical to what is already on disk, do
|
||||
// nothing — don't rewrite, don't restart. The hub may re-push the same config
|
||||
// idempotently, and a self-restart on every push would be a needless flap.
|
||||
if prior, rerr := os.ReadFile(r.configPath); rerr == nil && bytes.Equal(prior, body) {
|
||||
r.logger.Printf("[INFO] [api] Config apply: identical to current config (%d bytes) — no change, no restart", len(body))
|
||||
writeJSON(w, http.StatusOK, apiResponse{OK: true, Message: "A konfiguráció változatlan — nincs szükség újraindításra."})
|
||||
// Write config: try atomic rename first, fall back to direct write
|
||||
// (os.Rename fails on Docker bind mounts with "device or resource busy")
|
||||
tmpPath := r.configPath + ".tmp"
|
||||
if err := os.WriteFile(tmpPath, body, 0644); err != nil {
|
||||
r.logger.Printf("[ERROR] [api] Config apply: failed to write temp file: %v", err)
|
||||
writeJSON(w, http.StatusInternalServerError, apiResponse{OK: false, Error: "failed to write config"})
|
||||
return
|
||||
}
|
||||
|
||||
// Write config 0600: it holds infra credentials (cf_api_token, cf_tunnel_token, hub api_key) in
|
||||
// plaintext (F8), so it must not be world-readable. writeConfig0600 enforces the mode even when the
|
||||
// target file already existed with looser perms (os.WriteFile does not chmod an existing file).
|
||||
if err := writeConfig0600(r.configPath, body); err != nil {
|
||||
r.logger.Printf("[ERROR] [api] Config apply: failed to write config: %v", err)
|
||||
writeJSON(w, http.StatusInternalServerError, apiResponse{OK: false, Error: "failed to apply config"})
|
||||
return
|
||||
}
|
||||
|
||||
// Respond to the hub FIRST (and flush), THEN self-restart. The new config only takes
|
||||
// effect on restart — singletons such as the Cloudflare client are built once at
|
||||
// startup, so an in-process write alone would leave e.g. a rotated CF token unused.
|
||||
r.logger.Printf("[INFO] [api] Config applied from Hub (%d bytes) — self-restarting to take effect", len(body))
|
||||
writeJSON(w, http.StatusOK, apiResponse{OK: true, Message: "Konfiguráció alkalmazva — a vezérlő újraindul."})
|
||||
flushResponse(w)
|
||||
if r.restart != nil {
|
||||
r.restart()
|
||||
}
|
||||
}
|
||||
|
||||
// writeConfig0600 writes config bytes to path with mode 0600, atomically when possible (tmp+rename),
|
||||
// falling back to a direct write for Docker bind mounts (where os.Rename returns EBUSY). It ALWAYS
|
||||
// enforces 0600 on the final file — even if it already existed with looser perms — because controller.yaml
|
||||
// holds infra credentials in plaintext (F8); os.WriteFile only applies the mode when creating a new file.
|
||||
func writeConfig0600(path string, body []byte) error {
|
||||
tmpPath := path + ".tmp"
|
||||
if err := os.WriteFile(tmpPath, body, 0600); err != nil {
|
||||
return fmt.Errorf("writing temp config: %w", err)
|
||||
}
|
||||
if err := os.Rename(tmpPath, path); err != nil {
|
||||
if err := os.Rename(tmpPath, r.configPath); err != nil {
|
||||
os.Remove(tmpPath)
|
||||
if err := os.WriteFile(path, body, 0600); err != nil { // bind-mount fallback
|
||||
return fmt.Errorf("writing config: %w", err)
|
||||
// Rename failed (likely Docker bind mount) — write directly
|
||||
if err := os.WriteFile(r.configPath, body, 0644); err != nil {
|
||||
r.logger.Printf("[ERROR] [api] Config apply: failed to write config: %v", err)
|
||||
writeJSON(w, http.StatusInternalServerError, apiResponse{OK: false, Error: "failed to apply config"})
|
||||
return
|
||||
}
|
||||
r.logger.Printf("[INFO] [api] Config apply: rename failed, wrote directly (bind mount)")
|
||||
}
|
||||
if err := os.Chmod(path, 0600); err != nil {
|
||||
return fmt.Errorf("chmod config 0600: %w", err)
|
||||
|
||||
r.logger.Printf("[INFO] [api] Config applied from Hub (%d bytes), restart needed to take effect", len(body))
|
||||
writeJSON(w, http.StatusOK, apiResponse{OK: true, Message: "Config applied. Restart controller to apply changes."})
|
||||
|
||||
// Push updated infra backup so Hub has fresh config data immediately
|
||||
if r.OnConfigApplied != nil {
|
||||
go r.OnConfigApplied()
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (r *Router) configHash(w http.ResponseWriter, _ *http.Request) {
|
||||
|
||||
@@ -1,54 +0,0 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"log"
|
||||
"net/http"
|
||||
"os"
|
||||
"time"
|
||||
)
|
||||
|
||||
// restartDelay gives the in-flight HTTP response time to flush before the process exits.
|
||||
const restartDelay = 500 * time.Millisecond
|
||||
|
||||
// gracefulSelfRestart schedules a clean process exit after a short delay. The container
|
||||
// runs with `restart: unless-stopped`, so exiting 0 makes Docker start a fresh process
|
||||
// that re-reads controller.yaml. This is how a config-apply (e.g. a rotated Cloudflare
|
||||
// API token) and the manual restart button actually take effect: singletons such as the
|
||||
// Cloudflare client are built once at startup and are not reloaded in-process.
|
||||
func gracefulSelfRestart(logger *log.Logger) {
|
||||
GracefulSelfRestart(logger)
|
||||
}
|
||||
|
||||
// GracefulSelfRestart is the exported entry point to the same graceful restart, so non-api callers
|
||||
// (the config-refresh reconcile wired in main.go) reuse this one mechanism instead of reinventing an
|
||||
// os.Exit path. See gracefulSelfRestart for the rationale.
|
||||
func GracefulSelfRestart(logger *log.Logger) {
|
||||
go func() {
|
||||
time.Sleep(restartDelay)
|
||||
if logger != nil {
|
||||
logger.Println("[INFO] [api] Graceful self-restart: exiting (0) for container restart")
|
||||
}
|
||||
os.Exit(0)
|
||||
}()
|
||||
}
|
||||
|
||||
// flushResponse flushes buffered output to the client if the writer supports it, so the
|
||||
// caller receives the response body before a subsequent self-restart kills the process.
|
||||
func flushResponse(w http.ResponseWriter) {
|
||||
if f, ok := w.(http.Flusher); ok {
|
||||
f.Flush()
|
||||
}
|
||||
}
|
||||
|
||||
// selfRestart handles POST /api/selfrestart — a customer-facing self-serve restart so a
|
||||
// user can recover the controller without rebooting the whole guest. Auth + CSRF are
|
||||
// applied by the /api/ mux mount (same protection as every other state-changing endpoint).
|
||||
// Responds first, flushes, then triggers the graceful restart.
|
||||
func (r *Router) selfRestart(w http.ResponseWriter, _ *http.Request) {
|
||||
r.logger.Println("[INFO] [api] Manual controller restart requested")
|
||||
writeJSON(w, http.StatusOK, apiResponse{OK: true, Message: "Újraindítás folyamatban…"})
|
||||
flushResponse(w)
|
||||
if r.restart != nil {
|
||||
r.restart()
|
||||
}
|
||||
}
|
||||
@@ -1,99 +0,0 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"io"
|
||||
"log"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// newRestartTestRouter builds a minimal Router with a recording restart seam (so the
|
||||
// process is never actually killed) and a config path seeded with `prior`.
|
||||
func newRestartTestRouter(t *testing.T, prior []byte) (*Router, *int, string) {
|
||||
t.Helper()
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "controller.yaml")
|
||||
if prior != nil {
|
||||
if err := os.WriteFile(path, prior, 0o600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
calls := 0
|
||||
r := &Router{configPath: path, logger: log.New(io.Discard, "", 0)}
|
||||
r.restart = func() { calls++ }
|
||||
return r, &calls, path
|
||||
}
|
||||
|
||||
// a minimal config that passes config.LoadFromBytes (customer.id + customer.domain required).
|
||||
var validConfig = []byte("customer:\n id: test\n domain: test.example\n")
|
||||
|
||||
func TestConfigApply_ChangedConfig_Restarts(t *testing.T) {
|
||||
prior := []byte("customer:\n id: old\n domain: old.example\n")
|
||||
r, calls, path := newRestartTestRouter(t, prior)
|
||||
|
||||
req := httptest.NewRequest(http.MethodPost, "/api/config/apply", bytes.NewReader(validConfig))
|
||||
rec := httptest.NewRecorder()
|
||||
r.configApply(rec, req)
|
||||
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d, want 200; body=%s", rec.Code, rec.Body.String())
|
||||
}
|
||||
if *calls != 1 {
|
||||
t.Fatalf("restart called %d times, want 1 (config changed)", *calls)
|
||||
}
|
||||
got, _ := os.ReadFile(path)
|
||||
if !bytes.Equal(got, validConfig) {
|
||||
t.Fatalf("config not written: got %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
// COMPANION: an identical re-push must NOT restart (the old handler always restarted /
|
||||
// always ran its post-apply hook regardless of whether anything changed).
|
||||
func TestConfigApply_IdenticalConfig_NoRestart(t *testing.T) {
|
||||
r, calls, _ := newRestartTestRouter(t, validConfig) // prior == body
|
||||
|
||||
req := httptest.NewRequest(http.MethodPost, "/api/config/apply", bytes.NewReader(validConfig))
|
||||
rec := httptest.NewRecorder()
|
||||
r.configApply(rec, req)
|
||||
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d, want 200; body=%s", rec.Code, rec.Body.String())
|
||||
}
|
||||
if *calls != 0 {
|
||||
t.Fatalf("restart called %d times, want 0 (config byte-identical)", *calls)
|
||||
}
|
||||
}
|
||||
|
||||
func TestConfigApply_InvalidYAML_NoRestart(t *testing.T) {
|
||||
r, calls, _ := newRestartTestRouter(t, validConfig)
|
||||
|
||||
req := httptest.NewRequest(http.MethodPost, "/api/config/apply", bytes.NewReader([]byte("customer:\n id: only-id-no-domain\n")))
|
||||
rec := httptest.NewRecorder()
|
||||
r.configApply(rec, req)
|
||||
|
||||
if rec.Code != http.StatusBadRequest {
|
||||
t.Fatalf("status = %d, want 400 (missing customer.domain)", rec.Code)
|
||||
}
|
||||
if *calls != 0 {
|
||||
t.Fatalf("restart called %d times, want 0 (validation failed)", *calls)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSelfRestart_InvokesRestarter(t *testing.T) {
|
||||
r, calls, _ := newRestartTestRouter(t, nil)
|
||||
|
||||
req := httptest.NewRequest(http.MethodPost, "/api/selfrestart", nil)
|
||||
rec := httptest.NewRecorder()
|
||||
r.selfRestart(rec, req)
|
||||
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d, want 200", rec.Code)
|
||||
}
|
||||
if *calls != 1 {
|
||||
t.Fatalf("restart called %d times, want 1", *calls)
|
||||
}
|
||||
}
|
||||
@@ -1,7 +1,6 @@
|
||||
package appbackup
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"context"
|
||||
"fmt"
|
||||
"log"
|
||||
@@ -19,100 +18,11 @@ type StackDataProvider interface {
|
||||
GetStackComposePath(name string) (composePath string, ok bool)
|
||||
ListDeployedStacks() []StackSummary
|
||||
GetStackHDDMounts(name string) []string
|
||||
GetStackHDDPath(name string) string // raw HDD_PATH from app.yaml (empty if no HDD)
|
||||
// GetImportRoot returns the CANONICAL drop-zone root (R-75): <system namespace root>/userdata/import.
|
||||
// It is app-INDEPENDENT and lives on the SYSTEM drive, so ${IMPORT_PATH} binds cannot be resolved
|
||||
// from GetStackHDDPath. Empty when unresolvable — structuralGuard refuses such binds loudly.
|
||||
GetImportRoot() string
|
||||
GetStackHDDPath(name string) string // raw HDD_PATH from app.yaml (empty if no HDD)
|
||||
GetDockerVolumes(name string) []string // full Docker volume names (project-prefixed)
|
||||
StopStack(name string) error
|
||||
StartStack(name string) error
|
||||
RefreshAndIsRunning(name string) bool
|
||||
// GetStackRecoveryInfo returns the data needed to capture a recovery unit: the stack dir,
|
||||
// pinned image tags, the non-secret env, the NAMES of the secret/data-key env vars, and (D5)
|
||||
// the decrypted VALUES of the portable class. A WITHHELD secret's value is never returned —
|
||||
// it is recovered at restore time from the guest's app.yaml, or regenerated. ok=false if the
|
||||
// stack is unknown.
|
||||
GetStackRecoveryInfo(name string) (RecoveryInfo, bool)
|
||||
|
||||
// --- Phase 2b: restore-from-recovery-unit ---
|
||||
|
||||
// RecoverStackSecrets returns the live decrypted values for the named secret env vars that are
|
||||
// currently present (non-empty) in the stack's app.yaml (the guest's own — live rootfs, or
|
||||
// PBS-restored). Names that are absent/empty are simply omitted from the map; the caller's
|
||||
// fail-closed gate decides what to do. The unit is never the source of secrets.
|
||||
RecoverStackSecrets(name string, names []string) map[string]string
|
||||
|
||||
// RecreateStackDefinitionFromUnit restores an app's DEFINITION from the unit's compose dir into
|
||||
// the stack dir and writes app.yaml from fullEnv (encrypting secret fields). Secrets are NEVER
|
||||
// regenerated. It starts NOTHING: the caller owns the bring-up order, because a DB-bearing app
|
||||
// must have its database service started alone for the dump replay (R-47). It was
|
||||
// `RecreateStackFromUnit` until v0.153.0 and ended in a full `docker compose up -d` — that full
|
||||
// start before the replay IS the H4 race.
|
||||
RecreateStackDefinitionFromUnit(name, composeSrcDir string, fullEnv map[string]string) error
|
||||
|
||||
// StartStackServices brings up ONLY the named compose services, leaving the rest of the stack
|
||||
// down — the DB-only window in which a dump is replayed without the application racing it.
|
||||
// Implementations must REFUSE an empty list (an argument-less `up -d` is a full start).
|
||||
StartStackServices(name string, services []string) error
|
||||
|
||||
// GetStackClassifiedBinds returns the app's backup-classified compose binds + whether it carries a
|
||||
// (valid) backup block (Task 2, referential coupling). INERT — no tier consumes it yet; wired now
|
||||
// so Task 3 gets a tested seam. Implemented by delegating to stacks.Manager.ClassifiedBinds.
|
||||
GetStackClassifiedBinds(name string) ([]ClassifiedBind, bool)
|
||||
}
|
||||
|
||||
// RecoveryInfo carries everything needed to write a recovery unit for a stack.
|
||||
//
|
||||
// D5: it now carries the VALUES of the PORTABLE secret class (stacks.PortableSecretEnvVars — every
|
||||
// `type: secret` field bar the nonPortableSecrets register), because a Tier-1/2 restore that depends
|
||||
// on the guest for a data-encrypting key or a DB password is not independent of the guest at all: the
|
||||
// data sits safely on the drive and cannot be read back. The EXCLUDED class (`type: password` admin
|
||||
// logins) is still name-only and never leaves the guest.
|
||||
type RecoveryInfo struct {
|
||||
StackDir string // dir holding docker-compose.yml + .felhom.yml + app.yaml
|
||||
DisplayName string // app display name
|
||||
ImagePins []string // pinned image tags from compose `image:` lines (re-pulled on restore)
|
||||
NonSecretEnv map[string]string // env with ALL secret/password values removed (plaintext only)
|
||||
SecretEnvVars []string // NAMES of every secret/password field
|
||||
DataKeyEnvVars []string // NAMES of data-encrypting-key fields (fail-closed gate on restore)
|
||||
// PortableSecretEnvVars are the NAMES of the secrets that travel in the unit (D5), and
|
||||
// PortableSecrets their DECRYPTED values. A name present here but absent from PortableSecrets was
|
||||
// unset/empty in the guest's app.yaml — the restore's fail-closed gate decides what that means.
|
||||
// Never logged, never in the manifest's value space: the values reach disk only inside the unit's
|
||||
// 0600 app.yaml.
|
||||
PortableSecretEnvVars []string
|
||||
PortableSecrets map[string]string
|
||||
}
|
||||
|
||||
// ParseComposeImages extracts the pinned image references (`image: repo:tag`) from a
|
||||
// docker-compose.yml, in file order, de-duplicated. The image bytes are never stored in the
|
||||
// recovery unit — only these pins, so restore re-pulls from the registry.
|
||||
func ParseComposeImages(composePath string) []string {
|
||||
data, err := os.ReadFile(composePath)
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
var images []string
|
||||
seen := make(map[string]bool)
|
||||
scanner := bufio.NewScanner(strings.NewReader(string(data)))
|
||||
for scanner.Scan() {
|
||||
line := strings.TrimSpace(scanner.Text())
|
||||
if !strings.HasPrefix(line, "image:") {
|
||||
continue
|
||||
}
|
||||
img := strings.TrimSpace(strings.TrimPrefix(line, "image:"))
|
||||
img = strings.Trim(img, "\"'")
|
||||
// Skip variable-only images we can't pin (e.g. image: ${SOME_IMAGE})
|
||||
if img == "" || strings.HasPrefix(img, "${") {
|
||||
continue
|
||||
}
|
||||
if !seen[img] {
|
||||
seen[img] = true
|
||||
images = append(images, img)
|
||||
}
|
||||
}
|
||||
return images
|
||||
}
|
||||
|
||||
// StackSummary holds minimal stack info needed for app data discovery.
|
||||
|
||||
@@ -1,104 +0,0 @@
|
||||
package appbackup
|
||||
|
||||
import (
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// fp joins the elements under an HDD path with OS separators — mounts in the ParseComposeHDDMounts
|
||||
// shape are already filepath.Clean'd, so tests build them the same way.
|
||||
func fp(elems ...string) string { return filepath.Join(elems...) }
|
||||
|
||||
// TestAppDataDirNames is the pure derivation table (Group A). Every case asserts the RESOLVED name
|
||||
// list, never mere absence of error. Companion RP-1: a resolver that ignores mounts and returns
|
||||
// []string{stackName} fails the paperless, two-name, and dedupe cases.
|
||||
func TestAppDataDirNames(t *testing.T) {
|
||||
const hdd = "/mnt/felhom-usb"
|
||||
cases := []struct {
|
||||
name string
|
||||
stack string
|
||||
mounts []string
|
||||
want []string
|
||||
}{
|
||||
{
|
||||
// paperless shape: stack "paperless-ngx", dir "paperless" (F-S2/F-S3 core).
|
||||
name: "paperless mismatch",
|
||||
stack: "paperless-ngx",
|
||||
mounts: []string{
|
||||
fp(hdd, "appdata", "paperless", "media"),
|
||||
fp(hdd, "appdata", "paperless", "export"),
|
||||
},
|
||||
want: []string{"paperless"}, // media+export dedupe to one name
|
||||
},
|
||||
{
|
||||
// match shape: dir name == stack name (immich/nextcloud/romm).
|
||||
name: "matching name",
|
||||
stack: "nextcloud",
|
||||
mounts: []string{fp(hdd, "appdata", "nextcloud")},
|
||||
want: []string{"nextcloud"},
|
||||
},
|
||||
{
|
||||
// two DISTINCT names → both, sorted (no catalog app does this today).
|
||||
name: "two distinct names sorted",
|
||||
stack: "weird",
|
||||
mounts: []string{
|
||||
fp(hdd, "appdata", "zebra", "x"),
|
||||
fp(hdd, "appdata", "alpha", "y"),
|
||||
},
|
||||
want: []string{"alpha", "zebra"},
|
||||
},
|
||||
{
|
||||
// non-appdata HDD binds + a foreign-drive mount are filtered → fallback.
|
||||
name: "non-appdata and foreign filtered",
|
||||
stack: "romm",
|
||||
mounts: []string{
|
||||
fp(hdd, "roms"), // under HDD but not appdata/
|
||||
fp("/mnt/other-drive", "appdata", "ghost"), // foreign drive — wrong prefix
|
||||
},
|
||||
want: []string{"romm"},
|
||||
},
|
||||
{
|
||||
// whole-appdata-root bind (no name derivable) → ignored → fallback.
|
||||
name: "whole appdata root bind",
|
||||
stack: "root-binder",
|
||||
mounts: []string{fp(hdd, "appdata")},
|
||||
want: []string{"root-binder"},
|
||||
},
|
||||
{
|
||||
name: "empty mounts fallback",
|
||||
stack: "vaultwarden",
|
||||
mounts: nil,
|
||||
want: []string{"vaultwarden"},
|
||||
},
|
||||
{
|
||||
// unclean paths still resolve (Clean applied both sides).
|
||||
name: "unclean path",
|
||||
stack: "paperless-ngx",
|
||||
mounts: []string{hdd + "/appdata/paperless/../paperless/media"},
|
||||
want: []string{"paperless"},
|
||||
},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
got := AppDataDirNames(hdd, tc.stack, tc.mounts)
|
||||
if !reflect.DeepEqual(got, tc.want) {
|
||||
t.Errorf("AppDataDirNames(%q, %q, %v) = %v, want %v", hdd, tc.stack, tc.mounts, got, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestAppDataBindsPresent pins the WARN predicate: true only when a mount sits under appdata/.
|
||||
func TestAppDataBindsPresent(t *testing.T) {
|
||||
const hdd = "/mnt/felhom-usb"
|
||||
if !AppDataBindsPresent(hdd, []string{fp(hdd, "appdata", "paperless", "media")}) {
|
||||
t.Error("declared appdata bind should report present")
|
||||
}
|
||||
if AppDataBindsPresent(hdd, []string{fp(hdd, "roms")}) {
|
||||
t.Error("non-appdata bind should NOT report present")
|
||||
}
|
||||
if AppDataBindsPresent(hdd, nil) {
|
||||
t.Error("no mounts should NOT report present")
|
||||
}
|
||||
}
|
||||
@@ -1,356 +0,0 @@
|
||||
package appbackup
|
||||
|
||||
import (
|
||||
"path"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Capture-set computation — Task 3-core of the backup-classification-redesign arc
|
||||
// (felhom.eu/documentation/architecture/07-backup-architecture.md §3; tier×class matrix §2; SQ2/SQ5
|
||||
// in SPIKE-backup-classification-2026-07-14.md). This is a PURE path-algebra layer: given an app's
|
||||
// classified binds, a tier, and the app's live hddPath, it returns the tier-filtered,
|
||||
// structurally-guarded, containment-deduped absolute capture set that the 3a (offsite) and 3b
|
||||
// (tier-2) engines will capture. Deliberately INERT — no engine consumes it yet.
|
||||
//
|
||||
// Purity contract: no os, no exec, no logging, no filepath. Reasons for refused captures are DATA
|
||||
// (SkippedPath.Reason); the engines log them (a skipped MANDATORY is a capture GAP the engines must
|
||||
// surface loudly). All resolution and prefix algebra uses path.Join/path.Clean and "/" string ops —
|
||||
// never filepath.* — because RelPath is defined forward-slash (classify.go) and every resolved Abs
|
||||
// is an in-container Linux path; filepath on the Windows `go test` host would flip separators and
|
||||
// break both expectations and the containment prefix checks.
|
||||
|
||||
// CaptureTier selects the tier column of §2 that the filter applies.
|
||||
type CaptureTier string
|
||||
|
||||
const (
|
||||
TierOffsite CaptureTier = "offsite" // §2: mandatory only (optional is the customer's local tier)
|
||||
TierSecondary CaptureTier = "secondary" // §2: mandatory + optional
|
||||
)
|
||||
|
||||
// CapturePath is one resolved path in the capture set. Abs is the in-container Linux absolute path;
|
||||
// Root/RelPath preserve the bind's identity (the tier-2 layout and restore relpath-mirroring need it).
|
||||
type CapturePath struct {
|
||||
Abs string
|
||||
Root BindRoot
|
||||
RelPath string
|
||||
Class BindClass
|
||||
}
|
||||
|
||||
// SkippedPath is a would-be capture the tier filter selected but a structural guard refused. Reason
|
||||
// is operator-English; the engines log it (a skipped mandatory path = a silent capture gap otherwise).
|
||||
type SkippedPath struct {
|
||||
Root BindRoot
|
||||
RelPath string
|
||||
Class BindClass
|
||||
Reason string
|
||||
}
|
||||
|
||||
// CaptureSet is the result of ComputeCaptureSet. HasClassification mirrors the classifier's bool;
|
||||
// engines derive unit-only as (!HasClassification || len(Paths)==0). Paths is sorted by Abs.
|
||||
type CaptureSet struct {
|
||||
HasClassification bool
|
||||
Paths []CapturePath
|
||||
Skipped []SkippedPath
|
||||
}
|
||||
|
||||
// Structural-guard reasons (distinct strings; each names the rule it enforces).
|
||||
const (
|
||||
reasonEscape = "path escapes the drive root"
|
||||
reasonBareRoot = "bare drive-root bind would capture the backups tree"
|
||||
reasonReserved = "path inside the reserved backups zone"
|
||||
// reasonNoImportRoot: a ${IMPORT_PATH} bind with no resolvable system namespace root (R-75).
|
||||
reasonNoImportRoot = "canonical import root unresolvable (system_data_path unconfigured)"
|
||||
)
|
||||
|
||||
// ComputeCaptureSet resolves an app's classified binds into the tier-filtered absolute capture set.
|
||||
// Pipeline (fixed order, §8): legacy short-circuit → tier filter (§2 columns) → structural guards →
|
||||
// equal-Abs collapse (mandatory > optional) → containment dedup (keep ancestor) → sort by Abs.
|
||||
//
|
||||
// Tier columns (§2): TierOffsite carries mandatory only; TierSecondary carries mandatory + optional;
|
||||
// excluded is silently dropped at every tier (never in Paths, never in Skipped). A legacy app
|
||||
// (hasClassification=false) resolves NOTHING — {HasClassification:false} with nil Paths/Skipped —
|
||||
// so the engines' no-block branch stays byte-identical to today (the SQ5 cost-regression guard).
|
||||
//
|
||||
// Resolution: RootHDD → path.Join(hddPath, relPath); RootUserdata → path.Join(hddPath, "userdata",
|
||||
// relPath); RootImport → path.Join(importRoot, relPath) — the SYSTEM drive, never hddPath (R-75).
|
||||
// Guards run AFTER the tier filter, so Skipped means exactly "would have been captured by this tier,
|
||||
// refused for structural safety".
|
||||
func ComputeCaptureSet(binds []ClassifiedBind, hasClassification bool, tier CaptureTier, hddPath, importRoot string) CaptureSet {
|
||||
if !hasClassification {
|
||||
return CaptureSet{HasClassification: false}
|
||||
}
|
||||
cs := CaptureSet{HasClassification: true}
|
||||
|
||||
// Stages 1–3: tier filter → structural guards → equal-Abs collapse (shared with ComputeFabBuckets).
|
||||
uniq, skipped := resolveGuardCollapse(binds, hddPath, importRoot, func(c BindClass) bool { return tierKeeps(tier, c) })
|
||||
cs.Skipped = skipped
|
||||
|
||||
// Stage 4: containment dedup — drop any path whose ancestor is already present (keep the ancestor).
|
||||
for _, cp := range uniq {
|
||||
if hasStrictAncestor(cp.Abs, uniq) {
|
||||
continue
|
||||
}
|
||||
cs.Paths = append(cs.Paths, cp)
|
||||
}
|
||||
|
||||
// Stage 5: deterministic order.
|
||||
sort.Slice(cs.Paths, func(i, j int) bool { return cs.Paths[i].Abs < cs.Paths[j].Abs })
|
||||
sort.Slice(cs.Skipped, func(i, j int) bool {
|
||||
if cs.Skipped[i].Root != cs.Skipped[j].Root {
|
||||
return cs.Skipped[i].Root < cs.Skipped[j].Root
|
||||
}
|
||||
return cs.Skipped[i].RelPath < cs.Skipped[j].RelPath
|
||||
})
|
||||
return cs
|
||||
}
|
||||
|
||||
// resolveGuardCollapse is the pipeline shared by ComputeCaptureSet and ComputeFabBuckets: keep-filter
|
||||
// (the caller's predicate over class) → structural guards (Skipped) → resolve to Abs → equal-Abs
|
||||
// collapse (mandatory > optional > excluded; ties by smaller Root/RelPath). It does NOT apply
|
||||
// containment dedup — the caller decides (ComputeCaptureSet does; ComputeFabBuckets must not, so a
|
||||
// mandatory child inside an excluded parent stays independently addressable).
|
||||
func resolveGuardCollapse(binds []ClassifiedBind, hddPath, importRoot string, keep func(BindClass) bool) (uniq []CapturePath, skipped []SkippedPath) {
|
||||
var resolved []CapturePath
|
||||
for _, b := range binds {
|
||||
if !keep(b.Class) {
|
||||
continue
|
||||
}
|
||||
if reason, bad := structuralGuard(b.Root, b.RelPath, importRoot); bad {
|
||||
skipped = append(skipped, SkippedPath{Root: b.Root, RelPath: b.RelPath, Class: b.Class, Reason: reason})
|
||||
continue
|
||||
}
|
||||
resolved = append(resolved, CapturePath{
|
||||
Abs: resolveAbs(hddPath, importRoot, b.Root, b.RelPath), Root: b.Root, RelPath: b.RelPath, Class: b.Class,
|
||||
})
|
||||
}
|
||||
byAbs := make(map[string]CapturePath, len(resolved))
|
||||
for _, cp := range resolved {
|
||||
if cur, ok := byAbs[cp.Abs]; ok {
|
||||
byAbs[cp.Abs] = strongerCapture(cur, cp)
|
||||
continue
|
||||
}
|
||||
byAbs[cp.Abs] = cp
|
||||
}
|
||||
uniq = make([]CapturePath, 0, len(byAbs))
|
||||
for _, cp := range byAbs {
|
||||
uniq = append(uniq, cp)
|
||||
}
|
||||
return uniq, skipped
|
||||
}
|
||||
|
||||
// FabBuckets is the class-bucketed capture set for the manual `.fab` export (Task 4). Unlike
|
||||
// ComputeCaptureSet it keeps ALL classes (structural guards run over every class — a traversal path is
|
||||
// never plannable, opt-in or not) and does NOT collapse across containment (mandatory `media/books`
|
||||
// inside excluded `media` both survive, in their own buckets). HasClassification=false ⇒ empty (the
|
||||
// legacy full-root capture, unchanged).
|
||||
type FabBuckets struct {
|
||||
HasClassification bool
|
||||
Mandatory []CapturePath
|
||||
Optional []CapturePath
|
||||
Excluded []CapturePath
|
||||
Skipped []SkippedPath
|
||||
}
|
||||
|
||||
// ComputeFabBuckets resolves an app's classified binds into per-class buckets for the `.fab` export
|
||||
// selection UI + plan. Same resolution + structural guards + equal-Abs collapse as ComputeCaptureSet
|
||||
// (via resolveGuardCollapse), bucketed by class, no cross-bucket containment dedup. Each bucket is
|
||||
// Abs-sorted (deterministic).
|
||||
func ComputeFabBuckets(binds []ClassifiedBind, hasClassification bool, hddPath, importRoot string) FabBuckets {
|
||||
if !hasClassification {
|
||||
return FabBuckets{HasClassification: false}
|
||||
}
|
||||
fb := FabBuckets{HasClassification: true}
|
||||
uniq, skipped := resolveGuardCollapse(binds, hddPath, importRoot, func(BindClass) bool { return true })
|
||||
fb.Skipped = skipped
|
||||
for _, cp := range uniq {
|
||||
switch cp.Class {
|
||||
case ClassMandatory:
|
||||
fb.Mandatory = append(fb.Mandatory, cp)
|
||||
case ClassOptional:
|
||||
fb.Optional = append(fb.Optional, cp)
|
||||
default:
|
||||
fb.Excluded = append(fb.Excluded, cp)
|
||||
}
|
||||
}
|
||||
for _, b := range []*[]CapturePath{&fb.Mandatory, &fb.Optional, &fb.Excluded} {
|
||||
bk := *b
|
||||
sort.Slice(bk, func(i, j int) bool { return bk[i].Abs < bk[j].Abs })
|
||||
}
|
||||
sort.Slice(fb.Skipped, func(i, j int) bool {
|
||||
if fb.Skipped[i].Root != fb.Skipped[j].Root {
|
||||
return fb.Skipped[i].Root < fb.Skipped[j].Root
|
||||
}
|
||||
return fb.Skipped[i].RelPath < fb.Skipped[j].RelPath
|
||||
})
|
||||
return fb
|
||||
}
|
||||
|
||||
// tierKeeps applies the §2 tier column: mandatory everywhere, optional only for secondary, excluded
|
||||
// never.
|
||||
func tierKeeps(tier CaptureTier, class BindClass) bool {
|
||||
switch class {
|
||||
case ClassMandatory:
|
||||
return true
|
||||
case ClassOptional:
|
||||
return tier == TierSecondary
|
||||
default: // ClassExcluded (and any unexpected value) — never captured automatically
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
// structuralGuard refuses a (root, relPath) that would capture an unsafe location. Evaluated after
|
||||
// the tier filter. RelPath arrives path.Clean'd from the compose parser but is NOT traversal-checked
|
||||
// there (ParseComposeClassifiableBinds path.Cleans; ValidateBackupSpec vets only SPEC entries), so an
|
||||
// unlisted writable "${HDD_PATH}/../x" bind reaches here classed mandatory — this guard is
|
||||
// load-bearing security, not defence-in-depth.
|
||||
func structuralGuard(root BindRoot, relPath, importRoot string) (reason string, bad bool) {
|
||||
if relPathEscapes(relPath) {
|
||||
return reasonEscape, true
|
||||
}
|
||||
// RootImport (R-75) resolves against the SYSTEM drive, not hddPath. If that root is unresolvable
|
||||
// (system_data_path unconfigured) the bind cannot be placed at all — refuse it LOUDLY into Skipped
|
||||
// rather than let resolveAbs join onto "" and produce a relative, wrong-drive path. The other two
|
||||
// roots cannot hit this: hddPath is checked by their own callers.
|
||||
if root == RootImport && importRoot == "" {
|
||||
return reasonNoImportRoot, true
|
||||
}
|
||||
// A bare ${IMPORT_PATH} bind is allowed: it resolves to <sysNS>/userdata/import, which nests no
|
||||
// backups/ tree (backups live at <sysNS>/backups, a sibling of userdata).
|
||||
if root == RootHDD {
|
||||
if relPath == "" {
|
||||
return reasonBareRoot, true // bare ${HDD_PATH} would nest <hddPath>/backups into the capture
|
||||
}
|
||||
if relPath == "backups" || strings.HasPrefix(relPath, "backups/") {
|
||||
return reasonReserved, true
|
||||
}
|
||||
}
|
||||
// RootUserdata + "" is allowed: resolves to <hddPath>/userdata, which does not nest backups/.
|
||||
return "", false
|
||||
}
|
||||
|
||||
// relPathEscapes reports whether relPath is absolute or contains a ".." path segment. Detection is
|
||||
// SEGMENT-WISE on the slash-split path (".." as a whole component), so a legit dir literally named
|
||||
// "a..b" passes.
|
||||
func relPathEscapes(relPath string) bool {
|
||||
if path.IsAbs(relPath) {
|
||||
return true
|
||||
}
|
||||
for _, seg := range strings.Split(relPath, "/") {
|
||||
if seg == ".." {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// resolveAbs maps a guarded (root, relPath) to its in-container absolute path via slash algebra.
|
||||
//
|
||||
// RootImport is the one root that does NOT resolve against hddPath: the canonical drop-zone lives on
|
||||
// the SYSTEM drive (R-75), so importRoot is supplied separately by the caller. Resolving it against
|
||||
// hddPath would silently name a directory on the WRONG DRIVE — a .fab opt-in would then capture (or
|
||||
// on restore, write) somewhere that merely looks plausible. An empty importRoot is the unresolvable
|
||||
// case and is refused upstream by structuralGuard, never silently joined.
|
||||
func resolveAbs(hddPath, importRoot string, root BindRoot, relPath string) string {
|
||||
switch root {
|
||||
case RootUserdata:
|
||||
return path.Join(hddPath, "userdata", relPath)
|
||||
case RootImport:
|
||||
return path.Join(importRoot, relPath)
|
||||
default:
|
||||
return path.Join(hddPath, relPath)
|
||||
}
|
||||
}
|
||||
|
||||
// strongerCapture picks the winner of an equal-Abs collision: mandatory beats optional; on equal
|
||||
// class strength, the lexicographically-smaller (Root, RelPath) wins (determinism).
|
||||
func strongerCapture(a, b CapturePath) CapturePath {
|
||||
sa, sb := classStrength(a.Class), classStrength(b.Class)
|
||||
if sa != sb {
|
||||
if sa > sb {
|
||||
return a
|
||||
}
|
||||
return b
|
||||
}
|
||||
if a.Root != b.Root {
|
||||
if a.Root < b.Root {
|
||||
return a
|
||||
}
|
||||
return b
|
||||
}
|
||||
if a.RelPath <= b.RelPath {
|
||||
return a
|
||||
}
|
||||
return b
|
||||
}
|
||||
|
||||
// classStrength ranks capture classes for the equal-Abs collapse (mandatory must never degrade).
|
||||
func classStrength(c BindClass) int {
|
||||
switch c {
|
||||
case ClassMandatory:
|
||||
return 2
|
||||
case ClassOptional:
|
||||
return 1
|
||||
default:
|
||||
return 0
|
||||
}
|
||||
}
|
||||
|
||||
// hasStrictAncestor reports whether some OTHER path in set is a strict directory ancestor of abs
|
||||
// (abs == ancestor+"/"+…). Slash-aware prefix so "/x/paper" does not "contain" "/x/paperless".
|
||||
func hasStrictAncestor(abs string, set []CapturePath) bool {
|
||||
for _, o := range set {
|
||||
if o.Abs == abs {
|
||||
continue
|
||||
}
|
||||
if strings.HasPrefix(abs, o.Abs+"/") {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// Overlap is one absolute path claimed non-excluded by more than one app's capture set (§4.2). Apps
|
||||
// is sorted.
|
||||
type Overlap struct {
|
||||
Abs string
|
||||
Apps []string
|
||||
}
|
||||
|
||||
// CrossAppOverlaps reports absolute paths that appear in ≥2 apps' Paths — the catalog-convention
|
||||
// tripwire (§4: at most one app may class a host path non-excluded). Pure and advisory here; the
|
||||
// WARN wiring lands in 3a/3b, not in 3-core. Match is EXACT-Abs only: cross-app CONTAINMENT
|
||||
// (calibre-web's mandatory media/books sitting inside plex's excluded reader bind) is legitimate per
|
||||
// §4 and must NOT report. Deterministic despite the map input: app names are scanned in sorted order
|
||||
// and the output is sorted by Abs. Empty input / no overlap → empty (non-nil) slice.
|
||||
func CrossAppOverlaps(sets map[string]CaptureSet) []Overlap {
|
||||
apps := make([]string, 0, len(sets))
|
||||
for app := range sets {
|
||||
apps = append(apps, app)
|
||||
}
|
||||
sort.Strings(apps)
|
||||
|
||||
byAbs := make(map[string][]string)
|
||||
for _, app := range apps {
|
||||
seen := make(map[string]bool) // guard against an app listing the same Abs twice
|
||||
for _, cp := range sets[app].Paths {
|
||||
if seen[cp.Abs] {
|
||||
continue
|
||||
}
|
||||
seen[cp.Abs] = true
|
||||
byAbs[cp.Abs] = append(byAbs[cp.Abs], app)
|
||||
}
|
||||
}
|
||||
|
||||
out := make([]Overlap, 0)
|
||||
for abs, owners := range byAbs {
|
||||
if len(owners) < 2 {
|
||||
continue
|
||||
}
|
||||
sorted := append([]string(nil), owners...)
|
||||
sort.Strings(sorted)
|
||||
out = append(out, Overlap{Abs: abs, Apps: sorted})
|
||||
}
|
||||
sort.Slice(out, func(i, j int) bool { return out[i].Abs < out[j].Abs })
|
||||
return out
|
||||
}
|
||||
@@ -1,247 +0,0 @@
|
||||
package appbackup
|
||||
|
||||
import (
|
||||
"path"
|
||||
"reflect"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// absList extracts the sorted Abs slice from a CaptureSet's Paths (Paths is already Abs-sorted).
|
||||
func absList(cs CaptureSet) []string {
|
||||
out := make([]string, 0, len(cs.Paths))
|
||||
for _, p := range cs.Paths {
|
||||
out = append(out, p.Abs)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// classOfAbs finds the resolved class for an Abs in a CaptureSet (empty if absent).
|
||||
func classOfAbs(cs CaptureSet, abs string) BindClass {
|
||||
for _, p := range cs.Paths {
|
||||
if p.Abs == abs {
|
||||
return p.Class
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
const drv = "/mnt/drv"
|
||||
|
||||
func hdd(p string) string { return path.Join(drv, p) }
|
||||
func udat(p string) string { return path.Join(drv, "userdata", p) }
|
||||
|
||||
// --- Group A (Scenario A): classified per-tier split, immich shape ---
|
||||
|
||||
func TestComputeCaptureSet_PerTierSplit(t *testing.T) {
|
||||
binds := []ClassifiedBind{
|
||||
{ComposeBind: ComposeBind{Root: RootHDD, RelPath: "appdata/immich"}, Class: ClassMandatory, Origin: OriginExplicit},
|
||||
{ComposeBind: ComposeBind{Root: RootUserdata, RelPath: "media/photos", ReadOnly: true}, Class: ClassOptional, Origin: OriginExplicit},
|
||||
}
|
||||
|
||||
off := ComputeCaptureSet(binds, true, TierOffsite, drv, "")
|
||||
if got, want := absList(off), []string{hdd("appdata/immich")}; !reflect.DeepEqual(got, want) {
|
||||
t.Errorf("offsite Paths = %v, want %v (mandatory only — the :ro optional must NOT ship offsite)", got, want)
|
||||
}
|
||||
|
||||
sec := ComputeCaptureSet(binds, true, TierSecondary, drv, "")
|
||||
want := []string{hdd("appdata/immich"), udat("media/photos")}
|
||||
if got := absList(sec); !reflect.DeepEqual(got, want) {
|
||||
t.Errorf("secondary Paths = %v, want %v (sorted)", got, want)
|
||||
}
|
||||
// each CapturePath carries its originating identity
|
||||
for _, p := range sec.Paths {
|
||||
switch p.Abs {
|
||||
case hdd("appdata/immich"):
|
||||
if p.Root != RootHDD || p.RelPath != "appdata/immich" || p.Class != ClassMandatory {
|
||||
t.Errorf("immich CapturePath identity = %+v", p)
|
||||
}
|
||||
case udat("media/photos"):
|
||||
if p.Root != RootUserdata || p.RelPath != "media/photos" || p.Class != ClassOptional {
|
||||
t.Errorf("photos CapturePath identity = %+v", p)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// --- Group B (Scenario B): legacy inertness — THE single most important test (SQ5 guard) ---
|
||||
|
||||
func TestComputeCaptureSet_LegacyInert(t *testing.T) {
|
||||
// A legacy app still has binds (the parser returns them), but with no class and origin=legacy.
|
||||
binds := []ClassifiedBind{
|
||||
{ComposeBind: ComposeBind{Root: RootUserdata, RelPath: "media/tv"}, Origin: OriginLegacy},
|
||||
{ComposeBind: ComposeBind{Root: RootHDD, RelPath: "appdata/sonarr"}, Origin: OriginLegacy},
|
||||
}
|
||||
for _, tier := range []CaptureTier{TierOffsite, TierSecondary} {
|
||||
cs := ComputeCaptureSet(binds, false, tier, drv, "")
|
||||
if cs.HasClassification {
|
||||
t.Errorf("%s: HasClassification=true for a legacy app", tier)
|
||||
}
|
||||
if cs.Paths != nil {
|
||||
t.Errorf("%s: legacy app resolved Paths=%v — MUST be nil (unmigrated-sonarr-ships-its-TV-library regression)", tier, cs.Paths)
|
||||
}
|
||||
if cs.Skipped != nil {
|
||||
t.Errorf("%s: legacy app Skipped=%v — MUST be nil", tier, cs.Skipped)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// --- Group C (Scenario C): excluded is invisible — not in Paths, not in Skipped ---
|
||||
|
||||
func TestComputeCaptureSet_ExcludedInvisible(t *testing.T) {
|
||||
binds := []ClassifiedBind{
|
||||
{ComposeBind: ComposeBind{Root: RootHDD, RelPath: "appdata/paperless/media"}, Class: ClassMandatory, Origin: OriginExplicit},
|
||||
{ComposeBind: ComposeBind{Root: RootHDD, RelPath: "appdata/paperless/export"}, Class: ClassExcluded, Origin: OriginExplicit},
|
||||
{ComposeBind: ComposeBind{Root: RootUserdata, RelPath: "import/paperless"}, Class: ClassExcluded, Origin: OriginExplicit},
|
||||
}
|
||||
for _, tier := range []CaptureTier{TierOffsite, TierSecondary} {
|
||||
cs := ComputeCaptureSet(binds, true, tier, drv, "")
|
||||
if got, want := absList(cs), []string{hdd("appdata/paperless/media")}; !reflect.DeepEqual(got, want) {
|
||||
t.Errorf("%s Paths = %v, want %v (excluded filtered)", tier, got, want)
|
||||
}
|
||||
if len(cs.Skipped) != 0 {
|
||||
t.Errorf("%s: excluded binds must NOT appear in Skipped, got %v", tier, cs.Skipped)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// --- Group D (Scenario D): structural guards + allowed bare-userdata + legit a..b name ---
|
||||
|
||||
func TestComputeCaptureSet_StructuralGuards(t *testing.T) {
|
||||
binds := []ClassifiedBind{
|
||||
{ComposeBind: ComposeBind{Root: RootHDD, RelPath: "../evil"}, Class: ClassMandatory, Origin: OriginDefaultWritable}, // d1 traversal
|
||||
{ComposeBind: ComposeBind{Root: RootHDD, RelPath: ""}, Class: ClassMandatory, Origin: OriginDefaultWritable}, // d2 bare hdd root
|
||||
{ComposeBind: ComposeBind{Root: RootHDD, RelPath: "backups/primary/x"}, Class: ClassMandatory, Origin: OriginDefaultWritable}, // d3 reserved zone
|
||||
{ComposeBind: ComposeBind{Root: RootUserdata, RelPath: ""}, Class: ClassMandatory, Origin: OriginDefaultWritable}, // d4 bare userdata — ALLOWED
|
||||
}
|
||||
cs := ComputeCaptureSet(binds, true, TierOffsite, drv, "")
|
||||
|
||||
// Paths: ONLY d4's userdata root — no escaped root, no backups/ anywhere.
|
||||
if got, want := absList(cs), []string{udat("")}; !reflect.DeepEqual(got, want) {
|
||||
t.Errorf("Paths = %v, want %v (only the allowed bare-userdata)", got, want)
|
||||
}
|
||||
for _, p := range cs.Paths {
|
||||
if p.Abs == "/mnt/evil" {
|
||||
t.Fatal("escaped-root Abs present in Paths — traversal guard failed")
|
||||
}
|
||||
if containsSeg(p.Abs, "backups") {
|
||||
t.Fatalf("reserved backups/ path present in Paths: %s", p.Abs)
|
||||
}
|
||||
}
|
||||
|
||||
// Skipped: d1,d2,d3 each with a DISTINCT reason naming its rule.
|
||||
reasons := map[string]string{} // "<root>/<rel>" -> reason
|
||||
for _, s := range cs.Skipped {
|
||||
reasons[string(s.Root)+"/"+s.RelPath] = s.Reason
|
||||
}
|
||||
if len(cs.Skipped) != 3 {
|
||||
t.Fatalf("want 3 skipped, got %d: %+v", len(cs.Skipped), cs.Skipped)
|
||||
}
|
||||
if reasons["hdd/../evil"] != reasonEscape {
|
||||
t.Errorf("../evil reason = %q, want %q", reasons["hdd/../evil"], reasonEscape)
|
||||
}
|
||||
if reasons["hdd/"] != reasonBareRoot {
|
||||
t.Errorf("bare-hdd reason = %q, want %q", reasons["hdd/"], reasonBareRoot)
|
||||
}
|
||||
if reasons["hdd/backups/primary/x"] != reasonReserved {
|
||||
t.Errorf("backups reason = %q, want %q", reasons["hdd/backups/primary/x"], reasonReserved)
|
||||
}
|
||||
// distinctness
|
||||
if reasonEscape == reasonBareRoot || reasonBareRoot == reasonReserved || reasonEscape == reasonReserved {
|
||||
t.Error("guard reasons are not distinct")
|
||||
}
|
||||
}
|
||||
|
||||
// TestComputeCaptureSet_LegitDotDotName: a component literally named "a..b" is NOT traversal.
|
||||
func TestComputeCaptureSet_LegitDotDotName(t *testing.T) {
|
||||
binds := []ClassifiedBind{
|
||||
{ComposeBind: ComposeBind{Root: RootHDD, RelPath: "appdata/a..b"}, Class: ClassMandatory, Origin: OriginExplicit},
|
||||
}
|
||||
cs := ComputeCaptureSet(binds, true, TierOffsite, drv, "")
|
||||
if got, want := absList(cs), []string{hdd("appdata/a..b")}; !reflect.DeepEqual(got, want) {
|
||||
t.Errorf("Paths = %v, want %v (a..b is a legit name, not traversal)", got, want)
|
||||
}
|
||||
if len(cs.Skipped) != 0 {
|
||||
t.Errorf("a..b must not be skipped, got %v", cs.Skipped)
|
||||
}
|
||||
}
|
||||
|
||||
// --- Group E (Scenario E): containment dedup + equal-Abs mandatory-wins + determinism ---
|
||||
|
||||
func TestComputeCaptureSet_ContainmentAndCollision(t *testing.T) {
|
||||
binds := []ClassifiedBind{
|
||||
{ComposeBind: ComposeBind{Root: RootHDD, RelPath: "appdata/paperless"}, Class: ClassMandatory, Origin: OriginExplicit},
|
||||
{ComposeBind: ComposeBind{Root: RootHDD, RelPath: "appdata/paperless/media"}, Class: ClassMandatory, Origin: OriginExplicit}, // descendant
|
||||
{ComposeBind: ComposeBind{Root: RootHDD, RelPath: "userdata/media"}, Class: ClassOptional, Origin: OriginExplicit}, // Abs collides with next
|
||||
{ComposeBind: ComposeBind{Root: RootUserdata, RelPath: "media"}, Class: ClassMandatory, Origin: OriginExplicit}, // same Abs, mandatory
|
||||
}
|
||||
cs := ComputeCaptureSet(binds, true, TierSecondary, drv, "")
|
||||
|
||||
want := []string{hdd("appdata/paperless"), udat("media")}
|
||||
if got := absList(cs); !reflect.DeepEqual(got, want) {
|
||||
t.Errorf("Paths = %v, want %v (descendant dropped; two spellings collapsed)", got, want)
|
||||
}
|
||||
// mandatory beats optional on the equal-Abs collision
|
||||
if c := classOfAbs(cs, udat("media")); c != ClassMandatory {
|
||||
t.Errorf("collapsed /userdata/media class = %q, want mandatory (mandatory must never degrade)", c)
|
||||
}
|
||||
|
||||
// determinism: recompute and compare full struct
|
||||
cs2 := ComputeCaptureSet(binds, true, TierSecondary, drv, "")
|
||||
if !reflect.DeepEqual(cs, cs2) {
|
||||
t.Error("ComputeCaptureSet is non-deterministic across runs")
|
||||
}
|
||||
}
|
||||
|
||||
// --- Group F (Scenario F): cross-app overlap advisory (pure) ---
|
||||
|
||||
func TestCrossAppOverlaps(t *testing.T) {
|
||||
X, Y, Z := "/mnt/drv/x", "/mnt/drv/y", "/mnt/drv/z"
|
||||
sets := map[string]CaptureSet{
|
||||
"appA": {HasClassification: true, Paths: []CapturePath{{Abs: X}, {Abs: Y}}},
|
||||
"appB": {HasClassification: true, Paths: []CapturePath{{Abs: Y}}},
|
||||
"appC": {HasClassification: true, Paths: []CapturePath{{Abs: Z}}},
|
||||
}
|
||||
got := CrossAppOverlaps(sets)
|
||||
want := []Overlap{{Abs: Y, Apps: []string{"appA", "appB"}}}
|
||||
if !reflect.DeepEqual(got, want) {
|
||||
t.Errorf("CrossAppOverlaps = %+v, want %+v", got, want)
|
||||
}
|
||||
|
||||
// exact-match only: cross-app CONTAINMENT is legitimate, must NOT report.
|
||||
cont := map[string]CaptureSet{
|
||||
"plex": {Paths: []CapturePath{{Abs: "/mnt/drv/userdata/media"}}},
|
||||
"calibre-web": {Paths: []CapturePath{{Abs: "/mnt/drv/userdata/media/books"}}},
|
||||
}
|
||||
if got := CrossAppOverlaps(cont); len(got) != 0 {
|
||||
t.Errorf("containment across apps must NOT report an overlap, got %+v", got)
|
||||
}
|
||||
|
||||
// empty input → empty (non-nil) slice, not a flaky nil
|
||||
if got := CrossAppOverlaps(map[string]CaptureSet{}); got == nil || len(got) != 0 {
|
||||
t.Errorf("empty input → empty non-nil slice, got %#v", got)
|
||||
}
|
||||
}
|
||||
|
||||
// containsSeg reports whether abs has seg as a path component (test helper).
|
||||
func containsSeg(abs, seg string) bool {
|
||||
for _, s := range splitSlash(abs) {
|
||||
if s == seg {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func splitSlash(s string) []string {
|
||||
var out []string
|
||||
cur := ""
|
||||
for _, r := range s {
|
||||
if r == '/' {
|
||||
out = append(out, cur)
|
||||
cur = ""
|
||||
continue
|
||||
}
|
||||
cur += string(r)
|
||||
}
|
||||
return append(out, cur)
|
||||
}
|
||||
@@ -1,235 +0,0 @@
|
||||
package appbackup
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"path"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Backup classification (referential coupling) — Task 2 of the backup-classification-redesign arc
|
||||
// (felhom.eu/documentation/audits/SPIKE-backup-classification-2026-07-14.md). This file is the SCHEMA
|
||||
// + PURE CLASSIFIER only; it is deliberately INERT — no backup tier consumes it yet. Task 3 (tier
|
||||
// policy engine) and Task 4 (manual .fab UI) are the consumers. Classes describe how a bind couples
|
||||
// to the app's referential state:
|
||||
//
|
||||
// - mandatory: COUPLED — restoring the app WITHOUT this bind yields a broken (not merely empty)
|
||||
// app, because the DB/state references the content (SQ3: immich DB-only restore = broken).
|
||||
// - optional: DECOUPLED-precious — absent ⇒ empty-not-broken, but the content is user-precious
|
||||
// (not re-downloadable): an external photo library, a curated comic/ROM set.
|
||||
// - excluded: DECOUPLED-bulk/transient — re-downloadable media, scraper caches, ingest inboxes,
|
||||
// transient export/download dirs; never shipped offsite, opt-in only for a manual .fab.
|
||||
|
||||
// BindClass is the referential-coupling class of a single host bind.
|
||||
type BindClass string
|
||||
|
||||
const (
|
||||
ClassMandatory BindClass = "mandatory" // COUPLED: restore-without is broken, not empty (SQ3)
|
||||
ClassOptional BindClass = "optional" // DECOUPLED-precious: empty-not-broken, not re-downloadable
|
||||
ClassExcluded BindClass = "excluded" // DECOUPLED-bulk/transient: never offsite, .fab opt-in
|
||||
)
|
||||
|
||||
// BindRoot names the deploy-time variable a bind's host path is relative to.
|
||||
type BindRoot string
|
||||
|
||||
const (
|
||||
RootUserdata BindRoot = "userdata" // relative to ${USERDATA_PATH}
|
||||
RootHDD BindRoot = "hdd" // relative to ${HDD_PATH}
|
||||
// RootImport is relative to ${IMPORT_PATH} — the CANONICAL drop-zone root (R-75). Unlike the
|
||||
// other two it does NOT resolve against the app's own drive: it lives on the system drive's
|
||||
// namespace, so every app's ingest folder is in one place. Resolvers therefore need the import
|
||||
// root passed in separately; they cannot derive it from hddPath.
|
||||
RootImport BindRoot = "import"
|
||||
)
|
||||
|
||||
// BackupSpec is the .felhom.yml `backup:` block. Paths are forward-slash, relative, path.Clean'd.
|
||||
type BackupSpec struct {
|
||||
Userdata []BindSpec `yaml:"userdata,omitempty" json:"userdata,omitempty"`
|
||||
HDD []BindSpec `yaml:"hdd,omitempty" json:"hdd,omitempty"`
|
||||
// Import classifies ${IMPORT_PATH}-relative binds (R-75). An app whose ingest bind moved from
|
||||
// ${USERDATA_PATH}/import/<app> to ${IMPORT_PATH}/<app> MUST move its backup entry here in the
|
||||
// same change: ValidateBackupSpec rejects an entry matching no compose bind, and the rejection is
|
||||
// WHOLE-BLOCK, so a stale `userdata: import/<app>` would discard the app's OTHER classifications
|
||||
// (e.g. an hdd appdata path classed mandatory) and silently degrade it to legacy.
|
||||
Import []BindSpec `yaml:"import,omitempty" json:"import,omitempty"`
|
||||
}
|
||||
|
||||
// BindSpec is one classified entry in a BackupSpec.
|
||||
type BindSpec struct {
|
||||
Path string `yaml:"path" json:"path"`
|
||||
Class BindClass `yaml:"class" json:"class"`
|
||||
}
|
||||
|
||||
// ComposeBind is a ${VAR}-relative host bind extracted from docker-compose.yml (Part 2 parser). It
|
||||
// lives in relative ${VAR} space (NOT resolved to an absolute path) and carries the :ro flag, both of
|
||||
// which the classifier needs — this is why the classifier does NOT reuse ParseComposeHDDMounts (which
|
||||
// resolves absolutes and drops the mode).
|
||||
type ComposeBind struct {
|
||||
Root BindRoot
|
||||
RelPath string // path.Clean'd, forward-slash, relative; "" for a bare-root bind (${VAR} itself)
|
||||
ReadOnly bool
|
||||
}
|
||||
|
||||
// ClassOrigin records HOW a bind's class was decided — for logs/UI and to prove the precedence rule.
|
||||
type ClassOrigin string
|
||||
|
||||
const (
|
||||
OriginExplicit ClassOrigin = "explicit" // matched an entry in the backup block
|
||||
OriginDefaultWritable ClassOrigin = "default_writable" // unlisted + writable → mandatory (capture)
|
||||
OriginDefaultRO ClassOrigin = "default_ro" // unlisted + :ro → excluded (reader rule)
|
||||
OriginLegacy ClassOrigin = "legacy" // no backup block at all → no class semantics
|
||||
)
|
||||
|
||||
// ClassifiedBind pairs a compose bind with its resolved class + origin.
|
||||
type ClassifiedBind struct {
|
||||
ComposeBind
|
||||
Class BindClass
|
||||
Origin ClassOrigin
|
||||
}
|
||||
|
||||
// validClass reports whether c is one of the three known classes (empty is INVALID — a typoed
|
||||
// `clas:` key makes yaml.v3 silently leave Class "", which must be rejected, not defaulted).
|
||||
func validClass(c BindClass) bool {
|
||||
switch c {
|
||||
case ClassMandatory, ClassOptional, ClassExcluded:
|
||||
return true
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
// ValidateRelPath is THE path-safety refusal set for every ${VAR}-relative catalog path — the
|
||||
// `backup:` block and `data_paths:` both run through it, so there is exactly ONE definition of what
|
||||
// a safe relative path is. Refuses: empty, backslash, absolute, non-path.Clean'd, and any leading
|
||||
// ".." escape. It deliberately does NOT check "matches a compose bind" — that rule needs the bind
|
||||
// list and differs per caller (whole-block reject for backup:, per-entry for data_paths:).
|
||||
func ValidateRelPath(root BindRoot, p string) error {
|
||||
where := fmt.Sprintf("%s[%q]", root, p)
|
||||
if p == "" {
|
||||
return fmt.Errorf("%s: empty path", where)
|
||||
}
|
||||
if strings.ContainsRune(p, '\\') {
|
||||
return fmt.Errorf("%s: backslash in path (paths are forward-slash relative)", where)
|
||||
}
|
||||
if path.IsAbs(p) {
|
||||
return fmt.Errorf("%s: absolute path (must be relative to the %s root)", where, root)
|
||||
}
|
||||
if p != path.Clean(p) {
|
||||
return fmt.Errorf("%s: non-clean path (want %q)", where, path.Clean(p))
|
||||
}
|
||||
// path.Clean has run — ".." can only survive as a leading "../" segment.
|
||||
if p == ".." || strings.HasPrefix(p, "../") {
|
||||
return fmt.Errorf("%s: path escapes the root (..)", where)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// ValidRoot reports whether r is one of the three known bind roots.
|
||||
func ValidRoot(r BindRoot) bool {
|
||||
switch r {
|
||||
case RootUserdata, RootHDD, RootImport:
|
||||
return true
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
// ValidateBackupSpec checks a parsed backup block against the app's actual compose binds and returns
|
||||
// the FIRST defect (whole-block semantics — the caller rejects the ENTIRE block on any error, so the
|
||||
// app degrades to legacy rather than partially classifying). A nil spec is vacuously valid (legacy).
|
||||
//
|
||||
// Rejects: unknown/empty class; empty path; a path that is not already path.Clean'd, or is absolute,
|
||||
// or contains "..", or contains a backslash; a duplicate (root, path); an entry whose (root, path)
|
||||
// matches NO compose bind (a typo/stale entry must not silently shift the real bind onto the
|
||||
// mandatory default). Match is exact (Root, RelPath) equality.
|
||||
func ValidateBackupSpec(spec *BackupSpec, binds []ComposeBind) error {
|
||||
if spec == nil {
|
||||
return nil
|
||||
}
|
||||
present := make(map[BindRoot]map[string]bool)
|
||||
for _, b := range binds {
|
||||
if present[b.Root] == nil {
|
||||
present[b.Root] = make(map[string]bool)
|
||||
}
|
||||
present[b.Root][b.RelPath] = true
|
||||
}
|
||||
|
||||
seen := make(map[string]bool) // "<root>\x00<path>"
|
||||
check := func(root BindRoot, list []BindSpec) error {
|
||||
for _, e := range list {
|
||||
where := fmt.Sprintf("%s[%q]", root, e.Path)
|
||||
if !validClass(e.Class) {
|
||||
return fmt.Errorf("%s: invalid class %q (want mandatory|optional|excluded)", where, e.Class)
|
||||
}
|
||||
if err := ValidateRelPath(root, e.Path); err != nil {
|
||||
return err
|
||||
}
|
||||
key := string(root) + "\x00" + e.Path
|
||||
if seen[key] {
|
||||
return fmt.Errorf("%s: duplicate path in the backup block", where)
|
||||
}
|
||||
seen[key] = true
|
||||
if !present[root][e.Path] {
|
||||
return fmt.Errorf("%s: matches no compose bind (stale or typoed path)", where)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
if err := check(RootUserdata, spec.Userdata); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := check(RootHDD, spec.HDD); err != nil {
|
||||
return err
|
||||
}
|
||||
return check(RootImport, spec.Import)
|
||||
}
|
||||
|
||||
// ClassifyBinds resolves every compose bind to a class + origin, applying the two-level default. The
|
||||
// second return reports whether the app carries a backup block at all.
|
||||
//
|
||||
// - spec == nil → every bind is emitted with Origin=legacy and an EMPTY Class (no class semantics),
|
||||
// and hasClassification=false. This is the block-ABSENT branch: nothing downstream may change
|
||||
// behavior for it (SQ5 two-level default — no block means today's per-tier legacy behavior).
|
||||
// - spec present → an explicit block entry ALWAYS wins, regardless of the bind's :ro flag (an
|
||||
// explicit `optional` on immich's :ro external library beats the reader default). An UNLISTED
|
||||
// bind defaults by mode: writable → mandatory (default_writable — the C6B-F1 direction: capture
|
||||
// rather than silently drop), read-only → excluded (default_ro — reader rule, SQ2).
|
||||
//
|
||||
// Pure. Assumes a validated spec (see ValidateBackupSpec) but never panics on an unvalidated one:
|
||||
// unmatched/invalid spec entries simply don't match any bind here.
|
||||
//
|
||||
// A bare-root bind (RelPath "") can never be matched by an explicit entry — an empty path is invalid
|
||||
// in the spec — so it always falls to the ro/writable default.
|
||||
func ClassifyBinds(spec *BackupSpec, binds []ComposeBind) (classified []ClassifiedBind, hasClassification bool) {
|
||||
out := make([]ClassifiedBind, 0, len(binds))
|
||||
if spec == nil {
|
||||
for _, b := range binds {
|
||||
out = append(out, ClassifiedBind{ComposeBind: b, Origin: OriginLegacy})
|
||||
}
|
||||
return out, false
|
||||
}
|
||||
explicit := make(map[BindRoot]map[string]BindClass)
|
||||
add := func(root BindRoot, list []BindSpec) {
|
||||
for _, e := range list {
|
||||
if explicit[root] == nil {
|
||||
explicit[root] = make(map[string]BindClass)
|
||||
}
|
||||
explicit[root][e.Path] = e.Class
|
||||
}
|
||||
}
|
||||
add(RootUserdata, spec.Userdata)
|
||||
add(RootHDD, spec.HDD)
|
||||
add(RootImport, spec.Import)
|
||||
|
||||
for _, b := range binds {
|
||||
cb := ClassifiedBind{ComposeBind: b}
|
||||
if cls, ok := explicit[b.Root][b.RelPath]; ok {
|
||||
cb.Class, cb.Origin = cls, OriginExplicit
|
||||
} else if b.ReadOnly {
|
||||
cb.Class, cb.Origin = ClassExcluded, OriginDefaultRO
|
||||
} else {
|
||||
cb.Class, cb.Origin = ClassMandatory, OriginDefaultWritable
|
||||
}
|
||||
out = append(out, cb)
|
||||
}
|
||||
return out, true
|
||||
}
|
||||
@@ -1,165 +0,0 @@
|
||||
package appbackup
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// classOf finds the resolved class+origin for a (root, relpath) in a ClassifiedBind slice.
|
||||
func classOf(cbs []ClassifiedBind, root BindRoot, rel string) (BindClass, ClassOrigin, bool) {
|
||||
for _, c := range cbs {
|
||||
if c.Root == root && c.RelPath == rel {
|
||||
return c.Class, c.Origin, true
|
||||
}
|
||||
}
|
||||
return "", "", false
|
||||
}
|
||||
|
||||
// --- Group A: classifier ---
|
||||
|
||||
// TestClassify_ImmichShape is Scenario A: explicit classes resolve, and an EXPLICIT entry beats the
|
||||
// :ro reader-default (media/photos is :ro but ruled optional). Companion RP-2: making the ro-default
|
||||
// override explicit entries forces media/photos to excluded and fails the optional assertion.
|
||||
func TestClassify_ImmichShape(t *testing.T) {
|
||||
binds := []ComposeBind{
|
||||
{Root: RootHDD, RelPath: "appdata/immich", ReadOnly: false},
|
||||
{Root: RootUserdata, RelPath: "media/photos", ReadOnly: true}, // :ro external library
|
||||
}
|
||||
spec := &BackupSpec{
|
||||
HDD: []BindSpec{{Path: "appdata/immich", Class: ClassMandatory}},
|
||||
Userdata: []BindSpec{{Path: "media/photos", Class: ClassOptional}},
|
||||
}
|
||||
cbs, has := ClassifyBinds(spec, binds)
|
||||
if !has {
|
||||
t.Fatal("hasClassification should be true with a spec present")
|
||||
}
|
||||
if cls, org, ok := classOf(cbs, RootHDD, "appdata/immich"); !ok || cls != ClassMandatory || org != OriginExplicit {
|
||||
t.Errorf("appdata/immich = %v/%v, want mandatory/explicit", cls, org)
|
||||
}
|
||||
// The crux: an explicit optional beats the :ro default_ro that would otherwise force excluded.
|
||||
if cls, org, ok := classOf(cbs, RootUserdata, "media/photos"); !ok || cls != ClassOptional || org != OriginExplicit {
|
||||
t.Errorf("media/photos (:ro, explicit optional) = %v/%v, want optional/explicit (explicit beats ro-default)", cls, org)
|
||||
}
|
||||
}
|
||||
|
||||
// TestClassify_TwoLevelDefault is Scenario B: with a block PRESENT, an unlisted writable bind
|
||||
// defaults mandatory (capture, the C6B-F1 direction) and an unlisted :ro bind defaults excluded
|
||||
// (reader rule). Companion RP-3: flipping the unlisted-writable default to excluded fails the
|
||||
// mandatory assertion.
|
||||
func TestClassify_TwoLevelDefault(t *testing.T) {
|
||||
binds := []ComposeBind{
|
||||
{Root: RootHDD, RelPath: "appdata/app", ReadOnly: false}, // listed
|
||||
{Root: RootUserdata, RelPath: "data/extra", ReadOnly: false}, // UNLISTED writable
|
||||
{Root: RootUserdata, RelPath: "media/ro", ReadOnly: true}, // UNLISTED :ro
|
||||
}
|
||||
spec := &BackupSpec{HDD: []BindSpec{{Path: "appdata/app", Class: ClassMandatory}}}
|
||||
cbs, has := ClassifyBinds(spec, binds)
|
||||
if !has {
|
||||
t.Fatal("hasClassification should be true")
|
||||
}
|
||||
if cls, org, _ := classOf(cbs, RootUserdata, "data/extra"); cls != ClassMandatory || org != OriginDefaultWritable {
|
||||
t.Errorf("unlisted writable = %v/%v, want mandatory/default_writable (capture direction)", cls, org)
|
||||
}
|
||||
if cls, org, _ := classOf(cbs, RootUserdata, "media/ro"); cls != ClassExcluded || org != OriginDefaultRO {
|
||||
t.Errorf("unlisted :ro = %v/%v, want excluded/default_ro (reader rule)", cls, org)
|
||||
}
|
||||
}
|
||||
|
||||
// TestClassify_NilSpecLegacy is Scenario C: a nil spec → every bind is legacy with no class, and
|
||||
// hasClassification=false. This is the inertness gate at the classifier level.
|
||||
func TestClassify_NilSpecLegacy(t *testing.T) {
|
||||
binds := []ComposeBind{{Root: RootUserdata, RelPath: "media", ReadOnly: true}}
|
||||
cbs, has := ClassifyBinds(nil, binds)
|
||||
if has {
|
||||
t.Error("nil spec must report hasClassification=false")
|
||||
}
|
||||
if len(cbs) != 1 || cbs[0].Origin != OriginLegacy || cbs[0].Class != "" {
|
||||
t.Errorf("nil-spec bind = %+v, want origin=legacy, empty class", cbs[0])
|
||||
}
|
||||
}
|
||||
|
||||
// TestClassify_BareRootFallsToDefault: a bare-root bind (RelPath "") can't be matched by any explicit
|
||||
// entry (empty paths are invalid), so it falls to the ro/writable default.
|
||||
func TestClassify_BareRootFallsToDefault(t *testing.T) {
|
||||
spec := &BackupSpec{Userdata: []BindSpec{{Path: "media/x", Class: ClassOptional}}}
|
||||
cbs, _ := ClassifyBinds(spec, []ComposeBind{
|
||||
{Root: RootUserdata, RelPath: "", ReadOnly: false}, // bare ${USERDATA_PATH}
|
||||
{Root: RootHDD, RelPath: "", ReadOnly: true}, // bare ${HDD_PATH} :ro
|
||||
})
|
||||
if cls, org, _ := classOf(cbs, RootUserdata, ""); cls != ClassMandatory || org != OriginDefaultWritable {
|
||||
t.Errorf("bare writable root = %v/%v, want mandatory/default_writable", cls, org)
|
||||
}
|
||||
if cls, org, _ := classOf(cbs, RootHDD, ""); cls != ClassExcluded || org != OriginDefaultRO {
|
||||
t.Errorf("bare :ro root = %v/%v, want excluded/default_ro", cls, org)
|
||||
}
|
||||
}
|
||||
|
||||
// TestClassify_SameRelPathBothRoots: userdata/x and hdd/x are DISTINCT binds — Root is part of
|
||||
// identity, so an explicit hdd entry must not classify the userdata bind.
|
||||
func TestClassify_SameRelPathBothRoots(t *testing.T) {
|
||||
binds := []ComposeBind{
|
||||
{Root: RootUserdata, RelPath: "shared", ReadOnly: false},
|
||||
{Root: RootHDD, RelPath: "shared", ReadOnly: false},
|
||||
}
|
||||
spec := &BackupSpec{HDD: []BindSpec{{Path: "shared", Class: ClassExcluded}}}
|
||||
cbs, _ := ClassifyBinds(spec, binds)
|
||||
if cls, org, _ := classOf(cbs, RootHDD, "shared"); cls != ClassExcluded || org != OriginExplicit {
|
||||
t.Errorf("hdd/shared = %v/%v, want excluded/explicit", cls, org)
|
||||
}
|
||||
if cls, org, _ := classOf(cbs, RootUserdata, "shared"); cls != ClassMandatory || org != OriginDefaultWritable {
|
||||
t.Errorf("userdata/shared = %v/%v, want mandatory/default_writable (hdd entry must NOT match it)", cls, org)
|
||||
}
|
||||
}
|
||||
|
||||
// --- Group B: validation (Scenario D) — every defect rejects the WHOLE block; error names the entry ---
|
||||
|
||||
func TestValidateBackupSpec_Defects(t *testing.T) {
|
||||
// The compose binds the valid entries reference (so only the seeded defect is the failure).
|
||||
binds := []ComposeBind{
|
||||
{Root: RootUserdata, RelPath: "media/tv"},
|
||||
{Root: RootHDD, RelPath: "appdata/x"},
|
||||
}
|
||||
cases := []struct {
|
||||
name string
|
||||
spec *BackupSpec
|
||||
wantFrag string // substring the error must contain (the offending entry / rule)
|
||||
}{
|
||||
{"unknown class", &BackupSpec{Userdata: []BindSpec{{Path: "media/tv", Class: "keepit"}}}, "invalid class"},
|
||||
{"empty class (typoed key)", &BackupSpec{Userdata: []BindSpec{{Path: "media/tv", Class: ""}}}, "invalid class"},
|
||||
{"empty path", &BackupSpec{HDD: []BindSpec{{Path: "", Class: ClassMandatory}}}, "empty path"},
|
||||
{"absolute path", &BackupSpec{HDD: []BindSpec{{Path: "/etc/x", Class: ClassMandatory}}}, "absolute"},
|
||||
{"dotdot path", &BackupSpec{HDD: []BindSpec{{Path: "../escape", Class: ClassMandatory}}}, "escapes"},
|
||||
{"backslash path", &BackupSpec{HDD: []BindSpec{{Path: "appdata\\x", Class: ClassMandatory}}}, "backslash"},
|
||||
{"non-clean path", &BackupSpec{HDD: []BindSpec{{Path: "appdata/./x", Class: ClassMandatory}}}, "non-clean"},
|
||||
{"duplicate path", &BackupSpec{HDD: []BindSpec{
|
||||
{Path: "appdata/x", Class: ClassMandatory}, {Path: "appdata/x", Class: ClassExcluded},
|
||||
}}, "duplicate"},
|
||||
{"no matching bind (typo)", &BackupSpec{Userdata: []BindSpec{{Path: "media/tvv", Class: ClassExcluded}}}, "matches no compose bind"},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
err := ValidateBackupSpec(tc.spec, binds)
|
||||
if err == nil {
|
||||
t.Fatalf("expected rejection, got nil")
|
||||
}
|
||||
if !strings.Contains(err.Error(), tc.wantFrag) {
|
||||
t.Errorf("error %q must contain %q", err.Error(), tc.wantFrag)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestValidateBackupSpec_ValidAndNil: a clean block validates, and a nil spec is vacuously valid.
|
||||
func TestValidateBackupSpec_ValidAndNil(t *testing.T) {
|
||||
binds := []ComposeBind{{Root: RootHDD, RelPath: "appdata/x"}, {Root: RootUserdata, RelPath: "media/tv"}}
|
||||
spec := &BackupSpec{
|
||||
HDD: []BindSpec{{Path: "appdata/x", Class: ClassMandatory}},
|
||||
Userdata: []BindSpec{{Path: "media/tv", Class: ClassExcluded}},
|
||||
}
|
||||
if err := ValidateBackupSpec(spec, binds); err != nil {
|
||||
t.Errorf("clean block should validate: %v", err)
|
||||
}
|
||||
if err := ValidateBackupSpec(nil, binds); err != nil {
|
||||
t.Errorf("nil spec must be vacuously valid: %v", err)
|
||||
}
|
||||
}
|
||||
@@ -2,7 +2,6 @@ package appbackup
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"compress/gzip"
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
@@ -51,20 +50,6 @@ type DumpValidation struct {
|
||||
Error string
|
||||
FileSize int64
|
||||
ModTime time.Time
|
||||
// R-44 (v0.148.0) content sniff — a WARN-LEVEL signal, never a gate.
|
||||
//
|
||||
// Structural validity says nothing about whether a dump holds the customer's data. The immich
|
||||
// dump of 2026-07-19 was 52MB, had a valid header and 60+ CREATE TABLEs, and contained zero
|
||||
// users and zero assets: its whole bulk was the geodata reference tables immich ships. Size and
|
||||
// table count are therefore both useless as emptiness heuristics — but an accounts table with
|
||||
// no rows is a strong, cheap, app-agnostic hint that a dump predates the customer entirely.
|
||||
//
|
||||
// Deliberately NOT a refusal: plenty of legitimate apps have no users table (UserTableFound
|
||||
// false → inconclusive → silent), and a false positive that blocked a restore would be far
|
||||
// worse than the skew it guards against. The restore confirm shows it as one extra line.
|
||||
UserTableFound bool
|
||||
UserRows int
|
||||
LooksEmpty bool // UserTableFound && UserRows == 0
|
||||
}
|
||||
|
||||
// DumpFileInfo holds info about a dump file on disk.
|
||||
@@ -78,16 +63,7 @@ type DumpFileInfo struct {
|
||||
}
|
||||
|
||||
// DiscoverDatabases finds running database containers via docker ps.
|
||||
//
|
||||
// knownStacks is the set of actually-deployed stack names; it is used to attribute each DB container to
|
||||
// the correct stack (M19). Pass nil/empty for the legacy suffix-strip behaviour.
|
||||
func DiscoverDatabases(ctx context.Context, logger *log.Logger, debug bool, knownStacks []string) ([]DiscoveredDB, error) {
|
||||
known := make(map[string]bool, len(knownStacks))
|
||||
for _, s := range knownStacks {
|
||||
if s != "" {
|
||||
known[s] = true
|
||||
}
|
||||
}
|
||||
func DiscoverDatabases(ctx context.Context, logger *log.Logger, debug bool) ([]DiscoveredDB, error) {
|
||||
if debug {
|
||||
logger.Printf("[DEBUG] DiscoverDatabases: running docker ps to find database containers")
|
||||
}
|
||||
@@ -115,10 +91,12 @@ func DiscoverDatabases(ctx context.Context, logger *log.Logger, debug bool, know
|
||||
|
||||
id, name, image := parts[0], parts[1], strings.ToLower(parts[2])
|
||||
|
||||
// R-47: the same predicate that DBServiceNames applies to compose `image:` values, so a dump
|
||||
// that exists is always attributable to a startable service (see dbservices.go).
|
||||
dbType, isDB := dbTypeForImage(image)
|
||||
if !isDB {
|
||||
var dbType DBType
|
||||
if strings.Contains(image, "postgres") {
|
||||
dbType = DBTypePostgres
|
||||
} else if strings.Contains(image, "mariadb") || strings.Contains(image, "mysql") {
|
||||
dbType = DBTypeMariaDB
|
||||
} else {
|
||||
if debug {
|
||||
logger.Printf("[DEBUG] DiscoverDatabases: skipping container %s (image=%s, not a database)", name, image)
|
||||
}
|
||||
@@ -134,7 +112,7 @@ func DiscoverDatabases(ctx context.Context, logger *log.Logger, debug bool, know
|
||||
ContainerID: id,
|
||||
ContainerName: name,
|
||||
DBType: dbType,
|
||||
StackName: deriveStackName(name, known),
|
||||
StackName: deriveStackName(name),
|
||||
}
|
||||
|
||||
// Get env vars from container
|
||||
@@ -375,9 +353,6 @@ func ValidateDump(filePath string, dbType DBType) DumpValidation {
|
||||
lineNum := 0
|
||||
headerFound := false
|
||||
tableCount := 0
|
||||
// R-44 sniff state. inUserCopy tracks a postgres `COPY … FROM stdin;` block for an accounts
|
||||
// table; rows are counted until the `\.` terminator.
|
||||
inUserCopy := false
|
||||
for {
|
||||
lineBytes, isPrefix, err := reader.ReadLine()
|
||||
if err != nil {
|
||||
@@ -389,12 +364,7 @@ func ValidateDump(filePath string, dbType DBType) DumpValidation {
|
||||
break // EOF
|
||||
}
|
||||
if isPrefix {
|
||||
// Line exceeds buffer — skip remainder (COPY data, large INSERTs).
|
||||
// A long line inside a user COPY block is still a ROW: count it before discarding it,
|
||||
// or a table whose rows happen to be wide would sniff as empty and raise a false alarm.
|
||||
if inUserCopy {
|
||||
v.UserRows++
|
||||
}
|
||||
// Line exceeds buffer — skip remainder (COPY data, large INSERTs)
|
||||
for isPrefix && err == nil {
|
||||
_, isPrefix, err = reader.ReadLine()
|
||||
}
|
||||
@@ -404,23 +374,6 @@ func ValidateDump(filePath string, dbType DBType) DumpValidation {
|
||||
line := string(lineBytes)
|
||||
lineNum++
|
||||
|
||||
// R-44 content sniff (warn-level; see DumpValidation).
|
||||
if inUserCopy {
|
||||
if line == `\.` {
|
||||
inUserCopy = false
|
||||
} else {
|
||||
v.UserRows++
|
||||
}
|
||||
} else if isUserCopyStart(line, dbType) {
|
||||
inUserCopy = true
|
||||
v.UserTableFound = true
|
||||
} else if dbType == DBTypeMariaDB && isUserInsert(line) {
|
||||
// mysqldump writes multi-row `INSERT INTO \`users\` VALUES (…),(…);` — the row count is
|
||||
// not worth parsing out of it, and presence alone answers the only question asked here.
|
||||
v.UserTableFound = true
|
||||
v.UserRows++
|
||||
}
|
||||
|
||||
// Header check — scan first 10 lines for expected dump header
|
||||
// MariaDB 11.4+ prepends a sandbox comment before the header line
|
||||
if lineNum <= 10 && !headerFound {
|
||||
@@ -464,72 +417,12 @@ func ValidateDump(filePath string, dbType DBType) DumpValidation {
|
||||
return v
|
||||
}
|
||||
|
||||
v.LooksEmpty = v.UserTableFound && v.UserRows == 0
|
||||
if v.LooksEmpty {
|
||||
log.Printf("[WARN] [backup] ValidateDump: %s is structurally valid (%d tables) but its accounts table has NO rows — the dump may predate the customer's data", filePath, tableCount)
|
||||
}
|
||||
|
||||
v.Valid = true
|
||||
return v
|
||||
}
|
||||
|
||||
// userTableNames are the table names treated as "the accounts table" by the R-44 sniff. Kept
|
||||
// deliberately short: a wider net (anything containing "user") would match join/audit tables like
|
||||
// `user_metadata` or `album_user`, which are legitimately empty on a healthy single-user install
|
||||
// and would produce exactly the false alarm this signal must not raise.
|
||||
var userTableNames = []string{"user", "users", "account", "accounts"}
|
||||
|
||||
// isUserCopyStart reports whether a line opens a postgres `COPY <accounts-table> … FROM stdin;`
|
||||
// block. pg_dump writes the table qualified and optionally quoted — `COPY public."user" (…)`,
|
||||
// `COPY public.users (…)` — so both forms are matched.
|
||||
func isUserCopyStart(line string, dbType DBType) bool {
|
||||
if dbType != DBTypePostgres || !strings.HasPrefix(line, "COPY ") {
|
||||
return false
|
||||
}
|
||||
rest := strings.TrimPrefix(line, "COPY ")
|
||||
sp := strings.IndexByte(rest, ' ')
|
||||
if sp < 0 {
|
||||
return false
|
||||
}
|
||||
return matchesUserTable(rest[:sp])
|
||||
}
|
||||
|
||||
// isUserInsert reports whether a line is a mysqldump INSERT into an accounts table.
|
||||
func isUserInsert(line string) bool {
|
||||
const pfx = "INSERT INTO "
|
||||
if !strings.HasPrefix(line, pfx) {
|
||||
return false
|
||||
}
|
||||
rest := strings.TrimPrefix(line, pfx)
|
||||
sp := strings.IndexByte(rest, ' ')
|
||||
if sp < 0 {
|
||||
return false
|
||||
}
|
||||
return matchesUserTable(rest[:sp])
|
||||
}
|
||||
|
||||
// matchesUserTable strips schema qualification and quoting from a dumped table reference and
|
||||
// reports whether the bare name is an accounts table.
|
||||
func matchesUserTable(ref string) bool {
|
||||
if dot := strings.LastIndexByte(ref, '.'); dot >= 0 {
|
||||
ref = ref[dot+1:]
|
||||
}
|
||||
ref = strings.Trim(ref, "\"`")
|
||||
for _, n := range userTableNames {
|
||||
if strings.EqualFold(ref, n) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// ListDumpFiles returns info about SQL dump files on disk.
|
||||
//
|
||||
// M18: ValidateDump scans the dump line-by-line; on a customer with hundreds-of-MB dumps that is wasted
|
||||
// disk I/O + CPU on every ~5-min scheduler cycle. `cached` is an optional lookup that returns a
|
||||
// previously-computed DumpValidation for a file whose (name, size, modtime) match — when it returns ok,
|
||||
// the expensive ValidateDump is skipped. Pass nil to always validate (legacy fast path / other callers).
|
||||
func ListDumpFiles(dumpDir string, cached func(name string, size int64, mod time.Time) (DumpValidation, bool)) ([]DumpFileInfo, error) {
|
||||
func ListDumpFiles(dumpDir string) ([]DumpFileInfo, error) {
|
||||
entries, err := os.ReadDir(dumpDir)
|
||||
if err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
@@ -571,14 +464,7 @@ func ListDumpFiles(dumpDir string, cached func(name string, size int64, mod time
|
||||
f.StackName = base
|
||||
}
|
||||
|
||||
// M18: reuse a cached validation when the file is unchanged (name+size+modtime), else validate.
|
||||
if cached != nil {
|
||||
if v, ok := cached(e.Name(), info.Size(), info.ModTime()); ok {
|
||||
f.Validation = v
|
||||
files = append(files, f)
|
||||
continue
|
||||
}
|
||||
}
|
||||
// Run validation on the file
|
||||
fullPath := filepath.Join(dumpDir, e.Name())
|
||||
f.Validation = ValidateDump(fullPath, f.DBType)
|
||||
|
||||
@@ -627,113 +513,6 @@ func populateDBEnv(ctx context.Context, db *DiscoveredDB) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// ImportDump replays a (possibly gzipped) SQL dump into a RUNNING database container — the read-side
|
||||
// counterpart to DumpOne (F17). It reuses the per-engine clients (psql / mariadb) and the DiscoveredDB's
|
||||
// OWN credentials (discovered from the live container env), so the caller needs no external env map. The
|
||||
// container must already be running (the restore flow brings the stack up first); ImportDump briefly
|
||||
// waits for the engine to accept connections, then pipes the dump in. The backup dumps are produced with
|
||||
// DROP/CREATE (pg_dump --clean --if-exists; mariadb-dump's default --add-drop-table), so a replay fully
|
||||
// reconstructs the captured logical state.
|
||||
func ImportDump(ctx context.Context, db DiscoveredDB, dumpPath string, logger *log.Logger, debug bool) error {
|
||||
if err := waitDBReady(ctx, db, 30*time.Second); err != nil {
|
||||
return fmt.Errorf("waiting for %s (%s) readiness: %w", db.ContainerName, db.DBType, err)
|
||||
}
|
||||
|
||||
f, err := os.Open(dumpPath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("opening dump %s: %w", dumpPath, err)
|
||||
}
|
||||
defer f.Close()
|
||||
|
||||
var reader io.Reader = f
|
||||
if strings.HasSuffix(dumpPath, ".gz") {
|
||||
gr, err := gzip.NewReader(f)
|
||||
if err != nil {
|
||||
return fmt.Errorf("opening gzip %s: %w", dumpPath, err)
|
||||
}
|
||||
defer gr.Close()
|
||||
reader = gr
|
||||
}
|
||||
|
||||
impCtx, cancel := context.WithTimeout(ctx, 30*time.Minute)
|
||||
defer cancel()
|
||||
|
||||
var cmd *exec.Cmd
|
||||
switch db.DBType {
|
||||
case DBTypePostgres:
|
||||
user := db.DBUser
|
||||
if user == "" {
|
||||
user = "postgres"
|
||||
}
|
||||
dbName := db.DBName
|
||||
if dbName == "" {
|
||||
dbName = user
|
||||
}
|
||||
// ON_ERROR_STOP=1: a real import error must FAIL (and surface), not silently half-apply.
|
||||
cmd = exec.CommandContext(impCtx, "docker", "exec", "-i", db.ContainerID,
|
||||
"psql", "-v", "ON_ERROR_STOP=1", "-U", user, "-d", dbName)
|
||||
case DBTypeMariaDB:
|
||||
password := getMariaDBPassword(impCtx, db.ContainerID)
|
||||
if password == "" {
|
||||
return fmt.Errorf("could not determine MariaDB root password for %s", db.ContainerName)
|
||||
}
|
||||
cmd = exec.CommandContext(impCtx, "docker", "exec", "-i", db.ContainerID,
|
||||
"mariadb", "-u", "root", "-p"+password, db.DBName)
|
||||
default:
|
||||
return fmt.Errorf("unsupported DB type: %s", db.DBType)
|
||||
}
|
||||
|
||||
cmd.Stdin = reader
|
||||
var stderr strings.Builder
|
||||
cmd.Stderr = &stderr
|
||||
if debug && logger != nil {
|
||||
logger.Printf("[DEBUG] [backup] ImportDump: importing %s into %s (%s)", dumpPath, db.ContainerName, db.DBType)
|
||||
}
|
||||
if err := cmd.Run(); err != nil {
|
||||
msg := strings.TrimSpace(stderr.String())
|
||||
if len(msg) > 300 {
|
||||
msg = msg[:300]
|
||||
}
|
||||
return fmt.Errorf("%s import into %s failed: %s — %w", db.DBType, db.ContainerName, msg, err)
|
||||
}
|
||||
if logger != nil {
|
||||
logger.Printf("[INFO] [backup] Imported DB dump %s into %s (%s)", filepath.Base(dumpPath), db.ContainerName, db.DBType)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// waitDBReady polls until the database accepts connections (pg_isready / mariadb-admin ping).
|
||||
func waitDBReady(ctx context.Context, db DiscoveredDB, timeout time.Duration) error {
|
||||
deadline := time.Now().Add(timeout)
|
||||
for {
|
||||
c, cancel := context.WithTimeout(ctx, 5*time.Second)
|
||||
var cmd *exec.Cmd
|
||||
switch db.DBType {
|
||||
case DBTypePostgres:
|
||||
user := db.DBUser
|
||||
if user == "" {
|
||||
user = "postgres"
|
||||
}
|
||||
cmd = exec.CommandContext(c, "docker", "exec", db.ContainerID, "pg_isready", "-U", user)
|
||||
case DBTypeMariaDB:
|
||||
pw := getMariaDBPassword(c, db.ContainerID)
|
||||
cmd = exec.CommandContext(c, "docker", "exec", db.ContainerID, "mariadb-admin", "ping", "-u", "root", "-p"+pw)
|
||||
default:
|
||||
cancel()
|
||||
return fmt.Errorf("unsupported DB type: %s", db.DBType)
|
||||
}
|
||||
err := cmd.Run()
|
||||
cancel()
|
||||
if err == nil {
|
||||
return nil
|
||||
}
|
||||
if time.Now().After(deadline) {
|
||||
return fmt.Errorf("timeout after %s", timeout)
|
||||
}
|
||||
time.Sleep(2 * time.Second)
|
||||
}
|
||||
}
|
||||
|
||||
func getMariaDBPassword(ctx context.Context, containerID string) string {
|
||||
cmd := exec.CommandContext(ctx, "docker", "inspect", containerID,
|
||||
"--format", "{{range .Config.Env}}{{println .}}{{end}}")
|
||||
@@ -753,52 +532,8 @@ func getMariaDBPassword(ctx context.Context, containerID string) string {
|
||||
return ""
|
||||
}
|
||||
|
||||
// deriveStackName maps a DB container name to its owning stack name.
|
||||
//
|
||||
// M19: the old logic pure-suffix-stripped on `-` (postgres/db/mariadb/mysql/database/redis/cache),
|
||||
// which misattributes a stack whose real slug ENDS in a role token (e.g. a stack literally named
|
||||
// `my-cache` → stripped to `my`). When the set of actually-deployed stack names is known, cross-reference
|
||||
// it so the result is a real stack:
|
||||
// - candidate := suffix-strip result.
|
||||
// - known[candidate] → candidate (a real DB-role suffix of a real stack, e.g. romm-postgres→romm).
|
||||
// - else known[containerName] → containerName (the container name IS the stack — don't strip, e.g. my-cache).
|
||||
// - else longest known prefix → handles <stack>_postgres / <stack>-1 / compose-suffixed names.
|
||||
// - else → candidate (fall back to today's suffix-strip; preserves behaviour when
|
||||
// the stack list is empty/unavailable, so nothing regresses).
|
||||
//
|
||||
// A nil/empty `known` map = the legacy fast path (pure suffix-strip).
|
||||
func deriveStackName(containerName string, known map[string]bool) string {
|
||||
candidate := suffixStripStackName(containerName)
|
||||
|
||||
if len(known) == 0 {
|
||||
return candidate
|
||||
}
|
||||
if known[candidate] {
|
||||
return candidate
|
||||
}
|
||||
if known[containerName] {
|
||||
return containerName
|
||||
}
|
||||
// Longest known stack name that is a prefix of the container name (tie-break: longest wins).
|
||||
best := ""
|
||||
for name := range known {
|
||||
if name == "" || len(name) >= len(containerName) {
|
||||
continue
|
||||
}
|
||||
// boundary char so "rom" doesn't match "romm-..."; compose/role separators are - or _.
|
||||
sep := containerName[len(name)]
|
||||
if strings.HasPrefix(containerName, name) && (sep == '-' || sep == '_') && len(name) > len(best) {
|
||||
best = name
|
||||
}
|
||||
}
|
||||
if best != "" {
|
||||
return best
|
||||
}
|
||||
return candidate
|
||||
}
|
||||
|
||||
// suffixStripStackName is the legacy pure suffix-strip (the M19 fallback when no stack list is known).
|
||||
func suffixStripStackName(containerName string) string {
|
||||
// deriveStackName strips known DB suffixes from container name.
|
||||
func deriveStackName(containerName string) string {
|
||||
knownSuffixes := []string{"postgres", "db", "mariadb", "mysql", "database", "redis", "cache"}
|
||||
|
||||
parts := strings.Split(containerName, "-")
|
||||
|
||||
@@ -1,141 +0,0 @@
|
||||
package appbackup
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// R-44 content-sniff tests.
|
||||
//
|
||||
// The dump that triggered this work (DIAG-immich-restore-2026-07-19) was 52MB, had a valid
|
||||
// PostgreSQL header and 60+ CREATE TABLE statements, and contained zero users and zero assets —
|
||||
// its entire bulk was immich's shipped geodata reference tables. Both of the signals the product
|
||||
// already had (file size, table count) called it healthy. These tests pin the one signal that
|
||||
// would have caught it, and the boundaries that keep it from crying wolf.
|
||||
|
||||
func writeDump(t *testing.T, body string) string {
|
||||
t.Helper()
|
||||
p := filepath.Join(t.TempDir(), "d.sql")
|
||||
if err := os.WriteFile(p, []byte(body), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return p
|
||||
}
|
||||
|
||||
const pgHead = `-- PostgreSQL database dump
|
||||
-- Dumped from database version 16.10
|
||||
SET statement_timeout = 0;
|
||||
SET client_encoding = 'UTF8';
|
||||
CREATE TABLE public.asset (id uuid NOT NULL);
|
||||
CREATE TABLE public."user" (id uuid NOT NULL, email text);
|
||||
`
|
||||
|
||||
// TestSniffFlagsEmptyAccountsTable is the 2026-07-19 shape: structurally perfect, no customer.
|
||||
func TestSniffFlagsEmptyAccountsTable(t *testing.T) {
|
||||
body := pgHead + "COPY public.\"user\" (id, email) FROM stdin;\n\\.\n" +
|
||||
"COPY public.asset (id) FROM stdin;\n\\.\n"
|
||||
v := ValidateDump(writeDump(t, body), DBTypePostgres)
|
||||
if !v.Valid {
|
||||
t.Fatalf("the dump is structurally valid; sniff must not change that: %s", v.Error)
|
||||
}
|
||||
if !v.UserTableFound {
|
||||
t.Fatal("the accounts table COPY block was not recognised")
|
||||
}
|
||||
if v.UserRows != 0 {
|
||||
t.Fatalf("UserRows = %d, want 0", v.UserRows)
|
||||
}
|
||||
if !v.LooksEmpty {
|
||||
t.Fatal("a valid dump with zero account rows MUST raise the warn signal — this is the whole point of R-44")
|
||||
}
|
||||
}
|
||||
|
||||
// TestSniffQuietOnPopulatedDump — the common case must stay silent, or the warning becomes noise
|
||||
// and gets ignored precisely when it matters.
|
||||
func TestSniffQuietOnPopulatedDump(t *testing.T) {
|
||||
body := pgHead + "COPY public.\"user\" (id, email) FROM stdin;\n" +
|
||||
"a\tone@example.invalid\nb\ttwo@example.invalid\n\\.\n"
|
||||
v := ValidateDump(writeDump(t, body), DBTypePostgres)
|
||||
if v.UserRows != 2 {
|
||||
t.Fatalf("UserRows = %d, want 2", v.UserRows)
|
||||
}
|
||||
if v.LooksEmpty {
|
||||
t.Fatal("a dump with account rows must not be flagged")
|
||||
}
|
||||
}
|
||||
|
||||
// TestSniffInconclusiveWithoutAccountsTable — plenty of legitimate apps have no users table. No
|
||||
// table, no claim: a false positive here would warn on every restore of such an app forever.
|
||||
func TestSniffInconclusiveWithoutAccountsTable(t *testing.T) {
|
||||
body := "-- PostgreSQL database dump\nCREATE TABLE public.thing (id int);\n" +
|
||||
"COPY public.thing (id) FROM stdin;\n\\.\n" + strings.Repeat("-- pad\n", 20)
|
||||
v := ValidateDump(writeDump(t, body), DBTypePostgres)
|
||||
if v.UserTableFound {
|
||||
t.Fatal("no accounts table exists — none must be reported")
|
||||
}
|
||||
if v.LooksEmpty {
|
||||
t.Fatal("an app without an accounts table must be INCONCLUSIVE, never flagged empty")
|
||||
}
|
||||
}
|
||||
|
||||
// TestSniffIgnoresJoinAndAuditTables is the false-alarm guard that shaped the name list, and it is
|
||||
// written as the case that DISCRIMINATES: an app with NO accounts table but with `user_metadata` /
|
||||
// `album_user` / `user_audit` — all legitimately empty on a healthy box. Exact-matching leaves this
|
||||
// inconclusive (silent, correct). A substring match on "user" would treat a join table as the
|
||||
// accounts table, find zero rows, and shout "your backup looks empty" on every single restore of a
|
||||
// perfectly healthy app — which is how a warning signal becomes noise and then gets ignored.
|
||||
func TestSniffIgnoresJoinAndAuditTables(t *testing.T) {
|
||||
body := "-- PostgreSQL database dump\nCREATE TABLE public.album (id int);\n" +
|
||||
"COPY public.user_metadata (id) FROM stdin;\n\\.\n" +
|
||||
"COPY public.album_user (id) FROM stdin;\n\\.\n" +
|
||||
"COPY public.user_audit (id) FROM stdin;\n\\.\n" +
|
||||
"COPY public.album (id) FROM stdin;\n1\n\\.\n"
|
||||
v := ValidateDump(writeDump(t, body), DBTypePostgres)
|
||||
if v.UserTableFound {
|
||||
t.Fatal("a join/audit table must never be mistaken for the accounts table")
|
||||
}
|
||||
if v.LooksEmpty {
|
||||
t.Fatal("empty join/audit tables must not trigger the warning — this app has no accounts table at all")
|
||||
}
|
||||
}
|
||||
|
||||
// TestSniffCountsOnlyTheAccountsTable pins the counting boundary separately: with a real accounts
|
||||
// table present, rows from neighbouring user-ish tables must not inflate it.
|
||||
func TestSniffCountsOnlyTheAccountsTable(t *testing.T) {
|
||||
body := pgHead +
|
||||
"COPY public.user_metadata (id) FROM stdin;\nm1\nm2\nm3\n\\.\n" +
|
||||
"COPY public.\"user\" (id, email) FROM stdin;\na\tone@example.invalid\n\\.\n"
|
||||
v := ValidateDump(writeDump(t, body), DBTypePostgres)
|
||||
if v.UserRows != 1 {
|
||||
t.Fatalf("only the real accounts table may be counted; UserRows = %d, want 1", v.UserRows)
|
||||
}
|
||||
}
|
||||
|
||||
// TestSniffCountsWideRows — a row wider than the read buffer is skipped by the structural scan, but
|
||||
// it is still a row. Counting it wrong would flag a populated table as empty (immich asset rows are
|
||||
// genuinely long, which is what makes this reachable).
|
||||
func TestSniffCountsWideRows(t *testing.T) {
|
||||
wide := strings.Repeat("x", 300*1024)
|
||||
body := pgHead + "COPY public.\"user\" (id, email) FROM stdin;\n" + wide + "\n\\.\n"
|
||||
v := ValidateDump(writeDump(t, body), DBTypePostgres)
|
||||
if v.UserRows != 1 {
|
||||
t.Fatalf("a buffer-exceeding row must still count; UserRows = %d, want 1", v.UserRows)
|
||||
}
|
||||
if v.LooksEmpty {
|
||||
t.Fatal("a table whose single row is very wide must not sniff as empty")
|
||||
}
|
||||
}
|
||||
|
||||
// TestSniffMariaDBInsertForm — mysqldump writes multi-row INSERTs, not COPY blocks.
|
||||
func TestSniffMariaDBInsertForm(t *testing.T) {
|
||||
head := "-- MariaDB dump 10.19\nCREATE TABLE `users` (id int);\n" + strings.Repeat("-- pad\n", 20)
|
||||
empty := ValidateDump(writeDump(t, head), DBTypeMariaDB)
|
||||
if empty.UserTableFound {
|
||||
t.Fatal("a CREATE TABLE alone is not an accounts-table row source")
|
||||
}
|
||||
full := ValidateDump(writeDump(t, head+"INSERT INTO `users` VALUES (1),(2);\n"), DBTypeMariaDB)
|
||||
if !full.UserTableFound || full.LooksEmpty {
|
||||
t.Fatalf("a populated mariadb dump must not be flagged: %+v", full)
|
||||
}
|
||||
}
|
||||
@@ -1,79 +0,0 @@
|
||||
package appbackup
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
"gopkg.in/yaml.v3"
|
||||
)
|
||||
|
||||
// R-47 — naming the database SERVICE, not just the running container.
|
||||
//
|
||||
// A dump replay must never race the application's own schema management. Proven live on 2026-07-19
|
||||
// (DIAG-immich-restore-round2-2026-07-19, H4): the reconstitution started the whole stack before
|
||||
// replaying, immich-server rebuilt `clip_index` two seconds before the dump's own CREATE INDEX, and
|
||||
// the replay aborted `already exists` under ON_ERROR_STOP=1 — leaving a half-applied schema that the
|
||||
// app itself then reported as drift. The fix is to bring up ONLY the database service(s) for the
|
||||
// replay, which requires knowing their compose SERVICE names (docker `up -d <svc>` takes service
|
||||
// names, not container names).
|
||||
//
|
||||
// The symmetry that makes this safe: a `.sql` dump can only exist because DiscoverDatabases matched
|
||||
// the running container's image string, and the compose `image:` value IS that image string. So the
|
||||
// same predicate — dbTypeForImage — decides both "is there a dump" and "which service holds it".
|
||||
|
||||
// dbTypeForImage maps a container/compose image reference to the database engine the backup code
|
||||
// supports, or ok=false for anything else (redis/valkey/app images — never started in the DB-only
|
||||
// phase). Extracted from DiscoverDatabases so the discovery heuristic and the compose heuristic can
|
||||
// never drift apart; behaviour is byte-equivalent to the inline form it replaced.
|
||||
func dbTypeForImage(image string) (DBType, bool) {
|
||||
img := strings.ToLower(image)
|
||||
switch {
|
||||
case strings.Contains(img, "postgres"):
|
||||
return DBTypePostgres, true
|
||||
case strings.Contains(img, "mariadb"), strings.Contains(img, "mysql"):
|
||||
return DBTypeMariaDB, true
|
||||
}
|
||||
return DBType(""), false
|
||||
}
|
||||
|
||||
// composeServicesDoc is the minimal view of a compose file needed here: the `services:` MAP and each
|
||||
// service's `image:`. Deliberately a real YAML parse and not a line scan — a top-level `volumes:`
|
||||
// block (immich's `immich_ml_cache:`) has exactly the shape a naive scan misreads as a service, and
|
||||
// starting a phantom service, or missing the real one, both land in the wrong branch.
|
||||
type composeServicesDoc struct {
|
||||
Services map[string]struct {
|
||||
Image string `yaml:"image"`
|
||||
} `yaml:"services"`
|
||||
}
|
||||
|
||||
// DBServiceNames returns the sorted compose SERVICE names in composePath whose `image:` identifies a
|
||||
// supported database engine — the exact argument list for `docker compose up -d <svc>...`.
|
||||
//
|
||||
// A file with no (or an empty) `services:` key returns (nil, nil): an app with no identifiable DB
|
||||
// service is a legitimate, common case and the caller decides what it means. An unreadable or
|
||||
// unparseable file returns an error, because "cannot tell" must never silently read as "no database"
|
||||
// — the callers turn that into a refusal when a dump exists.
|
||||
//
|
||||
// Image values are matched literally. Catalog templates pin their images literally (enforced since
|
||||
// Campaign 7), so an interpolated `${...}` image simply does not match and lands in the caller's
|
||||
// fail-closed branch by design, rather than being guessed at.
|
||||
func DBServiceNames(composePath string) ([]string, error) {
|
||||
data, err := os.ReadFile(composePath)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("reading compose file: %w", err)
|
||||
}
|
||||
var doc composeServicesDoc
|
||||
if err := yaml.Unmarshal(data, &doc); err != nil {
|
||||
return nil, fmt.Errorf("parsing compose file %s: %w", composePath, err)
|
||||
}
|
||||
var names []string
|
||||
for name, svc := range doc.Services {
|
||||
if _, ok := dbTypeForImage(svc.Image); ok {
|
||||
names = append(names, name)
|
||||
}
|
||||
}
|
||||
sort.Strings(names)
|
||||
return names, nil
|
||||
}
|
||||
@@ -1,195 +0,0 @@
|
||||
package appbackup
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// R-47 (v0.153.0) — the DB-service resolver.
|
||||
//
|
||||
// These exist because a dump replay that starts the WHOLE stack races the application's own schema
|
||||
// management: proven live on 2026-07-19 (DIAG-immich-restore-round2-2026-07-19, H4) when
|
||||
// immich-server rebuilt `clip_index` two seconds before the dump's CREATE INDEX and the replay
|
||||
// aborted `already exists`. Closing that window means bringing up ONLY the database service, which
|
||||
// means naming it correctly — every case below is a way of naming it wrongly.
|
||||
|
||||
// writeCompose drops a compose file in a temp dir and returns its path.
|
||||
func writeCompose(t *testing.T, body string) string {
|
||||
t.Helper()
|
||||
p := filepath.Join(t.TempDir(), "docker-compose.yml")
|
||||
if err := os.WriteFile(p, []byte(body), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return p
|
||||
}
|
||||
|
||||
// TestDBTypeForImage pins the shared heuristic. It is the SAME predicate DiscoverDatabases applies to
|
||||
// a running container's image, which is what makes "a dump exists ⇒ a service can be named" hold:
|
||||
// the compose `image:` value IS the container's image string. The table reproduces the inline form
|
||||
// this function replaced, byte for byte, including the redis/valkey negatives that must never be
|
||||
// started in the DB-only window.
|
||||
func TestDBTypeForImage(t *testing.T) {
|
||||
cases := []struct {
|
||||
image string
|
||||
want DBType
|
||||
ok bool
|
||||
}{
|
||||
{"docker.io/library/postgres:16-alpine", DBTypePostgres, true},
|
||||
// immich's real pin — a vector-extended postgres whose REPO segment carries the substring.
|
||||
{"ghcr.io/immich-app/postgres:16-vectorchord0.4.3-pgvectors0.2.0", DBTypePostgres, true},
|
||||
{"postgres", DBTypePostgres, true},
|
||||
{"POSTGRES:16", DBTypePostgres, true}, // the discovery path lowercases; so does this
|
||||
{"mariadb:11", DBTypeMariaDB, true},
|
||||
{"mysql:8.4", DBTypeMariaDB, true},
|
||||
{"docker.io/library/MySQL:8", DBTypeMariaDB, true},
|
||||
{"redis:7-alpine", "", false},
|
||||
{"valkey/valkey:8", "", false},
|
||||
{"ghcr.io/immich-app/immich-server:v1.119.0", "", false},
|
||||
{"", "", false},
|
||||
}
|
||||
for _, c := range cases {
|
||||
got, ok := dbTypeForImage(c.image)
|
||||
if ok != c.ok || (ok && got != c.want) {
|
||||
t.Errorf("dbTypeForImage(%q) = (%q, %v), want (%q, %v)", c.image, got, ok, c.want, c.ok)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestDBServiceNames(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
body string
|
||||
want []string
|
||||
}{
|
||||
{
|
||||
name: "postgres service is named",
|
||||
body: "services:\n app:\n image: ghcr.io/x/app:1\n database:\n image: postgres:16\n",
|
||||
want: []string{"database"},
|
||||
},
|
||||
{
|
||||
name: "mariadb service is named",
|
||||
body: "services:\n db:\n image: mariadb:11\n web:\n image: nextcloud:30\n",
|
||||
want: []string{"db"},
|
||||
},
|
||||
{
|
||||
name: "mysql service is named",
|
||||
body: "services:\n mysql:\n image: mysql:8.4\n",
|
||||
want: []string{"mysql"},
|
||||
},
|
||||
{
|
||||
name: "redis-only app has no database service",
|
||||
body: "services:\n app:\n image: ghcr.io/x/app:1\n redis:\n image: redis:7-alpine\n",
|
||||
want: nil,
|
||||
},
|
||||
{
|
||||
name: "multiple databases are returned SORTED (one up -d carries them all)",
|
||||
body: "services:\n zdb:\n image: postgres:16\n adb:\n image: mariadb:11\n app:\n image: x:1\n",
|
||||
want: []string{"adb", "zdb"},
|
||||
},
|
||||
{
|
||||
name: "no services key at all",
|
||||
body: "volumes:\n data:\n",
|
||||
want: nil,
|
||||
},
|
||||
{
|
||||
name: "empty services map",
|
||||
body: "services:\n",
|
||||
want: nil,
|
||||
},
|
||||
{
|
||||
name: "an interpolated image is not guessed at",
|
||||
body: "services:\n db:\n image: ${DB_IMAGE}\n",
|
||||
want: nil,
|
||||
},
|
||||
}
|
||||
for _, c := range cases {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
got, err := DBServiceNames(writeCompose(t, c.body))
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
if !reflect.DeepEqual(got, c.want) {
|
||||
t.Errorf("DBServiceNames = %v, want %v", got, c.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestDBServiceNames_TopLevelKeysAreNotServices is the decoy test, and the reason this is a YAML
|
||||
// parse rather than a line scan. immich's real compose carries a top-level `volumes:` block whose
|
||||
// entry (`immich_ml_cache:`) sits at exactly the indentation a service name does, and a top-level
|
||||
// `networks:` block does the same. A scanner that collected "indented keys followed by image-ish
|
||||
// lines" would either invent a service that `docker compose up -d` cannot start, or — worse — match
|
||||
// the wrong one and leave the real database down while the app came up around the replay.
|
||||
func TestDBServiceNames_TopLevelKeysAreNotServices(t *testing.T) {
|
||||
// The service/volume/network names and the image pins are the catalog's real immich template.
|
||||
// `immich_postgres_data` is the trap made concrete: a top-level VOLUME key whose name contains
|
||||
// "postgres" and which no `up -d` could ever start.
|
||||
body := `services:
|
||||
immich-server:
|
||||
image: ghcr.io/immich-app/immich-server:v3.0.3
|
||||
immich-machine-learning:
|
||||
image: ghcr.io/immich-app/immich-machine-learning:v3.0.3
|
||||
immich-postgres:
|
||||
image: ghcr.io/immich-app/postgres:16-vectorchord0.4.3-pgvectors0.2.0
|
||||
immich-redis:
|
||||
image: redis:7-alpine
|
||||
volumes:
|
||||
immich_ml_cache:
|
||||
immich_postgres_data:
|
||||
immich_redis_data:
|
||||
networks:
|
||||
traefik-public:
|
||||
external: true
|
||||
immich-internal:
|
||||
`
|
||||
got, err := DBServiceNames(writeCompose(t, body))
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
if !reflect.DeepEqual(got, []string{"immich-postgres"}) {
|
||||
t.Fatalf("DBServiceNames = %v, want [immich-postgres] — a top-level volume/network key was mistaken for a service", got)
|
||||
}
|
||||
}
|
||||
|
||||
// TestDBServiceNames_UnreadableAndUnparseableError proves the fail-closed direction: "cannot tell"
|
||||
// must surface as an ERROR, never as the empty (= "this app has no database") answer. The callers
|
||||
// turn an empty result into a refusal only when a dump exists; if a read failure silently produced
|
||||
// the same empty slice for an app with no dump, a genuinely broken compose would flow on unnoticed.
|
||||
func TestDBServiceNames_UnreadableAndUnparseableError(t *testing.T) {
|
||||
if _, err := DBServiceNames(filepath.Join(t.TempDir(), "nope.yml")); err == nil {
|
||||
t.Fatal("a missing compose file must be an error, not an empty service list")
|
||||
}
|
||||
// Valid YAML scalar where a map is required, plus outright broken YAML.
|
||||
if _, err := DBServiceNames(writeCompose(t, "services: [1, 2, 3\n broken")); err == nil {
|
||||
t.Fatal("an unparseable compose file must be an error, not an empty service list")
|
||||
}
|
||||
}
|
||||
|
||||
// TestDiscoverAndComposeAgreeOnTheSameImages is the SYMMETRY guard: whatever image string makes
|
||||
// DiscoverDatabases produce a dump must also make DBServiceNames name a service. They now share one
|
||||
// predicate; this asserts the property that sharing is FOR, so a future edit to either side that
|
||||
// breaks it fails here rather than in a customer's restore.
|
||||
func TestDiscoverAndComposeAgreeOnTheSameImages(t *testing.T) {
|
||||
images := []string{"postgres:16", "mariadb:11", "mysql:8.4", "redis:7", "ghcr.io/x/app:1"}
|
||||
var body strings.Builder
|
||||
body.WriteString("services:\n")
|
||||
var wantDB []string
|
||||
for i, img := range images {
|
||||
svc := string(rune('a' + i))
|
||||
body.WriteString(" " + svc + ":\n image: " + img + "\n")
|
||||
if _, ok := dbTypeForImage(img); ok {
|
||||
wantDB = append(wantDB, svc)
|
||||
}
|
||||
}
|
||||
got, err := DBServiceNames(writeCompose(t, body.String()))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !reflect.DeepEqual(got, wantDB) {
|
||||
t.Fatalf("compose resolver named %v but the discovery predicate says %v — the two sides have drifted", got, wantDB)
|
||||
}
|
||||
}
|
||||
@@ -1,65 +0,0 @@
|
||||
package appbackup
|
||||
|
||||
import "testing"
|
||||
|
||||
// TestDeriveStackName_KnownCrossRef asserts M19: deriveStackName cross-references the deployed-stack set
|
||||
// so a stack whose slug ends in a DB-role token (e.g. `my-cache`) is NOT misattributed by pure
|
||||
// suffix-stripping. The `my-cache` case fails on the pre-fix code (which stripped it to `my`).
|
||||
func TestDeriveStackName_KnownCrossRef(t *testing.T) {
|
||||
known := map[string]bool{"romm": true, "my-cache": true, "paperless-ngx": true}
|
||||
|
||||
cases := []struct {
|
||||
container string
|
||||
want string
|
||||
note string
|
||||
}{
|
||||
{"romm-postgres", "romm", "role suffix of a real stack → strip"},
|
||||
{"my-cache", "my-cache", "container name IS a real stack → do NOT strip to 'my'"}, // pre-fix: "my"
|
||||
{"my-cache-postgres", "my-cache", "strip role, result is a known stack"}, // pre-fix: "my-cache" (ok) — but via prefix here
|
||||
{"paperless-ngx-postgres", "paperless-ngx", "multi-hyphen stack, role suffix"},
|
||||
{"romm_postgres", "romm", "underscore-separated compose name → longest known prefix"},
|
||||
{"romm-1", "romm", "compose numeric suffix → longest known prefix"},
|
||||
}
|
||||
for _, c := range cases {
|
||||
if got := deriveStackName(c.container, known); got != c.want {
|
||||
t.Errorf("deriveStackName(%q, known) = %q, want %q (%s)", c.container, got, c.want, c.note)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestDeriveStackName_LegacyFallback asserts the nil/empty-known fast path preserves the old behaviour
|
||||
// (pure suffix-strip), so callers without a stack list (e.g. appexport) don't regress.
|
||||
func TestDeriveStackName_LegacyFallback(t *testing.T) {
|
||||
cases := []struct {
|
||||
container string
|
||||
want string
|
||||
}{
|
||||
{"romm-postgres", "romm"},
|
||||
{"paperless-ngx-postgres", "paperless-ngx"},
|
||||
{"my-cache", "my"}, // legacy strips the role-token suffix
|
||||
{"unknown-db", "unknown"}, // legacy strip
|
||||
{"standalone", "standalone"},
|
||||
}
|
||||
for _, c := range cases {
|
||||
if got := deriveStackName(c.container, nil); got != c.want {
|
||||
t.Errorf("deriveStackName(%q, nil) = %q, want %q (legacy)", c.container, got, c.want)
|
||||
}
|
||||
// empty (non-nil) map must behave identically to nil
|
||||
if got := deriveStackName(c.container, map[string]bool{}); got != c.want {
|
||||
t.Errorf("deriveStackName(%q, {}) = %q, want %q (legacy)", c.container, got, c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestDeriveStackName_UnknownContainerFallsBack asserts that when the container matches no known stack at
|
||||
// all, the result falls back to the suffix-strip candidate (no spurious prefix match).
|
||||
func TestDeriveStackName_UnknownContainerFallsBack(t *testing.T) {
|
||||
known := map[string]bool{"romm": true}
|
||||
if got := deriveStackName("grafana-db", known); got != "grafana" {
|
||||
t.Fatalf("deriveStackName(grafana-db, {romm}) = %q, want grafana (fallback strip)", got)
|
||||
}
|
||||
// must NOT match "romm" as a prefix of an unrelated name
|
||||
if got := deriveStackName("rommother-db", known); got != "rommother" {
|
||||
t.Fatalf("deriveStackName(rommother-db) = %q, want rommother (no false prefix match)", got)
|
||||
}
|
||||
}
|
||||
@@ -1,96 +0,0 @@
|
||||
package appbackup
|
||||
|
||||
import (
|
||||
"reflect"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func bucketAbs(b []CapturePath) []string {
|
||||
out := make([]string, 0, len(b))
|
||||
for _, p := range b {
|
||||
out = append(out, p.Abs)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// classified app → three buckets, resolved + Abs-sorted.
|
||||
func TestComputeFabBuckets_Classified(t *testing.T) {
|
||||
binds := []ClassifiedBind{
|
||||
{ComposeBind: ComposeBind{Root: RootUserdata, RelPath: "media/books"}, Class: ClassMandatory},
|
||||
{ComposeBind: ComposeBind{Root: RootUserdata, RelPath: "media/comics"}, Class: ClassOptional},
|
||||
{ComposeBind: ComposeBind{Root: RootUserdata, RelPath: "media/movies"}, Class: ClassExcluded},
|
||||
{ComposeBind: ComposeBind{Root: RootHDD, RelPath: "appdata/app"}, Class: ClassMandatory},
|
||||
}
|
||||
fb := ComputeFabBuckets(binds, true, drv, "")
|
||||
if !fb.HasClassification {
|
||||
t.Fatal("HasClassification must be true")
|
||||
}
|
||||
if got, want := bucketAbs(fb.Mandatory), []string{hdd("appdata/app"), udat("media/books")}; !reflect.DeepEqual(got, want) {
|
||||
t.Errorf("Mandatory = %v, want %v", got, want)
|
||||
}
|
||||
if got, want := bucketAbs(fb.Optional), []string{udat("media/comics")}; !reflect.DeepEqual(got, want) {
|
||||
t.Errorf("Optional = %v, want %v", got, want)
|
||||
}
|
||||
if got, want := bucketAbs(fb.Excluded), []string{udat("media/movies")}; !reflect.DeepEqual(got, want) {
|
||||
t.Errorf("Excluded = %v, want %v", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
// legacy (no block) → empty buckets (the full-root capture stays out of the classified plan).
|
||||
func TestComputeFabBuckets_LegacyEmpty(t *testing.T) {
|
||||
binds := []ClassifiedBind{{ComposeBind: ComposeBind{Root: RootUserdata, RelPath: "media/tv"}, Origin: OriginLegacy}}
|
||||
fb := ComputeFabBuckets(binds, false, drv, "")
|
||||
if fb.HasClassification || fb.Mandatory != nil || fb.Optional != nil || fb.Excluded != nil {
|
||||
t.Errorf("legacy app must yield empty buckets, got %+v", fb)
|
||||
}
|
||||
}
|
||||
|
||||
// Scenario E: structural guards run over ALL classes — a traversal path in an EXCLUDED bind is Skipped,
|
||||
// never plannable (opt-in or not).
|
||||
func TestComputeFabBuckets_GuardsAllClasses(t *testing.T) {
|
||||
binds := []ClassifiedBind{
|
||||
{ComposeBind: ComposeBind{Root: RootHDD, RelPath: "../evil"}, Class: ClassExcluded},
|
||||
{ComposeBind: ComposeBind{Root: RootHDD, RelPath: "appdata/ok"}, Class: ClassMandatory},
|
||||
}
|
||||
fb := ComputeFabBuckets(binds, true, drv, "")
|
||||
for _, b := range [][]CapturePath{fb.Mandatory, fb.Optional, fb.Excluded} {
|
||||
for _, p := range b {
|
||||
if p.RelPath == "../evil" {
|
||||
t.Fatal("a traversal path must never enter a bucket (guards run over all classes)")
|
||||
}
|
||||
}
|
||||
}
|
||||
if len(fb.Skipped) != 1 || fb.Skipped[0].RelPath != "../evil" {
|
||||
t.Errorf("traversal excluded path must be Skipped, got %+v", fb.Skipped)
|
||||
}
|
||||
}
|
||||
|
||||
// no cross-bucket containment dedup: a mandatory CHILD inside an excluded PARENT both survive.
|
||||
func TestComputeFabBuckets_NoCrossBucketContainment(t *testing.T) {
|
||||
binds := []ClassifiedBind{
|
||||
{ComposeBind: ComposeBind{Root: RootUserdata, RelPath: "media"}, Class: ClassExcluded},
|
||||
{ComposeBind: ComposeBind{Root: RootUserdata, RelPath: "media/books"}, Class: ClassMandatory},
|
||||
}
|
||||
fb := ComputeFabBuckets(binds, true, drv, "")
|
||||
if got, want := bucketAbs(fb.Mandatory), []string{udat("media/books")}; !reflect.DeepEqual(got, want) {
|
||||
t.Errorf("mandatory child must survive independently: Mandatory = %v, want %v", got, want)
|
||||
}
|
||||
if got, want := bucketAbs(fb.Excluded), []string{udat("media")}; !reflect.DeepEqual(got, want) {
|
||||
t.Errorf("excluded parent must survive: Excluded = %v, want %v", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
// equal-Abs collapse: two spellings of one path collapse, mandatory wins (into the mandatory bucket).
|
||||
func TestComputeFabBuckets_EqualAbsMandatoryWins(t *testing.T) {
|
||||
binds := []ClassifiedBind{
|
||||
{ComposeBind: ComposeBind{Root: RootHDD, RelPath: "userdata/media"}, Class: ClassOptional},
|
||||
{ComposeBind: ComposeBind{Root: RootUserdata, RelPath: "media"}, Class: ClassMandatory},
|
||||
}
|
||||
fb := ComputeFabBuckets(binds, true, drv, "")
|
||||
if got, want := bucketAbs(fb.Mandatory), []string{udat("media")}; !reflect.DeepEqual(got, want) {
|
||||
t.Errorf("collapsed path must land in Mandatory, got Mandatory=%v", got)
|
||||
}
|
||||
if len(fb.Optional) != 0 {
|
||||
t.Errorf("optional spelling must collapse away, got %v", bucketAbs(fb.Optional))
|
||||
}
|
||||
}
|
||||
@@ -1,94 +0,0 @@
|
||||
package appbackup
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// writeDumpFile creates a minimal VALID postgres dump so a real ValidateDump would set Valid=true and
|
||||
// TableCount=1 — distinguishable from the sentinel the cache returns.
|
||||
func writeDumpFile(t *testing.T, dir string) string {
|
||||
t.Helper()
|
||||
path := filepath.Join(dir, "romm-postgres.sql")
|
||||
body := "-- PostgreSQL database dump\n" +
|
||||
"CREATE TABLE public.t (id int);\n" +
|
||||
"-- PostgreSQL database dump complete\n" +
|
||||
// pad past the 100-byte floor
|
||||
"-- padding ------------------------------------------------------------\n"
|
||||
if err := os.WriteFile(path, []byte(body), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return path
|
||||
}
|
||||
|
||||
// TestListDumpFiles_CacheHitSkipsValidate asserts M18: when the `cached` lookup returns ok for an
|
||||
// unchanged file, ListDumpFiles reuses that result and does NOT re-run the (expensive) ValidateDump.
|
||||
// Proven via a sentinel TableCount=999 that only the cache could supply (a real validate yields 1).
|
||||
func TestListDumpFiles_CacheHitSkipsValidate(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
writeDumpFile(t, dir)
|
||||
|
||||
const sentinel = 999
|
||||
calls := 0
|
||||
cached := func(name string, size int64, mod time.Time) (DumpValidation, bool) {
|
||||
calls++
|
||||
return DumpValidation{Valid: true, TableCount: sentinel}, true // always "hit"
|
||||
}
|
||||
|
||||
files, err := ListDumpFiles(dir, cached)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(files) != 1 {
|
||||
t.Fatalf("expected 1 dump file, got %d", len(files))
|
||||
}
|
||||
if files[0].Validation.TableCount != sentinel {
|
||||
t.Fatalf("validation TableCount = %d, want %d (sentinel from cache) — ValidateDump was re-run instead of using the cache",
|
||||
files[0].Validation.TableCount, sentinel)
|
||||
}
|
||||
if calls != 1 {
|
||||
t.Fatalf("cached lookup called %d times, want 1", calls)
|
||||
}
|
||||
}
|
||||
|
||||
// TestListDumpFiles_CacheMissValidates asserts that on a cache MISS (e.g. modtime changed) ListDumpFiles
|
||||
// falls through to a real ValidateDump (sentinel must NOT appear; real TableCount=1).
|
||||
func TestListDumpFiles_CacheMissValidates(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
writeDumpFile(t, dir)
|
||||
|
||||
cached := func(name string, size int64, mod time.Time) (DumpValidation, bool) {
|
||||
return DumpValidation{Valid: true, TableCount: 999}, false // always "miss"
|
||||
}
|
||||
|
||||
files, err := ListDumpFiles(dir, cached)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(files) != 1 {
|
||||
t.Fatalf("expected 1 dump file, got %d", len(files))
|
||||
}
|
||||
if files[0].Validation.TableCount == 999 {
|
||||
t.Fatalf("got the sentinel on a cache MISS — should have run a real ValidateDump")
|
||||
}
|
||||
if !files[0].Validation.Valid || files[0].Validation.TableCount != 1 {
|
||||
t.Fatalf("real validation expected Valid=true TableCount=1, got Valid=%v TableCount=%d",
|
||||
files[0].Validation.Valid, files[0].Validation.TableCount)
|
||||
}
|
||||
}
|
||||
|
||||
// TestListDumpFiles_NilCachedAlwaysValidates asserts the legacy fast path (cached==nil) still validates.
|
||||
func TestListDumpFiles_NilCachedAlwaysValidates(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
writeDumpFile(t, dir)
|
||||
|
||||
files, err := ListDumpFiles(dir, nil)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(files) != 1 || !files[0].Validation.Valid || files[0].Validation.TableCount != 1 {
|
||||
t.Fatalf("nil-cached path should validate: got %+v", files[0].Validation)
|
||||
}
|
||||
}
|
||||
@@ -1,69 +0,0 @@
|
||||
package appbackup
|
||||
|
||||
import "testing"
|
||||
|
||||
// R-203 — the ONE drive-kind rule. Table-driven over BOTH drive kinds on purpose: this defect
|
||||
// survived because it is invisible on the kind that already worked, so a test that only covers the
|
||||
// enrolled drive proves nothing about the fix.
|
||||
|
||||
func TestNamespaceRootFor_BothDriveKinds(t *testing.T) {
|
||||
const sys = "/mnt/sys_drive"
|
||||
cases := []struct {
|
||||
name, drive, want string
|
||||
}{
|
||||
// Scenario B — the enrolled drive must be BYTE-IDENTICAL to pre-R-203 behaviour. The
|
||||
// in-guest mount already IS the namespace root; appending felhom-data here would recreate
|
||||
// the .../felhom-data/felhom-data/... double-nest NamespaceRoot's comment exists to prevent.
|
||||
{"enrolled usb", "/mnt/felhom-usb", "/mnt/felhom-usb"},
|
||||
{"enrolled hdd", "/mnt/felhom-drives/hdd_1", "/mnt/felhom-drives/hdd_1"},
|
||||
{"enrolled nvme", "/mnt/felhom-drives/nvme-1tb", "/mnt/felhom-drives/nvme-1tb"},
|
||||
// Scenario A — the system-data fallback gains the segment. This is the case that was wrong.
|
||||
{"system drive", "/mnt/sys_drive", "/mnt/sys_drive/felhom-data"},
|
||||
// A trailing slash is the same drive. Before R-203 the backup package's copy of this rule
|
||||
// compared WITHOUT Clean while the stacks package's copy compared WITH it — so a config value
|
||||
// with a trailing slash would have flipped the mode in one package and not the other.
|
||||
{"system drive, trailing slash", "/mnt/sys_drive/", "/mnt/sys_drive/felhom-data"},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
if got := NamespaceRootFor(tc.drive, sys); got != tc.want {
|
||||
t.Fatalf("NamespaceRootFor(%q, %q) = %q, want %q", tc.drive, sys, got, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// The rule must survive a trailing slash on the SYSTEM path too — it comes from config.
|
||||
func TestIsEnrolledDrive_CleansBothSides(t *testing.T) {
|
||||
if IsEnrolledDrive("/mnt/sys_drive", "/mnt/sys_drive/") {
|
||||
t.Error("a trailing slash on the system path must not make the system drive look enrolled")
|
||||
}
|
||||
if IsEnrolledDrive("/mnt/sys_drive/", "/mnt/sys_drive") {
|
||||
t.Error("a trailing slash on the drive path must not make the system drive look enrolled")
|
||||
}
|
||||
if !IsEnrolledDrive("/mnt/felhom-usb", "/mnt/sys_drive") {
|
||||
t.Error("an enrolled drive must report enrolled")
|
||||
}
|
||||
}
|
||||
|
||||
// The consequence the whole item is about: the directory an app binds and the directory the capture
|
||||
// set looks in must be the SAME on both drive kinds.
|
||||
//
|
||||
// RED-PROOF: replace `UserdataDir(NamespaceRootFor(drive, sys))` with `UserdataDir(drive)` — the
|
||||
// pre-R-203 call — and the system-drive row FAILS with the two paths differing by exactly
|
||||
// `/felhom-data`. That is production behaviour up to v0.196.0.
|
||||
func TestAppBindAndCaptureRootAgree(t *testing.T) {
|
||||
const sys = "/mnt/sys_drive"
|
||||
for _, drive := range []string{"/mnt/felhom-usb", "/mnt/felhom-drives/hdd_1", "/mnt/sys_drive"} {
|
||||
nsRoot := NamespaceRootFor(drive, sys)
|
||||
appBind := UserdataDir(nsRoot) // what the deploy sets as ${USERDATA_PATH}
|
||||
captureRoot := UserdataDir(nsRoot) // what the capture set resolves RootUserdata against
|
||||
if appBind != captureRoot {
|
||||
t.Fatalf("drive %q: the app binds %q while the backup captures %q", drive, appBind, captureRoot)
|
||||
}
|
||||
// And it must be the canonical location — the one EnsureUserdataSkeleton creates.
|
||||
if drive == sys && appBind != "/mnt/sys_drive/felhom-data/userdata" {
|
||||
t.Fatalf("system drive resolved to %q, want the canonical /mnt/sys_drive/felhom-data/userdata", appBind)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -5,11 +5,7 @@
|
||||
// cross-drive, or drive-mount code in the backup package.
|
||||
package appbackup
|
||||
|
||||
import (
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
import "path/filepath"
|
||||
|
||||
// FelhomDataDir is the namespace directory on storage drives for all felhom-managed data.
|
||||
const FelhomDataDir = "felhom-data"
|
||||
@@ -32,127 +28,22 @@ func NamespaceRoot(drivePath string, inGuestDrive bool) string {
|
||||
return filepath.Join(drivePath, FelhomDataDir)
|
||||
}
|
||||
|
||||
// IsEnrolledDrive reports whether a drive path is an ENROLLED user-data drive (Model A: its in-guest
|
||||
// mount already IS the namespace root) rather than the system-data fallback. It is the ONE comparison
|
||||
// that decides which NamespaceRoot mode applies, and it lives here so no package re-derives it.
|
||||
//
|
||||
// Both sides are Clean'd: `/mnt/sys_drive/` and `/mnt/sys_drive` are the same drive, and a trailing
|
||||
// slash arriving from config must not silently flip the mode.
|
||||
func IsEnrolledDrive(drivePath, systemDataPath string) bool {
|
||||
return filepath.Clean(drivePath) != filepath.Clean(systemDataPath)
|
||||
}
|
||||
|
||||
// NamespaceRootFor is the resolver every caller should use when it holds a bare DRIVE path and the
|
||||
// system-data path — i.e. everywhere outside the backup package, which already had this rule.
|
||||
//
|
||||
// R-203: FIVE call sites passed a bare drive path straight to UserdataDir (and its siblings), which
|
||||
// take a NAMESPACE ROOT. On an enrolled drive the two coincide, so nothing showed; on the system-data
|
||||
// fallback they differ by exactly the felhom-data segment, and the app then bound a directory the
|
||||
// backup never looked at. The run still reported ok. Measured live on demo-hp 2026-08-04:
|
||||
// the app wrote to /mnt/sys_drive/userdata/media/books while the off-site capture set looked for
|
||||
// /mnt/sys_drive/felhom-data/userdata/media/books.
|
||||
//
|
||||
// THE CONTRACT, restated because four callers got it wrong and a fifth will: UserdataDir,
|
||||
// PrimaryBackupPath, RecoveryUnitPath and AppDataDir all take a NAMESPACE ROOT. If you are holding
|
||||
// something that came out of HDD_PATH or a StoragePath, it is a DRIVE path — put it through here
|
||||
// first. `UserdataDir(bareDrivePath)` still compiles and is still wrong; TestNoBareDrivePathToUserdataDir
|
||||
// is the guard that keeps the count from growing.
|
||||
func NamespaceRootFor(drivePath, systemDataPath string) string {
|
||||
return NamespaceRoot(drivePath, IsEnrolledDrive(drivePath, systemDataPath))
|
||||
}
|
||||
|
||||
// PrimaryBackupPath returns the root primary backup directory under a felhom-data namespace root.
|
||||
func PrimaryBackupPath(nsRoot string) string {
|
||||
return filepath.Join(nsRoot, "backups", "primary")
|
||||
}
|
||||
|
||||
// RecoveryUnitPath returns the per-app self-contained recovery-unit ROOT under a namespace root.
|
||||
// It is the existing per-app backup dir (`backups/primary/<stack>/`) — the legacy name is kept so the
|
||||
// db-dumps/ and volume-dumps/ already written there need no migration; the unit gains compose/ and
|
||||
// manifest.json as siblings, making the whole dir a complete, recreatable unit (Phase 2). Since D5 the
|
||||
// unit's compose/app.yaml CARRIES the portable secret class (data keys, DB passwords, internal signing
|
||||
// secrets) at mode 0600, so a Tier-1/2 restore needs the drive and nothing else; internet-reachable
|
||||
// admin logins are still withheld. See backup.recoveryUnit / restore for the capture + restore flow.
|
||||
func RecoveryUnitPath(nsRoot, stackName string) string {
|
||||
return filepath.Join(nsRoot, "backups", "primary", stackName)
|
||||
}
|
||||
|
||||
// RecoveryUnitComposePath returns the compose/config capture dir within an app's recovery unit
|
||||
// (docker-compose.yml + .felhom.yml + secret-stripped app.yaml).
|
||||
func RecoveryUnitComposePath(nsRoot, stackName string) string {
|
||||
return filepath.Join(RecoveryUnitPath(nsRoot, stackName), "compose")
|
||||
}
|
||||
|
||||
// RecoveryUnitManifestPath returns the manifest.json path within an app's recovery unit.
|
||||
func RecoveryUnitManifestPath(nsRoot, stackName string) string {
|
||||
return filepath.Join(RecoveryUnitPath(nsRoot, stackName), "manifest.json")
|
||||
}
|
||||
|
||||
// AppDBDumpPath returns the DB dump directory for an app under a felhom-data namespace root.
|
||||
func AppDBDumpPath(nsRoot, stackName string) string {
|
||||
return filepath.Join(RecoveryUnitPath(nsRoot, stackName), "db-dumps")
|
||||
return filepath.Join(nsRoot, "backups", "primary", stackName, "db-dumps")
|
||||
}
|
||||
|
||||
// AppVolumeDumpPath returns the Docker-volume dump-tar directory for an app under a namespace root.
|
||||
func AppVolumeDumpPath(nsRoot, stackName string) string {
|
||||
return filepath.Join(RecoveryUnitPath(nsRoot, stackName), "volume-dumps")
|
||||
return filepath.Join(nsRoot, "backups", "primary", stackName, "volume-dumps")
|
||||
}
|
||||
|
||||
// AppDataDir returns the app data directory under a felhom-data namespace root. The final segment
|
||||
// is the app's real appdata dir NAME — usually the stack name, but NOT always: paperless-ngx writes
|
||||
// appdata/paperless (F-S2/F-S3). Callers that key by stack name silently miss such apps; use
|
||||
// AppDataDirNames to resolve the real name(s) from the app's compose binds and pass them here.
|
||||
// AppDataDir returns the app data directory under a felhom-data namespace root.
|
||||
func AppDataDir(nsRoot, stackName string) string {
|
||||
return filepath.Join(nsRoot, "appdata", stackName)
|
||||
}
|
||||
|
||||
// AppDataDirNames returns the app's real directory name(s) under <hddPath>/appdata, derived from its
|
||||
// compose HDD bind mounts (F-S2/F-S3: the dir name is NOT always the stack name — paperless-ngx
|
||||
// writes appdata/paperless). hddMounts are resolved host paths in the ParseComposeHDDMounts shape
|
||||
// (each is <hddPath> itself or a subpath, filepath.Clean'd). The first path element under
|
||||
// <hddPath>/appdata/ is taken as the dir name; results are deduped and sorted. Falls back to
|
||||
// []string{stackName} when no appdata-prefixed mount is derivable (no HDD appdata binds, unreadable
|
||||
// compose, nil provider) — the exact legacy behavior.
|
||||
//
|
||||
// Today every catalog app resolves to exactly ONE name (immich→immich, nextcloud→nextcloud,
|
||||
// romm→romm, paperless-ngx→paperless). The N>1 return is defensive: tier-2 refuses it loudly,
|
||||
// migrate handles it naturally.
|
||||
func AppDataDirNames(hddPath, stackName string, hddMounts []string) []string {
|
||||
prefix := filepath.Clean(hddPath) + string(filepath.Separator) + "appdata" + string(filepath.Separator)
|
||||
seen := make(map[string]bool)
|
||||
var names []string
|
||||
for _, mnt := range hddMounts {
|
||||
cm := filepath.Clean(mnt)
|
||||
if !strings.HasPrefix(cm, prefix) {
|
||||
continue // not under appdata/ (a whole-root bind, a different subtree, a foreign drive)
|
||||
}
|
||||
rem := strings.TrimPrefix(cm, prefix)
|
||||
first := strings.Split(rem, string(filepath.Separator))[0]
|
||||
if first == "" {
|
||||
continue
|
||||
}
|
||||
if !seen[first] {
|
||||
seen[first] = true
|
||||
names = append(names, first)
|
||||
}
|
||||
}
|
||||
if len(names) == 0 {
|
||||
return []string{stackName}
|
||||
}
|
||||
sort.Strings(names)
|
||||
return names
|
||||
}
|
||||
|
||||
// AppDataBindsPresent reports whether any of the app's resolved HDD mounts sits under
|
||||
// <hddPath>/appdata/ — i.e. the compose actually DECLARES an appdata bind. Callers use it to
|
||||
// distinguish "no appdata to back up" (silent skip is correct) from "declared appdata dir missing
|
||||
// on disk" (the silence that hid F-S2 — worth a WARN). Same prefix rule as AppDataDirNames.
|
||||
func AppDataBindsPresent(hddPath string, hddMounts []string) bool {
|
||||
prefix := filepath.Clean(hddPath) + string(filepath.Separator) + "appdata" + string(filepath.Separator)
|
||||
for _, mnt := range hddMounts {
|
||||
if strings.HasPrefix(filepath.Clean(mnt), prefix) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
@@ -1,163 +0,0 @@
|
||||
package appbackup
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// R-75 Scenario C — DETERMINISM. This is the P6 gate.
|
||||
//
|
||||
// The spike measured the naive map-order derivation producing 20 DISTINCT outputs from 20 identical
|
||||
// runs. fbNeedsRecreate force-recreates FileBrowser on ANY byte difference in the generated config,
|
||||
// and SyncFileBrowserMounts has ~14 call sites — so a non-deterministic skeleton is a fleet-wide
|
||||
// FileBrowser restart loop, the v0.151-class bug. 20 identical generations or this fails.
|
||||
func TestScenarioC_SkeletonDeterminism(t *testing.T) {
|
||||
// Deliberately UNSORTED input, with duplicates and a deep path, so the function has real work to
|
||||
// normalise. A sort applied only to the input would not save a map-ordered implementation.
|
||||
derived := []string{
|
||||
"media/podcasts", "roms", "media/books", "downloads", "media",
|
||||
"media/photos", "media/books", "a/b/c/d",
|
||||
}
|
||||
const n = 20
|
||||
first := BuildUserdataSkeleton(derived)
|
||||
for i := 1; i < n; i++ {
|
||||
got := BuildUserdataSkeleton(derived)
|
||||
if !slices.Equal(got, first) {
|
||||
t.Fatalf("generation %d/%d differs — a non-deterministic skeleton force-recreates FileBrowser on every sync pass\n first: %v\n got: %v",
|
||||
i+1, n, first, got)
|
||||
}
|
||||
}
|
||||
if !slices.IsSorted(first) {
|
||||
t.Errorf("skeleton must be sorted, got %v", first)
|
||||
}
|
||||
// Ancestor expansion: a deep derived path implies its whole chain.
|
||||
for _, want := range []string{"a", "a/b", "a/b/c", "a/b/c/d"} {
|
||||
if !slices.Contains(first, want) {
|
||||
t.Errorf("ancestor chain incomplete: %q missing from %v", want, first)
|
||||
}
|
||||
}
|
||||
// Dedup: "media/books" appeared twice in the input and "media" both derived and as an ancestor.
|
||||
for _, d := range []string{"media", "media/books"} {
|
||||
if c := countOf(first, d); c != 1 {
|
||||
t.Errorf("%q appears %d times, want exactly 1", d, c)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func countOf(xs []string, want string) int {
|
||||
n := 0
|
||||
for _, x := range xs {
|
||||
if x == want {
|
||||
n++
|
||||
}
|
||||
}
|
||||
return n
|
||||
}
|
||||
|
||||
// R-75 Scenario D — ZERO REMOVALS, proven by construction.
|
||||
//
|
||||
// The derived set drops `documents` (implied by no catalog app) and, after the R-75 move, the two
|
||||
// import/* entries. The carry-list is what keeps them. This asserts the merged set is a strict
|
||||
// SUPERSET of the historical hardcoded skeleton for any derived input — including the empty one, the
|
||||
// fresh-box case where the catalog has not synced yet.
|
||||
func TestScenarioD_SkeletonNeverDropsACarriedDir(t *testing.T) {
|
||||
for _, derived := range [][]string{
|
||||
nil, // fresh box, catalog not yet synced
|
||||
{"media/podcasts"}, // the one genuinely new entry
|
||||
{"roms", "downloads", "media/photos"}, // a partial catalog
|
||||
} {
|
||||
got := BuildUserdataSkeleton(derived)
|
||||
for _, carried := range UserdataSkeletonCarry() {
|
||||
if !slices.Contains(got, carried) {
|
||||
t.Errorf("derived=%v: carried dir %q was DROPPED — zero-removals violated", derived, carried)
|
||||
}
|
||||
}
|
||||
}
|
||||
// And the new entry really is added when the catalog implies it.
|
||||
if !slices.Contains(BuildUserdataSkeleton([]string{"media/podcasts"}), "media/podcasts") {
|
||||
t.Error("media/podcasts must be added when the catalog implies it")
|
||||
}
|
||||
// `documents` is the specific entry the spike flagged: in the carry-list, in no catalog app.
|
||||
if !slices.Contains(BuildUserdataSkeleton([]string{"media/podcasts"}), "documents") {
|
||||
t.Error("`documents` must survive — it exists on both demo boxes and may hold customer files")
|
||||
}
|
||||
}
|
||||
|
||||
// A traversal or absolute entry reaching the skeleton would make EnsureUserdataSkeleton create a
|
||||
// directory outside the userdata root. The derived set comes from a compose parser, so this is a
|
||||
// guard on untrusted-ish catalog input, not defence in depth.
|
||||
func TestSkeletonRefusesEscapes(t *testing.T) {
|
||||
got := BuildUserdataSkeleton([]string{"../escape", "..", "", "/abs/path", "ok/dir"})
|
||||
for _, bad := range []string{"../escape", "..", "", "/abs/path"} {
|
||||
if slices.Contains(got, bad) {
|
||||
t.Errorf("escape entry %q must not reach the skeleton: %v", bad, got)
|
||||
}
|
||||
}
|
||||
for _, d := range got {
|
||||
if filepath.IsAbs(d) || d == ".." || len(d) > 3 && d[:3] == "../" {
|
||||
t.Errorf("unsafe skeleton entry %q", d)
|
||||
}
|
||||
}
|
||||
if !slices.Contains(got, "ok/dir") {
|
||||
t.Error("a legitimate entry alongside bad ones must still be kept")
|
||||
}
|
||||
// "/abs/path" is not dropped outright — it is normalised to a relative path and kept, which is
|
||||
// safe (it lands under the userdata root). Pin that so the behaviour is a decision, not a guess.
|
||||
if !slices.Contains(got, "abs/path") {
|
||||
t.Errorf("an absolute entry should be normalised to relative, got %v", got)
|
||||
}
|
||||
}
|
||||
|
||||
// EnsureUserdataSkeleton creates every dir it is given and NOTHING ELSE, and never removes.
|
||||
func TestEnsureUserdataSkeletonCreatesOnly(t *testing.T) {
|
||||
ns := t.TempDir()
|
||||
// A pre-existing customer dir that no catalog app implies and the carry-list does not contain.
|
||||
stray := filepath.Join(UserdataDir(ns), "sajat-mappa")
|
||||
if err := os.MkdirAll(stray, 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
dirs := BuildUserdataSkeleton([]string{"media/podcasts"})
|
||||
if err := EnsureUserdataSkeleton(ns, dirs); err != nil {
|
||||
// chown to gid 1000 fails for a non-root test user; the dirs are still created.
|
||||
t.Logf("EnsureUserdataSkeleton returned %v (expected when not running as root)", err)
|
||||
}
|
||||
for _, d := range dirs {
|
||||
if fi, err := os.Stat(filepath.Join(UserdataDir(ns), d)); err != nil || !fi.IsDir() {
|
||||
t.Errorf("skeleton dir %q not created: %v", d, err)
|
||||
}
|
||||
}
|
||||
if _, err := os.Stat(stray); err != nil {
|
||||
t.Errorf("a pre-existing customer dir was removed — zero-removals violated: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// R-75: a DATA drive must never get a per-drive drop-zone from the skeleton. Carrying the old
|
||||
// `import/*` entries would have the skeleton re-create a dead lookalike on every drive forever —
|
||||
// one that is also never backed up, since import paths are class: excluded.
|
||||
//
|
||||
// This is NOT a zero-removals violation: nothing deletes the dirs a box already has (see
|
||||
// TestEnsureUserdataSkeletonCreatesOnly). They stop being maintained and stop appearing on fresh boxes.
|
||||
func TestSkeletonNeverCreatesAPerDriveDropZone(t *testing.T) {
|
||||
// The catalog no longer implies any ${USERDATA_PATH}/import path — the binds moved to
|
||||
// ${IMPORT_PATH} — so the only way one could appear is via the carry-list.
|
||||
for _, derived := range [][]string{nil, {"media/podcasts", "roms"}} {
|
||||
for _, d := range BuildUserdataSkeleton(derived) {
|
||||
if d == "import" || strings.HasPrefix(d, "import/") {
|
||||
t.Errorf("derived=%v: skeleton created a per-drive drop-zone %q — the canonical root is on the SYSTEM drive", derived, d)
|
||||
}
|
||||
}
|
||||
}
|
||||
for _, c := range UserdataSkeletonCarry() {
|
||||
if c == "import" || strings.HasPrefix(c, "import/") {
|
||||
t.Errorf("the carry-list still holds %q", c)
|
||||
}
|
||||
}
|
||||
// A catalog app that genuinely declares a ${USERDATA_PATH}/import/... bind would still be
|
||||
// honoured — the rule is "don't carry them", not "filter them out".
|
||||
if !slices.Contains(BuildUserdataSkeleton([]string{"import/valami"}), "import/valami") {
|
||||
t.Error("a genuinely derived userdata import path must still be created")
|
||||
}
|
||||
}
|
||||
@@ -1,155 +0,0 @@
|
||||
package appbackup
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Customer-facing userdata layout + the shared-storage ownership convention (v0.66.0).
|
||||
//
|
||||
// userdata/ is a sibling of appdata/ and backups/ under a drive's felhom-data namespace. It is the
|
||||
// ONLY customer-browsable tree (FileBrowser mounts it). Apps that handle customer content write here.
|
||||
//
|
||||
// Ownership convention: every userdata dir is group-owned by SharedContentGID, mode 2775 (setgid +
|
||||
// group-rwx). Setgid makes new files/dirs inherit the shared group regardless of which app (or
|
||||
// FileBrowser) created them, so members collaborate without permission collisions. FileBrowser
|
||||
// (uid/gid 1000) and the content apps (PUID/PGID 1000, or pinned user 1000:1000) are all members.
|
||||
|
||||
// SharedContentGID is the group that owns the userdata tree.
|
||||
const SharedContentGID = 1000
|
||||
|
||||
// userdataDirMode is the on-disk mode for every userdata dir: setgid + group-rwx. os.ModeSetgid (NOT
|
||||
// the raw 0o2000) is how Go's Chmod requests S_ISGID. MkdirAll's mode is umask-masked AND drops the
|
||||
// setgid bit, so an explicit Chmod is mandatory after MkdirAll.
|
||||
const userdataDirMode = os.ModeSetgid | 0o775
|
||||
|
||||
// UserdataDir returns the customer-facing userdata root under a namespace root.
|
||||
func UserdataDir(nsRoot string) string {
|
||||
return filepath.Join(nsRoot, "userdata")
|
||||
}
|
||||
|
||||
// ImportDirName is the single import (drop-zone) subtree name under a userdata root.
|
||||
const ImportDirName = "import"
|
||||
|
||||
// ImportDir returns the CANONICAL drop-zone root under a namespace root (R-75).
|
||||
//
|
||||
// Unlike every other userdata dir, this one is drive-INDEPENDENT: the caller resolves it against the
|
||||
// SYSTEM drive's namespace root, never against the app's own HDD_PATH, so a multi-drive box has
|
||||
// exactly ONE import tree. That is the whole point. Each drop-zone app has exactly one ingest bind,
|
||||
// so a per-drive import/ would put a folder that LOOKS like a drop-zone on every drive while only
|
||||
// one of them does anything — and because import paths are `class: excluded`, files stranded in a
|
||||
// dead one are never backed up either.
|
||||
//
|
||||
// It deliberately stays INSIDE the userdata tree, so the 2775/setgid/GID-1000 convention, the
|
||||
// FileBrowser mount and the ownership rules all apply to it unchanged.
|
||||
func ImportDir(nsRoot string) string {
|
||||
return filepath.Join(UserdataDir(nsRoot), ImportDirName)
|
||||
}
|
||||
|
||||
// UserdataSkeletonCarry is the explicit NON-DERIVED carry-list: every entry the v0.171.0 hardcoded
|
||||
// skeleton created, retained verbatim and forever.
|
||||
//
|
||||
// It exists so the catalog-derived skeleton (R-75) can only ever ADD. That makes the zero-removals
|
||||
// invariant true BY CONSTRUCTION rather than by review, and it is not hypothetical:
|
||||
//
|
||||
// - `documents` is implied by NO catalog app (SPIKE P0(a)) yet exists on both demo boxes and is
|
||||
// customer-visible — it may hold customer files. Derivation alone would drop it.
|
||||
//
|
||||
// It doubles as the fresh-box floor: on a box whose catalog has not synced yet the derived set is
|
||||
// empty, and the customer still gets the full standard tree instead of a nearly-empty one.
|
||||
//
|
||||
// DELIBERATELY ABSENT: `import`, `import/paperless`, `import/calibre`. They were in the v0.171.0
|
||||
// hardcoded list, and carrying them would have the skeleton RE-CREATE a per-drive drop-zone on every
|
||||
// drive forever — the exact dead-lookalike R-75 exists to remove, and one that is never backed up
|
||||
// (`class: excluded`). Zero-removals is about not DELETING what a box already has, not about
|
||||
// re-creating it on boxes that never had it: nothing here removes the pre-existing dirs on
|
||||
// demo-felhom / demo-hp, they simply stop being maintained and stop appearing on fresh boxes.
|
||||
// Verified before the change: both boxes' old drop-zones held ZERO files (2026-07-26). A box with
|
||||
// pending files in an old drop-zone would need an operator-run move — see REPORT.md.
|
||||
//
|
||||
// ASCII, no spaces (flows through ${} interpolation, shell, and the rsync merge walk).
|
||||
func UserdataSkeletonCarry() []string {
|
||||
return []string{
|
||||
"media", "media/movies", "media/tv", "media/music", "media/audiobooks",
|
||||
"media/books", "media/comics", "media/photos",
|
||||
"downloads",
|
||||
"roms",
|
||||
"documents",
|
||||
}
|
||||
}
|
||||
|
||||
// BuildUserdataSkeleton merges the catalog-derived dirs with the carry-list into the final, SORTED
|
||||
// set. Each entry is expanded to its ancestor chain ("media/podcasts" implies "media"), deduped, and
|
||||
// sorted.
|
||||
//
|
||||
// SORTING IS A HARD REQUIREMENT, not tidiness. The FileBrowser config is regenerated from this set
|
||||
// and fbNeedsRecreate force-recreates the container on ANY byte difference. Go randomises map
|
||||
// iteration, and the spike measured the naive map-order derivation producing 20 DISTINCT outputs from
|
||||
// 20 identical runs (SPIKE P6) — which across SyncFileBrowserMounts' ~14 call sites is a fleet-wide
|
||||
// FileBrowser restart loop. TestSkeletonDeterminism pins this.
|
||||
func BuildUserdataSkeleton(derived []string) []string {
|
||||
set := make(map[string]bool, len(derived)+16)
|
||||
addChain := func(rel string) {
|
||||
rel = path.Clean(strings.TrimPrefix(filepath.ToSlash(rel), "/"))
|
||||
if rel == "" || rel == "." || rel == ".." || strings.HasPrefix(rel, "../") {
|
||||
return // never let a traversal or an empty entry become a directory to create
|
||||
}
|
||||
parts := strings.Split(rel, "/")
|
||||
for i := range parts {
|
||||
set[strings.Join(parts[:i+1], "/")] = true
|
||||
}
|
||||
}
|
||||
for _, d := range UserdataSkeletonCarry() {
|
||||
addChain(d)
|
||||
}
|
||||
for _, d := range derived {
|
||||
addChain(d)
|
||||
}
|
||||
out := make([]string, 0, len(set))
|
||||
for d := range set { // map order is RANDOM — the sort below is what makes this deterministic
|
||||
out = append(out, d)
|
||||
}
|
||||
sort.Strings(out)
|
||||
return out
|
||||
}
|
||||
|
||||
// EnsureDirOwned creates path (idempotent) and enforces the convention: mode 2775 via an explicit
|
||||
// Chmod incl. setgid (MkdirAll cannot) + group = gid. Setting an arbitrary group needs CAP_CHOWN —
|
||||
// the in-guest controller runs as root, so this succeeds in production. Returns the first hard error.
|
||||
func EnsureDirOwned(path string, gid int) error {
|
||||
if err := os.MkdirAll(path, 0o755); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := os.Chmod(path, userdataDirMode); err != nil {
|
||||
return err
|
||||
}
|
||||
return chownGID(path, gid)
|
||||
}
|
||||
|
||||
// EnsureUserdataDir applies the convention with the shared content group (GID 1000). Idempotent.
|
||||
func EnsureUserdataDir(path string) error { return EnsureDirOwned(path, SharedContentGID) }
|
||||
|
||||
// EnsureUserdataSkeleton creates the full userdata tree under a namespace root with the convention.
|
||||
// It creates ALL dirs even if one errors (so a single chown/chmod hiccup doesn't truncate the tree),
|
||||
// returning the first error seen for the caller to log.
|
||||
//
|
||||
// dirs is the merged, sorted set from BuildUserdataSkeleton. This function only ever CREATES: there
|
||||
// is no removal path here or anywhere in R-75, so a directory the current catalog no longer implies
|
||||
// simply stays where it is (Scenario D).
|
||||
func EnsureUserdataSkeleton(nsRoot string, dirs []string) error {
|
||||
base := UserdataDir(nsRoot)
|
||||
var firstErr error
|
||||
rec := func(e error) {
|
||||
if e != nil && firstErr == nil {
|
||||
firstErr = e
|
||||
}
|
||||
}
|
||||
rec(EnsureUserdataDir(base))
|
||||
for _, sub := range dirs {
|
||||
rec(EnsureUserdataDir(filepath.Join(base, sub)))
|
||||
}
|
||||
return firstErr
|
||||
}
|
||||
@@ -1,20 +0,0 @@
|
||||
//go:build linux
|
||||
|
||||
package appbackup
|
||||
|
||||
import (
|
||||
"os"
|
||||
"syscall"
|
||||
)
|
||||
|
||||
// chownGID sets the GROUP of path (owner unchanged via -1). Setting an arbitrary group requires
|
||||
// CAP_CHOWN; the in-guest controller runs as root, so this succeeds in production.
|
||||
func chownGID(path string, gid int) error { return os.Chown(path, -1, gid) }
|
||||
|
||||
// StatGID returns the owning GID of fi (Linux). ok=false when the underlying stat is unavailable.
|
||||
func StatGID(fi os.FileInfo) (int, bool) {
|
||||
if st, ok := fi.Sys().(*syscall.Stat_t); ok {
|
||||
return int(st.Gid), true
|
||||
}
|
||||
return -1, false
|
||||
}
|
||||
@@ -1,11 +0,0 @@
|
||||
//go:build !linux
|
||||
|
||||
package appbackup
|
||||
|
||||
import "os"
|
||||
|
||||
// chownGID is a no-op off Linux (the dev machine has no POSIX group ownership). Production is Linux.
|
||||
func chownGID(path string, gid int) error { return nil }
|
||||
|
||||
// StatGID is unavailable off Linux.
|
||||
func StatGID(fi os.FileInfo) (int, bool) { return -1, false }
|
||||
@@ -1,43 +0,0 @@
|
||||
//go:build linux
|
||||
|
||||
package appbackup
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// TestEnsureDirOwned_Setgid is the load-bearing assertion: EnsureDirOwned produces a dir with the
|
||||
// SETGID bit + group-rwx (mode 02775) and the requested group. Uses the test's own gid so the chown
|
||||
// succeeds without root. Companion: a plain MkdirAll(0755) does NOT get setgid — proving the explicit
|
||||
// Chmod is what sets it (the spike's collision fix). This test FAILS on a pre-fix MkdirAll-only impl.
|
||||
func TestEnsureDirOwned_Setgid(t *testing.T) {
|
||||
gid := os.Getgid()
|
||||
dir := filepath.Join(t.TempDir(), "userdata", "media", "movies")
|
||||
if err := EnsureDirOwned(dir, gid); err != nil {
|
||||
t.Fatalf("EnsureDirOwned: %v", err)
|
||||
}
|
||||
fi, err := os.Stat(dir)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if fi.Mode()&os.ModeSetgid == 0 {
|
||||
t.Errorf("dir is missing the setgid bit: mode=%v", fi.Mode())
|
||||
}
|
||||
if perm := fi.Mode().Perm(); perm != 0o775 {
|
||||
t.Errorf("dir perm = %o, want 0775", perm)
|
||||
}
|
||||
if g, ok := StatGID(fi); !ok || g != gid {
|
||||
t.Errorf("dir gid = %d (ok=%v), want %d", g, ok, gid)
|
||||
}
|
||||
|
||||
// Companion: the pre-fix behaviour (MkdirAll only, no explicit setgid Chmod) → NO setgid.
|
||||
plain := filepath.Join(t.TempDir(), "plain")
|
||||
if err := os.MkdirAll(plain, 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if pfi, _ := os.Stat(plain); pfi.Mode()&os.ModeSetgid != 0 {
|
||||
t.Errorf("plain MkdirAll unexpectedly has setgid — the explicit Chmod is not load-bearing")
|
||||
}
|
||||
}
|
||||
@@ -1,63 +0,0 @@
|
||||
package appbackup
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// TestSharedContentGID pins the shared content group to 1000 (FileBrowser's gid + the apps' PUID/PGID).
|
||||
func TestSharedContentGID(t *testing.T) {
|
||||
if SharedContentGID != 1000 {
|
||||
t.Errorf("SharedContentGID = %d, want 1000", SharedContentGID)
|
||||
}
|
||||
}
|
||||
|
||||
// TestUserdataSkeleton_List asserts the locked carry-list. R-75 renamed the hardcoded list to
|
||||
// UserdataSkeletonCarry (it is now the non-derived carry-list) and DELIBERATELY dropped the three
|
||||
// `import*` entries: carrying them would re-create a per-drive drop-zone on every drive forever, the
|
||||
// dead lookalike the canonical root exists to remove. That is not a removal — nothing deletes the
|
||||
// dirs an existing box has; they stop being maintained and stop appearing on fresh boxes. Every other
|
||||
// entry is unchanged, which is the zero-removals promise.
|
||||
func TestUserdataSkeleton_List(t *testing.T) {
|
||||
got := map[string]bool{}
|
||||
for _, s := range UserdataSkeletonCarry() {
|
||||
got[s] = true
|
||||
}
|
||||
for _, want := range []string{
|
||||
"media/movies", "media/tv", "media/music", "media/audiobooks", "media/books",
|
||||
"media/comics", "media/photos", "downloads",
|
||||
"roms", "documents",
|
||||
} {
|
||||
if !got[want] {
|
||||
t.Errorf("skeleton missing %q", want)
|
||||
}
|
||||
}
|
||||
for _, gone := range []string{"import", "import/paperless", "import/calibre"} {
|
||||
if got[gone] {
|
||||
t.Errorf("carry-list must NOT hold %q — the drop-zone is canonical on the system drive (R-75)", gone)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestUserdataDir confirms the userdata root is a sibling under the namespace.
|
||||
func TestUserdataDir(t *testing.T) {
|
||||
if got := UserdataDir("/mnt/felhom-usb"); got != filepath.Clean("/mnt/felhom-usb/userdata") {
|
||||
t.Errorf("UserdataDir = %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
// TestEnsureUserdataSkeleton_Structure: every skeleton dir is created (chown may fail off-root, which
|
||||
// is ignored — dirs + setgid still land). Runs cross-platform.
|
||||
func TestEnsureUserdataSkeleton_Structure(t *testing.T) {
|
||||
ns := t.TempDir()
|
||||
dirs := BuildUserdataSkeleton(nil) // no catalog derived → the carry-list floor
|
||||
_ = EnsureUserdataSkeleton(ns, dirs) // ignore chown error on a non-root CI host
|
||||
base := UserdataDir(ns)
|
||||
for _, sub := range append([]string{""}, dirs...) {
|
||||
p := filepath.Join(base, sub)
|
||||
if fi, err := os.Stat(p); err != nil || !fi.IsDir() {
|
||||
t.Errorf("skeleton dir missing: %s (%v)", p, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -15,8 +15,8 @@ import (
|
||||
)
|
||||
|
||||
const (
|
||||
magicHeader = "FABE" // Felhom App Bundle Encrypted
|
||||
scryptN = 1 << 15 // 32768
|
||||
magicHeader = "FABE" // Felhom App Bundle Encrypted
|
||||
scryptN = 1 << 15 // 32768
|
||||
scryptR = 8
|
||||
scryptP = 1
|
||||
saltSize = 32
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
package appexport
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"fmt"
|
||||
"os"
|
||||
@@ -23,31 +22,6 @@ type ExportEstimate struct {
|
||||
DestFreeBytes int64 `json:"dest_free_bytes"`
|
||||
DestFreeHuman string `json:"dest_free_human"`
|
||||
FitsOnDest bool `json:"fits_on_dest"`
|
||||
// SizeUnknown is set (v0.129.0 F-A) when a volume's size could not be read (docker helper
|
||||
// failed). When true, DataSizeBytes is a partial/understated sum and FitsOnDest is FORCED false
|
||||
// — a failed read must NEVER render as "fits". The UI shows "ismeretlen méret".
|
||||
SizeUnknown bool `json:"size_unknown"`
|
||||
|
||||
// Task 4 class split (classified apps only; empty for legacy — existing fields above are
|
||||
// unchanged, so old JSON consumers keep working). BaseBytes = config + DB + volumes + mandatory
|
||||
// (always in the bundle). OptionalItems are pre-selected, ExcludedItems are opt-in — each carries
|
||||
// its own size so the UI recomputes the total client-side per checkbox toggle (no extra du calls).
|
||||
HasClassification bool `json:"has_classification"`
|
||||
BaseBytes int64 `json:"base_bytes"`
|
||||
BaseHuman string `json:"base_human"`
|
||||
MandatoryItems []FabItem `json:"mandatory_items,omitempty"`
|
||||
OptionalItems []FabItem `json:"optional_items,omitempty"`
|
||||
ExcludedItems []FabItem `json:"excluded_items,omitempty"`
|
||||
}
|
||||
|
||||
// FabItem is one class-scoped path in the `.fab` selection UI: Key is the DeselectOptional/OptInExcluded
|
||||
// value ("root/rel"), RelPath is the display path, Bytes/Human its du size.
|
||||
type FabItem struct {
|
||||
Key string `json:"key"`
|
||||
Root string `json:"root"`
|
||||
RelPath string `json:"rel_path"`
|
||||
Bytes int64 `json:"bytes"`
|
||||
Human string `json:"human"`
|
||||
}
|
||||
|
||||
// EstimateExport calculates size estimates for an app export.
|
||||
@@ -65,8 +39,7 @@ func (e *Exporter) EstimateExport(stackName, destDrive string) (*ExportEstimate,
|
||||
est.ConfigSizeHuman = humanizeBytes(est.ConfigSizeBytes)
|
||||
e.debugf("EstimateExport: configSize=%s (%d bytes)", est.ConfigSizeHuman, est.ConfigSizeBytes)
|
||||
|
||||
// Data size: HDD bind mounts PLUS Docker volumes. v0.130.0 (C6B-F1): additive, mirroring the
|
||||
// export itself — a needs_hdd app bundles BOTH, so the fits-on-dest gate must count both.
|
||||
// Data size: HDD bind mounts or Docker volumes
|
||||
if e.provider.GetStackNeedsHDD(stackName) {
|
||||
mounts := e.provider.GetStackHDDMounts(stackName)
|
||||
e.debugf("EstimateExport: HDD mounts: %v", mounts)
|
||||
@@ -75,36 +48,20 @@ func (e *Exporter) EstimateExport(stackName, destDrive string) (*ExportEstimate,
|
||||
e.debugf("EstimateExport: mount %s = %s", mount, humanizeBytes(mountSize))
|
||||
est.DataSizeBytes += mountSize
|
||||
}
|
||||
}
|
||||
volumes := e.provider.GetDockerVolumes(stackName)
|
||||
e.debugf("EstimateExport: Docker volumes: %v", volumes)
|
||||
var volumeBytes int64
|
||||
for _, vol := range volumes {
|
||||
volSize, err := volumeSizer(vol)
|
||||
if err != nil {
|
||||
// F-A: the controller runs containerized, so a failed helper read must not
|
||||
// silently become 0-that-reads-as-fits. Mark unknown and keep going.
|
||||
e.logger.Printf("[WARN] appexport: volume size unknown for %s: %v", vol, err)
|
||||
est.SizeUnknown = true
|
||||
continue
|
||||
}
|
||||
e.debugf("EstimateExport: volume %s = %s", vol, humanizeBytes(volSize))
|
||||
est.DataSizeBytes += volSize
|
||||
volumeBytes += volSize
|
||||
}
|
||||
if est.SizeUnknown {
|
||||
est.DataSizeHuman = "ismeretlen méret"
|
||||
} else {
|
||||
est.DataSizeHuman = humanizeBytes(est.DataSizeBytes)
|
||||
volumes := e.provider.GetDockerVolumes(stackName)
|
||||
e.debugf("EstimateExport: Docker volumes: %v", volumes)
|
||||
for _, vol := range volumes {
|
||||
volSize := dockerVolumeSize(vol)
|
||||
e.debugf("EstimateExport: volume %s = %s", vol, humanizeBytes(volSize))
|
||||
est.DataSizeBytes += volSize
|
||||
}
|
||||
}
|
||||
est.DataSizeHuman = humanizeBytes(est.DataSizeBytes)
|
||||
|
||||
est.TotalSizeBytes = est.ConfigSizeBytes + est.DataSizeBytes
|
||||
est.TotalSizeHuman = humanizeBytes(est.TotalSizeBytes)
|
||||
|
||||
// Task 4: the class split (classified apps only). Independent du over each bucket path — additive,
|
||||
// never touches the fields/fits gate above.
|
||||
e.fabEstimateSplit(stackName, est, volumeBytes)
|
||||
|
||||
// Rough time estimate: ~500 MB/min for HDDs, minimum 1 minute
|
||||
minutes := int(est.TotalSizeBytes / (500 * 1024 * 1024))
|
||||
if minutes < 1 {
|
||||
@@ -115,13 +72,12 @@ func (e *Exporter) EstimateExport(stackName, destDrive string) (*ExportEstimate,
|
||||
// Destination free space
|
||||
exportDir := ExportDir(destDrive)
|
||||
os.MkdirAll(exportDir, 0755)
|
||||
est.DestFreeBytes = DiskFree(exportDir)
|
||||
est.DestFreeBytes = diskFree(exportDir)
|
||||
est.DestFreeHuman = humanizeBytes(est.DestFreeBytes)
|
||||
|
||||
// Need ~10% overhead for tar.gz metadata + compression margin. F-A: a size we could not read
|
||||
// must never render as "fits" — an unknown-size estimate is conservatively not-fits.
|
||||
// Need ~10% overhead for tar.gz metadata + compression margin
|
||||
needed := est.TotalSizeBytes + est.TotalSizeBytes/10
|
||||
est.FitsOnDest = !est.SizeUnknown && est.DestFreeBytes >= needed
|
||||
est.FitsOnDest = est.DestFreeBytes >= needed
|
||||
|
||||
e.debugf("EstimateExport: total=%s free=%s fits=%v needed=%s minutes=%d",
|
||||
est.TotalSizeHuman, est.DestFreeHuman, est.FitsOnDest, humanizeBytes(needed), est.EstimatedMinutes)
|
||||
@@ -162,38 +118,25 @@ func duBytes(path string) int64 {
|
||||
return size
|
||||
}
|
||||
|
||||
// volumeSizer returns the byte size of a named Docker volume as seen from a CONTAINER view.
|
||||
// Package var so unit tests inject a fake (returning a known size or an error) without shelling out
|
||||
// to real docker. F-A (v0.129.0): the old dockerVolumeSize `du`d the host mountpoint from
|
||||
// `docker volume inspect`, which is NOT visible inside the containerized controller → always 0.
|
||||
var volumeSizer = realVolumeSize
|
||||
|
||||
// realVolumeSize `du -sb`s the volume mounted read-only into a throwaway helper container — the same
|
||||
// container-view pattern the export path uses (appexport/export.go withVolumeHelper). It mounts the
|
||||
// NAMED VOLUME by name (never a controller-host path — the v0.125.0 strand class). Returns an error
|
||||
// on any failure; callers treat that as "unknown size", never as 0.
|
||||
func realVolumeSize(volumeName string) (int64, error) {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
||||
// dockerVolumeSize estimates the size of a Docker named volume.
|
||||
func dockerVolumeSize(volumeName string) int64 {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||
defer cancel()
|
||||
var out bytes.Buffer
|
||||
stderr, err := dockerExec(ctx, nil, &out, "run", "--rm", "-v", volumeName+":/vol:ro", "alpine", "du", "-sb", "/vol")
|
||||
// Use docker system df -v and parse, or inspect the volume mount path
|
||||
out, err := exec.CommandContext(ctx, "docker", "volume", "inspect",
|
||||
"--format", "{{.Mountpoint}}", volumeName).Output()
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("sizing volume %s: %s: %w", volumeName, stderr, err)
|
||||
return 0
|
||||
}
|
||||
fields := strings.Fields(out.String())
|
||||
if len(fields) == 0 {
|
||||
return 0, fmt.Errorf("sizing volume %s: empty du output", volumeName)
|
||||
mountpoint := strings.TrimSpace(string(out))
|
||||
if mountpoint == "" {
|
||||
return 0
|
||||
}
|
||||
var size int64
|
||||
if _, err := fmt.Sscanf(fields[0], "%d", &size); err != nil {
|
||||
return 0, fmt.Errorf("sizing volume %s: parse %q: %w", volumeName, fields[0], err)
|
||||
}
|
||||
return size, nil
|
||||
return duBytes(mountpoint)
|
||||
}
|
||||
|
||||
// DiskFree returns available bytes on the filesystem containing path (0 on any error).
|
||||
// Exported since v0.128.0 — the browser-upload space gate reuses it via a web-package seam.
|
||||
func DiskFree(path string) int64 {
|
||||
// diskFree returns available bytes on the filesystem containing path.
|
||||
func diskFree(path string) int64 {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
out, err := exec.CommandContext(ctx, "df", "--output=avail", "-B1", path).Output()
|
||||
|
||||
@@ -1,112 +0,0 @@
|
||||
package appexport
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"io"
|
||||
"log"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/appbackup"
|
||||
)
|
||||
|
||||
// hddProvider is an rtProvider that reports an HDD-backed stack (estimate scenario H + the
|
||||
// v0.130.0 additive-export tests in export_additive_test.go).
|
||||
type hddProvider struct {
|
||||
*rtProvider
|
||||
mounts []string
|
||||
hddPath string
|
||||
binds []appbackup.ClassifiedBind
|
||||
hasBinds bool
|
||||
}
|
||||
|
||||
func (p *hddProvider) GetStackNeedsHDD(string) bool { return true }
|
||||
func (p *hddProvider) GetStackHDDMounts(string) []string { return p.mounts }
|
||||
func (p *hddProvider) GetStackHDDPath(string) string { return p.hddPath }
|
||||
func (p *hddProvider) GetImportRoot() string { return "" } // R-75: no import binds in this fixture
|
||||
|
||||
// R-203: these fixtures use ENROLLED drive paths, where the namespace root IS the drive path.
|
||||
// Delegating keeps that identity explicit rather than hardcoding it.
|
||||
func (p *hddProvider) GetStackNamespaceRoot(name string) string { return p.GetStackHDDPath(name) }
|
||||
func (p *hddProvider) GetStackClassifiedBinds(string) ([]appbackup.ClassifiedBind, bool) {
|
||||
return p.binds, p.hasBinds
|
||||
}
|
||||
|
||||
func newEstimator(t *testing.T, provider ExportStackProvider) *Exporter {
|
||||
t.Helper()
|
||||
return NewExporter(provider, log.New(io.Discard, "", 0), "test")
|
||||
}
|
||||
|
||||
// Scenario F (the F-A fix): a volume-only app with a >1 GiB volume reports the REAL size via the
|
||||
// container-view sizer — not 0/"3.6 KB". This is the F-A red-proof anchor (revert EstimateExport to
|
||||
// dockerVolumeSize → reads 0).
|
||||
func TestEstimate_VolumeSize_RealNotZero(t *testing.T) {
|
||||
const twoGiB = int64(2) << 30
|
||||
orig := volumeSizer
|
||||
volumeSizer = func(vol string) (int64, error) { return twoGiB, nil }
|
||||
defer func() { volumeSizer = orig }()
|
||||
|
||||
e := newEstimator(t, &rtProvider{stackDir: t.TempDir(), volumes: []string{"app_data"}})
|
||||
est, err := e.EstimateExport("app", t.TempDir())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if est.SizeUnknown {
|
||||
t.Fatalf("size must be known when the sizer succeeds")
|
||||
}
|
||||
if est.DataSizeBytes != twoGiB {
|
||||
t.Fatalf("DataSizeBytes = %d, want %d (WRONG would be 0 — the F-A bug)", est.DataSizeBytes, twoGiB)
|
||||
}
|
||||
if !strings.Contains(est.DataSizeHuman, "GB") {
|
||||
t.Fatalf("DataSizeHuman = %q, want GB-scale (WRONG would be \"3.6 KB\")", est.DataSizeHuman)
|
||||
}
|
||||
}
|
||||
|
||||
// Scenario G: a failed volume read must never render as "fits". Size is marked unknown, the human
|
||||
// string says so, and FitsOnDest is forced false.
|
||||
func TestEstimate_VolumeSize_FailureNeverFits(t *testing.T) {
|
||||
orig := volumeSizer
|
||||
volumeSizer = func(vol string) (int64, error) { return 0, errors.New("docker: no such image") }
|
||||
defer func() { volumeSizer = orig }()
|
||||
|
||||
e := newEstimator(t, &rtProvider{stackDir: t.TempDir(), volumes: []string{"app_data"}})
|
||||
est, err := e.EstimateExport("app", t.TempDir())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !est.SizeUnknown {
|
||||
t.Fatalf("a failed volume read must set SizeUnknown")
|
||||
}
|
||||
if est.FitsOnDest {
|
||||
t.Fatalf("an unknown size must NEVER render as fits_on_dest:true")
|
||||
}
|
||||
if est.DataSizeHuman != "ismeretlen méret" {
|
||||
t.Fatalf("DataSizeHuman = %q, want \"ismeretlen méret\"", est.DataSizeHuman)
|
||||
}
|
||||
if est.DataSizeBytes != 0 {
|
||||
t.Fatalf("no successful read → DataSizeBytes should be 0, got %d", est.DataSizeBytes)
|
||||
}
|
||||
}
|
||||
|
||||
// Scenario H (regression): an HDD-backed stack must NOT touch the new volume sizer — the HDD branch
|
||||
// (duBytes on the mounted /mnt path) is unchanged. Platform-independent: assert the seam is not
|
||||
// invoked and SizeUnknown stays false.
|
||||
func TestEstimate_HDDPath_DoesNotUseVolumeSizer(t *testing.T) {
|
||||
called := false
|
||||
orig := volumeSizer
|
||||
volumeSizer = func(vol string) (int64, error) { called = true; return 0, nil }
|
||||
defer func() { volumeSizer = orig }()
|
||||
|
||||
p := &hddProvider{rtProvider: &rtProvider{stackDir: t.TempDir()}, mounts: []string{t.TempDir()}}
|
||||
e := newEstimator(t, p)
|
||||
est, err := e.EstimateExport("app", t.TempDir())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if called {
|
||||
t.Fatalf("HDD-backed stack must not call the docker volume sizer")
|
||||
}
|
||||
if est.SizeUnknown {
|
||||
t.Fatalf("HDD branch must not set SizeUnknown")
|
||||
}
|
||||
}
|
||||
@@ -2,7 +2,6 @@ package appexport
|
||||
|
||||
import (
|
||||
"archive/tar"
|
||||
"bytes"
|
||||
"compress/gzip"
|
||||
"context"
|
||||
"fmt"
|
||||
@@ -75,14 +74,6 @@ type ExportRequest struct {
|
||||
DestDrive string // drive mount path (e.g., "/mnt/hdd_1")
|
||||
Password string // empty = no encryption
|
||||
StopApp bool // stop app before export
|
||||
|
||||
// `.fab` class-scoped selection (Task 4; classified apps only — legacy apps ignore these). Both
|
||||
// empty = ruling #1 defaults (mandatory in, optional in, excluded out). Values are "root/rel" keys
|
||||
// (matching a CapturePath: "hdd/appdata/x" | "userdata/media/y"). The server enforces the floor:
|
||||
// a DeselectOptional entry naming a MANDATORY path is ignored with a WARN (the client cannot weaken
|
||||
// the mandatory floor). OptInExcluded pulls an excluded bind into the bundle.
|
||||
DeselectOptional []string
|
||||
OptInExcluded []string
|
||||
}
|
||||
|
||||
// Exporter manages app export/import operations.
|
||||
@@ -92,30 +83,10 @@ type Exporter struct {
|
||||
version string
|
||||
debug bool
|
||||
|
||||
// dirLister (Task 4) lists child DIR names of a path — the seam the `.fab` userdata-exclude
|
||||
// computation walks. Nil → the real os.ReadDir-based lister.
|
||||
dirLister func(dir string) []string
|
||||
|
||||
// stopGuard (R-166) marks the stop→export→start window so a controller killed inside it leaves a
|
||||
// durable record that the app is owed a restart. Declared consumer-side as a two-method interface
|
||||
// so this package does not import internal/backup; main.go passes the backup manager's guard, so
|
||||
// BOTH packages write ONE marker file — an exporter with its own file would be a second writer
|
||||
// racing the same recovery. Nil = not wired (tests): the export runs exactly as it did before.
|
||||
stopGuard appStopGuard
|
||||
|
||||
mu sync.Mutex
|
||||
activeJob *Job
|
||||
}
|
||||
|
||||
// appStopGuard is the app-stop crash-marker seam. The REASON is deliberately not a parameter: it is
|
||||
// always "app export" from here, and the adapter in main.go supplies it. Passing it as a string
|
||||
// would duplicate backup.ReasonAppExport's value in a second package with nothing keeping the two in
|
||||
// step — a drift this codebase has paid for before (the offbox key that was guessed, R-7b).
|
||||
type appStopGuard interface {
|
||||
Begin(opID string, stacks []string) error
|
||||
End()
|
||||
}
|
||||
|
||||
// NewExporter creates a new export/import engine.
|
||||
func NewExporter(provider ExportStackProvider, logger *log.Logger, version string) *Exporter {
|
||||
return &Exporter{
|
||||
@@ -125,18 +96,6 @@ func NewExporter(provider ExportStackProvider, logger *log.Logger, version strin
|
||||
}
|
||||
}
|
||||
|
||||
// SetStopGuard wires the app-stop crash marker. INIT-ONLY — call once at startup, before any export.
|
||||
func (e *Exporter) SetStopGuard(g appStopGuard) { e.stopGuard = g }
|
||||
|
||||
// stopGuardBegin records the app-stop marker before an export stops an app. An unwired guard is a
|
||||
// no-op (pre-v0.189.0 behaviour), never an error — a test exporter must not be forced to have one.
|
||||
func (e *Exporter) stopGuardBegin(stackName string) error {
|
||||
if e.stopGuard == nil {
|
||||
return nil
|
||||
}
|
||||
return e.stopGuard.Begin("app-export:"+stackName, []string{stackName})
|
||||
}
|
||||
|
||||
// SetDebug enables or disables verbose debug logging.
|
||||
func (e *Exporter) SetDebug(debug bool) {
|
||||
e.debug = debug
|
||||
@@ -237,14 +196,9 @@ func (e *Exporter) executeExport(req ExportRequest, job *Job) {
|
||||
if err != nil {
|
||||
e.debugf("estimate error (non-fatal): %v", err)
|
||||
} else {
|
||||
e.debugf("estimate: config=%s data=%s total=%s destFree=%s fits=%v unknown=%v",
|
||||
est.ConfigSizeHuman, est.DataSizeHuman, est.TotalSizeHuman, est.DestFreeHuman, est.FitsOnDest, est.SizeUnknown)
|
||||
// Hard-abort only on a KNOWN doesn't-fit. F-A: est.SizeUnknown forces FitsOnDest=false for
|
||||
// the UI honesty signal, but an unmeasured size must NOT block the export here — the tar
|
||||
// streaming and the destination filesystem surface a real ENOSPC if it genuinely won't fit.
|
||||
if est.SizeUnknown {
|
||||
e.logger.Printf("[WARN] appexport: export space pre-check skipped for %s — volume size unknown", req.StackName)
|
||||
} else if !est.FitsOnDest {
|
||||
e.debugf("estimate: config=%s data=%s total=%s destFree=%s fits=%v",
|
||||
est.ConfigSizeHuman, est.DataSizeHuman, est.TotalSizeHuman, est.DestFreeHuman, est.FitsOnDest)
|
||||
if !est.FitsOnDest {
|
||||
e.failJob(job, step, fmt.Sprintf("Nincs elég hely: szükséges ~%s, szabad %s",
|
||||
est.TotalSizeHuman, est.DestFreeHuman))
|
||||
return
|
||||
@@ -254,14 +208,6 @@ func (e *Exporter) executeExport(req ExportRequest, job *Job) {
|
||||
// Optionally stop the app
|
||||
wasRunning := false
|
||||
if req.StopApp && e.provider.IsStackRunning(req.StackName) {
|
||||
// R-166: mark BEFORE the stop. The defer below covers the graceful exits; it does NOT cover a
|
||||
// SIGKILL or a power cut, which run no deferred function (Campaign 8 fault 10, on live
|
||||
// hardware) — only this marker does, and a big export is a long window to be killed in.
|
||||
if err := e.stopGuardBegin(req.StackName); err != nil {
|
||||
e.failJob(job, step, "Az alkalmazás leállítása előtti jelölő nem menthető — az exportálás nem indult el.")
|
||||
e.logger.Printf("[ERROR] Export: could not record the app-stop marker for %s (refusing to stop it unprotected): %v", req.StackName, err)
|
||||
return
|
||||
}
|
||||
wasRunning = true
|
||||
e.logger.Printf("[INFO] Export: stopping %s", req.StackName)
|
||||
e.debugf("stopping stack %s before export", req.StackName)
|
||||
@@ -282,11 +228,6 @@ func (e *Exporter) executeExport(req ExportRequest, job *Job) {
|
||||
e.logger.Printf("[WARN] Export: could not restart %s: %v", req.StackName, err)
|
||||
} else {
|
||||
e.debugf("stack %s restarted successfully", req.StackName)
|
||||
// Cleared only on a restart that succeeded — a failed one keeps the marker so the
|
||||
// next startup retries.
|
||||
if e.stopGuard != nil {
|
||||
e.stopGuard.End()
|
||||
}
|
||||
}
|
||||
}()
|
||||
}
|
||||
@@ -377,24 +318,15 @@ func (e *Exporter) executeExport(req ExportRequest, job *Job) {
|
||||
dataDir := filepath.Join(tmpDir, "data")
|
||||
os.MkdirAll(dataDir, 0755)
|
||||
|
||||
// C6B-F1 cause 1 (v0.130.0): user-data capture is ADDITIVE, not either/or. A needs_hdd app
|
||||
// can hold state in BOTH its HDD binds and its named volumes (sonarr: ${USERDATA_PATH} media
|
||||
// binds + the sonarr_config volume with the entire app DB) — the old else-branch silently
|
||||
// dropped every named volume of every needs_hdd app.
|
||||
if e.provider.GetStackNeedsHDD(req.StackName) {
|
||||
e.debugf("exporting HDD data for %s", req.StackName)
|
||||
if err := e.exportHDDData(req, dataDir, manifest); err != nil {
|
||||
e.failJob(job, step, fmt.Sprintf("Felhasználói adatok mentése sikertelen: %v", err))
|
||||
return
|
||||
}
|
||||
e.exportHDDData(req.StackName, dataDir, manifest)
|
||||
e.debugf("HDD data exported: subdirs=%v hasData=%v", manifest.HDDSubdirs, manifest.HasHDDData)
|
||||
} else {
|
||||
e.debugf("exporting Docker volumes for %s", req.StackName)
|
||||
e.exportVolumeData(req.StackName, dataDir, manifest)
|
||||
e.debugf("volume data exported: volumes=%v hasData=%v", manifest.VolumeNames, manifest.HasVolumeData)
|
||||
}
|
||||
e.debugf("exporting Docker volumes for %s", req.StackName)
|
||||
if err := e.exportVolumeData(req.StackName, dataDir, manifest); err != nil {
|
||||
e.failJob(job, step, fmt.Sprintf("Kötet mentése sikertelen: %v", err))
|
||||
return
|
||||
}
|
||||
e.debugf("volume data exported: volumes=%v hasData=%v", manifest.VolumeNames, manifest.HasVolumeData)
|
||||
e.debugf("step 3 (user data) done in %v", time.Since(stepStart))
|
||||
|
||||
job.setStep(step, "done", "")
|
||||
@@ -404,13 +336,6 @@ func (e *Exporter) executeExport(req ExportRequest, job *Job) {
|
||||
job.setStep(step, "running", "")
|
||||
stepStart = time.Now()
|
||||
|
||||
// v0.125.0 fail-loud guard (scenario B): never package a bundle whose manifest claims data
|
||||
// that is not actually in the staging tree.
|
||||
if err := assertBundleDataComplete(tmpDir, manifest); err != nil {
|
||||
e.failJob(job, step, fmt.Sprintf("A csomag hiányos lenne — az export leállt: %v", err))
|
||||
return
|
||||
}
|
||||
|
||||
// Calculate total size
|
||||
manifest.TotalSizeBytes = calcDirSize(tmpDir)
|
||||
e.debugf("total bundle content size: %s (%d bytes)", humanizeBytes(manifest.TotalSizeBytes), manifest.TotalSizeBytes)
|
||||
@@ -509,8 +434,8 @@ func (e *Exporter) GetDebugInfo() map[string]interface{} {
|
||||
defer e.mu.Unlock()
|
||||
|
||||
info := map[string]interface{}{
|
||||
"debug_enabled": e.debug,
|
||||
"version": e.version,
|
||||
"debug_enabled": e.debug,
|
||||
"version": e.version,
|
||||
"has_active_job": e.activeJob != nil,
|
||||
}
|
||||
|
||||
@@ -585,7 +510,7 @@ func (e *Exporter) dumpDatabase(stackName, dbDir string, manifest *Manifest) boo
|
||||
defer cancel()
|
||||
|
||||
e.debugf("discovering databases (looking for stack %s)...", stackName)
|
||||
dbs, err := appbackup.DiscoverDatabases(ctx, e.logger, e.debug, nil)
|
||||
dbs, err := appbackup.DiscoverDatabases(ctx, e.logger, e.debug)
|
||||
if err != nil {
|
||||
e.logger.Printf("[WARN] Export: DB discovery error: %v", err)
|
||||
return false
|
||||
@@ -640,12 +565,8 @@ func (e *Exporter) dumpDatabase(stackName, dbDir string, manifest *Manifest) boo
|
||||
return true
|
||||
}
|
||||
|
||||
// exportHDDData copies HDD bind mount data for the export. v0.130.0 (C6B-F1 §8): a basename
|
||||
// collision between two mounts is a FATAL error (the manifest keys tars by basename; the old
|
||||
// code silently overwrote the first tar). A non-existent mount is still soft-skipped (honestly
|
||||
// absent from the manifest — the anti-hollow guard catches total emptiness).
|
||||
func (e *Exporter) exportHDDData(req ExportRequest, dataDir string, manifest *Manifest) error {
|
||||
stackName := req.StackName
|
||||
// exportHDDData copies HDD bind mount data for the export.
|
||||
func (e *Exporter) exportHDDData(stackName, dataDir string, manifest *Manifest) {
|
||||
hddDir := filepath.Join(dataDir, "hdd")
|
||||
os.MkdirAll(hddDir, 0755)
|
||||
|
||||
@@ -653,137 +574,33 @@ func (e *Exporter) exportHDDData(req ExportRequest, dataDir string, manifest *Ma
|
||||
e.debugf("HDD mounts for %s: %v (%d total)", stackName, mounts, len(mounts))
|
||||
if len(mounts) == 0 {
|
||||
e.debugf("no HDD mounts — skipping HDD data export")
|
||||
return nil
|
||||
return
|
||||
}
|
||||
|
||||
// Task 4: the class-scoped plan. Legacy / no-block apps get an EMPTY plan (all mounts kept, root
|
||||
// tar with zero excludes) → byte-identical v0.130.0 capture.
|
||||
plan := e.computeFabPlan(req, mounts)
|
||||
// R-203: a NAMESPACE ROOT, not the drive path (identical on an enrolled drive; one segment short
|
||||
// on the system-data fallback).
|
||||
ud := appbackup.UserdataDir(filepath.Clean(e.provider.GetStackNamespaceRoot(stackName)))
|
||||
|
||||
claimed := make(map[string]string) // subdir → mount that claimed it
|
||||
for _, mount := range mounts {
|
||||
if plan.SkipMounts[filepath.Clean(mount)] {
|
||||
e.debugf("HDD mount %s skipped — not selected (class-scoped plan)", mount)
|
||||
continue
|
||||
}
|
||||
isUserdataRoot := filepath.Clean(mount) == filepath.Clean(ud)
|
||||
if isUserdataRoot && plan.SkipUserdataTar {
|
||||
e.debugf("userdata root %s skipped — no selected userdata bind (Scenario B)", mount)
|
||||
continue
|
||||
}
|
||||
if _, err := os.Stat(mount); os.IsNotExist(err) {
|
||||
e.debugf("HDD mount %s does not exist — skipping", mount)
|
||||
continue
|
||||
}
|
||||
subdir := filepath.Base(mount)
|
||||
// C6B-F1 §8 (v0.130.0): the manifest keys HDD tars by BASENAME (the import side maps a
|
||||
// basename back to a path), so two mounts sharing a basename cannot round-trip — the old
|
||||
// code silently overwrote the first tar with the second (silent partial data loss).
|
||||
// Renaming can't help either (the import couldn't map the new name), so the only honest
|
||||
// outcome is a loud failure.
|
||||
if prev, dup := claimed[subdir]; dup {
|
||||
return fmt.Errorf("két adatkönyvtár azonos névvel végződik (%q: %s és %s) — a csomag nem tudná megkülönböztetni őket", subdir, prev, mount)
|
||||
}
|
||||
claimed[subdir] = mount
|
||||
manifest.HDDSubdirs = append(manifest.HDDSubdirs, subdir)
|
||||
|
||||
tarPath := filepath.Join(hddDir, subdir+".tar")
|
||||
var excludes []string
|
||||
if isUserdataRoot {
|
||||
excludes = plan.UserdataExcludeRels // R1-C: exclude-scoped root tar (empty for legacy)
|
||||
}
|
||||
e.debugf("tarring HDD mount: %s → %s (%d exclude(s))", mount, tarPath, len(excludes))
|
||||
e.debugf("tarring HDD mount: %s → %s", mount, tarPath)
|
||||
tarStart := time.Now()
|
||||
if err := tarDirectoryExcluding(mount, tarPath, excludes); err != nil {
|
||||
// v0.125.0: claim the subdir ONLY on success — a claimed-but-absent tar would trip
|
||||
// the packaging assertion; an honestly-skipped mount stays out of the manifest.
|
||||
e.logger.Printf("[WARN] Export: failed to tar %s (excluded from the bundle): %v", mount, err)
|
||||
os.Remove(tarPath)
|
||||
if err := tarDirectory(mount, tarPath); err != nil {
|
||||
e.logger.Printf("[WARN] Export: failed to tar %s: %v", mount, err)
|
||||
} else {
|
||||
manifest.HDDSubdirs = append(manifest.HDDSubdirs, subdir)
|
||||
if info, _ := os.Stat(tarPath); info != nil {
|
||||
e.debugf("HDD tar complete: %s (%s) in %v", subdir, humanizeBytes(info.Size()), time.Since(tarStart))
|
||||
}
|
||||
}
|
||||
}
|
||||
manifest.HasHDDData = len(manifest.HDDSubdirs) > 0
|
||||
return nil
|
||||
}
|
||||
|
||||
// dockerExec is the docker-CLI seam (v0.125.0): runs `docker args...` with optional
|
||||
// stdin/stdout STREAMING and returns captured stderr (truncated). The volume legs stream tars
|
||||
// over the docker API (docker cp) — NEVER via `docker run -v <controller-path>` host mounts,
|
||||
// which the daemon resolves against the GUEST filesystem and silently strands the tar when the
|
||||
// controller itself runs containerized (the v0.124.0 HIGH finding). Package var so unit tests
|
||||
// inject a recorder (no docker on test boxes).
|
||||
var dockerExec = func(ctx context.Context, stdin io.Reader, stdout io.Writer, args ...string) (string, error) {
|
||||
cmd := exec.CommandContext(ctx, "docker", args...)
|
||||
cmd.Stdin = stdin
|
||||
cmd.Stdout = stdout
|
||||
var errBuf bytes.Buffer
|
||||
cmd.Stderr = &errBuf
|
||||
err := cmd.Run()
|
||||
stderr := strings.TrimSpace(errBuf.String())
|
||||
if len(stderr) > 500 {
|
||||
stderr = stderr[:500] + "..."
|
||||
}
|
||||
return stderr, err
|
||||
}
|
||||
|
||||
// withVolumeHelper creates a stopped helper container pinning volName at /vol, runs fn(cid),
|
||||
// and ALWAYS force-removes the helper — including on fn failure (no leaked alpine containers).
|
||||
func (e *Exporter) withVolumeHelper(volName string, fn func(cid string) error) error {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
||||
var cidBuf bytes.Buffer
|
||||
stderr, err := dockerExec(ctx, nil, &cidBuf, "create", "-v", volName+":/vol", "alpine", "true")
|
||||
cancel()
|
||||
if err != nil {
|
||||
return fmt.Errorf("creating helper container for volume %s: %s — %w", volName, stderr, err)
|
||||
}
|
||||
cid := strings.TrimSpace(cidBuf.String())
|
||||
defer func() {
|
||||
rmCtx, rmCancel := context.WithTimeout(context.Background(), 30*time.Second)
|
||||
defer rmCancel()
|
||||
if _, rmErr := dockerExec(rmCtx, nil, nil, "rm", "-f", cid); rmErr != nil {
|
||||
e.logger.Printf("[WARN] appexport: helper container %s cleanup failed: %v", cid, rmErr)
|
||||
}
|
||||
}()
|
||||
return fn(cid)
|
||||
}
|
||||
|
||||
// exportVolumeTar streams one volume's content into tarPath via `docker cp <cid>:/vol/. -`
|
||||
// (tar on stdout — zero shared paths; live-probed 2026-07-13: content, subdirs, symlinks,
|
||||
// empty files and uid/gid all round-trip).
|
||||
func (e *Exporter) exportVolumeTar(volName, tarPath string) error {
|
||||
return e.withVolumeHelper(volName, func(cid string) error {
|
||||
f, err := os.Create(tarPath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("creating %s: %w", tarPath, err)
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Minute)
|
||||
defer cancel()
|
||||
stderr, err := dockerExec(ctx, nil, f, "cp", cid+":/vol/.", "-")
|
||||
closeErr := f.Close()
|
||||
if err != nil {
|
||||
os.Remove(tarPath)
|
||||
return fmt.Errorf("streaming volume %s: %s — %w", volName, stderr, err)
|
||||
}
|
||||
if closeErr != nil {
|
||||
os.Remove(tarPath)
|
||||
return fmt.Errorf("flushing %s: %w", tarPath, closeErr)
|
||||
}
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
// exportVolumeData exports the app's Docker named volumes. v0.130.0 (C6B-F1): runs for EVERY
|
||||
// app — needs_hdd apps hold state in named volumes too (sonarr_config = the whole app DB); the
|
||||
// pre-fix else-branch silently dropped them. v0.125.0: a failed volume export is FATAL (export
|
||||
// must never report success on a hollow bundle — the pre-fix WARN+continue is exactly how the
|
||||
// data-loss bundles were born).
|
||||
func (e *Exporter) exportVolumeData(stackName, dataDir string, manifest *Manifest) error {
|
||||
// exportVolumeData exports Docker named volumes for apps without HDD storage.
|
||||
func (e *Exporter) exportVolumeData(stackName, dataDir string, manifest *Manifest) {
|
||||
volDir := filepath.Join(dataDir, "volumes")
|
||||
os.MkdirAll(volDir, 0755)
|
||||
|
||||
@@ -791,15 +608,28 @@ func (e *Exporter) exportVolumeData(stackName, dataDir string, manifest *Manifes
|
||||
e.debugf("Docker volumes for %s: %v (%d total)", stackName, volumes, len(volumes))
|
||||
if len(volumes) == 0 {
|
||||
e.debugf("no Docker volumes — skipping volume data export")
|
||||
return nil
|
||||
return
|
||||
}
|
||||
|
||||
for _, volName := range volumes {
|
||||
tarPath := filepath.Join(volDir, volName+".tar")
|
||||
e.debugf("exporting volume %s via docker cp streaming...", volName)
|
||||
e.debugf("exporting volume %s via docker run alpine tar...", volName)
|
||||
volStart := time.Now()
|
||||
if err := e.exportVolumeTar(volName, tarPath); err != nil {
|
||||
return fmt.Errorf("volume %s export failed: %w", volName, err)
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Minute)
|
||||
cmd := exec.CommandContext(ctx, "docker", "run", "--rm",
|
||||
"-v", volName+":/vol:ro",
|
||||
"-v", volDir+":/out",
|
||||
"alpine", "tar", "cf", "/out/"+volName+".tar", "-C", "/vol", ".")
|
||||
out, err := cmd.CombinedOutput()
|
||||
cancel()
|
||||
|
||||
if err != nil {
|
||||
e.logger.Printf("[WARN] Export: volume %s export failed: %s — %v",
|
||||
volName, strings.TrimSpace(string(out)), err)
|
||||
e.debugf("volume %s export failed: %s", volName, strings.TrimSpace(string(out)))
|
||||
os.Remove(tarPath)
|
||||
continue
|
||||
}
|
||||
if info, _ := os.Stat(tarPath); info != nil {
|
||||
e.debugf("volume %s exported: %s in %v", volName, humanizeBytes(info.Size()), time.Since(volStart))
|
||||
@@ -807,36 +637,6 @@ func (e *Exporter) exportVolumeData(stackName, dataDir string, manifest *Manifes
|
||||
manifest.VolumeNames = append(manifest.VolumeNames, volName)
|
||||
}
|
||||
manifest.HasVolumeData = len(manifest.VolumeNames) > 0
|
||||
return nil
|
||||
}
|
||||
|
||||
// assertBundleDataComplete is the fail-loud post-export guard (v0.125.0, scenario B): every
|
||||
// manifest-CLAIMED data tar must exist non-empty in the staging tree before packaging. A
|
||||
// mismatch aborts the export — yesterday's outcome ("success" with a hollow bundle) is the
|
||||
// one this exists to make impossible. v0.130.0 (C6B-F1 cause 3): also refuses a needs_hdd
|
||||
// bundle that claims NO data at all — the claimed-tar checks pass trivially on 0 claims, which
|
||||
// is how a discovery gap shipped hollow bundles right past the v0.125.0 net.
|
||||
func assertBundleDataComplete(tmpDir string, manifest *Manifest) error {
|
||||
for _, v := range manifest.VolumeNames {
|
||||
fi, err := os.Stat(filepath.Join(tmpDir, "data", "volumes", v+".tar"))
|
||||
if err != nil || fi.Size() == 0 {
|
||||
return fmt.Errorf("bundle assertion: volume %q is claimed by the manifest but its tar is missing or empty", v)
|
||||
}
|
||||
}
|
||||
for _, s := range manifest.HDDSubdirs {
|
||||
fi, err := os.Stat(filepath.Join(tmpDir, "data", "hdd", s+".tar"))
|
||||
if err != nil || fi.Size() == 0 {
|
||||
return fmt.Errorf("bundle assertion: HDD subdir %q is claimed by the manifest but its tar is missing or empty", s)
|
||||
}
|
||||
}
|
||||
// C6B-F1 cause 3 (v0.130.0): the claimed-tar checks above pass TRIVIALLY when discovery finds
|
||||
// nothing (0 claims → 0 checks) — exactly how a 4.17 GB app shipped as a 2308-byte config-only
|
||||
// bundle. A needs_hdd app with NO data of any kind is a hollow bundle by definition; refuse it
|
||||
// loudly so a future discovery gap can never again ship silently.
|
||||
if manifest.NeedsHDD && !manifest.HasHDDData && !manifest.HasVolumeData {
|
||||
return fmt.Errorf("a mentés nem tartalmaz alkalmazásadatot (0 adatkönyvtár, 0 kötet egy adattárolós alkalmazásnál)")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// createTarGz creates a gzipped tar archive of a directory.
|
||||
@@ -891,10 +691,51 @@ func createTarGz(outputPath, sourceDir string) error {
|
||||
})
|
||||
}
|
||||
|
||||
// tarDirectory creates a tar (not gzipped) of a directory's contents. Thin wrapper over
|
||||
// tarDirectoryExcluding (Task 4) with no excludes — its existing callers are unchanged.
|
||||
// tarDirectory creates a tar (not gzipped) of a directory's contents.
|
||||
func tarDirectory(sourceDir, outputPath string) error {
|
||||
return tarDirectoryExcluding(sourceDir, outputPath, nil)
|
||||
outFile, err := os.Create(outputPath)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer outFile.Close()
|
||||
|
||||
tw := tar.NewWriter(outFile)
|
||||
defer tw.Close()
|
||||
|
||||
return filepath.Walk(sourceDir, func(path string, info os.FileInfo, err error) error {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
relPath, err := filepath.Rel(sourceDir, path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if relPath == "." {
|
||||
return nil
|
||||
}
|
||||
|
||||
header, err := tar.FileInfoHeader(info, "")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
header.Name = relPath
|
||||
|
||||
if err := tw.WriteHeader(header); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if info.IsDir() {
|
||||
return nil
|
||||
}
|
||||
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer f.Close()
|
||||
_, err = io.Copy(tw, f)
|
||||
return err
|
||||
})
|
||||
}
|
||||
|
||||
// gzipFile compresses a file with gzip.
|
||||
|
||||
@@ -1,306 +0,0 @@
|
||||
package appexport
|
||||
|
||||
import (
|
||||
"archive/tar"
|
||||
"compress/gzip"
|
||||
"fmt"
|
||||
"io"
|
||||
"log"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// C6B-F1 (v0.130.0) — the additive-export tests. A needs_hdd app bundles BOTH its HDD mounts
|
||||
// and its named volumes (the old either/or dropped every needs_hdd app's volumes); a basename
|
||||
// collision between mounts fails LOUDLY (the old code silently overwrote the first tar); the
|
||||
// HDD round-trip places a "userdata" tar back at <HDD_PATH>/userdata through the untouched
|
||||
// import mapping.
|
||||
|
||||
// listFabEntries returns the entry names inside an unencrypted .fab (tar.gz).
|
||||
func listFabEntries(t *testing.T, fabPath string) map[string]int64 {
|
||||
t.Helper()
|
||||
f, err := os.Open(fabPath)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer f.Close()
|
||||
gz, err := gzip.NewReader(f)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
tr := tar.NewReader(gz)
|
||||
entries := map[string]int64{}
|
||||
for {
|
||||
hdr, err := tr.Next()
|
||||
if err == io.EOF {
|
||||
break
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
entries[filepath.ToSlash(hdr.Name)] = hdr.Size
|
||||
}
|
||||
return entries
|
||||
}
|
||||
|
||||
func findFab(t *testing.T, drive string) string {
|
||||
t.Helper()
|
||||
entries, _ := os.ReadDir(ExportDir(drive))
|
||||
for _, en := range entries {
|
||||
if strings.HasSuffix(en.Name(), ".fab") {
|
||||
return filepath.Join(ExportDir(drive), en.Name())
|
||||
}
|
||||
}
|
||||
t.Fatal("no .fab produced")
|
||||
return ""
|
||||
}
|
||||
|
||||
// Scenario A (§7): the sonarr shape — a needs_hdd app with a populated userdata mount AND a
|
||||
// named volume. The bundle must contain BOTH tars; the manifest must claim both.
|
||||
// RED-PROOF (either/or): revert executeExport to the else-only volume branch → the volume tar
|
||||
// is absent and has_volume_data=false → this test fails.
|
||||
// RED-PROOF (discovery): with the pre-fix ${HDD_PATH}-only adapter the mount list is empty →
|
||||
// has_hdd_data=false → this test fails (proven at the adapter level in stacks/export_mounts_test.go).
|
||||
func TestExport_NeedsHDDBundlesBothUserdataAndVolumes(t *testing.T) {
|
||||
swapDockerExec(t, func(c dockerCall, stdin io.Reader, stdout io.Writer) (string, error) {
|
||||
switch c.args[0] {
|
||||
case "create":
|
||||
fmt.Fprint(stdout, "cid-123\n")
|
||||
return "", nil
|
||||
case "cp":
|
||||
// stream a plausible non-empty tar for the volume
|
||||
fmt.Fprint(stdout, strings.Repeat("VOLTAR", 100))
|
||||
return "", nil
|
||||
case "rm":
|
||||
return "", nil
|
||||
}
|
||||
return "", fmt.Errorf("unexpected docker call: %v", c.args)
|
||||
})
|
||||
|
||||
srcStack := t.TempDir()
|
||||
os.WriteFile(filepath.Join(srcStack, "docker-compose.yml"),
|
||||
[]byte("services:\n hdd-app:\n image: alpine\n"), 0644)
|
||||
|
||||
hdd := t.TempDir()
|
||||
ud := filepath.Join(hdd, "userdata")
|
||||
os.MkdirAll(filepath.Join(ud, "media", "tv"), 0755)
|
||||
os.WriteFile(filepath.Join(ud, "media", "tv", "marker.bin"), []byte("USERDATA-MARKER-7"), 0644)
|
||||
|
||||
prov := &hddProvider{
|
||||
rtProvider: &rtProvider{stackDir: srcStack, stacksDir: t.TempDir(), deployed: true,
|
||||
volumes: []string{"hdd-app_config"}},
|
||||
mounts: []string{ud}, hddPath: hdd,
|
||||
}
|
||||
drive := t.TempDir()
|
||||
e := NewExporter(prov, log.New(io.Discard, "", 0), "test")
|
||||
|
||||
if err := e.StartExport(ExportRequest{StackName: "hdd-app", DestDrive: drive}); err != nil {
|
||||
t.Fatalf("StartExport: %v", err)
|
||||
}
|
||||
job := waitJob(t, e)
|
||||
if msg := jobErr(job); msg != "" {
|
||||
t.Fatalf("export failed: %s", msg)
|
||||
}
|
||||
|
||||
fabPath := findFab(t, drive)
|
||||
man, err := ReadManifestFromFAB(fabPath)
|
||||
if err != nil {
|
||||
t.Fatalf("manifest: %v", err)
|
||||
}
|
||||
if !man.HasHDDData {
|
||||
t.Error("has_hdd_data=false — the userdata mount was dropped (C6B-F1 cause 2)")
|
||||
}
|
||||
if !man.HasVolumeData {
|
||||
t.Error("has_volume_data=false — the named volume was dropped (C6B-F1 cause 1, the either/or)")
|
||||
}
|
||||
|
||||
entries := listFabEntries(t, fabPath)
|
||||
if sz, ok := entries["data/hdd/userdata.tar"]; !ok || sz == 0 {
|
||||
t.Errorf("bundle is missing a non-empty data/hdd/userdata.tar (entries: %v)", entries)
|
||||
}
|
||||
if sz, ok := entries["data/volumes/hdd-app_config.tar"]; !ok || sz == 0 {
|
||||
t.Errorf("bundle is missing a non-empty data/volumes/hdd-app_config.tar (entries: %v)", entries)
|
||||
}
|
||||
}
|
||||
|
||||
// Scenario E (§7): a needs_hdd app whose volume export strands (cp writes nothing) must FAIL the
|
||||
// whole job loudly — never a partial-success bundle with userdata but silently-missing volumes.
|
||||
func TestExport_NeedsHDDVolumeStrandFailsLoud(t *testing.T) {
|
||||
swapDockerExec(t, func(c dockerCall, stdin io.Reader, stdout io.Writer) (string, error) {
|
||||
switch c.args[0] {
|
||||
case "create":
|
||||
fmt.Fprint(stdout, "cid-123\n")
|
||||
return "", nil
|
||||
case "cp":
|
||||
return "", nil // stranded: nothing written
|
||||
case "rm":
|
||||
return "", nil
|
||||
}
|
||||
return "", fmt.Errorf("unexpected docker call: %v", c.args)
|
||||
})
|
||||
|
||||
srcStack := t.TempDir()
|
||||
os.WriteFile(filepath.Join(srcStack, "docker-compose.yml"),
|
||||
[]byte("services:\n hdd-app:\n image: alpine\n"), 0644)
|
||||
hdd := t.TempDir()
|
||||
ud := filepath.Join(hdd, "userdata")
|
||||
os.MkdirAll(ud, 0755)
|
||||
os.WriteFile(filepath.Join(ud, "f.bin"), []byte("x"), 0644)
|
||||
|
||||
prov := &hddProvider{
|
||||
rtProvider: &rtProvider{stackDir: srcStack, stacksDir: t.TempDir(), deployed: true,
|
||||
volumes: []string{"vol1"}},
|
||||
mounts: []string{ud}, hddPath: hdd,
|
||||
}
|
||||
drive := t.TempDir()
|
||||
e := NewExporter(prov, log.New(io.Discard, "", 0), "test")
|
||||
if err := e.StartExport(ExportRequest{StackName: "hdd-app", DestDrive: drive}); err != nil {
|
||||
t.Fatalf("StartExport: %v", err)
|
||||
}
|
||||
job := waitJob(t, e)
|
||||
if msg := jobErr(job); msg == "" {
|
||||
t.Fatal("a stranded volume tar must FAIL a needs_hdd export too — got success")
|
||||
}
|
||||
entries, _ := os.ReadDir(ExportDir(drive))
|
||||
for _, en := range entries {
|
||||
if strings.HasSuffix(en.Name(), ".fab") {
|
||||
t.Fatalf("a bundle was produced despite the stranded volume: %s", en.Name())
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// §8: two mounts sharing a basename cannot round-trip through the basename-keyed manifest —
|
||||
// the export must fail loudly instead of silently overwriting the first tar (the pre-fix
|
||||
// behavior). RED-PROOF: drop the collision check in exportHDDData → this test fails.
|
||||
func TestExport_HDDMountBasenameCollisionFailsLoud(t *testing.T) {
|
||||
srcStack := t.TempDir()
|
||||
os.WriteFile(filepath.Join(srcStack, "docker-compose.yml"),
|
||||
[]byte("services:\n hdd-app:\n image: alpine\n"), 0644)
|
||||
|
||||
hdd := t.TempDir()
|
||||
a := filepath.Join(hdd, "a", "config")
|
||||
b := filepath.Join(hdd, "b", "config")
|
||||
os.MkdirAll(a, 0755)
|
||||
os.MkdirAll(b, 0755)
|
||||
os.WriteFile(filepath.Join(a, "one.txt"), []byte("A"), 0644)
|
||||
os.WriteFile(filepath.Join(b, "two.txt"), []byte("B"), 0644)
|
||||
|
||||
prov := &hddProvider{
|
||||
rtProvider: &rtProvider{stackDir: srcStack, stacksDir: t.TempDir(), deployed: true},
|
||||
mounts: []string{a, b}, hddPath: hdd,
|
||||
}
|
||||
drive := t.TempDir()
|
||||
e := NewExporter(prov, log.New(io.Discard, "", 0), "test")
|
||||
if err := e.StartExport(ExportRequest{StackName: "hdd-app", DestDrive: drive}); err != nil {
|
||||
t.Fatalf("StartExport: %v", err)
|
||||
}
|
||||
job := waitJob(t, e)
|
||||
msg := jobErr(job)
|
||||
if msg == "" {
|
||||
t.Fatal("a basename collision must FAIL the export — got success (silent overwrite)")
|
||||
}
|
||||
if !strings.Contains(msg, "config") {
|
||||
t.Errorf("the error must name the colliding basename, got %q", msg)
|
||||
}
|
||||
}
|
||||
|
||||
// Scenario A' — the round-trip placement proof: an exported "userdata" tar restores back to
|
||||
// <HDD_PATH>/userdata through the UNTOUCHED import mapping (basename → <HDD_PATH>/<subdir>
|
||||
// fallback). This is the property that dictated capturing the userdata ROOT rather than
|
||||
// per-bind subpaths.
|
||||
func TestFabRoundTrip_UserdataPlacement(t *testing.T) {
|
||||
const stack = "ud-app"
|
||||
lg := log.New(io.Discard, "", 0)
|
||||
|
||||
hdd := t.TempDir()
|
||||
ud := filepath.Join(hdd, "userdata")
|
||||
os.MkdirAll(filepath.Join(ud, "media", "tv"), 0755)
|
||||
marker := "ROUNDTRIP-MARKER-99"
|
||||
os.WriteFile(filepath.Join(ud, "media", "tv", "show.bin"), []byte(marker), 0644)
|
||||
|
||||
srcStack := t.TempDir()
|
||||
os.WriteFile(filepath.Join(srcStack, "docker-compose.yml"),
|
||||
[]byte("services:\n ud-app:\n image: alpine\n volumes:\n - ${USERDATA_PATH}/media/tv:/tv\n"), 0644)
|
||||
// app.yaml carries HDD_PATH into the bundle — the import derives every restore path from it.
|
||||
os.WriteFile(filepath.Join(srcStack, "app.yaml"),
|
||||
[]byte("deployed: true\nenv:\n HDD_PATH: "+hdd+"\n"), 0644)
|
||||
|
||||
prov := &hddProvider{
|
||||
rtProvider: &rtProvider{stackDir: srcStack, stacksDir: t.TempDir(), deployed: true},
|
||||
mounts: []string{ud}, hddPath: hdd,
|
||||
}
|
||||
drive := t.TempDir()
|
||||
e := NewExporter(prov, lg, "test")
|
||||
if err := e.StartExport(ExportRequest{StackName: stack, DestDrive: drive}); err != nil {
|
||||
t.Fatalf("StartExport: %v", err)
|
||||
}
|
||||
job := waitJob(t, e)
|
||||
if msg := jobErr(job); msg != "" {
|
||||
t.Fatalf("export failed: %s", msg)
|
||||
}
|
||||
fabPath := findFab(t, drive)
|
||||
|
||||
// wipe the source userdata — the import must bring it back to the same place
|
||||
if err := os.RemoveAll(ud); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
prov2 := &hddProvider{
|
||||
rtProvider: &rtProvider{stackDir: srcStack, stacksDir: t.TempDir(), deployed: false},
|
||||
hddPath: hdd,
|
||||
}
|
||||
e2 := NewExporter(prov2, lg, "test")
|
||||
if err := e2.StartImport(ImportRequest{FABPath: fabPath}); err != nil {
|
||||
t.Fatalf("StartImport: %v", err)
|
||||
}
|
||||
job = waitJob(t, e2)
|
||||
if msg := jobErr(job); msg != "" {
|
||||
t.Fatalf("import failed: %s", msg)
|
||||
}
|
||||
|
||||
got, err := os.ReadFile(filepath.Join(hdd, "userdata", "media", "tv", "show.bin"))
|
||||
if err != nil {
|
||||
t.Fatalf("restored userdata not at <HDD_PATH>/userdata/media/tv/show.bin: %v", err)
|
||||
}
|
||||
if string(got) != marker {
|
||||
t.Fatalf("restored content differs: got %q want %q", got, marker)
|
||||
}
|
||||
}
|
||||
|
||||
// Scenario D (§7) — the anti-hollow net (C6B-F1 cause 3): a needs_hdd app where discovery finds
|
||||
// NOTHING (mounts absent on disk, no volumes) must FAIL the export with an honest Hungarian
|
||||
// error — never success-with-a-hollow-bundle. RED-PROOF: remove the needs_hdd&&no-data assertion
|
||||
// from assertBundleDataComplete → this test fails (the pre-fix silent hollow success).
|
||||
func TestExport_NeedsHDDNoDataAtAllRefused(t *testing.T) {
|
||||
srcStack := t.TempDir()
|
||||
os.WriteFile(filepath.Join(srcStack, "docker-compose.yml"),
|
||||
[]byte("services:\n hdd-app:\n image: alpine\n"), 0644)
|
||||
|
||||
hdd := t.TempDir()
|
||||
prov := &hddProvider{
|
||||
rtProvider: &rtProvider{stackDir: srcStack, stacksDir: t.TempDir(), deployed: true},
|
||||
// the mount does NOT exist on disk and there are no volumes — total discovery blank
|
||||
mounts: []string{filepath.Join(hdd, "userdata")}, hddPath: hdd,
|
||||
}
|
||||
drive := t.TempDir()
|
||||
e := NewExporter(prov, log.New(io.Discard, "", 0), "test")
|
||||
if err := e.StartExport(ExportRequest{StackName: "hdd-app", DestDrive: drive}); err != nil {
|
||||
t.Fatalf("StartExport: %v", err)
|
||||
}
|
||||
job := waitJob(t, e)
|
||||
msg := jobErr(job)
|
||||
if msg == "" {
|
||||
t.Fatal("a needs_hdd export with ZERO discovered data must FAIL — got success (the hollow bundle)")
|
||||
}
|
||||
if !strings.Contains(msg, "nem tartalmaz alkalmazásadatot") {
|
||||
t.Errorf("expected the honest Hungarian no-app-data error, got %q", msg)
|
||||
}
|
||||
entries, _ := os.ReadDir(ExportDir(drive))
|
||||
for _, en := range entries {
|
||||
if strings.HasSuffix(en.Name(), ".fab") {
|
||||
t.Fatalf("a hollow bundle was produced: %s", en.Name())
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,138 +0,0 @@
|
||||
package appexport
|
||||
|
||||
import (
|
||||
"io"
|
||||
"log"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/appbackup"
|
||||
)
|
||||
|
||||
func fabWrite(t *testing.T, root, rel, content string) {
|
||||
t.Helper()
|
||||
p := filepath.Join(root, filepath.FromSlash(rel))
|
||||
if err := os.MkdirAll(filepath.Dir(p), 0755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(p, []byte(content), 0644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
// Scenario C at the export level: the userdata.tar keeps the mandatory subtree and NOT the siblings,
|
||||
// and the manifest still lists the single `userdata` basename (v1 unchanged). The bundle-level anchor
|
||||
// for the SQ6 fix (mirrors the §13 before/after).
|
||||
func TestFabExport_ExcludeScopedUserdataTar(t *testing.T) {
|
||||
drive := t.TempDir() // hddPath
|
||||
fabWrite(t, drive, "userdata/media/books/a.epub", "BOOK")
|
||||
fabWrite(t, drive, "userdata/media/movies/big.mkv", "MOVIE")
|
||||
fabWrite(t, drive, "userdata/music/s.flac", "SONG")
|
||||
|
||||
ud := appbackup.UserdataDir(filepath.Clean(drive))
|
||||
prov := &fabProv{
|
||||
rtProvider: &rtProvider{}, hddPath: drive, has: true,
|
||||
binds: []appbackup.ClassifiedBind{mUD("media/books"), xUD("media/movies")},
|
||||
mounts: []string{ud},
|
||||
}
|
||||
e := NewExporter(prov, log.New(io.Discard, "", 0), "test")
|
||||
|
||||
dataDir := t.TempDir()
|
||||
man := &Manifest{}
|
||||
if err := e.exportHDDData(ExportRequest{StackName: "calibre-web"}, dataDir, man); err != nil {
|
||||
t.Fatalf("exportHDDData: %v", err)
|
||||
}
|
||||
|
||||
entries := tarEntries(t, filepath.Join(dataDir, "hdd", "userdata.tar"))
|
||||
if !containsSuffix(entries, "media/books/a.epub") {
|
||||
t.Errorf("mandatory media/books missing from userdata.tar: %v", entries)
|
||||
}
|
||||
for _, sib := range []string{"media/movies/big.mkv", "media/movies", "music/s.flac", "music"} {
|
||||
if containsSuffix(entries, sib) {
|
||||
t.Errorf("sibling %q must NOT ride along (SQ6): %v", sib, entries)
|
||||
}
|
||||
}
|
||||
// v1 manifest: the single `userdata` basename, unchanged.
|
||||
if len(man.HDDSubdirs) != 1 || man.HDDSubdirs[0] != "userdata" {
|
||||
t.Errorf("manifest must list the single v1 `userdata` basename, got %v", man.HDDSubdirs)
|
||||
}
|
||||
}
|
||||
|
||||
// Scenario A at the export level: a legacy (no-block) app tars the FULL userdata root (every sibling)
|
||||
// — byte-identical to v0.130.0 (the SQ5 safety net).
|
||||
func TestFabExport_LegacyFullRoot(t *testing.T) {
|
||||
drive := t.TempDir()
|
||||
fabWrite(t, drive, "userdata/media/books/a.epub", "BOOK")
|
||||
fabWrite(t, drive, "userdata/media/movies/big.mkv", "MOVIE")
|
||||
|
||||
ud := appbackup.UserdataDir(filepath.Clean(drive))
|
||||
prov := &fabProv{rtProvider: &rtProvider{}, hddPath: drive, has: false, mounts: []string{ud}}
|
||||
e := NewExporter(prov, log.New(io.Discard, "", 0), "test")
|
||||
|
||||
dataDir := t.TempDir()
|
||||
man := &Manifest{}
|
||||
if err := e.exportHDDData(ExportRequest{StackName: "sonarr"}, dataDir, man); err != nil {
|
||||
t.Fatalf("exportHDDData: %v", err)
|
||||
}
|
||||
entries := tarEntries(t, filepath.Join(dataDir, "hdd", "userdata.tar"))
|
||||
for _, want := range []string{"media/books/a.epub", "media/movies/big.mkv"} {
|
||||
if !containsSuffix(entries, want) {
|
||||
t.Errorf("legacy app must capture the FULL root — %q missing: %v", want, entries)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Scenario B at the export level: an all-excluded app produces NO userdata.tar (root tar skipped).
|
||||
func TestFabExport_AllExcludedNoUserdataTar(t *testing.T) {
|
||||
drive := t.TempDir()
|
||||
fabWrite(t, drive, "userdata/media/movies/big.mkv", "MOVIE")
|
||||
ud := appbackup.UserdataDir(filepath.Clean(drive))
|
||||
prov := &fabProv{
|
||||
rtProvider: &rtProvider{}, hddPath: drive, has: true,
|
||||
binds: []appbackup.ClassifiedBind{xUD("media/movies")},
|
||||
mounts: []string{ud},
|
||||
}
|
||||
e := NewExporter(prov, log.New(io.Discard, "", 0), "test")
|
||||
dataDir := t.TempDir()
|
||||
man := &Manifest{}
|
||||
if err := e.exportHDDData(ExportRequest{StackName: "radarr"}, dataDir, man); err != nil {
|
||||
t.Fatalf("exportHDDData: %v", err)
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(dataDir, "hdd", "userdata.tar")); !os.IsNotExist(err) {
|
||||
t.Errorf("all-excluded app must produce NO userdata.tar (Scenario B), stat err=%v", err)
|
||||
}
|
||||
if len(man.HDDSubdirs) != 0 {
|
||||
t.Errorf("no userdata leg → no manifest subdir, got %v", man.HDDSubdirs)
|
||||
}
|
||||
}
|
||||
|
||||
// §7-F: EstimateExport populates the class split for a classified app (both web estimate pipelines
|
||||
// call this shared function, so both surface it). du returns 0 on the Windows test host, so this
|
||||
// asserts the STRUCTURE (HasClassification + item keys), not byte values.
|
||||
func TestEstimateExport_ClassifiedSplit(t *testing.T) {
|
||||
drive := t.TempDir()
|
||||
stackDir := t.TempDir()
|
||||
os.WriteFile(filepath.Join(stackDir, "docker-compose.yml"), []byte("services: {}\n"), 0644)
|
||||
prov := &fabProv{
|
||||
rtProvider: &rtProvider{stackDir: stackDir, deployed: true}, hddPath: drive, has: true,
|
||||
binds: []appbackup.ClassifiedBind{mUD("media/books"), oUD("media/comics"), xUD("media/movies")},
|
||||
}
|
||||
e := NewExporter(prov, log.New(io.Discard, "", 0), "test")
|
||||
est, err := e.EstimateExport("calibre-web", drive)
|
||||
if err != nil {
|
||||
t.Fatalf("EstimateExport: %v", err)
|
||||
}
|
||||
if !est.HasClassification {
|
||||
t.Fatal("classified app estimate must carry HasClassification")
|
||||
}
|
||||
if len(est.MandatoryItems) != 1 || est.MandatoryItems[0].Key != "userdata/media/books" {
|
||||
t.Errorf("MandatoryItems = %+v", est.MandatoryItems)
|
||||
}
|
||||
if len(est.OptionalItems) != 1 || est.OptionalItems[0].Key != "userdata/media/comics" {
|
||||
t.Errorf("OptionalItems = %+v", est.OptionalItems)
|
||||
}
|
||||
if len(est.ExcludedItems) != 1 || est.ExcludedItems[0].Key != "userdata/media/movies" {
|
||||
t.Errorf("ExcludedItems = %+v", est.ExcludedItems)
|
||||
}
|
||||
}
|
||||
@@ -1,311 +0,0 @@
|
||||
package appexport
|
||||
|
||||
import (
|
||||
"archive/tar"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/appbackup"
|
||||
)
|
||||
|
||||
// `.fab` class-scoped export plan (Task 4, architecture §2 `.fab` row + SQ5 exclusion-scoping verdict,
|
||||
// R1-C). Mechanics unchanged from v0.130.0: ONE exclude-scoped userdata-root tar + per-mount skip for
|
||||
// non-selected HDD binds. The manifest stays v1 (basename keying) and the import side is untouched.
|
||||
|
||||
// fabPlan is the class-scoped adjustment to the v0.130.0 mount/tar set. Empty (all zero) = the legacy
|
||||
// full capture (Scenario A — a no-block app produces this).
|
||||
type fabPlan struct {
|
||||
SkipMounts map[string]bool // absolute HDD mount paths to skip entirely
|
||||
SkipUserdataTar bool // no selected userdata bind ⇒ the whole root tar is skipped (Scenario B)
|
||||
UserdataExcludeRels []string // rels (relative to the userdata root) excluded from its tar (R1-C, Scenario C)
|
||||
}
|
||||
|
||||
func relKey(root appbackup.BindRoot, rel string) string { return string(root) + "/" + rel }
|
||||
func relKeyOf(cp appbackup.CapturePath) string { return relKey(cp.Root, cp.RelPath) }
|
||||
|
||||
// computeFabPlan resolves the class buckets + the caller's selection into the mount/userdata plan
|
||||
// (§8). Legacy / no-block ⇒ empty plan. The mandatory floor is enforced here: DeselectOptional can
|
||||
// never drop a mandatory path.
|
||||
func (e *Exporter) computeFabPlan(req ExportRequest, mounts []string) fabPlan {
|
||||
binds, has := e.provider.GetStackClassifiedBinds(req.StackName)
|
||||
if !has {
|
||||
return fabPlan{} // legacy: byte-identical v0.130.0 capture
|
||||
}
|
||||
// R-203: the shared resolver's root parameter is a NAMESPACE ROOT — that is what the off-site
|
||||
// side has always passed (ComputeCaptureSet ← offbox_capture.go). This site passed the bare drive
|
||||
// path, so on the system-data fallback the export's classified paths and the backup's capture set
|
||||
// described DIFFERENT directories for the same declared bind. They now agree by construction.
|
||||
nsRoot := filepath.Clean(e.provider.GetStackNamespaceRoot(req.StackName))
|
||||
fb := appbackup.ComputeFabBuckets(binds, has, nsRoot, e.provider.GetImportRoot())
|
||||
|
||||
deselect := sliceSet(req.DeselectOptional)
|
||||
optIn := sliceSet(req.OptInExcluded)
|
||||
|
||||
// Server-side floor: a request naming a mandatory path in DeselectOptional is ignored (loud WARN).
|
||||
for _, cp := range fb.Mandatory {
|
||||
if deselect[relKeyOf(cp)] {
|
||||
e.logger.Printf("[WARN] appexport: %s: request tried to deselect a MANDATORY path %s — ignored (floor enforced)", req.StackName, relKeyOf(cp))
|
||||
}
|
||||
}
|
||||
|
||||
// Resolve the selected set (mandatory always; optional default-in; excluded default-out).
|
||||
var selectedHDD, selectedUD []appbackup.CapturePath
|
||||
add := func(cp appbackup.CapturePath) {
|
||||
if cp.Root == appbackup.RootUserdata {
|
||||
selectedUD = append(selectedUD, cp)
|
||||
} else {
|
||||
selectedHDD = append(selectedHDD, cp)
|
||||
}
|
||||
}
|
||||
for _, cp := range fb.Mandatory {
|
||||
add(cp)
|
||||
}
|
||||
for _, cp := range fb.Optional {
|
||||
if !deselect[relKeyOf(cp)] {
|
||||
add(cp)
|
||||
}
|
||||
}
|
||||
for _, cp := range fb.Excluded {
|
||||
if optIn[relKeyOf(cp)] {
|
||||
add(cp)
|
||||
}
|
||||
}
|
||||
|
||||
// Every classified HDD bind (any class) — a mount matching NONE of these is "unclassified" and kept
|
||||
// (fail toward capture, the C6B-F1 direction).
|
||||
var classifiedHDD []string
|
||||
for _, bucket := range [][]appbackup.CapturePath{fb.Mandatory, fb.Optional, fb.Excluded} {
|
||||
for _, cp := range bucket {
|
||||
if cp.Root == appbackup.RootHDD {
|
||||
classifiedHDD = append(classifiedHDD, cp.Abs)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
plan := fabPlan{SkipMounts: map[string]bool{}}
|
||||
// R-203: UserdataDir takes a NAMESPACE ROOT, not the drive path. Identical on an enrolled drive;
|
||||
// one segment short on the system-data fallback, which is where the export plan then skipped (or
|
||||
// failed to skip) the wrong directory.
|
||||
ud := appbackup.UserdataDir(nsRoot)
|
||||
for _, m := range mounts {
|
||||
mc := filepath.Clean(m)
|
||||
if mc == filepath.Clean(ud) {
|
||||
if len(selectedUD) == 0 {
|
||||
plan.SkipUserdataTar = true // Scenario B: no selected userdata bind → no root tar
|
||||
}
|
||||
continue
|
||||
}
|
||||
// HDD mount. Unmatched by ANY classified bind → keep (fail toward capture), log it.
|
||||
if !relatedToAny(mc, classifiedHDD) {
|
||||
e.logger.Printf("[INFO] appexport: %s: HDD mount %s matches no classified bind — kept (fail toward capture)", req.StackName, mc)
|
||||
continue
|
||||
}
|
||||
// Matched a classified bind: keep iff ancestor-or-descendant of a SELECTED HDD path.
|
||||
if !relatedToAny(mc, absList(selectedHDD)) {
|
||||
plan.SkipMounts[mc] = true
|
||||
}
|
||||
}
|
||||
|
||||
if !plan.SkipUserdataTar && len(selectedUD) > 0 {
|
||||
plan.UserdataExcludeRels = e.fabUserdataExcludes(ud, udRels(selectedUD))
|
||||
}
|
||||
return plan
|
||||
}
|
||||
|
||||
// fabEstimateSplit populates the class-split estimate fields for a classified app (Task 4). Legacy /
|
||||
// no-block apps leave HasClassification=false (the UI shows the plain estimate). BaseBytes = config +
|
||||
// volumes + mandatory; optional/excluded carry per-path sizes for client-side total recomputation.
|
||||
func (e *Exporter) fabEstimateSplit(stackName string, est *ExportEstimate, volumeBytes int64) {
|
||||
binds, has := e.provider.GetStackClassifiedBinds(stackName)
|
||||
if !has {
|
||||
return
|
||||
}
|
||||
nsRoot := filepath.Clean(e.provider.GetStackNamespaceRoot(stackName)) // R-203, as above
|
||||
fb := appbackup.ComputeFabBuckets(binds, has, nsRoot, e.provider.GetImportRoot())
|
||||
est.HasClassification = true
|
||||
|
||||
toItems := func(cps []appbackup.CapturePath) ([]FabItem, int64) {
|
||||
var items []FabItem
|
||||
var sum int64
|
||||
for _, cp := range cps {
|
||||
sz := duBytes(cp.Abs)
|
||||
items = append(items, FabItem{Key: relKeyOf(cp), Root: string(cp.Root), RelPath: cp.RelPath, Bytes: sz, Human: humanizeBytes(sz)})
|
||||
sum += sz
|
||||
}
|
||||
return items, sum
|
||||
}
|
||||
var mandSum int64
|
||||
est.MandatoryItems, mandSum = toItems(fb.Mandatory)
|
||||
est.OptionalItems, _ = toItems(fb.Optional)
|
||||
est.ExcludedItems, _ = toItems(fb.Excluded)
|
||||
est.BaseBytes = est.ConfigSizeBytes + volumeBytes + mandSum
|
||||
est.BaseHuman = humanizeBytes(est.BaseBytes)
|
||||
}
|
||||
|
||||
func sliceSet(ss []string) map[string]bool {
|
||||
m := make(map[string]bool, len(ss))
|
||||
for _, s := range ss {
|
||||
m[s] = true
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
func absList(cps []appbackup.CapturePath) []string {
|
||||
out := make([]string, len(cps))
|
||||
for i, cp := range cps {
|
||||
out[i] = cp.Abs
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// udRels returns the userdata rels (relative to ${USERDATA_PATH}) of selected userdata paths.
|
||||
func udRels(cps []appbackup.CapturePath) []string {
|
||||
out := make([]string, 0, len(cps))
|
||||
for _, cp := range cps {
|
||||
out = append(out, cp.RelPath)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// relatedToAny reports whether path p is an ancestor OR descendant (or equal) of any path in set.
|
||||
func relatedToAny(p string, set []string) bool {
|
||||
pc := filepath.Clean(p)
|
||||
for _, s := range set {
|
||||
sc := filepath.Clean(s)
|
||||
if pc == sc || strings.HasPrefix(pc, sc+string(filepath.Separator)) || strings.HasPrefix(sc, pc+string(filepath.Separator)) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// fabRelClass classifies a dir rel (slash-form, relative to the userdata root) against the selected
|
||||
// userdata rels — the R1-C keep-rule (mirrors backup.classifyTier2Rel, copied not imported):
|
||||
// keepInside = a selected rel or inside one (keep, don't descend); keepAncestor = on the path to a
|
||||
// selected rel (keep, descend); else stale (exclude the topmost).
|
||||
type fabRelClass int
|
||||
|
||||
const (
|
||||
fabStale fabRelClass = iota
|
||||
fabKeepInside
|
||||
fabKeepAncestor
|
||||
)
|
||||
|
||||
func classifyFabRel(dirRel string, selectedRels []string) fabRelClass {
|
||||
for _, sr := range selectedRels {
|
||||
if dirRel == sr || strings.HasPrefix(dirRel, sr+"/") {
|
||||
return fabKeepInside
|
||||
}
|
||||
}
|
||||
for _, sr := range selectedRels {
|
||||
if strings.HasPrefix(sr, dirRel+"/") {
|
||||
return fabKeepAncestor
|
||||
}
|
||||
}
|
||||
return fabStale
|
||||
}
|
||||
|
||||
// fabUserdataExcludes walks the userdata root (via the dirLister seam) and returns the topmost rels
|
||||
// (relative to the root, slash-form) that are neither an ancestor nor a descendant of a selected
|
||||
// userdata rel — the exclude list for the root tar (R1-C). Deterministic (sorted).
|
||||
func (e *Exporter) fabUserdataExcludes(udRoot string, selectedRels []string) []string {
|
||||
lister := e.dirLister
|
||||
if lister == nil {
|
||||
lister = realDirLister
|
||||
}
|
||||
var excludes []string
|
||||
var walk func(dirAbs, dirRel string)
|
||||
walk = func(dirAbs, dirRel string) {
|
||||
for _, name := range lister(dirAbs) {
|
||||
childRel := name
|
||||
if dirRel != "" {
|
||||
childRel = dirRel + "/" + name
|
||||
}
|
||||
switch classifyFabRel(childRel, selectedRels) {
|
||||
case fabKeepInside:
|
||||
// selected leg or content inside it — keep, no descent
|
||||
case fabKeepAncestor:
|
||||
walk(filepath.Join(dirAbs, name), childRel)
|
||||
default:
|
||||
excludes = append(excludes, childRel) // topmost neither-ancestor-nor-descendant
|
||||
}
|
||||
}
|
||||
}
|
||||
walk(udRoot, "")
|
||||
sort.Strings(excludes)
|
||||
return excludes
|
||||
}
|
||||
|
||||
func realDirLister(dir string) []string {
|
||||
entries, err := os.ReadDir(dir)
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
var names []string
|
||||
for _, en := range entries {
|
||||
if en.IsDir() {
|
||||
names = append(names, en.Name())
|
||||
}
|
||||
}
|
||||
return names
|
||||
}
|
||||
|
||||
// tarDirectoryExcluding is tarDirectory with an exclude list: any path whose rel (relative to
|
||||
// sourceDir, slash-form) equals or descends from an exclude rel is skipped (a dir is pruned whole).
|
||||
// Empty excludes == tarDirectory. Anchored to the tar root exactly like tarDirectory's rel names.
|
||||
func tarDirectoryExcluding(sourceDir, outputPath string, excludeRels []string) error {
|
||||
excl := make([]string, len(excludeRels))
|
||||
for i, r := range excludeRels {
|
||||
excl[i] = filepath.ToSlash(r)
|
||||
}
|
||||
outFile, err := os.Create(outputPath)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer outFile.Close()
|
||||
tw := tar.NewWriter(outFile)
|
||||
defer tw.Close()
|
||||
|
||||
return filepath.Walk(sourceDir, func(path string, info os.FileInfo, err error) error {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
relPath, err := filepath.Rel(sourceDir, path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if relPath == "." {
|
||||
return nil
|
||||
}
|
||||
rel := filepath.ToSlash(relPath)
|
||||
for _, e := range excl {
|
||||
if rel == e || strings.HasPrefix(rel, e+"/") {
|
||||
if info.IsDir() {
|
||||
return filepath.SkipDir
|
||||
}
|
||||
return nil
|
||||
}
|
||||
}
|
||||
header, err := tar.FileInfoHeader(info, "")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
header.Name = relPath
|
||||
if err := tw.WriteHeader(header); err != nil {
|
||||
return err
|
||||
}
|
||||
if info.IsDir() {
|
||||
return nil
|
||||
}
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer f.Close()
|
||||
_, err = io.Copy(tw, f)
|
||||
return err
|
||||
})
|
||||
}
|
||||
@@ -1,270 +0,0 @@
|
||||
package appexport
|
||||
|
||||
import (
|
||||
"archive/tar"
|
||||
"io"
|
||||
"log"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"sort"
|
||||
"testing"
|
||||
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/appbackup"
|
||||
)
|
||||
|
||||
// fabProv is a minimal ExportStackProvider for the plan tests: configurable classified binds + hddPath.
|
||||
type fabProv struct {
|
||||
*rtProvider
|
||||
hddPath string
|
||||
binds []appbackup.ClassifiedBind
|
||||
has bool
|
||||
mounts []string
|
||||
}
|
||||
|
||||
func (p *fabProv) GetStackHDDPath(string) string { return p.hddPath }
|
||||
func (p *fabProv) GetImportRoot() string { return "" } // R-75: no import binds in this fixture
|
||||
|
||||
// R-203: these fixtures use ENROLLED drive paths, where the namespace root IS the drive path.
|
||||
// Delegating keeps that identity explicit rather than hardcoding it.
|
||||
func (p *fabProv) GetStackNamespaceRoot(name string) string { return p.GetStackHDDPath(name) }
|
||||
func (p *fabProv) GetStackHDDMounts(string) []string { return p.mounts }
|
||||
func (p *fabProv) GetStackClassifiedBinds(string) ([]appbackup.ClassifiedBind, bool) {
|
||||
return p.binds, p.has
|
||||
}
|
||||
|
||||
func newFabExporter(binds []appbackup.ClassifiedBind, has bool, hddPath string, tree map[string][]string) *Exporter {
|
||||
e := NewExporter(&fabProv{rtProvider: &rtProvider{}, hddPath: hddPath, binds: binds, has: has}, log.New(io.Discard, "", 0), "test")
|
||||
e.dirLister = func(dir string) []string { return tree[dir] }
|
||||
return e
|
||||
}
|
||||
|
||||
func mHDD(rel string) appbackup.ClassifiedBind {
|
||||
return appbackup.ClassifiedBind{ComposeBind: appbackup.ComposeBind{Root: appbackup.RootHDD, RelPath: rel}, Class: appbackup.ClassMandatory}
|
||||
}
|
||||
func mUD(rel string) appbackup.ClassifiedBind {
|
||||
return appbackup.ClassifiedBind{ComposeBind: appbackup.ComposeBind{Root: appbackup.RootUserdata, RelPath: rel}, Class: appbackup.ClassMandatory}
|
||||
}
|
||||
func oUD(rel string) appbackup.ClassifiedBind {
|
||||
return appbackup.ClassifiedBind{ComposeBind: appbackup.ComposeBind{Root: appbackup.RootUserdata, RelPath: rel}, Class: appbackup.ClassOptional}
|
||||
}
|
||||
func xUD(rel string) appbackup.ClassifiedBind {
|
||||
return appbackup.ClassifiedBind{ComposeBind: appbackup.ComposeBind{Root: appbackup.RootUserdata, RelPath: rel}, Class: appbackup.ClassExcluded}
|
||||
}
|
||||
|
||||
const hp = "/srv/data" // hddPath (plan is string-only; the lister is injected)
|
||||
|
||||
// udDir builds an OS-native userdata dir path (matches appbackup.UserdataDir + the walk's
|
||||
// filepath.Join, so injected-lister keys line up on any host).
|
||||
func udDir(rel ...string) string {
|
||||
return filepath.Join(append([]string{hp, "userdata"}, rel...)...)
|
||||
}
|
||||
|
||||
// A — legacy app: empty plan (byte-identical v0.130.0 capture).
|
||||
func TestFabPlan_LegacyEmpty(t *testing.T) {
|
||||
e := newFabExporter(nil, false, hp, nil)
|
||||
mounts := []string{hp + "/appdata/app", hp + "/userdata"}
|
||||
plan := e.computeFabPlan(ExportRequest{StackName: "x"}, mounts)
|
||||
if len(plan.SkipMounts) != 0 || plan.SkipUserdataTar || plan.UserdataExcludeRels != nil {
|
||||
t.Errorf("legacy app must yield an EMPTY plan (every mount kept, no excludes), got %+v", plan)
|
||||
}
|
||||
}
|
||||
|
||||
// B — all-excluded app: the userdata root tar is skipped entirely.
|
||||
func TestFabPlan_AllExcludedSkipsUserdataTar(t *testing.T) {
|
||||
binds := []appbackup.ClassifiedBind{xUD("media/movies"), xUD("downloads")}
|
||||
e := newFabExporter(binds, true, hp, map[string][]string{"/srv/data/userdata": {"media", "downloads"}})
|
||||
plan := e.computeFabPlan(ExportRequest{StackName: "radarr"}, []string{hp + "/userdata"})
|
||||
if !plan.SkipUserdataTar {
|
||||
t.Error("no selected userdata bind → the whole root tar must be skipped (Scenario B)")
|
||||
}
|
||||
}
|
||||
|
||||
// C — selected-complement: media/books kept, siblings excluded (R1-C).
|
||||
func TestFabPlan_SelectedComplement(t *testing.T) {
|
||||
binds := []appbackup.ClassifiedBind{mUD("media/books"), xUD("media/movies")}
|
||||
tree := map[string][]string{
|
||||
udDir(): {"media", "music"},
|
||||
udDir("media"): {"books", "movies", "comics"},
|
||||
}
|
||||
e := newFabExporter(binds, true, hp, tree)
|
||||
plan := e.computeFabPlan(ExportRequest{StackName: "calibre-web"}, []string{hp + "/userdata"})
|
||||
if plan.SkipUserdataTar {
|
||||
t.Fatal("a selected userdata bind exists — the root tar must NOT be skipped")
|
||||
}
|
||||
want := []string{"media/comics", "media/movies", "music"}
|
||||
if got := plan.UserdataExcludeRels; !reflect.DeepEqual(got, want) {
|
||||
t.Errorf("excludes = %v, want %v (media/books kept, siblings excluded)", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
// D — optional default-in / uncheck-out / excluded opt-in.
|
||||
func TestFabPlan_OptionalAndOptIn(t *testing.T) {
|
||||
// A mandatory anchor (data) keeps the root tar always produced, so comics/podcasts inclusion is
|
||||
// exercised via the EXCLUDE LIST (not the whole-tar skip).
|
||||
binds := []appbackup.ClassifiedBind{mUD("data"), oUD("media/comics"), xUD("media/podcasts")}
|
||||
tree := map[string][]string{
|
||||
udDir(): {"data", "media"},
|
||||
udDir("media"): {"comics", "podcasts", "junk"},
|
||||
}
|
||||
// D1 default: optional comics IN → not excluded; podcasts (excluded) + junk (unselected) excluded.
|
||||
e := newFabExporter(binds, true, hp, tree)
|
||||
p1 := e.computeFabPlan(ExportRequest{StackName: "komga"}, []string{hp + "/userdata"})
|
||||
if p1.SkipUserdataTar {
|
||||
t.Fatal("mandatory anchor selected — tar must be produced")
|
||||
}
|
||||
if effExcluded(p1.UserdataExcludeRels, "media/comics") {
|
||||
t.Error("D1: default → optional comics must be INCLUDED (not excluded)")
|
||||
}
|
||||
if !effExcluded(p1.UserdataExcludeRels, "media/podcasts") {
|
||||
t.Error("D1: excluded podcasts must be excluded by default")
|
||||
}
|
||||
// D2 uncheck the optional → excluded (effectively, via a topmost exclude covering it).
|
||||
p2 := e.computeFabPlan(ExportRequest{StackName: "komga", DeselectOptional: []string{"userdata/media/comics"}}, []string{hp + "/userdata"})
|
||||
if !effExcluded(p2.UserdataExcludeRels, "media/comics") {
|
||||
t.Errorf("D2: unchecked optional must be excluded, excludes=%v", p2.UserdataExcludeRels)
|
||||
}
|
||||
// D3 opt-in the excluded → included.
|
||||
p3 := e.computeFabPlan(ExportRequest{StackName: "komga", OptInExcluded: []string{"userdata/media/podcasts"}}, []string{hp + "/userdata"})
|
||||
if effExcluded(p3.UserdataExcludeRels, "media/podcasts") {
|
||||
t.Error("D3: opted-in excluded must be INCLUDED (not excluded)")
|
||||
}
|
||||
}
|
||||
|
||||
// effExcluded reports whether rel (or an ancestor of it) is in the topmost exclude list.
|
||||
func effExcluded(excludes []string, rel string) bool {
|
||||
for _, e := range excludes {
|
||||
if rel == e || len(rel) > len(e) && rel[:len(e)+1] == e+"/" {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// D floor — a request deselecting a MANDATORY path is IGNORED (mandatory stays in).
|
||||
func TestFabPlan_MandatoryFloor(t *testing.T) {
|
||||
binds := []appbackup.ClassifiedBind{mUD("media/books")}
|
||||
tree := map[string][]string{udDir(): {"media"}, udDir("media"): {"books"}}
|
||||
e := newFabExporter(binds, true, hp, tree)
|
||||
// client tries to deselect the mandatory path — must be ignored (books NOT excluded).
|
||||
plan := e.computeFabPlan(ExportRequest{StackName: "x", DeselectOptional: []string{"userdata/media/books"}}, []string{hp + "/userdata"})
|
||||
if contains(plan.UserdataExcludeRels, "media/books") || plan.SkipUserdataTar {
|
||||
t.Errorf("mandatory floor breached — media/books must stay in the bundle; plan=%+v", plan)
|
||||
}
|
||||
}
|
||||
|
||||
// §8 — an HDD mount matching NO classified bind is KEPT (fail toward capture).
|
||||
func TestFabPlan_UnmatchedMountKept(t *testing.T) {
|
||||
binds := []appbackup.ClassifiedBind{mHDD("appdata/known")}
|
||||
e := newFabExporter(binds, true, hp, nil)
|
||||
mounts := []string{hp + "/appdata/known", hp + "/appdata/mystery"}
|
||||
plan := e.computeFabPlan(ExportRequest{StackName: "x"}, mounts)
|
||||
if plan.SkipMounts[filepath.Clean(hp+"/appdata/mystery")] {
|
||||
t.Error("an unmatched mount must be KEPT (fail toward capture, C6B-F1)")
|
||||
}
|
||||
if plan.SkipMounts[filepath.Clean(hp+"/appdata/known")] {
|
||||
t.Error("a mandatory-matched mount must be kept")
|
||||
}
|
||||
}
|
||||
|
||||
// §8 — a classified HDD mount that is NOT selected is skipped.
|
||||
func TestFabPlan_UnselectedHDDMountSkipped(t *testing.T) {
|
||||
binds := []appbackup.ClassifiedBind{xHDD("appdata/cache")}
|
||||
e := newFabExporter(binds, true, hp, nil)
|
||||
mounts := []string{hp + "/appdata/cache"}
|
||||
plan := e.computeFabPlan(ExportRequest{StackName: "x"}, mounts)
|
||||
if !plan.SkipMounts[filepath.Clean(hp+"/appdata/cache")] {
|
||||
t.Error("an excluded, un-opted-in HDD mount must be skipped")
|
||||
}
|
||||
}
|
||||
|
||||
func xHDD(rel string) appbackup.ClassifiedBind {
|
||||
return appbackup.ClassifiedBind{ComposeBind: appbackup.ComposeBind{Root: appbackup.RootHDD, RelPath: rel}, Class: appbackup.ClassExcluded}
|
||||
}
|
||||
|
||||
// classifyFabRel keep-rule truth table (the R1-C core, pure).
|
||||
func TestClassifyFabRel(t *testing.T) {
|
||||
sel := []string{"media/books"}
|
||||
cases := []struct {
|
||||
rel string
|
||||
want fabRelClass
|
||||
}{
|
||||
{"media/books", fabKeepInside},
|
||||
{"media/books/covers", fabKeepInside},
|
||||
{"media", fabKeepAncestor},
|
||||
{"media/movies", fabStale},
|
||||
{"music", fabStale},
|
||||
}
|
||||
for _, c := range cases {
|
||||
if got := classifyFabRel(c.rel, sel); got != c.want {
|
||||
t.Errorf("classifyFabRel(%q) = %d, want %d", c.rel, got, c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// tarDirectoryExcluding FS-level: excluded subtrees are absent, kept content present.
|
||||
func TestTarDirectoryExcluding(t *testing.T) {
|
||||
src := t.TempDir()
|
||||
write := func(rel, content string) {
|
||||
p := filepath.Join(src, filepath.FromSlash(rel))
|
||||
os.MkdirAll(filepath.Dir(p), 0755)
|
||||
os.WriteFile(p, []byte(content), 0644)
|
||||
}
|
||||
write("media/books/a.epub", "BOOK")
|
||||
write("media/movies/big.mkv", "MOVIE")
|
||||
write("music/song.flac", "SONG")
|
||||
|
||||
out := filepath.Join(t.TempDir(), "userdata.tar")
|
||||
if err := tarDirectoryExcluding(src, out, []string{"media/movies", "music"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got := tarEntries(t, out)
|
||||
if !containsSuffix(got, "media/books/a.epub") {
|
||||
t.Errorf("kept content missing: %v", got)
|
||||
}
|
||||
for _, bad := range []string{"media/movies/big.mkv", "music/song.flac", "media/movies", "music"} {
|
||||
if containsSuffix(got, bad) {
|
||||
t.Errorf("excluded path %q present in tar: %v", bad, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func tarEntries(t *testing.T, tarPath string) []string {
|
||||
t.Helper()
|
||||
f, err := os.Open(tarPath)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer f.Close()
|
||||
tr := tar.NewReader(f)
|
||||
var names []string
|
||||
for {
|
||||
h, err := tr.Next()
|
||||
if err == io.EOF {
|
||||
break
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
names = append(names, filepath.ToSlash(h.Name))
|
||||
}
|
||||
sort.Strings(names)
|
||||
return names
|
||||
}
|
||||
|
||||
func contains(ss []string, want string) bool {
|
||||
for _, s := range ss {
|
||||
if s == want {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
func containsSuffix(ss []string, suffix string) bool {
|
||||
for _, s := range ss {
|
||||
if s == suffix || filepath.ToSlash(s) == suffix {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
@@ -38,11 +38,5 @@ func UnmarshalManifest(data []byte) (*Manifest, error) {
|
||||
if err := json.Unmarshal(data, &m); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// [CTRL-001] Reject path-traversal in any segment used to build a filesystem
|
||||
// path on import (app_name, hdd_subdirs, volume_names). A hostile .fab must
|
||||
// fail to parse rather than escape the stacks / HDD destination dir.
|
||||
if err := validateManifestPaths(&m); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &m, nil
|
||||
}
|
||||
|
||||
@@ -4,8 +4,6 @@
|
||||
// the app to its current state.
|
||||
package appexport
|
||||
|
||||
import "gitea.dooplex.hu/admin/felhom-controller/internal/appbackup"
|
||||
|
||||
// ExportStackProvider provides stack data without circular imports.
|
||||
// Implemented by exportAdapter in main.go (same pattern as backup.StackDataProvider).
|
||||
type ExportStackProvider interface {
|
||||
@@ -17,18 +15,6 @@ type ExportStackProvider interface {
|
||||
GetStackHDDMounts(name string) []string
|
||||
// GetStackHDDPath returns the raw HDD_PATH env var from app.yaml.
|
||||
GetStackHDDPath(name string) string
|
||||
// GetImportRoot returns the CANONICAL drop-zone root (R-75), on the SYSTEM drive. ${IMPORT_PATH}
|
||||
// binds resolve against THIS, never against GetStackHDDPath. Empty when unresolvable.
|
||||
GetImportRoot() string
|
||||
// GetStackNamespaceRoot returns the app's felhom-data NAMESPACE ROOT — the directory that directly
|
||||
// contains backups/ and userdata/. It is NOT GetStackHDDPath: on an enrolled drive the two are the
|
||||
// same, and on the system-data fallback the namespace root has one more segment (R-203). Every
|
||||
// appbackup path helper takes THIS, never the drive path. Empty when the app has no HDD_PATH.
|
||||
GetStackNamespaceRoot(name string) string
|
||||
// GetStackClassifiedBinds returns the app's backup-classified compose binds + whether it carries a
|
||||
// (valid) backup block (Task 2). Drives the `.fab` class-scoped export plan (Task 4); a legacy app
|
||||
// (false) exports the v0.130.0 full-root capture unchanged.
|
||||
GetStackClassifiedBinds(name string) ([]appbackup.ClassifiedBind, bool)
|
||||
// IsStackRunning returns true if the stack has running containers.
|
||||
IsStackRunning(name string) bool
|
||||
// StopStack stops the stack via docker compose down.
|
||||
|
||||
@@ -326,16 +326,6 @@ func (e *Exporter) executeImport(req ImportRequest, job *Job) {
|
||||
job.mu.Unlock()
|
||||
|
||||
e.logger.Printf("[INFO] Import: opening bundle for %s (%s)", manifest.AppName, manifest.DisplayName)
|
||||
|
||||
// v0.125.0 validate-before-destroy (scenario C): every manifest-claimed data tar must be
|
||||
// present and non-empty BEFORE the app is stopped and BEFORE any volume is removed. Hollow
|
||||
// bundles (a containerized v<=0.124.0 exporter stranded the tars host-side) land HERE — the
|
||||
// pre-fix order wiped the volumes first and only then discovered the emptiness.
|
||||
if err := validateBundleData(tmpDir, manifest); err != nil {
|
||||
e.failJob(job, step, fmt.Sprintf("A csomag hiányos — az importálás el sem indult, a meglévő alkalmazás érintetlen. (%v) A csomagot valószínűleg egy régebbi (≤0.124.0), konténerben futó vezérlő exportálta — készíts friss exportot.", err))
|
||||
return
|
||||
}
|
||||
|
||||
e.debugf("step 0 (open bundle) done in %v", time.Since(stepStart))
|
||||
|
||||
job.setStep(step, "done", "")
|
||||
@@ -599,17 +589,12 @@ func (e *Exporter) restoreHDDData(tmpDir string, manifest *Manifest, composePath
|
||||
}
|
||||
|
||||
for _, subdir := range manifest.HDDSubdirs {
|
||||
// [CTRL-001] defence-in-depth: refuse any subdir that is not a single
|
||||
// safe segment before it reaches MkdirAll/extractTar on a user drive.
|
||||
if err := ValidateSegment("hdd_subdir", subdir); err != nil {
|
||||
return err
|
||||
}
|
||||
tarPath := filepath.Join(hddDir, subdir+".tar")
|
||||
tarInfo, err := os.Stat(tarPath)
|
||||
if err != nil {
|
||||
// v0.125.0: a claimed-but-absent tar is an ERROR (validateBundleData already refused
|
||||
// it pre-destroy; this is defense in depth, not a reachable soft path).
|
||||
return fmt.Errorf("HDD tar missing from bundle: %s", subdir+".tar")
|
||||
e.logger.Printf("[WARN] Import: HDD tar not found: %s", tarPath)
|
||||
e.debugf("restoreHDDData: tar not found: %s", tarPath)
|
||||
continue
|
||||
}
|
||||
e.debugf("restoreHDDData: subdir=%s tarSize=%s", subdir, humanizeBytes(tarInfo.Size()))
|
||||
|
||||
@@ -680,82 +665,42 @@ func resolveHDDMounts(composePath string, env map[string]string) []string {
|
||||
return mounts
|
||||
}
|
||||
|
||||
// validateBundleData asserts every manifest-claimed data tar exists non-empty in the extracted
|
||||
// bundle (v0.125.0, scenario C). Pure read — called before ANY destructive import step.
|
||||
func validateBundleData(tmpDir string, manifest *Manifest) error {
|
||||
for _, v := range manifest.VolumeNames {
|
||||
if err := ValidateSegment("volume_name", v); err != nil {
|
||||
return err
|
||||
}
|
||||
fi, err := os.Stat(filepath.Join(tmpDir, "data", "volumes", v+".tar"))
|
||||
if err != nil || fi.Size() == 0 {
|
||||
return fmt.Errorf("a(z) %q kötet adata hiányzik a csomagból", v)
|
||||
}
|
||||
}
|
||||
for _, s := range manifest.HDDSubdirs {
|
||||
if err := ValidateSegment("hdd_subdir", s); err != nil {
|
||||
return err
|
||||
}
|
||||
fi, err := os.Stat(filepath.Join(tmpDir, "data", "hdd", s+".tar"))
|
||||
if err != nil || fi.Size() == 0 {
|
||||
return fmt.Errorf("a(z) %q adatkönyvtár tartalma hiányzik a csomagból", s)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// importVolumeTar streams tarPath into the (existing) volume via `docker cp - <cid>:/vol` —
|
||||
// zero shared paths, correct on bare metal AND under the containerized controller (the
|
||||
// docker-run -v population was the v0.124.0 strand's import half).
|
||||
func (e *Exporter) importVolumeTar(volName, tarPath string) error {
|
||||
return e.withVolumeHelper(volName, func(cid string) error {
|
||||
f, err := os.Open(tarPath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("opening %s: %w", tarPath, err)
|
||||
}
|
||||
defer f.Close()
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Minute)
|
||||
defer cancel()
|
||||
if stderr, err := dockerExec(ctx, f, nil, "cp", "-", cid+":/vol"); err != nil {
|
||||
return fmt.Errorf("streaming into volume %s: %s — %w", volName, stderr, err)
|
||||
}
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
// restoreVolumeData recreates Docker named volumes from bundle tarballs. v0.125.0: a missing
|
||||
// tar is an ERROR (defense in depth behind validateBundleData), and population streams via
|
||||
// docker cp instead of a docker-run -v host mount.
|
||||
// restoreVolumeData recreates Docker named volumes from bundle tarballs.
|
||||
func (e *Exporter) restoreVolumeData(tmpDir string, manifest *Manifest) error {
|
||||
volDir := filepath.Join(tmpDir, "data", "volumes")
|
||||
|
||||
for _, volName := range manifest.VolumeNames {
|
||||
// [CTRL-001] defence-in-depth: refuse any volume name that is not a
|
||||
// single safe segment before it reaches a tar path / docker volume op.
|
||||
if err := ValidateSegment("volume_name", volName); err != nil {
|
||||
return err
|
||||
}
|
||||
tarPath := filepath.Join(volDir, volName+".tar")
|
||||
tarInfo, err := os.Stat(tarPath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("volume tar missing from bundle: %s", volName+".tar")
|
||||
e.logger.Printf("[WARN] Import: volume tar not found: %s", tarPath)
|
||||
e.debugf("restoreVolumeData: tar not found: %s", tarPath)
|
||||
continue
|
||||
}
|
||||
e.debugf("restoreVolumeData: volume=%s tarSize=%s", volName, humanizeBytes(tarInfo.Size()))
|
||||
|
||||
// Create the Docker volume
|
||||
e.debugf("restoreVolumeData: creating docker volume %s", volName)
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
||||
stderr, err := dockerExec(ctx, nil, nil, "volume", "create", volName)
|
||||
out, err := exec.CommandContext(ctx, "docker", "volume", "create", volName).CombinedOutput()
|
||||
cancel()
|
||||
if err != nil {
|
||||
return fmt.Errorf("creating volume %s: %s — %w", volName, stderr, err)
|
||||
return fmt.Errorf("creating volume %s: %s — %w", volName, strings.TrimSpace(string(out)), err)
|
||||
}
|
||||
e.debugf("restoreVolumeData: volume %s created: %s", volName, strings.TrimSpace(string(out)))
|
||||
|
||||
// Populate volume from tar (docker cp streaming)
|
||||
// Populate volume from tar
|
||||
e.logger.Printf("[INFO] Import: populating volume %s", volName)
|
||||
e.debugf("restoreVolumeData: populating %s via docker run alpine tar xf...", volName)
|
||||
popStart := time.Now()
|
||||
if err := e.importVolumeTar(volName, tarPath); err != nil {
|
||||
return fmt.Errorf("populating volume %s: %w", volName, err)
|
||||
ctx, cancel = context.WithTimeout(context.Background(), 10*time.Minute)
|
||||
out, err = exec.CommandContext(ctx, "docker", "run", "--rm",
|
||||
"-v", volName+":/vol",
|
||||
"-v", volDir+":/in:ro",
|
||||
"alpine", "tar", "xf", "/in/"+volName+".tar", "-C", "/vol").CombinedOutput()
|
||||
cancel()
|
||||
if err != nil {
|
||||
return fmt.Errorf("populating volume %s: %s — %w", volName, strings.TrimSpace(string(out)), err)
|
||||
}
|
||||
e.debugf("restoreVolumeData: volume %s populated in %v", volName, time.Since(popStart))
|
||||
}
|
||||
|
||||
@@ -1,190 +0,0 @@
|
||||
package appexport
|
||||
|
||||
import (
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/appbackup"
|
||||
"io"
|
||||
"log"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// v0.124.0 Part 3 (§7D) — the .fab loop's proof at unit level: a real export through
|
||||
// executeExport produces a bundle that a real executeImport restores to identical content, and
|
||||
// a corrupted downloaded copy is REFUSED (the .fab's integrity is the gzip CRC + the manifest
|
||||
// segment validation — there is no per-file checksum; corruption breaks the extract, and the
|
||||
// import must fail loudly, not restore garbage).
|
||||
|
||||
// rtProvider is a filesystem-only fake: a config-only app (no HDD, no volumes, no DB) so the
|
||||
// whole loop runs without docker.
|
||||
type rtProvider struct {
|
||||
stackDir string
|
||||
stacksDir string
|
||||
deployed bool
|
||||
running bool
|
||||
volumes []string
|
||||
started bool
|
||||
stopped int
|
||||
removed int
|
||||
savedEnv map[string]string
|
||||
}
|
||||
|
||||
func (p *rtProvider) GetStackDir(string) (string, bool) { return p.stackDir, true }
|
||||
func (p *rtProvider) GetStackComposePath(string) (string, bool) {
|
||||
return filepath.Join(p.stackDir, "docker-compose.yml"), true
|
||||
}
|
||||
func (p *rtProvider) GetStackHDDMounts(string) []string { return nil }
|
||||
func (p *rtProvider) GetStackHDDPath(string) string { return "" }
|
||||
func (p *rtProvider) GetImportRoot() string { return "" } // R-75: no import binds in this fixture
|
||||
|
||||
// R-203: these fixtures use ENROLLED drive paths, where the namespace root IS the drive path.
|
||||
// Delegating keeps that identity explicit rather than hardcoding it.
|
||||
func (p *rtProvider) GetStackNamespaceRoot(name string) string { return p.GetStackHDDPath(name) }
|
||||
func (p *rtProvider) GetStackClassifiedBinds(string) ([]appbackup.ClassifiedBind, bool) {
|
||||
return nil, false
|
||||
}
|
||||
func (p *rtProvider) IsStackRunning(string) bool { return p.running }
|
||||
func (p *rtProvider) StopStack(string) error { p.stopped++; return nil }
|
||||
func (p *rtProvider) StartStack(string) error { p.started = true; return nil }
|
||||
func (p *rtProvider) GetStackDisplayName(n string) string { return "RT " + n }
|
||||
func (p *rtProvider) GetStackNeedsHDD(string) bool { return false }
|
||||
func (p *rtProvider) GetDockerVolumes(string) []string { return p.volumes }
|
||||
func (p *rtProvider) IsStackDeployed(string) bool { return p.deployed }
|
||||
func (p *rtProvider) GetDecryptedEnv(string) map[string]string { return nil }
|
||||
func (p *rtProvider) GetStacksBaseDir() string { return p.stacksDir }
|
||||
func (p *rtProvider) RefreshStacks() error { return nil }
|
||||
func (p *rtProvider) RemoveStackVolumes(string) error { p.removed++; return nil }
|
||||
func (p *rtProvider) SaveEncryptedAppConfig(stackDir string, env map[string]string) error {
|
||||
p.savedEnv = env
|
||||
return nil
|
||||
}
|
||||
|
||||
func waitJob(t *testing.T, e *Exporter) *Job {
|
||||
t.Helper()
|
||||
deadline := time.Now().Add(30 * time.Second)
|
||||
for time.Now().Before(deadline) {
|
||||
job := e.GetActiveJob()
|
||||
if job != nil {
|
||||
job.mu.RLock()
|
||||
done := job.Done
|
||||
job.mu.RUnlock()
|
||||
if done {
|
||||
return job
|
||||
}
|
||||
}
|
||||
time.Sleep(50 * time.Millisecond)
|
||||
}
|
||||
t.Fatal("job did not finish in time")
|
||||
return nil
|
||||
}
|
||||
|
||||
func jobErr(j *Job) string {
|
||||
j.mu.RLock()
|
||||
defer j.mu.RUnlock()
|
||||
if j.Error != "" {
|
||||
return j.Error
|
||||
}
|
||||
for _, s := range j.Steps {
|
||||
if s.Status == "failed" {
|
||||
return s.Error
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func TestFabRoundTrip_ExportImportContentEquality(t *testing.T) {
|
||||
const stack = "rt-app"
|
||||
lg := log.New(io.Discard, "", 0)
|
||||
|
||||
// Source stack: a compose file + a marker config with known content.
|
||||
srcStack := t.TempDir()
|
||||
compose := "services:\n rt-app:\n image: alpine\n"
|
||||
marker := "MARKER-CONTENT-42\n"
|
||||
os.WriteFile(filepath.Join(srcStack, "docker-compose.yml"), []byte(compose), 0644)
|
||||
os.WriteFile(filepath.Join(srcStack, "settings.conf"), []byte(marker), 0644)
|
||||
|
||||
drive := t.TempDir()
|
||||
prov := &rtProvider{stackDir: srcStack, stacksDir: t.TempDir(), deployed: true}
|
||||
e := NewExporter(prov, lg, "test")
|
||||
|
||||
// --- export (the REAL pipeline; same producer as a drive export) ---
|
||||
if err := e.StartExport(ExportRequest{StackName: stack, DestDrive: drive}); err != nil {
|
||||
t.Fatalf("StartExport: %v", err)
|
||||
}
|
||||
job := waitJob(t, e)
|
||||
if msg := jobErr(job); msg != "" {
|
||||
t.Fatalf("export failed: %s", msg)
|
||||
}
|
||||
entries, _ := os.ReadDir(ExportDir(drive))
|
||||
var fabPath string
|
||||
for _, en := range entries {
|
||||
if strings.HasSuffix(en.Name(), ".fab") {
|
||||
fabPath = filepath.Join(ExportDir(drive), en.Name())
|
||||
}
|
||||
}
|
||||
if fabPath == "" {
|
||||
t.Fatal("no .fab produced")
|
||||
}
|
||||
|
||||
// The manifest is readable and names the app (what /api/export/manifest shows pre-import).
|
||||
man, err := ReadManifestFromFAB(fabPath)
|
||||
if err != nil {
|
||||
t.Fatalf("manifest: %v", err)
|
||||
}
|
||||
if man.AppName != stack {
|
||||
t.Fatalf("manifest app = %q", man.AppName)
|
||||
}
|
||||
|
||||
// --- corrupted copy must be REFUSED (assert the refusal, §10 red-proof of the loop) ---
|
||||
corrupt := filepath.Join(t.TempDir(), "corrupt.fab")
|
||||
raw, _ := os.ReadFile(fabPath)
|
||||
mid := len(raw) / 2
|
||||
raw[mid] ^= 0xFF
|
||||
raw[mid+1] ^= 0xFF
|
||||
os.WriteFile(corrupt, raw, 0644)
|
||||
|
||||
prov2 := &rtProvider{stackDir: srcStack, stacksDir: t.TempDir(), deployed: false}
|
||||
e2 := NewExporter(prov2, lg, "test")
|
||||
if err := e2.StartImport(ImportRequest{FABPath: corrupt}); err != nil {
|
||||
t.Fatalf("StartImport(corrupt) should start (refusal is async): %v", err)
|
||||
}
|
||||
job = waitJob(t, e2)
|
||||
if msg := jobErr(job); msg == "" {
|
||||
t.Fatal("a corrupted bundle must FAIL the import (gzip CRC), got success")
|
||||
}
|
||||
if prov2.started {
|
||||
t.Fatal("a refused import must not start the app")
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(prov2.stacksDir, stack, "settings.conf")); !os.IsNotExist(err) {
|
||||
t.Fatal("a refused import must not restore content")
|
||||
}
|
||||
|
||||
// --- the clean bundle round-trips: restored content is byte-identical ---
|
||||
prov3 := &rtProvider{stackDir: srcStack, stacksDir: t.TempDir(), deployed: false}
|
||||
e3 := NewExporter(prov3, lg, "test")
|
||||
if err := e3.StartImport(ImportRequest{FABPath: fabPath}); err != nil {
|
||||
t.Fatalf("StartImport: %v", err)
|
||||
}
|
||||
job = waitJob(t, e3)
|
||||
if msg := jobErr(job); msg != "" {
|
||||
t.Fatalf("import failed: %s", msg)
|
||||
}
|
||||
restoredStack := filepath.Join(prov3.stacksDir, stack)
|
||||
for name, want := range map[string]string{
|
||||
"docker-compose.yml": compose,
|
||||
"settings.conf": marker,
|
||||
} {
|
||||
got, err := os.ReadFile(filepath.Join(restoredStack, name))
|
||||
if err != nil {
|
||||
t.Fatalf("restored %s missing: %v", name, err)
|
||||
}
|
||||
if string(got) != want {
|
||||
t.Errorf("restored %s differs:\n got %q\nwant %q", name, got, want)
|
||||
}
|
||||
}
|
||||
if !prov3.started {
|
||||
t.Error("import must start the restored app")
|
||||
}
|
||||
}
|
||||
@@ -1,82 +0,0 @@
|
||||
package appexport
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// Regression test for [CTRL-001] (path traversal on .fab import). Originated as
|
||||
// a failing audit test (audit/2026-06-13-deep-sweep); now a permanent guard.
|
||||
// UnmarshalManifest must REJECT any manifest whose AppName / HDDSubdirs /
|
||||
// VolumeNames contain a path-traversal or separator, and ACCEPT legitimate
|
||||
// single-segment names. Do NOT weaken these assertions.
|
||||
|
||||
func mustManifestJSON(t *testing.T, m Manifest) []byte {
|
||||
t.Helper()
|
||||
b, err := json.Marshal(m)
|
||||
if err != nil {
|
||||
t.Fatalf("marshal: %v", err)
|
||||
}
|
||||
return b
|
||||
}
|
||||
|
||||
func TestUnmarshalManifestRejectsTraversal(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
m Manifest
|
||||
}{
|
||||
{"appname-parent", Manifest{Version: 1, AppName: "../evil"}},
|
||||
{"appname-deep", Manifest{Version: 1, AppName: "../../etc/cron.d/x"}},
|
||||
{"appname-absolute", Manifest{Version: 1, AppName: "/etc/cron.d/x"}},
|
||||
{"appname-dotdot", Manifest{Version: 1, AppName: ".."}},
|
||||
{"appname-empty", Manifest{Version: 1, AppName: ""}},
|
||||
{"appname-backslash", Manifest{Version: 1, AppName: `..\evil`}},
|
||||
{"hdd-subdir-escape", Manifest{Version: 1, AppName: "romm", HDDSubdirs: []string{"../../mnt"}}},
|
||||
{"volume-escape", Manifest{Version: 1, AppName: "romm", VolumeNames: []string{"../../var/lib"}}},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
_, err := UnmarshalManifest(mustManifestJSON(t, tc.m))
|
||||
if err == nil {
|
||||
t.Fatalf("CTRL-001 regression: UnmarshalManifest accepted a traversal manifest %+v; expected rejection", tc.m)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestUnmarshalManifestAcceptsLegitNames(t *testing.T) {
|
||||
m := Manifest{
|
||||
Version: 1,
|
||||
AppName: "paperless-ngx",
|
||||
HDDSubdirs: []string{"felhom-usb", "romm"},
|
||||
VolumeNames: []string{"adventurelog_postgres_data", "romm_redis-data"},
|
||||
ConfigFiles: []string{".felhom.yml", "docker-compose.yml", "app.yaml"}, // dotfiles must NOT be rejected
|
||||
}
|
||||
got, err := UnmarshalManifest(mustManifestJSON(t, m))
|
||||
if err != nil {
|
||||
t.Fatalf("CTRL-001 regression: UnmarshalManifest rejected a legitimate manifest: %v", err)
|
||||
}
|
||||
if got.AppName != "paperless-ngx" {
|
||||
t.Fatalf("AppName round-trip mismatch: %q", got.AppName)
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidateSegment(t *testing.T) {
|
||||
good := []string{"romm", "paperless-ngx", "adventurelog_postgres_data", "felhom-usb", "a", "App1.2_3-4"}
|
||||
for _, s := range good {
|
||||
if err := ValidateSegment("x", s); err != nil {
|
||||
t.Errorf("ValidateSegment(%q) = %v; want nil", s, err)
|
||||
}
|
||||
}
|
||||
bad := []string{"", ".", "..", "../x", "a/b", `a\b`, "/abs", ".hidden", "-leadingdash", "a/../b"}
|
||||
for _, s := range bad {
|
||||
if err := ValidateSegment("x", s); err == nil {
|
||||
t.Errorf("ValidateSegment(%q) = nil; want rejection", s)
|
||||
}
|
||||
}
|
||||
// Sanity: a rejected value's message names the kind, for operator clarity.
|
||||
if err := ValidateSegment("app_name", "../x"); err == nil || !strings.Contains(err.Error(), "app_name") {
|
||||
t.Errorf("expected error mentioning app_name, got %v", err)
|
||||
}
|
||||
}
|
||||
@@ -1,66 +0,0 @@
|
||||
package appexport
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// safeSegment matches a single safe path component: starts with an
|
||||
// alphanumeric, then alphanumerics / dot / dash / underscore. It cannot be
|
||||
// "." or ".." (must start alnum), cannot contain a path separator, and cannot
|
||||
// be an absolute path. This covers the legitimate values these fields hold —
|
||||
// app slugs (e.g. "paperless-ngx"), HDD mount basenames (e.g. "felhom-usb"),
|
||||
// and docker volume names (e.g. "adventurelog_postgres_data").
|
||||
var safeSegment = regexp.MustCompile(`^[a-zA-Z0-9][a-zA-Z0-9._-]*$`)
|
||||
|
||||
// ValidateSegment rejects any value that is not a single safe path component.
|
||||
// It is the guard for [CTRL-001]: manifest fields that reach filepath.Join with
|
||||
// a trusted base (AppName, HDDSubdirs, VolumeNames) are fully attacker-controlled
|
||||
// JSON inside an imported .fab, so a value like "../../etc/cron.d/x" must be
|
||||
// refused before it can escape the stacks / HDD destination directory.
|
||||
//
|
||||
// NOTE: this is deliberately NOT applied to manifest.ConfigFiles — those are
|
||||
// dotfile-bearing names (e.g. ".felhom.yml") that are never used in a restore
|
||||
// join (restoreConfig enumerates the extracted dir via os.ReadDir, whose names
|
||||
// are already single components).
|
||||
func ValidateSegment(kind, s string) error {
|
||||
if s == "" {
|
||||
return fmt.Errorf("appexport: empty %s", kind)
|
||||
}
|
||||
if s == "." || s == ".." {
|
||||
return fmt.Errorf("appexport: %s %q is a path-traversal segment", kind, s)
|
||||
}
|
||||
if strings.ContainsAny(s, `/\`) || strings.ContainsRune(s, filepath.Separator) {
|
||||
return fmt.Errorf("appexport: %s %q must not contain a path separator", kind, s)
|
||||
}
|
||||
if filepath.IsAbs(s) {
|
||||
return fmt.Errorf("appexport: %s %q must not be an absolute path", kind, s)
|
||||
}
|
||||
if !safeSegment.MatchString(s) {
|
||||
return fmt.Errorf("appexport: %s %q is not a safe single-segment name", kind, s)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// validateManifestPaths checks every manifest field that is later used as a
|
||||
// path segment in a filepath.Join against a trusted base. Called from
|
||||
// UnmarshalManifest so a hostile bundle fails the parse, before executeImport
|
||||
// can MkdirAll/extract into a traversed location.
|
||||
func validateManifestPaths(m *Manifest) error {
|
||||
if err := ValidateSegment("app_name", m.AppName); err != nil {
|
||||
return err
|
||||
}
|
||||
for _, s := range m.HDDSubdirs {
|
||||
if err := ValidateSegment("hdd_subdir", s); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
for _, v := range m.VolumeNames {
|
||||
if err := ValidateSegment("volume_name", v); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -1,234 +0,0 @@
|
||||
package appexport
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"log"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// v0.125.0 — the containerized-.fab volume-strand fix (IA finding 1, HIGH). These tests pin:
|
||||
// (B) export can no longer lie — a stranded/empty tar aborts the export, no bundle;
|
||||
// (C) import validates BEFORE it destroys — a hollow bundle is refused with the app untouched;
|
||||
// the docker-cp command construction (image, mount shape, cp direction) and the always-remove
|
||||
// helper-container rule. The real docker legs are covered by the §3 live probe + §13 round-trip.
|
||||
|
||||
// dockerCall records one dockerExec invocation.
|
||||
type dockerCall struct {
|
||||
args []string
|
||||
stdin bool
|
||||
}
|
||||
|
||||
// swapDockerExec installs a scripted fake for the package seam and restores it on cleanup.
|
||||
func swapDockerExec(t *testing.T, fn func(call dockerCall, stdin io.Reader, stdout io.Writer) (string, error)) *[]dockerCall {
|
||||
t.Helper()
|
||||
var mu sync.Mutex
|
||||
calls := &[]dockerCall{}
|
||||
orig := dockerExec
|
||||
dockerExec = func(ctx context.Context, stdin io.Reader, stdout io.Writer, args ...string) (string, error) {
|
||||
mu.Lock()
|
||||
c := dockerCall{args: append([]string{}, args...), stdin: stdin != nil}
|
||||
*calls = append(*calls, c)
|
||||
mu.Unlock()
|
||||
return fn(c, stdin, stdout)
|
||||
}
|
||||
t.Cleanup(func() { dockerExec = orig })
|
||||
return calls
|
||||
}
|
||||
|
||||
func volTestExporter(t *testing.T, volumes []string) (*Exporter, *rtProvider, string) {
|
||||
t.Helper()
|
||||
srcStack := t.TempDir()
|
||||
os.WriteFile(filepath.Join(srcStack, "docker-compose.yml"), []byte("services:\n vol-app:\n image: alpine\n"), 0644)
|
||||
prov := &rtProvider{stackDir: srcStack, stacksDir: t.TempDir(), deployed: true, volumes: volumes}
|
||||
drive := t.TempDir()
|
||||
return NewExporter(prov, log.New(io.Discard, "", 0), "test"), prov, drive
|
||||
}
|
||||
|
||||
// Scenario B: a volume tar that fails to materialize (cp "succeeds" but writes nothing — the
|
||||
// strand's signature) must FAIL the export with the volume named, and NO bundle may exist.
|
||||
// Red-proof: remove the assertBundleDataComplete call → this test fails (hollow success).
|
||||
func TestExport_HollowVolumeTarAbortsExport(t *testing.T) {
|
||||
swapDockerExec(t, func(c dockerCall, stdin io.Reader, stdout io.Writer) (string, error) {
|
||||
switch c.args[0] {
|
||||
case "create":
|
||||
fmt.Fprint(stdout, "cid-123\n")
|
||||
return "", nil
|
||||
case "cp":
|
||||
return "", nil // writes NOTHING to stdout — the stranded-tar signature
|
||||
case "rm":
|
||||
return "", nil
|
||||
}
|
||||
return "", fmt.Errorf("unexpected docker call: %v", c.args)
|
||||
})
|
||||
|
||||
e, _, drive := volTestExporter(t, []string{"vol1"})
|
||||
if err := e.StartExport(ExportRequest{StackName: "vol-app", DestDrive: drive}); err != nil {
|
||||
t.Fatalf("StartExport: %v", err)
|
||||
}
|
||||
job := waitJob(t, e)
|
||||
msg := jobErr(job)
|
||||
if msg == "" {
|
||||
t.Fatal("a hollow volume tar must FAIL the export — got success")
|
||||
}
|
||||
if !strings.Contains(msg, "vol1") {
|
||||
t.Errorf("the error must NAME the missing volume, got %q", msg)
|
||||
}
|
||||
entries, _ := os.ReadDir(ExportDir(drive))
|
||||
for _, en := range entries {
|
||||
if strings.HasSuffix(en.Name(), ".fab") {
|
||||
t.Fatalf("a bundle was produced despite the hollow tar: %s", en.Name())
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Scenario C: a bundle whose manifest claims a volume without its tar is refused BEFORE any
|
||||
// destructive step — the app is not stopped, no volume is removed or recreated, zero docker
|
||||
// calls happen. Red-proof: disable the pre-flight (pre-fix order: wipe first, discover later)
|
||||
// → the zero-destruction assertions fail.
|
||||
func TestImport_HollowBundleRefusedBeforeDestroy(t *testing.T) {
|
||||
calls := swapDockerExec(t, func(c dockerCall, stdin io.Reader, stdout io.Writer) (string, error) {
|
||||
return "", nil
|
||||
})
|
||||
|
||||
// Handcraft a hollow bundle: manifest CLAIMS volume data, data/volumes is empty.
|
||||
tree := t.TempDir()
|
||||
os.MkdirAll(filepath.Join(tree, "config"), 0755)
|
||||
os.MkdirAll(filepath.Join(tree, "data", "volumes"), 0755)
|
||||
os.WriteFile(filepath.Join(tree, "config", "docker-compose.yml"), []byte("services: {}\n"), 0644)
|
||||
man := &Manifest{
|
||||
Version: ManifestVersion, AppName: "vol-app", DisplayName: "Vol App",
|
||||
HasVolumeData: true, VolumeNames: []string{"vol1"},
|
||||
ConfigFiles: []string{"docker-compose.yml"},
|
||||
}
|
||||
data, err := man.Marshal()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
os.WriteFile(filepath.Join(tree, "manifest.json"), data, 0644)
|
||||
fab := filepath.Join(t.TempDir(), "hollow.fab")
|
||||
if err := createTarGz(fab, tree); err != nil {
|
||||
t.Fatalf("createTarGz: %v", err)
|
||||
}
|
||||
|
||||
prov := &rtProvider{stackDir: t.TempDir(), stacksDir: t.TempDir(), deployed: true, running: true}
|
||||
e := NewExporter(prov, log.New(io.Discard, "", 0), "test")
|
||||
if err := e.StartImport(ImportRequest{FABPath: fab}); err != nil {
|
||||
t.Fatalf("StartImport: %v", err)
|
||||
}
|
||||
job := waitJob(t, e)
|
||||
msg := jobErr(job)
|
||||
if msg == "" {
|
||||
t.Fatal("a hollow bundle must be REFUSED — got success")
|
||||
}
|
||||
if !strings.Contains(msg, "érintetlen") {
|
||||
t.Errorf("refusal copy must state the app is untouched, got %q", msg)
|
||||
}
|
||||
// THE exact non-effects: nothing was stopped, wiped, recreated or started.
|
||||
if prov.stopped != 0 || prov.removed != 0 {
|
||||
t.Fatalf("refusal happened AFTER destruction: stopped=%d removedVolumes=%d", prov.stopped, prov.removed)
|
||||
}
|
||||
if prov.started {
|
||||
t.Fatal("a refused import must not start the app")
|
||||
}
|
||||
if len(*calls) != 0 {
|
||||
t.Fatalf("a refused import must make ZERO docker calls, got %v", *calls)
|
||||
}
|
||||
}
|
||||
|
||||
// Command construction + helper hygiene: the export leg uses create/cp/rm with the exact arg
|
||||
// shapes the §3 probe validated, and the helper container is force-removed EVEN when cp fails.
|
||||
func TestExportVolumeTar_CommandShapesAndHelperCleanup(t *testing.T) {
|
||||
t.Run("happy path shapes", func(t *testing.T) {
|
||||
calls := swapDockerExec(t, func(c dockerCall, stdin io.Reader, stdout io.Writer) (string, error) {
|
||||
switch c.args[0] {
|
||||
case "create":
|
||||
fmt.Fprint(stdout, "cid-abc\n")
|
||||
case "cp":
|
||||
fmt.Fprint(stdout, "TARBYTES")
|
||||
}
|
||||
return "", nil
|
||||
})
|
||||
e, _, _ := volTestExporter(t, nil)
|
||||
tarPath := filepath.Join(t.TempDir(), "v.tar")
|
||||
if err := e.exportVolumeTar("vol1", tarPath); err != nil {
|
||||
t.Fatalf("exportVolumeTar: %v", err)
|
||||
}
|
||||
got, _ := os.ReadFile(tarPath)
|
||||
if string(got) != "TARBYTES" {
|
||||
t.Fatalf("tar content = %q", got)
|
||||
}
|
||||
want := [][]string{
|
||||
{"create", "-v", "vol1:/vol", "alpine", "true"},
|
||||
{"cp", "cid-abc:/vol/.", "-"},
|
||||
{"rm", "-f", "cid-abc"},
|
||||
}
|
||||
assertCalls(t, *calls, want)
|
||||
})
|
||||
|
||||
t.Run("helper removed on cp failure", func(t *testing.T) {
|
||||
calls := swapDockerExec(t, func(c dockerCall, stdin io.Reader, stdout io.Writer) (string, error) {
|
||||
switch c.args[0] {
|
||||
case "create":
|
||||
fmt.Fprint(stdout, "cid-err\n")
|
||||
return "", nil
|
||||
case "cp":
|
||||
return "boom", fmt.Errorf("cp failed")
|
||||
}
|
||||
return "", nil
|
||||
})
|
||||
e, _, _ := volTestExporter(t, nil)
|
||||
tarPath := filepath.Join(t.TempDir(), "v.tar")
|
||||
if err := e.exportVolumeTar("vol1", tarPath); err == nil {
|
||||
t.Fatal("cp failure must surface")
|
||||
}
|
||||
if _, err := os.Stat(tarPath); !os.IsNotExist(err) {
|
||||
t.Error("a failed export must not leave a partial tar")
|
||||
}
|
||||
last := (*calls)[len(*calls)-1]
|
||||
if strings.Join(last.args, " ") != "rm -f cid-err" {
|
||||
t.Fatalf("helper container must be force-removed on failure, last call: %v", last.args)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("import leg shapes", func(t *testing.T) {
|
||||
calls := swapDockerExec(t, func(c dockerCall, stdin io.Reader, stdout io.Writer) (string, error) {
|
||||
if c.args[0] == "create" {
|
||||
fmt.Fprint(stdout, "cid-imp\n")
|
||||
}
|
||||
if c.args[0] == "cp" && !c.stdin {
|
||||
t.Error("import cp must stream the tar on stdin")
|
||||
}
|
||||
return "", nil
|
||||
})
|
||||
e, _, _ := volTestExporter(t, nil)
|
||||
tarPath := filepath.Join(t.TempDir(), "v.tar")
|
||||
os.WriteFile(tarPath, []byte("TAR"), 0644)
|
||||
if err := e.importVolumeTar("vol1", tarPath); err != nil {
|
||||
t.Fatalf("importVolumeTar: %v", err)
|
||||
}
|
||||
want := [][]string{
|
||||
{"create", "-v", "vol1:/vol", "alpine", "true"},
|
||||
{"cp", "-", "cid-imp:/vol"},
|
||||
{"rm", "-f", "cid-imp"},
|
||||
}
|
||||
assertCalls(t, *calls, want)
|
||||
})
|
||||
}
|
||||
|
||||
func assertCalls(t *testing.T, got []dockerCall, want [][]string) {
|
||||
t.Helper()
|
||||
if len(got) != len(want) {
|
||||
t.Fatalf("docker calls = %d, want %d (%v)", len(got), len(want), got)
|
||||
}
|
||||
for i := range want {
|
||||
if strings.Join(got[i].args, " ") != strings.Join(want[i], " ") {
|
||||
t.Errorf("call %d = %v, want %v", i, got[i].args, want[i])
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,252 +0,0 @@
|
||||
package backup
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// ── Backup admission (R-181) ─────────────────────────────────────────────────────────────────────
|
||||
//
|
||||
// WHAT WAS WRONG. B2's capture floor (v0.192.0, R-165) shipped as the deliberate replacement for the
|
||||
// bulkhead the `mp1` partition used to give, and it was consulted in exactly ONE place —
|
||||
// `captureAllRecoveryUnits`, which writes a manifest and three compose files: a few KB. The two legs
|
||||
// that write the BULK into the same `backups/primary/<app>` tree — the database dump and the volume
|
||||
// dump — ran FIRST and unguarded. Measured on demo-hp 2026-08-03 06:40:03: opengist's volume dump
|
||||
// wrote 2.0 GB with no check, free fell to 1.0 GB, and the floor then refused the cheap write it had
|
||||
// already lost the argument to. Its refusal message said *"the previous unit is untouched"*, which
|
||||
// was false by then — that app's tar had gone 182,272 B → 2,147,666,432 B under a stale manifest.
|
||||
//
|
||||
// WHAT THIS IS. ONE verdict per app per run, taken before that app's FIRST write of the run, covering
|
||||
// all three legs. The three write under one per-app root (`appbackup.RecoveryUnitPath`), which is
|
||||
// exactly why one verdict can honestly cover them — and why the message may now claim what it claims.
|
||||
//
|
||||
// WHY IT IS DECIDED LAZILY AND NOT ONCE AT THE START OF THE RUN. Space changes during a run: app A's
|
||||
// 2 GB dump can put app B under the reserve. A verdict taken at run start would wave B through on a
|
||||
// reading that was true before the disk filled — the same class of mistake as the one being fixed,
|
||||
// moved one level up.
|
||||
//
|
||||
// WHY IT IS REMEMBERED AND NOT RE-DECIDED PER LEG. Re-deciding between an app's own legs reintroduces
|
||||
// the split this closes: the DB leg admitted, the volume leg admitted, the capture refused — with the
|
||||
// bulk already written. Decide once, remember, reuse; reset per run, because a set carried between
|
||||
// runs is a wrong answer with a confident face.
|
||||
//
|
||||
// IT REFUSES; IT NEVER DELETES. Unchanged from B2 and load-bearing: nothing on this filesystem is
|
||||
// generational (one unit per app at one fixed path, refreshed in place), so "prune the oldest" could
|
||||
// only mean destroying a DIFFERENT app's only local copy. `pruneStalePrimaryDirs` removes ORPHANED
|
||||
// dirs an app left on a drive it moved off — it has no notion of age or of the current app — and must
|
||||
// never be repurposed for headroom.
|
||||
|
||||
// floorReason records WHICH term bound, so the operator can tell "the disk is full" from "this app's
|
||||
// backup is too big for what is left". An alert that says only "refused" sends them to read code.
|
||||
type floorReason int
|
||||
|
||||
const (
|
||||
floorAdmit floorReason = iota // admitted — no term binds
|
||||
floorHeadroom // the filesystem is ALREADY at/below the reserve
|
||||
floorSize // there is room now, but this app's own write would cross the reserve
|
||||
)
|
||||
|
||||
func (r floorReason) String() string {
|
||||
switch r {
|
||||
case floorHeadroom:
|
||||
return "headroom"
|
||||
case floorSize:
|
||||
return "size"
|
||||
default:
|
||||
return "admitted"
|
||||
}
|
||||
}
|
||||
|
||||
// admissionVerdict is one app's decision for one run. It carries everything the alert needs, so the
|
||||
// alert is rendered once from the same value every leg consults.
|
||||
type admissionVerdict struct {
|
||||
admitted bool
|
||||
reason floorReason
|
||||
usage *UnitSpace
|
||||
estGiB float64 // the estimated write in GiB — the arithmetic unit, matching the reserve's terms
|
||||
estBytes int64 // the same estimate in bytes — the RENDERING unit; see floorRefusal
|
||||
hasEst bool // whether an estimate was available at all (§8.2: distinct from "estimated 0")
|
||||
err error // the refusal, nil when admitted
|
||||
}
|
||||
|
||||
// admissionSet is the per-RUN memo. Deliberately not a field with a lifetime of its own: it is
|
||||
// created by beginAdmissionRun and cleared by the returned func, so an absent set means "no run is in
|
||||
// flight" rather than "a stale answer from last night".
|
||||
type admissionSet struct {
|
||||
v map[string]admissionVerdict
|
||||
}
|
||||
|
||||
// beginAdmissionRun opens the per-run admission scope and returns the closer. Called once at the top
|
||||
// of runDBDumpsInternal — which is the single orchestrator of all three legs — so the DB dump, the
|
||||
// volume dump and the capture of one app all consult the SAME verdict.
|
||||
//
|
||||
// A second call while a set is live REPLACES it and the returned closer restores the previous one, so
|
||||
// nesting cannot silently drop a caller's scope.
|
||||
func (m *Manager) beginAdmissionRun() func() {
|
||||
m.admissionMu.Lock()
|
||||
prev := m.admission
|
||||
m.admission = &admissionSet{v: map[string]admissionVerdict{}}
|
||||
m.admissionMu.Unlock()
|
||||
return func() {
|
||||
m.admissionMu.Lock()
|
||||
m.admission = prev
|
||||
m.admissionMu.Unlock()
|
||||
}
|
||||
}
|
||||
|
||||
// admitApp is THE gate. It returns true when this app may write, false when the reserve refuses it.
|
||||
//
|
||||
// On the first refusal for an app it logs and fires EXACTLY ONE operator alert; every later leg in
|
||||
// the same run reads the memo and stays silent, so a refused app produces one email and not three.
|
||||
//
|
||||
// With no run scope open (the periodic status refresh calls captureAllRecoveryUnits directly) it
|
||||
// decides fresh. That is not a gap: each app appears once in that sweep, so "once per app" still
|
||||
// holds — there is simply nothing to remember it across.
|
||||
func (m *Manager) admitApp(stackName string) bool {
|
||||
m.admissionMu.Lock()
|
||||
defer m.admissionMu.Unlock()
|
||||
|
||||
if set := m.admission; set != nil {
|
||||
if v, ok := set.v[stackName]; ok {
|
||||
return v.admitted // already decided this run — do NOT re-decide, do NOT re-alert
|
||||
}
|
||||
}
|
||||
|
||||
v := m.decideAdmission(stackName)
|
||||
if set := m.admission; set != nil {
|
||||
set.v[stackName] = v
|
||||
}
|
||||
if v.admitted {
|
||||
return true
|
||||
}
|
||||
|
||||
// The claim below is now literally true, and that is the whole point of R-181: the verdict is
|
||||
// taken before the FIRST of the three writes, so at this moment nothing under
|
||||
// backups/primary/<app> has been touched by this run. TestAdmission_RefusedAppsTreeIsByteIdentical
|
||||
// pins the consequence by checksumming the tree, not by reading this line.
|
||||
m.logger.Printf("[WARN] [backup] App backup REFUSED for %s (%s) — %v; NO database dump, NO volume "+
|
||||
"dump and NO recovery-unit capture was written for it, the previous unit is untouched and "+
|
||||
"NOTHING was deleted", stackName, v.reason, v.err)
|
||||
if m.unitNotify != nil {
|
||||
m.unitNotify(stackName, v.err, v.usage)
|
||||
}
|
||||
// R-182: the digest entry is recorded HERE, where the verdict is taken — once per app per run.
|
||||
// Not at the three call sites that consult the memo: R-181's whole contract is that ONE verdict
|
||||
// covers all three legs, so noting it per leg listed a single refused app three times and
|
||||
// produced counts like "2 of 1 apps failed". The leg name says what actually happened, which is
|
||||
// that nothing was attempted at all.
|
||||
m.noteFailure(stackName, "whole app (refused before any write)", v.err.Error())
|
||||
return false
|
||||
}
|
||||
|
||||
// decideAdmission applies the floor to a fresh reading plus this app's estimated write.
|
||||
func (m *Manager) decideAdmission(stackName string) admissionVerdict {
|
||||
estBytes, hasEst := m.estimatedWriteBytes(stackName)
|
||||
estGiB := float64(estBytes) / (1024 * 1024 * 1024)
|
||||
usage, reason := m.floorVerdict(m.readUnitSpace(stackName), estGiB)
|
||||
v := admissionVerdict{
|
||||
admitted: reason == floorAdmit,
|
||||
reason: reason,
|
||||
usage: usage,
|
||||
estGiB: estGiB,
|
||||
estBytes: estBytes,
|
||||
hasEst: hasEst,
|
||||
}
|
||||
if !v.admitted {
|
||||
v.err = floorRefusal(reason, usage, estBytes, hasEst)
|
||||
}
|
||||
return v
|
||||
}
|
||||
|
||||
// floorRefusal renders the refusal an operator reads. It names the reserve (not an I/O error — this
|
||||
// is a deliberate hold, not broken machinery), says WHICH term bound, and states plainly when the
|
||||
// decision was headroom-only because the app has no previous backup to estimate from (§8.2).
|
||||
//
|
||||
// THE ESTIMATE IS RENDERED IN BYTES-HUMANIZED, NOT GiB, and that is not cosmetic. Fixed to two
|
||||
// decimal GiB, every app under ~10 MB prints `0.00 GiB` — which reads as "no estimate was available"
|
||||
// and is the opposite of what happened. Observed on the live proof run: opengist's real 178 KB
|
||||
// estimate rendered as `estimated 0.00 GiB write`. The arithmetic stays in GiB (the reserve's own
|
||||
// unit); only the rendering changes.
|
||||
func floorRefusal(reason floorReason, usage *UnitSpace, estBytes int64, hasEst bool) error {
|
||||
var b strings.Builder
|
||||
fmt.Fprintf(&b, "%%w (reserve: %.0f%%%% used or %.1f GiB free", FloorUsedPercent, FloorFreeGiB)
|
||||
switch {
|
||||
case reason == floorSize:
|
||||
fmt.Fprintf(&b, "; this app's last backup was %s and writing it again would cross the reserve", humanizeBytes(estBytes))
|
||||
case hasEst:
|
||||
fmt.Fprintf(&b, "; the filesystem is already below it, before this app's estimated %s write", humanizeBytes(estBytes))
|
||||
default:
|
||||
b.WriteString("; this app has no previous backup on disk, so only current headroom was considered")
|
||||
}
|
||||
b.WriteString(") — %s")
|
||||
return fmt.Errorf(b.String(), ErrCaptureFloor, usage)
|
||||
}
|
||||
|
||||
// estimatedWriteBytes estimates what this app's three legs are about to write, from what the PREVIOUS
|
||||
// run left in its unit: the `.sql` dumps and the `.tar` volume archives already on disk for this app.
|
||||
//
|
||||
// WHY THIS ESTIMATOR. It is free — two ReadDirs of a directory the caller is about to write into — and
|
||||
// the next write is usually close to the last one. The alternative, a container-based `du` of every
|
||||
// named volume, was measured on the demo box before being rejected; the figure is in REPORT.md §6.
|
||||
//
|
||||
// NO HISTORY → (0, false), and the caller falls back to headroom-only. Refusing an app because it has
|
||||
// never been backed up would make the first backup the one that can never happen (Scenario E).
|
||||
//
|
||||
// It reads the app's CURRENT unit root, so an app that moved drives estimates from its new (probably
|
||||
// empty) location and is treated as history-less — conservative in the admitting direction, which is
|
||||
// the right way round for an estimate that only ever tightens a threshold.
|
||||
func (m *Manager) estimatedWriteBytes(stackName string) (int64, bool) {
|
||||
drivePath := m.GetAppDrivePath(stackName)
|
||||
if drivePath == "" {
|
||||
return 0, false
|
||||
}
|
||||
nsRoot := m.namespaceRoot(drivePath)
|
||||
var total int64
|
||||
var found bool
|
||||
for _, d := range []struct {
|
||||
dir string
|
||||
ext string
|
||||
}{
|
||||
{AppDBDumpPath(nsRoot, stackName), ".sql"},
|
||||
{AppVolumeDumpPath(nsRoot, stackName), ".tar"},
|
||||
} {
|
||||
n, ok := sumFileSizes(d.dir, d.ext)
|
||||
total += n
|
||||
found = found || ok
|
||||
}
|
||||
if !found {
|
||||
return 0, false
|
||||
}
|
||||
return total, true
|
||||
}
|
||||
|
||||
// sumFileSizes totals the sizes of files with the given suffix in dir. The bool reports whether ANY
|
||||
// such file was seen — distinct from a zero total, because a 0-byte dump is history (a real, if
|
||||
// alarming, previous result) while an absent directory is not.
|
||||
//
|
||||
// A stat error on one entry is skipped rather than aborting the sum: an estimate built from the
|
||||
// readable files is worth more than no estimate, and the entry that could not be read is logged
|
||||
// nowhere because this is a hint, not a measurement — it can only tighten a threshold, never relax
|
||||
// one below what the headroom term already enforces.
|
||||
func sumFileSizes(dir, suffix string) (int64, bool) {
|
||||
entries, err := os.ReadDir(dir)
|
||||
if err != nil {
|
||||
return 0, false
|
||||
}
|
||||
var total int64
|
||||
var found bool
|
||||
for _, e := range entries {
|
||||
if e.IsDir() || !strings.HasSuffix(e.Name(), suffix) {
|
||||
continue
|
||||
}
|
||||
fi, err := os.Stat(filepath.Join(dir, e.Name()))
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
found = true
|
||||
total += fi.Size()
|
||||
}
|
||||
return total, found
|
||||
}
|
||||
@@ -1,741 +0,0 @@
|
||||
package backup
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"fmt"
|
||||
"go/ast"
|
||||
"go/parser"
|
||||
"go/token"
|
||||
"io"
|
||||
"log"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/settings"
|
||||
)
|
||||
|
||||
// R-181 — the reserve guards the write that fills the disk, and its promise is true.
|
||||
//
|
||||
// WHAT THESE ASSERT, AND WHY IT IS THE TREE AND NOT THE LOG. The defect being closed is precisely a
|
||||
// log line that claimed something the filesystem contradicted: B2 printed *"the previous unit is
|
||||
// untouched"* while the volume leg had already rewritten that unit's tar 182,272 B → 2,147,666,432 B.
|
||||
// So a test that reads the message and believes it would have passed against the broken code. Every
|
||||
// refusal test here checksums the whole `backups/primary` tree before and after and compares.
|
||||
|
||||
// ── Harness ──────────────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
// admissionProvider records the two acts a refused app must never suffer: its recovery info being
|
||||
// read (a capture that was ATTEMPTED) and its stack being stopped (which DumpAppVolumesSafe does as
|
||||
// its first act, before any check of its own).
|
||||
type admissionProvider struct {
|
||||
stacks []string
|
||||
volumes map[string][]string
|
||||
hdd map[string]string // per-app drive path, for the drive-state skip tests
|
||||
dir string
|
||||
infoHits []string
|
||||
stopped []string
|
||||
}
|
||||
|
||||
func (p *admissionProvider) GetStackComposePath(string) (string, bool) { return "", false }
|
||||
func (p *admissionProvider) ListDeployedStacks() []StackSummary {
|
||||
out := make([]StackSummary, 0, len(p.stacks))
|
||||
for _, s := range p.stacks {
|
||||
out = append(out, StackSummary{Name: s})
|
||||
}
|
||||
return out
|
||||
}
|
||||
func (p *admissionProvider) GetStackHDDMounts(string) []string { return nil }
|
||||
func (p *admissionProvider) GetStackHDDPath(n string) string { return p.hdd[n] }
|
||||
func (p *admissionProvider) GetImportRoot() string { return "" }
|
||||
func (p *admissionProvider) GetDockerVolumes(name string) []string {
|
||||
if p.volumes == nil {
|
||||
return []string{name + "_data"} // every app is volume-bearing unless told otherwise
|
||||
}
|
||||
return p.volumes[name]
|
||||
}
|
||||
func (p *admissionProvider) StopStack(name string) error {
|
||||
p.stopped = append(p.stopped, name)
|
||||
return nil
|
||||
}
|
||||
func (p *admissionProvider) StartStack(string) error { return nil }
|
||||
func (p *admissionProvider) RefreshAndIsRunning(string) bool { return true }
|
||||
func (p *admissionProvider) GetStackRecoveryInfo(name string) (RecoveryInfo, bool) {
|
||||
p.infoHits = append(p.infoHits, name)
|
||||
return RecoveryInfo{StackDir: filepath.Join(p.dir, "stacks", name)}, true
|
||||
}
|
||||
func (p *admissionProvider) RecoverStackSecrets(string, []string) map[string]string { return nil }
|
||||
func (p *admissionProvider) RecreateStackDefinitionFromUnit(string, string, map[string]string) error {
|
||||
return nil
|
||||
}
|
||||
func (p *admissionProvider) StartStackServices(string, []string) error { return nil }
|
||||
func (p *admissionProvider) GetStackClassifiedBinds(string) ([]ClassifiedBind, bool) {
|
||||
return nil, false
|
||||
}
|
||||
|
||||
type admissionHarness struct {
|
||||
m *Manager
|
||||
prov *admissionProvider
|
||||
events []unitEvent
|
||||
usage map[string]*UnitSpace
|
||||
dir string
|
||||
logs *bytes.Buffer
|
||||
volDumped []string
|
||||
}
|
||||
|
||||
func newAdmissionHarness(t *testing.T, stacks ...string) *admissionHarness {
|
||||
t.Helper()
|
||||
dir := t.TempDir()
|
||||
h := &admissionHarness{
|
||||
prov: &admissionProvider{stacks: stacks, dir: dir, hdd: map[string]string{}},
|
||||
usage: map[string]*UnitSpace{},
|
||||
dir: dir,
|
||||
logs: &bytes.Buffer{},
|
||||
}
|
||||
h.m = &Manager{
|
||||
logger: log.New(h.logs, "", 0),
|
||||
systemDataPath: dir,
|
||||
stackProvider: h.prov,
|
||||
unitSpaceFn: func(name string) *UnitSpace { return h.usage[name] },
|
||||
}
|
||||
// The volume-dump seam records the leg that writes the BULK — the one B2 never gated. A refused
|
||||
// app must not reach it.
|
||||
h.m.dumpVolumesSafe = func(name string) error {
|
||||
h.volDumped = append(h.volDumped, name)
|
||||
// Write what the real leg writes, so an ungated call is visible in the tree checksum too.
|
||||
dumpDir := AppVolumeDumpPath(h.nsRoot(), name)
|
||||
if err := os.MkdirAll(dumpDir, 0o755); err != nil {
|
||||
return err
|
||||
}
|
||||
return os.WriteFile(filepath.Join(dumpDir, name+"_data.tar"), []byte("FRESH TAR FROM THIS RUN"), 0o644)
|
||||
}
|
||||
h.m.SetUnitNotify(func(name string, err error, u *UnitSpace) {
|
||||
h.events = append(h.events, unitEvent{app: name, err: err.Error(), usage: u})
|
||||
})
|
||||
return h
|
||||
}
|
||||
|
||||
// markDisconnected / markDecommissioned put a real settings row behind the drive-state skips, so
|
||||
// Scenario F exercises the production guards rather than a stub of them.
|
||||
func (h *admissionHarness) markDisconnected(app string) {
|
||||
h.driveState(app, true, false)
|
||||
}
|
||||
|
||||
func (h *admissionHarness) markDecommissioned(app string) {
|
||||
h.driveState(app, false, true)
|
||||
}
|
||||
|
||||
func (h *admissionHarness) driveState(app string, disconnected, decommissioned bool) {
|
||||
if h.m.settings == nil {
|
||||
sett, err := settings.Load(filepath.Join(h.dir, "settings.json"), log.New(io.Discard, "", 0))
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
h.m.settings = sett
|
||||
}
|
||||
// Each such app gets its OWN drive path, or marking one would skip them all.
|
||||
p := filepath.Join(h.dir, "drives", app)
|
||||
if err := os.MkdirAll(p, 0o755); err != nil {
|
||||
panic(err)
|
||||
}
|
||||
h.prov.hdd[app] = p
|
||||
if err := h.m.settings.AddStoragePath(settings.StoragePath{Path: p, Label: app}); err != nil {
|
||||
panic(err)
|
||||
}
|
||||
if disconnected {
|
||||
if err := h.m.settings.SetDisconnected(p, true, nil); err != nil {
|
||||
panic(err)
|
||||
}
|
||||
}
|
||||
if decommissioned {
|
||||
if err := h.m.settings.SetDecommissioned(p, ""); err != nil {
|
||||
panic(err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (h *admissionHarness) nsRoot() string { return filepath.Join(h.dir, "felhom-data") }
|
||||
|
||||
// setSpace states the filesystem's occupancy as a test INPUT — the whole point of the unitSpaceFn
|
||||
// seam, so no test has to manufacture disk pressure on a real disk.
|
||||
func (h *admissionHarness) setSpace(app string, usedPct, availGB, totalGB float64) {
|
||||
h.usage[app] = &UnitSpace{
|
||||
Path: h.dir, UsedPercent: usedPct, AvailGB: availGB,
|
||||
TotalGB: totalGB, UsedGB: totalGB * usedPct / 100,
|
||||
}
|
||||
}
|
||||
|
||||
// seedUnit writes a previous recovery unit for an app: a manifest, a captured app.yaml, a DB dump and
|
||||
// a volume tar of the given size. The tar is SPARSE (Truncate), so a 2 GiB "previous backup" costs no
|
||||
// disk — the estimator reads st_size, which is what the next write will actually cost.
|
||||
func (h *admissionHarness) seedUnit(t *testing.T, app string, tarBytes int64) {
|
||||
t.Helper()
|
||||
ns := h.nsRoot()
|
||||
for _, d := range []string{
|
||||
RecoveryUnitComposePath(ns, app),
|
||||
AppDBDumpPath(ns, app),
|
||||
AppVolumeDumpPath(ns, app),
|
||||
} {
|
||||
if err := os.MkdirAll(d, 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
write := func(p string, b []byte, mode os.FileMode) {
|
||||
if err := os.WriteFile(p, b, mode); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
write(RecoveryUnitManifestPath(ns, app), []byte(`{"app_name":"`+app+`","created_at":"2026-08-02T00:00:00Z"}`), 0o644)
|
||||
write(filepath.Join(RecoveryUnitComposePath(ns, app), "app.yaml"), []byte("deployed: true\nenv:\n A: previous-good-value\n"), 0o600)
|
||||
write(filepath.Join(AppDBDumpPath(ns, app), app+"-postgres.sql"), []byte("-- previous good dump\n"), 0o644)
|
||||
|
||||
tar := filepath.Join(AppVolumeDumpPath(ns, app), app+"_data.tar")
|
||||
f, err := os.Create(tar)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := f.WriteString("PREVIOUS GOOD TAR"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if tarBytes > 0 {
|
||||
if err := f.Truncate(tarBytes); err != nil { // sparse — st_size is the estimate, blocks are not spent
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if err := f.Close(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
// runOneBackupRun performs exactly the sequence runDBDumpsInternal performs for the two legs that can
|
||||
// be driven without Docker: the admission scope is opened, the volume leg runs, then the capture leg.
|
||||
// The DB leg's wiring is pinned structurally by TestAdmission_IsWiredIntoEveryProductionWriteLeg,
|
||||
// because DiscoverDatabases shells out to `docker` and cannot honestly run here.
|
||||
func (h *admissionHarness) runOneBackupRun() {
|
||||
done := h.m.beginAdmissionRun()
|
||||
defer done()
|
||||
h.m.runVolumeDumps()
|
||||
h.m.captureAllRecoveryUnits()
|
||||
}
|
||||
|
||||
// ── The instrument: a checksum of the whole backup tree ──────────────────────────────────────────
|
||||
|
||||
// treeFingerprint walks every file under backups/primary and returns "relpath mode sha256" lines,
|
||||
// sorted. It is the ONLY honest way to check the refusal's claim: it detects a rewritten payload, an
|
||||
// added file and a deleted one alike, which a log line and an exit code both fail to do.
|
||||
func treeFingerprint(t *testing.T, root string) string {
|
||||
t.Helper()
|
||||
var lines []string
|
||||
err := filepath.Walk(root, func(p string, fi os.FileInfo, err error) error {
|
||||
if err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
}
|
||||
if fi.IsDir() {
|
||||
return nil
|
||||
}
|
||||
f, err := os.Open(p)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer f.Close()
|
||||
sum := sha256.New()
|
||||
if _, err := io.Copy(sum, f); err != nil {
|
||||
return err
|
||||
}
|
||||
rel, _ := filepath.Rel(root, p)
|
||||
lines = append(lines, fmt.Sprintf("%s %o %d %s", rel, fi.Mode().Perm(), fi.Size(), hex.EncodeToString(sum.Sum(nil))))
|
||||
return nil
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("fingerprinting %s: %v", root, err)
|
||||
}
|
||||
sort.Strings(lines)
|
||||
return strings.Join(lines, "\n")
|
||||
}
|
||||
|
||||
// treeStatFingerprint is the instrument for trees holding a multi-GiB fixture, where hashing every
|
||||
// byte costs more than it proves: name + mode + SIZE. It still catches the act being tested — the
|
||||
// volume leg replacing a 2 GiB tar with a freshly written one — because that changes the size, and it
|
||||
// catches an added or deleted file by name. Content-identical-but-different-bytes is the one thing it
|
||||
// cannot see, which is why the small-tree tests use treeFingerprint instead.
|
||||
func treeStatFingerprint(t *testing.T, root string) string {
|
||||
t.Helper()
|
||||
var lines []string
|
||||
_ = filepath.Walk(root, func(p string, fi os.FileInfo, err error) error {
|
||||
if err != nil || fi.IsDir() {
|
||||
return nil
|
||||
}
|
||||
rel, _ := filepath.Rel(root, p)
|
||||
lines = append(lines, fmt.Sprintf("%s %o %d", rel, fi.Mode().Perm(), fi.Size()))
|
||||
return nil
|
||||
})
|
||||
sort.Strings(lines)
|
||||
return strings.Join(lines, "\n")
|
||||
}
|
||||
|
||||
// treeFileList is the weaker instrument used for Scenario F: names only, so the assertion is
|
||||
// specifically about DELETION and cannot be satisfied or broken by a content change.
|
||||
func treeFileList(t *testing.T, root string) []string {
|
||||
t.Helper()
|
||||
var names []string
|
||||
_ = filepath.Walk(root, func(p string, fi os.FileInfo, err error) error {
|
||||
if err != nil || fi.IsDir() {
|
||||
return nil
|
||||
}
|
||||
rel, _ := filepath.Rel(root, p)
|
||||
names = append(names, rel)
|
||||
return nil
|
||||
})
|
||||
sort.Strings(names)
|
||||
return names
|
||||
}
|
||||
|
||||
func (h *admissionHarness) primaryRoot() string {
|
||||
return PrimaryBackupPath(h.nsRoot())
|
||||
}
|
||||
|
||||
// ── Scenario A — one decision, taken before the first byte ───────────────────────────────────────
|
||||
|
||||
func TestAdmission_RefusedAppWritesNothingAndIsNotStopped(t *testing.T) {
|
||||
h := newAdmissionHarness(t, "privatebin", "opengist", "homebox")
|
||||
h.setSpace("privatebin", 40, 60, 100)
|
||||
h.setSpace("opengist", 98, 0.4, 70) // below the reserve on BOTH terms
|
||||
h.setSpace("homebox", 40, 60, 100)
|
||||
h.seedUnit(t, "opengist", 0)
|
||||
|
||||
// Scoped to the REFUSED app's own unit: its two siblings are admitted and legitimately write
|
||||
// theirs, so a whole-tree fingerprint would change for the right reason and prove nothing here.
|
||||
// Scenario F below takes the whole-tree view, where every app is refused.
|
||||
refusedUnit := RecoveryUnitPath(h.nsRoot(), "opengist")
|
||||
before := treeFingerprint(t, refusedUnit)
|
||||
if before == "" {
|
||||
t.Fatal("the fixture seeded no previous unit, so 'byte-identical' would be vacuously true")
|
||||
}
|
||||
h.runOneBackupRun()
|
||||
after := treeFingerprint(t, refusedUnit)
|
||||
|
||||
// 1. NOT ONE of the three legs ran for the refused app.
|
||||
for _, got := range h.volDumped {
|
||||
if got == "opengist" {
|
||||
t.Fatal("the VOLUME leg ran for a refused app — this is the R-181 defect exactly: the leg " +
|
||||
"that writes the bulk was never gated, so the reserve it protects was consumed by the " +
|
||||
"very step it exists to bound")
|
||||
}
|
||||
}
|
||||
for _, got := range h.prov.infoHits {
|
||||
if got == "opengist" {
|
||||
t.Fatal("the CAPTURE leg was attempted for a refused app — the verdict must be taken before " +
|
||||
"any write is prepared, not partway through one")
|
||||
}
|
||||
}
|
||||
|
||||
// 2. The tree is byte-identical. This is the assertion the broken code could not pass.
|
||||
if after != before {
|
||||
t.Fatalf("the backup tree CHANGED across a refusal.\n--- before ---\n%s\n--- after ---\n%s\n"+
|
||||
"A refusal that has already rewritten the payload is the defect, not the fix", before, after)
|
||||
}
|
||||
|
||||
// 3. The app was never stopped. DumpAppVolumesSafe stops the stack as its FIRST act, so a gate
|
||||
// placed inside it would bounce the app it is refusing to back up.
|
||||
for _, got := range h.prov.stopped {
|
||||
if got == "opengist" {
|
||||
t.Fatal("the refused app was STOPPED — the reserve check has drifted behind the stop")
|
||||
}
|
||||
}
|
||||
|
||||
// 4. Exactly ONE alert, for that app, carrying the space figures. Three legs must not mean three
|
||||
// emails about one disk.
|
||||
if len(h.events) != 1 {
|
||||
t.Fatalf("got %d alerts, want exactly 1 (one app refused, three legs): %+v", len(h.events), h.events)
|
||||
}
|
||||
if h.events[0].app != "opengist" {
|
||||
t.Fatalf("alert names %q, want opengist", h.events[0].app)
|
||||
}
|
||||
if h.events[0].usage == nil || h.events[0].usage.AvailGB != 0.4 {
|
||||
t.Fatalf("the alert carries no/incorrect space figures: %+v", h.events[0].usage)
|
||||
}
|
||||
}
|
||||
|
||||
// ── Scenario B — the other apps are unaffected ───────────────────────────────────────────────────
|
||||
|
||||
func TestAdmission_SiblingAppsProceedAndOnlyTheRefusedOneAlerts(t *testing.T) {
|
||||
h := newAdmissionHarness(t, "privatebin", "opengist", "homebox")
|
||||
h.setSpace("privatebin", 40, 60, 100)
|
||||
h.setSpace("opengist", 99, 0.2, 70)
|
||||
h.setSpace("homebox", 40, 60, 100)
|
||||
|
||||
h.runOneBackupRun()
|
||||
|
||||
for _, app := range []string{"privatebin", "homebox"} {
|
||||
if !hasStr(h.volDumped, app) {
|
||||
t.Errorf("%s was not volume-dumped (dumped=%v) — one app's refusal silenced its siblings", app, h.volDumped)
|
||||
}
|
||||
if !hasStr(h.prov.infoHits, app) {
|
||||
t.Errorf("%s was not captured (attempted=%v) — the loop did not continue past the refusal", app, h.prov.infoHits)
|
||||
}
|
||||
if _, err := os.Stat(RecoveryUnitManifestPath(h.nsRoot(), app)); err != nil {
|
||||
t.Errorf("%s has no manifest after the run: %v — an admitted app must be backed up normally", app, err)
|
||||
}
|
||||
}
|
||||
if len(h.events) != 1 {
|
||||
t.Fatalf("got %d alerts, want exactly 1: %+v", len(h.events), h.events)
|
||||
}
|
||||
}
|
||||
|
||||
// ── Scenario C — the promise is true ─────────────────────────────────────────────────────────────
|
||||
|
||||
// Every claim the shipped message makes is checked against the tree it describes. The wording is NOT
|
||||
// weakened to fit the behaviour; the behaviour was moved so the wording became true (§8.3).
|
||||
func TestAdmission_EveryClaimInTheRefusalMessageHoldsAgainstTheTree(t *testing.T) {
|
||||
h := newAdmissionHarness(t, "opengist")
|
||||
h.setSpace("opengist", 98, 0.5, 70)
|
||||
h.seedUnit(t, "opengist", 0)
|
||||
|
||||
beforeFP := treeFingerprint(t, h.primaryRoot())
|
||||
beforeList := treeFileList(t, h.primaryRoot())
|
||||
h.runOneBackupRun()
|
||||
msg := h.logs.String()
|
||||
|
||||
if !strings.Contains(msg, "REFUSED for opengist") {
|
||||
t.Fatalf("no refusal was logged for opengist; log was:\n%s", msg)
|
||||
}
|
||||
|
||||
// Claim 1: "NO database dump, NO volume dump and NO recovery-unit capture was written for it".
|
||||
for _, claim := range []string{"NO database dump", "NO volume dump", "NO recovery-unit capture"} {
|
||||
if !strings.Contains(msg, claim) {
|
||||
t.Fatalf("the message no longer claims %q — if a leg cannot be brought under the verdict the "+
|
||||
"wording must be narrowed deliberately and the gap named, not dropped silently.\n%s", claim, msg)
|
||||
}
|
||||
}
|
||||
if len(h.volDumped) != 0 || len(h.prov.infoHits) != 0 {
|
||||
t.Fatalf("the message claims no leg ran, but volume=%v capture=%v", h.volDumped, h.prov.infoHits)
|
||||
}
|
||||
|
||||
// Claim 2: "the previous unit is untouched" — the claim that was MEASURED FALSE in R-181.
|
||||
if !strings.Contains(msg, "the previous unit is untouched") {
|
||||
t.Fatalf("the message dropped the untouched claim: %s", msg)
|
||||
}
|
||||
if got := treeFingerprint(t, h.primaryRoot()); got != beforeFP {
|
||||
t.Fatalf("the message says the previous unit is untouched; the tree says otherwise.\n"+
|
||||
"--- before ---\n%s\n--- after ---\n%s", beforeFP, got)
|
||||
}
|
||||
|
||||
// Claim 3: "NOTHING was deleted".
|
||||
if !strings.Contains(msg, "NOTHING was deleted") {
|
||||
t.Fatalf("the message dropped the no-deletion claim: %s", msg)
|
||||
}
|
||||
if got := treeFileList(t, h.primaryRoot()); !equalStrs(got, beforeList) {
|
||||
t.Fatalf("files disappeared across a refusal: before=%v after=%v", beforeList, got)
|
||||
}
|
||||
|
||||
// Claim 4: the reason is named, so the operator can tell which term bound.
|
||||
if !strings.Contains(msg, "headroom") {
|
||||
t.Fatalf("the message does not name WHICH term bound — an operator cannot tell 'the disk is "+
|
||||
"full' from 'this app's backup is too big for what is left':\n%s", msg)
|
||||
}
|
||||
}
|
||||
|
||||
// ── Scenario D — size-aware, not just headroom-aware ─────────────────────────────────────────────
|
||||
|
||||
// The live R-181 sequence, reproduced as a unit: the filesystem is ABOVE the reserve on both terms
|
||||
// when the run reaches the app, and the app's own write is what crosses it. Under B2 this app was
|
||||
// admitted at 96% and then allowed to write 2 GB.
|
||||
func TestAdmission_SizeTermRefusesAnAppWhoseOwnWriteWouldCrossTheReserve(t *testing.T) {
|
||||
h := newAdmissionHarness(t, "opengist")
|
||||
// 96% used of 70 GiB, 3.0 GiB free — BOTH reserve terms deliberately still clear (97% / 1.0 GiB),
|
||||
// exactly as on demo-hp at 06:40:03, so a headroom-only rule starts the run.
|
||||
h.setSpace("opengist", 96, 3.0, 70)
|
||||
if _, r := h.m.floorVerdict(h.usage["opengist"], 0); r != floorAdmit {
|
||||
t.Fatalf("fixture is wrong: the headroom term already refuses (%v), so this test would pass "+
|
||||
"without a size term and prove nothing", r)
|
||||
}
|
||||
h.seedUnit(t, "opengist", 2<<30) // its last backup was 2 GiB — the figure measured live
|
||||
|
||||
before := treeStatFingerprint(t, h.primaryRoot())
|
||||
h.runOneBackupRun()
|
||||
|
||||
if len(h.events) != 1 {
|
||||
t.Fatalf("got %d alerts, want 1 — the app was admitted at 96%% and would have been allowed to "+
|
||||
"write 2 GiB, which is the R-181 sequence: %+v", len(h.events), h.events)
|
||||
}
|
||||
if !strings.Contains(h.logs.String(), "(size)") {
|
||||
t.Fatalf("the refusal was not attributed to the SIZE term:\n%s", h.logs.String())
|
||||
}
|
||||
if !strings.Contains(h.events[0].err, "last backup was 2.0 GB") {
|
||||
t.Fatalf("the alert does not carry the estimate that produced the refusal: %q", h.events[0].err)
|
||||
}
|
||||
if len(h.volDumped) != 0 {
|
||||
t.Fatalf("the volume leg ran anyway: %v", h.volDumped)
|
||||
}
|
||||
if got := treeStatFingerprint(t, h.primaryRoot()); got != before {
|
||||
t.Fatalf("the tree changed despite the size-term refusal.\nbefore=%s\nafter =%s", before, got)
|
||||
}
|
||||
}
|
||||
|
||||
// ── Scenario E — a first-ever backup is not blocked by having no history ─────────────────────────
|
||||
|
||||
func TestAdmission_FirstEverBackupIsAdmitted(t *testing.T) {
|
||||
h := newAdmissionHarness(t, "brandnew")
|
||||
h.setSpace("brandnew", 40, 600, 1000) // ample room, and NO previous unit on disk
|
||||
|
||||
if est, ok := h.m.estimatedWriteBytes("brandnew"); ok || est != 0 {
|
||||
t.Fatalf("estimatedWriteBytes = (%v, %v) for an app with no history, want (0, false)", est, ok)
|
||||
}
|
||||
h.runOneBackupRun()
|
||||
|
||||
if len(h.events) != 0 {
|
||||
t.Fatalf("a brand-new app was refused: %+v — refusing every app that has no size to estimate "+
|
||||
"from would make the FIRST backup the one that can never happen", h.events)
|
||||
}
|
||||
if !hasStr(h.volDumped, "brandnew") || !hasStr(h.prov.infoHits, "brandnew") {
|
||||
t.Fatalf("the app was not backed up (volume=%v capture=%v)", h.volDumped, h.prov.infoHits)
|
||||
}
|
||||
}
|
||||
|
||||
// ── Scenario F — the reserve still never deletes ─────────────────────────────────────────────────
|
||||
|
||||
func TestAdmission_NothingUnderBackupsIsEverRemoved(t *testing.T) {
|
||||
h := newAdmissionHarness(t, "privatebin", "opengist", "homebox")
|
||||
for _, app := range []string{"privatebin", "opengist", "homebox"} {
|
||||
h.setSpace(app, 99, 0.1, 70) // every app refused — maximum pressure to "make room"
|
||||
h.seedUnit(t, app, 0)
|
||||
}
|
||||
|
||||
before := treeFileList(t, h.primaryRoot())
|
||||
h.runOneBackupRun()
|
||||
after := treeFileList(t, h.primaryRoot())
|
||||
|
||||
if !equalStrs(before, after) {
|
||||
t.Fatalf("the file list changed under the reserve.\nbefore=%v\nafter =%v\n"+
|
||||
"Nothing here is generational — a unit is ONE fixed path per app — so 'prune the oldest' "+
|
||||
"could only mean destroying a DIFFERENT app's only local recovery unit", before, after)
|
||||
}
|
||||
if len(before) == 0 {
|
||||
t.Fatal("the fixture seeded no files, so this test would pass against code that deleted everything")
|
||||
}
|
||||
}
|
||||
|
||||
// ── §8.1 — one verdict per app per run, and it resets between runs ───────────────────────────────
|
||||
|
||||
// The verdict must not be re-taken between an app's own legs. Re-deciding is how the split this fixes
|
||||
// came about: DB leg admitted, volume leg admitted, capture refused — with the bulk already written.
|
||||
func TestAdmission_VerdictIsTakenOncePerAppPerRunAndNotRedecidedBetweenLegs(t *testing.T) {
|
||||
h := newAdmissionHarness(t, "opengist")
|
||||
reads := 0
|
||||
h.m.unitSpaceFn = func(string) *UnitSpace {
|
||||
reads++
|
||||
if reads == 1 {
|
||||
return &UnitSpace{Path: h.dir, UsedPercent: 99, AvailGB: 0.1, TotalGB: 70, UsedGB: 69.3}
|
||||
}
|
||||
// The disk "recovers" mid-run. A re-decided verdict would admit the capture leg here — which
|
||||
// is precisely the split R-181 closes, arriving from the other direction.
|
||||
return &UnitSpace{Path: h.dir, UsedPercent: 10, AvailGB: 60, TotalGB: 70, UsedGB: 7}
|
||||
}
|
||||
|
||||
h.runOneBackupRun()
|
||||
|
||||
if reads != 1 {
|
||||
t.Fatalf("the filesystem was read %d times for ONE app in ONE run — the verdict is being "+
|
||||
"re-decided between legs, which reintroduces the split (bulk written, capture refused)", reads)
|
||||
}
|
||||
if len(h.prov.infoHits) != 0 {
|
||||
t.Fatal("the capture leg ran after the app was refused earlier in the same run")
|
||||
}
|
||||
if len(h.events) != 1 {
|
||||
t.Fatalf("got %d alerts, want exactly 1 per app per run: %+v", len(h.events), h.events)
|
||||
}
|
||||
}
|
||||
|
||||
// A set carried between runs is a wrong answer with a confident face: tonight's question answered
|
||||
// with last night's disk.
|
||||
func TestAdmission_TheRememberedSetResetsBetweenRuns(t *testing.T) {
|
||||
h := newAdmissionHarness(t, "opengist")
|
||||
h.setSpace("opengist", 99, 0.1, 70)
|
||||
h.runOneBackupRun()
|
||||
if len(h.events) != 1 {
|
||||
t.Fatalf("run 1: want 1 alert, got %+v", h.events)
|
||||
}
|
||||
|
||||
h.setSpace("opengist", 20, 55, 70) // space freed between runs
|
||||
h.runOneBackupRun()
|
||||
|
||||
if !hasStr(h.volDumped, "opengist") {
|
||||
t.Fatal("the second run still refused the app — the previous run's verdict was carried over, " +
|
||||
"so freeing space could never take effect")
|
||||
}
|
||||
if len(h.events) != 1 {
|
||||
t.Fatalf("the second (admitted) run alerted again: %+v", h.events)
|
||||
}
|
||||
}
|
||||
|
||||
// ── §8.4 — a nil reading neither refuses nor warns, across ALL THREE legs ────────────────────────
|
||||
|
||||
// Unchanged behaviour, re-pinned because the decision now governs three legs instead of one: an
|
||||
// unreadable filesystem must not silently stop an app being backed up at all.
|
||||
func TestAdmission_UnreadableFilesystemAdmitsEveryLegAndDoesNotWarn(t *testing.T) {
|
||||
h := newAdmissionHarness(t, "opengist")
|
||||
// No usage entry → the reader returns nil, which is what system.GetDiskUsage does on error.
|
||||
|
||||
h.runOneBackupRun()
|
||||
|
||||
if len(h.events) != 0 {
|
||||
t.Fatalf("an unreadable filesystem produced %d alert(s): %+v — that is the drive gate's "+
|
||||
"business and has its own alert", len(h.events), h.events)
|
||||
}
|
||||
if !hasStr(h.volDumped, "opengist") {
|
||||
t.Fatal("the VOLUME leg was refused on an unreadable read — a drive that merely blipped would " +
|
||||
"now stop the bulk of the backup, not just the capture")
|
||||
}
|
||||
if !hasStr(h.prov.infoHits, "opengist") {
|
||||
t.Fatal("the CAPTURE leg was refused on an unreadable read")
|
||||
}
|
||||
}
|
||||
|
||||
// ── The estimator, through the production path (no seam) ─────────────────────────────────────────
|
||||
|
||||
func TestEstimatedWriteBytes_SumsTheAppsPreviousDumpsFromRealFiles(t *testing.T) {
|
||||
h := newAdmissionHarness(t, "opengist")
|
||||
h.seedUnit(t, "opengist", 3<<30) // 3 GiB sparse tar + a small .sql
|
||||
|
||||
est, ok := h.m.estimatedWriteBytes("opengist")
|
||||
if !ok {
|
||||
t.Fatal("history on disk was not recognised as history")
|
||||
}
|
||||
if est < 3<<30 || est > (3<<30)+4096 {
|
||||
t.Fatalf("estimate = %d B, want ~%d (the .tar plus the small .sql)", est, int64(3)<<30)
|
||||
}
|
||||
|
||||
// An app whose unit exists but holds no dumps yet is history-LESS, not a zero-byte estimate.
|
||||
other := AppVolumeDumpPath(h.nsRoot(), "empty")
|
||||
if err := os.MkdirAll(other, 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if est, ok := h.m.estimatedWriteBytes("empty"); ok || est != 0 {
|
||||
t.Fatalf("an empty unit reported history (%v, %v) — an absent dump is not a 0-byte one", est, ok)
|
||||
}
|
||||
}
|
||||
|
||||
// ── The seam is WIRED — walked as an AST, not grepped ────────────────────────────────────────────
|
||||
|
||||
// FOUR mechanisms in this project have been built and left disconnected (REUSE.md's seam register).
|
||||
// The behavioural tests above drive the two legs that can run without Docker; the DB leg cannot, so
|
||||
// its gate is pinned HERE, structurally. `strings.Contains` is deliberately not used: a commented-out
|
||||
// call still contains the string, and so does a call inside dead code.
|
||||
func TestAdmission_IsWiredIntoEveryProductionWriteLeg(t *testing.T) {
|
||||
fset := token.NewFileSet()
|
||||
file, err := parser.ParseFile(fset, "backup.go", nil, 0) // comments dropped — only real calls survive
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
calls := map[string][]string{} // enclosing func → called names, in source order
|
||||
var current string
|
||||
ast.Inspect(file, func(n ast.Node) bool {
|
||||
switch v := n.(type) {
|
||||
case *ast.FuncDecl:
|
||||
current = v.Name.Name
|
||||
case *ast.CallExpr:
|
||||
name := ""
|
||||
switch fn := v.Fun.(type) {
|
||||
case *ast.Ident:
|
||||
name = fn.Name
|
||||
case *ast.SelectorExpr:
|
||||
name = fn.Sel.Name
|
||||
}
|
||||
if name != "" && current != "" {
|
||||
calls[current] = append(calls[current], name)
|
||||
}
|
||||
}
|
||||
return true
|
||||
})
|
||||
|
||||
// 1. The run scope is opened by the orchestrator of all three legs.
|
||||
if !hasStr(calls["runDBDumpsInternal"], "beginAdmissionRun") {
|
||||
t.Fatal("runDBDumpsInternal does not open the admission scope — without it every leg decides " +
|
||||
"independently and the per-run memo never exists, which is the pre-R-181 behaviour")
|
||||
}
|
||||
|
||||
// 2. The DB leg consults it BEFORE the dump. Order is the whole point: a gate after the write is
|
||||
// the defect, relocated.
|
||||
assertGateBefore(t, calls["runDBDumpsInternal"], "admitApp", "DumpOne",
|
||||
"the DATABASE leg dumps before consulting the reserve")
|
||||
|
||||
// 3. The volume leg consults it BEFORE the dump seam — which stops the stack as its first act.
|
||||
assertGateBefore(t, calls["runVolumeDumps"], "admitApp", "dump",
|
||||
"the VOLUME leg — the one that writes the bulk, and the one B2 never gated — dumps before "+
|
||||
"consulting the reserve")
|
||||
|
||||
// 4. The capture leg, in its own file.
|
||||
rfset := token.NewFileSet()
|
||||
rfile, err := parser.ParseFile(rfset, "recovery_unit.go", nil, 0)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
capCalls := map[string][]string{}
|
||||
current = ""
|
||||
ast.Inspect(rfile, func(n ast.Node) bool {
|
||||
switch v := n.(type) {
|
||||
case *ast.FuncDecl:
|
||||
current = v.Name.Name
|
||||
case *ast.CallExpr:
|
||||
if sel, ok := v.Fun.(*ast.SelectorExpr); ok && current != "" {
|
||||
capCalls[current] = append(capCalls[current], sel.Sel.Name)
|
||||
}
|
||||
}
|
||||
return true
|
||||
})
|
||||
assertGateBefore(t, capCalls["captureAllRecoveryUnits"], "admitApp", "CaptureRecoveryUnit",
|
||||
"the CAPTURE leg captures before consulting the reserve")
|
||||
}
|
||||
|
||||
// assertGateBefore checks that `gate` appears in the call list before `act`.
|
||||
func assertGateBefore(t *testing.T, calls []string, gate, act, why string) {
|
||||
t.Helper()
|
||||
gi, ai := -1, -1
|
||||
for i, c := range calls {
|
||||
if c == gate && gi < 0 {
|
||||
gi = i
|
||||
}
|
||||
if c == act && ai < 0 {
|
||||
ai = i
|
||||
}
|
||||
}
|
||||
if gi < 0 {
|
||||
t.Fatalf("%s: %q is never called there at all (calls=%v)", why, gate, calls)
|
||||
}
|
||||
if ai < 0 {
|
||||
t.Fatalf("fixture drift: %q is no longer called in that function (calls=%v) — this test can no "+
|
||||
"longer see the act it is ordering the gate against", act, calls)
|
||||
}
|
||||
if gi > ai {
|
||||
t.Fatalf("%s: %q first appears at %d, after %q at %d", why, gate, gi, act, ai)
|
||||
}
|
||||
}
|
||||
|
||||
func hasStr(hay []string, needle string) bool {
|
||||
for _, s := range hay {
|
||||
if s == needle {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func equalStrs(a, b []string) bool {
|
||||
if len(a) != len(b) {
|
||||
return false
|
||||
}
|
||||
for i := range a {
|
||||
if a[i] != b[i] {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
@@ -14,7 +14,6 @@ package backup
|
||||
import (
|
||||
"context"
|
||||
"log"
|
||||
"time"
|
||||
|
||||
"gitea.dooplex.hu/admin/felhom-controller/internal/appbackup"
|
||||
)
|
||||
@@ -26,8 +25,6 @@ type StackSummary = appbackup.StackSummary
|
||||
type AppBackupInfo = appbackup.AppBackupInfo
|
||||
type AppDataPath = appbackup.AppDataPath
|
||||
type AppDockerVolume = appbackup.AppDockerVolume
|
||||
type RecoveryInfo = appbackup.RecoveryInfo
|
||||
type ClassifiedBind = appbackup.ClassifiedBind
|
||||
|
||||
// --- type aliases (dbdump) ---
|
||||
|
||||
@@ -44,21 +41,13 @@ const (
|
||||
DBTypeMariaDB = appbackup.DBTypeMariaDB
|
||||
)
|
||||
|
||||
// Backup-classification class constants (Task 3-core) — aliased so the tier engines can switch on
|
||||
// class without importing appbackup directly.
|
||||
const (
|
||||
ClassMandatory = appbackup.ClassMandatory
|
||||
ClassOptional = appbackup.ClassOptional
|
||||
ClassExcluded = appbackup.ClassExcluded
|
||||
)
|
||||
|
||||
// FelhomDataDir is the namespace directory on storage drives for all felhom-managed data.
|
||||
const FelhomDataDir = appbackup.FelhomDataDir
|
||||
|
||||
// --- function forwarders (dbdump) ---
|
||||
|
||||
func DiscoverDatabases(ctx context.Context, logger *log.Logger, debug bool, knownStacks []string) ([]DiscoveredDB, error) {
|
||||
return appbackup.DiscoverDatabases(ctx, logger, debug, knownStacks)
|
||||
func DiscoverDatabases(ctx context.Context, logger *log.Logger, debug bool) ([]DiscoveredDB, error) {
|
||||
return appbackup.DiscoverDatabases(ctx, logger, debug)
|
||||
}
|
||||
|
||||
func DumpAll(ctx context.Context, dbs []DiscoveredDB, dumpDir string, logger *log.Logger, debug bool) []DumpResult {
|
||||
@@ -69,17 +58,12 @@ func DumpOne(ctx context.Context, db DiscoveredDB, dumpDir string, logger *log.L
|
||||
return appbackup.DumpOne(ctx, db, dumpDir, logger, debug)
|
||||
}
|
||||
|
||||
// ImportDump replays a captured .sql dump back into a running DB container (F17 restore path).
|
||||
func ImportDump(ctx context.Context, db DiscoveredDB, dumpPath string, logger *log.Logger, debug bool) error {
|
||||
return appbackup.ImportDump(ctx, db, dumpPath, logger, debug)
|
||||
}
|
||||
|
||||
func ValidateDump(filePath string, dbType DBType) DumpValidation {
|
||||
return appbackup.ValidateDump(filePath, dbType)
|
||||
}
|
||||
|
||||
func ListDumpFiles(dumpDir string, cached func(name string, size int64, mod time.Time) (DumpValidation, bool)) ([]DumpFileInfo, error) {
|
||||
return appbackup.ListDumpFiles(dumpDir, cached)
|
||||
func ListDumpFiles(dumpDir string) ([]DumpFileInfo, error) {
|
||||
return appbackup.ListDumpFiles(dumpDir)
|
||||
}
|
||||
|
||||
// --- function forwarders (appdata) ---
|
||||
@@ -96,16 +80,6 @@ func ResolveDockerVolumeNames(composePath string) []string {
|
||||
return appbackup.ResolveDockerVolumeNames(composePath)
|
||||
}
|
||||
|
||||
func ParseComposeImages(composePath string) []string {
|
||||
return appbackup.ParseComposeImages(composePath)
|
||||
}
|
||||
|
||||
// DBServiceNames forwards to appbackup.DBServiceNames — the compose SERVICE names holding a database,
|
||||
// i.e. the argument list for the DB-only bring-up both restore paths use before a dump replay (R-47).
|
||||
func DBServiceNames(composePath string) ([]string, error) {
|
||||
return appbackup.DBServiceNames(composePath)
|
||||
}
|
||||
|
||||
// humanizeBytes forwards to appbackup.HumanizeBytes; kept unexported so the
|
||||
// many in-package call sites (backup.go, crossdrive.go, restore code) need no edit.
|
||||
func humanizeBytes(b int64) string {
|
||||
@@ -121,11 +95,6 @@ func NamespaceRoot(drivePath string, inGuestDrive bool) string {
|
||||
return appbackup.NamespaceRoot(drivePath, inGuestDrive)
|
||||
}
|
||||
|
||||
// NamespaceRootFor re-exports the ONE drive-kind-aware resolver (R-203).
|
||||
func NamespaceRootFor(drivePath, systemDataPath string) string {
|
||||
return appbackup.NamespaceRootFor(drivePath, systemDataPath)
|
||||
}
|
||||
|
||||
func PrimaryBackupPath(nsRoot string) string {
|
||||
return appbackup.PrimaryBackupPath(nsRoot)
|
||||
}
|
||||
@@ -138,26 +107,6 @@ func AppVolumeDumpPath(nsRoot, stackName string) string {
|
||||
return appbackup.AppVolumeDumpPath(nsRoot, stackName)
|
||||
}
|
||||
|
||||
func RecoveryUnitPath(nsRoot, stackName string) string {
|
||||
return appbackup.RecoveryUnitPath(nsRoot, stackName)
|
||||
}
|
||||
|
||||
func RecoveryUnitComposePath(nsRoot, stackName string) string {
|
||||
return appbackup.RecoveryUnitComposePath(nsRoot, stackName)
|
||||
}
|
||||
|
||||
func RecoveryUnitManifestPath(nsRoot, stackName string) string {
|
||||
return appbackup.RecoveryUnitManifestPath(nsRoot, stackName)
|
||||
}
|
||||
|
||||
func AppDataDir(nsRoot, stackName string) string {
|
||||
return appbackup.AppDataDir(nsRoot, stackName)
|
||||
}
|
||||
|
||||
func AppDataDirNames(hddPath, stackName string, hddMounts []string) []string {
|
||||
return appbackup.AppDataDirNames(hddPath, stackName, hddMounts)
|
||||
}
|
||||
|
||||
func AppDataBindsPresent(hddPath string, hddMounts []string) bool {
|
||||
return appbackup.AppDataBindsPresent(hddPath, hddMounts)
|
||||
}
|
||||
|
||||
@@ -1,229 +0,0 @@
|
||||
package backup
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"log"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// R-174 — the app-stop guard's crash recovery must not start an app onto a MISSING drive.
|
||||
//
|
||||
// The defect these pin, found by review on 2026-08-02 in code shipped 2026-08-01 (v0.189.0):
|
||||
// `appStopGuard.SetStarter(stackMgr)` handed Recover the raw stack manager, whose `StartStack` has
|
||||
// no drive gate. Recover runs at STARTUP — exactly when an external drive may not have come back —
|
||||
// so a backup that stopped an app, followed by a power cut and a drive that did not remount, ended
|
||||
// with the app started onto a missing drive. R-171 one path over.
|
||||
//
|
||||
// THE SEAM UNDER TEST IS THE STARTER, not the gate: `internal/backup` must not import `stacks` or
|
||||
// `settings`, so the production gate lives in `cmd/controller`. What is pinned here is the contract
|
||||
// between them — that a starter returning ErrStartRefused produces a REFUSAL (marker kept, no alarm)
|
||||
// and not a FAILURE. The production wiring itself is pinned by TestMainWiresGatedAppStopStarter.
|
||||
|
||||
// gatingStarter is a starter whose gate refuses a named set of apps, in the shape the production
|
||||
// `gatedAppStopStarter` uses: refuse BEFORE calling through, and wrap ErrStartRefused with a reason.
|
||||
type gatingStarter struct {
|
||||
inner *fakeStarter
|
||||
refuse map[string]string // app → reason
|
||||
refused []string
|
||||
}
|
||||
|
||||
func (s *gatingStarter) StartStack(name string) error {
|
||||
if why, ok := s.refuse[name]; ok {
|
||||
s.refused = append(s.refused, name)
|
||||
return fmt.Errorf("%w: %s", ErrStartRefused, why)
|
||||
}
|
||||
return s.inner.StartStack(name)
|
||||
}
|
||||
|
||||
func newGatedGuard(t *testing.T, dir string, refuse map[string]string) (*AppStopGuard, *gatingStarter) {
|
||||
t.Helper()
|
||||
s := &gatingStarter{inner: &fakeStarter{}, refuse: refuse}
|
||||
g := NewAppStopGuard(filepath.Join(dir, "appstop-state.json"), log.New(io.Discard, "", 0))
|
||||
g.SetStarter(s)
|
||||
return g, s
|
||||
}
|
||||
|
||||
// --- Scenario A — the guard does not start an app onto a missing drive ---------------------------
|
||||
|
||||
func TestRecover_DriveAbsent_RefusesTheStartAndKEEPSTheMarker(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
|
||||
// process 1: a volume dump stops immich, then the box loses power. No End(), no defer.
|
||||
g1, _ := newGatedGuard(t, dir, nil)
|
||||
if err := g1.Begin("volume-dump:immich", ReasonVolumeDump, []string{"immich"}); err != nil {
|
||||
t.Fatalf("Begin: %v", err)
|
||||
}
|
||||
// <power cut> — and immich's drive does NOT come back.
|
||||
|
||||
// process 2: a fresh controller starts. The drive is absent.
|
||||
g2, starter := newGatedGuard(t, dir, map[string]string{
|
||||
"immich": "drive /mnt/felhom-drives/hdd_1 is not a live mountpoint",
|
||||
})
|
||||
res := g2.Recover()
|
||||
|
||||
if len(starter.inner.starts) != 0 {
|
||||
t.Fatalf("started %v — the app was started onto a MISSING drive, which is the whole defect",
|
||||
starter.inner.starts)
|
||||
}
|
||||
if res == nil {
|
||||
t.Fatal("Recover returned nil — the refusal is invisible to the caller, so nothing can report it")
|
||||
}
|
||||
if len(res.Refused) != 1 || res.Refused[0] != "immich" {
|
||||
t.Fatalf("refused=%v, want [immich]", res.Refused)
|
||||
}
|
||||
if len(res.Failed) != 0 {
|
||||
t.Fatalf("failed=%v — a deliberate hold was recorded as a FAILURE. That bucket reaches "+
|
||||
"NotifyBackupFailed, which is customer-enabled by default, so the customer would be "+
|
||||
"emailed \"A biztonsági mentés sikertelen!\" about an app nothing is wrong with (R-171's "+
|
||||
"false-alarm shape one path over)", res.Failed)
|
||||
}
|
||||
if !markerExists(t, dir) {
|
||||
t.Fatal("the marker was CLEARED after a refused start — the operation is genuinely " +
|
||||
"unfinished, and clearing it erases the only durable record that immich is owed a restart")
|
||||
}
|
||||
// The refusal must name the app AND the reason, or an operator cannot act on it.
|
||||
if d := res.Detail(); !strings.Contains(d, "held_by_drive") || !strings.Contains(d, "immich") {
|
||||
t.Fatalf("detail %q does not name the held app", d)
|
||||
}
|
||||
if msg := res.Message(); !strings.Contains(msg, "HELD") || !strings.Contains(msg, "drive") {
|
||||
t.Fatalf("operator message %q does not say the app is held by an absent drive", msg)
|
||||
}
|
||||
}
|
||||
|
||||
// A refusal-only recovery MUST NOT alarm. This is the assertion that keeps the fix from being the
|
||||
// bug it fixes: the drive gate doing its job is not a backup failure.
|
||||
func TestRecover_RefusalOnly_IsNotAlarming(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
g1, _ := newGatedGuard(t, dir, nil)
|
||||
if err := g1.Begin("volume-dump:immich", ReasonVolumeDump, []string{"immich"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
g2, _ := newGatedGuard(t, dir, map[string]string{"immich": "drive /mnt/felhom-drives/hdd_1 is not a live mountpoint"})
|
||||
res := g2.Recover()
|
||||
|
||||
if res.Alarming() {
|
||||
t.Fatal("a recovery that only REFUSED starts reports as alarming — main.go would push it " +
|
||||
"through NotifyBackupFailed and email the customer about a working drive gate")
|
||||
}
|
||||
}
|
||||
|
||||
// A genuine failure alongside a refusal still alarms, and the two stay in different buckets.
|
||||
func TestRecover_FailureAlongsideRefusal_StillAlarmsAndKeepsThemApart(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
g1, _ := newGatedGuard(t, dir, nil)
|
||||
if err := g1.Begin("volume-dump:batch", ReasonVolumeDump, []string{"immich", "nextcloud", "homebox"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
g2, starter := newGatedGuard(t, dir, map[string]string{"immich": "drive /mnt/felhom-drives/hdd_1 is not a live mountpoint"})
|
||||
starter.inner.failWith = map[string]error{"nextcloud": errors.New("compose up: no such image")}
|
||||
res := g2.Recover()
|
||||
|
||||
if len(res.Refused) != 1 || res.Refused[0] != "immich" {
|
||||
t.Fatalf("refused=%v, want [immich]", res.Refused)
|
||||
}
|
||||
if len(res.Failed) != 1 || res.Failed[0] != "nextcloud" {
|
||||
t.Fatalf("failed=%v, want [nextcloud]", res.Failed)
|
||||
}
|
||||
if len(res.Restarted) != 1 || res.Restarted[0] != "homebox" {
|
||||
t.Fatalf("restarted=%v, want [homebox] — neither a refusal nor a failure may abort the loop",
|
||||
res.Restarted)
|
||||
}
|
||||
if !res.Alarming() {
|
||||
t.Fatal("a genuine restart FAILURE alongside a refusal no longer alarms — the refusal " +
|
||||
"swallowed a real fault")
|
||||
}
|
||||
if !markerExists(t, dir) {
|
||||
t.Fatal("the marker was cleared with work still owed")
|
||||
}
|
||||
// The message must not let the held app inflate the failure count.
|
||||
msg := res.Message()
|
||||
if !strings.Contains(msg, "1 of 2 app(s) could NOT be restarted") {
|
||||
t.Fatalf("operator message %q miscounts: the held app must not be counted as a failure", msg)
|
||||
}
|
||||
if !strings.Contains(msg, "not counted as failures") {
|
||||
t.Fatalf("operator message %q does not disclose the held app at all", msg)
|
||||
}
|
||||
}
|
||||
|
||||
// --- Scenario B — a live drive still recovers normally, byte-identical to before -----------------
|
||||
|
||||
func TestRecover_DriveLive_RecoversExactlyAsBefore(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
g1, _ := newGatedGuard(t, dir, nil)
|
||||
if err := g1.Begin("volume-dump:immich", ReasonVolumeDump, []string{"immich", "nextcloud"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// Nothing refused — the gate says yes for both.
|
||||
g2, starter := newGatedGuard(t, dir, nil)
|
||||
res := g2.Recover()
|
||||
|
||||
if len(starter.inner.starts) != 2 {
|
||||
t.Fatalf("started %v, want both apps — the new gate refused a LEGITIMATE recovery",
|
||||
starter.inner.starts)
|
||||
}
|
||||
if len(res.Refused) != 0 || len(res.Failed) != 0 {
|
||||
t.Fatalf("refused=%v failed=%v, want neither on a live drive", res.Refused, res.Failed)
|
||||
}
|
||||
if len(res.Restarted) != 2 {
|
||||
t.Fatalf("restarted=%v, want both", res.Restarted)
|
||||
}
|
||||
if markerExists(t, dir) {
|
||||
t.Fatal("the marker survived a fully successful recovery — the next boot would restart the apps again")
|
||||
}
|
||||
if !res.Alarming() {
|
||||
t.Fatal("a successful recovery no longer reports to the operator — the interrupted operation " +
|
||||
"itself is what §2.4 wants reported, and it went silent")
|
||||
}
|
||||
}
|
||||
|
||||
// The next startup, with the drive back, completes the recovery and clears the marker. This is what
|
||||
// makes "keep the marker" a recovery rather than a leak.
|
||||
func TestRecover_HeldAppIsRestartedOnceTheDriveReturns(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
g1, _ := newGatedGuard(t, dir, nil)
|
||||
if err := g1.Begin("volume-dump:immich", ReasonVolumeDump, []string{"immich"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// Boot 1 — drive absent: refused, marker kept.
|
||||
g2, _ := newGatedGuard(t, dir, map[string]string{"immich": "drive /mnt/felhom-drives/hdd_1 is not a live mountpoint"})
|
||||
if res := g2.Recover(); len(res.Refused) != 1 {
|
||||
t.Fatalf("boot 1 refused=%v, want [immich]", res.Refused)
|
||||
}
|
||||
if !markerExists(t, dir) {
|
||||
t.Fatal("boot 1 cleared the marker — boot 2 has nothing to act on and immich stays down forever")
|
||||
}
|
||||
|
||||
// Boot 2 — the drive is back.
|
||||
g3, starter := newGatedGuard(t, dir, nil)
|
||||
res := g3.Recover()
|
||||
if len(starter.inner.starts) != 1 || starter.inner.starts[0] != "immich" {
|
||||
t.Fatalf("boot 2 started %v, want [immich] — the held app was never picked up again",
|
||||
starter.inner.starts)
|
||||
}
|
||||
if len(res.Restarted) != 1 {
|
||||
t.Fatalf("boot 2 restarted=%v, want [immich]", res.Restarted)
|
||||
}
|
||||
if markerExists(t, dir) {
|
||||
t.Fatal("boot 2 kept the marker after a fully successful recovery")
|
||||
}
|
||||
}
|
||||
|
||||
// ErrStartRefused must be matched with errors.Is, i.e. it survives wrapping. A starter that returns
|
||||
// a bare string reason would land in Failed and alarm — the exact collapse this type prevents.
|
||||
func TestErrStartRefused_SurvivesWrapping(t *testing.T) {
|
||||
err := fmt.Errorf("%w: drive /mnt/felhom-drives/hdd_1 is not a live mountpoint", ErrStartRefused)
|
||||
if !errors.Is(err, ErrStartRefused) {
|
||||
t.Fatal("a wrapped ErrStartRefused is no longer matched by errors.Is — every refusal would " +
|
||||
"be recorded as a restart failure and alarm the customer")
|
||||
}
|
||||
if errors.Is(errors.New("compose up: no such image"), ErrStartRefused) {
|
||||
t.Fatal("an ordinary restart failure matches ErrStartRefused — real faults would go silent")
|
||||
}
|
||||
}
|
||||
@@ -1,365 +0,0 @@
|
||||
package backup
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"log"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"time"
|
||||
)
|
||||
|
||||
// ── The app-stop marker (R-166 part 2, decision D-b "in-flight operations") ───────────────────────
|
||||
//
|
||||
// Several operations stop a customer's app, do something to its data, and start it again. Between
|
||||
// the stop and the start, NOTHING ON DISK RECORDED THAT AN APP WAS OWED A RESTART. A controller that
|
||||
// died in that window left the app down with no explanation anywhere — and because a stopped app has
|
||||
// zero containers, the boot reconciler read it as a deliberate customer stop and deliberately left
|
||||
// it alone. Silently, indefinitely.
|
||||
//
|
||||
// A `defer` is NOT the fix and must never be described as one. Campaign 8 fault 10 established this
|
||||
// on live hardware: a SIGKILL runs no deferred function, and what brought the quiesce loop's stacks
|
||||
// back was its persisted marker read by Recover() one second after restart. The defer covers the
|
||||
// graceful exits; the marker covers the hard crash and the power cut. This file is that marker for
|
||||
// the app-data path, modelled directly on internal/quiesce's.
|
||||
//
|
||||
// WHY ITS OWN FILE, not quiesce's: one file, one writer. Quiesce's marker records a whole-guest
|
||||
// backup window and is written by the quiesce loop; this one records an app-data operation and is
|
||||
// written by the backup manager and the exporter. Sharing the file would give it two writers with
|
||||
// two lifetimes, and one clearing the other's record is a stranded app by a different route.
|
||||
//
|
||||
// SAFETY (D-b's binding rule): losing this file must never be worse than not having it. A lost or
|
||||
// corrupt marker means the app is not auto-restarted by THIS mechanism — which is precisely the
|
||||
// pre-v0.189.0 position, not a new hazard. It never deletes, restores, or touches a backup artifact.
|
||||
|
||||
// AppStopReason names WHY an app was stopped, so the recovery log tells an operator which operation
|
||||
// was interrupted rather than merely that something was.
|
||||
type AppStopReason string
|
||||
|
||||
const (
|
||||
// ReasonVolumeDump — DumpAppVolumesSafe: stop, tar the volumes consistently, start.
|
||||
ReasonVolumeDump AppStopReason = "volume_dump"
|
||||
// ReasonOffboxReconstitute — a full offsite restore overwriting the app's files.
|
||||
ReasonOffboxReconstitute AppStopReason = "offbox_reconstitute"
|
||||
// ReasonAppExport — a .fab export taken with "stop the app first".
|
||||
ReasonAppExport AppStopReason = "app_export"
|
||||
)
|
||||
|
||||
// humanReason is the operator-facing phrasing for each reason.
|
||||
func (r AppStopReason) humanReason() string {
|
||||
switch r {
|
||||
case ReasonVolumeDump:
|
||||
return "an app-data backup (volume dump)"
|
||||
case ReasonOffboxReconstitute:
|
||||
return "an off-site restore"
|
||||
case ReasonAppExport:
|
||||
return "an app export"
|
||||
default:
|
||||
return string(r)
|
||||
}
|
||||
}
|
||||
|
||||
// AppStopMarker is the persisted "these apps were stopped by an operation that has not reported
|
||||
// finishing — they are owed a restart" note.
|
||||
type AppStopMarker struct {
|
||||
Active bool `json:"active"`
|
||||
OpID string `json:"op_id"`
|
||||
Reason AppStopReason `json:"reason"`
|
||||
Stacks []string `json:"stacks"`
|
||||
StartedAt time.Time `json:"started_at"`
|
||||
}
|
||||
|
||||
// AppStopStarter is the one thing recovery needs: the ability to start a stack. StartStack must be
|
||||
// idempotent (it is — `compose up -d` on a running stack is a no-op).
|
||||
//
|
||||
// R-174: production MUST pass a GATED starter, never the raw stack manager. Recover runs at STARTUP —
|
||||
// exactly when an external drive may not have come back — and `Manager.StartStack` has no drive gate
|
||||
// of its own. See `gatedAppStopStarter` in cmd/controller/main.go.
|
||||
type AppStopStarter interface {
|
||||
StartStack(name string) error
|
||||
}
|
||||
|
||||
// ErrStartRefused is what a gated starter returns when a DELIBERATE HOLDER — today the drive gate —
|
||||
// says an app must not be started. Wrap it (`fmt.Errorf("%w: …", ErrStartRefused)`) so the reason
|
||||
// survives; Recover matches with errors.Is.
|
||||
//
|
||||
// IT IS NOT A FAILURE, AND THE DISTINCTION IS THE WHOLE POINT OF THE TYPE. A refusal means the
|
||||
// holder is doing its job and owns the restart; a failure means the restart was attempted and broke.
|
||||
// Collapsing the two would put a deliberately-held app into `Failed`, which main.go reports through
|
||||
// `NotifyBackupFailed` — a type that is customer-enabled by default (`settings.DefaultEnabledEvents`)
|
||||
// and carries the Hungarian "A biztonsági mentés sikertelen!". That is R-171's defect one path over:
|
||||
// a false alarm about an app the drive gate is deliberately holding. Both buckets keep the marker;
|
||||
// only `Failed` alarms.
|
||||
var ErrStartRefused = errors.New("start refused by a deliberate holder")
|
||||
|
||||
// AppStopGuard owns one marker file. Construct with NewAppStopGuard; the zero value is inert (every
|
||||
// method is a no-op on a nil guard), so a caller that was never wired degrades to pre-v0.189.0
|
||||
// behaviour instead of panicking.
|
||||
type AppStopGuard struct {
|
||||
path string
|
||||
logger *log.Logger
|
||||
now func() time.Time
|
||||
// starter is only needed by Recover; Begin/End work without one.
|
||||
starter AppStopStarter
|
||||
}
|
||||
|
||||
// AppStopRecovery is what Recover found and did. Returned rather than pushed through a notifier
|
||||
// seam, because of a hard ordering constraint: Recover must COMPLETE before the boot reconciler is
|
||||
// launched (§8.4, main.go:236) and the hub notifier is not constructed until main.go:307. A seam
|
||||
// wired after the fact would be a seam that never fires — the "built but never wired" shape this
|
||||
// project has now hit four times. Returning the outcome lets main.go report it the moment the
|
||||
// notifier exists, and makes the reporting decision visible at the call site instead of buried here.
|
||||
type AppStopRecovery struct {
|
||||
Reason AppStopReason
|
||||
OpID string
|
||||
StartedAt time.Time
|
||||
Restarted []string // apps started again by this recovery
|
||||
Failed []string // apps whose restart was ATTEMPTED and broke (the marker was kept for these)
|
||||
// Refused are apps a deliberate holder said must not start — today, an absent data drive
|
||||
// (R-174). The marker is kept for these too, but they are NOT a fault and MUST NOT alarm: the
|
||||
// holder owns the restart. Separate from Failed for the reason recorded on ErrStartRefused.
|
||||
Refused []string
|
||||
}
|
||||
|
||||
// Alarming reports whether this recovery is worth paging an operator about. A recovery that only
|
||||
// REFUSED starts is the drive gate working as designed, and reporting it through the customer-enabled
|
||||
// `backup_failed` type would be the R-171 false alarm one path over.
|
||||
func (r *AppStopRecovery) Alarming() bool {
|
||||
if r == nil {
|
||||
return false
|
||||
}
|
||||
return len(r.Failed) > 0 || len(r.Restarted) > 0
|
||||
}
|
||||
|
||||
// Message is the operator-facing headline for an interrupted operation.
|
||||
func (r *AppStopRecovery) Message() string {
|
||||
if r == nil {
|
||||
return ""
|
||||
}
|
||||
if len(r.Failed) > 0 {
|
||||
m := fmt.Sprintf("%s was interrupted by a controller restart and %d of %d app(s) could NOT be restarted",
|
||||
r.Reason.humanReason(), len(r.Failed), len(r.Restarted)+len(r.Failed))
|
||||
if len(r.Refused) > 0 {
|
||||
m += fmt.Sprintf(" (a further %d are held by an absent drive and are not counted as failures)", len(r.Refused))
|
||||
}
|
||||
return m
|
||||
}
|
||||
if len(r.Refused) > 0 && len(r.Restarted) == 0 {
|
||||
return fmt.Sprintf("%s was interrupted by a controller restart — %d app(s) are left stopped and HELD: their data drive is not available, so the drive gate restarts them when it returns",
|
||||
r.Reason.humanReason(), len(r.Refused))
|
||||
}
|
||||
m := fmt.Sprintf("%s was interrupted by a controller restart — %d app(s) were left stopped and have been restarted",
|
||||
r.Reason.humanReason(), len(r.Restarted))
|
||||
if len(r.Refused) > 0 {
|
||||
m += fmt.Sprintf("; %d more are held by an absent drive", len(r.Refused))
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
// Detail is the machine-readable tail. App/stack NAMES only — never env values (§9.5).
|
||||
func (r *AppStopRecovery) Detail() string {
|
||||
if r == nil {
|
||||
return ""
|
||||
}
|
||||
d := fmt.Sprintf("op=%s reason=%s started_at=%s restarted=%v", r.OpID, r.Reason,
|
||||
r.StartedAt.UTC().Format(time.RFC3339), r.Restarted)
|
||||
if len(r.Failed) > 0 {
|
||||
d += fmt.Sprintf(" restart_failed=%v", r.Failed)
|
||||
}
|
||||
if len(r.Refused) > 0 {
|
||||
d += fmt.Sprintf(" held_by_drive=%v", r.Refused)
|
||||
}
|
||||
return d
|
||||
}
|
||||
|
||||
// NewAppStopGuard builds a guard over the given marker path.
|
||||
func NewAppStopGuard(path string, logger *log.Logger) *AppStopGuard {
|
||||
if logger == nil {
|
||||
logger = log.Default()
|
||||
}
|
||||
return &AppStopGuard{path: path, logger: logger, now: time.Now}
|
||||
}
|
||||
|
||||
// SetStarter wires the stack-start seam used by Recover. INIT-ONLY — call once at startup, before
|
||||
// Recover. Separate from the constructor because the guard is built alongside the backup manager,
|
||||
// which learns its stack provider later (the same shape as SetStackProvider).
|
||||
func (g *AppStopGuard) SetStarter(s AppStopStarter) {
|
||||
if g == nil {
|
||||
return
|
||||
}
|
||||
g.starter = s
|
||||
}
|
||||
|
||||
// Begin records that `stacks` are about to be stopped by `reason`. It MUST be called BEFORE the
|
||||
// first stop — an error here means the marker could not be written, and the caller must not proceed
|
||||
// to stop an app it cannot promise to restart.
|
||||
func (g *AppStopGuard) Begin(opID string, reason AppStopReason, stackNames []string) error {
|
||||
if g == nil || g.path == "" {
|
||||
return nil // not wired — pre-v0.189.0 behaviour, never a hard failure
|
||||
}
|
||||
if len(stackNames) == 0 {
|
||||
return nil
|
||||
}
|
||||
return g.write(AppStopMarker{
|
||||
Active: true,
|
||||
OpID: opID,
|
||||
Reason: reason,
|
||||
Stacks: append([]string(nil), stackNames...),
|
||||
StartedAt: g.now(),
|
||||
})
|
||||
}
|
||||
|
||||
// End clears the marker after a successful restart. Best-effort by contract: a failure to clear is
|
||||
// logged, never returned as the operation's error — a stale marker costs one idempotent StartStack
|
||||
// on the next boot, which is exactly D-b's "worst acceptable outcome" and far cheaper than failing
|
||||
// a backup that actually succeeded.
|
||||
func (g *AppStopGuard) End() {
|
||||
if g == nil || g.path == "" {
|
||||
return
|
||||
}
|
||||
if err := os.Remove(g.path); err != nil && !os.IsNotExist(err) {
|
||||
g.logger.Printf("[ERROR] [appstop] could not clear the app-stop marker at %s: %v (a stale marker costs one idempotent restart at next startup)", g.path, err)
|
||||
}
|
||||
}
|
||||
|
||||
// Recover restarts any apps left stopped by an operation that died before restarting them, then
|
||||
// clears the marker. Call ONCE at startup, and — critically — call it to COMPLETION before the boot
|
||||
// reconciler is launched, so an app this marker explains is not also reported as an unexplained boot
|
||||
// orphan (§8.4).
|
||||
//
|
||||
// Idempotent: StartStack on a running stack is tolerated, and an absent or inactive marker is a
|
||||
// no-op. On a restart FAILURE the marker is deliberately LEFT IN PLACE — the next startup retries,
|
||||
// and in the meantime the app is down with desired_state:running, so the boot reconciler sees it as
|
||||
// an orphan and the dead-app alarm owns it. Clearing a marker whose restart failed would erase the
|
||||
// only durable record that an app is owed one.
|
||||
//
|
||||
// Returns nil when there was nothing to recover — so "no interrupted operation" and "the recovery
|
||||
// never ran" are distinguishable to the caller, not only in a log (standing rule 3).
|
||||
func (g *AppStopGuard) Recover() *AppStopRecovery {
|
||||
if g == nil || g.path == "" {
|
||||
return nil
|
||||
}
|
||||
m, ok := g.read()
|
||||
if !ok || !m.Active || len(m.Stacks) == 0 {
|
||||
return nil
|
||||
}
|
||||
if g.starter == nil {
|
||||
g.logger.Printf("[ERROR] [appstop] crash recovery: %d app(s) were stopped by %s and are owed a restart, but no stack starter is wired — leaving the marker for the next startup: %v",
|
||||
len(m.Stacks), m.Reason.humanReason(), m.Stacks)
|
||||
return nil
|
||||
}
|
||||
|
||||
g.logger.Printf("[WARN] [appstop] crash recovery: %s (op %q) was interrupted and left %d app(s) stopped — restarting them: %v",
|
||||
m.Reason.humanReason(), m.OpID, len(m.Stacks), m.Stacks)
|
||||
|
||||
res := &AppStopRecovery{Reason: m.Reason, OpID: m.OpID, StartedAt: m.StartedAt}
|
||||
for _, name := range m.Stacks {
|
||||
if err := g.starter.StartStack(name); err != nil {
|
||||
// R-174: a REFUSAL is not a failure. The starter's gate has said this app must not be
|
||||
// started (an absent data drive), so the app is left down deliberately and the holder
|
||||
// owns the restart. Logged at WARN with the reason, and kept out of Failed so it never
|
||||
// reaches the customer-enabled backup_failed alarm — see ErrStartRefused.
|
||||
if errors.Is(err, ErrStartRefused) {
|
||||
g.logger.Printf("[WARN] [appstop] crash recovery: NOT restarting %s — %v; the marker is KEPT and the holder owns the restart", name, err)
|
||||
res.Refused = append(res.Refused, name)
|
||||
continue
|
||||
}
|
||||
g.logger.Printf("[ERROR] [appstop] crash recovery: restart %s failed: %v", name, err)
|
||||
res.Failed = append(res.Failed, name)
|
||||
continue
|
||||
}
|
||||
g.logger.Printf("[INFO] [appstop] crash recovery: restarted %s after the interrupted %s", name, m.Reason.humanReason())
|
||||
res.Restarted = append(res.Restarted, name)
|
||||
}
|
||||
sort.Strings(res.Failed)
|
||||
sort.Strings(res.Refused)
|
||||
sort.Strings(res.Restarted)
|
||||
|
||||
// The marker is kept for BOTH unfinished outcomes, for the same reason and with different
|
||||
// urgency: a failed restart is retried next startup, and a refused one is genuinely unfinished
|
||||
// until its drive returns. Clearing it in either case would erase the only durable record that
|
||||
// an app is owed a restart.
|
||||
if len(res.Failed) > 0 {
|
||||
g.logger.Printf("[ERROR] [appstop] crash recovery: %d app(s) could not be restarted — KEEPING the marker so the next startup retries; the dead-app alarm owns them meanwhile: %v",
|
||||
len(res.Failed), res.Failed)
|
||||
return res
|
||||
}
|
||||
if len(res.Refused) > 0 {
|
||||
g.logger.Printf("[WARN] [appstop] crash recovery: %d app(s) were deliberately NOT restarted (drive absent) — KEEPING the marker; this is the gate working, not a fault: %v",
|
||||
len(res.Refused), res.Refused)
|
||||
return res
|
||||
}
|
||||
g.End()
|
||||
return res
|
||||
}
|
||||
|
||||
// HeldStacks returns the stacks an app-data operation is CURRENTLY holding down, or nil.
|
||||
//
|
||||
// Read-only and nil-safe. It exists for the boot reconciler (§8.2): once R-157 mechanism A widened
|
||||
// the boot window, the sweep could overlap a running volume dump or export and "recover" an app that
|
||||
// is deliberately stopped mid-operation — restarting it under a tar, which is the inconsistency the
|
||||
// stop was taken to avoid. Recover() has already run to completion by then, so a marker seen through
|
||||
// this method belongs to an operation running NOW, not to a crashed one.
|
||||
func (g *AppStopGuard) HeldStacks() []string {
|
||||
if g == nil || g.path == "" {
|
||||
return nil
|
||||
}
|
||||
m, ok := g.read()
|
||||
if !ok || !m.Active {
|
||||
return nil
|
||||
}
|
||||
return append([]string(nil), m.Stacks...)
|
||||
}
|
||||
|
||||
// ---- marker persistence (atomic, 0600) — the quiesce shape ------------------------------------
|
||||
|
||||
func (g *AppStopGuard) write(m AppStopMarker) error {
|
||||
data, err := json.MarshalIndent(m, "", " ")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := os.MkdirAll(filepath.Dir(g.path), 0o755); err != nil {
|
||||
return err
|
||||
}
|
||||
tmp := g.path + ".tmp"
|
||||
f, err := os.OpenFile(tmp, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0o600)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := f.Write(data); err != nil {
|
||||
f.Close()
|
||||
os.Remove(tmp)
|
||||
return err
|
||||
}
|
||||
// fsync before rename: the whole point is surviving a power cut, and a rename that lands ahead
|
||||
// of the bytes it points at is a marker that reads as corrupt at exactly the wrong moment.
|
||||
if err := f.Sync(); err != nil {
|
||||
f.Close()
|
||||
os.Remove(tmp)
|
||||
return err
|
||||
}
|
||||
if err := f.Close(); err != nil {
|
||||
os.Remove(tmp)
|
||||
return err
|
||||
}
|
||||
return os.Rename(tmp, g.path)
|
||||
}
|
||||
|
||||
func (g *AppStopGuard) read() (AppStopMarker, bool) {
|
||||
data, err := os.ReadFile(g.path)
|
||||
if err != nil {
|
||||
return AppStopMarker{}, false
|
||||
}
|
||||
var m AppStopMarker
|
||||
if err := json.Unmarshal(data, &m); err != nil {
|
||||
// Never a silent skip (§9.4): a corrupt marker is LOUD and the bad file is quarantined, so a
|
||||
// genuinely interrupted operation leaves a trace instead of vanishing. Still returns false —
|
||||
// "no usable marker ⇒ no recovery" is the correct contract, and matches quiesce's.
|
||||
g.logger.Printf("[WARN] [appstop] the app-stop marker at %s is corrupt (%v) — quarantining; apps are NOT auto-restarted from it", g.path, err)
|
||||
_ = os.Rename(g.path, fmt.Sprintf("%s.corrupt-%d", g.path, g.now().Unix()))
|
||||
return AppStopMarker{}, false
|
||||
}
|
||||
return m, true
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user