10 KiB
10 KiB
TEST-REPORT — Unattended test campaign (N100 / guest 9201)
Run start: 2026-06-22 (CC, unattended) Operator: Viktor (away, glancing remotely) Scope: RUNBOOK phased test matrix. Only code change permitted = komga catalog HC fix. Legend: PASS / FAIL / SKIPPED(reason). Evidence is raw (command output / volid / screenshot path).
Baseline state (Phase 0 "before")
- Host: felhom-pve (
demo-felhom), agent v0.39.0 (service active). - Controller: image
gitea.dooplex.hu/admin/felhom-controller:0.73.0, containerUp 5 days (healthy). pct list: 9201 running (demo-felhom); 9001 stopped (spike-lxc); 9999 stopped (felhom-selftest-scratch).- Infra containers (9201): felhom-controller, cloudflared, traefik, filebrowser — all Up 5 days, healthy.
- App containers (9201): 24 containers Up 18h. All healthy except
komga= unhealthy (Phase-1 target). - Disk (9201): rootfs
/32G (4% used);/var/lib/docker252G (8%, 19G);/mnt/felhom-drives(pve-root) 94G (31%). - Drives:
felhom-flashsdc1 118G (~159M used)/mnt/felhom-drives/felhom-flash;felhom-usbsdb1 916G (~2M used)/mnt/felhom-drives/felhom-usb. - PBS storage:
felhom-pbsactive, 9.6 TB total, 21% used. Nightly 9201 backups present (latest2026-06-21T18:55:12Z, ~19.5 GB). - pvesm: felhom-flash, felhom-usb, felhom-pbs, local, local-lvm — all active.
Phase 0 — Baseline + safety net
State: 9201 healthy, controller up, tunnel up (cloudflared Up 5 days).
| Check | Result | Evidence |
|---|---|---|
| Baseline capture | PASS | see above |
| GUI screenshots (before) | pending | Chrome bridge check next |
| Fresh PBS whole-CT backup of 9201 | PASS | volid felhom-pbs:backup/ct/9201/2026-06-22T12:45:37Z, 19.62 GB, crash-consistent=true, 161.5s, success=true. uncovered_volumes (by design): /etc/felhom-bootstrap, /mnt/felhom-drives (drives backed up separately) |
| PBS verify of new snapshot | PASS | verify_state: ok for backup_id 9201 backup_time 2026-06-22T12:45:37Z (datastore felhom-spike, verify UPID present) |
DR source for Phase 4: felhom-pbs:backup/ct/9201/2026-06-22T12:45:37Z (verified ok). Gate to destructive phases: OPEN.
GUI render smoke-test (before): dashboard PASS (17 running / 0 stopped / 55 total; mem 5.8/12 GB; CPU 5%; 58°C; v0.73.0; app-backup "Még nem futott"), /monitoring PASS, /stacks PASS.
Access path note (important finding)
- On-LAN, split-horizon DNS (resolver 192.168.0.250) maps
*.demo-felhom.eu→ guest 9201 (192.168.0.141), bypassing Cloudflare (nocf-rayon responses). - The guest's traefik
:443(websecure) returns Go404 page not foundfor ALL hosts (controller AND apps, e.g. romm), while serving a valid LE*.demo-felhom.eucert.:80301-redirects to https → so the browser dead-ends at the 404. → on-LAN HTTPS to the dashboard/apps is non-functional. (The supported external path is Cloudflare; couldn't validate it from the LAN due to DNS hijack.) FINDING — flag for supervised review. - Workaround used for UI testing (reversible): a python TCP forwarder inside guest 9201 (
/tmp/felhom-test-fwd.py, listens0.0.0.0:18080→ controller172.18.0.10:8080). Browser →http://felhom.demo-felhom.eu:18080/(LAN DNS → guest:18080 → controller).CatchAllMiddlewarestrips the port so Host matchesfelhom.<domain>→ dashboard serves over plain HTTP. To be killed at cleanup. - The dashboard currently serves with no auth wall (
/→ 200, no login redirect, no session cookie) — no password configured/enforced. (Bears on Phase 6 "password change" step.)
Scope reality vs runbook
- Runbook assumed "14 enabled apps"; catalog is now 55 total, 38 not-deployed. Deploying all 38 unattended would exhaust the 12 GiB guest. Phase 2 will deploy a representative sample (a few light apps) + cite the already-deployed heavy multi-container case (immich, 4 containers healthy). Documented, not silently truncated.
Phase 0 verdict: PASS — gate OPEN.
Phase 1 — komga HC fix + git-sync + health detection — PASS
State: 9201 healthy, controller up, tunnel up.
Diagnosis (live, guest 9201): docker HC curl -f .../api/v1/actuator/health → exit 22 (HTTP 401, auth-gated) → unhealthy for 18h while serving fine on :25600. Endpoint probe matrix (wget in image): /, /actuator/health, /api/v1/oauth2/providers, /login all 200; only /api/v1/actuator/health → 401. Image ships curl (/usr/bin/curl). Found a second probe: the controller's own .felhom.yml healthcheck.checks[].path (drives the UI badge + route publishing) also pointed at the 401 path.
| Check | Result | Evidence |
|---|---|---|
| Catalog HC fix (docker-compose.yml) | PASS | /api/v1/actuator/health → /actuator/health; commit 3faa5ae pushed to main |
| Catalog probe fix (.felhom.yml) | PASS | controller-side probe path fixed; commit 9b066de pushed to main |
| CHANGELOG + REPORT (app-catalog) | PASS | created CHANGELOG.md (newest-on-top) + overwrote REPORT.md (f51f794) |
| GUI git-sync ("Sablonok frissítése") | PASS | logs: [sync] Updated komga/docker-compose.yml + komga/.felhom.yml; Sablonok frissítve — frissítve: komga. Live stack /opt/docker/stacks/komga/docker-compose.yml updated to /actuator/health (content-hash copy verified) |
| GUI redeploy ("Frissítés") | PASS | POST /api/stacks/komga/update → 200; docker compose up -d --remove-orphans; "Stack komga updated successfully (4.2s)"; container recreated (StartedAt 2026-06-22T13:09:50Z) |
| komga reaches healthy (docker) | PASS | docker ps: komga Up (healthy); HC now http://localhost:25600/actuator/health |
| Controller health-state machine | PASS | controller log Health probes: N ok (of N probed) (no more komga WARN); card badge Nem egészséges → Fut |
| Route publishing | PASS | "⚠ URL nem elérhető – útvonal nincs publikálva" warning cleared; komga.demo-felhom.eu ↗ link shown |
Minor cosmetic note (not fixed): a few catalog logo SVGs 404 (adventurelog-logo.svg, cloudflared-logo.svg, crafty-controller-logo.svg).
Phase 2 — App deployment sweep — PASS (with 1 logged catalog defect)
State: 9201 healthy, controller up, tunnel up. Scope: representative sample (see Phase-0 scope note) — catalog has 38 not-deployed apps; deploying all on a 12 GiB guest is unsafe.
| App | Result | Evidence |
|---|---|---|
| gitea (deploy via GUI) | PASS (after HC fix) | GUI deploy page → memory-gate bar (6043/11904 MB 50%, two-segment current+new) → 3-step progress panel (✅ Konfiguráció mentve → ✅ Konténerek elindultak → ⏳ inicializálás) → POST /api/stacks/gitea/update 200. Initially unhealthy: HC curl -f /api/v1/version → 404 (404 until install-lock) while /api/healthz → 200. Fixed catalog (commit 17e00b7 + CHANGELOG 1a44c14), re-synced, redeployed → docker inspect: healthy, HC now /api/healthz; endpoint http://127.0.0.1:3000/ → HTTP/1.1 200 OK. |
| glance (deploy via GUI) | FAIL — logged, not fixed | GUI deploy → 3-step panel OK, but container Restarting (1) crash-loop. Logs: parsing config: reading main YAML file: open /app/config/glance.yml: no such file or directory. Root cause: catalog template mounts an empty glance_config:/app/config named volume but never seeds the required glance.yml; glance refuses to start without it. Not a trivial HC/start_period fix (needs a seeded default config asset + wiring) → logged per runbook, not fixed unattended. Cleaned up: POST .../stop + POST .../remove → container gone. |
| immich (heavy multi-container, pre-deployed) | PASS (cited) | 4 containers (server/ML/postgres/redis) all Up (healthy) — the heavy-app coverage case. |
Bonus coverage: stop + remove endpoints exercised (glance teardown). Catalog defects found this phase: gitea HC (fixed), glance missing-config (logged for supervised fix).
Phase 3 — Backup + restore — PASS (2 findings)
State: 9201 healthy, controller up, tunnel up.
| Check | Result | Evidence |
|---|---|---|
| Manual DB-dump backup ("Mentés most") | PASS | POST /api/backup/run → 3 DBs dumped: romm-mariadb 38.5 KB/14 tbl, paperless-postgres 360 KB/67 tbl, immich-postgres 43.2 MB/60 tbl; 43.6 MB total in 2.18s; status.success=true, last_run 2026-06-22T13:27:26Z. Files on disk fresh 13:28: …/primary/immich/db-dumps/immich-postgres.sql (45.3 MB), …/primary/romm/db-dumps/romm-mariadb.sql (39 KB) |
| Cross-drive copy (Tier-2) | PASS | POST /api/backup/tier2 → 8 HDD apps copied to /mnt/sys_drive/felhom-data/backups/secondary/<app> (immich 155.3 MB verified 156 MB on disk; romm has appdata + recovery-unit{db-dump,manifest,compose}). crossdrive_completed events pushed |
| Backup page reflects status | PASS | page shows "Adatmentés aktív", 3 DBs, schedule 02:30, last DB backup timestamp; status API success=true |
| Restore flow (per-app) | PASS | POST /backup/restore stack_name=romm → RestoreFromRecoveryUnit: compose down → redeploy from recovery unit (6 env, 3 encrypted) → "Imported DB dump romm-mariadb.sql into romm-db" → containers up → "Restore-from-unit completed: romm" |
| Restore is non-hollow | PASS | post-restore romm DB has real platforms table + 17 tables; all 3 romm containers Up (healthy). Marker-table test: planted restore_test_marker(id=42) pre-restore → survived restore (see finding) but real snapshot tables were reimported; marker dropped afterward (DB back to 17-table baseline) |
Findings:
GET /api/backup/snapshotshas no server handler (api-router returns "endpoint not found"). The restore UI's snapshot dropdown (backups.html:682) can't populate → a pure-UI restore is blocked at snapshot selection. Restore itself works viaPOST /backup/restore(the form's action). Flag for fix.- Restore DB import is additive —
RestoreFromRecoveryUnitreplays the dump but does not drop tables absent from it (my marker table persisted). Real snapshot data is restored; stale/extra tables are not pruned. Minor behavioral note. - Whole-server ("Rendszermentés") backup shows "A host-ügynök jelenleg nem elérhető" (agent local API unreachable from controller) — same root as the monitoring host-metrics gap; tracked in Phase 6.