92 lines
8.2 KiB
Markdown
92 lines
8.2 KiB
Markdown
# TEST-REPORT — Unattended test campaign (N100 / guest 9201)
|
||
|
||
**Run start:** 2026-06-22 (CC, unattended)
|
||
**Operator:** Viktor (away, glancing remotely)
|
||
**Scope:** RUNBOOK phased test matrix. Only code change permitted = komga catalog HC fix.
|
||
**Legend:** PASS / FAIL / SKIPPED(reason). Evidence is raw (command output / volid / screenshot path).
|
||
|
||
---
|
||
|
||
## Baseline state (Phase 0 "before")
|
||
|
||
- **Host:** felhom-pve (`demo-felhom`), agent **v0.39.0** (service active).
|
||
- **Controller:** image `gitea.dooplex.hu/admin/felhom-controller:0.73.0`, container `Up 5 days (healthy)`.
|
||
- **`pct list`:** 9201 running (`demo-felhom`); 9001 stopped (`spike-lxc`); 9999 stopped (`felhom-selftest-scratch`).
|
||
- **Infra containers (9201):** felhom-controller, cloudflared, traefik, filebrowser — all Up 5 days, healthy.
|
||
- **App containers (9201):** 24 containers Up 18h. All healthy **except `komga` = unhealthy** (Phase-1 target).
|
||
- **Disk (9201):** rootfs `/` 32G (4% used); `/var/lib/docker` 252G (8%, 19G); `/mnt/felhom-drives` (pve-root) 94G (31%).
|
||
- **Drives:** `felhom-flash` sdc1 118G (~159M used) `/mnt/felhom-drives/felhom-flash`; `felhom-usb` sdb1 916G (~2M used) `/mnt/felhom-drives/felhom-usb`.
|
||
- **PBS storage:** `felhom-pbs` active, 9.6 TB total, 21% used. Nightly 9201 backups present (latest `2026-06-21T18:55:12Z`, ~19.5 GB).
|
||
- **pvesm:** felhom-flash, felhom-usb, felhom-pbs, local, local-lvm — all active.
|
||
|
||
---
|
||
|
||
## Phase 0 — Baseline + safety net
|
||
|
||
State: 9201 healthy, controller up, tunnel up (cloudflared Up 5 days).
|
||
|
||
| Check | Result | Evidence |
|
||
|---|---|---|
|
||
| Baseline capture | PASS | see above |
|
||
| GUI screenshots (before) | _pending_ | Chrome bridge check next |
|
||
| Fresh PBS whole-CT backup of 9201 | **PASS** | volid `felhom-pbs:backup/ct/9201/2026-06-22T12:45:37Z`, 19.62 GB, crash-consistent=true, 161.5s, success=true. uncovered_volumes (by design): /etc/felhom-bootstrap, /mnt/felhom-drives (drives backed up separately) |
|
||
| PBS verify of new snapshot | **PASS** | `verify_state: ok` for backup_id 9201 backup_time 2026-06-22T12:45:37Z (datastore felhom-spike, verify UPID present) |
|
||
|
||
**DR source for Phase 4:** `felhom-pbs:backup/ct/9201/2026-06-22T12:45:37Z` (verified ok). Gate to destructive phases: **OPEN**.
|
||
|
||
**GUI render smoke-test (before):** dashboard PASS (17 running / 0 stopped / 55 total; mem 5.8/12 GB; CPU 5%; 58°C; v0.73.0; app-backup "Még nem futott"), `/monitoring` PASS, `/stacks` PASS.
|
||
|
||
### Access path note (important finding)
|
||
- On-LAN, split-horizon DNS (resolver 192.168.0.250) maps `*.demo-felhom.eu` → guest 9201 (192.168.0.141), bypassing Cloudflare (no `cf-ray` on responses).
|
||
- **The guest's traefik `:443` (websecure) returns Go `404 page not found` for ALL hosts** (controller AND apps, e.g. romm), while serving a valid LE `*.demo-felhom.eu` cert. `:80` 301-redirects to https → so the browser dead-ends at the 404. → on-LAN HTTPS to the dashboard/apps is non-functional. (The supported external path is Cloudflare; couldn't validate it from the LAN due to DNS hijack.) **FINDING — flag for supervised review.**
|
||
- **Workaround used for UI testing (reversible):** a python TCP forwarder inside guest 9201 (`/tmp/felhom-test-fwd.py`, listens `0.0.0.0:18080` → controller `172.18.0.10:8080`). Browser → `http://felhom.demo-felhom.eu:18080/` (LAN DNS → guest:18080 → controller). `CatchAllMiddleware` strips the port so Host matches `felhom.<domain>` → dashboard serves over plain HTTP. **To be killed at cleanup.**
|
||
- The dashboard currently serves with **no auth wall** (`/` → 200, no login redirect, no session cookie) — no password configured/enforced. (Bears on Phase 6 "password change" step.)
|
||
|
||
### Scope reality vs runbook
|
||
- Runbook assumed "14 enabled apps"; catalog is now **55 total, 38 not-deployed**. Deploying all 38 unattended would exhaust the 12 GiB guest. **Phase 2 will deploy a representative sample** (a few light apps) + cite the already-deployed heavy multi-container case (immich, 4 containers healthy). Documented, not silently truncated.
|
||
|
||
**Phase 0 verdict: PASS — gate OPEN.**
|
||
|
||
---
|
||
|
||
## Phase 1 — komga HC fix + git-sync + health detection — **PASS**
|
||
|
||
State: 9201 healthy, controller up, tunnel up.
|
||
|
||
**Diagnosis (live, guest 9201):** docker HC `curl -f .../api/v1/actuator/health` → exit 22 (HTTP 401, auth-gated) → `unhealthy` for 18h while serving fine on :25600. Endpoint probe matrix (wget in image): `/`, `/actuator/health`, `/api/v1/oauth2/providers`, `/login` all 200; only `/api/v1/actuator/health` → 401. Image ships `curl` (`/usr/bin/curl`). **Found a second probe:** the controller's own `.felhom.yml` `healthcheck.checks[].path` (drives the UI badge + route publishing) also pointed at the 401 path.
|
||
|
||
| Check | Result | Evidence |
|
||
|---|---|---|
|
||
| Catalog HC fix (docker-compose.yml) | PASS | `/api/v1/actuator/health` → `/actuator/health`; commit `3faa5ae` pushed to `main` |
|
||
| Catalog probe fix (.felhom.yml) | PASS | controller-side probe path fixed; commit `9b066de` pushed to `main` |
|
||
| CHANGELOG + REPORT (app-catalog) | PASS | created `CHANGELOG.md` (newest-on-top) + overwrote `REPORT.md` (`f51f794`) |
|
||
| GUI git-sync ("Sablonok frissítése") | PASS | logs: `[sync] Updated komga/docker-compose.yml` + `komga/.felhom.yml`; `Sablonok frissítve — frissítve: komga`. Live stack `/opt/docker/stacks/komga/docker-compose.yml` updated to `/actuator/health` (content-hash copy verified) |
|
||
| GUI redeploy ("Frissítés") | PASS | `POST /api/stacks/komga/update` → 200; `docker compose up -d --remove-orphans`; "Stack komga updated successfully (4.2s)"; container recreated (StartedAt 2026-06-22T13:09:50Z) |
|
||
| komga reaches healthy (docker) | PASS | `docker ps`: `komga Up (healthy)`; HC now `http://localhost:25600/actuator/health` |
|
||
| Controller health-state machine | PASS | controller log `Health probes: N ok (of N probed)` (no more komga WARN); card badge `Nem egészséges` → **Fut** |
|
||
| Route publishing | PASS | "⚠ URL nem elérhető – útvonal nincs publikálva" warning cleared; `komga.demo-felhom.eu ↗` link shown |
|
||
|
||
Minor cosmetic note (not fixed): a few catalog logo SVGs 404 (`adventurelog-logo.svg`, `cloudflared-logo.svg`, `crafty-controller-logo.svg`).
|
||
## Phase 2 — App deployment sweep — **PASS (with 1 logged catalog defect)**
|
||
|
||
State: 9201 healthy, controller up, tunnel up. Scope: representative sample (see Phase-0 scope note) — catalog has 38 not-deployed apps; deploying all on a 12 GiB guest is unsafe.
|
||
|
||
| App | Result | Evidence |
|
||
|---|---|---|
|
||
| **gitea** (deploy via GUI) | PASS (after HC fix) | GUI deploy page → memory-gate bar (`6043/11904 MB 50%`, two-segment current+new) → 3-step progress panel (✅ Konfiguráció mentve → ✅ Konténerek elindultak → ⏳ inicializálás) → `POST /api/stacks/gitea/update` 200. Initially **unhealthy**: HC `curl -f /api/v1/version` → 404 (404 until install-lock) while `/api/healthz` → 200. **Fixed catalog** (commit `17e00b7` + CHANGELOG `1a44c14`), re-synced, redeployed → `docker inspect`: `healthy`, HC now `/api/healthz`; endpoint `http://127.0.0.1:3000/` → `HTTP/1.1 200 OK`. |
|
||
| **glance** (deploy via GUI) | **FAIL — logged, not fixed** | GUI deploy → 3-step panel OK, but container `Restarting (1)` crash-loop. Logs: `parsing config: reading main YAML file: open /app/config/glance.yml: no such file or directory`. **Root cause:** catalog template mounts an empty `glance_config:/app/config` named volume but never seeds the required `glance.yml`; glance refuses to start without it. **Not a trivial HC/start_period fix** (needs a seeded default config asset + wiring) → logged per runbook, not fixed unattended. Cleaned up: `POST .../stop` + `POST .../remove` → container gone. |
|
||
| **immich** (heavy multi-container, pre-deployed) | PASS (cited) | 4 containers (server/ML/postgres/redis) all `Up (healthy)` — the heavy-app coverage case. |
|
||
|
||
**Bonus coverage:** stop + remove endpoints exercised (glance teardown). **Catalog defects found this phase:** gitea HC (fixed), glance missing-config (logged for supervised fix).
|
||
## Phase 3 — Backup + restore — _pending_
|
||
## Phase 4 — DR rebuild (9300) — _pending_
|
||
## Phase 5 — Storage lifecycle — _pending_
|
||
## Phase 6 — Monitoring/alerts/hub — _pending_
|
||
## Phase 7 — Resilience (optional) — _pending_
|
||
|
||
---
|
||
|
||
## Feature matrix (filled at end)
|
||
## Cleanup confirmation (filled at end)
|
||
## Deferred — needs supervised run (filled at end)
|