2026-09-24: 09 decisions 24-25, part 5 shipped; the whole-copy truth table; fourth suppression; rows; STATUS; evidence
gates / gates (push) Successful in 27s
gates / gates (push) Successful in 27s
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
This commit is contained in:
@@ -0,0 +1,106 @@
|
||||
# 2026-09-24 — the fleet to 0.267.0 and 0.268.0; the undo after a restore; option A; the ladder on the box
|
||||
|
||||
Architecture read first and named: `architecture/09-update-architecture.md` (§3 decisions 13–25, §6.1, §6.1a,
|
||||
§6.4 parts 4–5), `07-backup-architecture.md` §6, `08-alarm-ladder.md` §5. **Method: endpoint-level** — every
|
||||
product act is the endpoint the UI invokes; pages fetched as HTML in both languages (`?lang=`); no browser.
|
||||
Scratch guest **9202** on demo-hp (Tier 0), drill catalog for every failing edge; the live catalog carried only
|
||||
the step files and gates.
|
||||
|
||||
## Not done, or changed
|
||||
|
||||
- **Nothing not done.** Parts 0, A, B, C, D and E all ran; the interim "one step per app per day" gate was not
|
||||
needed (Part D shipped).
|
||||
- **The operator's English sentence lost one word, „please"** — the house rule (`i18n_missing_gate.py`,
|
||||
`EN_FORBIDDEN`) refuses it. The Hungarian is verbatim; its formal register raised the R-516 ceiling 18 → 19 with
|
||||
the reason written beside it.
|
||||
- **The hold names ONE copy (the newest whole one), not a list.** A list would need new sentence copy; the existing
|
||||
sentence names one copy with what it holds.
|
||||
- **Part A's second failing update (g) did not fail:** right after a restore the stack dir's `.felhom.yml` was the
|
||||
restored one (real probe port), so that press was a good update. The proof the brief asks for is (e) — an app
|
||||
restored under v0.267.0, then a failing update on v0.268.0 — and it passed. (g) still shows the v0.268.0 restore's
|
||||
volumes copied with no "unlabelled" warning. Filed as R-665 (not diagnosed).
|
||||
- **R-653 and R-656 are unit-tested and red-proofed, not run on a bench** — no bench was provisioned this session.
|
||||
- **Two catalog test suites were already red at the start** (`test_gate_decoys.py`, `test_ladder_writer.py` — last
|
||||
night's pin moves under literals). Fixed and filed as R-663 (closed).
|
||||
- **One of the first R-660 control's two alarm events is not attributable** (the dropped-event line names no app);
|
||||
the repeat control produced exactly one. Likely gokapi (R-667), stated as inference.
|
||||
- An older wiring test (`TestR475_AdapterReadsEveryTier`) asserted the adapter hands the hold the precondition copy
|
||||
— the thing R-659 changes on purpose. Its assertion was replaced, with the reason in the test.
|
||||
|
||||
## Claims in the brief, checked
|
||||
|
||||
| the brief's claim | what was found |
|
||||
|---|---|
|
||||
| compose resolves volume names as `<project>_<volume>` for every template (some set `name:`) | **No template sets `name:`**, top-level or per volume (all 53 read). Supported anyway, and tested. |
|
||||
| the syncer can carry a `steps/` folder | **It does not need to.** The syncer copies two files into the stack dir; the box reads `steps/` from its catalog CLONE, which (depth 1) holds the whole tree. Measured on 9202: `steps/spike-B.yml` present in the clone after one sync (`partD/01`). |
|
||||
| „saját meghajtó" never holds files for a file-leg app (from `07` §6) | **Held, and it goes further:** read from source, the SECOND-DRIVE unit restore refuses such an app too (the R-538 guard is in the shared function), and the second drive's file restore brings back no database. So only off-site brings a file app back whole (R-661). An app with only OPTIONAL legs (romm, komga, audiobookshelf) is accepted by the unit restore. |
|
||||
| one press = one step is safe for an engine step whose datadir converts | **Held for MariaDB, measured once:** romm press 2 ran `mariadb-upgrade` 11.4.13 → 11.8.9 and the account read back. The undo copy covers the pre-conversion volume if the step fails (not exercised on an engine step this session). PostgreSQL majors stay refused (part 10). |
|
||||
| today's press jumps A → C | **Held** on v0.267.0: vikunja 2.4.0 → 2.6.0 in one press, 2.5.0 never ran (`partD/00-spike.json`). |
|
||||
|
||||
## Part 0 / Part E — the floor
|
||||
|
||||
| save (UTC) | floor / MinAgent | read back | demo-felhom | demo-hp |
|
||||
|---|---|---|---|---|
|
||||
| 05:12:14 | 0.267.0 / 0.131.0 | `min_controller_version=0.267.0`, `min_agent=0.131.0`; hub `SERVED … from declared` | 0.267.0 healthy at +28 s | same |
|
||||
| 06:47:13 | 0.268.0 / 0.131.0 | same shape | 0.268.0 healthy at +30 s | same |
|
||||
|
||||
Hub 0.122.0 rolled out first (ArgoCD, manifest `500488c`, pod image read back). **Stays below:** `drill-r50`
|
||||
(DOWN, agent 0.129.0 — held by MinAgent, a Tier-0 leftover). Evidence `part0/`.
|
||||
|
||||
## Red-proofs (each seen failing, then restored)
|
||||
|
||||
| row | mutation | failed at | file |
|
||||
|---|---|---|---|
|
||||
| R-658 undo | `appVolumes` returns the label selector | `copied 0 of 2 declared volumes` | `redproofs/A-r658-undo.txt` |
|
||||
| R-658 restore | create without labels | `created vikunja_files WITHOUT the project label` | `A-r658-restore.txt` |
|
||||
| R-658 remove report | `appVolumeSet` = labelled only | `volumes_removed = []` | `C-r651-and-A-remove-report.txt` |
|
||||
| R-651 | skip deleting the applied files | both files survive; undo would probe the REMOVED install's file | same |
|
||||
| R-659 hold | `WholeOnTier` always true | round-11 case names „saját meghajtó" | `B-r659-hold.txt` |
|
||||
| R-659 page | gate dropped from `stacks.html` | Mentések button beside a no-whole-copy hold | `B-r659-page.txt` |
|
||||
| R-659 hub | dropped from `operatorOnlyEvents` | "must be operator-only" | `B-hub-operator-only.txt` |
|
||||
| R-660 | `updateHeldSet` returns nil / union line removed | "held app reported DOWN" / wiring | `C-r660.txt` |
|
||||
| ladder (box) | `nextLadderStep` always the template | press 1 pinned C; C brought up after B failed; missing step file → `done` | `D-ladder.txt` |
|
||||
| ladder (catalog gate) | rule 4 removed | 3 decoys pass wrongly | `D-catalog-gate.txt` |
|
||||
| ladder (writer) | STEP block off | "has no definition" / file absent | `D-catalog-writer.txt` |
|
||||
| R-653 / R-656 | `load_verdict` always reached / no removal | ghost's all-`err` watch reads reached; last run's config survives | `C-r653-r656.txt` |
|
||||
|
||||
Green: controller `go build/vet/test` rc 0 (31 packages); `controller_gates.py` all OK; hub `go test` OK +
|
||||
`repo_gates.py` OK; catalog `catalog_gates.py` OK, decoys 84 OK, writer 6 OK, bench 5 OK.
|
||||
|
||||
## Live proofs on 9202
|
||||
|
||||
- **A (R-658)** — v0.267.0: vikunja installed, a pull-failing update took its own backup, restore from „helyi" →
|
||||
both volumes' labels `null` (`partA/01`). v0.268.0: failing update (2.5.0 → 2.6.0, probe 8999) → log
|
||||
*„carry no compose label … copied by name"*, *„will hold 2 named volume(s)"*, undone in 7 s, seeds A (before) and
|
||||
B (after the restore) read back. v0.268.0 restore → labels `project`/`volume`/`version`, no compose warning
|
||||
(`partA/02`). Remove → `volumes_removed` names both.
|
||||
- **B (R-659)** — nextcloud installed at ladder step 1, seeded; failing update with the undo copy's marker removed
|
||||
during `verifying` → HELD, `hold_no_whole_copy: true`; the sentence in hu and en exactly as ruled (ASCII
|
||||
fragments, positive and negative control); no Mentések button on the app page or the list, both languages; log
|
||||
*„NO copy on this box brings it back whole (seen: tier 2 …, tier 1 …; drive files declared: true)"* and
|
||||
*„DROPPED event app_hold_no_whole_copy (severity critical)"* (R-620's line). Steps line „… lépések: 1"
|
||||
(`partB/01`).
|
||||
- **C (R-660, R-651)** — dead-app heartbeat 2 s after the hold: *„0 currently down"*; no `app_start_failed` for
|
||||
the held app in ~12 min; a throwaway stopped out of band raised one (`partB/03`, `06`). Remove: `applied-compose.yml`
|
||||
and `applied-meta/` present before, gone after; every undo copy gone (`partB/07`).
|
||||
- **D (the ladder)** — romm installed at 5.3.0/11.4, seeded; the page read „Hátralévő frissítési lépések: 2" /
|
||||
"Update steps remaining: 2". Press 1 → *„ladder — step 2 of 3 … from 90dd9d68258286ef.yml"*, pin 5.3.1/11.4,
|
||||
applied == that step file, MariaDB „upgrade not required", account read back, count 1. Press 2 → *„the last step
|
||||
(3 of 3)"*, pin 5.3.1/11.8, `mariadb-upgrade` 11.4.13 → 11.8.9, account read back, count gone (`partD/10`). A
|
||||
second seed after press 1 was refused by RomM (403 for a non-admin) — the readback rests on seed A.
|
||||
|
||||
## Teardown — three layers
|
||||
|
||||
- **Machine (9202):** vikunja, nextcloud, romm, actualbudget ×2 removed THROUGH THE PRODUCT; 0 volumes, 0 undo
|
||||
copies; drive folders nextcloud/romm removed by name (R-442 keeps them on 9202); 11 test images removed by name
|
||||
(`/var/lib/docker` 27 % → 18 %), no prune. `controller.yaml` restored from `controller.yaml.pre-ladder0924`, git
|
||||
URL read back as the live catalog, cache at live `5ed599c`. **9202 stays on v0.268.0.** Standing apps as found
|
||||
(gokapi was crash-looping before the session — R-644/R-667).
|
||||
- **Host (demo-hp):** no guest created; nothing else touched.
|
||||
- **Hub:** no customer or appliance record created. Floor 0.268.0 is the intended end state.
|
||||
- **Drill repo:** reset to live `main` `5ed599c`, `has_actions: False`, private — read back.
|
||||
|
||||
## Rows
|
||||
|
||||
Closed: R-658, R-659, R-660, R-651, R-653, R-656, R-40 (superseded), R-663 (filed and fixed). Opened: R-661, R-662,
|
||||
R-664, R-665, R-666, R-667. Narrowed: R-450 (part 5 shipped). Register **336 → 335 rows, 683,233 → 679,393 bytes**.
|
||||
@@ -0,0 +1,10 @@
|
||||
2026/09/24 07:29:33 [INFO] host-report from demo-hp-bb76ea (1 guests, 4 storage targets, 2 backups, 2 restore-tests, 2 pbs-snapshots, 14808 bytes)
|
||||
2026/09/24 07:29:33 [INFO] DR-recipe host-half stored for customer demo-hp (host demo-hp-bb76ea, v1)
|
||||
2026/09/24 07:31:48 [INFO] Offsite pool-box refreshed: 0.3% full (3.0 GB of 1.00 TB), Σ shared quota 250 GB, oversub 0.24x
|
||||
2026/09/24 07:32:46 [INFO] PBS-DR box refreshed: 16.1% full (15.7 GB of 97.9 GB)
|
||||
2026/09/24 07:33:27 [INFO] wgsync: pushed 4 peers to 167.233.158.164:22
|
||||
2026/09/24 07:34:43 [INFO] felhom-hub 0.122.0 starting
|
||||
2026/09/24 07:34:43 [INFO] Default controller-version floor: 0.120.0
|
||||
2026/09/24 07:34:45 [INFO] Gitea artifact browser enabled (Day-0 version dropdowns) via http://gitea.gitea-system.svc.cluster.local:3000
|
||||
2026/09/24 07:34:45 [INFO] Registry version checker started (every 6h)
|
||||
2026/09/24 07:34:45 [DEBUG] Registry version check: latest = 0.267.0
|
||||
@@ -0,0 +1,15 @@
|
||||
privatebin privatebin/pdo:2.0.6 Up 5 hours (healthy)
|
||||
paperless-webserver ghcr.io/paperless-ngx/paperless-ngx:2.20.15 Up 5 hours (healthy)
|
||||
paperless-postgres postgres:16-alpine Up 5 hours (healthy)
|
||||
paperless-redis redis:7-alpine Up 5 hours (healthy)
|
||||
gokapi f0rc3/gokapi:v1.9.6 Restarting (1) 45 seconds ago
|
||||
felhom-controller gitea.dooplex.hu/admin/felhom-controller:0.267.0 Up 7 hours (healthy)
|
||||
filebrowser gtstef/filebrowser:1.3.3-stable Up 9 hours (healthy)
|
||||
traefik traefik:v3.6.7 Up 9 hours
|
||||
/dev/loop1 69G 18G 48G 27% /var/lib/docker
|
||||
git:
|
||||
branch: main
|
||||
repo_url: https://gitea.dooplex.hu/admin/app-catalog-felhom.eu.git
|
||||
sync_interval: 15m
|
||||
token: <redacted>
|
||||
actualbudget adventurelog audiobookshelf bentopdf bookstack calcom calibre-web claper code-server crafty-controller docmost emby filebrowser ghost gitea glance gokapi grafana gramps-web home-assistant homebox homepage immich jellyfin kimai komga mealie n8n navidrome nextcloud onlyoffice opengist outline paperless-ngx papra plant-it plex privatebin radarr rallly recipe-importer romm seerr sonarr sparkyfitness tandoor termix traefik uptime-kuma vaultwarden vikunja wanderer wger wishlist zipline
|
||||
@@ -0,0 +1,8 @@
|
||||
gitea.dooplex.hu/admin/felhom-controller:0.268.0
|
||||
gitea.dooplex.hu/admin/felhom-controller:0.268.0 Up 6 seconds (healthy)
|
||||
2026/09/24 06:16:41 pin.go:290: [INFO] [stacks] pin adoption: 0 pinned, 4 already pinned, 0 left unpinned (0 not completely observed, 0 running something the template no longer offers)
|
||||
2026/09/24 06:16:41 undo.go:656: [INFO] [stacks] applied-meta backfill (R-646): recorded 0 []; skipped 0 [] — not current with the catalog, their pinned version's .felhom.yml is no longer on the box
|
||||
2026/09/24 06:16:41 main.go:527: [INFO] Metrics collector started (60s interval)
|
||||
2026/09/24 06:16:41 scheduler.go:67: [DEBUG] [scheduler] scheduler started: periodic=8 daily=8
|
||||
2026/09/24 06:16:46 notifier.go:1255: [WARN] notifier disabled (no hub configured): DROPPED event controller_started (severity info) — further controller_started events are logged at DEBUG only
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
gitea.dooplex.hu/admin/felhom-hub:0.122.0
|
||||
2026-09-24T06:47:13Z
|
||||
HTTP/1.1 303 See Other
|
||||
Location: /configuration?flash=floor_set
|
||||
name="min_agent" value="0.131.0"
|
||||
name="min_controller_version" value="0.268.0"
|
||||
@@ -0,0 +1,6 @@
|
||||
06:47:26 felhom-pve=gitea.dooplex.hu/admin/felhom-controller:0.268.0|Up 3 seconds (health: starting) demo-hp=gitea.dooplex.hu/admin/felhom-controller:0.268.0|Up 3 seconds (health: starting)
|
||||
06:47:43 felhom-pve=gitea.dooplex.hu/admin/felhom-controller:0.268.0|Up 20 seconds (healthy) demo-hp=gitea.dooplex.hu/admin/felhom-controller:0.268.0|Up 20 seconds (healthy)
|
||||
both-arrived
|
||||
2026/09/24 08:47:14 [INFO] Global controller-version floor set to "0.268.0" (declared MinAgent "0.131.0")
|
||||
2026/09/24 08:47:16 [INFO] managed floor SERVED for demo-felhom: floor 0.268.0, agent requirement "0.131.0" from declared (golden 0.258.0)
|
||||
2026/09/24 08:47:17 [INFO] managed floor SERVED for demo-hp: floor 0.268.0, agent requirement "0.131.0" from declared (golden 0.258.0)
|
||||
@@ -0,0 +1,100 @@
|
||||
{
|
||||
"controller": "gitea.dooplex.hu/admin/felhom-controller:0.267.0",
|
||||
"drill_A": "58c7c938bc7c",
|
||||
"deployed": true,
|
||||
"seed_A": true,
|
||||
"labels_after_install": "vikunja_vikunja_data {\"com.docker.compose.config-hash\":\"fe3fc6629444c7799d20fe7f96a201fcf52bd22fe1dcea326288a2a8963b48e0\",\"com.docker.compose.project\":\"vikunja\",\"com.docker.compose.version\":\"5.5.0\",\"com.docker.compose.volume\":\"vikunja_data\"}\nvikunja_vikunja_db {\"com.docker.compose.config-hash\":\"9c995441cdf40d38f9a48997f7376d7d20f88e50557ea8bca04237eeb3b2c9d1\",\"com.docker.compose.project\":\"vikunja\",\"com.docker.compose.version\":\"5.5.0\",\"com.docker.compose.volume\":\"vikunja_db\"}\n",
|
||||
"drill_E": "ef42b4f34455",
|
||||
"badge_wait": 22.8,
|
||||
"press_backup": {
|
||||
"accepted": true,
|
||||
"http": "202",
|
||||
"phases": [
|
||||
{
|
||||
"t": 0.0,
|
||||
"phase": "backing-up",
|
||||
"label": "Biztonsági mentés készül a frissítés előtt…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 2.1,
|
||||
"phase": "safety-dump",
|
||||
"label": "Adatbázis pillanatkép…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 3.1,
|
||||
"phase": "pulling",
|
||||
"label": "Új verzió letöltése…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 4.1,
|
||||
"phase": "failed",
|
||||
"label": "A frissítés nem sikerült",
|
||||
"updating": false,
|
||||
"error": "Az új verzió letöltése nem sikerült, ezért a frissítés elmaradt. Az alkalmazás a korábbi verzióval fut tovább.",
|
||||
"hold": null
|
||||
}
|
||||
],
|
||||
"duration_s": 4.1,
|
||||
"final_phase": "failed",
|
||||
"update_error": "Az új verzió letöltése nem sikerült, ezért a frissítés elmaradt. Az alkalmazás a korábbi verzióval fut tovább.",
|
||||
"hold_reason": null,
|
||||
"state": "running"
|
||||
},
|
||||
"snapshots": [
|
||||
{
|
||||
"time": "2026-09-24T06:11:05Z",
|
||||
"short_id": "helyi",
|
||||
"tier": 1,
|
||||
"drive_label": "Belső SSD (rendszer)"
|
||||
}
|
||||
],
|
||||
"drill_back": "087b3edb7475",
|
||||
"restore": {
|
||||
"ok": true,
|
||||
"snapshot_id": "helyi",
|
||||
"snapshots": [
|
||||
{
|
||||
"time": "2026-09-24T06:11:05Z",
|
||||
"short_id": "helyi",
|
||||
"tier": 1,
|
||||
"drive_label": "Belső SSD (rendszer)"
|
||||
}
|
||||
],
|
||||
"http": "HTTP/2 302",
|
||||
"location": [
|
||||
"location: /backups/restore?flash=flash.restore.started"
|
||||
],
|
||||
"seconds": 6.1,
|
||||
"state_after": "running",
|
||||
"hold_after": null,
|
||||
"observables_after": {
|
||||
"pinned_images": {
|
||||
"vikunja": "vikunja/vikunja:2.5.0"
|
||||
},
|
||||
"installed_images": {
|
||||
"vikunja": "vikunja/vikunja:2.5.0"
|
||||
},
|
||||
"catalog_images": {
|
||||
"vikunja": "vikunja/vikunja:2.5.99-notag"
|
||||
},
|
||||
"live_compose_image_lines": [
|
||||
"image: vikunja/vikunja:2.5.0"
|
||||
],
|
||||
"docker_inspect": [
|
||||
"vikunja vikunja/vikunja:2.5.0 running=true restarts=0"
|
||||
]
|
||||
}
|
||||
},
|
||||
"labels_after_restore_v0267": "vikunja_vikunja_data null\nvikunja_vikunja_db null\n",
|
||||
"A_after_restore": true,
|
||||
"seed_B_after_restore": true
|
||||
}
|
||||
@@ -0,0 +1,30 @@
|
||||
08:10:26 drill 58c7c938bc7c: vikunja 2.5.0 (the install version) push rc=0
|
||||
08:10:30 [1] deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}
|
||||
08:10:35 [1] deployed, controller state=running, pinned={'vikunja': 'vikunja/vikunja:2.5.0'}
|
||||
08:10:36 vikunja: register http=200
|
||||
08:10:36 vikunja: create project http=201
|
||||
08:10:39 labels after install:
|
||||
vikunja_vikunja_data {"com.docker.compose.config-hash":"fe3fc6629444c7799d20fe7f96a201fcf52bd22fe1dcea326288a2a8963b48e0","com.docker.compose.project":"vikunja","com.docker.compose.version":"5.5.0","com.docker.compose.volume":"vikunja_data"}
|
||||
vikunja_vikunja_db {"com.docker.compose.config-hash":"9c995441cdf40d38f9a48997f7376d7d20f88e50557ea8bca04237eeb3b2c9d1","com.docker.compose.project":"vikunja","com.docker.compose.version":"5.5.0","com.docker.compose.volume":"vikunja_db"}
|
||||
|
||||
08:10:40 drill ef42b4f34455: vikunja 2.5.99-notag (a tag that does not exist: backing-up runs, the pull fails, nothing moves) push rc=0
|
||||
08:11:03 [sync] the badge needed 22.8s and 4 sync+rescan rounds to catch up to vikunja/vikunja:2.5.99-notag — R-607's window, measured
|
||||
08:11:03 [6] Update -> 202 {'ok': True, 'data': {'accepted': True, 'completed': False}, 'message': 'Frissítés elindult – az állapot a kártyán követhető'}
|
||||
08:11:03 + 0.0s phase=backing-up label=Biztonsági mentés készül a frissítés előtt… err=None hold=None
|
||||
08:11:05 + 2.1s phase=safety-dump label=Adatbázis pillanatkép… err=None hold=None
|
||||
08:11:06 + 3.1s phase=pulling label=Új verzió letöltése… err=None hold=None
|
||||
08:11:07 + 4.1s phase=failed label=A frissítés nem sikerült err=Az új verzió letöltése nem sikerült, ezért a frissítés elmaradt. Az alkalmazás a korábbi verzióval fut tovább. hold=None
|
||||
08:11:07 snapshots offered: [{"time": "2026-09-24T06:11:05Z", "short_id": "helyi", "tier": 1, "drive_label": "Bels\u0151 SSD (rendszer)"}]
|
||||
08:11:08 drill 087b3edb7475: vikunja 2.5.0 (back to the install version) push rc=0
|
||||
08:11:13 [R] restoring vikunja from snapshot 'helyi' (of 1 offered)
|
||||
08:11:13 [R] POST /backup/restore -> HTTP/2 302 ['location: /backups/restore?flash=flash.restore.started']
|
||||
08:11:13 + 0.0s restore (True, None, None)
|
||||
08:11:19 + 6.1s restore (False, None, None)
|
||||
08:11:19 [R] after restore: state=running hold=None phase=failed
|
||||
08:11:25 labels after the v0.267.0 restore:
|
||||
vikunja_vikunja_data null
|
||||
vikunja_vikunja_db null
|
||||
|
||||
08:11:26 vikunja: readback of the seeded project http=200 ok=True
|
||||
08:11:26 vikunja: register http=200
|
||||
08:11:26 vikunja: create project http=201
|
||||
@@ -0,0 +1,30 @@
|
||||
08:10:26 drill 58c7c938bc7c: vikunja 2.5.0 (the install version) push rc=0
|
||||
08:10:30 [1] deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}
|
||||
08:10:35 [1] deployed, controller state=running, pinned={'vikunja': 'vikunja/vikunja:2.5.0'}
|
||||
08:10:36 vikunja: register http=200
|
||||
08:10:36 vikunja: create project http=201
|
||||
08:10:39 labels after install:
|
||||
vikunja_vikunja_data {"com.docker.compose.config-hash":"fe3fc6629444c7799d20fe7f96a201fcf52bd22fe1dcea326288a2a8963b48e0","com.docker.compose.project":"vikunja","com.docker.compose.version":"5.5.0","com.docker.compose.volume":"vikunja_data"}
|
||||
vikunja_vikunja_db {"com.docker.compose.config-hash":"9c995441cdf40d38f9a48997f7376d7d20f88e50557ea8bca04237eeb3b2c9d1","com.docker.compose.project":"vikunja","com.docker.compose.version":"5.5.0","com.docker.compose.volume":"vikunja_db"}
|
||||
|
||||
08:10:40 drill ef42b4f34455: vikunja 2.5.99-notag (a tag that does not exist: backing-up runs, the pull fails, nothing moves) push rc=0
|
||||
08:11:03 [sync] the badge needed 22.8s and 4 sync+rescan rounds to catch up to vikunja/vikunja:2.5.99-notag — R-607's window, measured
|
||||
08:11:03 [6] Update -> 202 {'ok': True, 'data': {'accepted': True, 'completed': False}, 'message': 'Frissítés elindult – az állapot a kártyán követhető'}
|
||||
08:11:03 + 0.0s phase=backing-up label=Biztonsági mentés készül a frissítés előtt… err=None hold=None
|
||||
08:11:05 + 2.1s phase=safety-dump label=Adatbázis pillanatkép… err=None hold=None
|
||||
08:11:06 + 3.1s phase=pulling label=Új verzió letöltése… err=None hold=None
|
||||
08:11:07 + 4.1s phase=failed label=A frissítés nem sikerült err=Az új verzió letöltése nem sikerült, ezért a frissítés elmaradt. Az alkalmazás a korábbi verzióval fut tovább. hold=None
|
||||
08:11:07 snapshots offered: [{"time": "2026-09-24T06:11:05Z", "short_id": "helyi", "tier": 1, "drive_label": "Bels\u0151 SSD (rendszer)"}]
|
||||
08:11:08 drill 087b3edb7475: vikunja 2.5.0 (back to the install version) push rc=0
|
||||
08:11:13 [R] restoring vikunja from snapshot 'helyi' (of 1 offered)
|
||||
08:11:13 [R] POST /backup/restore -> HTTP/2 302 ['location: /backups/restore?flash=flash.restore.started']
|
||||
08:11:13 + 0.0s restore (True, None, None)
|
||||
08:11:19 + 6.1s restore (False, None, None)
|
||||
08:11:19 [R] after restore: state=running hold=None phase=failed
|
||||
08:11:25 labels after the v0.267.0 restore:
|
||||
vikunja_vikunja_data null
|
||||
vikunja_vikunja_db null
|
||||
|
||||
08:11:26 vikunja: readback of the seeded project http=200 ok=True
|
||||
08:11:26 vikunja: register http=200
|
||||
08:11:26 vikunja: create project http=201
|
||||
@@ -0,0 +1,205 @@
|
||||
{
|
||||
"controller": "gitea.dooplex.hu/admin/felhom-controller:0.268.0",
|
||||
"probe_port_real": 3456,
|
||||
"labels_before": "vikunja_vikunja_data null\nvikunja_vikunja_db null\n",
|
||||
"drill_e": "17a5e25f99bc",
|
||||
"badge_e": 4.5,
|
||||
"press_e": {
|
||||
"accepted": true,
|
||||
"http": "202",
|
||||
"phases": [
|
||||
{
|
||||
"t": 0.0,
|
||||
"phase": "safety-dump",
|
||||
"label": "Adatbázis pillanatkép…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 1.1,
|
||||
"phase": "pulling",
|
||||
"label": "Új verzió letöltése…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 3.1,
|
||||
"phase": "copying",
|
||||
"label": "Az adatok másolása a frissítés előtt…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 4.1,
|
||||
"phase": "verifying",
|
||||
"label": "Működés ellenőrzése…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 94.3,
|
||||
"phase": "undoing",
|
||||
"label": "Visszaállítás az előző változatra…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 102.5,
|
||||
"phase": "undone",
|
||||
"label": "Visszaállítva az előző változatra",
|
||||
"updating": false,
|
||||
"error": null,
|
||||
"hold": null
|
||||
}
|
||||
],
|
||||
"duration_s": 102.5,
|
||||
"final_phase": "undone",
|
||||
"update_error": null,
|
||||
"hold_reason": null,
|
||||
"state": "running"
|
||||
},
|
||||
"log_e": "2026/09/24 06:17:27 update.go:722: [INFO] [stacks] update vikunja: ladder — the installed version vikunja=vikunja/vikunja:2.5.0 matches no update_ladder entry (1 entries) — an app older than the ladder has no record to climb; the catalog's current definition\n2026/09/24 06:17:27 undo.go:387: [WARN] [stacks] update vikunja: volume(s) [vikunja_vikunja_data vikunja_vikunja_db] carry no compose label (recreated by a restore before v0.268.0 — R-658) — copied by name\n2026/09/24 06:17:28 undo.go:273: [INFO] [stacks] update vikunja: the undo copy will hold 2 named volume(s), 2.6 MiB\n2026/09/24 06:17:30 undo.go:423: [INFO] [stacks] update vikunja: copied vikunja_vikunja_data → vikunja_vikunja_data.pre-update-20260924T061729Z in 431ms\n2026/09/24 06:17:30 undo.go:423: [INFO] [stacks] update vikunja: copied vikunja_vikunja_db → vikunja_vikunja_db.pre-update-20260924T061729Z in 428ms\n2026/09/24 06:19:01 undo.go:501: [WARN] [stacks] update vikunja: UNDO — putting back the previous version and its 2 volume copy(ies) (reason: not healthy: not healthy within 1m30s (last: state unhealthy))\n2026/09/24 06:19:09 undo.go:549: [INFO] [stacks] update vikunja: UNDONE in 7s — the previous version is running on the data from before the update (the app's health check passed)\n",
|
||||
"read_e": {
|
||||
"A": true,
|
||||
"B": true
|
||||
},
|
||||
"obs_e": {
|
||||
"pinned_images": {
|
||||
"vikunja": "vikunja/vikunja:2.5.0"
|
||||
},
|
||||
"installed_images": {
|
||||
"vikunja": "vikunja/vikunja:2.5.0"
|
||||
},
|
||||
"catalog_images": {
|
||||
"vikunja": "vikunja/vikunja:2.6.0"
|
||||
},
|
||||
"live_compose_image_lines": [
|
||||
"image: vikunja/vikunja:2.5.0"
|
||||
],
|
||||
"docker_inspect": [
|
||||
"vikunja vikunja/vikunja:2.5.0 running=true restarts=0"
|
||||
]
|
||||
},
|
||||
"restore_f": {
|
||||
"ok": true,
|
||||
"snapshot_id": "helyi",
|
||||
"snapshots": [
|
||||
{
|
||||
"time": "2026-09-24T06:16:42Z",
|
||||
"short_id": "helyi",
|
||||
"tier": 1,
|
||||
"drive_label": "Belső SSD (rendszer)"
|
||||
}
|
||||
],
|
||||
"http": "HTTP/2 302",
|
||||
"location": [
|
||||
"location: /backups/restore?flash=flash.restore.started"
|
||||
],
|
||||
"seconds": 97.2,
|
||||
"state_after": "unhealthy",
|
||||
"hold_after": null,
|
||||
"observables_after": {
|
||||
"pinned_images": {
|
||||
"vikunja": "vikunja/vikunja:2.5.0"
|
||||
},
|
||||
"installed_images": {
|
||||
"vikunja": "vikunja/vikunja:2.5.0"
|
||||
},
|
||||
"catalog_images": {
|
||||
"vikunja": "vikunja/vikunja:2.6.0"
|
||||
},
|
||||
"live_compose_image_lines": [
|
||||
"image: vikunja/vikunja:2.5.0"
|
||||
],
|
||||
"docker_inspect": [
|
||||
"vikunja vikunja/vikunja:2.5.0 running=true restarts=0"
|
||||
]
|
||||
}
|
||||
},
|
||||
"labels_after_restore_v0268": "vikunja_vikunja_data {\"com.docker.compose.project\":\"vikunja\",\"com.docker.compose.version\":\"5.5.0\",\"com.docker.compose.volume\":\"vikunja_data\"}\nvikunja_vikunja_db {\"com.docker.compose.project\":\"vikunja\",\"com.docker.compose.version\":\"5.5.0\",\"com.docker.compose.volume\":\"vikunja_db\"}\n",
|
||||
"log_f": "2026/09/24 06:19:17 restore.go:152: [INFO] [backup] Restoring Docker volume vikunja_vikunja_data for vikunja\n2026/09/24 06:19:18 restore.go:152: [INFO] [backup] Restoring Docker volume vikunja_vikunja_db for vikunja\n",
|
||||
"compose_warn_f": "",
|
||||
"read_f": {
|
||||
"A": true
|
||||
},
|
||||
"press_g": {
|
||||
"accepted": true,
|
||||
"http": "202",
|
||||
"phases": [
|
||||
{
|
||||
"t": 0.0,
|
||||
"phase": "backing-up",
|
||||
"label": "Biztonsági mentés készül a frissítés előtt…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 2.1,
|
||||
"phase": "safety-dump",
|
||||
"label": "Adatbázis pillanatkép…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 3.1,
|
||||
"phase": "pulling",
|
||||
"label": "Új verzió letöltése…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 4.1,
|
||||
"phase": "copying",
|
||||
"label": "Az adatok másolása a frissítés előtt…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 5.2,
|
||||
"phase": "starting",
|
||||
"label": "Indítás az új verzióval…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 6.2,
|
||||
"phase": "verifying",
|
||||
"label": "Működés ellenőrzése…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 11.3,
|
||||
"phase": "done",
|
||||
"label": "Frissítve",
|
||||
"updating": false,
|
||||
"error": null,
|
||||
"hold": null
|
||||
}
|
||||
],
|
||||
"duration_s": 11.3,
|
||||
"final_phase": "done",
|
||||
"update_error": null,
|
||||
"hold_reason": null,
|
||||
"state": "running"
|
||||
},
|
||||
"log_g": "2026/09/24 06:21:07 update.go:722: [INFO] [stacks] update vikunja: ladder — the installed version vikunja=vikunja/vikunja:2.5.0 matches no update_ladder entry (1 entries) — an app older than the ladder has no record to climb; the catalog's current definition\n2026/09/24 06:21:09 tier2.go:425: [INFO] [backup] Tier 2 copied vikunja → /mnt/felhom-drives/scratch_hdd/backups/secondary/vikunja (2.6 MB, 0 leg(s), 0s)\n2026/09/24 06:21:10 undo.go:273: [INFO] [stacks] update vikunja: the undo copy will hold 2 named volume(s), 2.6 MiB\n2026/09/24 06:21:12 undo.go:423: [INFO] [stacks] update vikunja: copied vikunja_vikunja_data → vikunja_vikunja_data.pre-update-20260924T062112Z in 481ms\n2026/09/24 06:21:13 undo.go:423: [INFO] [stacks] update vikunja: copied vikunja_vikunja_db → vikunja_vikunja_db.pre-update-20260924T062112Z in 462ms\n",
|
||||
"read_g": {
|
||||
"A": true,
|
||||
"C": true
|
||||
},
|
||||
"drill_h": "418e8ed2aeb2",
|
||||
"remove_h": "200",
|
||||
"leftovers_h": "total 24\ndrwxr-xr-x 3 root root 4096 Sep 24 06:21 .\ndrwxr-xr-x 57 root root 4096 Sep 13 20:22 ..\n-rw-r--r-- 1 root root 4419 Sep 24 06:19 .felhom.yml\n-rw-r--r-- 1 root root 1280 Sep 24 06:21 docker-compose.yml\ndrwxr-xr-x 12 root root 4096 Sep 24 06:19 hold-logs\nno vikunja volumes\n"
|
||||
}
|
||||
@@ -0,0 +1,69 @@
|
||||
08:17:21 labels before (unlabelled after the v0.267.0 restore):
|
||||
vikunja_vikunja_data null
|
||||
vikunja_vikunja_db null
|
||||
|
||||
08:17:22 drill 17a5e25f99bc: vikunja 2.6.0 + probe 8999 (the failing edge) push rc=0
|
||||
08:17:27 [6] Update -> 202 {'ok': True, 'data': {'accepted': True, 'completed': False}, 'message': 'Frissítés elindult – az állapot a kártyán követhető'}
|
||||
08:17:27 + 0.0s phase=safety-dump label=Adatbázis pillanatkép… err=None hold=None
|
||||
08:17:28 + 1.1s phase=pulling label=Új verzió letöltése… err=None hold=None
|
||||
08:17:30 + 3.1s phase=copying label=Az adatok másolása a frissítés előtt… err=None hold=None
|
||||
08:17:31 + 4.1s phase=verifying label=Működés ellenőrzése… err=None hold=None
|
||||
08:19:01 + 94.3s phase=undoing label=Visszaállítás az előző változatra… err=None hold=None
|
||||
08:19:10 + 102.5s phase=undone label=Visszaállítva az előző változatra err=None hold=None
|
||||
08:19:13 controller log (e):
|
||||
2026/09/24 06:17:27 update.go:722: [INFO] [stacks] update vikunja: ladder — the installed version vikunja=vikunja/vikunja:2.5.0 matches no update_ladder entry (1 entries) — an app older than the ladder has no record to climb; the catalog's current definition
|
||||
2026/09/24 06:17:27 undo.go:387: [WARN] [stacks] update vikunja: volume(s) [vikunja_vikunja_data vikunja_vikunja_db] carry no compose label (recreated by a restore before v0.268.0 — R-658) — copied by name
|
||||
2026/09/24 06:17:28 undo.go:273: [INFO] [stacks] update vikunja: the undo copy will hold 2 named volume(s), 2.6 MiB
|
||||
2026/09/24 06:17:30 undo.go:423: [INFO] [stacks] update vikunja: copied vikunja_vikunja_data → vikunja_vikunja_data.pre-update-20260924T061729Z in 431ms
|
||||
2026/09/24 06:17:30 undo.go:423: [INFO] [stacks] update vikunja: copied vikunja_vikunja_db → vikunja_vikunja_db.pre-update-20260924T061729Z in 428ms
|
||||
2026/09/24 06:19:01 undo.go:501: [WARN] [stacks] update vikunja: UNDO — putting back the previous version and its 2 volume copy(ies) (reason: not healthy: not healthy within 1m30s (last: state unhealthy))
|
||||
2026/09/24 06:19:09 undo.go:549: [INFO] [stacks] update vikunja: UNDONE in 7s — the previous version is running on the data from before the update (the app's health check passed)
|
||||
|
||||
08:19:13 vikunja: readback of the seeded project http=200 ok=True
|
||||
08:19:14 vikunja: readback of the seeded project http=200 ok=True
|
||||
08:19:14 READBACK after the undo (e): {'A': True, 'B': True}
|
||||
08:19:17 [R] restoring vikunja from snapshot 'helyi' (of 1 offered)
|
||||
08:19:17 [R] POST /backup/restore -> HTTP/2 302 ['location: /backups/restore?flash=flash.restore.started']
|
||||
08:19:17 + 0.0s restore (True, None, None)
|
||||
08:20:54 + 97.2s restore (False, None, None)
|
||||
08:20:54 [R] after restore: state=unhealthy hold=None phase=undone
|
||||
08:21:00 labels after the v0.268.0 restore:
|
||||
vikunja_vikunja_data {"com.docker.compose.project":"vikunja","com.docker.compose.version":"5.5.0","com.docker.compose.volume":"vikunja_data"}
|
||||
vikunja_vikunja_db {"com.docker.compose.project":"vikunja","com.docker.compose.version":"5.5.0","com.docker.compose.volume":"vikunja_db"}
|
||||
|
||||
08:21:06 compose warnings after the restore (want none): (none)
|
||||
08:21:07 vikunja: readback of the seeded project http=200 ok=True
|
||||
08:21:07 READBACK after the v0.268.0 restore (f): {'A': True}
|
||||
08:21:07 vikunja: register http=200
|
||||
08:21:07 vikunja: create project http=201
|
||||
08:21:07 [6] Update -> 202 {'ok': True, 'data': {'accepted': True, 'completed': False}, 'message': 'Frissítés elindult – az állapot a kártyán követhető'}
|
||||
08:21:08 + 0.0s phase=backing-up label=Biztonsági mentés készül a frissítés előtt… err=None hold=None
|
||||
08:21:10 + 2.1s phase=safety-dump label=Adatbázis pillanatkép… err=None hold=None
|
||||
08:21:11 + 3.1s phase=pulling label=Új verzió letöltése… err=None hold=None
|
||||
08:21:12 + 4.1s phase=copying label=Az adatok másolása a frissítés előtt… err=None hold=None
|
||||
08:21:13 + 5.2s phase=starting label=Indítás az új verzióval… err=None hold=None
|
||||
08:21:14 + 6.2s phase=verifying label=Működés ellenőrzése… err=None hold=None
|
||||
08:21:19 + 11.3s phase=done label=Frissítve err=None hold=None
|
||||
08:21:22 controller log (g):
|
||||
2026/09/24 06:21:07 update.go:722: [INFO] [stacks] update vikunja: ladder — the installed version vikunja=vikunja/vikunja:2.5.0 matches no update_ladder entry (1 entries) — an app older than the ladder has no record to climb; the catalog's current definition
|
||||
2026/09/24 06:21:09 tier2.go:425: [INFO] [backup] Tier 2 copied vikunja → /mnt/felhom-drives/scratch_hdd/backups/secondary/vikunja (2.6 MB, 0 leg(s), 0s)
|
||||
2026/09/24 06:21:10 undo.go:273: [INFO] [stacks] update vikunja: the undo copy will hold 2 named volume(s), 2.6 MiB
|
||||
2026/09/24 06:21:12 undo.go:423: [INFO] [stacks] update vikunja: copied vikunja_vikunja_data → vikunja_vikunja_data.pre-update-20260924T062112Z in 481ms
|
||||
2026/09/24 06:21:13 undo.go:423: [INFO] [stacks] update vikunja: copied vikunja_vikunja_db → vikunja_vikunja_db.pre-update-20260924T062112Z in 462ms
|
||||
|
||||
08:21:23 vikunja: readback of the seeded project http=200 ok=True
|
||||
08:21:23 vikunja: readback of the seeded project http=200 ok=True
|
||||
08:21:23 READBACK after the second undo (g): {'A': True, 'C': True}
|
||||
08:21:24 drill 418e8ed2aeb2: vikunja back to 2.5.0 + its real probe push rc=0
|
||||
08:21:24 [X] stop -> 200 {'ok': True, 'message': 'Stack vikunja stop completed'}
|
||||
08:21:56 [X] remove (with drive data) -> 200 {'ok': True, 'data': {'removed': 'vikunja', 'volumes_removed': ['vikunja_vikunja_data', 'vikunja_vikunja_db'], 'hdd_paths_removed': [], 'hdd_paths_preserved': [
|
||||
08:22:04 [X] after remove: deployed=False leftovers='/opt/docker/stacks/vikunja'
|
||||
08:22:07 after remove:
|
||||
total 24
|
||||
drwxr-xr-x 3 root root 4096 Sep 24 06:21 .
|
||||
drwxr-xr-x 57 root root 4096 Sep 13 20:22 ..
|
||||
-rw-r--r-- 1 root root 4419 Sep 24 06:19 .felhom.yml
|
||||
-rw-r--r-- 1 root root 1280 Sep 24 06:21 docker-compose.yml
|
||||
drwxr-xr-x 12 root root 4096 Sep 24 06:19 hold-logs
|
||||
no vikunja volumes
|
||||
|
||||
@@ -0,0 +1,69 @@
|
||||
08:17:21 labels before (unlabelled after the v0.267.0 restore):
|
||||
vikunja_vikunja_data null
|
||||
vikunja_vikunja_db null
|
||||
|
||||
08:17:22 drill 17a5e25f99bc: vikunja 2.6.0 + probe 8999 (the failing edge) push rc=0
|
||||
08:17:27 [6] Update -> 202 {'ok': True, 'data': {'accepted': True, 'completed': False}, 'message': 'Frissítés elindult – az állapot a kártyán követhető'}
|
||||
08:17:27 + 0.0s phase=safety-dump label=Adatbázis pillanatkép… err=None hold=None
|
||||
08:17:28 + 1.1s phase=pulling label=Új verzió letöltése… err=None hold=None
|
||||
08:17:30 + 3.1s phase=copying label=Az adatok másolása a frissítés előtt… err=None hold=None
|
||||
08:17:31 + 4.1s phase=verifying label=Működés ellenőrzése… err=None hold=None
|
||||
08:19:01 + 94.3s phase=undoing label=Visszaállítás az előző változatra… err=None hold=None
|
||||
08:19:10 + 102.5s phase=undone label=Visszaállítva az előző változatra err=None hold=None
|
||||
08:19:13 controller log (e):
|
||||
2026/09/24 06:17:27 update.go:722: [INFO] [stacks] update vikunja: ladder — the installed version vikunja=vikunja/vikunja:2.5.0 matches no update_ladder entry (1 entries) — an app older than the ladder has no record to climb; the catalog's current definition
|
||||
2026/09/24 06:17:27 undo.go:387: [WARN] [stacks] update vikunja: volume(s) [vikunja_vikunja_data vikunja_vikunja_db] carry no compose label (recreated by a restore before v0.268.0 — R-658) — copied by name
|
||||
2026/09/24 06:17:28 undo.go:273: [INFO] [stacks] update vikunja: the undo copy will hold 2 named volume(s), 2.6 MiB
|
||||
2026/09/24 06:17:30 undo.go:423: [INFO] [stacks] update vikunja: copied vikunja_vikunja_data → vikunja_vikunja_data.pre-update-20260924T061729Z in 431ms
|
||||
2026/09/24 06:17:30 undo.go:423: [INFO] [stacks] update vikunja: copied vikunja_vikunja_db → vikunja_vikunja_db.pre-update-20260924T061729Z in 428ms
|
||||
2026/09/24 06:19:01 undo.go:501: [WARN] [stacks] update vikunja: UNDO — putting back the previous version and its 2 volume copy(ies) (reason: not healthy: not healthy within 1m30s (last: state unhealthy))
|
||||
2026/09/24 06:19:09 undo.go:549: [INFO] [stacks] update vikunja: UNDONE in 7s — the previous version is running on the data from before the update (the app's health check passed)
|
||||
|
||||
08:19:13 vikunja: readback of the seeded project http=200 ok=True
|
||||
08:19:14 vikunja: readback of the seeded project http=200 ok=True
|
||||
08:19:14 READBACK after the undo (e): {'A': True, 'B': True}
|
||||
08:19:17 [R] restoring vikunja from snapshot 'helyi' (of 1 offered)
|
||||
08:19:17 [R] POST /backup/restore -> HTTP/2 302 ['location: /backups/restore?flash=flash.restore.started']
|
||||
08:19:17 + 0.0s restore (True, None, None)
|
||||
08:20:54 + 97.2s restore (False, None, None)
|
||||
08:20:54 [R] after restore: state=unhealthy hold=None phase=undone
|
||||
08:21:00 labels after the v0.268.0 restore:
|
||||
vikunja_vikunja_data {"com.docker.compose.project":"vikunja","com.docker.compose.version":"5.5.0","com.docker.compose.volume":"vikunja_data"}
|
||||
vikunja_vikunja_db {"com.docker.compose.project":"vikunja","com.docker.compose.version":"5.5.0","com.docker.compose.volume":"vikunja_db"}
|
||||
|
||||
08:21:06 compose warnings after the restore (want none): (none)
|
||||
08:21:07 vikunja: readback of the seeded project http=200 ok=True
|
||||
08:21:07 READBACK after the v0.268.0 restore (f): {'A': True}
|
||||
08:21:07 vikunja: register http=200
|
||||
08:21:07 vikunja: create project http=201
|
||||
08:21:07 [6] Update -> 202 {'ok': True, 'data': {'accepted': True, 'completed': False}, 'message': 'Frissítés elindult – az állapot a kártyán követhető'}
|
||||
08:21:08 + 0.0s phase=backing-up label=Biztonsági mentés készül a frissítés előtt… err=None hold=None
|
||||
08:21:10 + 2.1s phase=safety-dump label=Adatbázis pillanatkép… err=None hold=None
|
||||
08:21:11 + 3.1s phase=pulling label=Új verzió letöltése… err=None hold=None
|
||||
08:21:12 + 4.1s phase=copying label=Az adatok másolása a frissítés előtt… err=None hold=None
|
||||
08:21:13 + 5.2s phase=starting label=Indítás az új verzióval… err=None hold=None
|
||||
08:21:14 + 6.2s phase=verifying label=Működés ellenőrzése… err=None hold=None
|
||||
08:21:19 + 11.3s phase=done label=Frissítve err=None hold=None
|
||||
08:21:22 controller log (g):
|
||||
2026/09/24 06:21:07 update.go:722: [INFO] [stacks] update vikunja: ladder — the installed version vikunja=vikunja/vikunja:2.5.0 matches no update_ladder entry (1 entries) — an app older than the ladder has no record to climb; the catalog's current definition
|
||||
2026/09/24 06:21:09 tier2.go:425: [INFO] [backup] Tier 2 copied vikunja → /mnt/felhom-drives/scratch_hdd/backups/secondary/vikunja (2.6 MB, 0 leg(s), 0s)
|
||||
2026/09/24 06:21:10 undo.go:273: [INFO] [stacks] update vikunja: the undo copy will hold 2 named volume(s), 2.6 MiB
|
||||
2026/09/24 06:21:12 undo.go:423: [INFO] [stacks] update vikunja: copied vikunja_vikunja_data → vikunja_vikunja_data.pre-update-20260924T062112Z in 481ms
|
||||
2026/09/24 06:21:13 undo.go:423: [INFO] [stacks] update vikunja: copied vikunja_vikunja_db → vikunja_vikunja_db.pre-update-20260924T062112Z in 462ms
|
||||
|
||||
08:21:23 vikunja: readback of the seeded project http=200 ok=True
|
||||
08:21:23 vikunja: readback of the seeded project http=200 ok=True
|
||||
08:21:23 READBACK after the second undo (g): {'A': True, 'C': True}
|
||||
08:21:24 drill 418e8ed2aeb2: vikunja back to 2.5.0 + its real probe push rc=0
|
||||
08:21:24 [X] stop -> 200 {'ok': True, 'message': 'Stack vikunja stop completed'}
|
||||
08:21:56 [X] remove (with drive data) -> 200 {'ok': True, 'data': {'removed': 'vikunja', 'volumes_removed': ['vikunja_vikunja_data', 'vikunja_vikunja_db'], 'hdd_paths_removed': [], 'hdd_paths_preserved': [
|
||||
08:22:04 [X] after remove: deployed=False leftovers='/opt/docker/stacks/vikunja'
|
||||
08:22:07 after remove:
|
||||
total 24
|
||||
drwxr-xr-x 3 root root 4096 Sep 24 06:21 .
|
||||
drwxr-xr-x 57 root root 4096 Sep 13 20:22 ..
|
||||
-rw-r--r-- 1 root root 4419 Sep 24 06:19 .felhom.yml
|
||||
-rw-r--r-- 1 root root 1280 Sep 24 06:21 docker-compose.yml
|
||||
drwxr-xr-x 12 root root 4096 Sep 24 06:19 hold-logs
|
||||
no vikunja volumes
|
||||
|
||||
@@ -0,0 +1,31 @@
|
||||
2026/09/24 06:19:17 handlers.go:1701: [WARN] [web] Restore requested (async): stack=vikunja, snapshot=helyi from 172.18.0.9:57016
|
||||
2026/09/24 06:19:17 restore_unit.go:379: [INFO] [backup] Restoring vikunja from recovery unit /mnt/sys_drive/felhom-data/backups/primary/vikunja: images=1, secrets recovered=1/1, data_keys=0
|
||||
2026/09/24 06:19:17 manager.go:1265: [INFO] [stacks] Stopping stack: vikunja
|
||||
2026/09/24 06:19:17 restore.go:152: [INFO] [backup] Restoring Docker volume vikunja_vikunja_data for vikunja
|
||||
2026/09/24 06:19:18 restore.go:152: [INFO] [backup] Restoring Docker volume vikunja_vikunja_db for vikunja
|
||||
2026/09/24 06:19:18 restore.go:195: [INFO] [backup] Restored 2 Docker volume(s) for vikunja
|
||||
2026/09/24 06:19:18 deploy.go:645: [INFO] [stacks] Redeploying vikunja from recovery unit with 3 env vars
|
||||
2026/09/24 06:19:18 pin.go:93: [INFO] [stacks] pin vikunja: vikunja=vikunja/vikunja:2.5.0
|
||||
2026/09/24 06:19:21 healthprobe.go:190: [WARN] Health probe vikunja: API GET :8999/api/v1/info → Get "http://vikunja:8999/api/v1/info": dial tcp 172.18.0.5:8999: connect: connection refused
|
||||
2026/09/24 06:19:31 healthprobe.go:190: [WARN] Health probe vikunja: API GET :8999/api/v1/info → Get "http://vikunja:8999/api/v1/info": dial tcp 172.18.0.5:8999: connect: connection refused
|
||||
2026/09/24 06:19:41 healthprobe.go:190: [WARN] Health probe vikunja: API GET :8999/api/v1/info → Get "http://vikunja:8999/api/v1/info": dial tcp 172.18.0.5:8999: connect: connection refused
|
||||
2026/09/24 06:19:51 healthprobe.go:190: [WARN] Health probe vikunja: API GET :8999/api/v1/info → Get "http://vikunja:8999/api/v1/info": dial tcp 172.18.0.5:8999: connect: connection refused
|
||||
2026/09/24 06:20:01 healthprobe.go:190: [WARN] Health probe vikunja: API GET :8999/api/v1/info → Get "http://vikunja:8999/api/v1/info": dial tcp 172.18.0.5:8999: connect: connection refused
|
||||
2026/09/24 06:20:21 healthprobe.go:190: [WARN] Health probe vikunja: API GET :8999/api/v1/info → Get "http://vikunja:8999/api/v1/info": dial tcp 172.18.0.5:8999: connect: connection refused
|
||||
2026/09/24 06:20:41 healthprobe.go:190: [WARN] Health probe vikunja: API GET :8999/api/v1/info → Get "http://vikunja:8999/api/v1/info": dial tcp 172.18.0.5:8999: connect: connection refused
|
||||
2026/09/24 06:20:52 restore_unit.go:458: [WARN] [backup] vikunja restored but health check failed: stack vikunja did not reach running state within 1m30s after restore
|
||||
2026/09/24 06:20:52 restore_unit.go:464: [INFO] [backup] Restore-from-unit completed: vikunja — 2 volume(s) of 2 listed, 0 database(s) of 0 listed
|
||||
2026/09/24 06:20:52 handlers.go:1713: [INFO] [web] Restore completed (async): stack=vikunja in 1m35.21599499s (volumes 2/2, dbs 0/0)
|
||||
2026/09/24 06:21:07 update.go:733: [INFO] [stacks] update vikunja: no usable copy on any tier — younger than 24h0m0s and not older than this install's deploy (2026-09-24T06:19:18Z) (found: Tier 2 (second drive) at 2026-09-24T06:11:05Z (10m0s old); Tier 1 (own recovery unit) at 2026-09-24T06:16:42Z (4m0s old)) — backing up first
|
||||
2026/09/24 06:21:07 update_guard.go:360: [INFO] [backup] update pre-backup for vikunja: starting (DB dump → volume dump → unit capture → Tier 2)
|
||||
2026/09/24 06:21:08 backup.go:918: [INFO] [backup] Stopping vikunja for safe volume dump
|
||||
2026/09/24 06:21:08 manager.go:1265: [INFO] [stacks] Stopping stack: vikunja
|
||||
2026/09/24 06:21:09 update_guard.go:416: [INFO] [backup] update pre-backup for vikunja: recovery unit captured (0 database dump(s))
|
||||
2026/09/24 06:21:09 update.go:763: [INFO] [stacks] update vikunja: safety dump done (0 file(s)) []
|
||||
2026/09/24 06:21:10 update.go:1213: [INFO] [stacks] update vikunja: phase pinning
|
||||
2026/09/24 06:21:10 pin.go:370: [INFO] [stacks] update vikunja: pin advanced to the catalog's current definition (vikunja=vikunja/vikunja:2.6.0)
|
||||
2026/09/24 06:21:13 healthprobe.go:190: [WARN] Health probe vikunja: API GET :3456/api/v1/info → Get "http://vikunja:3456/api/v1/info": dial tcp 172.18.0.5:3456: connect: connection refused
|
||||
2026/09/24 06:21:18 update.go:884: [INFO] [stacks] update vikunja: healthy after 5s (the app's health check passed)
|
||||
2026/09/24 06:21:18 update.go:892: [INFO] [stacks] update vikunja: DONE in 11s
|
||||
2026/09/24 06:21:24 manager.go:1265: [INFO] [stacks] Stopping stack: vikunja
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"A": {
|
||||
"user": "drill9ba9d9",
|
||||
"pw": "<redacted \u2014 a throwaway account on a removed app>",
|
||||
"title": "drill-d60cb3a1c9",
|
||||
"pid": 2
|
||||
},
|
||||
"B": {
|
||||
"user": "drilld26feb",
|
||||
"pw": "<redacted \u2014 a throwaway account on a removed app>",
|
||||
"title": "drill-4821c514ac",
|
||||
"pid": 4
|
||||
},
|
||||
"sub": "vikunja-la"
|
||||
}
|
||||
@@ -0,0 +1,75 @@
|
||||
{
|
||||
"controller": "gitea.dooplex.hu/admin/felhom-controller:0.268.0",
|
||||
"drill_1": "6a41e807228f",
|
||||
"deployed": true,
|
||||
"seed_A": true,
|
||||
"drill_2": "a6d49ca9eabc",
|
||||
"badge_wait": 4.4,
|
||||
"steps_left_line_hu": [
|
||||
[
|
||||
"1",
|
||||
"Hátralévő frissítési lépések: 1"
|
||||
]
|
||||
],
|
||||
"steps_left_line_en": [
|
||||
[
|
||||
"1",
|
||||
"Update steps remaining: 1"
|
||||
]
|
||||
],
|
||||
"phases": [
|
||||
[
|
||||
0.0,
|
||||
"backing-up"
|
||||
],
|
||||
[
|
||||
18.9,
|
||||
"safety-dump"
|
||||
],
|
||||
[
|
||||
21.0,
|
||||
"pulling"
|
||||
],
|
||||
[
|
||||
22.6,
|
||||
"copying"
|
||||
],
|
||||
[
|
||||
36.8,
|
||||
"starting"
|
||||
],
|
||||
[
|
||||
43.1,
|
||||
"verifying"
|
||||
],
|
||||
[
|
||||
133.4,
|
||||
"undoing"
|
||||
],
|
||||
[
|
||||
136.0,
|
||||
"failed"
|
||||
]
|
||||
],
|
||||
"cutoff": "copy: nextcloud_nextcloud_db_data.pre-update-20260924T062548Z\ndata\n",
|
||||
"api": {
|
||||
"state": "stopped",
|
||||
"update_phase": "failed",
|
||||
"hold_reason": "A(z) nextcloud frissítése nem sikerült, és az automatikus visszaállítás sem. Ezen a dobozon nincs olyan másolat, amely az alkalmazást a fájljaival együtt vissza tudná hozni. A Felhom ügyfélszolgálatát értesítettük — kérjük, addig ne indítsa újra és ne törölje az alkalmazást.",
|
||||
"hold_no_whole_copy": true,
|
||||
"update_error": "A(z) nextcloud frissítése nem sikerült, és az automatikus visszaállítás sem. Ezen a dobozon nincs olyan másolat, amely az alkalmazást a fájljaival együtt vissza tudná hozni. A Felhom ügyfélszolgálatát értesítettük — kérjük, addig ne indítsa újra és ne törölje az alkalmazást."
|
||||
},
|
||||
"app_page_hu": "A(z) nextcloud frissítése nem sikerült, és az automatikus visszaállítás sem. Ezen a dobozon nincs olyan másolat, amely az alkalmazást a fájljaival együtt vissza tudná hozni. A Felhom ügyfélszolgálatát értesítettük — kérjük, addig ne indítsa újra és ne törölje az alkalmazást.",
|
||||
"app_page_hu_restore_button": false,
|
||||
"stacks_page_hu_restore_button": false,
|
||||
"app_page_en": "The update of nextcloud did not work, and neither did the automatic undo. This box has no copy that can bring the app back together with its files. Felhom support has been told — until then, do not restart or remove the app.",
|
||||
"app_page_en_restore_button": false,
|
||||
"stacks_page_en_restore_button": false,
|
||||
"ascii_checks": {
|
||||
"hu has 'gyfelszolgalat' (positive)": true,
|
||||
"hu has 'Visszaallithato a Mentesek' (must be absent)": false,
|
||||
"en has 'Felhom support has been told' (positive)": true
|
||||
},
|
||||
"log": "2026/09/24 06:25:42 notifier.go:1259: [DEBUG] notifier disabled: dropped event crossdrive_completed (severity info)\n2026/09/24 06:26:42 notifier.go:1259: [DEBUG] notifier disabled: dropped event health_change (severity warning)\n2026/09/24 06:27:37 undo.go:501: [WARN] [stacks] update nextcloud: UNDO — putting back the previous version and its 3 volume copy(ies) (reason: not healthy: not healthy within 1m30s (last: state unhealthy))\n2026/09/24 06:27:40 update.go:937: [ERROR] [stacks] update nextcloud: the UNDO failed too (untouched) — HOLDING the app; the undo copies are kept: [{nextcloud_nextcloud_db_data nextcloud_nextcloud_db_data.pre-update-20260924T062548Z} {nextcloud_nextcloud_html nextcloud_nextcloud_html.pre-update-20260924T062548Z} {nextcloud_nextcloud_redis_data nextcloud_nextcloud_redis_data.pre-update-20260924T062548Z}]\n2026/09/24 06:27:40 update_guard.go:705: [ERROR] [backup] nextcloud is HELD STOPPED after a failed update and NO copy on this box brings it back whole (seen: [tier 2 at 2026-09-24T06:25:42Z tier 1 at 2026-09-24T06:25:43Z]; drive files declared: true; undo: \"untouched\") — support must act (R-659)\n2026/09/24 06:27:40 notifier.go:1255: [WARN] notifier disabled (no hub configured): DROPPED event app_update_held (severity error) — further app_update_held events are logged at DEBUG only\n2026/09/24 06:27:40 notifier.go:1255: [WARN] notifier disabled (no hub configured): DROPPED event app_hold_no_whole_copy (severity critical) — further app_hold_no_whole_copy events are logged at DEBUG only\n",
|
||||
"debug_ring_events": []
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
08:23:32 drill 6a41e807228f: nextcloud template = ladder step 1's own definition (34.0.1 / mariadb 12.3) push rc=0
|
||||
08:23:40 [1] made the drive paths this app requires: ['/mnt/felhom-drives/scratch_hdd/userdata/nextcloud']
|
||||
08:23:40 [1] required fields filled beyond DOMAIN/SUBDOMAIN: ['NEXTCLOUD_ADMIN_PASSWORD', 'HDD_PATH']
|
||||
08:23:41 [1] deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}
|
||||
08:25:06 [1] deployed, controller state=running, pinned={'nextcloud': 'nextcloud:34.0.1-apache', 'nextcloud-db': 'mariadb:12.3', 'nextcloud-redis': 'redis:7-alpine'}
|
||||
08:25:15 nextcloud: occ user:add :: The account "drill10a42a" was created successfully Display name set to "drill10a42a"
|
||||
08:25:18 nextcloud: seeded user drill10a42a
|
||||
08:25:19 drill a6d49ca9eabc: nextcloud back to the head (34.0.4) + probe port 8999 push rc=0
|
||||
08:25:24 steps-left line hu=[('1', 'Hátralévő frissítési lépések: 1')] en=[('1', 'Update steps remaining: 1')]
|
||||
08:25:24 Update -> 202
|
||||
08:25:24 + 0.0s phase=backing-up
|
||||
08:25:43 + 18.9s phase=safety-dump
|
||||
08:25:45 + 21.0s phase=pulling
|
||||
08:25:47 + 22.6s phase=copying
|
||||
08:26:01 + 36.8s phase=starting
|
||||
08:26:07 + 43.1s phase=verifying
|
||||
08:26:13 >>> finished-marker removed from one undo copy:
|
||||
copy: nextcloud_nextcloud_db_data.pre-update-20260924T062548Z
|
||||
data
|
||||
|
||||
08:27:37 + 133.4s phase=undoing
|
||||
08:27:40 + 136.0s phase=failed
|
||||
08:27:40 API: {"state": "stopped", "update_phase": "failed", "hold_reason": "A(z) nextcloud frissítése nem sikerült, és az automatikus visszaállítás sem. Ezen a dobozon nincs olyan másolat, amely az alkalmazást a fájljaival együtt vissza tudná hozni. A Felhom ügyfélszolgálatát értesítettük — kérjük, addig ne indítsa újra és ne törölje az alkalmazást.", "hold_no_whole_copy": true, "update_error": "A(z) nextcloud frissítése nem sikerült, és az automatikus visszaállítás sem. Ezen a dobozon nincs olyan másolat, amely az alkalmazást a fájljaival együtt vissza tudná hozni. A Felhom ügyfélszolgálatát értesítettük — kérjük, addig ne indítsa újra és ne törölje az alkalmazást."}
|
||||
08:27:40 [hu] app page held block: 'A(z) nextcloud frissítése nem sikerült, és az automatikus visszaállítás sem. Ezen a dobozon nincs olyan másolat, amely az alkalmazást a fájljaival együtt vissza tudná hozni. A Felhom ügyfélszolgálatát értesítettük — kérjük, addig ne indítsa újra és ne törölje az alkalmazást.' restore-button app=False list=False
|
||||
08:27:40 [en] app page held block: 'The update of nextcloud did not work, and neither did the automatic undo. This box has no copy that can bring the app back together with its files. Felhom support has been told — until then, do not restart or remove the app.' restore-button app=False list=False
|
||||
08:27:40 checks: {"hu has 'gyfelszolgalat' (positive)": true, "hu has 'Visszaallithato a Mentesek' (must be absent)": false, "en has 'Felhom support has been told' (positive)": true}
|
||||
08:30:13 controller log:
|
||||
2026/09/24 06:25:42 notifier.go:1259: [DEBUG] notifier disabled: dropped event crossdrive_completed (severity info)
|
||||
2026/09/24 06:26:42 notifier.go:1259: [DEBUG] notifier disabled: dropped event health_change (severity warning)
|
||||
2026/09/24 06:27:37 undo.go:501: [WARN] [stacks] update nextcloud: UNDO — putting back the previous version and its 3 volume copy(ies) (reason: not healthy: not healthy within 1m30s (last: state unhealthy))
|
||||
2026/09/24 06:27:40 update.go:937: [ERROR] [stacks] update nextcloud: the UNDO failed too (untouched) — HOLDING the app; the undo copies are kept: [{nextcloud_nextcloud_db_data nextcloud_nextcloud_db_data.pre-update-20260924T062548Z} {nextcloud_nextcloud_html nextcloud_nextcloud_html.pre-update-20260924T062548Z} {nextcloud_nextcloud_redis_data nextcloud_nextcloud_redis_data.pre-update-20260924T062548Z}]
|
||||
2026/09/24 06:27:40 update_guard.go:705: [ERROR] [backup] nextcloud is HELD STOPPED after a failed update and NO copy on this box brings it back whole (seen: [tier 2 at 2026-09-24T06:25:42Z tier 1 at 2026-09-24T06:25:43Z]; drive files declared: true; undo: "untouched") — support must act (R-659)
|
||||
2026/09/24 06:27:40 notifier.go:1255: [WARN] notifier disabled (no hub configured): DROPPED event app_update_held (severity error) — further app_update_held events are logged at DEBUG only
|
||||
2026/09/24 06:27:40 notifier.go:1255: [WARN] notifier disabled (no hub configured): DROPPED event app_hold_no_whole_copy (severity critical) — further app_hold_no_whole_copy events are logged at DEBUG only
|
||||
|
||||
08:30:14 debug ring (events since the press):
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
08:23:32 drill 6a41e807228f: nextcloud template = ladder step 1's own definition (34.0.1 / mariadb 12.3) push rc=0
|
||||
08:23:40 [1] made the drive paths this app requires: ['/mnt/felhom-drives/scratch_hdd/userdata/nextcloud']
|
||||
08:23:40 [1] required fields filled beyond DOMAIN/SUBDOMAIN: ['NEXTCLOUD_ADMIN_PASSWORD', 'HDD_PATH']
|
||||
08:23:41 [1] deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}
|
||||
08:25:06 [1] deployed, controller state=running, pinned={'nextcloud': 'nextcloud:34.0.1-apache', 'nextcloud-db': 'mariadb:12.3', 'nextcloud-redis': 'redis:7-alpine'}
|
||||
08:25:15 nextcloud: occ user:add :: The account "drill10a42a" was created successfully Display name set to "drill10a42a"
|
||||
08:25:18 nextcloud: seeded user drill10a42a
|
||||
08:25:19 drill a6d49ca9eabc: nextcloud back to the head (34.0.4) + probe port 8999 push rc=0
|
||||
08:25:24 steps-left line hu=[('1', 'Hátralévő frissítési lépések: 1')] en=[('1', 'Update steps remaining: 1')]
|
||||
08:25:24 Update -> 202
|
||||
08:25:24 + 0.0s phase=backing-up
|
||||
08:25:43 + 18.9s phase=safety-dump
|
||||
08:25:45 + 21.0s phase=pulling
|
||||
08:25:47 + 22.6s phase=copying
|
||||
08:26:01 + 36.8s phase=starting
|
||||
08:26:07 + 43.1s phase=verifying
|
||||
08:26:13 >>> finished-marker removed from one undo copy:
|
||||
copy: nextcloud_nextcloud_db_data.pre-update-20260924T062548Z
|
||||
data
|
||||
|
||||
08:27:37 + 133.4s phase=undoing
|
||||
08:27:40 + 136.0s phase=failed
|
||||
08:27:40 API: {"state": "stopped", "update_phase": "failed", "hold_reason": "A(z) nextcloud frissítése nem sikerült, és az automatikus visszaállítás sem. Ezen a dobozon nincs olyan másolat, amely az alkalmazást a fájljaival együtt vissza tudná hozni. A Felhom ügyfélszolgálatát értesítettük — kérjük, addig ne indítsa újra és ne törölje az alkalmazást.", "hold_no_whole_copy": true, "update_error": "A(z) nextcloud frissítése nem sikerült, és az automatikus visszaállítás sem. Ezen a dobozon nincs olyan másolat, amely az alkalmazást a fájljaival együtt vissza tudná hozni. A Felhom ügyfélszolgálatát értesítettük — kérjük, addig ne indítsa újra és ne törölje az alkalmazást."}
|
||||
08:27:40 [hu] app page held block: 'A(z) nextcloud frissítése nem sikerült, és az automatikus visszaállítás sem. Ezen a dobozon nincs olyan másolat, amely az alkalmazást a fájljaival együtt vissza tudná hozni. A Felhom ügyfélszolgálatát értesítettük — kérjük, addig ne indítsa újra és ne törölje az alkalmazást.' restore-button app=False list=False
|
||||
08:27:40 [en] app page held block: 'The update of nextcloud did not work, and neither did the automatic undo. This box has no copy that can bring the app back together with its files. Felhom support has been told — until then, do not restart or remove the app.' restore-button app=False list=False
|
||||
08:27:40 checks: {"hu has 'gyfelszolgalat' (positive)": true, "hu has 'Visszaallithato a Mentesek' (must be absent)": false, "en has 'Felhom support has been told' (positive)": true}
|
||||
08:30:13 controller log:
|
||||
2026/09/24 06:25:42 notifier.go:1259: [DEBUG] notifier disabled: dropped event crossdrive_completed (severity info)
|
||||
2026/09/24 06:26:42 notifier.go:1259: [DEBUG] notifier disabled: dropped event health_change (severity warning)
|
||||
2026/09/24 06:27:37 undo.go:501: [WARN] [stacks] update nextcloud: UNDO — putting back the previous version and its 3 volume copy(ies) (reason: not healthy: not healthy within 1m30s (last: state unhealthy))
|
||||
2026/09/24 06:27:40 update.go:937: [ERROR] [stacks] update nextcloud: the UNDO failed too (untouched) — HOLDING the app; the undo copies are kept: [{nextcloud_nextcloud_db_data nextcloud_nextcloud_db_data.pre-update-20260924T062548Z} {nextcloud_nextcloud_html nextcloud_nextcloud_html.pre-update-20260924T062548Z} {nextcloud_nextcloud_redis_data nextcloud_nextcloud_redis_data.pre-update-20260924T062548Z}]
|
||||
2026/09/24 06:27:40 update_guard.go:705: [ERROR] [backup] nextcloud is HELD STOPPED after a failed update and NO copy on this box brings it back whole (seen: [tier 2 at 2026-09-24T06:25:42Z tier 1 at 2026-09-24T06:25:43Z]; drive files declared: true; undo: "untouched") — support must act (R-659)
|
||||
2026/09/24 06:27:40 notifier.go:1255: [WARN] notifier disabled (no hub configured): DROPPED event app_update_held (severity error) — further app_update_held events are logged at DEBUG only
|
||||
2026/09/24 06:27:40 notifier.go:1255: [WARN] notifier disabled (no hub configured): DROPPED event app_hold_no_whole_copy (severity critical) — further app_hold_no_whole_copy events are logged at DEBUG only
|
||||
|
||||
08:30:14 debug ring (events since the press):
|
||||
|
||||
@@ -0,0 +1,27 @@
|
||||
2026/09/24 06:23:41 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
|
||||
2026/09/24 06:24:11 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
|
||||
2026/09/24 06:24:41 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
|
||||
2026/09/24 06:25:11 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
|
||||
2026/09/24 06:25:41 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
|
||||
2026/09/24 06:26:11 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
|
||||
2026/09/24 06:26:41 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
|
||||
2026/09/24 06:27:11 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
|
||||
2026/09/24 06:27:41 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
|
||||
2026/09/24 06:27:42 main.go:1951: [INFO] [deadapp] check alive: 20 scans since boot, 4 deployed app(s) evaluated, 0 currently down
|
||||
2026/09/24 06:28:11 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
|
||||
2026/09/24 06:28:41 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
|
||||
2026/09/24 06:29:11 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
|
||||
2026/09/24 06:29:41 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
|
||||
2026/09/24 06:30:11 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
|
||||
|
||||
200 ['entries', 'total']
|
||||
200 {'timestamp': '2026-09-24T06:27:41Z', 'level': 'DEBUG', 'message': '[stacks] RunHealthProbes: collected 0 targets (2 skipped not due, 0 skipped no container)', 'source': 'healthprobe.go:104'}
|
||||
{"timestamp": "2026-09-24T06:27:42Z", "level": "INFO", "message": "[deadapp] check alive: 20 scans since boot, 4 deployed app(s) evaluated, 0 currently down", "source": "main.go:1951"}
|
||||
{"timestamp": "2026-09-24T06:28:11Z", "level": "DEBUG", "message": "[scheduler] job deadapp-check: execution starting", "source": "scheduler.go:67"}
|
||||
{"timestamp": "2026-09-24T06:28:41Z", "level": "DEBUG", "message": "[scheduler] job deadapp-check: execution starting", "source": "scheduler.go:67"}
|
||||
{"timestamp": "2026-09-24T06:29:11Z", "level": "DEBUG", "message": "[scheduler] job deadapp-check: execution starting", "source": "scheduler.go:67"}
|
||||
{"timestamp": "2026-09-24T06:29:41Z", "level": "DEBUG", "message": "[scheduler] job deadapp-check: execution starting", "source": "scheduler.go:67"}
|
||||
{"timestamp": "2026-09-24T06:30:11Z", "level": "DEBUG", "message": "[scheduler] job deadapp-check: execution starting", "source": "scheduler.go:67"}
|
||||
2026/09/24 06:27:42 main.go:1951: [INFO] [deadapp] check alive: 20 scans since boot, 4 deployed app(s) evaluated, 0 currently down
|
||||
gokapi Restarting (1) 28 seconds ago
|
||||
|
||||
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"events": [
|
||||
"2026-09-24T06:31:41Z notifier disabled: dropped event app_start_failed (severity warning)",
|
||||
"2026-09-24T06:31:41Z notifier disabled: dropped event app_start_failed (severity warning)"
|
||||
],
|
||||
"log": "2026/09/24 06:31:11 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting\n2026/09/24 06:31:41 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting\n2026/09/24 06:31:41 notifier.go:1259: [DEBUG] notifier disabled: dropped event app_start_failed (severity warning)\n2026/09/24 06:31:41 notifier.go:1259: [DEBUG] notifier disabled: dropped event app_start_failed (severity warning)\n",
|
||||
"nextcloud_state": "stopped"
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
08:31:02 [1] deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}
|
||||
08:31:12 [1] deployed, controller state=running, pinned={'actualbudget': 'actualbudget/actual-server:26.9.0'}
|
||||
08:31:16 stopping the throwaway's container out of band (not through the product): actualbudget
|
||||
08:31:46 events/heartbeats since the control started:
|
||||
2026-09-24T06:31:41Z notifier disabled: dropped event app_start_failed (severity warning)
|
||||
2026-09-24T06:31:41Z notifier disabled: dropped event app_start_failed (severity warning)
|
||||
08:31:49 controller log lines:
|
||||
2026/09/24 06:31:11 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
|
||||
2026/09/24 06:31:41 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
|
||||
2026/09/24 06:31:41 notifier.go:1259: [DEBUG] notifier disabled: dropped event app_start_failed (severity warning)
|
||||
2026/09/24 06:31:41 notifier.go:1259: [DEBUG] notifier disabled: dropped event app_start_failed (severity warning)
|
||||
|
||||
08:31:49 nextcloud meanwhile: state=stopped held=True
|
||||
08:31:49 [X] stop -> 200 {'ok': True, 'message': 'Stack actualbudget stop completed'}
|
||||
08:32:21 [X] remove (with drive data) -> 200 {'ok': True, 'data': {'removed': 'actualbudget', 'volumes_removed': ['actualbudget_actualbudget_data'], 'hdd_paths_removed': [], 'hdd_paths_preserved': [], 'hdd
|
||||
08:32:30 [X] after remove: deployed=False leftovers='/opt/docker/stacks/actualbudget'
|
||||
@@ -0,0 +1,5 @@
|
||||
gokapi restarting running
|
||||
nextcloud stopped held running
|
||||
paperless-ngx running running
|
||||
privatebin running stopped
|
||||
[]
|
||||
@@ -0,0 +1,2 @@
|
||||
2026-09-24T06:34:42Z [stacks] ScanStacks: found stack "gokapi" deployed=true composePath=/opt/docker/stacks/gokapi/docker-compose.yml
|
||||
2026-09-24T06:34:42Z [stacks] ScanStacks: found stack "nextcloud" deployed=true composePath=/opt/docker/stacks/nextcloud/docker-compose.yml
|
||||
@@ -0,0 +1,128 @@
|
||||
[
|
||||
{
|
||||
"t": 10,
|
||||
"banner": [],
|
||||
"events": [
|
||||
"2026-09-24T06:37:11Z notifier disabled: dropped event app_start_failed (severity warning)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"t": 15,
|
||||
"banner": [],
|
||||
"events": [
|
||||
"2026-09-24T06:37:11Z notifier disabled: dropped event app_start_failed (severity warning)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"t": 20,
|
||||
"banner": [],
|
||||
"events": [
|
||||
"2026-09-24T06:37:11Z notifier disabled: dropped event app_start_failed (severity warning)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"t": 25,
|
||||
"banner": [],
|
||||
"events": [
|
||||
"2026-09-24T06:37:11Z notifier disabled: dropped event app_start_failed (severity warning)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"t": 30,
|
||||
"banner": [],
|
||||
"events": [
|
||||
"2026-09-24T06:37:11Z notifier disabled: dropped event app_start_failed (severity warning)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"t": 35,
|
||||
"banner": [],
|
||||
"events": [
|
||||
"2026-09-24T06:37:11Z notifier disabled: dropped event app_start_failed (severity warning)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"t": 40,
|
||||
"banner": [],
|
||||
"events": [
|
||||
"2026-09-24T06:37:11Z notifier disabled: dropped event app_start_failed (severity warning)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"t": 45,
|
||||
"banner": [],
|
||||
"events": [
|
||||
"2026-09-24T06:37:11Z notifier disabled: dropped event app_start_failed (severity warning)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"t": 50,
|
||||
"banner": [],
|
||||
"events": [
|
||||
"2026-09-24T06:37:11Z notifier disabled: dropped event app_start_failed (severity warning)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"t": 55,
|
||||
"banner": [],
|
||||
"events": [
|
||||
"2026-09-24T06:37:11Z notifier disabled: dropped event app_start_failed (severity warning)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"t": 60,
|
||||
"banner": [],
|
||||
"events": [
|
||||
"2026-09-24T06:37:11Z notifier disabled: dropped event app_start_failed (severity warning)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"t": 65,
|
||||
"banner": [],
|
||||
"events": [
|
||||
"2026-09-24T06:37:11Z notifier disabled: dropped event app_start_failed (severity warning)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"t": 70,
|
||||
"banner": [],
|
||||
"events": [
|
||||
"2026-09-24T06:37:11Z notifier disabled: dropped event app_start_failed (severity warning)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"t": 75,
|
||||
"banner": [],
|
||||
"events": [
|
||||
"2026-09-24T06:37:11Z notifier disabled: dropped event app_start_failed (severity warning)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"t": 80,
|
||||
"banner": [],
|
||||
"events": [
|
||||
"2026-09-24T06:37:11Z notifier disabled: dropped event app_start_failed (severity warning)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"t": 85,
|
||||
"banner": [],
|
||||
"events": [
|
||||
"2026-09-24T06:37:11Z notifier disabled: dropped event app_start_failed (severity warning)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"t": 90,
|
||||
"banner": [],
|
||||
"events": [
|
||||
"2026-09-24T06:37:11Z notifier disabled: dropped event app_start_failed (severity warning)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"t": 95,
|
||||
"banner": [],
|
||||
"events": [
|
||||
"2026-09-24T06:37:11Z notifier disabled: dropped event app_start_failed (severity warning)"
|
||||
]
|
||||
}
|
||||
]
|
||||
@@ -0,0 +1,30 @@
|
||||
08:39:36 before remove:
|
||||
nextcloud_nextcloud_db_data
|
||||
nextcloud_nextcloud_db_data.pre-update-20260924T062548Z
|
||||
nextcloud_nextcloud_html
|
||||
nextcloud_nextcloud_html.pre-update-20260924T062548Z
|
||||
nextcloud_nextcloud_redis_data
|
||||
nextcloud_nextcloud_redis_data.pre-update-20260924T062548Z
|
||||
app.yaml
|
||||
applied-compose.yml
|
||||
applied-meta
|
||||
docker-compose.yml
|
||||
hold-logs
|
||||
|
||||
08:39:36 [X] stop -> 200 {'ok': True, 'message': 'Stack nextcloud stop completed'}
|
||||
08:39:41 [X] remove (with drive data) -> 409 {'ok': False, 'error': 'A(z) /mnt/felhom-drives/scratch_hdd/userdata/nextcloud tárhely jelenleg nem elérhető — az alkalmazás nem távolítható el, amíg a meghajtó
|
||||
08:39:41 [X] refused because the drive path cannot be resolved (R-442, fail-closed and right) — removing the app and KEEPING the drive data instead
|
||||
08:40:10 [X] remove (keeping drive data) -> 200 {'ok': True, 'data': {'removed': 'nextcloud', 'volumes_removed': ['nextcloud_nextcloud_db_data', 'nextcloud_nextcloud_html', 'nextcloud_nextcloud_redis_data'],
|
||||
08:40:18 [X] after remove: deployed=False leftovers='/opt/docker/stacks/nextcloud'
|
||||
08:40:21 after remove:
|
||||
no nextcloud volumes (incl. undo copies)
|
||||
total 32
|
||||
drwxr-xr-x 3 root root 4096 Sep 24 06:40 .
|
||||
drwxr-xr-x 57 root root 4096 Sep 13 20:22 ..
|
||||
-rw-r--r-- 1 root root 9182 Sep 24 06:25 .felhom.yml
|
||||
-rw-r--r-- 1 root root 4593 Sep 24 06:25 docker-compose.yml
|
||||
drwxr-xr-x 4 root root 4096 Sep 24 06:27 hold-logs
|
||||
ls: cannot access '/mnt/felhom-drives/scratch_hdd/appdata/nextcloud': No such file or directory
|
||||
/mnt/felhom-drives/scratch_hdd/userdata/nextcloud
|
||||
|
||||
08:40:24 drive folders removed by name (R-442 keeps them on 9202): ls: cannot access '/mnt/felhom-drives/scratch_hdd/userdata/nextcloud': No such file or directory
|
||||
@@ -0,0 +1,10 @@
|
||||
git:
|
||||
branch: main
|
||||
repo_url: https://gitea.dooplex.hu/admin/app-catalog-drill.git
|
||||
sync_interval: 15m
|
||||
token: <redacted>
|
||||
username: "admin"
|
||||
hub:
|
||||
update:
|
||||
health_timeout: 90s
|
||||
|
||||
@@ -0,0 +1,110 @@
|
||||
{
|
||||
"deployed_A": true,
|
||||
"A": {
|
||||
"pinned_images": {
|
||||
"vikunja": "vikunja/vikunja:2.4.0"
|
||||
},
|
||||
"installed_images": {
|
||||
"vikunja": "vikunja/vikunja:2.4.0"
|
||||
},
|
||||
"catalog_images": null,
|
||||
"live_compose_image_lines": [
|
||||
"image: vikunja/vikunja:2.4.0"
|
||||
],
|
||||
"docker_inspect": [
|
||||
"vikunja vikunja/vikunja:2.4.0 running=true restarts=0"
|
||||
]
|
||||
},
|
||||
"badge_wait_s": 10.5,
|
||||
"cache_steps": "ls: cannot access '/var/lib/docker/volumes/felhom-controller-data/_data/catalog-cache/templates/vikunja/': No such file or directory\nls: cannot access '/var/lib/docker/volumes/felhom-controller-data/_data/catalog-cache/templates/vikunja/steps/': No such file or directory\napp.yaml\napplied-compose.yml\napplied-meta\ndocker-compose.yml\nhold-logs\n",
|
||||
"badges": {
|
||||
"hu": [
|
||||
{
|
||||
"title": "Újabb változat érhető el ehhez az alkalmazáshoz. A frissítés indításához nyomd meg a Frissítés gombot.",
|
||||
"text": "Frissítés elérhető — 2 napja"
|
||||
}
|
||||
],
|
||||
"en": [
|
||||
{
|
||||
"title": "A newer version of this app is available. Select the Update button to start it.",
|
||||
"text": "Update available — 2 days ago"
|
||||
}
|
||||
]
|
||||
},
|
||||
"press": {
|
||||
"accepted": true,
|
||||
"http": "202",
|
||||
"phases": [
|
||||
{
|
||||
"t": 0.0,
|
||||
"phase": "backing-up",
|
||||
"label": "Biztonsági mentés készül a frissítés előtt…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 2.1,
|
||||
"phase": "safety-dump",
|
||||
"label": "Adatbázis pillanatkép…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 3.1,
|
||||
"phase": "pulling",
|
||||
"label": "Új verzió letöltése…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 4.1,
|
||||
"phase": "copying",
|
||||
"label": "Az adatok másolása a frissítés előtt…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 12.3,
|
||||
"phase": "verifying",
|
||||
"label": "Működés ellenőrzése…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 32.8,
|
||||
"phase": "done",
|
||||
"label": "Frissítve",
|
||||
"updating": false,
|
||||
"error": null,
|
||||
"hold": null
|
||||
}
|
||||
],
|
||||
"duration_s": 32.9,
|
||||
"final_phase": "done",
|
||||
"update_error": null,
|
||||
"hold_reason": null,
|
||||
"state": "running"
|
||||
},
|
||||
"after": {
|
||||
"pinned_images": {
|
||||
"vikunja": "vikunja/vikunja:2.6.0"
|
||||
},
|
||||
"installed_images": {
|
||||
"vikunja": "vikunja/vikunja:2.6.0"
|
||||
},
|
||||
"catalog_images": {
|
||||
"vikunja": "vikunja/vikunja:2.6.0"
|
||||
},
|
||||
"live_compose_image_lines": [
|
||||
"image: vikunja/vikunja:2.6.0"
|
||||
],
|
||||
"docker_inspect": [
|
||||
"vikunja vikunja/vikunja:2.6.0 running=true restarts=0"
|
||||
]
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
07:44:29 drill commit 15a4107ef0c0: SPIKE vikunja A=2.4.0 (push rc=0)
|
||||
07:45:48 [sync] the badge NEVER caught up to vikunja/vikunja:2.4.0 in 78.9s — catalog_images = None
|
||||
07:45:48 [1] deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}
|
||||
07:45:58 [1] deployed, controller state=running, pinned={'vikunja': 'vikunja/vikunja:2.4.0'}
|
||||
07:46:03 drill commit 4d557385d146: SPIKE vikunja B=2.5.0 + steps/spike-B.yml (push rc=0)
|
||||
07:46:03 drill commit 1cb96ca718dd: SPIKE vikunja C=2.6.0 (push rc=0)
|
||||
07:46:14 [sync] the badge needed 10.5s and 2 sync+rescan rounds to catch up to vikunja/vikunja:2.6.0 — R-607's window, measured
|
||||
07:46:18 cache:
|
||||
ls: cannot access '/var/lib/docker/volumes/felhom-controller-data/_data/catalog-cache/templates/vikunja/': No such file or directory
|
||||
ls: cannot access '/var/lib/docker/volumes/felhom-controller-data/_data/catalog-cache/templates/vikunja/steps/': No such file or directory
|
||||
app.yaml
|
||||
applied-compose.yml
|
||||
applied-meta
|
||||
docker-compose.yml
|
||||
hold-logs
|
||||
|
||||
07:46:18 [6] Update -> 202 {'ok': True, 'data': {'accepted': True, 'completed': False}, 'message': 'Frissítés elindult – az állapot a kártyán követhető'}
|
||||
07:46:18 + 0.0s phase=backing-up label=Biztonsági mentés készül a frissítés előtt… err=None hold=None
|
||||
07:46:20 + 2.1s phase=safety-dump label=Adatbázis pillanatkép… err=None hold=None
|
||||
07:46:21 + 3.1s phase=pulling label=Új verzió letöltése… err=None hold=None
|
||||
07:46:22 + 4.1s phase=copying label=Az adatok másolása a frissítés előtt… err=None hold=None
|
||||
07:46:30 + 12.3s phase=verifying label=Működés ellenőrzése… err=None hold=None
|
||||
07:46:51 + 32.8s phase=done label=Frissítve err=None hold=None
|
||||
07:46:54 after: {"pinned_images": {"vikunja": "vikunja/vikunja:2.6.0"}, "installed_images": {"vikunja": "vikunja/vikunja:2.6.0"}, "catalog_images": {"vikunja": "vikunja/vikunja:2.6.0"}, "live_compose_image_lines": ["image: vikunja/vikunja:2.6.0"], "docker_inspect": ["vikunja vikunja/vikunja:2.6.0 running=true restarts=0"]}
|
||||
07:46:55 [X] stop -> 200 {'ok': True, 'message': 'Stack vikunja stop completed'}
|
||||
07:47:26 [X] remove (with drive data) -> 200 {'ok': True, 'data': {'removed': 'vikunja', 'volumes_removed': ['vikunja_vikunja_data', 'vikunja_vikunja_db'], 'hdd_paths_removed': [], 'hdd_paths_preserved': [
|
||||
07:47:35 [X] after remove: deployed=False leftovers='/opt/docker/stacks/vikunja'
|
||||
@@ -0,0 +1,27 @@
|
||||
07:44:29 drill commit 15a4107ef0c0: SPIKE vikunja A=2.4.0 (push rc=0)
|
||||
07:45:48 [sync] the badge NEVER caught up to vikunja/vikunja:2.4.0 in 78.9s — catalog_images = None
|
||||
07:45:48 [1] deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}
|
||||
07:45:58 [1] deployed, controller state=running, pinned={'vikunja': 'vikunja/vikunja:2.4.0'}
|
||||
07:46:03 drill commit 4d557385d146: SPIKE vikunja B=2.5.0 + steps/spike-B.yml (push rc=0)
|
||||
07:46:03 drill commit 1cb96ca718dd: SPIKE vikunja C=2.6.0 (push rc=0)
|
||||
07:46:14 [sync] the badge needed 10.5s and 2 sync+rescan rounds to catch up to vikunja/vikunja:2.6.0 — R-607's window, measured
|
||||
07:46:18 cache:
|
||||
ls: cannot access '/var/lib/docker/volumes/felhom-controller-data/_data/catalog-cache/templates/vikunja/': No such file or directory
|
||||
ls: cannot access '/var/lib/docker/volumes/felhom-controller-data/_data/catalog-cache/templates/vikunja/steps/': No such file or directory
|
||||
app.yaml
|
||||
applied-compose.yml
|
||||
applied-meta
|
||||
docker-compose.yml
|
||||
hold-logs
|
||||
|
||||
07:46:18 [6] Update -> 202 {'ok': True, 'data': {'accepted': True, 'completed': False}, 'message': 'Frissítés elindult – az állapot a kártyán követhető'}
|
||||
07:46:18 + 0.0s phase=backing-up label=Biztonsági mentés készül a frissítés előtt… err=None hold=None
|
||||
07:46:20 + 2.1s phase=safety-dump label=Adatbázis pillanatkép… err=None hold=None
|
||||
07:46:21 + 3.1s phase=pulling label=Új verzió letöltése… err=None hold=None
|
||||
07:46:22 + 4.1s phase=copying label=Az adatok másolása a frissítés előtt… err=None hold=None
|
||||
07:46:30 + 12.3s phase=verifying label=Működés ellenőrzése… err=None hold=None
|
||||
07:46:51 + 32.8s phase=done label=Frissítve err=None hold=None
|
||||
07:46:54 after: {"pinned_images": {"vikunja": "vikunja/vikunja:2.6.0"}, "installed_images": {"vikunja": "vikunja/vikunja:2.6.0"}, "catalog_images": {"vikunja": "vikunja/vikunja:2.6.0"}, "live_compose_image_lines": ["image: vikunja/vikunja:2.6.0"], "docker_inspect": ["vikunja vikunja/vikunja:2.6.0 running=true restarts=0"]}
|
||||
07:46:55 [X] stop -> 200 {'ok': True, 'message': 'Stack vikunja stop completed'}
|
||||
07:47:26 [X] remove (with drive data) -> 200 {'ok': True, 'data': {'removed': 'vikunja', 'volumes_removed': ['vikunja_vikunja_data', 'vikunja_vikunja_db'], 'hdd_paths_removed': [], 'hdd_paths_preserved': [
|
||||
07:47:35 [X] after remove: deployed=False leftovers='/opt/docker/stacks/vikunja'
|
||||
@@ -0,0 +1,25 @@
|
||||
1cb96ca718dd
|
||||
1
|
||||
true
|
||||
/var/lib/docker/volumes/felhom-controller-data/_data/data/catalog-cache/templates/vikunja/:
|
||||
total 24
|
||||
drwxr-xr-x 3 root root 4096 Sep 24 05:46 .
|
||||
drwxr-xr-x 55 root root 4096 Sep 24 05:44 ..
|
||||
-rw-r--r-- 1 root root 4419 Sep 24 05:44 .felhom.yml
|
||||
-rw-r--r-- 1 root root 1280 Sep 24 05:46 docker-compose.yml
|
||||
drwxr-xr-x 2 root root 4096 Sep 24 05:46 steps
|
||||
|
||||
/var/lib/docker/volumes/felhom-controller-data/_data/data/catalog-cache/templates/vikunja/steps/:
|
||||
total 12
|
||||
drwxr-xr-x 2 root root 4096 Sep 24 05:46 .
|
||||
drwxr-xr-x 3 root root 4096 Sep 24 05:46 ..
|
||||
-rw-r--r-- 1 root root 1280 Sep 24 05:46 spike-B.yml
|
||||
total 32
|
||||
drwxr-xr-x 4 root root 4096 Sep 24 05:47 .
|
||||
drwxr-xr-x 57 root root 4096 Sep 13 20:22 ..
|
||||
-rw-r--r-- 1 root root 4419 Sep 23 22:15 .felhom.yml
|
||||
-rw-r--r-- 1 root root 1280 Sep 24 05:46 applied-compose.yml
|
||||
drwxr-xr-x 2 root root 4096 Sep 24 05:46 applied-meta
|
||||
-rw-r--r-- 1 root root 1280 Sep 24 05:46 docker-compose.yml
|
||||
drwxr-xr-x 11 root root 4096 Sep 23 21:05 hold-logs
|
||||
|
||||
@@ -0,0 +1,295 @@
|
||||
{
|
||||
"controller": "gitea.dooplex.hu/admin/felhom-controller:0.268.0",
|
||||
"drill_install": "95d83b7ab698",
|
||||
"deployed": true,
|
||||
"seed_A": true,
|
||||
"drill_head": "0e4e12aed1f2",
|
||||
"badge_wait": 4.4,
|
||||
"before": {
|
||||
"obs": {
|
||||
"pinned_images": {
|
||||
"romm": "rommapp/romm:5.3.0",
|
||||
"romm-db": "mariadb:11.4",
|
||||
"romm-redis": "redis:7-alpine"
|
||||
},
|
||||
"installed_images": {
|
||||
"romm": "rommapp/romm:5.3.0",
|
||||
"romm-db": "mariadb:11.4",
|
||||
"romm-redis": "redis:7-alpine"
|
||||
},
|
||||
"catalog_images": {
|
||||
"romm": "rommapp/romm:5.3.1",
|
||||
"romm-db": "mariadb:11.8",
|
||||
"romm-redis": "redis:7-alpine"
|
||||
},
|
||||
"live_compose_image_lines": [
|
||||
"image: rommapp/romm:5.3.0",
|
||||
"image: mariadb:11.4",
|
||||
"image: redis:7-alpine"
|
||||
],
|
||||
"docker_inspect": [
|
||||
"romm rommapp/romm:5.3.0 running=true restarts=0",
|
||||
"romm-redis redis:7-alpine running=true restarts=0",
|
||||
"romm-db mariadb:11.4 running=true restarts=0"
|
||||
]
|
||||
},
|
||||
"steps_line": {
|
||||
"hu": [
|
||||
[
|
||||
"2",
|
||||
"Hátralévő frissítési lépések: 2"
|
||||
]
|
||||
],
|
||||
"en": [
|
||||
[
|
||||
"2",
|
||||
"Update steps remaining: 2"
|
||||
]
|
||||
]
|
||||
},
|
||||
"badges": {
|
||||
"hu": [
|
||||
{
|
||||
"title": "Újabb változat érhető el ehhez az alkalmazáshoz. A frissítés indításához nyomd meg a Frissítés gombot.",
|
||||
"text": "Frissítés elérhető — 1 napja"
|
||||
}
|
||||
],
|
||||
"en": [
|
||||
{
|
||||
"title": "A newer version of this app is available. Select the Update button to start it.",
|
||||
"text": "Update available — 1 day ago"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"press1": {
|
||||
"press": {
|
||||
"accepted": true,
|
||||
"http": "202",
|
||||
"phases": [
|
||||
{
|
||||
"t": 0.0,
|
||||
"phase": "safety-dump",
|
||||
"label": "Adatbázis pillanatkép…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 2.1,
|
||||
"phase": "pulling",
|
||||
"label": "Új verzió letöltése…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 3.1,
|
||||
"phase": "copying",
|
||||
"label": "Az adatok másolása a frissítés előtt…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 10.3,
|
||||
"phase": "starting",
|
||||
"label": "Indítás az új verzióval…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 21.6,
|
||||
"phase": "verifying",
|
||||
"label": "Működés ellenőrzése…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 57.4,
|
||||
"phase": "done",
|
||||
"label": "Frissítve",
|
||||
"updating": false,
|
||||
"error": null,
|
||||
"hold": null
|
||||
}
|
||||
],
|
||||
"duration_s": 57.5,
|
||||
"final_phase": "done",
|
||||
"update_error": null,
|
||||
"hold_reason": null,
|
||||
"state": "running"
|
||||
},
|
||||
"obs": {
|
||||
"pinned_images": {
|
||||
"romm": "rommapp/romm:5.3.1",
|
||||
"romm-db": "mariadb:11.4",
|
||||
"romm-redis": "redis:7-alpine"
|
||||
},
|
||||
"installed_images": {
|
||||
"romm": "rommapp/romm:5.3.1",
|
||||
"romm-db": "mariadb:11.4",
|
||||
"romm-redis": "redis:7-alpine"
|
||||
},
|
||||
"catalog_images": {
|
||||
"romm": "rommapp/romm:5.3.1",
|
||||
"romm-db": "mariadb:11.8",
|
||||
"romm-redis": "redis:7-alpine"
|
||||
},
|
||||
"live_compose_image_lines": [
|
||||
"image: rommapp/romm:5.3.1",
|
||||
"image: mariadb:11.4",
|
||||
"image: redis:7-alpine"
|
||||
],
|
||||
"docker_inspect": [
|
||||
"romm rommapp/romm:5.3.1 running=true restarts=0",
|
||||
"romm-redis redis:7-alpine running=true restarts=0",
|
||||
"romm-db mariadb:11.4 running=true restarts=0"
|
||||
]
|
||||
},
|
||||
"readback_A": true,
|
||||
"log": "2026/09/24 06:42:31 update.go:722: [INFO] [stacks] update romm: ladder — step 2 of 3: romm=rommapp/romm:5.3.0, romm-db=mariadb:11.4, romm-redis=redis:7-alpine → romm=rommapp/romm:5.3.1, romm-db=mariadb:11.4, romm-redis=redis:7-alpine, from 90dd9d68258286ef.yml\n2026/09/24 06:42:33 pin.go:370: [INFO] [stacks] update romm: pin advanced to the catalog's current definition (romm=rommapp/romm:5.3.1, romm-db=mariadb:11.4, romm-redis=redis:7-alpine)\n2026/09/24 06:43:28 update.go:884: [INFO] [stacks] update romm: healthy after 35s (the app's health check passed)\n2026/09/24 06:43:28 update.go:892: [INFO] [stacks] update romm: DONE in 57s\n",
|
||||
"db_log": "2026-09-24 08:42:42+02:00 [Note] [Entrypoint]: MariaDB upgrade not required\nVersion: '11.4.13-MariaDB-ubu2404' socket: '/run/mysqld/mysqld.sock' port: 3306 mariadb.org binary distribution\n",
|
||||
"files": "26: image: rommapp/romm:5.3.1\n86: memory: 768M\n102: image: mariadb:11.4\n123: memory: 384M\n132: image: redis:7-alpine\n145: memory: 128M\n---\napplied == steps/90dd9d68258286ef.yml\napplied != template\n",
|
||||
"steps_line": {
|
||||
"hu": [
|
||||
[
|
||||
"1",
|
||||
"Hátralévő frissítési lépések: 1"
|
||||
]
|
||||
],
|
||||
"en": [
|
||||
[
|
||||
"1",
|
||||
"Update steps remaining: 1"
|
||||
]
|
||||
]
|
||||
},
|
||||
"badges": {
|
||||
"hu": [
|
||||
{
|
||||
"title": "Újabb változat érhető el ehhez az alkalmazáshoz. A frissítés indításához nyomd meg a Frissítés gombot.",
|
||||
"text": "Frissítés elérhető — 1 napja"
|
||||
}
|
||||
],
|
||||
"en": [
|
||||
{
|
||||
"title": "A newer version of this app is available. Select the Update button to start it.",
|
||||
"text": "Update available — 1 day ago"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"press2": {
|
||||
"press": {
|
||||
"accepted": true,
|
||||
"http": "202",
|
||||
"phases": [
|
||||
{
|
||||
"t": 0.0,
|
||||
"phase": "safety-dump",
|
||||
"label": "Adatbázis pillanatkép…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 2.1,
|
||||
"phase": "pulling",
|
||||
"label": "Új verzió letöltése…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 3.1,
|
||||
"phase": "copying",
|
||||
"label": "Az adatok másolása a frissítés előtt…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 15.4,
|
||||
"phase": "starting",
|
||||
"label": "Indítás az új verzióval…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 31.8,
|
||||
"phase": "verifying",
|
||||
"label": "Működés ellenőrzése…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 67.7,
|
||||
"phase": "done",
|
||||
"label": "Frissítve",
|
||||
"updating": false,
|
||||
"error": null,
|
||||
"hold": null
|
||||
}
|
||||
],
|
||||
"duration_s": 67.7,
|
||||
"final_phase": "done",
|
||||
"update_error": null,
|
||||
"hold_reason": null,
|
||||
"state": "running"
|
||||
},
|
||||
"obs": {
|
||||
"pinned_images": {
|
||||
"romm": "rommapp/romm:5.3.1",
|
||||
"romm-db": "mariadb:11.8",
|
||||
"romm-redis": "redis:7-alpine"
|
||||
},
|
||||
"installed_images": {
|
||||
"romm": "rommapp/romm:5.3.1",
|
||||
"romm-db": "mariadb:11.8",
|
||||
"romm-redis": "redis:7-alpine"
|
||||
},
|
||||
"catalog_images": {
|
||||
"romm": "rommapp/romm:5.3.1",
|
||||
"romm-db": "mariadb:11.8",
|
||||
"romm-redis": "redis:7-alpine"
|
||||
},
|
||||
"live_compose_image_lines": [
|
||||
"image: rommapp/romm:5.3.1",
|
||||
"image: mariadb:11.8",
|
||||
"image: redis:7-alpine"
|
||||
],
|
||||
"docker_inspect": [
|
||||
"romm-db mariadb:11.8 running=true restarts=0",
|
||||
"romm rommapp/romm:5.3.1 running=true restarts=0",
|
||||
"romm-redis redis:7-alpine running=true restarts=0"
|
||||
]
|
||||
},
|
||||
"readback_A": true,
|
||||
"log": "2026/09/24 06:43:46 update.go:722: [INFO] [stacks] update romm: ladder — the last step (3 of 3) — the catalog's current definition\n2026/09/24 06:43:48 pin.go:370: [INFO] [stacks] update romm: pin advanced to the catalog's current definition (romm=rommapp/romm:5.3.1, romm-db=mariadb:11.8, romm-redis=redis:7-alpine)\n2026/09/24 06:44:53 update.go:884: [INFO] [stacks] update romm: healthy after 35s (the app's health check passed)\n2026/09/24 06:44:53 update.go:892: [INFO] [stacks] update romm: DONE in 1m7s\n",
|
||||
"db_log": "Version: '11.8.9-MariaDB-ubu2404' socket: '/run/mysqld/mysqld.sock' port: 0 mariadb.org binary distribution\n2026-09-24 08:44:05+02:00 [Note] [Entrypoint]: Starting mariadb-upgrade\nThe --upgrade-system-tables option was used, user tables won't be touched.\nMajor version upgrade detected from 11.4.13-MariaDB to 11.8.9-MariaDB. Check required!\n2026-09-24 08:44:12+02:00 [Note] [Entrypoint]: Finished mariadb-upgrade\nVersion: '11.8.9-MariaDB-ubu2404' socket: '/run/mysqld/mysqld.sock' port: 3306 mariadb.org binary distribution\n",
|
||||
"files": "26: image: rommapp/romm:5.3.1\n86: memory: 768M\n102: image: mariadb:11.8\n123: memory: 384M\n132: image: redis:7-alpine\n145: memory: 128M\n---\napplied != steps/90dd…\napplied == template\n",
|
||||
"steps_line": {
|
||||
"hu": [],
|
||||
"en": []
|
||||
},
|
||||
"badges": {
|
||||
"hu": [
|
||||
{
|
||||
"title": "Ez az alkalmazás a legfrissebb elérhető változatot futtatja.",
|
||||
"text": "Naprakész"
|
||||
}
|
||||
],
|
||||
"en": [
|
||||
{
|
||||
"title": "This app is running the newest version available.",
|
||||
"text": "Up to date"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,70 @@
|
||||
08:41:05 drill 95d83b7ab698: romm template = step 1's own definition (5.3.0 / mariadb 11.4) for the install push rc=0
|
||||
08:41:12 [1] made the drive paths this app requires: ['/mnt/felhom-drives/scratch_hdd/userdata/romm']
|
||||
08:41:12 [1] required fields filled beyond DOMAIN/SUBDOMAIN: ['HDD_PATH']
|
||||
08:41:12 [1] deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}
|
||||
08:42:13 [1] deployed, controller state=running, pinned={'romm': 'rommapp/romm:5.3.0', 'romm-db': 'mariadb:11.4', 'romm-redis': 'redis:7-alpine'}
|
||||
08:42:22 romm: POST /api/users http=201
|
||||
08:42:23 drill 0e4e12aed1f2: romm back to the head (5.3.1 / mariadb 11.8) push rc=0
|
||||
08:42:31 before: steps line {"hu": [["2", "Hátralévő frissítési lépések: 2"]], "en": [["2", "Update steps remaining: 2"]]} badges {"hu": [{"title": "Újabb változat érhető el ehhez az alkalmazáshoz. A frissítés indításához nyomd meg a Frissítés gombot.", "text": "Frissítés elérhető — 1 napja"}], "en": [{"title": "A newer version of this app is available. Select the Update button to start it.", "text": "Update available — 1 day
|
||||
08:42:31 [6] Update -> 202 {'ok': True, 'data': {'accepted': True, 'completed': False}, 'message': 'Frissítés elindult – az állapot a kártyán követhető'}
|
||||
08:42:31 + 0.0s phase=safety-dump label=Adatbázis pillanatkép… err=None hold=None
|
||||
08:42:33 + 2.1s phase=pulling label=Új verzió letöltése… err=None hold=None
|
||||
08:42:34 + 3.1s phase=copying label=Az adatok másolása a frissítés előtt… err=None hold=None
|
||||
08:42:41 + 10.3s phase=starting label=Indítás az új verzióval… err=None hold=None
|
||||
08:42:53 + 21.6s phase=verifying label=Működés ellenőrzése… err=None hold=None
|
||||
08:43:28 + 57.4s phase=done label=Frissítve err=None hold=None
|
||||
08:43:37 romm: login as the seeded user http=200 ok=True
|
||||
08:43:43 romm: POST /api/users http=403
|
||||
08:43:43 romm: refused {"detail":"Forbidden"}
|
||||
08:43:46 PRESS 1: phase=done pinned={'romm': 'rommapp/romm:5.3.1', 'romm-db': 'mariadb:11.4', 'romm-redis': 'redis:7-alpine'} readback A=True
|
||||
2026/09/24 06:42:31 update.go:722: [INFO] [stacks] update romm: ladder — step 2 of 3: romm=rommapp/romm:5.3.0, romm-db=mariadb:11.4, romm-redis=redis:7-alpine → romm=rommapp/romm:5.3.1, romm-db=mariadb:11.4, romm-redis=redis:7-alpine, from 90dd9d68258286ef.yml
|
||||
2026/09/24 06:42:33 pin.go:370: [INFO] [stacks] update romm: pin advanced to the catalog's current definition (romm=rommapp/romm:5.3.1, romm-db=mariadb:11.4, romm-redis=redis:7-alpine)
|
||||
2026/09/24 06:43:28 update.go:884: [INFO] [stacks] update romm: healthy after 35s (the app's health check passed)
|
||||
2026/09/24 06:43:28 update.go:892: [INFO] [stacks] update romm: DONE in 57s
|
||||
|
||||
DB: 2026-09-24 08:42:42+02:00 [Note] [Entrypoint]: MariaDB upgrade not required
|
||||
Version: '11.4.13-MariaDB-ubu2404' socket: '/run/mysqld/mysqld.sock' port: 3306 mariadb.org binary distribution
|
||||
|
||||
files: 26: image: rommapp/romm:5.3.1
|
||||
86: memory: 768M
|
||||
102: image: mariadb:11.4
|
||||
123: memory: 384M
|
||||
132: image: redis:7-alpine
|
||||
145: memory: 128M
|
||||
---
|
||||
applied == steps/90dd9d68258286ef.yml
|
||||
applied != template
|
||||
|
||||
steps line: {'hu': [('1', 'Hátralévő frissítési lépések: 1')], 'en': [('1', 'Update steps remaining: 1')]}
|
||||
08:43:46 [6] Update -> 202 {'ok': True, 'data': {'accepted': True, 'completed': False}, 'message': 'Frissítés elindult – az állapot a kártyán követhető'}
|
||||
08:43:46 + 0.0s phase=safety-dump label=Adatbázis pillanatkép… err=None hold=None
|
||||
08:43:48 + 2.1s phase=pulling label=Új verzió letöltése… err=None hold=None
|
||||
08:43:50 + 3.1s phase=copying label=Az adatok másolása a frissítés előtt… err=None hold=None
|
||||
08:44:02 + 15.4s phase=starting label=Indítás az új verzióval… err=None hold=None
|
||||
08:44:18 + 31.8s phase=verifying label=Működés ellenőrzése… err=None hold=None
|
||||
08:44:54 + 67.7s phase=done label=Frissítve err=None hold=None
|
||||
08:45:02 romm: login as the seeded user http=200 ok=True
|
||||
08:45:11 PRESS 2: phase=done pinned={'romm': 'rommapp/romm:5.3.1', 'romm-db': 'mariadb:11.8', 'romm-redis': 'redis:7-alpine'} readback A=True
|
||||
2026/09/24 06:43:46 update.go:722: [INFO] [stacks] update romm: ladder — the last step (3 of 3) — the catalog's current definition
|
||||
2026/09/24 06:43:48 pin.go:370: [INFO] [stacks] update romm: pin advanced to the catalog's current definition (romm=rommapp/romm:5.3.1, romm-db=mariadb:11.8, romm-redis=redis:7-alpine)
|
||||
2026/09/24 06:44:53 update.go:884: [INFO] [stacks] update romm: healthy after 35s (the app's health check passed)
|
||||
2026/09/24 06:44:53 update.go:892: [INFO] [stacks] update romm: DONE in 1m7s
|
||||
|
||||
DB: Version: '11.8.9-MariaDB-ubu2404' socket: '/run/mysqld/mysqld.sock' port: 0 mariadb.org binary distribution
|
||||
2026-09-24 08:44:05+02:00 [Note] [Entrypoint]: Starting mariadb-upgrade
|
||||
The --upgrade-system-tables option was used, user tables won't be touched.
|
||||
Major version upgrade detected from 11.4.13-MariaDB to 11.8.9-MariaDB. Check required!
|
||||
2026-09-24 08:44:12+02:00 [Note] [Entrypoint]: Finished mariadb-upgrade
|
||||
Version: '11.8.9-MariaDB-ubu2404' socket: '/run/mysqld/mysqld.sock' port: 3306 mariadb.org binary distribution
|
||||
|
||||
files: 26: image: rommapp/romm:5.3.1
|
||||
86: memory: 768M
|
||||
102: image: mariadb:11.8
|
||||
123: memory: 384M
|
||||
132: image: redis:7-alpine
|
||||
145: memory: 128M
|
||||
---
|
||||
applied != steps/90dd…
|
||||
applied == template
|
||||
|
||||
steps line: {'hu': [], 'en': []}
|
||||
@@ -0,0 +1,70 @@
|
||||
08:41:05 drill 95d83b7ab698: romm template = step 1's own definition (5.3.0 / mariadb 11.4) for the install push rc=0
|
||||
08:41:12 [1] made the drive paths this app requires: ['/mnt/felhom-drives/scratch_hdd/userdata/romm']
|
||||
08:41:12 [1] required fields filled beyond DOMAIN/SUBDOMAIN: ['HDD_PATH']
|
||||
08:41:12 [1] deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}
|
||||
08:42:13 [1] deployed, controller state=running, pinned={'romm': 'rommapp/romm:5.3.0', 'romm-db': 'mariadb:11.4', 'romm-redis': 'redis:7-alpine'}
|
||||
08:42:22 romm: POST /api/users http=201
|
||||
08:42:23 drill 0e4e12aed1f2: romm back to the head (5.3.1 / mariadb 11.8) push rc=0
|
||||
08:42:31 before: steps line {"hu": [["2", "Hátralévő frissítési lépések: 2"]], "en": [["2", "Update steps remaining: 2"]]} badges {"hu": [{"title": "Újabb változat érhető el ehhez az alkalmazáshoz. A frissítés indításához nyomd meg a Frissítés gombot.", "text": "Frissítés elérhető — 1 napja"}], "en": [{"title": "A newer version of this app is available. Select the Update button to start it.", "text": "Update available — 1 day
|
||||
08:42:31 [6] Update -> 202 {'ok': True, 'data': {'accepted': True, 'completed': False}, 'message': 'Frissítés elindult – az állapot a kártyán követhető'}
|
||||
08:42:31 + 0.0s phase=safety-dump label=Adatbázis pillanatkép… err=None hold=None
|
||||
08:42:33 + 2.1s phase=pulling label=Új verzió letöltése… err=None hold=None
|
||||
08:42:34 + 3.1s phase=copying label=Az adatok másolása a frissítés előtt… err=None hold=None
|
||||
08:42:41 + 10.3s phase=starting label=Indítás az új verzióval… err=None hold=None
|
||||
08:42:53 + 21.6s phase=verifying label=Működés ellenőrzése… err=None hold=None
|
||||
08:43:28 + 57.4s phase=done label=Frissítve err=None hold=None
|
||||
08:43:37 romm: login as the seeded user http=200 ok=True
|
||||
08:43:43 romm: POST /api/users http=403
|
||||
08:43:43 romm: refused {"detail":"Forbidden"}
|
||||
08:43:46 PRESS 1: phase=done pinned={'romm': 'rommapp/romm:5.3.1', 'romm-db': 'mariadb:11.4', 'romm-redis': 'redis:7-alpine'} readback A=True
|
||||
2026/09/24 06:42:31 update.go:722: [INFO] [stacks] update romm: ladder — step 2 of 3: romm=rommapp/romm:5.3.0, romm-db=mariadb:11.4, romm-redis=redis:7-alpine → romm=rommapp/romm:5.3.1, romm-db=mariadb:11.4, romm-redis=redis:7-alpine, from 90dd9d68258286ef.yml
|
||||
2026/09/24 06:42:33 pin.go:370: [INFO] [stacks] update romm: pin advanced to the catalog's current definition (romm=rommapp/romm:5.3.1, romm-db=mariadb:11.4, romm-redis=redis:7-alpine)
|
||||
2026/09/24 06:43:28 update.go:884: [INFO] [stacks] update romm: healthy after 35s (the app's health check passed)
|
||||
2026/09/24 06:43:28 update.go:892: [INFO] [stacks] update romm: DONE in 57s
|
||||
|
||||
DB: 2026-09-24 08:42:42+02:00 [Note] [Entrypoint]: MariaDB upgrade not required
|
||||
Version: '11.4.13-MariaDB-ubu2404' socket: '/run/mysqld/mysqld.sock' port: 3306 mariadb.org binary distribution
|
||||
|
||||
files: 26: image: rommapp/romm:5.3.1
|
||||
86: memory: 768M
|
||||
102: image: mariadb:11.4
|
||||
123: memory: 384M
|
||||
132: image: redis:7-alpine
|
||||
145: memory: 128M
|
||||
---
|
||||
applied == steps/90dd9d68258286ef.yml
|
||||
applied != template
|
||||
|
||||
steps line: {'hu': [('1', 'Hátralévő frissítési lépések: 1')], 'en': [('1', 'Update steps remaining: 1')]}
|
||||
08:43:46 [6] Update -> 202 {'ok': True, 'data': {'accepted': True, 'completed': False}, 'message': 'Frissítés elindult – az állapot a kártyán követhető'}
|
||||
08:43:46 + 0.0s phase=safety-dump label=Adatbázis pillanatkép… err=None hold=None
|
||||
08:43:48 + 2.1s phase=pulling label=Új verzió letöltése… err=None hold=None
|
||||
08:43:50 + 3.1s phase=copying label=Az adatok másolása a frissítés előtt… err=None hold=None
|
||||
08:44:02 + 15.4s phase=starting label=Indítás az új verzióval… err=None hold=None
|
||||
08:44:18 + 31.8s phase=verifying label=Működés ellenőrzése… err=None hold=None
|
||||
08:44:54 + 67.7s phase=done label=Frissítve err=None hold=None
|
||||
08:45:02 romm: login as the seeded user http=200 ok=True
|
||||
08:45:11 PRESS 2: phase=done pinned={'romm': 'rommapp/romm:5.3.1', 'romm-db': 'mariadb:11.8', 'romm-redis': 'redis:7-alpine'} readback A=True
|
||||
2026/09/24 06:43:46 update.go:722: [INFO] [stacks] update romm: ladder — the last step (3 of 3) — the catalog's current definition
|
||||
2026/09/24 06:43:48 pin.go:370: [INFO] [stacks] update romm: pin advanced to the catalog's current definition (romm=rommapp/romm:5.3.1, romm-db=mariadb:11.8, romm-redis=redis:7-alpine)
|
||||
2026/09/24 06:44:53 update.go:884: [INFO] [stacks] update romm: healthy after 35s (the app's health check passed)
|
||||
2026/09/24 06:44:53 update.go:892: [INFO] [stacks] update romm: DONE in 1m7s
|
||||
|
||||
DB: Version: '11.8.9-MariaDB-ubu2404' socket: '/run/mysqld/mysqld.sock' port: 0 mariadb.org binary distribution
|
||||
2026-09-24 08:44:05+02:00 [Note] [Entrypoint]: Starting mariadb-upgrade
|
||||
The --upgrade-system-tables option was used, user tables won't be touched.
|
||||
Major version upgrade detected from 11.4.13-MariaDB to 11.8.9-MariaDB. Check required!
|
||||
2026-09-24 08:44:12+02:00 [Note] [Entrypoint]: Finished mariadb-upgrade
|
||||
Version: '11.8.9-MariaDB-ubu2404' socket: '/run/mysqld/mysqld.sock' port: 3306 mariadb.org binary distribution
|
||||
|
||||
files: 26: image: rommapp/romm:5.3.1
|
||||
86: memory: 768M
|
||||
102: image: mariadb:11.8
|
||||
123: memory: 384M
|
||||
132: image: redis:7-alpine
|
||||
145: memory: 128M
|
||||
---
|
||||
applied != steps/90dd…
|
||||
applied == template
|
||||
|
||||
steps line: {'hu': [], 'en': []}
|
||||
@@ -0,0 +1,27 @@
|
||||
08:45:32 [X] stop -> 200 {'ok': True, 'message': 'Stack romm stop completed'}
|
||||
08:45:37 [X] remove (with drive data) -> 409 {'ok': False, 'error': 'A(z) /mnt/felhom-drives/scratch_hdd/userdata/romm tárhely jelenleg nem elérhető — az alkalmazás nem távolítható el, amíg a meghajtó viss
|
||||
08:45:37 [X] refused because the drive path cannot be resolved (R-442, fail-closed and right) — removing the app and KEEPING the drive data instead
|
||||
08:46:04 [X] remove (keeping drive data) -> 200 {'ok': True, 'data': {'removed': 'romm', 'volumes_removed': ['romm_romm_config', 'romm_romm_db_data', 'romm_romm_redis_data'], 'hdd_paths_removed': [], 'hdd_pat
|
||||
08:46:12 [X] after remove: deployed=False leftovers='/opt/docker/stacks/romm'
|
||||
08:45:32 [X] stop -> 200 {'ok': True, 'message': 'Stack romm stop completed'}
|
||||
08:45:37 [X] remove (with drive data) -> 409 {'ok': False, 'error': 'A(z) /mnt/felhom-drives/scratch_hdd/userdata/romm tárhely jelenleg nem elérhető — az alkalmazás nem távolítható el, amíg a meghajtó viss
|
||||
08:45:37 [X] refused because the drive path cannot be resolved (R-442, fail-closed and right) — removing the app and KEEPING the drive data instead
|
||||
08:46:04 [X] remove (keeping drive data) -> 200 {'ok': True, 'data': {'removed': 'romm', 'volumes_removed': ['romm_romm_config', 'romm_romm_db_data', 'romm_romm_redis_data'], 'hdd_paths_removed': [], 'hdd_pat
|
||||
08:46:12 [X] after remove: deployed=False leftovers='/opt/docker/stacks/romm'
|
||||
no-romm-volumes
|
||||
total 36
|
||||
drwxr-xr-x 3 root root 4096 Sep 24 06:46 .
|
||||
drwxr-xr-x 57 root root 4096 Sep 13 20:22 ..
|
||||
-rw-r--r-- 1 root root 12798 Sep 23 22:15 .felhom.yml
|
||||
-rw-r--r-- 1 root root 5771 Sep 24 06:43 docker-compose.yml
|
||||
drwxr-xr-x 11 root root 4096 Sep 23 10:13 hold-logs
|
||||
ls: cannot access '/mnt/felhom-drives/scratch_hdd/userdata/romm': No such file or directory
|
||||
felhom-controller Up 29 minutes (healthy)
|
||||
privatebin Up 6 hours (healthy)
|
||||
paperless-webserver Up 6 hours (healthy)
|
||||
paperless-postgres Up 6 hours (healthy)
|
||||
paperless-redis Up 6 hours (healthy)
|
||||
gokapi Restarting (1) 58 seconds ago
|
||||
filebrowser Up 10 hours (healthy)
|
||||
traefik Up 10 hours
|
||||
|
||||
@@ -0,0 +1,28 @@
|
||||
git:
|
||||
branch: main
|
||||
repo_url: https://gitea.dooplex.hu/admin/app-catalog-felhom.eu.git
|
||||
sync_interval: 15m
|
||||
token: <redacted>
|
||||
username: ""
|
||||
hub:
|
||||
0
|
||||
|
||||
/dev/loop1 69G 18G 48G 27% /var/lib/docker
|
||||
0
|
||||
removed vikunja/vikunja:2.4.0
|
||||
removed vikunja/vikunja:2.5.0
|
||||
removed vikunja/vikunja:2.6.0
|
||||
removed nextcloud:34.0.1-apache
|
||||
removed nextcloud:34.0.4-apache
|
||||
removed mariadb:12.3
|
||||
removed rommapp/romm:5.3.0
|
||||
removed rommapp/romm:5.3.1
|
||||
removed mariadb:11.4
|
||||
removed mariadb:11.8
|
||||
removed actualbudget/actual-server:26.9.0
|
||||
/dev/loop1 69G 12G 54G 18% /var/lib/docker
|
||||
ref: refs/heads/main
|
||||
5ed599cd5f50
|
||||
|
||||
drill=5ed599cd5f50 live=5ed599cd5f50
|
||||
drill has_actions: False private: True
|
||||
@@ -0,0 +1,9 @@
|
||||
--- FAIL: TestR659_TheHoldNamesOnlyAWholeCopy (0.01s)
|
||||
--- FAIL: TestR659_TheHoldNamesOnlyAWholeCopy/files_/_unit_only_(round_11) (0.00s)
|
||||
r659_whole_copy_test.go:95: no whole copy exists, yet the hold names "A frissítés nem sikerült, és az automatikus visszaállítás sem. Az adatok az új változat által hagyott állapotban vannak. A(z) nextcloud frissítése 2026-09-23 23:42-kor nem sikerült, és az alkalmazás nem indult el az új verzióval. Az alkalmazás biztonsági okból leállítva marad, hogy az adatai ne sérüljenek. Visszaállítható a Mentések oldalon ebből a biztonsági mentésből: saját meghajtó, 2026-09-23 21:42 — ez a másolat csak a beállításokat és az adatbázist tartalmazza, a fájlokat nem." (none=false)
|
||||
--- FAIL: TestR659_TheHoldNamesOnlyAWholeCopy/files_/_second_drive_only (0.00s)
|
||||
r659_whole_copy_test.go:95: no whole copy exists, yet the hold names "A frissítés nem sikerült, és az automatikus visszaállítás sem. Az adatok az új változat által hagyott állapotban vannak. A(z) nextcloud frissítése 2026-09-23 23:42-kor nem sikerült, és az alkalmazás nem indult el az új verzióval. Az alkalmazás biztonsági okból leállítva marad, hogy az adatai ne sérüljenek. Visszaállítható a Mentések oldalon ebből a biztonsági mentésből: második meghajtó, 2026-09-23 03:42 — ez a másolat a beállításokat, az adatbázist és a fájlokat tartalmazza." (none=false)
|
||||
--- FAIL: TestR659_TheHoldNamesOnlyAWholeCopy/files_/_unit_+_second_drive (0.00s)
|
||||
r659_whole_copy_test.go:95: no whole copy exists, yet the hold names "A frissítés nem sikerült, és az automatikus visszaállítás sem. Az adatok az új változat által hagyott állapotban vannak. A(z) nextcloud frissítése 2026-09-23 23:42-kor nem sikerült, és az alkalmazás nem indult el az új verzióval. Az alkalmazás biztonsági okból leállítva marad, hogy az adatai ne sérüljenek. Visszaállítható a Mentések oldalon ebből a biztonsági mentésből: saját meghajtó, 2026-09-23 21:42 — ez a másolat csak a beállításokat és az adatbázist tartalmazza, a fájlokat nem." (none=false)
|
||||
--- FAIL: TestR659_TheHoldNamesOnlyAWholeCopy/files_/_off-site_+_unit (0.00s)
|
||||
ok gitea.dooplex.hu/admin/felhom-controller/internal/backup 0.019s
|
||||
@@ -0,0 +1,6 @@
|
||||
--- FAIL: TestR659_HeldPage_NoWholeCopyOffersNoRestoreButton (0.24s)
|
||||
r659_held_page_test.go:33: stacks: a hold naming no whole copy still offers the Mentések button — the restore there would refuse
|
||||
FAIL
|
||||
FAIL gitea.dooplex.hu/admin/felhom-controller/internal/web 0.252s
|
||||
FAIL
|
||||
ok gitea.dooplex.hu/admin/felhom-controller/internal/web 0.283s
|
||||
@@ -0,0 +1,9 @@
|
||||
FAIL
|
||||
FAIL: test_the_apps_own_folders_are_cleared_and_said (__main__.ClearScratch.test_the_apps_own_folders_are_cleared_and_said)
|
||||
AssertionError: True is not false : the last run's files are still there
|
||||
FAIL: test_every_request_errored_is_inconclusive (__main__.LoadVerdict.test_every_request_errored_is_inconclusive)
|
||||
AssertionError: 'reached' != 'inconclusive'
|
||||
FAIL: test_under_half_is_inconclusive_and_none_is_inconclusive (__main__.LoadVerdict.test_under_half_is_inconclusive_and_none_is_inconclusive)
|
||||
AssertionError: 'reached' != 'inconclusive'
|
||||
FAILED (failures=3)
|
||||
OK
|
||||
@@ -0,0 +1,5 @@
|
||||
-- probe-matches-compose: the decoys (the label moves, the fact does not)
|
||||
XX FACT: a two-step ladder with NO steps/ file for the first step rc=0 (expected 1)
|
||||
XX DECOY: the steps/ file has the right NAME and names the head's image rc=0 (expected 1)
|
||||
XX DECOY: the step's definition sits beside the template under another name rc=0 (expected 1)
|
||||
test-record gate — 53 template(s) read, 25 carry a ladder, 0 convicted
|
||||
@@ -0,0 +1,5 @@
|
||||
FAIL
|
||||
FAIL: test_a_second_step_keeps_the_first_steps_definition (__main__.WriterTest.test_a_second_step_keeps_the_first_steps_definition)
|
||||
AssertionError: False is not true : WROTE navidrome: {'navidrome': 'deluan/navidrome:0.64.1-next'} -> {'navidrome': 'deluan/navidrome:0.64.1-after'} peak 41.0% marks {'files_may_change': False, 'needs_person': None, 'memory_tight': False}
|
||||
FAILED (failures=1)
|
||||
OK
|
||||
@@ -0,0 +1,7 @@
|
||||
--- FAIL: TestLadder_TwoPressesTwoSteps (0.01s)
|
||||
ladder_test.go:83: press 1 pinned nextcloud:34.0.1-apache, want the tested step B nextcloud:33.0.0-apache — one press must be one step
|
||||
--- FAIL: TestLadder_FailedStepStopsTheLadder (0.01s)
|
||||
ladder_test.go:152: C was brought up after B failed: ups=[nextcloud:34.0.1-apache nextcloud:31.0.14-apache]
|
||||
--- FAIL: TestLadder_MissingStepFileRefusesBeforeAnythingMoves (0.01s)
|
||||
ladder_test.go:169: phase="done" key="", want failed/pin_failed
|
||||
ok gitea.dooplex.hu/admin/felhom-controller/internal/stacks 0.051s
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,64 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Part A, first half — on controller v0.267.0 (the field's version): vikunja installed, seeded (A),
|
||||
backed up by the product (the update's own `backing-up` leg, then a pull that fails: nothing moves),
|
||||
RESTORED from its own unit — which recreates its volumes WITHOUT compose labels (R-658) — and seeded
|
||||
again (B, after the restore). Evidence only; every act is a product endpoint."""
|
||||
import json, re, sys, time
|
||||
sys.path.insert(0, ".")
|
||||
import walk as w
|
||||
import fixtures as fx
|
||||
|
||||
ST = "../partA/state.json"
|
||||
F = fx.Vikunja()
|
||||
SUB = "vikunja-la"
|
||||
out = {"controller": w.guest("cat /etc/felhom-controller-image").strip()}
|
||||
w.login()
|
||||
|
||||
|
||||
def drill_set(ref, msg):
|
||||
import fcntl
|
||||
with open(f"{w.SC}/drill.lock", "w") as lk:
|
||||
fcntl.flock(lk, fcntl.LOCK_EX)
|
||||
w.sh(["git", "-C", w.DRILL, "pull", "-q", "--rebase", "origin", "main"], timeout=120)
|
||||
p = f"{w.DRILL}/templates/vikunja/docker-compose.yml"
|
||||
s = open(p).read()
|
||||
s = re.sub(r"image: vikunja/vikunja:\S+", f"image: vikunja/vikunja:{ref}", s)
|
||||
open(p, "w").write(s)
|
||||
w.sh(["git", "-C", w.DRILL, "commit", "-qam", f"LIVE-A vikunja {ref}: {msg}"])
|
||||
r = w.sh(["git", "-C", w.DRILL, "push", "-q", "origin", "main"], timeout=120)
|
||||
h = w.sh(["git", "-C", w.DRILL, "rev-parse", "--short=12", "HEAD"]).stdout.strip()
|
||||
w.say(f"drill {h}: vikunja {ref} ({msg}) push rc={r.returncode}")
|
||||
return h
|
||||
|
||||
|
||||
def labels():
|
||||
return w.guest("for v in $(docker volume ls -q | grep '^vikunja_'); do echo \"$v $(docker volume inspect $v --format '{{json .Labels}}')\"; done")
|
||||
|
||||
|
||||
out["drill_A"] = drill_set("2.5.0", "the install version")
|
||||
w.sync_rescan()
|
||||
out["deployed"] = w.deploy("vikunja", SUB)
|
||||
w.wait_app(SUB, "/", tries=60, delay=3)
|
||||
A = F.seed(w, SUB, w.say)
|
||||
out["seed_A"] = bool(A)
|
||||
out["labels_after_install"] = labels()
|
||||
w.say("labels after install:\n" + out["labels_after_install"])
|
||||
|
||||
out["drill_E"] = drill_set("2.5.99-notag", "a tag that does not exist: backing-up runs, the pull fails, nothing moves")
|
||||
out["badge_wait"] = w.sync_rescan("vikunja", "vikunja/vikunja:2.5.99-notag")
|
||||
out["press_backup"] = w.press_update("vikunja")
|
||||
out["snapshots"] = w.snapshots("vikunja")
|
||||
w.say("snapshots offered: " + json.dumps(out["snapshots"])[:400])
|
||||
out["drill_back"] = drill_set("2.5.0", "back to the install version")
|
||||
w.sync_rescan()
|
||||
|
||||
out["restore"] = w.restore("vikunja")
|
||||
out["labels_after_restore_v0267"] = labels()
|
||||
w.say("labels after the v0.267.0 restore:\n" + out["labels_after_restore_v0267"])
|
||||
w.wait_app(SUB, "/", tries=60, delay=3)
|
||||
out["A_after_restore"] = bool(F.verify(w, SUB, A, w.say))
|
||||
B = F.seed(w, SUB, w.say)
|
||||
out["seed_B_after_restore"] = bool(B)
|
||||
json.dump({"A": A, "B": B, "sub": SUB}, open(ST, "w"))
|
||||
json.dump(out, open("../partA/01-v0267-install-backup-restore.json", "w"), indent=2, ensure_ascii=False, default=str)
|
||||
open("../partA/01-v0267-install-backup-restore.log", "w").write("\n".join(w.LOG) + "\n")
|
||||
@@ -0,0 +1,96 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Part A, second half — on controller v0.268.0. vikunja's volumes are UNLABELLED (restored under v0.267.0,
|
||||
liveA1). (e) a failing update (real move 2.5.0 -> 2.6.0 + a probe port the app does not answer): the undo
|
||||
must copy BOTH volumes and seeds A and B must read back. (f) a restore under v0.268.0: the recreated
|
||||
volumes carry compose's labels. (g) seed C, the same failing update again: undone, both copied, A B C read
|
||||
back. (h) remove: volumes_removed names both, no applied-compose.yml / applied-meta/ left (R-651)."""
|
||||
import json, re, sys, time
|
||||
sys.path.insert(0, ".")
|
||||
import walk as w
|
||||
import fixtures as fx
|
||||
|
||||
S = json.load(open("../partA/state.json"))
|
||||
F, SUB = fx.Vikunja(), S["sub"]
|
||||
out = {"controller": w.guest("cat /etc/felhom-controller-image").strip()}
|
||||
w.login()
|
||||
|
||||
|
||||
def drill(edit, msg):
|
||||
import fcntl
|
||||
with open(f"{w.SC}/drill.lock", "w") as lk:
|
||||
fcntl.flock(lk, fcntl.LOCK_EX)
|
||||
w.sh(["git", "-C", w.DRILL, "pull", "-q", "--rebase", "origin", "main"], timeout=120)
|
||||
edit()
|
||||
w.sh(["git", "-C", w.DRILL, "commit", "-qam", "LIVE-A " + msg])
|
||||
r = w.sh(["git", "-C", w.DRILL, "push", "-q", "origin", "main"], timeout=120)
|
||||
h = w.sh(["git", "-C", w.DRILL, "rev-parse", "--short=12", "HEAD"]).stdout.strip()
|
||||
w.say(f"drill {h}: {msg} push rc={r.returncode}")
|
||||
return h
|
||||
|
||||
|
||||
def set_vik(ref, port):
|
||||
p = f"{w.DRILL}/templates/vikunja/docker-compose.yml"
|
||||
s = re.sub(r"image: vikunja/vikunja:\S+", f"image: vikunja/vikunja:{ref}", open(p).read())
|
||||
open(p, "w").write(s)
|
||||
fy = f"{w.DRILL}/templates/vikunja/.felhom.yml"
|
||||
f = open(fy).read()
|
||||
f2 = re.sub(r"(healthcheck:\n(?:.*\n){0,8}?\s+port: )\d+", lambda m: m.group(1) + str(port), f, count=1)
|
||||
assert f2 != f or str(port) in f, "probe port not found"
|
||||
open(fy, "w").write(f2)
|
||||
|
||||
|
||||
def labels():
|
||||
return w.guest("for v in $(docker volume ls -q | grep '^vikunja_'); do echo \"$v $(docker volume inspect $v --format '{{json .Labels}}')\"; done")
|
||||
|
||||
|
||||
def ctl_log(since):
|
||||
return w.guest(f"docker logs --since {since} felhom-controller 2>&1 | grep -E 'vikunja' | grep -E 'undo copy will hold|carry no compose label|ladder|copied|UNDONE|UNDO|RestoreStack|Restoring Docker volume|was not created|volume .* already exists|created WITHOUT' | tail -30")
|
||||
|
||||
|
||||
def readall(tag, seeds):
|
||||
w.wait_app(SUB, "/", tries=60, delay=3)
|
||||
r = {k: bool(F.verify(w, SUB, v, w.say)) for k, v in seeds.items()}
|
||||
w.say(f"READBACK {tag}: {r}")
|
||||
return r
|
||||
|
||||
|
||||
PORT_REAL = int(re.search(r"port: (\d+)", open(f"{w.DRILL}/templates/vikunja/.felhom.yml").read().split("healthcheck:")[1]).group(1))
|
||||
out["probe_port_real"] = PORT_REAL
|
||||
out["labels_before"] = labels()
|
||||
w.say("labels before (unlabelled after the v0.267.0 restore):\n" + out["labels_before"])
|
||||
|
||||
# (e)
|
||||
t0 = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
|
||||
out["drill_e"] = drill(lambda: set_vik("2.6.0", 8999), "vikunja 2.6.0 + probe 8999 (the failing edge)")
|
||||
out["badge_e"] = w.sync_rescan("vikunja", "vikunja/vikunja:2.6.0")
|
||||
out["press_e"] = w.press_update("vikunja")
|
||||
out["log_e"] = ctl_log(t0)
|
||||
w.say("controller log (e):\n" + out["log_e"])
|
||||
out["read_e"] = readall("after the undo (e)", {"A": S["A"], "B": S["B"]})
|
||||
out["obs_e"] = w.observables("vikunja")
|
||||
|
||||
# (f)
|
||||
t1 = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
|
||||
out["restore_f"] = w.restore("vikunja")
|
||||
out["labels_after_restore_v0268"] = labels()
|
||||
w.say("labels after the v0.268.0 restore:\n" + out["labels_after_restore_v0268"])
|
||||
out["log_f"] = ctl_log(t1)
|
||||
out["compose_warn_f"] = w.guest(f"docker logs --since {t1} felhom-controller 2>&1 | grep -iE 'not created by Docker Compose|already exists|Recreate' | tail -5")
|
||||
w.say("compose warnings after the restore (want none): " + (out["compose_warn_f"].strip() or "(none)"))
|
||||
out["read_f"] = readall("after the v0.268.0 restore (f)", {"A": S["A"]})
|
||||
|
||||
# (g)
|
||||
C = F.seed(w, SUB, w.say)
|
||||
t2 = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
|
||||
out["press_g"] = w.press_update("vikunja")
|
||||
out["log_g"] = ctl_log(t2)
|
||||
w.say("controller log (g):\n" + out["log_g"])
|
||||
out["read_g"] = readall("after the second undo (g)", {"A": S["A"], "C": C})
|
||||
|
||||
# (h)
|
||||
out["drill_h"] = drill(lambda: set_vik("2.5.0", PORT_REAL), "vikunja back to 2.5.0 + its real probe")
|
||||
out["remove_h"] = w.remove("vikunja")
|
||||
out["leftovers_h"] = w.guest("ls -la /opt/docker/stacks/vikunja/; docker volume ls -q | grep -E '^vikunja' || echo 'no vikunja volumes'")
|
||||
w.say("after remove:\n" + out["leftovers_h"])
|
||||
json.dump(out, open("../partA/02-v0268-undo-after-restore.json", "w"), indent=2, ensure_ascii=False, default=str)
|
||||
open("../partA/02-v0268-undo-after-restore.log", "w").write("\n".join(w.LOG) + "\n")
|
||||
@@ -0,0 +1,117 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Part B — R-659 on controller v0.268.0: chaos round 11 reproduced. nextcloud (declared drive files),
|
||||
a failing update (34.0.1 -> 34.0.4 + a probe port it does not answer), the undo copy CUT OFF during
|
||||
`verifying` (its finished-marker removed — `09` §6.1a's measured case) -> HOLD. No off-site tier on 9202.
|
||||
Expected: the no-whole-copy sentence in hu and en, no Mentések button beside it, the operator event
|
||||
(DROPPED at WARN on this hub-less box — R-620), and NO app_start_failed after the hold (R-660)."""
|
||||
import html as H, json, re, sys, time
|
||||
sys.path.insert(0, ".")
|
||||
import walk as w
|
||||
import fixtures as fx
|
||||
|
||||
F, SUB, APP = fx.Nextcloud(), "cloud-lb", "nextcloud"
|
||||
CAT = f"{w.DRILL}/templates/{APP}"
|
||||
out = {"controller": w.guest("cat /etc/felhom-controller-image").strip()}
|
||||
w.login()
|
||||
|
||||
|
||||
def drill(edit, msg):
|
||||
import fcntl
|
||||
with open(f"{w.SC}/drill.lock", "w") as lk:
|
||||
fcntl.flock(lk, fcntl.LOCK_EX)
|
||||
w.sh(["git", "-C", w.DRILL, "pull", "-q", "--rebase", "origin", "main"], timeout=120)
|
||||
edit()
|
||||
w.sh(["git", "-C", w.DRILL, "commit", "-qam", "LIVE-B " + msg])
|
||||
r = w.sh(["git", "-C", w.DRILL, "push", "-q", "origin", "main"], timeout=120)
|
||||
h = w.sh(["git", "-C", w.DRILL, "rev-parse", "--short=12", "HEAD"]).stdout.strip()
|
||||
w.say(f"drill {h}: {msg} push rc={r.returncode}")
|
||||
return h
|
||||
|
||||
|
||||
HEAD = open(f"{CAT}/docker-compose.yml").read()
|
||||
STEP1 = open(f"{CAT}/steps/eef2e4afe1218021.yml").read()
|
||||
FY = open(f"{CAT}/.felhom.yml").read()
|
||||
|
||||
|
||||
def set_compose(body):
|
||||
return lambda: open(f"{CAT}/docker-compose.yml", "w").write(body)
|
||||
|
||||
|
||||
def set_probe(port):
|
||||
def _e():
|
||||
f = open(f"{CAT}/.felhom.yml").read()
|
||||
open(f"{CAT}/.felhom.yml", "w").write(re.sub(r"(healthcheck:\n\s+checks:\n\s+- type: api\n\s+port: )\d+", lambda m: m.group(1) + str(port), f, count=1))
|
||||
return _e
|
||||
|
||||
|
||||
out["drill_1"] = drill(set_compose(STEP1), "nextcloud template = ladder step 1's own definition (34.0.1 / mariadb 12.3)")
|
||||
w.sync_rescan()
|
||||
out["deployed"] = w.deploy(APP, SUB)
|
||||
A = F.seed(w, SUB, w.say)
|
||||
out["seed_A"] = bool(A)
|
||||
out["drill_2"] = drill(lambda: (set_compose(HEAD)(), set_probe(8999)()), "nextcloud back to the head (34.0.4) + probe port 8999")
|
||||
out["badge_wait"] = w.sync_rescan(APP, "nextcloud:34.0.4-apache", tries=60)
|
||||
h = H.unescape(w.page(f"/apps/{APP}"))
|
||||
out["steps_left_line_hu"] = re.findall(r'data-ladder-steps="(\d+)">([^<]*)<', h)
|
||||
out["steps_left_line_en"] = re.findall(r'data-ladder-steps="(\d+)">([^<]*)<', H.unescape(w.page(f"/apps/{APP}?lang=en")))
|
||||
w.say(f"steps-left line hu={out['steps_left_line_hu']} en={out['steps_left_line_en']}")
|
||||
|
||||
cut = {}
|
||||
|
||||
|
||||
def on_phase(ph):
|
||||
if ph == "verifying" and not cut:
|
||||
cut["out"] = w.guest(f"""c=$(docker volume ls -q --filter label=felhom.undo-copy-of={APP} | head -1); echo "copy: $c"
|
||||
docker run --rm -v $c:/c alpine sh -c 'rm -f /c/felhom-undo-complete; ls /c | head -5'""")
|
||||
w.say(" >>> finished-marker removed from one undo copy:\n" + cut["out"])
|
||||
|
||||
|
||||
t0 = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
|
||||
code, d = w.ctl("POST", f"/api/stacks/{APP}/update")
|
||||
w.say(f"Update -> {code}")
|
||||
phases, seen, tp = [], None, time.time()
|
||||
while time.time() - tp < 1500:
|
||||
st = w.stack(APP)
|
||||
ph = st.get("update_phase")
|
||||
if ph != seen and ph:
|
||||
seen = ph
|
||||
phases.append((round(time.time() - tp, 1), ph))
|
||||
w.say(f" +{phases[-1][0]:6.1f}s phase={ph}")
|
||||
on_phase(ph)
|
||||
if not st.get("updating") and ph in ("done", "failed", "undone") and time.time() - tp > 3:
|
||||
break
|
||||
time.sleep(0.5)
|
||||
held_at = time.time()
|
||||
out["phases"], out["cutoff"] = phases, cut.get("out")
|
||||
st = w.stack(APP)
|
||||
out["api"] = {k: st.get(k) for k in ("state", "update_phase", "hold_reason", "hold_no_whole_copy", "update_error")}
|
||||
w.say("API: " + json.dumps(out["api"], ensure_ascii=False))
|
||||
for lang in ("hu", "en"):
|
||||
page = H.unescape(w.page(f"/apps/{APP}?lang={lang}"))
|
||||
m = re.search(r'data-held="true">(.*?)</div>', page, re.S)
|
||||
out[f"app_page_{lang}"] = m.group(1).strip() if m else None
|
||||
out[f"app_page_{lang}_restore_button"] = bool(m and 'href="/backups/apps"' in m.group(1))
|
||||
lst = H.unescape(w.page(f"/stacks?lang={lang}"))
|
||||
m2 = re.search(r'data-held="true">(.*?)</div>', lst, re.S)
|
||||
out[f"stacks_page_{lang}_restore_button"] = bool(m2 and 'href="/backups/apps"' in m2.group(1))
|
||||
w.say(f"[{lang}] app page held block: {out[f'app_page_{lang}']!r} restore-button app={out[f'app_page_{lang}_restore_button']} list={out[f'stacks_page_{lang}_restore_button']}")
|
||||
# ASCII-fragment checks, with controls (the ui-hungarian rule)
|
||||
hu = out["app_page_hu"] or ""
|
||||
out["ascii_checks"] = {"hu has 'gyfelszolgalat' (positive)": "gyfélszolgálat" in hu,
|
||||
"hu has 'Visszaallithato a Mentesek' (must be absent)": "Visszaállítható a Mentések" in hu,
|
||||
"en has 'Felhom support has been told' (positive)": "Felhom support has been told" in (out["app_page_en"] or "")}
|
||||
w.say("checks: " + json.dumps(out["ascii_checks"]))
|
||||
|
||||
# R-660: two and a half minutes of the dead-app scan after the hold
|
||||
while time.time() - held_at < 150:
|
||||
time.sleep(10)
|
||||
out["log"] = w.guest(f"docker logs --since {t0} felhom-controller 2>&1 | grep -E 'nextcloud|DROPPED|dropped event' | grep -E 'NO copy|DROPPED|dropped event|HELD|UNDO|app_start_failed|no_whole' | tail -30")
|
||||
w.say("controller log:\n" + out["log"])
|
||||
c2, dl = w.ctl("GET", "/api/debug/logs?level=DEBUG&lines=4000")
|
||||
ents = ((dl.get("data") or {}).get("entries") or []) if isinstance(dl, dict) else []
|
||||
out["debug_ring_events"] = [e.get("timestamp", "") + " " + e.get("message", "") for e in ents
|
||||
if ("app_start_failed" in e.get("message", "") or "app_hold_no_whole_copy" in e.get("message", "")
|
||||
or "app_update_held" in e.get("message", "")) and e.get("timestamp", "") >= t0]
|
||||
w.say("debug ring (events since the press):\n " + "\n ".join(out["debug_ring_events"]))
|
||||
json.dump(out, open("../partB/01-round11-reproduced.json", "w"), indent=2, ensure_ascii=False, default=str)
|
||||
open("../partB/01-round11-reproduced.log", "w").write("\n".join(w.LOG) + "\n")
|
||||
@@ -0,0 +1,29 @@
|
||||
#!/usr/bin/env python3
|
||||
"""R-660 positive control: a throwaway app (actualbudget) stopped OUT OF BAND must raise app_start_failed
|
||||
while the held nextcloud (held since 06:27:40Z) must not. Then the throwaway is removed through the product."""
|
||||
import json, sys, time
|
||||
sys.path.insert(0, ".")
|
||||
import walk as w
|
||||
w.login()
|
||||
t0 = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
|
||||
ok = w.deploy("actualbudget", "budget-c660")
|
||||
w.wait_app("budget-c660", "/", tries=40, delay=3)
|
||||
w.say("stopping the throwaway's container out of band (not through the product): " + w.guest("docker stop actualbudget 2>&1").strip())
|
||||
found = None
|
||||
for i in range(24):
|
||||
time.sleep(10)
|
||||
c, d = w.ctl("GET", "/api/debug/logs?level=DEBUG&lines=6000")
|
||||
ents = ((d.get("data") or {}).get("entries") or []) if isinstance(d, dict) else []
|
||||
evs = [e["timestamp"] + " " + e["message"] for e in ents if e.get("timestamp", "") >= t0 and
|
||||
("app_start_failed" in e.get("message", "") or "currently down" in e.get("message", ""))]
|
||||
if any("app_start_failed" in x for x in evs):
|
||||
found = evs
|
||||
break
|
||||
w.say("events/heartbeats since the control started:\n " + "\n ".join(found or evs))
|
||||
names = w.guest(f"docker logs --since {t0} felhom-controller 2>&1 | grep -iE 'app_start_failed|not running|DeadApp|nem fut' | tail -8")
|
||||
w.say("controller log lines:\n" + names)
|
||||
st = w.stack("nextcloud")
|
||||
w.say(f"nextcloud meanwhile: state={st.get('state')} held={bool(st.get('hold_reason'))}")
|
||||
w.remove("actualbudget")
|
||||
json.dump({"events": found or evs, "log": names, "nextcloud_state": st.get("state")}, open("../partB/03-r660-control.json", "w"), indent=2)
|
||||
open("../partB/03-r660-control.log", "w").write("\n".join(w.LOG) + "\n")
|
||||
@@ -0,0 +1,26 @@
|
||||
import sys, time, re, html as H, json
|
||||
sys.path.insert(0, ".")
|
||||
import walk as w
|
||||
w.login()
|
||||
w.deploy("actualbudget", "budget-c660b")
|
||||
w.wait_app("budget-c660b", "/", tries=40, delay=3)
|
||||
t0 = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
|
||||
w.say("out-of-band stop: " + w.guest("docker stop actualbudget 2>&1").strip())
|
||||
seen = []
|
||||
for i in range(20):
|
||||
time.sleep(5)
|
||||
h = H.unescape(w.page("/"))
|
||||
txt = re.sub(r"\s+", " ", re.sub("<[^>]+>", " ", h))
|
||||
for name in ("actualbudget", "Actual", "nextcloud", "Nextcloud", "gokapi", "Gokapi"):
|
||||
pass
|
||||
m = re.findall(r"(nem fut[^.]{0,200})", txt)
|
||||
c, d = w.ctl("GET", "/api/debug/logs?level=DEBUG&lines=6000")
|
||||
ents = ((d.get("data") or {}).get("entries") or [])
|
||||
ev = [e["timestamp"] + " " + e["message"] for e in ents if e.get("timestamp", "") >= t0 and "app_start_failed" in e["message"]]
|
||||
if m or ev:
|
||||
seen.append({"t": i * 5, "banner": m[:3], "events": ev})
|
||||
w.say(f"+{i*5}s banner={m[:3]} events={len(ev)}")
|
||||
if ev and m:
|
||||
break
|
||||
json.dump(seen, open("../partB/06-r660-control-banner.json", "w"), indent=2, ensure_ascii=False)
|
||||
w.remove("actualbudget")
|
||||
@@ -0,0 +1,71 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Part D live — the ladder on controller v0.268.0. romm installed at 5.3.0 / mariadb 11.4 (the ladder's
|
||||
step-1 definition, steps/7f9c6a74b5891f8e.yml, served as the drill template for the install only); the
|
||||
drill template then back to the head (5.3.1 / 11.8). Two presses: press 1 must apply ONLY the app step
|
||||
(5.3.1 / 11.4, from steps/90dd9d68258286ef.yml), press 2 the engine step (11.4 -> 11.8, the template).
|
||||
Seeded data read back after each press; the page's steps-left line read in both languages each time."""
|
||||
import html as H, json, re, sys, time
|
||||
sys.path.insert(0, ".")
|
||||
import walk as w
|
||||
import fixtures as fx
|
||||
|
||||
F, SUB, APP = fx.Romm(), "arcade-ld", "romm"
|
||||
CAT = f"{w.DRILL}/templates/{APP}"
|
||||
out = {"controller": w.guest("cat /etc/felhom-controller-image").strip()}
|
||||
w.login()
|
||||
|
||||
|
||||
def drill(body, msg):
|
||||
import fcntl
|
||||
with open(f"{w.SC}/drill.lock", "w") as lk:
|
||||
fcntl.flock(lk, fcntl.LOCK_EX)
|
||||
w.sh(["git", "-C", w.DRILL, "pull", "-q", "--rebase", "origin", "main"], timeout=120)
|
||||
open(f"{CAT}/docker-compose.yml", "w").write(body)
|
||||
w.sh(["git", "-C", w.DRILL, "commit", "-qam", "LIVE-D " + msg])
|
||||
r = w.sh(["git", "-C", w.DRILL, "push", "-q", "origin", "main"], timeout=120)
|
||||
h = w.sh(["git", "-C", w.DRILL, "rev-parse", "--short=12", "HEAD"]).stdout.strip()
|
||||
w.say(f"drill {h}: {msg} push rc={r.returncode}")
|
||||
return h
|
||||
|
||||
|
||||
def steps_line():
|
||||
r = {}
|
||||
for lang, q in (("hu", ""), ("en", "?lang=en")):
|
||||
r[lang] = re.findall(r'data-ladder-steps="(\d+)">([^<]*)<', H.unescape(w.page(f"/apps/{APP}{q}")))
|
||||
return r
|
||||
|
||||
|
||||
def box_files():
|
||||
return w.guest(f"grep -nE 'image:|memory:' /opt/docker/stacks/{APP}/docker-compose.yml; echo ---; cmp -s /opt/docker/stacks/{APP}/applied-compose.yml /var/lib/docker/volumes/felhom-controller-data/_data/data/catalog-cache/templates/{APP}/steps/90dd9d68258286ef.yml && echo 'applied == steps/90dd9d68258286ef.yml' || echo 'applied != steps/90dd…'; cmp -s /opt/docker/stacks/{APP}/applied-compose.yml /var/lib/docker/volumes/felhom-controller-data/_data/data/catalog-cache/templates/{APP}/docker-compose.yml && echo 'applied == template' || echo 'applied != template'")
|
||||
|
||||
|
||||
HEAD = open(f"{CAT}/docker-compose.yml").read()
|
||||
STEP1 = open(f"{CAT}/steps/7f9c6a74b5891f8e.yml").read()
|
||||
out["drill_install"] = drill(STEP1, "romm template = step 1's own definition (5.3.0 / mariadb 11.4) for the install")
|
||||
w.sync_rescan()
|
||||
out["deployed"] = w.deploy(APP, SUB)
|
||||
A = F.seed(w, SUB, w.say)
|
||||
out["seed_A"] = bool(A)
|
||||
out["drill_head"] = drill(HEAD, "romm back to the head (5.3.1 / mariadb 11.8)")
|
||||
out["badge_wait"] = w.sync_rescan(APP, "mariadb:11.8", tries=60)
|
||||
out["before"] = {"obs": w.observables(APP), "steps_line": steps_line(), "badges": w.badges(APP)}
|
||||
w.say("before: steps line " + json.dumps(out["before"]["steps_line"], ensure_ascii=False) + " badges " + json.dumps(out["before"]["badges"], ensure_ascii=False)[:300])
|
||||
|
||||
for n in (1, 2):
|
||||
t0 = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
|
||||
p = w.press_update(APP)
|
||||
obs = w.observables(APP)
|
||||
w.wait_app(SUB, "/api/heartbeat", want=("200",), tries=60, delay=5)
|
||||
rb = bool(F.verify(w, SUB, A, w.say))
|
||||
log = w.guest(f"docker logs --since {t0} felhom-controller 2>&1 | grep -E 'update romm' | grep -E 'ladder|pin advanced|DONE|UNDO|healthy after' | tail -8")
|
||||
dblog = w.guest(f"docker logs --since {t0} romm-db 2>&1 | grep -iE 'upgrade|mariadb-upgrade|Version:' | tail -6")
|
||||
B = F.seed(w, SUB, w.say) if n == 1 else None
|
||||
out[f"press{n}"] = {"press": p, "obs": obs, "readback_A": rb, "log": log, "db_log": dblog,
|
||||
"files": box_files(), "steps_line": steps_line(), "badges": w.badges(APP)}
|
||||
if B:
|
||||
out["seed_B"] = B
|
||||
w.say(f"PRESS {n}: phase={p.get('final_phase')} pinned={obs.get('pinned_images')} readback A={rb}\n{log}\nDB: {dblog}\nfiles: {out[f'press{n}']['files']}\nsteps line: {out[f'press{n}']['steps_line']}")
|
||||
if out.get("seed_B"):
|
||||
out["readback_B_after_press2"] = bool(F.verify(w, SUB, out["seed_B"], w.say))
|
||||
json.dump(out, open("../partD/10-romm-two-steps.json", "w"), indent=2, ensure_ascii=False, default=str)
|
||||
open("../partD/10-romm-two-steps.log", "w").write("\n".join(w.LOG) + "\n")
|
||||
@@ -0,0 +1,60 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Point guest 9202 at the drill catalog (and a 90 s health timeout), or restore the saved config.
|
||||
|
||||
`09` §6.5: `git.repo_url` alone is INERT (R-615) — the cache dir must go too. The saved copy is
|
||||
`controller.yaml.pre-ladder0924` (NOT the older `.pre-28`, which a restore must never pick up).
|
||||
"""
|
||||
import re, sys, io
|
||||
sys.path.insert(0, '.')
|
||||
import walk as w
|
||||
|
||||
VOL = "/var/lib/docker/volumes/felhom-controller-data/_data"
|
||||
DRILL_REPO = "https://gitea.dooplex.hu/admin/app-catalog-drill.git"
|
||||
|
||||
|
||||
def creds():
|
||||
for l in io.open("/home/kisfenyo/.git-credentials").read().strip().split("\n"):
|
||||
m = re.match(r'https://(admin):([^@]+)@gitea\.dooplex\.hu', l)
|
||||
if m:
|
||||
return m.group(1), m.group(2)
|
||||
raise SystemExit("no admin credential")
|
||||
|
||||
|
||||
def to_drill():
|
||||
u, t = creds()
|
||||
print(w.guest(f"""
|
||||
set -e
|
||||
test -f {VOL}/controller.yaml.pre-ladder0924 || cp -p {VOL}/controller.yaml {VOL}/controller.yaml.pre-ladder0924
|
||||
python3 - <<'PY'
|
||||
import re
|
||||
p = "{VOL}/controller.yaml"
|
||||
s = open(p).read()
|
||||
s = re.sub(r'(^\\s+repo_url: ).*$', r'\\g<1>{DRILL_REPO}', s, count=1, flags=re.M)
|
||||
s = re.sub(r'(^git:(?:\\n\\s+.*)*?\\n\\s+token: ).*$', r'\\g<1>"{t}"', s, count=1, flags=re.M)
|
||||
s = re.sub(r'(^git:(?:\\n\\s+.*)*?\\n\\s+username: ).*$', r'\\g<1>"{u}"', s, count=1, flags=re.M)
|
||||
if not re.search(r'^update:', s, re.M):
|
||||
s += "update:\\n health_timeout: 90s\\n"
|
||||
open(p, "w").write(s)
|
||||
PY
|
||||
rm -rf {VOL}/catalog-cache {VOL}/data/catalog-cache
|
||||
docker restart felhom-controller >/dev/null
|
||||
sleep 15
|
||||
grep -A6 '^git:' {VOL}/controller.yaml | sed 's/token:.*/token: <redacted>/'
|
||||
grep -A2 '^update:' {VOL}/controller.yaml
|
||||
"""))
|
||||
|
||||
|
||||
def restore():
|
||||
print(w.guest(f"""
|
||||
set -e
|
||||
cp -p {VOL}/controller.yaml.pre-ladder0924 {VOL}/controller.yaml
|
||||
rm -rf {VOL}/catalog-cache {VOL}/data/catalog-cache
|
||||
docker restart felhom-controller >/dev/null
|
||||
sleep 15
|
||||
grep -A6 '^git:' {VOL}/controller.yaml | sed 's/token:.*/token: <redacted>/'
|
||||
grep -c '^update:' {VOL}/controller.yaml || true
|
||||
"""))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
to_drill() if sys.argv[1] == "drill" else restore()
|
||||
@@ -0,0 +1,60 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Part D spike (30 min) on guest 9202, controller v0.267.0, drill catalog.
|
||||
|
||||
1. an app two steps behind (vikunja A=2.4.0 -> B=2.5.0 -> C=2.6.0): does ONE press jump A -> C today?
|
||||
2. does a `templates/<app>/steps/` folder reach the box? (it is read from the catalog clone)
|
||||
Evidence only; every act is a product endpoint."""
|
||||
import json, os, sys, time
|
||||
sys.path.insert(0, ".")
|
||||
import walk as w
|
||||
|
||||
DRILL = w.DRILL
|
||||
CACHE = "/var/lib/docker/volumes/felhom-controller-data/_data/catalog-cache"
|
||||
|
||||
|
||||
def commit(msg, edit):
|
||||
edit()
|
||||
w.sh(["git", "-C", DRILL, "add", "-A"])
|
||||
w.sh(["git", "-C", DRILL, "commit", "-q", "-m", msg])
|
||||
r = w.sh(["git", "-C", DRILL, "push", "-q", "origin", "main"], timeout=120)
|
||||
h = w.sh(["git", "-C", DRILL, "rev-parse", "--short=12", "HEAD"]).stdout.strip()
|
||||
w.say(f"drill commit {h}: {msg} (push rc={r.returncode})")
|
||||
return h
|
||||
|
||||
|
||||
def set_img(ref):
|
||||
p = f"{DRILL}/templates/vikunja/docker-compose.yml"
|
||||
s = open(p).read()
|
||||
import re
|
||||
s = re.sub(r"image: vikunja/vikunja:\S+", f"image: vikunja/vikunja:{ref}", s)
|
||||
open(p, "w").write(s)
|
||||
|
||||
|
||||
out = {}
|
||||
commit("SPIKE vikunja A=2.4.0", lambda: set_img("2.4.0"))
|
||||
w.login()
|
||||
w.sync_rescan("vikunja", "vikunja/vikunja:2.4.0")
|
||||
ok = w.deploy("vikunja", "vikunja-sp")
|
||||
out["deployed_A"] = ok
|
||||
out["A"] = w.observables("vikunja")
|
||||
|
||||
|
||||
def step_b():
|
||||
set_img("2.5.0")
|
||||
os.makedirs(f"{DRILL}/templates/vikunja/steps", exist_ok=True)
|
||||
open(f"{DRILL}/templates/vikunja/steps/spike-B.yml", "w").write(
|
||||
open(f"{DRILL}/templates/vikunja/docker-compose.yml").read())
|
||||
|
||||
|
||||
commit("SPIKE vikunja B=2.5.0 + steps/spike-B.yml", step_b)
|
||||
commit("SPIKE vikunja C=2.6.0", lambda: set_img("2.6.0"))
|
||||
out["badge_wait_s"] = w.sync_rescan("vikunja", "vikunja/vikunja:2.6.0")
|
||||
out["cache_steps"] = w.guest(f"git -C {CACHE} rev-parse --short=12 HEAD; git -C {CACHE} rev-list --count HEAD; ls -la {CACHE}/templates/vikunja/ {CACHE}/templates/vikunja/steps/ 2>&1; ls /opt/docker/stacks/vikunja/")
|
||||
w.say("cache:\n" + out["cache_steps"])
|
||||
out["badges"] = w.badges("vikunja")
|
||||
out["press"] = w.press_update("vikunja")
|
||||
out["after"] = w.observables("vikunja")
|
||||
w.say("after: " + json.dumps(out["after"]))
|
||||
w.remove("vikunja")
|
||||
json.dump(out, open("../partD/00-spike.json", "w"), indent=2, ensure_ascii=False)
|
||||
open("../partD/00-spike.log", "w").write("\n".join(w.LOG) + "\n")
|
||||
@@ -0,0 +1,21 @@
|
||||
import sys, json, re
|
||||
sys.path.insert(0, ".")
|
||||
import walk as w
|
||||
w.login()
|
||||
import fcntl
|
||||
CAT = f"{w.DRILL}/templates/nextcloud"
|
||||
with open(f"{w.SC}/drill.lock", "w") as lk:
|
||||
fcntl.flock(lk, fcntl.LOCK_EX)
|
||||
w.sh(["git", "-C", w.DRILL, "pull", "-q", "--rebase", "origin", "main"], timeout=120)
|
||||
f = open(f"{CAT}/.felhom.yml").read()
|
||||
open(f"{CAT}/.felhom.yml", "w").write(re.sub(r"(healthcheck:\n\s+checks:\n\s+- type: api\n\s+port: )8999", r"\g<1>80", f, count=1))
|
||||
w.sh(["git", "-C", w.DRILL, "commit", "-qam", "LIVE-B teardown: nextcloud real probe back"])
|
||||
w.sh(["git", "-C", w.DRILL, "push", "-q", "origin", "main"], timeout=120)
|
||||
before = w.guest("docker volume ls -q | grep -E '^nextcloud' ; ls /opt/docker/stacks/nextcloud/")
|
||||
w.say("before remove:\n" + before)
|
||||
code = w.remove("nextcloud")
|
||||
after = w.guest("docker volume ls -q | grep -E '^nextcloud' || echo 'no nextcloud volumes (incl. undo copies)'; ls -la /opt/docker/stacks/nextcloud/; ls -d /mnt/felhom-drives/scratch_hdd/userdata/nextcloud /mnt/felhom-drives/scratch_hdd/appdata/nextcloud 2>&1")
|
||||
w.say("after remove:\n" + after)
|
||||
rm = w.guest("rm -rf /mnt/felhom-drives/scratch_hdd/userdata/nextcloud /mnt/felhom-drives/scratch_hdd/appdata/nextcloud && ls -d /mnt/felhom-drives/scratch_hdd/userdata/nextcloud 2>&1")
|
||||
w.say("drive folders removed by name (R-442 keeps them on 9202): " + rm.strip())
|
||||
open("../partB/07-teardown.log", "w").write("\n".join(w.LOG) + "\n")
|
||||
@@ -0,0 +1,507 @@
|
||||
#!/usr/bin/env python3
|
||||
"""walk.py — ONE app's full update walk on guest 9202, through the product's own endpoints.
|
||||
|
||||
EVIDENCE, NOT PRODUCT. It presses exactly the buttons a person presses:
|
||||
POST /api/stacks/<n>/deploy · POST /api/sync · POST /api/stacks/rescan
|
||||
POST /api/stacks/<n>/update · POST /api/stacks/<n>/remove
|
||||
and reads GET /api/stacks/<n>. No controller code exists for it.
|
||||
|
||||
The walk, per `09` §6.4 and the update-night brief §4:
|
||||
1 deploy from the DRILL catalog at the LIVE pin
|
||||
2 seed through the app's OWN front door (R-156: never a volume, never SQL)
|
||||
3 read the seed back <- control C1; a fixture that cannot prove itself proves nothing
|
||||
4 „Mentés most"
|
||||
5 commit the real one-step bump to the DRILL repo, sync, rescan, read the badge in BOTH languages
|
||||
6 press the guarded Update, record every phase with timestamps
|
||||
7 read the seed back through the front door
|
||||
8 the four version observables side by side
|
||||
9 write the verdict record in `09`'s JSON shape
|
||||
|
||||
`inconclusive` is a first-class verdict and is NEVER collapsed into `failed`.
|
||||
"""
|
||||
import argparse, json, os, re, subprocess, sys, time
|
||||
from datetime import datetime, timezone
|
||||
|
||||
SC = "/tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/25947a06-40b7-43dd-8334-2519e02142da/scratchpad"
|
||||
EV = "/mnt/5_hdd/felhom.eu/git/felhom.eu/documentation/audits/ladder-2026-09-24"
|
||||
DRILL = "/mnt/5_hdd/felhom.eu/drill/app-catalog-drill"
|
||||
# GUEST=9201 selects demo-hp's hub-enabled guest (the mail proof); default 9202, the scratch guest.
|
||||
GUEST = os.environ.get("GUEST", "9202")
|
||||
BASE = {"9202": "https://192.168.0.114", "9201": "https://192.168.0.138"}[GUEST]
|
||||
DOMAIN = os.environ.get("DOMAIN", "enkisfelhom.hu")
|
||||
HOSTHDR = f"Host: felhom.{DOMAIN}"
|
||||
HP = "demo-hp"
|
||||
|
||||
LOG = []
|
||||
|
||||
|
||||
def say(*a):
|
||||
line = " ".join(str(x) for x in a)
|
||||
ts = datetime.now().strftime("%H:%M:%S")
|
||||
print(f"{ts} {line}", flush=True)
|
||||
LOG.append(f"{ts} {line}")
|
||||
|
||||
|
||||
def sh(args, timeout=300, inp=None):
|
||||
try:
|
||||
return subprocess.run(args, capture_output=True, text=True, timeout=timeout, input=inp)
|
||||
except (subprocess.TimeoutExpired, OSError) as e:
|
||||
return subprocess.CompletedProcess(args, 124, "", f"{e}")
|
||||
|
||||
|
||||
def guest(script, timeout=600):
|
||||
"""Run a bash script inside guest 9202. Piped as a file — never as an argument (quoting)."""
|
||||
# ONE TEMP FILE PER CALL (night 2026-09-23): the shared /tmp/w<guest>.sh swapped scripts under
|
||||
# two concurrent walks (memory: guest-helper-shares-one-tmp-file).
|
||||
import secrets as _s
|
||||
t = f"/tmp/w{GUEST}-{os.getpid()}-{_s.token_hex(4)}.sh"
|
||||
r = sh(["ssh", "-o", "ConnectTimeout=20", "-o", "StrictHostKeyChecking=accept-new", HP,
|
||||
f"export LC_ALL=C; cat > {t}; pct push {GUEST} {t} {t} >/dev/null 2>&1; "
|
||||
f"pct exec {GUEST} -- bash {t}; pct exec {GUEST} -- rm -f {t}; rm -f {t}"],
|
||||
timeout=timeout, inp=script)
|
||||
return r.stdout or ""
|
||||
|
||||
|
||||
def login():
|
||||
pw = open(f"{SC}/.ctlpw").read().strip()
|
||||
sh(["curl", "-sk", "-D", f"{SC}/hdr{os.getpid()}.txt", "-o", "/dev/null", "-H", HOSTHDR,
|
||||
"-X", "POST", "--data-urlencode", f"password={pw}", f"{BASE}/login"])
|
||||
h = open(f"{SC}/hdr{os.getpid()}.txt").read()
|
||||
m = re.search(r"felhom_session=[A-Za-z0-9._-]+", h, re.I)
|
||||
if not m:
|
||||
sys.exit("login failed: no session cookie")
|
||||
open(f"{SC}/sess{os.getpid()}.txt", "w").write(m.group(0))
|
||||
r = sh(["curl", "-sk", "-L", "-H", HOSTHDR, "-H", f"Cookie: {m.group(0)}", f"{BASE}/"])
|
||||
c = re.search(r'<meta name="csrf-token" content="([^"]+)"', r.stdout or "")
|
||||
if not c:
|
||||
sys.exit("login failed: no csrf token")
|
||||
open(f"{SC}/csrf{os.getpid()}.txt", "w").write(c.group(1))
|
||||
|
||||
|
||||
def ctl(method, path, data=None, raw=False, tries=2):
|
||||
"""One controller API call. Re-logs in once on a 302/401 — the controller's session store is
|
||||
in memory, so any controller restart during the night invalidates it silently."""
|
||||
for attempt in range(tries):
|
||||
sess = open(f"{SC}/sess{os.getpid()}.txt").read().strip()
|
||||
csrf = open(f"{SC}/csrf{os.getpid()}.txt").read().strip()
|
||||
args = ["curl", "-sk", "-H", HOSTHDR, "-H", f"Cookie: {sess}", "-w", "\n%{http_code}"]
|
||||
if method != "GET":
|
||||
args += ["-H", f"X-CSRF-Token: {csrf}", "-H", "Content-Type: application/json",
|
||||
"-X", method]
|
||||
if data is not None:
|
||||
args += ["--data", json.dumps(data)]
|
||||
args.append(f"{BASE}{path}")
|
||||
r = sh(args)
|
||||
body, _, code = (r.stdout or "").rpartition("\n")
|
||||
if code.strip() in ("302", "401") and attempt + 1 < tries:
|
||||
login()
|
||||
continue
|
||||
if raw:
|
||||
return code.strip(), body
|
||||
try:
|
||||
return code.strip(), json.loads(body)
|
||||
except Exception:
|
||||
return code.strip(), {"_raw": body[:600]}
|
||||
return code.strip(), {"_raw": body[:600]}
|
||||
|
||||
|
||||
def page(path):
|
||||
sess = open(f"{SC}/sess{os.getpid()}.txt").read().strip()
|
||||
r = sh(["curl", "-sk", "-H", HOSTHDR, "-H", f"Cookie: {sess}", f"{BASE}{path}"])
|
||||
return r.stdout or ""
|
||||
|
||||
|
||||
def app_curl(sub, path, *extra, method=None, data=None, timeout=45):
|
||||
"""A call to the APP's own front door on 9202 — the household's route, not ours."""
|
||||
args = ["curl", "-sSk", "--max-time", str(timeout), "-H", f"Host: {sub}.{DOMAIN}",
|
||||
"-w", "\n%{http_code}"]
|
||||
if method:
|
||||
args += ["-X", method]
|
||||
if data is not None:
|
||||
args += ["--data-binary", "@-"]
|
||||
args += list(extra) + [f"{BASE}{path}"]
|
||||
r = sh(args, timeout=timeout + 30, inp=data)
|
||||
body, _, code = (r.stdout or "").rpartition("\n")
|
||||
return r.returncode, code.strip(), body
|
||||
|
||||
|
||||
def stack(name):
|
||||
_, d = ctl("GET", f"/api/stacks/{name}")
|
||||
return (d.get("data") or {}) if isinstance(d, dict) else {}
|
||||
|
||||
|
||||
def wait_app(sub, path="/", want=("200", "302", "303", "401", "403"), tries=60, delay=5):
|
||||
"""Settling says the container runs; this says the APP answers. Not the same thing."""
|
||||
last = None
|
||||
for _ in range(tries):
|
||||
rc, code, _ = app_curl(sub, path, timeout=15)
|
||||
last = (rc, code)
|
||||
if rc == 0 and code in want:
|
||||
return True
|
||||
time.sleep(delay)
|
||||
say(f" app never answered on {sub}{path} (last rc={last[0]} code={last[1]})")
|
||||
return False
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ the walk
|
||||
|
||||
|
||||
DRIVE = "/mnt/felhom-drives/scratch_hdd/userdata"
|
||||
|
||||
# What THIS run generated for a deploy, per app. Deploy secrets are ENCRYPTED AT REST in
|
||||
# `app.yaml` (`ENC:…`), which is right and which means a fixture cannot read an app's admin
|
||||
# password back off the box — the household sees it once. So the value the harness itself
|
||||
# generated is kept here for the life of the run, and nowhere else.
|
||||
GENERATED = {}
|
||||
|
||||
|
||||
def deploy_values(name, sub):
|
||||
"""Fill EVERY required deploy field the way the wizard would, by asking the box what this app
|
||||
asks for — `GET /api/stacks/<n>/deploy-fields` — instead of assuming DOMAIN+SUBDOMAIN.
|
||||
|
||||
Measured 2026-09-21: three apps in one batch refused at the deploy with a correct 400 because
|
||||
a required field was absent — `HDD_PATH` (navidrome, audiobookshelf) and an admin password
|
||||
(grafana). The refusals happen BEFORE anything is created (`deploy.go:324`), which is the only
|
||||
reason this was safe to discover by running it (live-probes rule).
|
||||
|
||||
A `path` field must name a directory that ALREADY EXISTS (`deploy.go:330`), so one is made on
|
||||
the scratch drive first — the same act the drive browser performs for a household.
|
||||
"""
|
||||
code, d = ctl("GET", f"/api/stacks/{name}/deploy-fields")
|
||||
fields = (((d.get("data") or {}).get("metadata") or {}).get("deploy_fields")) or []
|
||||
values = {"DOMAIN": DOMAIN, "SUBDOMAIN": sub}
|
||||
made = []
|
||||
for f in fields:
|
||||
ev, ty = f.get("env_var"), f.get("type")
|
||||
if ev in values:
|
||||
continue
|
||||
# `type: password` is MANDATORY whatever `required` says — `deploy.go:305-312` refuses
|
||||
# when the caller sends none, deliberately ("the user needs to know their password"),
|
||||
# while `.felhom.yml` declares `required: false` and the API serves that verbatim. A
|
||||
# caller that trusts the contract gets a 400. Measured tonight on grafana; filed.
|
||||
if not f.get("required") and ty != "password":
|
||||
continue # the controller generates the optional secrets itself
|
||||
if ty == "path":
|
||||
p = f"{DRIVE}/{name}"
|
||||
values[ev] = p
|
||||
made.append(p)
|
||||
elif ty in ("secret", "password"):
|
||||
import secrets as _s
|
||||
values[ev] = "Drill-" + _s.token_hex(12)
|
||||
GENERATED.setdefault(name, {})[ev] = values[ev]
|
||||
elif f.get("default"):
|
||||
values[ev] = f["default"]
|
||||
else:
|
||||
values[ev] = f"drill-{name}"
|
||||
if made:
|
||||
guest("mkdir -p " + " ".join(made) + "; ls -ld " + " ".join(made))
|
||||
say(f" [1] made the drive paths this app requires: {made}")
|
||||
extra = [k for k in values if k not in ("DOMAIN", "SUBDOMAIN")]
|
||||
if extra:
|
||||
say(f" [1] required fields filled beyond DOMAIN/SUBDOMAIN: {extra}")
|
||||
return values
|
||||
|
||||
|
||||
def deploy(name, sub, extra_values=None):
|
||||
st = stack(name)
|
||||
if st.get("deployed"):
|
||||
say(f" [1] {name} already deployed — reusing")
|
||||
return True
|
||||
values = deploy_values(name, sub)
|
||||
if extra_values:
|
||||
values.update(extra_values)
|
||||
code, d = ctl("POST", f"/api/stacks/{name}/deploy", {"values": values})
|
||||
say(f" [1] deploy -> {code} {str(d)[:120]}")
|
||||
if code != "202":
|
||||
return False
|
||||
# WAIT FOR `deployed`, NOT FOR `running`. Measured 2026-09-21 on tandoor: docker reported the
|
||||
# container `healthy` while the controller's own state read `unhealthy` — a gate on `running`
|
||||
# alone therefore times out on an app that is up. The state is RECORDED rather than required;
|
||||
# the real gate is the fixture's own `wait_app`, which asks whether the APP answers.
|
||||
seen = None
|
||||
for _ in range(90):
|
||||
time.sleep(5)
|
||||
st = stack(name)
|
||||
seen = st.get("state")
|
||||
# `deployed` alone is NOT enough and `state` alone is NOT right. Measured 2026-09-21:
|
||||
# tandoor reads `unhealthy` while serving (R-618), so gating on "running" hangs; and romm
|
||||
# read `deployed=True, state=degraded, pinned_images=None` twenty seconds in, i.e. the
|
||||
# deploy had not finished writing app.yaml. The PIN is the deploy's own completion mark
|
||||
# (`runComposeDeploy` writes it), so that is what to wait for.
|
||||
pins = (st.get("app_config") or {}).get("pinned_images")
|
||||
if st.get("deployed") and pins and seen in ("running", "unhealthy", "degraded"):
|
||||
say(f" [1] deployed, controller state={seen}, "
|
||||
f"pinned={(st.get('app_config') or {}).get('pinned_images')}")
|
||||
if seen != "running":
|
||||
say(f" [1] NOTE: the controller's own state is {seen!r}, not 'running' — recorded, "
|
||||
f"not treated as a failure; the fixture's front-door wait is the real gate")
|
||||
return True
|
||||
say(f" [1] never became deployed (last controller state={seen!r})")
|
||||
return False
|
||||
|
||||
|
||||
def backup_now(name):
|
||||
"""R-648 (2026-09-23): NO whole-box „Mentés most" from a drill, ever.
|
||||
|
||||
`POST /api/backup/run` is the only backup endpoint and it is WHOLE-BOX: on 9201 it stopped and
|
||||
restarted 9 of 10 standing apps twice, and on 9202 it broke a deploy in flight (R-634). The product
|
||||
has NO per-app backup endpoint (router.go: /backup/run, /backup/tier2 only); the per-app backup
|
||||
exists only inside the guarded update, whose `backing-up` phase calls RunAppBackupNow for the one
|
||||
app. So this presses nothing: the update takes the throwaway app's own backup, and says so in its
|
||||
phase list. A seed written "after the backup" is therefore written before the update's own backup
|
||||
— the undo's last-second copy is still the one that must bring it back."""
|
||||
say(f" [4] backup press SKIPPED for {name} (R-648: whole-box only; the update's backing-up phase backs up {name} alone)")
|
||||
return None
|
||||
|
||||
def drill_bump(app, frm, to, service_hint=None):
|
||||
"""Serialised across concurrent walks: one git working tree, one lock."""
|
||||
import fcntl
|
||||
with open(f"{SC}/drill.lock", "w") as lk:
|
||||
fcntl.flock(lk, fcntl.LOCK_EX)
|
||||
sh(["git", "-C", DRILL, "pull", "-q", "--rebase", "origin", "main"], timeout=120)
|
||||
return _drill_bump(app, frm, to, service_hint)
|
||||
|
||||
|
||||
def _drill_bump(app, frm, to, service_hint=None):
|
||||
"""Commit the edge to the DRILL repo. catalog_since set by hand (the drill repo has no gates).
|
||||
|
||||
`frm`/`to` may be comma-separated lists of the SAME length: an app whose own version lives in
|
||||
two images (adventurelog's backend and frontend) moves both in one edge, while its engine
|
||||
sidecar stays where it is — `09` §3b Q3's rule is per SERVICE, and an app-half edge must move
|
||||
every service that carries the app's own version and no others.
|
||||
"""
|
||||
comp = f"{DRILL}/templates/{app}/docker-compose.yml"
|
||||
fy = f"{DRILL}/templates/{app}/.felhom.yml"
|
||||
s = open(comp).read()
|
||||
froms = [x.strip() for x in frm.split(",") if x.strip()]
|
||||
tos = [x.strip() for x in to.split(",") if x.strip()]
|
||||
if len(froms) != len(tos):
|
||||
say(f" [5] from/to lists differ in length: {froms} vs {tos}")
|
||||
return None
|
||||
for f1, t1 in zip(froms, tos):
|
||||
if f"image: {f1}" not in s:
|
||||
say(f" [5] FROM ref not found in compose: {f1}")
|
||||
return None
|
||||
s = s.replace(f"image: {f1}", f"image: {t1}")
|
||||
open(comp, "w").write(s)
|
||||
f = open(fy).read()
|
||||
today = datetime.now().strftime("%Y-%m-%d")
|
||||
f = re.sub(r'^catalog_since:.*$', f'catalog_since: "{today}"', f, count=1, flags=re.M)
|
||||
open(fy, "w").write(f)
|
||||
sh(["git", "-C", DRILL, "add", "-A"])
|
||||
sh(["git", "-C", DRILL, "commit", "-q", "-m", f"DRILL {app}: {frm} -> {to}"])
|
||||
r = sh(["git", "-C", DRILL, "push", "-q", "origin", "main"], timeout=120)
|
||||
h = sh(["git", "-C", DRILL, "rev-parse", "--short=12", "HEAD"]).stdout.strip()
|
||||
say(f" [5] drill commit {h}: {app} {frm} -> {to} (push rc={r.returncode})")
|
||||
return h
|
||||
|
||||
|
||||
def sync_rescan(expect_app=None, expect_ref=None, tries=12, delay=5):
|
||||
"""Sync, rescan, and — when told what to expect — WAIT FOR THE BADGE TO CATCH UP.
|
||||
|
||||
R-607: `POST /api/sync` answers "nincs valtozas" while the catalog HAS moved, and
|
||||
`catalog_images` stays stale until a separate rescan. Tonight showed the rescan alone is not
|
||||
enough either: mealie's badge read "Naprakesz" seconds after its bump was pushed, and the
|
||||
Update that followed moved nothing and still reported "Frissitve". So when the caller knows
|
||||
which reference should appear, this polls for it and SAYS HOW LONG IT TOOK — which is the
|
||||
NUMBER R-607 asks for and has never had.
|
||||
"""
|
||||
t0 = time.time()
|
||||
ctl("POST", "/api/sync")
|
||||
time.sleep(2)
|
||||
ctl("POST", "/api/stacks/rescan")
|
||||
time.sleep(2)
|
||||
if not expect_app or not expect_ref:
|
||||
return None
|
||||
for i in range(tries):
|
||||
cat = stack(expect_app).get("catalog_images") or {}
|
||||
if expect_ref in cat.values():
|
||||
waited = round(time.time() - t0, 1)
|
||||
if i:
|
||||
say(f" [sync] the badge needed {waited}s and {i+1} sync+rescan rounds to catch up "
|
||||
f"to {expect_ref} — R-607's window, measured")
|
||||
return waited
|
||||
time.sleep(delay)
|
||||
ctl("POST", "/api/sync")
|
||||
time.sleep(1)
|
||||
ctl("POST", "/api/stacks/rescan")
|
||||
say(f" [sync] the badge NEVER caught up to {expect_ref} in {round(time.time()-t0,1)}s — "
|
||||
f"catalog_images = {stack(expect_app).get('catalog_images')}")
|
||||
return None
|
||||
|
||||
|
||||
def badges(name):
|
||||
out = {}
|
||||
for lang, suffix in (("hu", ""), ("en", "?lang=en")):
|
||||
h = page(f"/apps/{name}{suffix}")
|
||||
m = re.findall(r'<span class="tag tag-[^"]*"[^>]*title="([^"]*)"[^>]*>([^<]*)<', h)
|
||||
out[lang] = [{"title": a.strip(), "text": b.strip()} for a, b in m][:3]
|
||||
return out
|
||||
|
||||
|
||||
def press_update(name, poll=1.0, cap_s=1800):
|
||||
code, d = ctl("POST", f"/api/stacks/{name}/update")
|
||||
say(f" [6] Update -> {code} {str(d)[:220]}")
|
||||
if code not in ("202", "200"):
|
||||
return {"accepted": False, "http": code, "refusal": d, "phases": [], "duration_s": 0}
|
||||
phases, seen, t0 = [], None, time.time()
|
||||
while time.time() - t0 < cap_s:
|
||||
st = stack(name)
|
||||
ph = st.get("update_phase")
|
||||
if ph != seen:
|
||||
seen = ph
|
||||
rec = {"t": round(time.time() - t0, 1), "phase": ph,
|
||||
"label": st.get("update_phase_label"), "updating": st.get("updating"),
|
||||
"error": st.get("update_error"), "hold": st.get("hold_reason")}
|
||||
phases.append(rec)
|
||||
say(f" +{rec['t']:>6.1f}s phase={ph} label={rec['label']} "
|
||||
f"err={rec['error']} hold={rec['hold']}")
|
||||
if not st.get("updating") and ph in ("done", "failed", "undone", None) and time.time() - t0 > 3:
|
||||
break
|
||||
time.sleep(poll)
|
||||
st = stack(name)
|
||||
return {"accepted": True, "http": code, "phases": phases,
|
||||
"duration_s": round(time.time() - t0, 1),
|
||||
"final_phase": st.get("update_phase"), "update_error": st.get("update_error"),
|
||||
"hold_reason": st.get("hold_reason"), "state": st.get("state")}
|
||||
|
||||
|
||||
def observables(name):
|
||||
st = stack(name)
|
||||
ac = st.get("app_config") or {}
|
||||
live = guest(f"""
|
||||
grep -E '^\\s+image:' /opt/docker/stacks/{name}/docker-compose.yml 2>/dev/null | sed 's/^ *//'
|
||||
echo '---inspect---'
|
||||
for c in $(docker ps -a --filter label=com.docker.compose.project={name} --format '{{{{.Names}}}}'); do
|
||||
echo -n "$c "; docker inspect "$c" --format '{{{{.Config.Image}}}} running={{{{.State.Running}}}} restarts={{{{.RestartCount}}}}'
|
||||
done
|
||||
""")
|
||||
a, _, b = live.partition("---inspect---")
|
||||
return {
|
||||
"pinned_images": ac.get("pinned_images"),
|
||||
"installed_images": {k: (v.get("ref") if isinstance(v, dict) else v)
|
||||
for k, v in (ac.get("installed_images") or {}).items()},
|
||||
"catalog_images": st.get("catalog_images"),
|
||||
"live_compose_image_lines": [x for x in a.strip().splitlines() if x.strip()],
|
||||
"docker_inspect": [x for x in b.strip().splitlines() if x.strip()],
|
||||
}
|
||||
|
||||
|
||||
def app_logs(name, lines=400):
|
||||
"""The app's own container log, DECODED. The endpoint answers a JSON envelope whose `logs` is
|
||||
one string with escaped newlines — a scan over the envelope sees a single enormous line and
|
||||
finds nothing, which reads exactly like "the app printed no migration line" and is not. R-96
|
||||
rule 3 in a new place: an absent line is not evidence when the instrument cannot see lines."""
|
||||
code, d = ctl("GET", f"/api/stacks/{name}/logs?lines={lines}")
|
||||
if isinstance(d, dict):
|
||||
data = d.get("data")
|
||||
if isinstance(data, dict) and isinstance(data.get("logs"), str):
|
||||
return data["logs"]
|
||||
if isinstance(d.get("_raw"), str):
|
||||
return d["_raw"]
|
||||
return str(d)
|
||||
|
||||
|
||||
def write_verdict(rec, appdir):
|
||||
os.makedirs(appdir, exist_ok=True)
|
||||
p = os.path.join(appdir, "verdict.json")
|
||||
json.dump(rec, open(p, "w"), indent=2, ensure_ascii=False)
|
||||
say(f" [9] verdict {rec['verdict']} -> {p}")
|
||||
|
||||
|
||||
def remove(name):
|
||||
"""Remove through the PRODUCT, never `docker rm` (live-probes rule). The remove endpoint
|
||||
refuses a running stack — `409 still running` — so the stop is part of the act, not a tidy-up."""
|
||||
c1, d1 = ctl("POST", f"/api/stacks/{name}/stop")
|
||||
say(f" [X] stop -> {c1} {str(d1)[:100]}")
|
||||
for _ in range(24):
|
||||
time.sleep(5)
|
||||
if stack(name).get("state") != "running":
|
||||
break
|
||||
code, d = ctl("POST", f"/api/stacks/{name}/remove",
|
||||
{"remove_hdd_data": True, "remove_backups": True})
|
||||
say(f" [X] remove (with drive data) -> {code} {str(d)[:160]}")
|
||||
if code == "409":
|
||||
# R-442's fail-closed guard: when the storage subsystem cannot RESOLVE the app's drive
|
||||
# path, the removal is REFUSED and the app is kept rather than half-deleted. On guest 9202
|
||||
# `/api/disks` answers `agent not configured`, so every app deployed with an HDD_PATH hits
|
||||
# this. The household's other choice — remove the app, KEEP the data — is accepted, and the
|
||||
# harness takes it, then tidies its own directory by name at teardown.
|
||||
say(" [X] refused because the drive path cannot be resolved (R-442, fail-closed and right)"
|
||||
" — removing the app and KEEPING the drive data instead")
|
||||
code, d = ctl("POST", f"/api/stacks/{name}/remove",
|
||||
{"remove_hdd_data": False, "remove_backups": True})
|
||||
say(f" [X] remove (keeping drive data) -> {code} {str(d)[:160]}")
|
||||
time.sleep(5)
|
||||
st = stack(name)
|
||||
left = guest(f"ls -d /opt/docker/stacks/{name} 2>/dev/null; "
|
||||
f"docker ps -a --filter label=com.docker.compose.project={name} --format '{{{{.Names}}}}'")
|
||||
say(f" [X] after remove: deployed={st.get('deployed')} leftovers={left.strip()!r}")
|
||||
return code
|
||||
|
||||
|
||||
def app_env(name, key):
|
||||
"""Read one deploy value the CUSTOMER was given (e.g. the generated admin password) from the
|
||||
app's own `app.yaml`. This is not seeding — it is how the household logs in; the controller
|
||||
shows them the same value. Data still goes in through the app's own front door."""
|
||||
out = guest(f"grep -E '^\\s*{key}:' /opt/docker/stacks/{name}/app.yaml 2>/dev/null | head -1")
|
||||
if ":" in out:
|
||||
return out.split(":", 1)[1].strip().strip('"').strip("'")
|
||||
return ""
|
||||
|
||||
|
||||
def snapshots(name):
|
||||
"""The restorable copies the backups page offers for this app."""
|
||||
code, d = ctl("GET", f"/api/backup/snapshots?stack={name}")
|
||||
data = d.get("data") if isinstance(d, dict) else None
|
||||
if isinstance(data, dict):
|
||||
for k in ("snapshots", "items", "restore_points"):
|
||||
if isinstance(data.get(k), list):
|
||||
return data[k]
|
||||
return data if isinstance(data, list) else []
|
||||
|
||||
|
||||
def restore(name, snapshot_id=None, wait_s=1200):
|
||||
"""The household's own way out: the „Visszaállítás a mentésből" button on the backups page.
|
||||
|
||||
A FORM post, not an API call — `POST /backup/restore` with `_csrf`, `stack_name`,
|
||||
`snapshot_id` — because that is the button the sentence tells them to press.
|
||||
"""
|
||||
snaps = snapshots(name)
|
||||
if snapshot_id is None:
|
||||
if not snaps:
|
||||
say(f" [R] no restorable copy offered for {name}")
|
||||
return {"ok": False, "why": "no snapshot offered", "snapshots": snaps}
|
||||
first = snaps[0]
|
||||
snapshot_id = first.get("id") or first.get("snapshot_id") or first.get("short_id")
|
||||
say(f" [R] restoring {name} from snapshot {snapshot_id!r} (of {len(snaps)} offered)")
|
||||
sess = open(f"{SC}/sess{os.getpid()}.txt").read().strip()
|
||||
csrf = open(f"{SC}/csrf{os.getpid()}.txt").read().strip()
|
||||
r = sh(["curl", "-sk", "-D", "-", "-o", "/dev/null", "-H", HOSTHDR, "-H", f"Cookie: {sess}",
|
||||
"-X", "POST",
|
||||
"--data-urlencode", f"_csrf={csrf}",
|
||||
"--data-urlencode", f"stack_name={name}",
|
||||
"--data-urlencode", f"snapshot_id={snapshot_id}",
|
||||
f"{BASE}/backup/restore"], timeout=180)
|
||||
head = (r.stdout or "").split("\n")[0].strip()
|
||||
loc = [l for l in (r.stdout or "").split("\n") if l.lower().startswith("location:")]
|
||||
say(f" [R] POST /backup/restore -> {head} {loc[:1]}")
|
||||
t0 = time.time()
|
||||
last = None
|
||||
while time.time() - t0 < wait_s:
|
||||
code, d = ctl("GET", "/api/backup/restore-status")
|
||||
dd = d.get("data") or {}
|
||||
cur = (dd.get("running"), dd.get("phase") or dd.get("state"), dd.get("message"))
|
||||
if cur != last:
|
||||
say(f" +{round(time.time()-t0,1):>6.1f}s restore {cur}")
|
||||
last = cur
|
||||
if not dd.get("running", False) and time.time() - t0 > 5:
|
||||
break
|
||||
time.sleep(2)
|
||||
st = stack(name)
|
||||
say(f" [R] after restore: state={st.get('state')} hold={st.get('hold_reason')!r} "
|
||||
f"phase={st.get('update_phase')}")
|
||||
return {"ok": True, "snapshot_id": snapshot_id, "snapshots": snaps,
|
||||
"http": head, "location": loc[:1], "seconds": round(time.time() - t0, 1),
|
||||
"state_after": st.get("state"), "hold_after": st.get("hold_reason"),
|
||||
"observables_after": observables(name)}
|
||||
Reference in New Issue
Block a user