Evening 2026-10-18: fresh install proven (R-1148), OnlyOffice walk (R-1018), one-sign-in spike (R-1100 READY), Redis volumes (R-1117), CI rows, golden 0.331.0 baked; register 168 -> 168 (closed 6, opened R-1149..R-1154)
gates / gates (push) Successful in 7m34s
gates / gates (push) Successful in 7m34s
This commit is contained in:
@@ -16,6 +16,14 @@
|
||||
> and holds nothing of its own; this file does hold its own content, namely the standing rulings below.
|
||||
|
||||
|
||||
> **2026-10-18 (evening) — controller 0.331.0 is the floor (MinAgent 0.158.1): OnlyOffice opens and saves from FileBrowser
|
||||
> and Nextcloud, FileBrowser back at once after a restart (R-1018), a removal deletes a volume the definition dropped
|
||||
> (R-1117); catalog `30eea97`: Immich/Nextcloud Redis without a volume; golden 0.331.0 baked, NOT vouched; a fresh install
|
||||
> from the ISO proved installer 1.34.0 + vouched golden/agent + auto-floor (R-1148); agent `main` carries R-978, R-973,
|
||||
> R-1093 unreleased. *Decided by CC unattended — operator may reverse:* the R-973 cap numbers (3 successful daily-tier runs
|
||||
> per 24 h, one weekly off-site run per 6 days; `07` §6.1). Tester4 is kept as the standing fresh-install customer
|
||||
> (operator). `REPORT-evening-2026-10-18.md`.**
|
||||
|
||||
> **2026-10-18 (day) — hub 0.158.0 DEPLOYED (access log live, agent-only vouch, R-895 first verdict 2026-10-20);
|
||||
> controller 0.330.0 is the floor (R-1144); agent 0.159.2 released and on the three test hosts, NOT vouched; installer
|
||||
> 1.34.0 published (vault-first step 4b, R-1145; fresh install unproven, R-1148); estimator Immich peak 3,983 MB
|
||||
|
||||
@@ -0,0 +1,149 @@
|
||||
# REPORT — evening → night 2026-10-18
|
||||
|
||||
## How I read the brief
|
||||
|
||||
- **Part 5, R-1135:** the brief says the mail should use the public Gitea URL — the register row says the same, and that
|
||||
is what was done (the CI helper first read the brief's wording the other way round and followed the row).
|
||||
- **R-1052** was already closed (2026-10-15); Part 1 proves R-1148 and re-proves R-1052's path with tonight's golden.
|
||||
- **Part 6, Immich:** 9202's Immich lives on `/mnt/sizing-bench`, a hand-made bench folder outside `/mnt/felhom-drives/`,
|
||||
so the allow-list refuses its definition (see Observations); the restore half of the measurement ran on Nextcloud.
|
||||
- **The operator kept Tester4** (asked at 21:20): not deleted; its old host must go before its next install (R-1154).
|
||||
|
||||
## In plain words
|
||||
|
||||
- **The morning list:** vouch golden 0.331.0; say yes or no to the agent release with three fixes; two decisions (one
|
||||
sign-in, local backup retention).
|
||||
- **A brand-new box from nothing works end to end** with installer 1.34.0: emergency password saved in the hub first,
|
||||
then set, then the web login took it; the approved golden and agent; the box moved itself to the floor.
|
||||
- **OnlyOffice now really works** from FileBrowser and from Nextcloud. Before tonight FileBrowser could not open a
|
||||
document and Nextcloud lost every save while saying „saved". Fixed in controller 0.331.0, now the floor for every box.
|
||||
- **One sign-in for the file manager can be built safely**, but only with two safeguards — measured, design note written.
|
||||
- **Three agent fixes are written and tested, not released.**
|
||||
- **CI:** the failure mail's link opens from outside; the 5-minute CI timeout never worked (filed); a rerun hides the
|
||||
first try's log, and the check now looks for it.
|
||||
- **Redis for Immich and Nextcloud has no disk volume any more**, after a test; the test found that removing an app left
|
||||
an old volume behind — fixed too.
|
||||
- **Golden 0.331.0 is baked** (not vouched).
|
||||
- **The old-restic check:** §9.
|
||||
|
||||
## 1. The morning list
|
||||
|
||||
1. **Vouch golden 0.331.0** — sha256 `3881de62b0986ccbe5e55580e3f1bb9765d65b8d898f67c6339bac105244f84f` (registry round
|
||||
trip = the bake), agent 0.159.2 unchanged, **Min agent 0.158.1**.
|
||||
2. **Agent release (R-978, R-973, R-1093) — may I?** If yes: release, prove on demo-hp, ask for the vouch.
|
||||
3. **Decide D16** (R-1100: build one sign-in with the two safeguards?) and **D17** (R-1149: local whole-box backups
|
||||
„last 3" or „one per day, 3 days"?). STATUS has both.
|
||||
4. Before Tester4's next install: delete its old host `Tester4-31fa0e` (Hosts page).
|
||||
|
||||
## 2. Fresh install from nothing (R-1148, R-1052) — `audits/newbox-2026-10-18/`
|
||||
|
||||
| UTC | Step | Result |
|
||||
|---|---|---|
|
||||
| 17:42:21 | Tester4 created by the operator | bind mail sent |
|
||||
| 17:43:15 | VM 9302 `newbox-r1148` on demo-hp, ISO 1.29.0 (sha = published) | 8 GB / 4 cores / 120 GB on `/mnt/hdd_1` root |
|
||||
| 17:50:19 | Install pressed (graphical, Hungarian keyboard kept) | installed, rebooted, pairing code on the console |
|
||||
| 18:02:58 | bind: mailed link + pairing code + owner phrase (operator's 0600 file, shredded) | „Sikeres összekötés" |
|
||||
| 18:03:43 | step 4b: vault → chpasswd → PVE login | `[OK] … vaulted, set, and the PVE login takes it`; hub `set_at` same second |
|
||||
| 18:03:44 | step 5: the hub manifest | agent **0.159.2**, golden **0.329.0** — the vouched pair, both sha-verified |
|
||||
| 18:05:37 → 18:05:51 | first controller reports | 0.329.0, then **0.330.0** (`initiated_by: auto-floor` at 18:05:38) — 2 min 47 s after the bind |
|
||||
| 18:17:14 | CC's PVE login from DooPlex | vaulted value **200**, installer value **401** |
|
||||
| 18:18:32 → 18:20:05 | claim (setup code mail), BentoPDF | claimed; BentoPDF healthy |
|
||||
| 18:24:15 | household „turn off" | hub `deliberate_off_at` set; VM stopped |
|
||||
| — | teardown | name read twice, VM 9302 destroyed, disk gone; other VMs untouched; Tester4 kept (operator) |
|
||||
|
||||
## 3. OnlyOffice (R-1018) — `audits/onlyoffice-2026-10-18/`
|
||||
|
||||
On 0.330.0: FileBrowser → the editor loaded, the document never did (OnlyOffice fetched `https://files.<domain>` → 403);
|
||||
Nextcloud → edit and „All changes saved", but the file kept the old text (`http://onlyoffice:80cache/…`). Switching the
|
||||
link on/off left files.<domain> 502 for 4 min 8 s / 3 min 38 s. **Fixed (controller 0.331.0):** FileBrowser
|
||||
`server.internalUrl`, Nextcloud's slash, the gate after a FileBrowser restart. On rc.1: both open, edit, save, read back
|
||||
(„… RC1-FB-OK", „… RC1-NC-OK"); the stop revokes both (no editor; Nextcloud 404); the stop's outage one 2-s sample.
|
||||
|
||||
## 4. One sign-in spike (R-1100 → READY) — `audits/proxyauth-spike-2026-10-18/DESIGN-NOTE.md`
|
||||
|
||||
Traefik forwardAuth strips a client's header (any case, twice) and its own value wins — measured. But an app container
|
||||
reaching FileBrowser directly logs in with a forged header (200), and the dashboard cookie never reaches `files.`. Verdict:
|
||||
build only with a files-only ticket cookie and a per-box secret header name. Everything on 9202 removed (users, routes,
|
||||
stand-in container, config).
|
||||
|
||||
## 5. Agent fixes (committed, NOT released) — felhom-agent `REPORT.md`
|
||||
|
||||
R-1093 `f7b7308` + `0963bae` (and a fail-open hole closed after a background review), R-978 `8561dc8`, R-973 `f17e197`;
|
||||
tests and red-proofs for each; CI job 2062 success. R-1093 live: a loop disk on demo-hp showed the probe's condition holds.
|
||||
|
||||
## 6. CI (R-1135, R-1136, R-1137 closed)
|
||||
|
||||
Link public and by run id (forced mail 18:46:01Z, job 2067, one mail). The runner's real limit is 3 h; the job-level
|
||||
5-minute timeout is not enforced (R-265 corrected; R-1152 for the step-level one). A rerun replaces the job's attempt; the
|
||||
pull-check recipe in `CLAUDE.md` now reads `actions/tasks` (instruction-file edit by the CI helper: felhom.eu `CLAUDE.md`
|
||||
CI recipe, item 3 added — a proven fact, stricter; `b9b56c47`).
|
||||
|
||||
## 7. Redis volumes (R-1117 closed) — `audits/redis-volumes-2026-10-18/`
|
||||
|
||||
(a) Both apps healthy on the new shape; Immich's metadata job ran (0 → 11, 0 failed); Nextcloud `cron.php` rc 0, WebDAV
|
||||
200. (b) The dump skips the dropped volume (the old tar gone from the unit); a second-drive restore from a copy made WITH
|
||||
the volume: 3/3 volumes, 1/1 database, healthy — it brings back that copy's compose (by design). (c) The old volume stays
|
||||
attached under the tmpfs; **a removal left it behind** — fixed (controller 0.331.0, red-proofed). Templates changed
|
||||
(catalog `30eea97`); Nextcloud on 9202 took it through the sync + restart (tmpfs, `save ""`).
|
||||
|
||||
## 8. Controller release and golden — `audits/evening-2026-10-18/P7/`, `tests/golden-0.331.0-2026-10-18/`
|
||||
|
||||
0.331.0 (digest `1b11d25a…`): per-box floors 20:44:00 → all three on 0.331.0 within 20 s; box walk green (0 refusals,
|
||||
0 unhealthy, FileBrowser re-published in 5–6 s, public pages 200 after one 20-s blip each on two names, https-only 28 OK);
|
||||
**global floor 0.331.0 / 0.158.1 at 20:48:56 UTC**, read back with the WAL; per-box floors cleared. Golden 0.331.0 baked
|
||||
after it, round trip = bake sha, token checks 0 with a working control, teardown clean. No controller push after the bake.
|
||||
|
||||
## 9. The old-restic check (Part 8)
|
||||
|
||||
PENDING — filled in when demo-hp's clean-up window has run (after its 04:15 CEST night run). A watcher runs the check by
|
||||
itself; at 20:57 UTC both restics listed 310 snapshots (`audits/evening-2026-10-18/P8-precheck.txt`).
|
||||
|
||||
## 10. Versions, commits, fixtures, rows
|
||||
|
||||
- Controller **0.331.0** — `772ef36`, `8ae9178` (+ `615940d` CI); rc.1 on 9202 only. Agent `main` `f7b7308` `8561dc8`
|
||||
`0963bae` `f17e197` `79ba884` `1693da4` — unreleased. Catalog `c97eb1c`, `30eea97`. felhom.eu `1fe9f848`, `b9b56c47`,
|
||||
this commit. Golden **0.331.0** baked. Hub unchanged (0.158.0).
|
||||
- Fixtures: none changed — no page changed.
|
||||
- **Register: 168 before, 168 after; closed 6** (R-1148, R-1018, R-1117, R-1135, R-1136, R-1137), **opened 6**
|
||||
(R-1149, R-1150, R-1151, R-1152, R-1153, R-1154). Updated: R-1100 (READY), R-978, R-973, R-1093 (fixed in agent
|
||||
`main`), R-265 (corrected).
|
||||
|
||||
## Observations
|
||||
|
||||
1. Installer password, golden and agent all proven in one fresh install. — closes R-1148 (not a finding).
|
||||
2. FileBrowser's office preview answered 500 with the link on. FILED: R-1153.
|
||||
3. A reinstall reuses an old host and skips two re-issues; now recurring with the kept Tester4. FILED: R-1154.
|
||||
4. The R-973 cap leaves the local keep-last 3 floodable within a day. FILED: R-1149.
|
||||
5. The cap's refusal reaches the operator with a misleading event text. FILED: R-1150.
|
||||
6. R-978's typed-back confirmation is browser-only. FILED: R-1151.
|
||||
7. The CI job timeout is inert. FILED: R-1152.
|
||||
8. 9202's Immich sits on `/mnt/sizing-bench`, outside `/mnt/felhom-drives/`, so the allow-list refuses its definition every
|
||||
15 minutes since 0.328.0 and a restore of it is refused. NOT-A-FINDING: the check works as designed; the folder is a
|
||||
hand-made bench artifact (R-994), not a shape a real box has.
|
||||
9. A restore brings back the backup's own compose (the old Redis volume with it). NOT-A-FINDING: by design (R-47: the
|
||||
unit's compose becomes the live one); the older shape works.
|
||||
10. My commit `7f7081d1` (day session) carries a „Co-Authored-By" trailer, against both briefs' rule 10. NOT-A-FINDING:
|
||||
my slip; it is pushed and cannot be rewritten without a force-push (forbidden); no later commit has one.
|
||||
11. I started the catalog's FULL gate run on DooPlex, whose third gate starts containers. NOT-A-FINDING: stopped at gate 2
|
||||
(a network check), no container was made (checked), the fast run used instead.
|
||||
12. One OnlyOffice „Connection is lost" on the first rc.1 attempt, after two blind clicks. NOT-A-FINDING: not reproduced;
|
||||
the next attempt saved; no container restarted in between.
|
||||
13. Two public names answered 000 once in the box walk. NOT-A-FINDING: 18 of 18 retries answered 200 in < 0.2 s.
|
||||
14. The R-1100 row named the config key `proxyAuth`; it is `auth.methods.proxy`. NOT-A-FINDING: corrected in the row.
|
||||
|
||||
## Teardown — three layers
|
||||
|
||||
- **Machines:** VM 9302 destroyed (its disk gone). 9202: OnlyOffice and Nextcloud removed through the product (their
|
||||
volumes, data and backup copies), the test documents deleted, the spike's route file, container, FileBrowser users and
|
||||
proxy setting removed, helper files and the Immich scratch password shredded; Immich keeps the new Redis shape by hand
|
||||
(= the template); 9202 on controller 0.331.0. `nextcloud_nextcloud_redis_data` (the orphan the pre-fix
|
||||
removal left) removed by name afterwards; no Nextcloud volume remains. demo-hp host: the loop-disk file removed. Drill VM: CT 9100 purged, VM off,
|
||||
disk `virgin`.
|
||||
- **Hub:** global floor 0.331.0 / 0.158.1, no per-box floor; Tester4 and its host `Tester4-31fa0e` kept (operator); no
|
||||
vouch, no deploy.
|
||||
- **DooPlex:** builds and the golden bake only; the headless browser as a test client; no Docker cleanup.
|
||||
|
||||
Architecture read: `01`, `02`, `03` §8a, `04`, `05`, `07` §6/§8, `09` §3 decisions 162/250; `target-selection.md`,
|
||||
`nightly-rotation.md`, `RUNBOOK-manual-build.md` §4. Documents changed: `03` §8a, `07` §6.1 and §8 row 10,
|
||||
`target-selection.md` (Tester4). System poster: none of the three (no listed fact changed). `unproven.py --summary`:
|
||||
not walked 35 of 55 — unchanged.
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
**Ready for the first real tester (Tester-2): yes. Tester 2 (a laptop) is off; nothing was sent to it.**
|
||||
|
||||
**Updated 2026-10-18 (day).** Hub 0.158.0 is live (your yes): its own 90-day access log works and writes no secret (tested with a fake token); you can vouch an agent without a new golden; the off-site snapshot check gives its first verdict on 20 October. Controller 0.330.0 is the floor (13:10 local): if the host refuses „retire anyway", the drive's apps start again. Agent 0.159.2 runs on the three test hosts (your yes; no pool check on a good backup) — **vouch it**. Installer 1.34.0 is published (your yes): the emergency password is saved in the hub first, then set. The calculator shows Immich's bigger figure (your yes). A fresh reviewer read last night's code: one medium finding, already closed by today's installer change — `REPORT-day-2026-10-18.md`. **Earlier:** **Updated 2026-10-18 (night).** Controller 0.329.0 is the floor for every box (21:36 UTC, MinAgent 0.158.1), after the three test boxes: each app reaches only its own data folder on the drive; homepage no longer controls Docker, so no app has a special right; when the host refuses to retire a drive, nothing changes and the household reads why; a virtual disk without a serial gets a plain sentence. Immich gets 3 GB (it was killed twice at 2 GB on a 4,000-photo import). Golden 0.329.0 is baked, NOT vouched. Hub 0.158.0 is built, NOT deployed — `REPORT-night-2026-10-18.md`. **Earlier:** **Updated 2026-10-17 (evening).** Controller 0.328.0 is the floor for every box (17:22 local): the box refuses a catalog change that asks for dangerous container rights (your D2 = A). Agent 0.159.1 (a gone, decommissioned drive is forgotten; an orphan thin snapshot is named in the backup failure; new Go) — runs on demo-hp, Tester 1 and demo-felhom; **your vouch in the hub is still open**. New emergency root passwords on demo-hp, Tester 1, demo-felhom (your D1 = A; read them in the hub). Hub 0.157.0 deployed on your yes — `REPORT-agent-allowlist-rotation-2026-10-17.md`. **Earlier:** **Updated 2026-10-17 (afternoon, later).** Controller 0.327.1 is the floor for every box since 12:06 UTC: FileBrowser is published only after its default password is checked (your D11 = A), and the install check uses real memory amounts with a clear message (your D10 = A) — `REPORT-filebrowser-memory-ci-2026-10-17.md`. **Earlier:** Controller 0.326.0 is the floor for every box since 09:55 UTC: FileBrowser's default password is checked at every FileBrowser start. Tester 1 is back on the live app list (test apps and drives removed); six memory limits raised — `REPORT-tester1-limits-filebrowser-2026-10-17.md`. **Earlier today:** controller 0.325.0 was the floor since 08:21 UTC (after the three test boxes): two more password doors count a try before checking it, family sign-ins are stored as fingerprints, a file-share rename always applies; five apps' Redis saves snapshots now, which a full disk cannot damage — `REPORT-burndown-3-2026-10-17.md`. **Earlier (2026-10-17 night):** controller 0.324.0 was the floor for every box since 02:46 UTC (all three test boxes made tonight's off-site backup on it first): a restore now brings the database password back with the data (your 1A), restic 0.19.1 (your 2A), dashboard hardening — `REPORT-night-2026-10-17.md`. **Earlier (2026-10-16 evening):** 0.323.0, Debian 13 — `REPORT-debian13-restic-2026-10-16.md`. **Earlier (midday):** Controller 0.319.0 (the app page in panels, the login in one panel) is the floor, MinAgent 0.158.1 — `REPORT-app-page-tuning-2026-10-16.md`. **Earlier (morning):** Storage identity build B shipped: agent 0.158.1 and golden 0.318.0 vouched by you, controller 0.318.0 is the floor (MinAgent 0.158.1), installer 1.33.0 published — new one-disk boxes get their own data area. Open items: see `documentation/backlog/OPEN-ITEMS.md`. Older notes, day by day:
|
||||
**Updated 2026-10-18 (evening).** A brand-new box was installed from nothing on demo-hp (your test customer Tester4): the emergency password was saved in the hub first, then set, and the web login took it; the box came up on the approved golden and moved to the floor by itself in under 3 minutes; its agent was the approved one. OnlyOffice now opens AND saves documents from FileBrowser and from Nextcloud (before tonight: FileBrowser could not open one, and Nextcloud lost every save while saying „saved") — controller 0.331.0 is the floor for every box (22:48 local). Immich's and Nextcloud's Redis no longer have a disk volume. Three agent fixes are written and tested, NOT released (your yes). Golden 0.331.0 is baked, NOT vouched — `REPORT-evening-2026-10-18.md`. **Earlier:** **Updated 2026-10-18 (day).** Hub 0.158.0 is live (your yes): its own 90-day access log works and writes no secret (tested with a fake token); you can vouch an agent without a new golden; the off-site snapshot check gives its first verdict on 20 October. Controller 0.330.0 is the floor (13:10 local): if the host refuses „retire anyway", the drive's apps start again. Agent 0.159.2 runs on the three test hosts (your yes; no pool check on a good backup) — **vouch it**. Installer 1.34.0 is published (your yes): the emergency password is saved in the hub first, then set. The calculator shows Immich's bigger figure (your yes). A fresh reviewer read last night's code: one medium finding, already closed by today's installer change — `REPORT-day-2026-10-18.md`. **Earlier:** **Updated 2026-10-18 (night).** Controller 0.329.0 is the floor for every box (21:36 UTC, MinAgent 0.158.1), after the three test boxes: each app reaches only its own data folder on the drive; homepage no longer controls Docker, so no app has a special right; when the host refuses to retire a drive, nothing changes and the household reads why; a virtual disk without a serial gets a plain sentence. Immich gets 3 GB (it was killed twice at 2 GB on a 4,000-photo import). Golden 0.329.0 is baked, NOT vouched. Hub 0.158.0 is built, NOT deployed — `REPORT-night-2026-10-18.md`. **Earlier:** **Updated 2026-10-17 (evening).** Controller 0.328.0 is the floor for every box (17:22 local): the box refuses a catalog change that asks for dangerous container rights (your D2 = A). Agent 0.159.1 (a gone, decommissioned drive is forgotten; an orphan thin snapshot is named in the backup failure; new Go) — runs on demo-hp, Tester 1 and demo-felhom; **your vouch in the hub is still open**. New emergency root passwords on demo-hp, Tester 1, demo-felhom (your D1 = A; read them in the hub). Hub 0.157.0 deployed on your yes — `REPORT-agent-allowlist-rotation-2026-10-17.md`. **Earlier:** **Updated 2026-10-17 (afternoon, later).** Controller 0.327.1 is the floor for every box since 12:06 UTC: FileBrowser is published only after its default password is checked (your D11 = A), and the install check uses real memory amounts with a clear message (your D10 = A) — `REPORT-filebrowser-memory-ci-2026-10-17.md`. **Earlier:** Controller 0.326.0 is the floor for every box since 09:55 UTC: FileBrowser's default password is checked at every FileBrowser start. Tester 1 is back on the live app list (test apps and drives removed); six memory limits raised — `REPORT-tester1-limits-filebrowser-2026-10-17.md`. **Earlier today:** controller 0.325.0 was the floor since 08:21 UTC (after the three test boxes): two more password doors count a try before checking it, family sign-ins are stored as fingerprints, a file-share rename always applies; five apps' Redis saves snapshots now, which a full disk cannot damage — `REPORT-burndown-3-2026-10-17.md`. **Earlier (2026-10-17 night):** controller 0.324.0 was the floor for every box since 02:46 UTC (all three test boxes made tonight's off-site backup on it first): a restore now brings the database password back with the data (your 1A), restic 0.19.1 (your 2A), dashboard hardening — `REPORT-night-2026-10-17.md`. **Earlier (2026-10-16 evening):** 0.323.0, Debian 13 — `REPORT-debian13-restic-2026-10-16.md`. **Earlier (midday):** Controller 0.319.0 (the app page in panels, the login in one panel) is the floor, MinAgent 0.158.1 — `REPORT-app-page-tuning-2026-10-16.md`. **Earlier (morning):** Storage identity build B shipped: agent 0.158.1 and golden 0.318.0 vouched by you, controller 0.318.0 is the floor (MinAgent 0.158.1), installer 1.33.0 published — new one-disk boxes get their own data area. Open items: see `documentation/backlog/OPEN-ITEMS.md`. Older notes, day by day:
|
||||
`documentation/backlog/STATUS-HISTORY.md`. Reports: `REPORT-night-2026-10-14.md`, `REPORT-immich-rollout-2026-10-14.md`, `REPORT-seo-felho-tarhely-2026-10-14.md`, `REPORT-immich-repair-2026-10-14.md`, `REPORT-power-cut-fixes-2026-10-14.md`, `REPORT-night-power-2026-10-13.md`, `REPORT-poweroff.md`.
|
||||
|
||||
## What works
|
||||
@@ -29,7 +29,7 @@
|
||||
- 65 apps in the catalog. New today: Prowlarr — you enter your download sources once, and Sonarr and Radarr get them, linked by the box with no button (the website lists 63 apps — R-1036). OS updates, the kernel lane too, run at night.
|
||||
- The hub is closed again after the 2026-10-12 security review; one box cannot read another's data.
|
||||
|
||||
- Felhom runs in a virtual machine on OpenMediaVault 8, tried on an exact copy of the first outside tester's box: install, bind, claim, apps, backup, and back by itself after an OMV restart. The Hungarian guide for him is `documentation/pilot/GUIDE-omv-kvm-hu.md`; open: one-disk Jellyfin (R-1068), NAS films for Jellyfin (R-1071, yours), no disk health in a VM (R-1073, yours). Waits on you: send the guide; delete test customer Tester4 in the hub.
|
||||
- Felhom runs in a virtual machine on OpenMediaVault 8, tried on an exact copy of the first outside tester's box: install, bind, claim, apps, backup, and back by itself after an OMV restart. The Hungarian guide for him is `documentation/pilot/GUIDE-omv-kvm-hu.md`; open: one-disk Jellyfin (R-1068), NAS films for Jellyfin (R-1071, yours), no disk health in a VM (R-1073, yours). Waits on you: send the guide.
|
||||
- A second local copy counts only when it is on a different physical disk, and the box can prove it. Inside a virtual machine the box makes the copy but says „A Felhom nem tudja ellenőrizni…" (Tester 1). On real disks nothing changes (demo-hp, N100). A NAS never hides the „only one copy" notice. A box with no drive refuses a media app in words, with a link to the Meghajtók page. Live on the three test boxes; the NAS and no-drive parts are unit-tested only. Next: build B (the internal place on one-disk boxes, R-1079). Your rulings D1–D6 are in the design (`architecture/16-storage-physical-identity.md`).
|
||||
- **One-disk boxes get their own data area** (2026-10-16): a new install makes a „Belső tárhely" on the disk, outside every whole-box copy, and media apps install there with no extra step (proven on a fresh 512 GB box: 213 GiB area, Jellyfin with no extra step). A disk split into parts can be used part by part. The backup offer no longer promises disk-failure safety when it cannot check the disk, and `/mnt/sys_drive` is no longer offered.
|
||||
|
||||
@@ -41,7 +41,7 @@
|
||||
|
||||
## What we work on
|
||||
- **CC's secrets** (R-1118): done 2026-10-17 — the store, `sec`, the guards in every CC session (option B: `bash -x` and `curl -v` allowed inside `sec run`), the nightly check (linger on). The old credentials file and its three copies are deleted.
|
||||
- **After 2026-10-19 02:15 UTC** (demo-hp's next clean-up window: the hub allows one from 2026-10-18 14:16 UTC, but the box asks only after its 04:15 CEST night run; not open on 2026-10-18 at 14:33 UTC, `audits/day-2026-10-18/P8/`): check that the old restic (image 0.322.0) still lists and restores demo-hp's store after its first real 0.19.1 prune (`audits/night-restore-2026-10-17/tools/rx.sh`). Everything else of R-1105/R-1107 is done.
|
||||
- **After 2026-10-19 02:15 UTC** (demo-hp's next clean-up window, after its 04:15 CEST night run; the evening session left a watcher that runs the check by itself when the hub records the window closed — `scratchpad p8-watch.sh`; by hand: `rx.sh` + `xver-after-prune.sh` in guest 9201; both restics listed 310 snapshots at 20:57 UTC, `audits/evening-2026-10-18/P8-precheck.txt`): check that the old restic (image 0.322.0) still lists and restores demo-hp's store after its first real 0.19.1 prune (`audits/night-restore-2026-10-17/tools/rx.sh`). Everything else of R-1105/R-1107 is done.
|
||||
- The power-cut findings still open (R-1004, R-1005, R-1010, R-1013). R-1012 closed: agent 0.159.1 names an orphan thin snapshot in the backup failure.
|
||||
- Media apps on every drive (R-1023): live with the 0.313.0 floor — every drive in every media app, a new drive made ready by the box, a pulled drive loses nothing, a safe removal lets go first and the drive comes back by itself.
|
||||
|
||||
@@ -64,14 +64,16 @@
|
||||
| D11 | R-1126 | **DECIDED 2026-10-17: A** — FileBrowser is published only after its default password is checked; it never answers on the default (controller 0.327.1). The tunnel still runs before the claim, by design (`01` §5). | done | — | — | — |
|
||||
| D12 | R-1129 | **CLOSED 2026-10-17: not a defect** — CC misread the CI log; the failure mail always worked. A forced test mail went to admin@felhom.eu at 10:48:54 UTC (CC saw it arrive). Your key change did no harm. | done | — | — | — |
|
||||
| D9 | R-1062 | The N100's USB drive is also its whole-box backup drive. Split it in two anyway? | **No — keep one drive**; the lived-in box keeps its nightly whole-box backup | Split: the box loses its local whole-box backup (or add a second drive first) | it stays one drive; multi-drive tests stay on Tester 1 | — |
|
||||
| D16 | R-1100 | One sign-in for the dashboard and the file manager: build it? (tested tonight: the router can pass the user safely, but any app on the box could fake it unless the header name is a per-box secret) | **Yes, with the two safeguards in the design note** (a files-only ticket + a secret header name), about one release | No: households keep signing in to the file manager once | nothing changes; the second sign-in stays | — |
|
||||
| D17 | R-1149 | The box's local whole-box backups keep the last 3 copies; a broken-in box can replace all 3 in one day. Keep „last 3", or keep „one per day, 3 days"? | **One per day, 3 days** — same disk use, cannot be flooded in a day | Keep „last 3": flooding stays possible within one day (the new cap slows it) | it stays „last 3" | — |
|
||||
|
||||
## Waiting on you — do
|
||||
- **Vouch agent 0.159.2** (Configuration → Day-0 artifacts — with hub 0.158.0 the golden can stay as it is): agent sha256 `4d98868e922d6c10665a6c0da9323e8627ea9c589094e81d27d741e625d9d8f9`, bundle `477704ea2980948433ffc34bb59f008cc11410fd9683d386bfd3e2f085d55bc9` (read from the registry 2026-10-18), Min agent stays **0.158.1**. The three test hosts run it since 14:26. If you do not: a new box installs agent 0.159.1 (works; it takes the pool check before every backup).
|
||||
- **Sunday night's golden:** the floor is 0.330.0 since 2026-10-18 13:10 (the box release of the day), above the vouched golden 0.329.0 — so the Sunday rotation bakes a golden at **0.330.0** and puts its vouch line here.
|
||||
- **Vouch golden 0.331.0** (Configuration → Day-0 artifacts): golden sha256 `3881de62b0986ccbe5e55580e3f1bb9765d65b8d898f67c6339bac105244f84f` (read back from the registry 2026-10-18), agent **0.159.2** unchanged (already vouched), **Min agent 0.158.1**. This is the Sunday golden (the floor 0.331.0 is above the vouched 0.329.0). If you do not: a new box installs 0.329.0 and updates itself to the floor in about 3 minutes (proven tonight).
|
||||
- **Agent release with three fixes (R-978, R-973, R-1093) — may I?** Starting the recovery-code setup again no longer spoils the written-down code; a broken-into box program cannot run the whole-box backup again and again to push out old copies (3 a day, 1 off-site a week — numbers I chose, you may change them); an empty partition no longer asks „data will be erased". Tested, red-proved, in the agent's main branch. If you say yes: I release, prove on demo-hp, and ask you to vouch. If not: they wait.
|
||||
- **Nézd meg a demo-hp-n a Calibre-Web „Adatok és mentés" fülét és a Fájlkezelőt** (`felhom.enkisfelhom.hu`, controller 0.321.0, minden dobozon kiadva). A mentés egy részben, egyszerű szavakkal; a Fájlkezelő nem kér frissítést és nem ajánl „Install app"-ot.
|
||||
- **Look at `roms/roms/` on the N100's drive in the file manager (R-1099).** RomM now scans `roms/<platform>/`. A game uploaded through RomM before today sits one folder deeper: if there are platform folders in `roms/roms/`, move them up into `roms/` and press „Scan” in RomM. If not: those games show under a platform called „roms”.
|
||||
- **Nézd meg a Calibre-Web oldalát a demo-hp-n** (`felhom.enkisfelhom.hu`, controller 0.319.0, minden dobozon kiadva). A fejléc, a fülsor és minden rész most külön panel (az „A” stílus). A belépési adatok egy panelben vannak, egymás mellett. A Calibre-Web jelszaváról a doboz nem tud, mert az alkalmazás előbb települt, mint hogy a sablonban jelszó lett volna — ezt most egy mondat mondja ki. Ha nem tetszik, írd meg, mit változtassak. Ha nem nézed meg: semmi nem romlik el.
|
||||
- **Discard two unclaimed boxes and delete Tester4** (hub, browser only): the build B scratch VMs registered as unclaimed boxes — registration ids 40 (`b43afa10…`) and 41 (`9c78ea30…`); both VMs are destroyed. Tester4 (the throwaway customer) is switched off the household way. If not: two dead entries stay in the Hosts list and Tester4 stays in Customers.
|
||||
- **Discard two unclaimed boxes** (hub, browser only): the build B scratch VMs registered as unclaimed boxes — registration ids 40 (`b43afa10…`) and 41 (`9c78ea30…`); both VMs are destroyed. If not: two dead entries stay in the Hosts list. **Tester4 stays** (your word, 2026-10-18) as the fresh-install customer; **before its next install, delete its old host `Tester4-31fa0e`** (Hosts page) — otherwise the new install silently reuses the old host and skips two re-issues.
|
||||
- System poster needs a refresh: one-disk boxes now have a data area („Belső tárhely") on the system disk, outside the whole-guest backups (facts file updated 2026-10-16).
|
||||
- **Olvasd el a Plex és a Sonarr útmutatót a demo-hp-n** (az alkalmazás oldalán az „Útmutató” fül). Ha más formát szeretnél, egy menetben átírom mind a 62-t. A robot 2026-10-16-án bejelentkezve megnézte: mindkét oldal megjelenik magyarul és angolul, 4 rész és a GYIK, minden sor egyezik a katalógussal. Ha nem nézed meg: csak a forma marad emberi szemmel nem látott.
|
||||
- **Nézd meg az AdventureLog és a Plex oldalát a demo-hp-n, a rendes címén** (`felhom.enkisfelhom.hu`). A túl nagy keret javítva: a tartalom most 32 képpontra van a bal oszloptól és 36-ra a tetejétől (előtte 80 és 64 volt). Az alkalmazás oldala mostantól egy hasábban van: egy szélesség, vékony vonalak a szakaszok között, aláhúzott fül, és doboz csak ott, ahol az alkalmazás vár rád (kék) vagy ahol valami kockázatos (sárga). A „sikeres frissítés" sáv kis szürke sorrá vált a név alatt. Ha nem nézed meg: a felület él, csak senki nem látta még böngészőben.
|
||||
|
||||
@@ -544,6 +544,12 @@ nicety.
|
||||
**loses Felhom's backup guarantees** — no PBS verify / monitoring on storage we can't reach. An
|
||||
honest signup-time tradeoff, not a hidden one.
|
||||
|
||||
**When the new package reaches the hub (R-978, agent `main` 2026-10-18, unreleased):** a ceremony started again no
|
||||
longer replaces the hub's package when it starts. The package travels with the pending ceremony and is uploaded when its
|
||||
code is claimed, before the code is handed out (an upload failure is a 502 and the code stays claimable). An abandoned
|
||||
ceremony — TTL, a new start, a restart — leaves the first code opening the hub's package. Residual: the typed-back check
|
||||
is the browser's, so a code opened and not written down still replaces the package (R-1151).
|
||||
|
||||
#### Custody across generations — what the hub retains when a blob is superseded (hub v0.60.0; **contract corrected hub v0.93.0, R-198**)
|
||||
|
||||
A ceremony that seals a *different* passphrase does not destroy the blob it replaces: the hub copies
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -0,0 +1,24 @@
|
||||
loop=/dev/loop2
|
||||
--- lsblk p1 (the bug input)
|
||||
{"blockdevices":[{"name":"loop2p1","fstype":null,"pttype":"gpt","mountpoint":null}]}
|
||||
--- blkid -p p1 (empty part)
|
||||
DEVNAME=/dev/loop2p1
|
||||
PART_ENTRY_SCHEME=gpt
|
||||
PART_ENTRY_UUID=2b13a20b-f5a4-4906-9e30-985c170a50ad
|
||||
PART_ENTRY_TYPE=0fc63daf-8483-4772-8e79-3d69d8477de4
|
||||
PART_ENTRY_NUMBER=1
|
||||
PART_ENTRY_OFFSET=2048
|
||||
PART_ENTRY_SIZE=1048576
|
||||
PART_ENTRY_DISK=7:2
|
||||
rc=0
|
||||
--- blkid -p p2 (ext4)
|
||||
TYPE=ext4
|
||||
PART_ENTRY_SCHEME=gpt
|
||||
PART_ENTRY_NUMBER=2
|
||||
rc=0
|
||||
Re-reading the partition table failed.: Invalid argument
|
||||
--- blkid -p p1 (a dos table inside)
|
||||
PTTYPE=dos
|
||||
PART_ENTRY_SCHEME=gpt
|
||||
rc=0
|
||||
ls: cannot access '/root/r1093-scratch.img': No such file or directory
|
||||
@@ -0,0 +1,4 @@
|
||||
## per-customer floors 0.331.0 / MinAgent 0.158.1 — 20:44:00
|
||||
tester-1: HTTP/1.1 303 See Other Location: /customers/tester-1?flash=floor_set
|
||||
demo-hp: HTTP/1.1 303 See Other Location: /customers/demo-hp?flash=floor_set
|
||||
demo-felhom: HTTP/1.1 303 See Other Location: /customers/demo-felhom?flash=floor_set
|
||||
@@ -0,0 +1,6 @@
|
||||
demo-felhom|2026-10-18 20:44:04|0.330.0
|
||||
demo-felhom|2026-10-18 20:44:16|0.331.0
|
||||
demo-hp|2026-10-18 20:44:04|0.330.0
|
||||
demo-hp|2026-10-18 20:44:19|0.331.0
|
||||
tester-1|2026-10-18 20:44:03|0.330.0
|
||||
tester-1|2026-10-18 20:44:18|0.331.0
|
||||
@@ -0,0 +1,52 @@
|
||||
=== demo-hp (enkisfelhom.hu) 20:45:54
|
||||
containers: 24 running; not healthy/running: traefik Up 2 days;
|
||||
unhealthy: 0
|
||||
2026/10/18 20:44:12 updater.go:855: [INFO] [selfupdate] Post-update startup: update successful (0.330.0 → 0.331.0)
|
||||
allow-list refusals since start: 0
|
||||
2026/10/18 20:44:13 sync.go:142: [INFO] [sync] Initial sync: Sablonok naprakészek — nincs változás
|
||||
installed apps (stacks with app.yaml): 12
|
||||
2026/10/18 20:44:14 handlers.go:3887: [INFO] [web] FileBrowser mounts synced (recreated) — 1 storage path(s), config updated
|
||||
2026/10/18 20:44:19 filebrowser_gate.go:94: [INFO] [infra] filebrowser: default login refused for this run — joined traefik-public, files.<domain> s
|
||||
public https://felhom.enkisfelhom.hu/login: 000
|
||||
public https://files.enkisfelhom.hu/: 200
|
||||
=== demo-felhom (demo-felhom.eu) 20:46:13
|
||||
containers: 9 running; not healthy/running: traefik Up 2 days;
|
||||
unhealthy: 0
|
||||
2026/10/18 20:44:02 [INFO] [selfupdate] Post-update startup: update successful (0.330.0 → 0.331.0)
|
||||
allow-list refusals since start: 0
|
||||
2026/10/18 20:44:02 [INFO] [sync] Initial sync: Sablonok naprakészek — nincs változás
|
||||
installed apps (stacks with app.yaml): 5
|
||||
2026/10/18 20:44:03 [INFO] [web] FileBrowser mounts synced (recreated) — 1 storage path(s), config updated
|
||||
2026/10/18 20:44:09 [INFO] [infra] filebrowser: default login refused for this run — joined traefik-public, files.<domain> served (R-1126)
|
||||
public https://felhom.demo-felhom.eu/login: 200
|
||||
public https://files.demo-felhom.eu/: 200
|
||||
=== tester-1 (enkicsifelhom.hu) 20:46:16
|
||||
containers: 10 running; not healthy/running: traefik Up 2 days;
|
||||
unhealthy: 0
|
||||
2026/10/18 20:44:11 [INFO] [selfupdate] Post-update startup: update successful (0.330.0 → 0.331.0)
|
||||
allow-list refusals since start: 0
|
||||
2026/10/18 20:44:12 [INFO] [sync] Initial sync: Sablonok naprakészek — nincs változás
|
||||
installed apps (stacks with app.yaml): 3
|
||||
2026/10/18 20:44:13 [INFO] [web] FileBrowser mounts synced (recreated) — 5 storage path(s), config updated
|
||||
2026/10/18 20:44:18 [INFO] [infra] filebrowser: default login refused for this run — joined traefik-public, files.<domain> served (R-1126)
|
||||
public https://felhom.enkicsifelhom.hu/login: 200
|
||||
public https://files.enkicsifelhom.hu/: 000
|
||||
## retries 20:47:49
|
||||
20:47:49 https://felhom.enkisfelhom.hu/login: 200 0.156759s
|
||||
20:47:49 https://files.enkisfelhom.hu/: 200 0.151312s
|
||||
20:47:49 https://felhom.enkicsifelhom.hu/login: 200 0.154273s
|
||||
20:47:49 https://files.enkicsifelhom.hu/: 200 0.186791s
|
||||
20:47:50 https://felhom.demo-felhom.eu/login: 200 0.129644s
|
||||
20:47:50 https://files.demo-felhom.eu/: 200 0.148989s
|
||||
20:48:00 https://felhom.enkisfelhom.hu/login: 200 0.138598s
|
||||
20:48:00 https://files.enkisfelhom.hu/: 200 0.137951s
|
||||
20:48:00 https://felhom.enkicsifelhom.hu/login: 200 0.168040s
|
||||
20:48:00 https://files.enkicsifelhom.hu/: 200 0.160295s
|
||||
20:48:01 https://felhom.demo-felhom.eu/login: 200 0.172388s
|
||||
20:48:01 https://files.demo-felhom.eu/: 200 0.177437s
|
||||
20:48:11 https://felhom.enkisfelhom.hu/login: 200 0.151194s
|
||||
20:48:11 https://files.enkisfelhom.hu/: 200 0.166961s
|
||||
20:48:11 https://felhom.enkicsifelhom.hu/login: 200 0.130657s
|
||||
20:48:11 https://files.enkicsifelhom.hu/: 200 0.139520s
|
||||
20:48:12 https://felhom.demo-felhom.eu/login: 200 0.166568s
|
||||
20:48:12 https://files.demo-felhom.eu/: 200 0.148200s
|
||||
@@ -0,0 +1,33 @@
|
||||
control NEGATIVE (a stub answering 200 over plain http): convicted, as it must (200 -)
|
||||
== tester-1 (enkicsifelhom.hu): 6 public name(s)
|
||||
OK control POSITIVE http://felhom.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://felhom.enkicsifelhom.hu/r1033-check?q=1
|
||||
OK http://files.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://files.enkicsifelhom.hu/r1033-check?q=1
|
||||
OK http://no-such-name-r1033.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://no-such-name-r1033.enkicsifelhom.hu/r1033-check?q=1
|
||||
OK http://paperless.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://paperless.enkicsifelhom.hu/r1033-check?q=1
|
||||
OK http://paste.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://paste.enkicsifelhom.hu/r1033-check?q=1
|
||||
OK http://wiki.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://wiki.enkicsifelhom.hu/r1033-check?q=1
|
||||
== demo-hp (enkisfelhom.hu): 14 public name(s)
|
||||
OK http://arcade.enkisfelhom.hu/r1033-check?q=1 -> 301 https://arcade.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://books.enkisfelhom.hu/r1033-check?q=1 -> 301 https://books.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://docs.enkisfelhom.hu/r1033-check?q=1 -> 301 https://docs.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://felhom.enkisfelhom.hu/r1033-check?q=1 -> 301 https://felhom.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://files.enkisfelhom.hu/r1033-check?q=1 -> 301 https://files.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://gist.enkisfelhom.hu/r1033-check?q=1 -> 301 https://gist.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://ha.enkisfelhom.hu/r1033-check?q=1 -> 301 https://ha.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://no-such-name-r1033.enkisfelhom.hu/r1033-check?q=1 -> 301 https://no-such-name-r1033.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://papir.enkisfelhom.hu/r1033-check?q=1 -> 301 https://papir.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://paste.enkisfelhom.hu/r1033-check?q=1 -> 301 https://paste.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://pdf.enkisfelhom.hu/r1033-check?q=1 -> 301 https://pdf.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://plex.enkisfelhom.hu/r1033-check?q=1 -> 301 https://plex.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://travel.enkisfelhom.hu/r1033-check?q=1 -> 301 https://travel.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://wiki.enkisfelhom.hu/r1033-check?q=1 -> 301 https://wiki.enkisfelhom.hu/r1033-check?q=1
|
||||
== demo-felhom (demo-felhom.eu): 8 public name(s)
|
||||
OK http://felhom.demo-felhom.eu/r1033-check?q=1 -> 301 https://felhom.demo-felhom.eu/r1033-check?q=1
|
||||
OK http://files.demo-felhom.eu/r1033-check?q=1 -> 301 https://files.demo-felhom.eu/r1033-check?q=1
|
||||
OK http://gist.demo-felhom.eu/r1033-check?q=1 -> 301 https://gist.demo-felhom.eu/r1033-check?q=1
|
||||
OK http://no-such-name-r1033.demo-felhom.eu/r1033-check?q=1 -> 301 https://no-such-name-r1033.demo-felhom.eu/r1033-check?q=1
|
||||
OK http://prowlarr.demo-felhom.eu/r1033-check?q=1 -> 301 https://prowlarr.demo-felhom.eu/r1033-check?q=1
|
||||
OK http://radarr.demo-felhom.eu/r1033-check?q=1 -> 301 https://radarr.demo-felhom.eu/r1033-check?q=1
|
||||
OK http://sonarr.demo-felhom.eu/r1033-check?q=1 -> 301 https://sonarr.demo-felhom.eu/r1033-check?q=1
|
||||
OK http://torrent.demo-felhom.eu/r1033-check?q=1 -> 301 https://torrent.demo-felhom.eu/r1033-check?q=1
|
||||
== every public name redirects http -> https
|
||||
@@ -0,0 +1,15 @@
|
||||
## global floor 20:48:56
|
||||
HTTP/1.1 303 See Other Location: /configuration?flash=floor_set
|
||||
## read back (hub.db + -wal + -shm in one tar)
|
||||
min_controller_version|0.331.0
|
||||
min_controller_version_declared_min_agent|0.331.0=0.158.1
|
||||
## clear the per-customer floors 20:49:09
|
||||
tester-1: HTTP/1.1 303 See Other Location: /customers/tester-1?flash=floor_set
|
||||
demo-hp: HTTP/1.1 303 See Other Location: /customers/demo-hp?flash=floor_set
|
||||
demo-felhom: HTTP/1.1 303 See Other Location: /customers/demo-felhom?flash=floor_set
|
||||
, status TEXT NOT NULL DEFAULT 'active', min_controller_version TEXT NOT NULL DEFAULT '', config_version INTEGER NOT NULL DEFAULT 1, dr_tier INTEGER NOT NULL DEFAULT 0, min_controller_declared_min_agent TEXT NOT NULL DEFAULT '', language TEXT NOT NULL DEFAULT 'hu', min_controller_set_at TEXT NOT NULL DEFAULT '', api_key_hash TEXT NOT NULL DEFAULT '');
|
||||
demo-felhom|[]|[]
|
||||
demo-hp|[]|[]
|
||||
tester-1|[]|[]
|
||||
Tester-2|[]|[]
|
||||
Tester4|[]|[]
|
||||
@@ -0,0 +1,2 @@
|
||||
0.322.0: 310 snapshots restic=restic 0.14.0
|
||||
0.331.0: 310 snapshots restic=restic 0.19.1
|
||||
@@ -0,0 +1,12 @@
|
||||
iso=/var/lib/vz/template/iso/felhom-installer-1.29.0-pve9.2-1.iso sha256=dceacae5da247d76
|
||||
Formatting '/mnt/hdd_1/images/9302/vm-9302-disk-0.raw', fmt=raw size=128849018880 preallocation=off
|
||||
scsi0: successfully created disk 'nvme-scratch:9302/vm-9302-disk-0.raw,size=120G'
|
||||
update VM 9302: -boot order=scsi0;ide2
|
||||
boot: order=scsi0;ide2
|
||||
cores: 4
|
||||
ide2: local:iso/felhom-installer-1.29.0-pve9.2-1.iso,media=cdrom,size=1665356K
|
||||
memory: 8192
|
||||
name: newbox-r1148
|
||||
net0: virtio=BC:24:11:0F:7C:57,bridge=vmbr0
|
||||
scsi0: nvme-scratch:9302/vm-9302-disk-0.raw,size=120G
|
||||
19:43:15
|
||||
@@ -0,0 +1,22 @@
|
||||
install pressed 19:50:19 CEST
|
||||
bind posted 20:02:58 CEST
|
||||
--- read from the hub DB (copied with its WAL), the box's journal, and the box's own /api/selfupdate/status (all UTC) ---
|
||||
17:42:21 hub: customer Tester4 created by the operator; the bind mail sent at once (Gmail catch-all)
|
||||
17:43:15 VM 9302 created on demo-hp (8 GB / 4 cores / 120 GB on nvme-scratch at /mnt/hdd_1 root); ISO 1.29.0 (sha256 dceacae5… = published)
|
||||
17:50:19 Install pressed (graphical installer, Hungarian layout kept, driven by qm sendkey; screens/)
|
||||
~18:02 console: pairing code shown (screens/13-now.png)
|
||||
18:02:58 bind through the mailed link + pairing code + owner phrase: „Sikeres összekötés" (03-bind.txt)
|
||||
18:03:13 setup-code mail
|
||||
18:03:42 hub: host Tester4-31fa0e created
|
||||
18:03:43 step 4b/8: „recovery password vaulted at the hub (stored) — now setting it on root@pam"; chpasswd; „[OK] root@pam password vaulted, set, and the PVE login takes it"
|
||||
18:03:43 hub: host_recovery.set_at for Tester4-31fa0e (the same second)
|
||||
18:03:44 step 5/8: manifest agent v0.159.2 (sha 4d98868e…), golden v0.329.0; agent sha256 verified = hub manifest; bundle 477704ea… verified
|
||||
18:04:02 first host report: agent 0.159.2
|
||||
18:05:37 first controller report: 0.329.0 (the vouched golden)
|
||||
18:05:38 the box's own record: update 0.329.0 -> 0.330.0 initiated_by „auto-floor"; completed 18:05:45
|
||||
18:05:51 controller report: 0.330.0
|
||||
18:05:54 „Day-0 provision SUCCESS — vmid=9201 host_id=Tester4-31fa0e customer=Tester4"
|
||||
18:17:14 CC's own PVE login from DooPlex: vaulted value 200; the installer's typed value 401 (04-pve-login-check.txt)
|
||||
18:18:32 dashboard claimed with the mailed setup code (302 /)
|
||||
18:19:25 BentoPDF deploy 202; 18:20:05 running, container healthy (07-first-app.txt)
|
||||
=> Install pressed to the floor: 15 min 26 s. Bind to the floor: 2 min 47 s. Nobody pressed an update.
|
||||
@@ -0,0 +1 @@
|
||||
Felhom — Doboz összekötése Felhom doboz összekötése Sikeres összekötés. A doboz kb. egy percen belül folytatja a telepítést. Ezt az oldalt bezárhatod — a beállítás a háttérben befejeződik, és a vezérlőpultod hamarosan elérhető lesz. Felhom.eu
|
||||
@@ -0,0 +1,2 @@
|
||||
18:17:14 PVE login with the VAULTED value: HTTP 200
|
||||
18:17:17 PVE login with the INSTALLER value (typed at install): HTTP 401
|
||||
@@ -0,0 +1,7 @@
|
||||
GET /claim 200
|
||||
csrf_len 64
|
||||
HTTP/1.1 302 Found
|
||||
Location: /
|
||||
18:18:32
|
||||
{"ok":true,"data":{"running":false,"last_check":{"current_version":"0.330.0","latest_version":"0.330.0","update_available":false,"checked_at":"2026-10-18T18:06:16Z"},"last_state":{"status":"success","previous_version":"0.329.0","previous_image":"gitea.dooplex.hu/admin/felhom-controller:0.329.0","target_version":"0.330.0","target_image":"gitea.dooplex.hu/admin/felhom-controller:0.330.0","initiated_at":"2026-10-18T18:05:38Z","initiated_by":"auto-floor","completed_at":"2026-10-18T18:05:45Z"}}}
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
{"ok":false,"error":"dashboard not yet claimed"}
|
||||
@@ -0,0 +1,8 @@
|
||||
login 302
|
||||
csrf_len 64
|
||||
{"ok":true,"data":{"app_config":null,"domain":"tester4.test","logo_url":"/static/assets/bentopdf-logo.svg","metadata":{"display_name":"BentoPDF","description":"Adatvédelmi fókuszú PDF eszköztár","category":"tools","subdomain":"pdf","slug":"bentopdf","catalog_since":"2026-07-12","resources":{"me
|
||||
{"ok":true,"message":"Telepítés elindítva – az állapot a kártyán követhető"}
|
||||
deploy 202
|
||||
18:19:25
|
||||
18:20:05 "state":"running" "deployed":true "state":"running" "deployed":true
|
||||
bentopdf ghcr.io/alam00000/bentopdf:v2.8.6 Up 20 seconds (healthy)
|
||||
@@ -0,0 +1,255 @@
|
||||
2026-10-18T19:53:32+02:00 tester4 systemd[1]: Starting felhom-bootstrap.service - Felhom host bootstrap (fetch + run felhom-host-install.sh unattended, retry until success)...
|
||||
2026-10-18T19:53:33+02:00 tester4 felhom-bootstrap.sh[1122]: felhom-bootstrap: console: pvebanner.service masked (it would rewrite /etc/issue with the Proxmox admin URL on every boot)
|
||||
2026-10-18T19:53:33+02:00 tester4 felhom-bootstrap.sh[1122]: felhom-bootstrap: console: /etc/issue is the Felhom text (no admin URL)
|
||||
2026-10-18T19:53:33+02:00 tester4 felhom-bootstrap.sh[1122]: felhom-bootstrap: PAIRING mode (generic ISO, no baked customer/passphrase) — hub=https://hub.felhom.eu
|
||||
2026-10-18T19:53:33+02:00 tester4 felhom-bootstrap.sh[1122]: felhom-bootstrap: registering unclaimed appliance at the hub
|
||||
2026-10-18T19:53:33+02:00 tester4 felhom-bootstrap.sh[1122]: felhom-bootstrap: registered — appliance token stored (0600); waiting for the operator or a customer self-bind
|
||||
2026-10-18T19:53:33+02:00 tester4 felhom-bootstrap.sh[1122]: felhom-bootstrap: console font -> Lat2-Terminus16 (Latin-2, ő/ű capable)
|
||||
2026-10-18T19:53:33+02:00 tester4 felhom-bootstrap.sh[1122]: felhom-bootstrap: not bound yet — polling every 30s until the operator or a customer self-bind lands (this is the normal waiting state, not an error)
|
||||
2026-10-18T20:03:04+02:00 tester4 felhom-bootstrap.sh[1122]: felhom-bootstrap: bind DELIVERED — writing credentials to the env and switching to direct install
|
||||
2026-10-18T20:03:04+02:00 tester4 felhom-bootstrap.sh[1122]: felhom-bootstrap: fetching host-install: https://felhom.eu/scripts/felhom-host-install.sh
|
||||
2026-10-18T20:03:05+02:00 tester4 felhom-bootstrap.sh[1122]: felhom-bootstrap: running host-install (customer=Tester4 mode=appliance hub=https://hub.felhom.eu)
|
||||
2026-10-18T20:03:05+02:00 tester4 felhom-bootstrap.sh[3647]: [INFO] auto-sized guest RAM: 4096 MiB (host 7939 MiB; clamp(host-4096, min 4096, max host-2048), ceiling host-1024)
|
||||
2026-10-18T20:03:05+02:00 tester4 felhom-bootstrap.sh[3647]: [INFO] auto-sized guest cores: 3 (host 4 cores; host-1, min 2)
|
||||
2026-10-18T20:03:05+02:00 tester4 felhom-bootstrap.sh[3647]: [INFO] felhom-host-install v1.34.0 — mode=appliance customer=Tester4 vmid=9201
|
||||
2026-10-18T20:03:05+02:00 tester4 felhom-bootstrap.sh[3647]: [STEP] 1/8 pre-flight
|
||||
2026-10-18T20:03:06+02:00 tester4 felhom-bootstrap.sh[3647]: [INFO] pve-manager/9.2.2/b9984c6d90a4bd80 (running kernel: 7.0.2-6-pve)
|
||||
2026-10-18T20:03:07+02:00 tester4 felhom-bootstrap.sh[3647]: [INFO] node: tester4 (auto)
|
||||
2026-10-18T20:03:07+02:00 tester4 felhom-bootstrap.sh[3647]: [INFO] agent config: /etc/felhom-agent/agent.json
|
||||
2026-10-18T20:03:07+02:00 tester4 felhom-bootstrap.sh[3647]: [INFO] agent: not installed yet — will be fetched + installed in step 5/8
|
||||
2026-10-18T20:03:07+02:00 tester4 felhom-bootstrap.sh[3647]: [INFO] local-lvm free: ~54 GiB
|
||||
2026-10-18T20:03:07+02:00 tester4 felhom-bootstrap.sh[3647]: [WARN] local-lvm free ~54 GiB is below the recommended 120 GiB — the install continues; proceed only if you sized the grows deliberately
|
||||
2026-10-18T20:03:07+02:00 tester4 felhom-bootstrap.sh[3647]: [INFO] free RAM: ~6371 MiB
|
||||
2026-10-18T20:03:09+02:00 tester4 felhom-bootstrap.sh[3647]: [INFO] existing guests on this host: 0 (pct+qm)
|
||||
2026-10-18T20:03:09+02:00 tester4 felhom-bootstrap.sh[3647]: [INFO] archive storage 'local' present
|
||||
2026-10-18T20:03:11+02:00 tester4 felhom-bootstrap.sh[3647]: [INFO] acl storage 'felhom-pbs' not present yet — expected: the PBS-DR tier creates it; the grant is pre-positioned deliberately
|
||||
2026-10-18T20:03:11+02:00 tester4 felhom-bootstrap.sh[3647]: [INFO] dnsmasq: not present before Felhom — recorded; uninstall will remove it again if we install it
|
||||
2026-10-18T20:03:12+02:00 tester4 felhom-bootstrap.sh[3647]: [INFO] hub reachable (https://hub.felhom.eu)
|
||||
2026-10-18T20:03:12+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] customer 'Tester4' exists + passphrase valid
|
||||
2026-10-18T20:03:13+02:00 tester4 felhom-bootstrap.sh[3647]: [INFO] golden: none local — will fetch + verify from Gitea in step 7/8
|
||||
2026-10-18T20:03:15+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] pre-flight passed
|
||||
2026-10-18T20:03:15+02:00 tester4 felhom-bootstrap.sh[3647]: [STEP] 2/8 Proxmox API token
|
||||
2026-10-18T20:03:19+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] token minted (secret captured, not logged)
|
||||
2026-10-18T20:03:41+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] scoped ACL applied (Base@/, Guest@/pool/felhom + /vms/990000..990009, Store@[local local-lvm felhom-pbs])
|
||||
2026-10-18T20:03:42+02:00 tester4 felhom-bootstrap.sh[3647]: [SKIP] old broad role FelhomAgent already absent
|
||||
2026-10-18T20:03:42+02:00 tester4 felhom-bootstrap.sh[3647]: [STEP] 3/8 compute volume grows
|
||||
2026-10-18T20:03:42+02:00 tester4 felhom-bootstrap.sh[3647]: [INFO] auto-computed from a 54 GiB thin pool (16 §7: 50/100/130 GiB)
|
||||
2026-10-18T20:03:42+02:00 tester4 felhom-bootstrap.sh[3647]: [INFO] grows: rootfs +0G (->32G), data +26G (->50G, ONE volume)
|
||||
2026-10-18T20:03:42+02:00 tester4 felhom-bootstrap.sh[3647]: [STEP] 4/8 host enrollment (POST /host-enroll)
|
||||
2026-10-18T20:03:42+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] host MINTED (first enroll)
|
||||
2026-10-18T20:03:43+02:00 tester4 felhom-bootstrap.sh[3647]: [INFO] host_id: Tester4-31fa0e (api_key captured, not logged)
|
||||
2026-10-18T20:03:43+02:00 tester4 felhom-bootstrap.sh[3647]: [STEP] 4b/8 break-glass credential (root@pam console password → hub vault)
|
||||
2026-10-18T20:03:43+02:00 tester4 felhom-bootstrap.sh[3647]: [INFO] recovery password vaulted at the hub (stored) — now setting it on root@pam
|
||||
2026-10-18T20:03:43+02:00 tester4 chpasswd[3936]: pam_unix(chpasswd:chauthtok): password changed for root
|
||||
2026-10-18T20:03:43+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] root@pam password vaulted, set, and the PVE login takes it (retrieve via the operator /admin path; never logged here)
|
||||
2026-10-18T20:03:43+02:00 tester4 felhom-bootstrap.sh[3647]: [WARN] NOTE: the root@pam password just CHANGED — the old one now fails at the PVE web GUI (:8006).
|
||||
2026-10-18T20:03:43+02:00 tester4 felhom-bootstrap.sh[3647]: [WARN] Retrieve the new one at hub → host page (vaulted recovery credential).
|
||||
2026-10-18T20:03:43+02:00 tester4 felhom-bootstrap.sh[3647]: [STEP] 5/8 agent install (fetch + verify + install)
|
||||
2026-10-18T20:03:43+02:00 tester4 felhom-bootstrap.sh[3647]: [INFO] disabled pve-enterprise.sources (Enabled: no)
|
||||
2026-10-18T20:03:43+02:00 tester4 felhom-bootstrap.sh[3647]: [INFO] disabled ceph.sources (Enabled: no)
|
||||
2026-10-18T20:03:43+02:00 tester4 felhom-bootstrap.sh[3647]: [INFO] added pve-no-subscription.sources (suite=trixie)
|
||||
2026-10-18T20:03:44+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] apt repos aligned to no-subscription (changed; apt-get update OK)
|
||||
2026-10-18T20:03:44+02:00 tester4 felhom-bootstrap.sh[3647]: [WARN] no git credential in controller.yaml — fetching artifacts ANONYMOUSLY (they are world-readable; sha256 verification unchanged)
|
||||
2026-10-18T20:03:44+02:00 tester4 felhom-bootstrap.sh[3647]: [INFO] manifest: agent v0.159.2 (sha 4d98868e922d6c10…), golden v0.329.0
|
||||
2026-10-18T20:03:44+02:00 tester4 felhom-bootstrap.sh[3647]: [INFO] fetching agent binary v0.159.2 from Gitea …
|
||||
2026-10-18T20:03:45+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] verified sha256 4d98868e922d6c10… matches the hub manifest
|
||||
2026-10-18T20:03:45+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] installed /usr/local/bin/felhom-agent (felhom-agent 0.159.2)
|
||||
2026-10-18T20:03:45+02:00 tester4 felhom-bootstrap.sh[3647]: [INFO] installing the 'sudo' package (required for the non-root agent model) …
|
||||
2026-10-18T20:03:48+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] sudo installed (Sudo version 1.9.16p2)
|
||||
2026-10-18T20:03:48+02:00 tester4 felhom-bootstrap.sh[3647]: [INFO] installing the 'age' package (escrow ceremony identity-wrap dependency) …
|
||||
2026-10-18T20:03:51+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] age installed (1.2.1)
|
||||
2026-10-18T20:03:51+02:00 tester4 useradd[4161]: new group: name=felhom-agent, GID=990
|
||||
2026-10-18T20:03:51+02:00 tester4 useradd[4161]: new user: name=felhom-agent, UID=999, GID=990, home=/home/felhom-agent, shell=/usr/sbin/nologin, from=none
|
||||
2026-10-18T20:03:51+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] created service user felhom-agent
|
||||
2026-10-18T20:03:51+02:00 tester4 usermod[4169]: add 'felhom-agent' to group 'systemd-journal'
|
||||
2026-10-18T20:03:51+02:00 tester4 usermod[4169]: add 'felhom-agent' to shadow group 'systemd-journal'
|
||||
2026-10-18T20:03:51+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] added felhom-agent to systemd-journal (unprivileged journal read for NAS verify)
|
||||
2026-10-18T20:03:53+02:00 tester4 useradd[4209]: new group: name=felhom-op, GID=1000
|
||||
2026-10-18T20:03:53+02:00 tester4 useradd[4209]: new user: name=felhom-op, UID=1000, GID=1000, home=/home/felhom-op, shell=/bin/bash, from=none
|
||||
2026-10-18T20:03:53+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] verified sha256 477704ea29809484… matches the hub manifest
|
||||
2026-10-18T20:03:54+02:00 tester4 felhom-os-apply[4246]: os-apply: BUNDLE START agent=0.159.2 sha=477704ea29809484 authority=installer files=31 write=30 same=1 kept=0 skipped=0
|
||||
2026-10-18T20:03:54+02:00 tester4 felhom-os-apply[4248]: os-apply: BUNDLE WROTE /usr/local/sbin/felhom-selfupdate-guarded (new)
|
||||
2026-10-18T20:03:54+02:00 tester4 felhom-os-apply[4249]: os-apply: BUNDLE WROTE /usr/local/sbin/felhom-pbs-apply (new)
|
||||
2026-10-18T20:03:54+02:00 tester4 felhom-os-apply[4250]: os-apply: BUNDLE WROTE /usr/local/sbin/felhom-backup-target-apply (new)
|
||||
2026-10-18T20:03:54+02:00 tester4 felhom-os-apply[4251]: os-apply: BUNDLE WROTE /usr/local/sbin/felhom-vzdump-partial-clean (new)
|
||||
2026-10-18T20:03:54+02:00 tester4 felhom-os-apply[4252]: os-apply: BUNDLE WROTE /usr/local/sbin/felhom-thin-check (new)
|
||||
2026-10-18T20:03:54+02:00 tester4 felhom-os-apply[4253]: os-apply: BUNDLE WROTE /usr/local/sbin/felhom-internal-place (new)
|
||||
2026-10-18T20:03:54+02:00 tester4 felhom-os-apply[4254]: os-apply: BUNDLE WROTE /usr/local/sbin/felhom-crash-guard (new)
|
||||
2026-10-18T20:03:54+02:00 tester4 felhom-os-apply[4255]: os-apply: BUNDLE WROTE /usr/local/sbin/felhom-priv-apply (new)
|
||||
2026-10-18T20:03:54+02:00 tester4 felhom-os-apply[4256]: os-apply: BUNDLE WROTE /var/lib/vz/snippets/felhom-guest-hook.sh (new)
|
||||
2026-10-18T20:03:54+02:00 tester4 felhom-os-apply[4257]: os-apply: BUNDLE WROTE /usr/local/sbin/felhom-shared-parent.sh (new)
|
||||
2026-10-18T20:03:54+02:00 tester4 felhom-os-apply[4258]: os-apply: BUNDLE WROTE /etc/systemd/system/felhom-shared-parent.service (new)
|
||||
2026-10-18T20:03:54+02:00 tester4 felhom-os-apply[4259]: os-apply: BUNDLE WROTE /etc/systemd/system/felhom-crash-guard.service (new)
|
||||
2026-10-18T20:03:54+02:00 tester4 felhom-os-apply[4260]: os-apply: BUNDLE WROTE /etc/systemd/system/felhom-crash-guard-check.service (new)
|
||||
2026-10-18T20:03:54+02:00 tester4 felhom-os-apply[4261]: os-apply: BUNDLE WROTE /etc/systemd/system/felhom-crash-guard-check.timer (new)
|
||||
2026-10-18T20:03:54+02:00 tester4 felhom-os-apply[4262]: os-apply: BUNDLE WROTE /etc/felhom/crash-guard.conf (new)
|
||||
2026-10-18T20:03:54+02:00 tester4 felhom-os-apply[4263]: os-apply: BUNDLE WROTE /etc/grub.d/01_felhom_oneshot (new)
|
||||
2026-10-18T20:03:54+02:00 tester4 felhom-os-apply[4264]: os-apply: BUNDLE WROTE /etc/grub.d/42_felhom_oneshot (new)
|
||||
2026-10-18T20:03:54+02:00 tester4 felhom-os-apply[4265]: os-apply: BUNDLE WROTE /etc/systemd/system/felhom-agent.service (new)
|
||||
2026-10-18T20:03:54+02:00 tester4 felhom-os-apply[4266]: os-apply: BUNDLE WROTE /etc/systemd/system/felhom-agent-rollback.service (new)
|
||||
2026-10-18T20:03:54+02:00 tester4 felhom-os-apply[4267]: os-apply: BUNDLE WROTE /etc/systemd/system/felhom-agent.service.d/felhom-agent-limits.conf (new)
|
||||
2026-10-18T20:03:54+02:00 tester4 felhom-os-apply[4268]: os-apply: BUNDLE WROTE /usr/local/sbin/felhom-mgmt-watchdog (new)
|
||||
2026-10-18T20:03:54+02:00 tester4 felhom-os-apply[4269]: os-apply: BUNDLE WROTE /etc/tmpfiles.d/felhom-privsep.conf (new)
|
||||
2026-10-18T20:03:54+02:00 tester4 felhom-os-apply[4270]: os-apply: BUNDLE WROTE /etc/systemd/system/felhom-mgmt-watchdog.service (new)
|
||||
2026-10-18T20:03:54+02:00 tester4 felhom-os-apply[4271]: os-apply: BUNDLE WROTE /etc/systemd/system/felhom-mgmt-watchdog.timer (new)
|
||||
2026-10-18T20:03:54+02:00 tester4 felhom-os-apply[4272]: os-apply: BUNDLE WROTE /etc/systemd/system/felhom-sshd.service (new)
|
||||
2026-10-18T20:03:54+02:00 tester4 felhom-os-apply[4273]: os-apply: BUNDLE WROTE /etc/felhom-oob.nft (new)
|
||||
2026-10-18T20:03:54+02:00 tester4 felhom-os-apply[4274]: os-apply: BUNDLE WROTE /etc/systemd/system/felhom-oob-nft.service (new)
|
||||
2026-10-18T20:03:54+02:00 tester4 felhom-os-apply[4275]: os-apply: BUNDLE WROTE /etc/sudoers.d/felhom-op (new)
|
||||
2026-10-18T20:03:54+02:00 tester4 felhom-os-apply[4276]: os-apply: BUNDLE WROTE /etc/sudoers.d/felhom-agent (new)
|
||||
2026-10-18T20:03:55+02:00 tester4 felhom-os-apply[4395]: os-apply: BUNDLE DONE agent=0.159.2 written=30 same=1 self-check=ok signers-created=False
|
||||
2026-10-18T20:03:55+02:00 tester4 felhom-bootstrap.sh[4398]: os-apply: BUNDLE START agent=0.159.2 sha=477704ea29809484 authority=installer files=31 write=30 same=1 kept=0 skipped=0
|
||||
2026-10-18T20:03:55+02:00 tester4 felhom-bootstrap.sh[4398]: os-apply: BUNDLE WROTE /usr/local/sbin/felhom-mkfs-guarded (new)
|
||||
2026-10-18T20:03:55+02:00 tester4 felhom-bootstrap.sh[4398]: os-apply: BUNDLE WROTE /usr/local/sbin/felhom-selfupdate-guarded (new)
|
||||
2026-10-18T20:03:55+02:00 tester4 felhom-bootstrap.sh[4398]: os-apply: BUNDLE WROTE /usr/local/sbin/felhom-pbs-apply (new)
|
||||
2026-10-18T20:03:55+02:00 tester4 felhom-bootstrap.sh[4398]: os-apply: BUNDLE WROTE /usr/local/sbin/felhom-backup-target-apply (new)
|
||||
2026-10-18T20:03:55+02:00 tester4 felhom-bootstrap.sh[4398]: os-apply: BUNDLE WROTE /usr/local/sbin/felhom-vzdump-partial-clean (new)
|
||||
2026-10-18T20:03:55+02:00 tester4 felhom-bootstrap.sh[4398]: os-apply: BUNDLE WROTE /usr/local/sbin/felhom-thin-check (new)
|
||||
2026-10-18T20:03:55+02:00 tester4 felhom-bootstrap.sh[4398]: os-apply: BUNDLE WROTE /usr/local/sbin/felhom-internal-place (new)
|
||||
2026-10-18T20:03:55+02:00 tester4 felhom-bootstrap.sh[4398]: os-apply: BUNDLE WROTE /usr/local/sbin/felhom-crash-guard (new)
|
||||
2026-10-18T20:03:55+02:00 tester4 felhom-bootstrap.sh[4398]: os-apply: BUNDLE WROTE /usr/local/sbin/felhom-priv-apply (new)
|
||||
2026-10-18T20:03:55+02:00 tester4 felhom-bootstrap.sh[4398]: os-apply: BUNDLE WROTE /var/lib/vz/snippets/felhom-guest-hook.sh (new)
|
||||
2026-10-18T20:03:55+02:00 tester4 felhom-bootstrap.sh[4398]: os-apply: BUNDLE WROTE /usr/local/sbin/felhom-shared-parent.sh (new)
|
||||
2026-10-18T20:03:55+02:00 tester4 felhom-bootstrap.sh[4398]: os-apply: BUNDLE WROTE /etc/systemd/system/felhom-shared-parent.service (new)
|
||||
2026-10-18T20:03:55+02:00 tester4 felhom-bootstrap.sh[4398]: os-apply: BUNDLE WROTE /etc/systemd/system/felhom-crash-guard.service (new)
|
||||
2026-10-18T20:03:55+02:00 tester4 felhom-bootstrap.sh[4398]: os-apply: BUNDLE WROTE /etc/systemd/system/felhom-crash-guard-check.service (new)
|
||||
2026-10-18T20:03:55+02:00 tester4 felhom-bootstrap.sh[4398]: os-apply: BUNDLE WROTE /etc/systemd/system/felhom-crash-guard-check.timer (new)
|
||||
2026-10-18T20:03:55+02:00 tester4 felhom-bootstrap.sh[4398]: os-apply: BUNDLE WROTE /etc/felhom/crash-guard.conf (new)
|
||||
2026-10-18T20:03:55+02:00 tester4 felhom-bootstrap.sh[4398]: os-apply: BUNDLE WROTE /etc/grub.d/01_felhom_oneshot (new)
|
||||
2026-10-18T20:03:55+02:00 tester4 felhom-bootstrap.sh[4398]: os-apply: BUNDLE WROTE /etc/grub.d/42_felhom_oneshot (new)
|
||||
2026-10-18T20:03:55+02:00 tester4 felhom-bootstrap.sh[4398]: os-apply: BUNDLE WROTE /etc/systemd/system/felhom-agent.service (new)
|
||||
2026-10-18T20:03:55+02:00 tester4 felhom-bootstrap.sh[4398]: os-apply: BUNDLE WROTE /etc/systemd/system/felhom-agent-rollback.service (new)
|
||||
2026-10-18T20:03:55+02:00 tester4 felhom-bootstrap.sh[4398]: os-apply: BUNDLE WROTE /etc/systemd/system/felhom-agent.service.d/felhom-agent-limits.conf (new)
|
||||
2026-10-18T20:03:55+02:00 tester4 felhom-bootstrap.sh[4398]: os-apply: BUNDLE WROTE /usr/local/sbin/felhom-mgmt-watchdog (new)
|
||||
2026-10-18T20:03:55+02:00 tester4 felhom-bootstrap.sh[4398]: os-apply: BUNDLE WROTE /etc/tmpfiles.d/felhom-privsep.conf (new)
|
||||
2026-10-18T20:03:55+02:00 tester4 felhom-bootstrap.sh[4398]: os-apply: BUNDLE WROTE /etc/systemd/system/felhom-mgmt-watchdog.service (new)
|
||||
2026-10-18T20:03:55+02:00 tester4 felhom-bootstrap.sh[4398]: os-apply: BUNDLE WROTE /etc/systemd/system/felhom-mgmt-watchdog.timer (new)
|
||||
2026-10-18T20:03:55+02:00 tester4 felhom-bootstrap.sh[4398]: os-apply: BUNDLE WROTE /etc/systemd/system/felhom-sshd.service (new)
|
||||
2026-10-18T20:03:55+02:00 tester4 felhom-bootstrap.sh[4398]: os-apply: BUNDLE WROTE /etc/felhom-oob.nft (new)
|
||||
2026-10-18T20:03:55+02:00 tester4 felhom-bootstrap.sh[4398]: os-apply: BUNDLE WROTE /etc/systemd/system/felhom-oob-nft.service (new)
|
||||
2026-10-18T20:03:55+02:00 tester4 felhom-bootstrap.sh[4398]: os-apply: BUNDLE WROTE /etc/sudoers.d/felhom-op (new)
|
||||
2026-10-18T20:03:55+02:00 tester4 felhom-bootstrap.sh[4398]: os-apply: BUNDLE WROTE /etc/sudoers.d/felhom-agent (new)
|
||||
2026-10-18T20:03:55+02:00 tester4 felhom-bootstrap.sh[4398]: os-apply: BUNDLE DONE agent=0.159.2 written=30 same=1 self-check=ok signers-created=False
|
||||
2026-10-18T20:03:56+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] installed every root-owned file from config bundle v0.159.2 (sha 477704ea29809484…; checked, previous copies kept)
|
||||
2026-10-18T20:03:56+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] OOB felhom-sshd instance + static belt from the bundle (agent renders config + fills sets once oob.enabled)
|
||||
2026-10-18T20:03:56+02:00 tester4 felhom-bootstrap.sh[3647]: [STEP] 5b/8 internal data place (Belső tárhely)
|
||||
2026-10-18T20:03:58+02:00 tester4 felhom-bootstrap.sh[3647]: [INFO] no internal place: only -44 GiB free for it (pool 54, reserve 16, promised 0, planned 82; needs 20)
|
||||
2026-10-18T20:03:58+02:00 tester4 felhom-bootstrap.sh[3647]: [STEP] 6/8 agent config + service
|
||||
2026-10-18T20:03:59+02:00 tester4 felhom-bootstrap.sh[3647]: [WARN] backup target: DEGRADED — no eligible second drive, so the whole-system backup stays on the SYSTEM drive.
|
||||
2026-10-18T20:03:59+02:00 tester4 felhom-bootstrap.sh[3647]: [WARN] It protects against file corruption but NOT against a disk failure. Attach a second drive and assign it in the dashboard.
|
||||
2026-10-18T20:03:59+02:00 tester4 felhom-bootstrap.sh[3647]: [INFO] creating island bridge vmbr9 (portless, 169.254.253.1/30)
|
||||
2026-10-18T20:03:59+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] vmbr9 up: 169.254.253.1/30
|
||||
2026-10-18T20:03:59+02:00 tester4 felhom-bootstrap.sh[3647]: [INFO] R-50 island ON: local_api=169.254.253.1:8443 (vmbr9); guest net1=169.254.253.2/30; lan_resolver.host_ip=192.168.0.109
|
||||
2026-10-18T20:03:59+02:00 tester4 felhom-bootstrap.sh[3647]: [INFO] node=tester4 local_api=169.254.253.1:8443 tls_fp=25:C4:B3:61:78:4A…
|
||||
2026-10-18T20:03:59+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] wrote /etc/felhom-agent/agent.json (0600 felhom-agent)
|
||||
2026-10-18T20:04:00+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] agent --selftest (read-only) passed
|
||||
2026-10-18T20:04:03+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] felhom-agent service active (non-root felhom-agent reads the config OK)
|
||||
2026-10-18T20:04:03+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] wrote /etc/felhom/os-trust.json + operator-signers (felhom-op-1) — the root-owned slow-lane anchors
|
||||
2026-10-18T20:04:03+02:00 tester4 felhom-bootstrap.sh[3647]: [STEP] 7/8 golden archive
|
||||
2026-10-18T20:04:03+02:00 tester4 felhom-bootstrap.sh[3647]: [WARN] no git credential in controller.yaml — fetching artifacts ANONYMOUSLY (they are world-readable; sha256 verification unchanged)
|
||||
2026-10-18T20:04:04+02:00 tester4 felhom-bootstrap.sh[3647]: [INFO] fetching golden v0.329.0 from Gitea → /var/lib/vz/dump/vzdump-lxc-9100-2026_10_18-20_04_04.tar.zst
|
||||
2026-10-18T20:04:15+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] verified sha256 58dc40df555676d8… matches the hub manifest
|
||||
2026-10-18T20:04:16+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] golden imported + verified: local:backup/vzdump-lxc-9100-2026_10_18-20_04_04.tar.zst
|
||||
2026-10-18T20:04:16+02:00 tester4 felhom-bootstrap.sh[3647]: [STEP] 8/8 provision guest 9201
|
||||
2026-10-18T20:04:17+02:00 tester4 felhom-bootstrap.sh[3647]: [SKIP] pool felhom already exists
|
||||
2026-10-18T20:04:17+02:00 tester4 felhom-bootstrap.sh[5665]: === felhom-agent 0.159.2 selftest=provision (vmid=9201 customer=Tester4 hostname=tester4) ===
|
||||
2026-10-18T20:04:17+02:00 tester4 felhom-bootstrap.sh[5665]: --- front half: bring-up (provision) local:backup/vzdump-lxc-9100-2026_10_18-20_04_04.tar.zst → vmid 9201 ---
|
||||
2026-10-18T20:05:11+02:00 tester4 felhom-bootstrap.sh[5665]: time=2026-10-18T20:05:11.381+02:00 level=INFO msg="bring-up: pool membership re-asserted" vmid=9201 pool=felhom
|
||||
2026-10-18T20:05:11+02:00 tester4 felhom-bootstrap.sh[5665]: [OK] front half: vmid 9201 up (boot+running) in 54s; MAC=BC:24:11:13:D8:4F
|
||||
2026-10-18T20:05:11+02:00 tester4 felhom-bootstrap.sh[5665]: --- back half: mint per-guest token + populate bootstrap config mount ---
|
||||
2026-10-18T20:05:11+02:00 tester4 sudo[6764]: root : PWD=/ ; USER=root ; COMMAND=/usr/bin/chown --reference=/var/lib/felhom-agent /var/lib/felhom-agent/guests /var/lib/felhom-agent/guests/9201
|
||||
2026-10-18T20:05:11+02:00 tester4 sudo[6764]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=0)
|
||||
2026-10-18T20:05:11+02:00 tester4 sudo[6764]: pam_unix(sudo:session): session closed for user root
|
||||
2026-10-18T20:05:11+02:00 tester4 sudo[6767]: root : PWD=/ ; USER=root ; COMMAND=/usr/bin/chown -R 100000:100000 /var/lib/felhom-agent/guests/9201/bootstrap
|
||||
2026-10-18T20:05:11+02:00 tester4 sudo[6767]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=0)
|
||||
2026-10-18T20:05:11+02:00 tester4 sudo[6767]: pam_unix(sudo:session): session closed for user root
|
||||
2026-10-18T20:05:11+02:00 tester4 sudo[6770]: root : PWD=/ ; USER=root ; COMMAND=/usr/sbin/pct set 9201 -mp9 /var/lib/felhom-agent/guests/9201/bootstrap,mp=/etc/felhom-bootstrap,ro=1
|
||||
2026-10-18T20:05:11+02:00 tester4 sudo[6770]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=0)
|
||||
2026-10-18T20:05:12+02:00 tester4 sudo[6770]: pam_unix(sudo:session): session closed for user root
|
||||
2026-10-18T20:05:12+02:00 tester4 sudo[6851]: root : PWD=/ ; USER=root ; COMMAND=/usr/sbin/pct set 9201 -onboot 1
|
||||
2026-10-18T20:05:12+02:00 tester4 sudo[6851]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=0)
|
||||
2026-10-18T20:05:13+02:00 tester4 sudo[6851]: pam_unix(sudo:session): session closed for user root
|
||||
2026-10-18T20:05:13+02:00 tester4 sudo[7056]: root : PWD=/ ; USER=root ; COMMAND=/usr/sbin/pct set 9201 --hookscript local:snippets/felhom-guest-hook.sh
|
||||
2026-10-18T20:05:13+02:00 tester4 sudo[7056]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=0)
|
||||
2026-10-18T20:05:14+02:00 tester4 sudo[7056]: pam_unix(sudo:session): session closed for user root
|
||||
2026-10-18T20:05:14+02:00 tester4 sudo[7479]: root : PWD=/ ; USER=root ; COMMAND=/usr/bin/mkdir -p /mnt/felhom-drives
|
||||
2026-10-18T20:05:14+02:00 tester4 sudo[7479]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=0)
|
||||
2026-10-18T20:05:14+02:00 tester4 sudo[7479]: pam_unix(sudo:session): session closed for user root
|
||||
2026-10-18T20:05:14+02:00 tester4 sudo[7485]: root : PWD=/ ; USER=root ; COMMAND=/usr/sbin/pct set 9201 -mp8 /mnt/felhom-drives,mp=/mnt/felhom-drives
|
||||
2026-10-18T20:05:14+02:00 tester4 sudo[7485]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=0)
|
||||
2026-10-18T20:05:15+02:00 tester4 sudo[7485]: pam_unix(sudo:session): session closed for user root
|
||||
2026-10-18T20:05:15+02:00 tester4 felhom-bootstrap.sh[5665]: time=2026-10-18T20:05:15.845+02:00 level=INFO msg="provision: back-half complete" vmid=9201 mount=mp9 guest_path=/etc/felhom-bootstrap endpoint=169.254.253.1:8443
|
||||
2026-10-18T20:05:15+02:00 tester4 felhom-bootstrap.sh[5665]: [OK] back half: bootstrap mount mp9 → /etc/felhom-bootstrap on vmid 9201 (host dir /var/lib/felhom-agent/guests/9201/bootstrap)
|
||||
2026-10-18T20:05:15+02:00 tester4 felhom-bootstrap.sh[5665]: local-api endpoint 169.254.253.1:8443 · leaf fp 7df7cb57504e66d1a1c688fd6d54e4c6c0a1970781c6ac65d2198a1b6e448f22 · token: minted (not printed)
|
||||
2026-10-18T20:05:15+02:00 tester4 felhom-bootstrap.sh[5665]: === selftest=provision OK — guest 9201 provisioned + bootstrap-mounted (KEPT) ===
|
||||
2026-10-18T20:05:15+02:00 tester4 felhom-bootstrap.sh[5665]: next: reboot the guest → the golden's baked controller-bootstrap unit deploys the controller,
|
||||
2026-10-18T20:05:15+02:00 tester4 felhom-bootstrap.sh[5665]: which PULLS its controller.yaml from the hub (retrieval passphrase) and merges in this local_api.
|
||||
2026-10-18T20:05:15+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] provision completed
|
||||
2026-10-18T20:05:15+02:00 tester4 felhom-bootstrap.sh[3647]: [INFO] rebooting guest 9201 so the baked controller-bootstrap unit picks up the mount
|
||||
2026-10-18T20:05:16+02:00 tester4 pct[7627]: <root@pam> starting task UPID:tester4:00001DDF:00011AB9:6AD50A5C:vzreboot:9201:root@pam:
|
||||
2026-10-18T20:05:16+02:00 tester4 pct[7647]: requesting reboot of CT 9201: UPID:tester4:00001DDF:00011AB9:6AD50A5C:vzreboot:9201:root@pam:
|
||||
2026-10-18T20:05:27+02:00 tester4 pct[7627]: <root@pam> end task UPID:tester4:00001DDF:00011AB9:6AD50A5C:vzreboot:9201:root@pam: OK
|
||||
2026-10-18T20:05:27+02:00 tester4 felhom-bootstrap.sh[3647]: [STEP] verify
|
||||
2026-10-18T20:05:28+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] pct status: running
|
||||
2026-10-18T20:05:29+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] onboot: 1
|
||||
2026-10-18T20:05:30+02:00 tester4 felhom-bootstrap.sh[8525]: mp0: local-lvm:vm-9201-disk-1,mp=/var/lib/felhom,backup=1,size=50G
|
||||
2026-10-18T20:05:30+02:00 tester4 felhom-bootstrap.sh[8525]: mp8: /mnt/felhom-drives,mp=/mnt/felhom-drives
|
||||
2026-10-18T20:05:30+02:00 tester4 felhom-bootstrap.sh[8525]: rootfs: local-lvm:vm-9201-disk-0,size=32G
|
||||
2026-10-18T20:05:32+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] pool: guest 9201 is a member of felhom
|
||||
2026-10-18T20:05:33+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] acl: FelhomAgentBase@/ present (user+token)
|
||||
2026-10-18T20:05:34+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] acl: FelhomAgentGuest@/pool/felhom present (user+token)
|
||||
2026-10-18T20:05:36+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] acl: FelhomAgentStore@/storage/local present (user+token)
|
||||
2026-10-18T20:05:37+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] acl: FelhomAgentStore@/storage/local-lvm present (user+token)
|
||||
2026-10-18T20:05:38+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] acl: FelhomAgentStore@/storage/felhom-pbs present (user+token)
|
||||
2026-10-18T20:05:40+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] acl: FelhomAgentGuest@/vms/990000 present (user+token)
|
||||
2026-10-18T20:05:41+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] acl: FelhomAgentGuest@/vms/990001 present (user+token)
|
||||
2026-10-18T20:05:42+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] acl: FelhomAgentGuest@/vms/990002 present (user+token)
|
||||
2026-10-18T20:05:43+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] acl: FelhomAgentGuest@/vms/990003 present (user+token)
|
||||
2026-10-18T20:05:45+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] acl: FelhomAgentGuest@/vms/990004 present (user+token)
|
||||
2026-10-18T20:05:46+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] acl: FelhomAgentGuest@/vms/990005 present (user+token)
|
||||
2026-10-18T20:05:47+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] acl: FelhomAgentGuest@/vms/990006 present (user+token)
|
||||
2026-10-18T20:05:48+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] acl: FelhomAgentGuest@/vms/990007 present (user+token)
|
||||
2026-10-18T20:05:49+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] acl: FelhomAgentGuest@/vms/990008 present (user+token)
|
||||
2026-10-18T20:05:51+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] acl: FelhomAgentGuest@/vms/990009 present (user+token)
|
||||
2026-10-18T20:05:51+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] authz signers: 2 (operator-signed self-update armed)
|
||||
2026-10-18T20:05:52+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] controller: Up 6 seconds (healthy) (after ~0s)
|
||||
2026-10-18T20:05:53+02:00 tester4 felhom-bootstrap.sh[3647]: [INFO] controller image: gitea.dooplex.hu/admin/felhom-controller:0.330.0
|
||||
2026-10-18T20:05:54+02:00 tester4 felhom-bootstrap.sh[3647]: [WARN] cloudflared not visible yet
|
||||
2026-10-18T20:05:54+02:00 tester4 felhom-bootstrap.sh[3647]: [INFO] (confirm in the hub UI that host Tester4-31fa0e reports guest 9201)
|
||||
2026-10-18T20:05:54+02:00 tester4 felhom-bootstrap.sh[3647]: [OK] Day-0 provision SUCCESS — vmid=9201 host_id=Tester4-31fa0e customer=Tester4 golden=local:backup/vzdump-lxc-9100-2026_10_18-20_04_04.tar.zst
|
||||
2026-10-18T20:05:54+02:00 tester4 felhom-bootstrap.sh[3647]: [INFO] root@pam was rotated + vaulted at step 4b — retrieve at hub → host page (the old GUI password no longer works).
|
||||
2026-10-18T20:05:54+02:00 tester4 felhom-bootstrap.sh[1122]: felhom-bootstrap: host-install SUCCESS — writing done-flag, disabling unit, scrubbing secrets
|
||||
2026-10-18T20:05:54+02:00 tester4 systemd[1]: felhom-bootstrap.service: Deactivated successfully.
|
||||
2026-10-18T20:05:54+02:00 tester4 systemd[1]: Finished felhom-bootstrap.service - Felhom host bootstrap (fetch + run felhom-host-install.sh unattended, retry until success).
|
||||
2026-10-18T20:05:54+02:00 tester4 systemd[1]: felhom-bootstrap.service: Consumed 1min 32.507s CPU time, 884.4M memory peak.
|
||||
=== agent
|
||||
2026-10-18T20:04:00+02:00 tester4 systemd[1]: Started felhom-agent.service - Felhom host agent (Proxmox host tier; hub control loop + PBS verify + storage watchdog).
|
||||
2026-10-18T20:04:00+02:00 tester4 felhom-agent[4670]: time=2026-10-18T20:04:00.468+02:00 level=INFO msg="felhom-agent daemon starting" version=0.159.2 host_id=Tester4-31fa0e hub_url=https://hub.felhom.eu interval_s=900
|
||||
2026-10-18T20:04:01+02:00 tester4 felhom-agent[4670]: time=2026-10-18T20:04:01.336+02:00 level=INFO msg="backup: restore-test scheduler starting (per-archive due-check)" eval_interval=6h0m0s settle=24h0m0s
|
||||
2026-10-18T20:04:01+02:00 tester4 felhom-agent[4670]: time=2026-10-18T20:04:01.336+02:00 level=INFO msg="pbs: verify loop starting" cadence=6h0m0s
|
||||
2026-10-18T20:04:01+02:00 tester4 felhom-agent[4670]: time=2026-10-18T20:04:01.336+02:00 level=INFO msg="storage: watchdog starting" interval=5s debounce=15s
|
||||
2026-10-18T20:04:01+02:00 tester4 sudo[4892]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/lvs --reportformat json --units b -o lv_name,data_percent,metadata_percent -- pve/data
|
||||
2026-10-18T20:04:01+02:00 tester4 sudo[4936]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/lvs --reportformat json --units b -o lv_name,data_percent,metadata_percent -- pve/data
|
||||
2026-10-18T20:04:01+02:00 tester4 felhom-agent[4670]: time=2026-10-18T20:04:01.874+02:00 level=INFO msg="selfheal: node watchdog starting" mode=appliance interval_s=60
|
||||
2026-10-18T20:04:01+02:00 tester4 felhom-agent[4670]: time=2026-10-18T20:04:01.875+02:00 level=INFO msg="guestnet: watchdog starting" interval=1m0s min_heal_interval=10m0s max_heals_per_hour=3 settle=3m0s
|
||||
2026-10-18T20:04:01+02:00 tester4 felhom-agent[4670]: time=2026-10-18T20:04:01.925+02:00 level=INFO msg="guest-power: watchdog started" interval=1m0s max_attempts=3
|
||||
2026-10-18T20:04:01+02:00 tester4 felhom-agent[4670]: time=2026-10-18T20:04:01.925+02:00 level=INFO msg="fstrim: weekly guest disk trim starting" schedule="weekly, due Wednesday from 10:00 host-local time; starts only 10:00-20:59; never beside a backup or restore-test"
|
||||
2026-10-18T20:04:01+02:00 tester4 felhom-agent[4670]: time=2026-10-18T20:04:01.925+02:00 level=INFO msg="janitor: starting (restore-test scratch retry + stale-lock sweep)" interval=10m0s
|
||||
2026-10-18T20:04:01+02:00 tester4 felhom-agent[4670]: time=2026-10-18T20:04:01.925+02:00 level=INFO msg="controller-supervisor: started" interval=30s confirm_sweeps=2 crashloop_max=3 crashloop_window=15m0s slow_crashloop_max=5 slow_crashloop_window=24h0m0s guests_dir=/var/lib/felhom-agent/guests
|
||||
2026-10-18T20:04:02+02:00 tester4 felhom-agent[4670]: time=2026-10-18T20:04:02.544+02:00 level=INFO msg="wgtunnel: registered with hub" pubkey="pUKC3lzCxXNARPkI70LtbsxdyW9QateJH1VDhAnvYD8=" assigned_ip=10.77.0.6/32 existed=false generation=1 sync=ok
|
||||
2026-10-18T20:04:03+02:00 tester4 felhom-agent[4670]: time=2026-10-18T20:04:03.198+02:00 level=INFO msg="wgtunnel: tunnel conf applied" endpoint=167.233.158.164:443 dns_name=ep0.felhom.eu assigned_ip=10.77.0.6/32 action=enable
|
||||
2026-10-18T20:04:07+02:00 tester4 sudo[5602]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/bin/systemctl restart dnsmasq
|
||||
2026-10-18T20:04:22+02:00 tester4 sudo[5850]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/lvs --reportformat json --units b -o lv_name,data_percent,metadata_percent -- pve/data
|
||||
2026-10-18T20:04:42+02:00 tester4 sudo[5963]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/lvs --reportformat json --units b -o lv_name,data_percent,metadata_percent -- pve/data
|
||||
2026-10-18T20:05:02+02:00 tester4 sudo[6050]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/lvs --reportformat json --units b -o lv_name,data_percent,metadata_percent -- pve/data
|
||||
2026-10-18T20:05:13+02:00 tester4 sudo[6939]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/lvs --reportformat json --units b -o lv_name,data_percent,metadata_percent -- pve/data
|
||||
2026-10-18T20:05:16+02:00 tester4 sudo[7665]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/lvs --reportformat json --units b -o lv_name,data_percent,metadata_percent -- pve/data
|
||||
2026-10-18T20:05:17+02:00 tester4 sudo[7695]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/lvs --reportformat json --units b -o lv_name,data_percent,metadata_percent -- pve/data
|
||||
2026-10-18T20:05:22+02:00 tester4 sudo[7847]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/lvs --reportformat json --units b -o lv_name,data_percent,metadata_percent -- pve/data
|
||||
2026-10-18T20:05:31+02:00 tester4 sudo[9006]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/lvs --reportformat json --units b -o lv_name,data_percent,metadata_percent -- pve/data
|
||||
2026-10-18T20:05:31+02:00 tester4 sudo[9182]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/lvs --reportformat json --units b -o lv_name,data_percent,metadata_percent -- pve/data
|
||||
2026-10-18T20:05:32+02:00 tester4 sudo[9449]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/lvs --reportformat json --units b -o lv_name,data_percent,metadata_percent -- pve/data
|
||||
2026-10-18T20:05:33+02:00 tester4 sudo[9536]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/lvs --reportformat json --units b -o lv_name,data_percent,metadata_percent -- pve/data
|
||||
2026-10-18T20:05:42+02:00 tester4 sudo[9917]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/lvs --reportformat json --units b -o lv_name,data_percent,metadata_percent -- pve/data
|
||||
2026-10-18T20:05:43+02:00 tester4 felhom-agent[4670]: time=2026-10-18T20:05:43.703+02:00 level=INFO msg="controller-swap: image file written, restarting bootstrap" vmid=9201 target=gitea.dooplex.hu/admin/felhom-controller:0.330.0
|
||||
2026-10-18T20:05:43+02:00 tester4 sudo[9961]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/pct exec 9201 -- systemctl restart felhom-controller-bootstrap.service
|
||||
@@ -0,0 +1,7 @@
|
||||
2026-10-18 18:04:02|0.159.2|0|0||
|
||||
2026-10-18 18:19:07|0.159.2|1|1||9201
|
||||
33652|2026-10-18 18:05:37|0.329.0|1
|
||||
33653|2026-10-18 18:05:51|0.330.0|1
|
||||
33657|2026-10-18 18:18:34|0.330.0|1
|
||||
33658|2026-10-18 18:19:27|0.330.0|1
|
||||
33659|2026-10-18 18:20:46|0.330.0|1
|
||||
@@ -0,0 +1,6 @@
|
||||
login 302
|
||||
{"data":{"powering_off":true},"ok":true}
|
||||
poweroff 202
|
||||
18:24:15
|
||||
18:24:45 VM 9302: status: stopped
|
||||
Tester4-31fa0e|2026-10-18 18:24:18|2026-10-18T18:24:15Z
|
||||
@@ -0,0 +1,12 @@
|
||||
read 1: 9302 name=newbox-r1148
|
||||
read 2: 9302 name=newbox-r1148
|
||||
scsi0: nvme-scratch:9302/vm-9302-disk-0.raw,size=120G
|
||||
purging VM 9302 from related configurations..
|
||||
Configuration file 'nodes/demo-hp/qemu-server/9302.conf' does not exist
|
||||
ls: cannot access '/mnt/hdd_1/images/9302': No such file or directory
|
||||
VMID NAME STATUS MEM(MB) BOOTDISK(GB) PID
|
||||
341 night1004-tester1 running 8192 200.00 1851
|
||||
VMID Status Lock Name
|
||||
9201 running demo-hp
|
||||
9202 running demo-hp-scratch
|
||||
9401 stopped upgrade-harness
|
||||
@@ -0,0 +1,33 @@
|
||||
# R-1148 — a fresh install from nothing with installer 1.34.0 (2026-10-18 evening)
|
||||
|
||||
Throwaway customer **Tester4** (created by the operator; domain `tester4.test`, a reserved name that never resolves; no
|
||||
off-site, no PBS-DR — nothing on ep0). VM 9302 `newbox-r1148` on demo-hp (Tier 1; 8 GB / 4 cores / 120 GB raw on
|
||||
`nvme-scratch` = `/mnt/hdd_1` root) from the public ISO 1.29.0 (checksum = the published one). The ISO fetches
|
||||
`felhom-host-install.sh` at run time from `https://felhom.eu/scripts/` — served 1.34.0 (`SCRIPT_VERSION` read live). The
|
||||
graphical installer was driven with `qm sendkey`, the Hungarian keyboard kept, as a household would. The household's
|
||||
path: the mailed bind link + the pairing code on the console + the owner phrase (the operator put it in a 0600 file;
|
||||
shredded after), then the mailed setup code.
|
||||
|
||||
| file | what it shows |
|
||||
|---|---|
|
||||
| `02-timeline.txt` | every step with its time (UTC below the line) |
|
||||
| `03-bind.txt` | the bind page's answer: „Sikeres összekötés" |
|
||||
| `04-pve-login-check.txt` | CC's own PVE API login from DooPlex, status codes only: the **vaulted** value 200; the value typed at install 401 |
|
||||
| `05-claim-and-version.txt` | the claim (302 /) and the box's own `/api/selfupdate/status`: 0.329.0 → 0.330.0, `initiated_by: auto-floor` |
|
||||
| `07-first-app.txt` | BentoPDF installed through the dashboard's API: deploy 202, running, healthy |
|
||||
| `08-bootstrap-journal.txt` | the box's `felhom-bootstrap` journal (scanned for secret shapes: 0) |
|
||||
| `09-hub-reports.txt` | the hub's host reports (agent 0.159.2) and the controller's own reports (0.329.0, then 0.330.0) |
|
||||
| `10-poweroff.txt` | the household's „turn off" (202), the hub's `deliberate_off_at` |
|
||||
| `99-teardown.txt` | the name read back twice, VM 9302 destroyed with its disk; the other VMs listed |
|
||||
| `screens/` | installer and console |
|
||||
|
||||
**Step 4b (R-1145) in the full sequence:** `recovery password vaulted at the hub (stored) — now setting it on root@pam`
|
||||
→ chpasswd → `[OK] root@pam password vaulted, set, and the PVE login takes it`, all at 18:03:43 UTC; the hub's
|
||||
`host_recovery.set_at` is the same second. **The golden and the agent:** the hub manifest named agent v0.159.2 (sha
|
||||
`4d98868e…`) and golden v0.329.0 — the two vouched artifacts; both verified by sha256. The box came up on 0.329.0 and
|
||||
moved to the floor 0.330.0 by itself 2 min 47 s after the bind.
|
||||
|
||||
Not walked: an SSH key login — CC logged in with the vaulted password (`sshpass -f`, file 0600, shredded after).
|
||||
|
||||
**Left for the operator (session only):** delete customer Tester4 on the hub — allowed from 19:09 UTC (the last report
|
||||
18:24:18 + the 45-minute stale threshold); a 409 before that is by design (R-599).
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 21 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 157 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 137 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 131 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 134 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 134 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 136 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 134 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 133 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 133 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 150 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 149 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 133 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 141 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 126 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 15 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 14 KiB |
@@ -0,0 +1,8 @@
|
||||
18:41:08
|
||||
25293 bytes, mtime 2026-10-18 18:40:29.429360235 +0000, owner 1000
|
||||
text: R1018 BEFORE EDITED-FB-1849
|
||||
2026/10/18 20:39:38 POST | 200 | 172.18.0.7 | admin | 0ms | "/api/office/callback?auth=[redacted]&path=%2Fr1018%2Fteszt.docx&source=SCRATCH+HDD"
|
||||
2026/10/18 20:39:53 POST | 200 | 172.18.0.7 | admin | 0ms | "/api/office/callback?auth=[redacted]&path=%2Fr1018%2Fteszt.docx&source=SCRATCH+HDD"
|
||||
2026/10/18 20:39:53 [INFO ] OnlyOffice Server is downloading file: /r1018/teszt.docx (documentId: 795feeebca16ab3ab1aae0dd352e9932fffa9efe0a9a927d6c3100ce98f20e6d)
|
||||
2026/10/18 20:40:29 [INFO ] OnlyOffice callback: successfully saved document to path=/r1018/teszt.docx
|
||||
2026/10/18 20:40:29 POST | 200 | 172.18.0.7 | admin | 5ms | "/api/office/callback?auth=[redacted]&path=%2Fr1018%2Fteszt.docx&source=SCRATCH+HDD"
|
||||
@@ -0,0 +1,4 @@
|
||||
text: R1018 NC BEFORE
|
||||
18:45:21
|
||||
text: EDITED-NC-1847R1018 NC BEFORE EDITED-NC-1844
|
||||
18:47:25
|
||||
@@ -0,0 +1,14 @@
|
||||
18:48:16 files /login 502
|
||||
18:48:31 files /login 502
|
||||
18:48:46 files /login 502
|
||||
18:49:01 files /login 502
|
||||
18:49:16 files /login 502
|
||||
18:49:31 files /login 502
|
||||
18:49:46 files /login 502
|
||||
18:50:01 files /login 502
|
||||
18:50:16 files /login 502
|
||||
18:50:31 files /login 502
|
||||
18:50:46 files /login 502
|
||||
18:51:01 files /login 502
|
||||
18:51:16 files /login 502
|
||||
18:51:31 files /login 200
|
||||
@@ -0,0 +1,7 @@
|
||||
18:51:56 goto https://files.enkisfelhom.hu/files/SCRATCH%20HDD/r1018/teszt.docx -> https://files.enkisfelhom.hu/files/SCRATCH%20HDD/r1018/teszt.docx
|
||||
18:52:11 shot 11-fb-after-stop
|
||||
18:52:11 eval -> []
|
||||
18:52:11 goto https://cloud.enkisfelhom.hu/apps/onlyoffice/78 -> https://cloud.enkisfelhom.hu/apps/onlyoffice/78
|
||||
18:52:27 shot 12-nc-after-stop
|
||||
18:52:27 eval -> ["Nextcloud", [], "Nextcloud\nPage not found\n\nThe page could not be found on the server or you may not be allowed to view it.\n\nBack to Nextcloud\n\nNextcloud – a safe home for all your data"]
|
||||
LOG ('http', 404, 'https://cloud.enkisfelhom.hu/apps/onlyoffice/78')
|
||||
@@ -0,0 +1,6 @@
|
||||
## rc.1 walk 19:12:09
|
||||
FileBrowser file text: R1018 BEFORE EDITED-FB-1849
|
||||
Nextcloud file text: EDITED-NC-1847R1018 NC BEFORE EDITED-N RC1-NC-OKC-1844
|
||||
## rc.1 FileBrowser walk, second try 19:14:39
|
||||
mtime 2026-10-18 19:13:58.691684571 +0000
|
||||
FileBrowser file text: R1018 BEFORE EDITED-FB-1849 RC1-FB-OK
|
||||
@@ -0,0 +1,10 @@
|
||||
stop pressed 19:14:54
|
||||
{"ok": true, "message": "Stack onlyoffice stop completed"}
|
||||
1 19:15:06 files 200
|
||||
1 19:15:08 files 502
|
||||
43 19:15:10 files 200
|
||||
2026/10/18 19:15:06 manager.go:1463: [INFO] [stacks] Restarting stack: filebrowser
|
||||
2026/10/18 19:15:07 manager.go:1493: [INFO] [stacks] Stack filebrowser restarted successfully (took 1.1s)
|
||||
2026/10/18 19:15:07 lifecycle.go:61: [INFO] [integrations] Integration onlyoffice:filebrowser suspended (stack onlyoffice stopped)
|
||||
2026/10/18 19:15:07 lifecycle.go:61: [INFO] [integrations] Integration onlyoffice:nextcloud suspended (stack onlyoffice stopped)
|
||||
2026/10/18 19:15:09 filebrowser_gate.go:94: [INFO] [infra] filebrowser: default login refused for this run — joined traefik-public, files.<domain> served (R-1126)
|
||||
@@ -0,0 +1,31 @@
|
||||
# R-1018 — the OnlyOffice links, live (2026-10-18 evening)
|
||||
|
||||
Scratch 9202 (demo-hp, Tier 0), controller 0.330.0 then 0.331.0-rc.1. OnlyOffice 9.4.0 and Nextcloud 34.0.4 installed
|
||||
through the dashboard API (`POST /api/stacks/<n>/deploy`), each link switched on through `POST /api/integrations/
|
||||
onlyoffice/<target>` — the calls the app page makes. The browser: Playwright's headless Chrome shell on DooPlex, the box's
|
||||
public names (`files.`, `office.`, `cloud.` `enkisfelhom.hu`) mapped to 9202's LAN address (`--host-resolver-rules`), as
|
||||
in `audits/filebrowser-memory-ci-2026-10-17/tools/cdp_shot.py`; the driver is in the session scratchpad (`cdp.py`). The
|
||||
FileBrowser and Nextcloud passwords came from the dashboard's own reveal buttons into 0600 files, typed by CDP, never
|
||||
printed. Test documents: `r1018/teszt.docx` on the scratch drive, `r1018-nc.docx` in Nextcloud (WebDAV).
|
||||
|
||||
| Step | 0.330.0 | 0.331.0-rc.1 |
|
||||
|---|---|---|
|
||||
| link on → files.<domain> | **502 for 4 min 8 s** (FileBrowser re-created off traefik-public, R-1126, until the next 5-min round) | 200 throughout |
|
||||
| FileBrowser → open .docx | editor frame loads, document never does: OnlyOffice fetched `https://files.<domain>/api/resources/download…` → **403** (the public name leads to another box, 9201) — `screens/04` | opens (`screens/15`) |
|
||||
| FileBrowser → edit, close, read the file on the drive | — (by-hand `server.internalUrl` on 0.330.0: „R1018 BEFORE EDITED-FB-1849", `01-filebrowser-save-readback.txt`) | „… RC1-FB-OK", mtime 19:13:58 (`05-rc1-walk.txt`) |
|
||||
| Nextcloud → open .docx | opens, edits, „All changes saved" (`screens/09`) — **but the file keeps the old text**: Nextcloud fetched `http://onlyoffice:80cache/files/…`, „Track: 78 status 2 error" (`02-nextcloud-save-readback.txt`) | the link re-applied the value with its slash at OnlyOffice's start; the file holds „RC1-NC-OK" |
|
||||
| OnlyOffice stopped | FileBrowser: no editor frame; Nextcloud `/apps/onlyoffice/78` **404** — the revoke-on-stop rule (`lifecycle.go:10-13`) holds (`04-after-stop-browser.txt`, `screens/11`, `12`); files.<domain> **502 for 3 min 38 s** (`03-files-outage-after-stop.txt`) | one 502 sample (≈ 2 s), FileBrowser re-published 3 s after its restart (`06-rc1-files-after-stop.txt`) |
|
||||
|
||||
Red-proofs of the three fixes: `redproof-fix1.txt` (FileBrowser `server.internalUrl`), `redproof-fix2.txt` (Nextcloud
|
||||
`DocumentServerInternalUrl` slash), `redproof-fix3.txt` (the gate after `RestartStack("filebrowser")`).
|
||||
|
||||
Also seen, not fixed (outside the links): FileBrowser's `/api/resources/preview` answers 500 for the .docx (an office
|
||||
preview goes through OnlyOffice's converter); Nextcloud's previews log `Request to http://onlyoffice:80converter` on
|
||||
0.330.0 — the same missing slash, fixed with it.
|
||||
|
||||
One first rc.1 FileBrowser attempt showed OnlyOffice's „Connection is lost" after the driver clicked two fixed spots
|
||||
where pop-ups had been the time before (none were there); the next attempt, with no blind clicks, saved. No restart of
|
||||
either container happened in between (both „Up 5–6 minutes").
|
||||
|
||||
Teardown: OnlyOffice removed through the product (`remove_hdd_data`, `remove_backups`: both volumes removed). Nextcloud
|
||||
stays on 9202 for Part 6 of the same brief; the test documents are removed with it.
|
||||
@@ -0,0 +1,9 @@
|
||||
## red-proof 1: the internalUrl line removed
|
||||
=== RUN TestFileBrowserConfig_R1018_InternalURLIsTheContainer
|
||||
infra_test.go:269: server.internalUrl = "" — OnlyOffice would be handed the public files address
|
||||
--- FAIL: TestFileBrowserConfig_R1018_InternalURLIsTheContainer (0.00s)
|
||||
FAIL
|
||||
FAIL gitea.dooplex.hu/admin/felhom-controller/internal/infra 0.008s
|
||||
## restored
|
||||
--- PASS: TestFileBrowserConfig_R1018_InternalURLIsTheContainer (0.00s)
|
||||
ok gitea.dooplex.hu/admin/felhom-controller/internal/infra 0.007s
|
||||
@@ -0,0 +1,10 @@
|
||||
## red-proof 2: the slash removed from ncDocServerInternalURL
|
||||
=== RUN TestNextcloudApply_R1018_InternalAddressesTakeAPath
|
||||
onlyoffice_nextcloud_test.go:34: DocumentServerInternalUrl "http://onlyoffice:80" + "cache/files/data/2153553391_5872/output.docx/output.docx" is not a URL: parse "http://onlyoffice:80cache/files/data/2153553391_5872/output.docx/output.docx": invalid port ":80cache" after host
|
||||
onlyoffice_nextcloud_test.go:34: DocumentServerInternalUrl "http://onlyoffice:80" + "converter?shardkey=thumb_1" is not a URL: parse "http://onlyoffice:80converter?shardkey=thumb_1": invalid port ":80converter" after host
|
||||
--- FAIL: TestNextcloudApply_R1018_InternalAddressesTakeAPath (0.00s)
|
||||
FAIL
|
||||
FAIL gitea.dooplex.hu/admin/felhom-controller/internal/integrations 0.007s
|
||||
## restored
|
||||
--- PASS: TestNextcloudApply_R1018_InternalAddressesTakeAPath (0.00s)
|
||||
ok gitea.dooplex.hu/admin/felhom-controller/internal/integrations (cached)
|
||||
@@ -0,0 +1,9 @@
|
||||
## red-proof 3: the gate call removed from RestartStack
|
||||
=== RUN TestRestartStack_R1018_FileBrowserPublishedAgainAtOnce
|
||||
filebrowser_restart_gate_test.go:129: FileBrowser was not published again after its restart — files.<domain> stays 502 until a base-stack round
|
||||
--- FAIL: TestRestartStack_R1018_FileBrowserPublishedAgainAtOnce (10.31s)
|
||||
FAIL
|
||||
FAIL gitea.dooplex.hu/admin/felhom-controller/internal/stacks 10.318s
|
||||
## restored
|
||||
--- PASS: TestRestartStack_R1018_FileBrowserPublishedAgainAtOnce (0.31s)
|
||||
ok gitea.dooplex.hu/admin/felhom-controller/internal/stacks 0.320s
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 7.9 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 17 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 69 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 46 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 71 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 45 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 89 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 54 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 55 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 66 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 12 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 323 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 50 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 56 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 44 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 45 KiB |
@@ -0,0 +1,47 @@
|
||||
## 19:43:05 — FileBrowser 1.5.6 with auth.methods.proxy {enabled, header X-Felhom-User} on scratch 9202
|
||||
### control route filesopen (no middleware)
|
||||
no header: 401
|
||||
forged header: 200 body: {"name":"scratch_hdd","size":132976640,"modified":"2026-10-18T20:34:20.752138936+02:00","type":"directory","hidden":fals
|
||||
### strip route filesspike (forwardAuth -> controller /api/health, authResponseHeaders [X-Felhom-User])
|
||||
no header: 404 body:
|
||||
<!DOCTYPE html>
|
||||
<html lang="hu">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initi
|
||||
forged header: 404 body:
|
||||
<!DOCTYPE html>
|
||||
<html lang="hu">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initi
|
||||
forged, other case: 404
|
||||
forged, twice: 404
|
||||
|
||||
## 19:44:02 — auth stand-in: /always = 200 and NO user header; /cookie = 200 + X-Felhom-User: household-sso only with cookie spike=ok
|
||||
### filesspike (forwardAuth /always, authResponseHeaders [X-Felhom-User])
|
||||
header [none]: 401 {"status":401,"message":"no token present in request"}
|
||||
header [X-Felhom-User: spike-stranger2]: 401 {"status":401,"message":"no token present in request"}
|
||||
header [x-felhom-user: spike-stranger3]: 401 {"status":401,"message":"no token present in request"}
|
||||
two forged headers: 401 {"status":401,"message":"no token present in request"}
|
||||
### filessso (forwardAuth /cookie)
|
||||
forged header, no cookie: 401
|
||||
cookie ok, no header: 404 404 page not found
|
||||
cookie ok + forged header attacker: 404 404 page not found
|
||||
### filessso again, on the drive listing (19:44:14)
|
||||
cookie ok, no header: 200 {"name":"scratch_hdd","size":132976640,"modified":"2026-10-18T20:34:20.752138936+02:00","t
|
||||
cookie ok + forged header attacker: 200 {"name":"scratch_hdd","size":132976640,"modified":"2026-10-18T20:34:20.752138936+02:00","t
|
||||
cookie wrong + forged header attacker: 401
|
||||
### FileBrowser's own request log (user column) since 19:43
|
||||
2026/10/18 21:43:05[33m GET | 401 | 172.18.0.3 | N/A | 0ms | "/api/resources?path=%2F&source=SCRATCH%20HDD"[0m
|
||||
2026/10/18 21:43:05 GET | 200 | 172.18.0.3 | spike-stra.. | 80ms | "/api/resources?path=%2F&source=SCRATCH%20HDD"
|
||||
2026/10/18 21:44:02[33m GET | 401 | 172.18.0.3 | N/A | 0ms | "/api/resources?path=%2F&source=SCRATCH%20HDD"[0m
|
||||
2026/10/18 21:44:02[33m GET | 401 | 172.18.0.3 | N/A | 0ms | "/api/resources?path=%2F&source=SCRATCH%20HDD"[0m
|
||||
2026/10/18 21:44:02[33m GET | 401 | 172.18.0.3 | N/A | 0ms | "/api/resources?path=%2F&source=SCRATCH%20HDD"[0m
|
||||
2026/10/18 21:44:02[33m GET | 401 | 172.18.0.3 | N/A | 0ms | "/api/resources?path=%2F&source=SCRATCH%20HDD"[0m
|
||||
2026/10/18 21:44:14 GET | 200 | 172.18.0.3 | household-.. | 79ms | "/api/resources?path=%2F&source=SCRATCH%20HDD"
|
||||
2026/10/18 21:44:14 GET | 200 | 172.18.0.3 | household-.. | 2ms | "/api/resources?path=%2F&source=SCRATCH%20HDD"
|
||||
### an app container on traefik-public talks to FileBrowser directly, no router (19:44:27)
|
||||
from nextcloud, no header: 401
|
||||
from nextcloud, forged X-Felhom-User: app-forged: 200
|
||||
nextcloud networks: nextcloud_nextcloud-internal traefik-public
|
||||
@@ -0,0 +1,6 @@
|
||||
login 200 token_len 355
|
||||
users 200 [(1, 'admin', 'password'), (2, 'spike-stranger', 'proxy'), (3, 'household-sso', 'proxy'), (4, 'app-forged', 'proxy')]
|
||||
delete spike-stranger 200
|
||||
delete household-sso 200
|
||||
delete app-forged 200
|
||||
users after 200 [(1, 'admin')]
|
||||
@@ -0,0 +1,11 @@
|
||||
0
|
||||
2026/10/18 21:46:05 [INFO ] Auth Methods : [password]
|
||||
0
|
||||
controller.yml
|
||||
filebrowser.yml
|
||||
forwarded.yml
|
||||
serverstransports.yml
|
||||
felhom-filebrowser traefik-public
|
||||
filesspike: 404
|
||||
filesopen: 404
|
||||
files: 401
|
||||
@@ -0,0 +1,59 @@
|
||||
# R-1100 — one sign-in for the dashboard and FileBrowser: the spike (2026-10-18 evening)
|
||||
|
||||
Measured on scratch 9202 (demo-hp, Tier 0): FileBrowser Quantum `gtstef/filebrowser:1.5.6-stable` (source read at tag
|
||||
`v1.5.6-stable`, commit `5d9b4df2`), Traefik v3.7.13, controller 0.331.0-rc.1. Nothing was built; every change on 9202 was
|
||||
removed (`03-teardown.txt`). Evidence: `01-strip.txt` (every request and answer), `02-teardown-users.txt`.
|
||||
|
||||
## What FileBrowser's proxy auth is (source, at the pinned tag)
|
||||
|
||||
- The config key is **`auth.methods.proxy`** (`backend/common/settings/auth.go:58`, json `proxy` — not `proxyAuth`, as the
|
||||
row said), with `enabled` and `header` (`auth.go:75-78`: „FileBrowser blindly accepts the header value as username").
|
||||
- Any request that carries the header is that user (`backend/http/middleware.go:302-307` on login, `:384-401` on every API
|
||||
call, `:494-520` `getProxyUser`). **A name FileBrowser has never seen is created on the spot**
|
||||
(`backend/http/auth.go:81-106`), with the user defaults. A name that exists as a password user is refused
|
||||
(`auth.go:107-108`, `ErrWrongLoginMethod`) — so the household's `admin` cannot be reached by a header; a header user
|
||||
is a separate FileBrowser user.
|
||||
|
||||
## What was measured
|
||||
|
||||
| Request | Answer |
|
||||
|---|---|
|
||||
| a route with no middleware, forged `X-Felhom-User: spike-stranger` | **200 — the drive's listing**; FileBrowser created the user |
|
||||
| Traefik `forwardAuth` that answers 200 and sets NO user header, `authResponseHeaders: [X-Felhom-User]`; forged header (also lower-case, also twice) | **401** from FileBrowser („no token present") — Traefik removed the client's header |
|
||||
| `forwardAuth` that sets `X-Felhom-User: household-sso` only for a valid cookie; cookie ok + forged `attacker` | 200 as **`household-sso`** (FileBrowser's own request log names the user) — the auth answer replaced the forged value |
|
||||
| the same, wrong cookie + forged header | 401 from the auth step; FileBrowser never reached |
|
||||
| **from inside the Nextcloud container**, straight to `http://filebrowser:80`, forged header | **200** — no Traefik on that path |
|
||||
|
||||
So: **Traefik can pass the header only for a valid session and strips one a client sends — proven.** But the header is
|
||||
also trusted from any container that reaches FileBrowser's port, and every app sits on `traefik-public` with it.
|
||||
|
||||
## The three things a build must solve
|
||||
|
||||
1. **The session cannot be read on `files.<domain>`.** The dashboard cookie `felhom_session` has no `Domain`
|
||||
(`controller/internal/web/auth.go:197-205`), so a browser never sends it to `files.<domain>`, and a forward-auth there
|
||||
sees nothing. **Do not widen it to `Domain=<domain>`**: every app's subdomain — third-party code — would then receive the
|
||||
dashboard session. Instead a files-only ticket: „Fájlok ↗" goes through the dashboard, which mints a one-time,
|
||||
~60-second token; `files.<domain>/_felhom/sso?t=…` (a Traefik route to the controller) swaps it for a host-only,
|
||||
HttpOnly, Secure cookie for `files.<domain>`; the forward-auth checks that cookie and answers the header.
|
||||
2. **Any app container can forge the header** (measured). Two cheap closes, either enough, both better:
|
||||
(a) **the header name is a per-box secret** (`X-Felhom-User-<32 hex>`), written only into FileBrowser's config and
|
||||
Traefik's dynamic file — an app that does not know the name cannot send it; (b) FileBrowser **off `traefik-public`**:
|
||||
Traefik joins `felhom-filebrowser` instead (and OnlyOffice too, for its `server.internalUrl`, R-1018). (b) changes the
|
||||
R-1126 gate's shape; (a) does not.
|
||||
3. **Header users are new FileBrowser users.** The household's files permissions for that user must be set by the box (the
|
||||
user defaults decide today); shares the household made as `admin` stay `admin`'s; logout needs
|
||||
`logoutRedirectUrl` to the dashboard.
|
||||
|
||||
## Unchanged by it
|
||||
|
||||
- **The password reveal (R-513)** stays: `admin` remains a password user — the fallback when the ticket path fails, and
|
||||
the operator's. The „Fájlok ↗" button would no longer need it.
|
||||
- **The R-1126 gate** keeps checking `admin`'s default login; header users do not touch it.
|
||||
|
||||
## Cost and recommendation
|
||||
|
||||
Controller only (no agent, no hub): a ticket endpoint and a forward-auth endpoint, one Traefik route + middleware, the
|
||||
FileBrowser config block with a secret header name, the header user's permissions, tests for each refusal (forged header
|
||||
via Traefik; forged header from an app container — with a secret name; stale/replayed ticket; wrong host). About one
|
||||
release. **Recommendation: build it with (1) the ticket cookie and (2a) the secret header name**; do not ship header
|
||||
trust without 2a — the measured app-container path would hand every app the household's files.
|
||||
@@ -0,0 +1,41 @@
|
||||
run at 19:49:37
|
||||
{"ok": true, "message": "Mentés elindítva"}
|
||||
2026/10/18 19:49:38 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container immich-server (image=ghcr.io/immich-app/immich-server:v3.2.4, not a database)
|
||||
2026/10/18 19:49:38 dbdump.go:178: [DEBUG] DiscoverDatabases: found postgres container: immich-postgres (id=5865007e1ebd)
|
||||
2026/10/18 19:49:38 dbdump.go:198: [DEBUG] DiscoverDatabases: immich-postgres → stack=immich, dbUser=immich, dbName=immich
|
||||
2026/10/18 19:49:38 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container immich-machine-learning (image=ghcr.io/immich-app/immich-machine-learning:v3.2.4, not a database)
|
||||
2026/10/18 19:49:38 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container immich-redis (image=redis:7-alpine, not a database)
|
||||
2026/10/18 19:49:39 dbdump.go:277: [DEBUG] DumpOne: starting dump for container=immich-postgres, stack=immich, dbType=postgres, dest=/mnt/sizing-bench/backups/primary/immich/db-dumps/immich-postgres.s
|
||||
2026/10/18 19:49:39 dbdump.go:314: [DEBUG] DumpOne: pg_dump command: docker exec 5865007e1ebd pg_dump -U immich -d immich --clean --if-exists --no-owner --no-privileges
|
||||
2026/10/18 19:49:40 dbdump.go:424: [DEBUG] DumpOne: completed immich-postgres → immich-postgres.sql (size=51.2 MB, valid=true, tables=71, duration=1.179s)
|
||||
2026/10/18 19:49:40 dbdump.go:429: [INFO] [backup] DB dump: immich-postgres → immich-postgres.sql (51.2 MB, 1.179s, 71 tables)
|
||||
2026/10/18 19:49:40 data_versions.go:131: [DEBUG] [backup] immich: stamped db-dumps/immich-postgres.sql (53711499 B) with pins [ghcr.io/immich-app/immich-server:v3.2.4@sha256:d317916b28090c33eb36b3084
|
||||
2026/10/18 19:49:40 backup.go:1003: [INFO] [backup] Stopping immich for safe volume dump
|
||||
2026/10/18 19:49:41 backup.go:877: [DEBUG] [backup] Dumping volume immich_immich_ml_cache for immich
|
||||
unit=
|
||||
/mnt/sizing-bench/backups/primary/immich/:
|
||||
total 36
|
||||
drwxr-xr-x 5 root root 4096 19:50:44 .
|
||||
drwxr-xr-x 3 root root 4096 14:09:21 ..
|
||||
drwxr-xr-x 2 root root 4096 19:50:44 compose
|
||||
-rw-r--r-- 1 root root 4889 19:49:49 data-stamps.json
|
||||
drwxr-xr-x 2 root root 4096 19:49:40 db-dumps
|
||||
-rw-r--r-- 1 root root 7983 19:50:44 manifest.json
|
||||
drwxr-xr-x 2 root root 4096 19:49:49 volume-dumps
|
||||
|
||||
/mnt/sizing-bench/backups/primary/immich/volume-dumps/:
|
||||
total 1111744
|
||||
drwxr-xr-x 2 root root 4096 19:49:49 .
|
||||
drwxr-xr-x 5 root root 4096 19:50:44 ..
|
||||
-rw-r--r-- 1 root root 823764480 19:49:42 immich_immich_ml_cache.tar
|
||||
-rw-r--r-- 1 root root 283049472 19:49:47 immich_immich_postgres_data.tar
|
||||
-rw-r--r-- 1 root root 31591424 19:49:49 immich_immich_redis_data.tar
|
||||
2026/10/18 19:49:41 backup.go:877: [DEBUG] [backup] Dumping volume immich_immich_ml_cache for immich
|
||||
2026/10/18 19:49:46 backup.go:877: [DEBUG] [backup] Dumping volume immich_immich_postgres_data for immich
|
||||
2026/10/18 19:49:49 backup.go:877: [DEBUG] [backup] Dumping volume immich_immich_redis_data for immich
|
||||
2026/10/18 19:49:49 backup.go:1019: [INFO] [backup] Restarting immich after volume dump
|
||||
2026/10/18 19:49:59 manager.go:920: [DEBUG] [stacks] restart-policy of down member "immich-server" = "unless-stopped"
|
||||
2026/10/18 19:50:44 recovery_unit.go:289: [INFO] [backup] Recovery unit captured for immich → /mnt/sizing-bench/backups/primary/immich (images=4, secrets-referenced=1, data_keys=
|
||||
immich_immich_ml_cache
|
||||
immich_immich_postgres_data
|
||||
immich_immich_redis_data
|
||||
@@ -0,0 +1,13 @@
|
||||
edited /opt/docker/stacks/immich/docker-compose.yml
|
||||
132c132
|
||||
< command: redis-server --appendonly no
|
||||
---
|
||||
> command: redis-server --save "" --appendonly no
|
||||
135,136c135,136
|
||||
< volumes:
|
||||
< - immich_redis_data:/data
|
||||
---
|
||||
> tmpfs:
|
||||
> - /data
|
||||
152d151
|
||||
< immich_redis_data:
|
||||
@@ -0,0 +1,44 @@
|
||||
restart at 19:52:31
|
||||
{"ok": true, "data": {"state": "starting"}, "message": "Stack immich restart requested — state now: starting"}
|
||||
immich-server Up About a minute (healthy)
|
||||
immich-postgres Up About a minute (healthy)
|
||||
immich-machine-learning Up About a minute (healthy)
|
||||
immich-redis Up About a minute (healthy)
|
||||
mounts: volume:immich_immich_redis_data/data tmpfs: map[/data:]
|
||||
save
|
||||
39
|
||||
backgroundTask backupDatabase duplicateDetection editor faceDetection facialRecognition integrityCheck library metadataExtraction migration notifications ocr
|
||||
{"res":"pong"}
|
||||
[32m[Nest] 29 - [39m10/18/2026, 9:52:51 PM [32m LOG[39m [33m[Api:RoutesResolver][39m [32mJobController {/api/jobs}:[39m
|
||||
[32m[Nest] 29 - [39m10/18/2026, 9:52:51 PM [32m LOG[39m [33m[Api:RouterExplorer][39m [32mMapped {/api/jobs, GET} route[39m
|
||||
[32m[Nest] 29 - [39m10/18/2026, 9:52:51 PM [32m LOG[39m [33m[Api:RouterExplorer][39m [32mMapped {/api/jobs, POST} route[39m
|
||||
[32m[Nest] 29 - [39m10/18/2026, 9:52:51 PM [32m LOG[39m [33m[Api:RouterExplorer][39m [32mMapped {/api/jobs/:name, PUT} route[39m
|
||||
[32m[Nest] 29 - [39m10/18/2026, 9:52:51 PM [32m LOG[39m [33m[Api:RouterExplorer][39m [32mMapped {/api/queues/:name/jobs, GET} route[39m
|
||||
[32m[Nest] 29 - [39m10/18/2026, 9:52:51 PM [32m LOG[39m [33m[Api:RouterExplorer][39m [32mMapped {/api/queues/:name/jobs, DELETE} route[39m
|
||||
old volume: immich_immich_redis_data labels=map[com.docker.compose.config-hash:730e1778edc1a7fb43e276ccfc1c6e7d304775266ab19bae03c9ebec05eaf65b com.docker.compose.project:immich com.docker.compose.version:5.5.1 com.docker.compose.volume:immich_redis_data]
|
||||
used by: immich-redis
|
||||
tmpfs /data tmpfs rw,nosuid,nodev,noexec,relatime,mode=755,uid=100000,gid=100000,inode64 0 0
|
||||
total 4
|
||||
drwxr-xr-x 2 redis root 40 Oct 18 21:52 .
|
||||
drwxr-xr-x 1 root root 4096 Oct 18 21:52 ..
|
||||
[{"Type":"volume","Name":"immich_immich_redis_data","Source":"/var/lib/docker/volumes/immich_immich_redis_data/_data","Destination":"/data","Driver":"local","Mode":"z","RW":true,"Propagation":""}]
|
||||
{"com.docker.compose.config-hash":"888272c2acd984cb6996f81bd49b41bdd8d6b2318870cd1a2088ec554dbf0fed"
|
||||
"com.docker.compose.depends_on":""
|
||||
"com.docker.compose.replace":"immich-redis"
|
||||
19:54:10 backgroundTask id= completed=0 failed=0
|
||||
19:54:10 library id= completed=0 failed=0
|
||||
19:54:10 migration id= completed=0 failed=0
|
||||
19:54:10 notifications id= completed=0 failed=0
|
||||
19:54:10 metadataExtraction id= completed=0 failed=0
|
||||
19:54:10 thumbnailGeneration id= completed=0 failed=0
|
||||
19:55:27 backgroundTask id= completed=0 failed=0
|
||||
19:55:27 library id= completed=0 failed=0
|
||||
19:55:27 migration id= completed=0 failed=0
|
||||
19:55:27 notifications id= completed=0 failed=0
|
||||
19:55:27 metadataExtraction id= completed=0 failed=0
|
||||
19:55:27 thumbnailGeneration id= completed=0 failed=0
|
||||
token_len 39
|
||||
photos 10 videos 0
|
||||
start metadataExtraction (force): 200
|
||||
metadataExtraction {'active': 0, 'completed': 0, 'failed': 0, 'delayed': 0, 'waiting': 0, 'paused': 0} {'isPaused': False, 'isActive': False}
|
||||
redis: metadataExtraction id=11
|
||||
@@ -0,0 +1,16 @@
|
||||
restore pressed 19:56:29
|
||||
302
|
||||
2026/10/18 19:56:29 handlers.go:1831: [DEBUG] [web] backupRestoreHandler: stack=immich snapshot=latest from 172.18.0.3:48626
|
||||
2026/10/18 19:56:29 handlers.go:1868: [WARN] [web] restore refused for immich: unit carries no file leg (1 drive path(s))
|
||||
---
|
||||
immich-server Up 9 minutes (healthy)
|
||||
immich-postgres Up 9 minutes (healthy)
|
||||
immich-machine-learning Up 9 minutes (healthy)
|
||||
immich-redis Up 9 minutes (healthy)
|
||||
{"res":"pong"}
|
||||
redis mounts: volume:immich_immich_redis_data/data
|
||||
tmpfs /data tmpfs
|
||||
immich_immich_redis_data created=2026-10-14T14:06:06Z labels=map[com.docker.compose.config-hash:730e1778edc1a7fb43e276ccfc1c6e7d304775266ab19bae03c9ebec05eaf65b com.docker.compose.project:immich com.docker.compose.version:5.5.1 com.docker.compose.volume:immich_redis_data]
|
||||
|
||||
## second-drive unit restore pressed 20:03:36 (unit of 2026-10-18 00:30 UTC, made WITH immich_immich_redis_data.tar)
|
||||
302
|
||||
@@ -0,0 +1,31 @@
|
||||
## Nextcloud: tier-2 copy WITH the Redis volume, started 20:05:16
|
||||
{"ok": true, "message": "2. mentés elindítva"}
|
||||
20:05:17
|
||||
2026/10/18 20:05:17 auth.go:144: [DEBUG] [web] auth: valid session for POST /api/backup/tier2
|
||||
2026/10/18 20:05:17 router.go:78: [DEBUG] [api] POST /api/backup/tier2 (path=/backup/tier2)
|
||||
2026/10/18 20:05:17 router.go:1323: [INFO] [api] Manual Tier 2 (off-drive) backup triggered
|
||||
2026/10/18 20:05:27 tier2.go:470: [INFO] [backup] Tier 2 copied immich → /mnt/felhom-drives/scratch_hdd/backups/secondary/immich (1.2 GB, 2 leg(s), 11s)
|
||||
2026/10/18 20:05:30 tier2.go:470: [INFO] [backup] Tier 2 copied nextcloud → /mnt/sizing-bench/backups/secondary/nextcloud (1011.9 MB, 1 leg(s), 3s)
|
||||
2026/10/18 20:05:30 tier2.go:470: [INFO] [backup] Tier 2 copied paperless-ngx → /mnt/sizing-bench/backups/secondary/paperless-ngx (125.8 MB, 1 leg(s), 0s)
|
||||
2026/10/18 20:05:31 tier2.go:470: [INFO] [backup] Tier 2 copied privatebin → /mnt/felhom-drives/scratch_hdd/backups/secondary/privatebin (8.3 KB, 0 leg(s), 0s)
|
||||
2026/10/18 20:05:31 tier2.go:526: [INFO] [backup] Tier 2 run complete: 4 app(s) processed (incl. volume-only — F6)
|
||||
/mnt/felhom-drives/scratch_hdd/backups/secondary/immich
|
||||
/mnt/felhom-drives/scratch_hdd/backups/secondary/privatebin
|
||||
/mnt/sizing-bench/backups/secondary/nextcloud
|
||||
/mnt/sizing-bench/backups/secondary/paperless-ngx
|
||||
/mnt/sys_drive/felhom-data/backups/secondary/paperless-ngx
|
||||
/mnt/felhom-drives/scratch_hdd/backups/primary/nextcloud/volume-dumps/:
|
||||
total 964000
|
||||
drwxr-xr-x 2 root root 4096 19:50:15 .
|
||||
drwxr-xr-x 5 root root 4096 19:50:44 ..
|
||||
-rw-r--r-- 1 root root 170889216 19:50:05 nextcloud_nextcloud_db_data.tar
|
||||
-rw-r--r-- 1 root root 816219136 19:50:11 nextcloud_nextcloud_html.tar
|
||||
-rw-r--r-- 1 root root 11776 19:50:04 nextcloud_nextcloud_redis_data.tar
|
||||
|
||||
/mnt/sizing-bench/backups/secondary/nextcloud/recovery-unit/volume-dumps/:
|
||||
total 964004
|
||||
drwxr-xr-x 2 root root 4096 19:50:15 .
|
||||
drwxr-xr-x 5 root root 4096 19:50:44 ..
|
||||
-rw-r--r-- 1 root root 170889216 19:50:05 nextcloud_nextcloud_db_data.tar
|
||||
-rw-r--r-- 1 root root 816219136 19:50:11 nextcloud_nextcloud_html.tar
|
||||
-rw-r--r-- 1 root root 11776 19:50:04 nextcloud_nextcloud_redis_data.tar
|
||||
@@ -0,0 +1,26 @@
|
||||
edited /opt/docker/stacks/nextcloud/docker-compose.yml
|
||||
110c110
|
||||
< command: redis-server --appendonly no
|
||||
---
|
||||
> command: redis-server --save "" --appendonly no
|
||||
113,114c113,114
|
||||
< volumes:
|
||||
< - nextcloud_redis_data:/data
|
||||
---
|
||||
> tmpfs:
|
||||
> - /data
|
||||
131d130
|
||||
< nextcloud_redis_data:
|
||||
restart at 20:06:47
|
||||
{"ok": true, "data": {"state": "starting"}, "message": "Stack nextcloud restart requested — state
|
||||
nextcloud Up 52 seconds (healthy)
|
||||
nextcloud-db Up 58 seconds (healthy)
|
||||
nextcloud-redis Up 57 seconds (healthy)
|
||||
tmpfs /data tmpfs
|
||||
save
|
||||
{"installed":true,"version":"34.0.4.1","versionstring":"34.0.4","edition":"","maintenance":false,"needsDbUpgrade":false,"productname":"Nextcloud","extendedSupport":false}
|
||||
|
||||
cron rc=0
|
||||
5
|
||||
nextcloud-redis
|
||||
webdav GET: 200
|
||||
@@ -0,0 +1,49 @@
|
||||
## Nextcloud second-drive restore pressed 20:09:28 — the copy made WITH nextcloud_nextcloud_redis_data.tar; the live compose has NO Redis volume
|
||||
302
|
||||
2026/10/18 20:09:28 tier2_whole.go:304: [WARN] [backup] WHOLE restore for nextcloud from the second drive /mnt/sizing-bench/backups/secondary/nextcloud: files by the four rules (never delete, never ov
|
||||
2026/10/18 20:09:30 tier2_whole.go:326: [INFO] [backup] whole restore nextcloud: files restored=1 replaced=0 (live kept beside as *.felhom-20261018T200930Z) kept-newer=2 unchanged=106
|
||||
2026/10/18 20:09:30 restore_dbcreds.go:114: [INFO] [backup] Restore nextcloud: database password(s) [DB_PASSWORD MYSQL_ROOT_PASSWORD] come from the backup, [] stay as installed (older than the data) â
|
||||
2026/10/18 20:09:30 restore_unit.go:393: [INFO] [backup] Restoring nextcloud from recovery unit /mnt/sizing-bench/backups/secondary/nextcloud/recovery-unit: images=3, secrets recovered=3/3, data_keys=
|
||||
2026/10/18 20:09:30 restore.go:215: [INFO] [backup] Restoring Docker volume nextcloud_nextcloud_db_data for nextcloud
|
||||
---
|
||||
Error response from daemon: No such container: nextcloud
|
||||
|
||||
|
||||
error: no such object: nextcloud-redis
|
||||
Error response from daemon: No such container: nextcloud-redis
|
||||
nextcloud_nextcloud_redis_data created=2026-10-18T18:31:13Z labels=map[com.docker.compose.config-hash:d0c5aae5dee81a25758afbc72c23a510ae97ef5e5c37c02afce24bb9b83e5515 com.docker.compose.project:nextcloud com.docker.compose.version:5.6.0 com.docker.compose.volume:nextcloud_redis_data]
|
||||
webdav GET after restore: 404
|
||||
## after the restore (20:10:37)
|
||||
2026/10/18 20:09:30 restore.go:215: [INFO] [backup] Restoring Docker volume nextcloud_nextcloud_db_data for nextcloud
|
||||
2026/10/18 20:09:32 restore.go:253: [DEBUG] [backup] Volume nextcloud_nextcloud_db_data restored successfully
|
||||
2026/10/18 20:09:32 restore.go:215: [INFO] [backup] Restoring Docker volume nextcloud_nextcloud_html for nextcloud
|
||||
2026/10/18 20:09:42 restore.go:253: [DEBUG] [backup] Volume nextcloud_nextcloud_html restored successfully
|
||||
2026/10/18 20:09:42 restore.go:215: [INFO] [backup] Restoring Docker volume nextcloud_nextcloud_redis_data for nextcloud
|
||||
2026/10/18 20:09:43 restore.go:253: [DEBUG] [backup] Volume nextcloud_nextcloud_redis_data restored successfully
|
||||
2026/10/18 20:09:43 restore.go:258: [INFO] [backup] Restored 3 Docker volume(s) for nextcloud
|
||||
2026/10/18 20:09:43 restore_db.go:87: [INFO] [backup] Restore nextcloud: replaying DB dump into nextcloud-db (mariadb)
|
||||
2026/10/18 20:09:57 restore_db.go:97: [INFO] [backup] Restore nextcloud: replayed 1 DB dump(s)
|
||||
2026/10/18 20:10:06 restore.go:298: [DEBUG] [backup] Post-restore health check: nextcloud not yet running, waiting...
|
||||
2026/10/18 20:10:11 restore.go:293: [DEBUG] [backup] Post-restore health check: nextcloud is running
|
||||
2026/10/18 20:10:11 restore_unit.go:494: [INFO] [backup] Restore-from-unit completed: nextcloud — 3 volume(s) of 3 listed, 1 database(s) of 1 listed
|
||||
2026/10/18 20:10:11 tier2_restore.go:255: [INFO] [backup] nextcloud: primary unit already carries data — left untouched (R-403 never overwrites a richer package with a poorer one)
|
||||
2026/10/18 20:10:11 handlers.go:2281: [INFO] [web] Tier-2 whole restore completed (async): stack=nextcloud in 42.745438966s (files restored 1, replaced 0, kept-newer 2, unchanged 106; volumes 3/3, dbs 1/1)
|
||||
---
|
||||
nextcloud Up 36 seconds (healthy)
|
||||
nextcloud-redis Up 41 seconds (healthy)
|
||||
nextcloud-db Up 55 seconds (healthy)
|
||||
{"installed":true,"version":"34.0.4.1","versionstring":"34.0.4","edition":"","maintenance":false,"needsDbUpgrade":false,"productname":"Nextcloud","extendedSupport":false}
|
||||
|
||||
redis mounts: volume:nextcloud_nextcloud_redis_data/data
|
||||
/dev/loop1 /data ext4
|
||||
drwxr-xr-x 2 redis redis 4096 Oct 18 22:09 .
|
||||
drwxr-xr-x 1 root root 4096 Oct 18 22:09 ..
|
||||
-rw------- 1 redis redis 9368 Oct 18 21:50 dump.rdb
|
||||
nextcloud_nextcloud_redis_data created=2026-10-18T20:09:42Z labels=map[com.docker.compose.project:nextcloud com.docker.compose.version:5.6.0 com.docker.compose.volume:nextcloud_redis_data]
|
||||
total 20
|
||||
drwxr-xr-x 2 999 1000 4096 Oct 18 20:09 .
|
||||
drwxr-xr-x 1 root root 4096 Oct 18 20:10 ..
|
||||
-rw------- 1 999 1000 9368 Oct 18 19:50 dump.rdb
|
||||
used by: nextcloud-redis
|
||||
2
|
||||
webdav GET after restore: 200
|
||||
@@ -0,0 +1,13 @@
|
||||
## backup run with Immich on the NO-volume compose, 20:11:01
|
||||
1
|
||||
{"ok": true, "message": "Mentés elindítva"}
|
||||
20:12:06
|
||||
2026/10/18 20:11:04 backup.go:877: [DEBUG] [backup] Dumping volume immich_immich_ml_cache for immich
|
||||
2026/10/18 20:11:09 backup.go:877: [DEBUG] [backup] Dumping volume immich_immich_postgres_data for immich
|
||||
2026/10/18 20:11:20 recovery_unit.go:289: [INFO] [backup] Recovery unit captured for immich → /mnt/sizing-bench/backups/primary/immich (images=4, secrets-referenced=1, data_keys=0, portable-carried=
|
||||
total 1080892
|
||||
drwxr-xr-x 2 root root 4096 20:11:12 .
|
||||
drwxr-xr-x 5 root root 4096 20:11:20 ..
|
||||
-rw-r--r-- 1 root root 823764480 20:11:05 immich_immich_ml_cache.tar
|
||||
-rw-r--r-- 1 root root 283049472 20:11:10 immich_immich_postgres_data.tar
|
||||
manifest volume_dumps: ['immich_immich_ml_cache.tar', 'immich_immich_postgres_data.tar']
|
||||
@@ -0,0 +1,13 @@
|
||||
## after the template push (catalog 30eea97) and the sync, 20:25:49
|
||||
/opt/docker/stacks/nextcloud/docker-compose.yml:110: command: redis-server --save "" --appendonly no
|
||||
/opt/docker/stacks/nextcloud/docker-compose.yml:113: tmpfs:
|
||||
/opt/docker/stacks/immich/docker-compose.yml:17:# Redis data → immich_redis_data (named volume, NVMe)
|
||||
/opt/docker/stacks/immich/docker-compose.yml:132: command: redis-server --save "" --appendonly no
|
||||
/opt/docker/stacks/immich/docker-compose.yml:135: tmpfs:
|
||||
{"ok": true, "data": {"state": "starting"}, "message": "Stack nextcloud restart
|
||||
nextcloud Up 52 seconds (healthy)
|
||||
nextcloud-redis Up 57 seconds (healthy)
|
||||
nextcloud-db Up 57 seconds (healthy)
|
||||
tmpfs /data tmpfs
|
||||
save
|
||||
{"installed":true,"version":"34.0.4.1","versionstring":"34.0.4","edition":"","ma
|
||||
@@ -0,0 +1,22 @@
|
||||
22:28:23 [X] remove (with drive data) -> 200 {'ok': True, 'data': {'removed': 'nextcloud', 'volumes_removed': ['nextcloud_nextcloud_db_data', 'nextcloud_nextcloud_html'], 'hdd_paths_removed': ['/mnt/felhom
|
||||
22:28:32 [X] after remove: deployed=False leftovers='/opt/docker/stacks/nextcloud'
|
||||
remove: 200
|
||||
0
|
||||
nextcloud_nextcloud_redis_data
|
||||
ls: cannot access '/mnt/felhom-drives/scratch_hdd/backups/primary/nextcloud': No such file or directory
|
||||
ls: cannot access '/mnt/sizing-bench/backups/secondary/nextcloud': No such file or directory
|
||||
/mnt/felhom-drives/scratch_hdd/userdata/nextcloud
|
||||
ls: cannot access '/mnt/felhom-drives/scratch_hdd/userdata/r1018': No such file or directory
|
||||
S10-box-fresh-768-noswap.txt
|
||||
S2-box-positive.txt
|
||||
controller.yaml.pre-lock-probe
|
||||
daemon.json.bak-2026-10-04
|
||||
pgsample.sh
|
||||
r626-events.txt
|
||||
repoint-night1005.sh
|
||||
shots
|
||||
sizing
|
||||
0
|
||||
nextcloud_nextcloud_redis_data map[com.docker.compose.project:nextcloud com.docker.compose.version:5.6.0 com.docker.compose.volume:nextcloud_redis_data]
|
||||
nextcloud_nextcloud_redis_data
|
||||
0
|
||||
@@ -0,0 +1,9 @@
|
||||
## red-proof: the leftover loop removed from RemoveStack
|
||||
=== RUN TestR1117_RemoveDeletesALabelledVolumeTheDefinitionNoLongerDeclares
|
||||
r1117_leftover_volume_test.go:66: left behind: [app_app_redis_data] — a volume with the app's project label outlived its removal
|
||||
--- FAIL: TestR1117_RemoveDeletesALabelledVolumeTheDefinitionNoLongerDeclares (26.02s)
|
||||
FAIL
|
||||
FAIL gitea.dooplex.hu/admin/felhom-controller/internal/stacks 26.027s
|
||||
## restored
|
||||
--- PASS: TestR1117_RemoveDeletesALabelledVolumeTheDefinitionNoLongerDeclares (26.02s)
|
||||
ok gitea.dooplex.hu/admin/felhom-controller/internal/stacks 26.027s
|
||||
@@ -26,6 +26,17 @@
|
||||
|
||||
---
|
||||
|
||||
## 2026-10-18 (evening) — a fresh install from scratch, OnlyOffice saves, Redis without a volume, CI mail link (controller v0.331.0, catalog `30eea97`)
|
||||
|
||||
| Row | What | Closed | Evidence |
|
||||
|---|---|---|---|
|
||||
| **R-1148** | **Installer 1.34.0 is published (operator yes, 2026-10-18) but its default path has not run on a fresh install.** Step 4b now vaults FIRST, then sets the password, then checks a PVE login (R-1145); the SC2066 loop became an `if`. Proven: the harness (52/0, the R1145 cases run the real step with fake commands), `test_hostinstall.py` 40/0, a real `--rotate-recovery-only` on demo-hp (vault time moved, new value 200, old 401). Not proven: a whole fresh install on a machine, where step 4b runs inside the full sequence and `https://127.0.0.1:8006` must answer during it. (P3) | CLOSED 2026-10-18 — **proven on a fresh install:** VM 9302 on demo-hp from the public ISO 1.29.0, which fetched installer **1.34.0** at run time; throwaway customer Tester4 (the operator's). Step 4b in the full sequence: „vaulted at the hub (stored) — now setting it” → chpasswd → „[OK] … the PVE login takes it”, 18:03:43 UTC, the hub's `host_recovery.set_at` the same second; CC's own PVE login from DooPlex: vaulted value 200, the installer's value 401. The manifest named the vouched agent 0.159.2 and golden 0.329.0 (sha-verified); the box came up on 0.329.0 and moved to the floor 0.330.0 by itself (`initiated_by: auto-floor`) 2 min 47 s after the bind; claimed, BentoPDF installed and healthy; turned off by the household's button (deliberate). VM destroyed by name. **The operator keeps Tester4 as the standing fresh-install customer** (2026-10-18); before its next install the old host `Tester4-31fa0e` must be deleted on the Hosts page, or the enroll reuses it and skips the re-issues (memory „reinstall without deleting the host”). | `audits/newbox-2026-10-18/` |
|
||||
| **R-1018** | **The OnlyOffice links (FileBrowser, Nextcloud) have no recorded live proof since the move to Proxmox.** `controller/internal/integrations/` holds two handlers; no audit after the Proxmox move shows a document opened from FileBrowser or Nextcloud in OnlyOffice (searched 2026-10-14). The media-app links of `15` §6 will copy this code, including its revoke-on-stop rule (`lifecycle.go:10-13`). (P3) | CLOSED 2026-10-18 — **walked live and fixed, controller v0.331.0 (floor):** on 0.330.0 FileBrowser → OnlyOffice never loaded a document (OnlyOffice fetched the public `files.<domain>` URL: 403) and Nextcloud → OnlyOffice lost every save (`http://onlyoffice:80cache/…` — no slash); switching the link on or off left files.<domain> 502 for 3.5–4 min. Fixed: FileBrowser `server.internalUrl: http://filebrowser:80`, Nextcloud `DocumentServerInternalUrl` with its slash, the R-1126 gate after any FileBrowser restart. On rc.1 both paths open, edit, save and read back; the revoke-on-stop rule holds (no editor in FileBrowser, Nextcloud 404); the stop's outage ≈ 2 s. Each fix red-proofed. | `audits/onlyoffice-2026-10-18/` |
|
||||
| **R-1117** | **Immich's and Nextcloud's Redis keep a volume they do not need** (upstream runs both without one; queues and caches only). Deferred from R-1096 on purpose: removing a volume from the compose leaves the old volume on installed boxes, and the nightly dump / an old backup's restore for a volume the compose no longer declares is unchecked. The RDB switch (catalog `5d9450b`) already removed the full-disk danger. (P4) | CLOSED 2026-10-18 — **measured, then changed (catalog `30eea97`):** both Redis services run with `--save ""`, tmpfs `/data`, no volume. On scratch 9202 with the new shape: both apps healthy, Immich's forced metadata job ran (counter 0 → 11, 0 failed), Nextcloud `cron.php` rc 0; the nightly dump skips the dropped volume; a second-drive restore from a copy made WITH the volume succeeded and, by design, brought back that copy's compose. The old volume stays attached under the tmpfs; **a removal left it behind — fixed in controller v0.331.0** (the removal deletes volumes still labelled with the app's project; red-proofed). | `audits/redis-volumes-2026-10-18/` |
|
||||
| **R-1135** | **CI's failure mail links the run at the in-cluster Gitea address** (`http://gitea.gitea-system.svc.cluster.local:3000/...`), which a reader outside the cluster cannot open. Seen in the forced test mail 2026-10-17 (job 1997). (P4) | CLOSED 2026-10-18 — the alarm link is now `https://gitea.dooplex.hu/<repo>/actions/runs/<GITHUB_RUN_ID>` in all four workflows (felhom.eu `1fe9f848`, controller `615940d`, agent `79ba884`, catalog `c97eb1c`). The run NUMBER was a second defect: Gitea 1.26 reads it as an id first and opened an unrelated run (felhom.eu runs/1116 → id 1116 = #613). Proof: forced job 2067 (catalog) at 18:46:01Z, one mail (Gmail, 18:46:02Z), link `…/actions/runs/2067`, also printed as `ALARM-LINK` in the job log. | helper evidence in the session scratchpad `p5/`; job 2067 |
|
||||
| **R-1136** | **felhom.eu CI runs take ~6.9 minutes and complete, so the 5-minute job timeout R-265 assumes does not hold there** (runs 1996, 1997; `script-tests` alone ~356 s). Found by the R-1129 proof helper 2026-10-17. (P4) | CLOSED 2026-10-18 — measured: the runner's real limit is act_runner 0.6.1's default `runner.timeout` **3 h** (`act-runner-config` sets only capacity, fetch_timeout, fetch_interval); the job-level `timeout-minutes: 5` is parsed and NOT enforced by act_runner 0.6.1 (only step-level is) — 231 felhom.eu jobs ran past 300 s and passed (longest 529 s, job 2038). `script-tests` (~7 min) is far from 3 h: no speed-up. R-265 corrected; the step-level timeout is R-1152. | act_runner v0.6.1 `config.go:128`, `act/runner/runner.go:164`, `step.go:206` |
|
||||
| **R-1137** | **A CI rerun may overwrite a failed job's log** — failure mails arrived for controller commits `c02c9d83` and `29a3ce8c` while their jobs (1944, 1959) now read green. UNPROVEN (a reading, 2026-10-17). (P4) | CLOSED 2026-10-18 — **confirmed:** a rerun replaces the job record the `jobs` endpoint shows (same id, `run_attempt` 2, the rerun's conclusion and log); the first attempt survives only as a task in `actions/tasks` (`status` failure) and its log is not readable by the API (controller job 1944 = tasks 1962 fail + 1964 ok; 1959 = 1978 fail + 1980 ok). The pull check in felhom.eu `CLAUDE.md` now reads `actions/tasks` when `run_attempt` > 1 (`b9b56c47`). A scratch-repo red run was not possible: the admin token lacks `write:user` and push-to-create is off — measured on the two real cases instead. | felhom.eu `CLAUDE.md` CI pull-check item 3 |
|
||||
|
||||
## 2026-10-18 (day) — hub 0.158.0 live, „anyway" restarts the apps, vault-first installer 1.34.0, the estimator's Immich figure (controller v0.330.0, agent v0.159.2)
|
||||
|
||||
| Row | What | Closed | Evidence |
|
||||
@@ -1254,7 +1265,7 @@ Nine rows closed. **Full original text: `git show <this commit>^ -- documentatio
|
||||
| **R-258** | **C3 — the customer's per-app backup tick is green on the PRESENCE of a restore point, and its only red condition is a GLOBAL one.** | **CLOSED 2026-08-08 — controller v0.210.0.** `appDumpVerdict` reads THIS app's own dump result; three states, no icon when nothing is known. **Recency deliberately not added** — see the observation in the follow-through section | full text: `git show fddfe00ce268:documentation/backlog/OPEN-ITEMS.md` |
|
||||
| **R-259** | **C4 — a disk read that FAILS renders as „0.0 GB / 0.0 GB (0%)" in the nominal colour, on the dashboard's most-looked-at meter.** | **CLOSED 2026-08-08 — controller v0.210.0.** `readDiskUsage` reports success; `SystemInfo.DiskKnown`/`HDDKnown`; the template draws no figure, no percentage and no meter fill when unknown. **The hub leg is deliberately NOT fixed and is now R-266** | full text: `git show fddfe00ce268:documentation/backlog/OPEN-ITEMS.md` |
|
||||
| **R-260** | **C5 — the agent reports at least eight decision-bearing facts the hub models NOWHERE, and the sharpest one blinds the check that answers „can the operator get into this box".** | **CLOSED 2026-08-08** — the class is GATED (G-1, `scripts/wire_contract_gate.py`) and the sharpest instance is fixed (hub v0.99.0). The remaining unconsumed facts are **R-264, OPEN** — allowlisted with reasons, which is not the same as decided. See the follow-through section below | full text: `git show fddfe00ce268:documentation/backlog/OPEN-ITEMS.md` |
|
||||
| **R-265** | **A CI run can fail with NO LOG PERSISTED, and the alarm mail then points the operator at a log that does not exist.** | **CLOSED 2026-08-08 — `timeout-minutes: 5` on the gates job, and the alarm mail now states elapsed seconds and qualifies its own "names itself in the run log" sentence.** ⚠ **The unknown is NOT closed and must not be read as closed:** whether the `if: failure()` alarm fires for a REAPED job is still unverified. The timeout makes the reap unreachable in practice; it does not answer what happens inside one | full text: `git show fddfe00ce268:documentation/backlog/OPEN-ITEMS.md` |
|
||||
| **R-265** | **A CI run can fail with NO LOG PERSISTED, and the alarm mail then points the operator at a log that does not exist.** | **CLOSED 2026-08-08 — `timeout-minutes: 5` on the gates job, and the alarm mail now states elapsed seconds and qualifies its own "names itself in the run log" sentence.** ⚠ **The unknown is NOT closed and must not be read as closed:** whether the `if: failure()` alarm fires for a REAPED job is still unverified. The timeout makes the reap unreachable in practice; it does not answer what happens inside one **⚠ CORRECTED 2026-10-18 (R-1136): the `timeout-minutes: 5` this row closed on has NEVER been in force.** act_runner 0.6.1 parses a job-level `timeout-minutes` and does not enforce it (only step-level is), and 231 felhom.eu jobs ran past 5 minutes and passed (longest 529 s, job 2038). The limits really in force are the runner's 3 h default (`runner.timeout` not set in `act-runner-config`) and Gitea's server-side reapers (defaults: 10 min with no task update, 3 h endless; `app.ini` not read). A hung step on a live runner runs up to 3 h. The unknown stays open: whether `if: failure()` runs for a reaped job. Step-level timeout: R-1152. | full text: `git show fddfe00ce268:documentation/backlog/OPEN-ITEMS.md` |
|
||||
| **R-267** | **The Configuration page is 2.6× faster and is still ~10 s, and the remaining cost is ONE Gitea call whose latency swings 20× with load.** Shipped in 0.100.2, v0.100.0. | **CLOSED 2026-08-08 — hub v0.101.0 + a registry prune.** **Final: cold 5.4 s, warm 0.14 s** (was 26.2 s). Three serialisation legs took it to 9.85 s mean, the 60 s in-memory memo took the warm path to a quarter-second, and the prune halved what remains of the cold path. **⚠ TWO CORRECTIONS TO THIS ROW'S OWN EARLIER TEXT, because both were wrong and both mattered.** **(1) "Only 50 generic versions exist" WAS NOT A COUNT, IT WAS A PAGE LIMIT.** `?type=generic&limit=1000` returns at most 50; the 50 I measured was exactly the cap, and three older agent versions (0.81.0, 0.80.0, 0.79.0) only became visible after the first 30 deletions moved them onto page one. **An unpaginated listing is not evidence of a total** — this repo's own "an empty listing is not evidence of emptiness" rule, walked into while measuring. **(2) THE OPERATOR'S "REDUCE THE NUMBER OF ARTIFACTS" WAS THE BETTER CALL AND MY MEASUREMENT SAID OTHERWISE.** I reported it helps "sub-linearly" and "is not the lever". Measured after: trimming to 10+10 took the COLD load from 13.4 s to 5.4 s — a 2.5× improvement on the path the memo cannot help, because the fan-out is per-version. Recorded rather than quietly dropped (the R-96 standing rule). **Pruned to the newest 10 per package on the operator's rule**, with the live-vouched golden/agent/floor asserted into the KEEP set before a single DELETE was issued; 33 deletions, all HTTP 204, and golden 0.210.0 / agent 0.128.0 / agent 0.127.0 verified still fetchable afterwards. `drill-r50` runs agent 0.113.0, now deleted — flagged to the operator first; it is a disposable nested drill VM and only its re-download path is gone | full text: `git show fddfe00ce268:documentation/backlog/OPEN-ITEMS.md` |
|
||||
| **R-268** | **A live per-guest local-API token was printed into a session transcript.** Shipped in 169.254.253. | **CLOSED — ROTATED + PROVEN LIVE 2026-08-09** (rehearsal pre-phase, `audits/REHEARSAL-byo-reinstall-2026-08-09.md` §3) | full text: `git show fddfe00ce268:documentation/backlog/OPEN-ITEMS.md` |
|
||||
| **R-273** | **RANK 1 — the hub vouched an agent version that was never git-tagged, and every install fleet-wide now fails at step 5/8.** Shipped in 0.127.0, 0.128.0, v0.127.0. | **CLOSED 2026-08-09 — tag pushed, install PROVEN** | full text: `git show fddfe00ce268:documentation/backlog/OPEN-ITEMS.md` |
|
||||
|
||||
@@ -123,16 +123,15 @@ stopping line that lies.
|
||||
| **R-993** | Install & onboarding | P3 | **Operator: approve the minimum machine in `documentation/audits/sizing-2026-10-13/SIZING.md` §5** (the input R-991's hardware page needs). MEASURED 2026-10-13 on the N100 demo box (61 apps deployed and removed through the product, 15 min at rest each; heavy apps with one real action). Proposal: **minimum** x86-64, 4 cores (N100 class), **8 GB**, 256 GB SSD; **recommended 16 GB**, 512 GB SSD; out: ARM / Raspberry Pi, Synology-QNAP, routers, cloud VPS. Worked from the floor (2.5 GB for the whole box with no apps) + Immich, Jellyfin, Nextcloud, Home Assistant, Vaultwarden, Paperless-ngx at their peaks (5.0 GB) + 50 % headroom ≈ 12.3 GB. 8 GB runs that set without Immich; 16 GB runs all of it twice over. Data for the estimator: `sizing.json`. **2026-10-13 (estimator session) — the installer accepts a host BELOW this minimum:** `autosize_guest_caps` has no floor on the host, only on the guest; on a 4 GB box (MemTotal ≈ 3.5 GiB) the guest gets host − 1 GB ≈ 2.4 GB and Proxmox keeps ≈ 1 GB, where it used ≈ 2.0 GB on the N100 (SIZING.md §1). Whatever minimum is approved, nothing enforces it at install today. The three-verdict wording of §5 now matches the estimator page (`/gepigeny`, R-997). | **WAITING-ON-OPERATOR** — filed 2026-10-13 | the operator's decision | Operator: approve or change §5. If nothing is done: no hardware page; a buyer with a Raspberry Pi can still download the installer (R-991 stays open) | operator |
|
||||
| **R-494** | Install & onboarding | P4 | **NARROWED 2026-09-14 by operator ruling → [P3-LOW] the hub COULD create the tunnel at customer creation, for a domain already on Cloudflare. Not blocking: every customer has their own domain and the operator creates the tunnel per day-0 A.1 (`architecture/01-topology-and-trust.md`).** *Original finding, kept:* **[P1-HIGH] A new customer's dashboard has NO reachable address unless the operator hand-makes a Cloudflare tunnel — the link in the setup-code mail is dead.** MEASURED 2026-09-14 on a fresh install from the public ISO (drill intervention **I1**): the claim mail points at `https://felhom.drill0242.felhom.eu`; that name has **no A and no AAAA** record (`dig @1.1.1.1`, control `felhom.enkisfelhom.hu` resolves); the hub has **no tunnel- or DNS-creation code** (`hub/internal/cloudflare/` holds only geo-rule removal; `cf_tunnel_token` is a pasted, optional form field, `configs.go:1478`) — day-0 runbook A.1 makes it a manual Cloudflare-dashboard step that nothing on the customer-create page asks for; the box's own split-horizon resolver on the appliance LAN IP answered `google.com` but not the dashboard name at 13:27:39Z; the agent applied the record at **13:27:44Z** (`lanresolver: applied split-horizon record … ip=192.168.0.158`, 3 m 46 s after the controller started), so the box CAN answer the name — **but only to a device that uses the box as its DNS server, and no document, screen or mail tells a household to do that**; the router and the installer-offered DNS answer nothing. The page was reachable only at the guest's LAN address with the name forced (`curl --resolve …:443:192.168.0.158`). **A volunteer could not have done that.** **What it needs:** an operator ruling — the hub creates the tunnel and DNS at customer creation, or the product gives a household a LAN address that works with no DNS change. | **READY — rank P3-LOW; owner: CC** **Re-ranked 2026-10-03: P3→P4: operator-side automation; the operator creates the tunnel by hand per the day-0 runbook.** | — | — | CC |
|
||||
| **R-504** | Install & onboarding | P4 | **[P3-LOW] `iso.felhom.eu` cannot show an index page on its own — its root returns 404, and the download page lives on the website instead.** MEASURED 2026-09-14: `https://iso.felhom.eu/` and `/index.html` → 404; only named objects answer. The host is an R2 bucket behind a custom domain; whether R2 would serve an uploaded `index.html` at `/` was **not measured** (uploading anything to the public bucket is a publication). The ISO v1.27.0 task puts the Hungarian download page at `felhom.eu/letoltes` (published with the ISO, after the operator's yes). **Remaining:** a redirect from `iso.felhom.eu/` to that page needs a Cloudflare rule the session has no credential for. | **WAITING-ON-OPERATOR — rank P3-LOW; owner: operator (Cloudflare rule)** **Re-ranked 2026-10-03: P3→P4: households are sent to the website's download page; the bare address is cosmetic.** | — | — | operator |
|
||||
| **R-1148** | Install & onboarding | P3 | **Installer 1.34.0 is published (operator yes, 2026-10-18) but its default path has not run on a fresh install.** Step 4b now vaults FIRST, then sets the password, then checks a PVE login (R-1145); the SC2066 loop became an `if`. Proven: the harness (52/0, the R1145 cases run the real step with fake commands), `test_hostinstall.py` 40/0, a real `--rotate-recovery-only` on demo-hp (vault time moved, new value 200, old 401). Not proven: a whole fresh install on a machine, where step 4b runs inside the full sequence and `https://127.0.0.1:8006` must answer during it. | OPEN — filed 2026-10-18 | — | One fresh appliance install with 1.34.0 on a scratch VM on demo-hp (the ISO proof-VM shape): step 4b's three lines, the vault time, a PVE login with the vaulted value; then discard the unclaimed appliance (the operator's press) | CC |
|
||||
| **R-1154** | Install & onboarding | P3 | **A reinstall while the customer's OLD host record still exists skips both clean-slate re-issues (the claim code and the off-site credential) in silence.** `handleHostEnroll` returns the existing host early (mint-once-reuse), so `reissueOnReenroll` is never reached; its own comment assumes „the clean-slate flow deletes the stale host” — nobody does on a plain reinstall. Proven live on the demo-hp reinstall 2026-08-21 (CC memory), never filed. Now recurring: the operator keeps Tester4 as the standing fresh-install customer (2026-10-18), and its old host `Tester4-31fa0e` stays after each teardown. Filed 2026-10-18. | OPEN — filed 2026-10-18 | — | Decide: the enroll re-issues when the old host is stale or off, or the teardown deletes the host first (operator, session only); a test on the enroll path | CC |
|
||||
|
||||
## Apps & catalog — 16 rows (P3 7, P4 9)
|
||||
## Apps & catalog — 15 rows (P3 6, P4 9)
|
||||
|
||||
| ID | Category | Sev | What | State | Blocked on | Next action | Owner |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| **R-1091** | Apps & catalog | P3 | **Catalog copy that is wrong or stale at the pinned tag, found writing the guides — each needs an operator word or a live check, so none was changed tonight.** From source at each app's tag (evidence per app in `audits/app-guides-2026-10-15/B/`): (1) and (2) **DONE 2026-10-16** (operator decision D12: reword): calcom's use case now names Apple Calendar or a CalDAV calendar instead of Google/Outlook, and bentopdf's tagline and use case say the browser does the work and the file stays on the device (catalog CHANGELOG 2026-10-16); (3) **sabnzbd**, **nzbget** use_cases say Radarr/Sonarr/Jellyseerr hand them jobs — the box wires only qBittorrent and Transmission (`mediawire.go`), and a hand link needs a path mapping; (4) **onlyoffice** first_steps 3–4 tell the household to wire Nextcloud by hand — the integration toggle does it (`internal/integrations/onlyoffice_nextcloud.go`); (5) **glance**, **homepage** first_steps send the household into config files in a Docker volume no household tool reaches; (6) **lubelogger** starts with only the „Dashboard" tab and in miles (shipped `appsettings.json`), so first step 3 cannot be done as written; (7) **home-assistant** first_steps promise auto-discovered devices — the app runs on a Docker bridge, where mDNS/SSDP were measured deaf (spike R-6); (8) **homebox** and **gitea** `add_people` name menus that moved or read differently at the tag; **bookstack** `default_creds` still says `admin@admin.com / password`, which after_install replaces; **dawarich** names the API key's page „Settings" (source suggests the account page); **code-server** says „install your tools" with no sudo; **komga** names the discontinued Tachiyomi; **seerr** `docs_url` points at Overseerr; **wanderer**'s comment still says it cannot close sign-up (R-714 is fixed in the same file); (9) **nzbget** and **grafana** deploy-field descriptions say the password can be changed later — nzbget re-applies its env password at every start and grafana ignores the field after first start (R-1086 is the qBittorrent/Transmission side). | OPEN — filed 2026-10-15 | — | Live-check (5)–(7) and (9) on 9202; fix (3), (4), (8) as copy with the freeze re-admitted; re-run the guide label check for any app whose copy moves | CC |
|
||||
| **R-1092** | Apps & catalog | P4 | **Three apps show no guide: plant-it and home-assistant carry one HIDDEN (`checked_version: ""`), emby has none.** plant-it: `lifecycle: abandoned`, and its pinned image does not exist upstream, so there is no screen to check labels against. home-assistant: its frontend follows the browser language and the Hungarian strings are fetched at build time, not kept in git, so the Hungarian labels cannot be checked against source (the draft is written; the shipped frontend in the image could check them). emby: its server's web UI is in no public source at 4.11.0.4. Evidence: `audits/app-guides-2026-10-15/B/`. The other 62 templates show a checked guide (controller ≥ 0.315.0). | OPEN — filed 2026-10-15 | — | home-assistant: check the Hungarian labels against the frontend files in the pinned image (read-only, on a box running it), then set checked_version; emby: label-check against the image's own web files the same way; plant-it: nothing while abandoned | CC |
|
||||
| **R-676** | Apps & catalog | P3 | **[P3-LOW] Watch: immich's first start restarted 12 times — decision 28's crash-loop stop (6 in 10 min) would stop it.** From the 2026-09-17 chaos night (DB connection dropped during the first-start geocoding import on a 6 GB guest; it did not recover that night). No healthy app in any drill evidence restarts on a first start (1831 samples, 40 live containers), so the threshold stands; this row exists so the first immich install under v0.269.x is watched. `audits/night-2026-09-24/A3/40-first-start-restarts.txt` **2026-09-25 night (read from source, v0.271.0): a DEPLOY's first start is NOT covered by decision 28's suppression** — `Deploying` clears when `compose up -d` returns (`deploy.go` "Clear deploying flag"), and `ObserveUnhealthy` then samples the app; an automatic update's step, verify and undo ARE covered (`Updating`, pinned by `TestD28_NoCrashLoopStopDuringAnAutomaticStep`). So a first start that restarts ≥ 6 times in 10 min is stopped — which R-676 already accepts for a broken first start; a healthy slow first start would be stopped too. **-- 2026-09-30: the first-start restarts are explained.** immich's first-start geodata import OOM-kills its database at 512M on a guest with no swap (R-732, measured: 61–104 kills); the 2026-09-17 chaos-night case (DB connection dropped during the import on a 6 GB guest) fits it. Fixed in the catalog (`56c4888`, 768M). The watch itself (decision 28 on a DEPLOY's first start) is unchanged. | **OPEN — P3; owner: CC (watch)** | — | — | CC |
|
||||
| **R-1018** | Apps & catalog | P3 | **The OnlyOffice links (FileBrowser, Nextcloud) have no recorded live proof since the move to Proxmox.** `controller/internal/integrations/` holds two handlers; no audit after the Proxmox move shows a document opened from FileBrowser or Nextcloud in OnlyOffice (searched 2026-10-14). The media-app links of `15` §6 will copy this code, including its revoke-on-stop rule (`lifecycle.go:10-13`). | **OPEN** — filed 2026-10-14 | — | One live walk on a Tier-0 box: install OnlyOffice + FileBrowser, open a document, save, read it back; then stop one app and record what the link does | CC |
|
||||
| **R-1028** | Apps & catalog | P3 | **Plex's claim code can run out during the install, and the box cannot recover it.** plex.tv's code lives 4 minutes from the moment the household copies it; the clock covers the wizard, the image pull (`plexinc/pms-docker` is 351 MB, measured) and the first start. Plex reads `PLEX_CLAIM` only on its first run, and the field is `locked_after_deploy` — so a late code leaves an unclaimed server and the only way back is remove + install again. Reported by the operator during the 2026-10-14 test; on Tester 1's line the pull+start took 8.5 s and the claim worked (`audits/media-all-drives-2026-10-14/``G-auth/G3-plex-claim.txt`), so it bites on a slow line or a slow hand. **K6, 2026-10-14 night (`audits/media-drive-events-2026-10-14/``K6-1-unclaimed-plex.txt`):** an unclaimed Plex starts with no code at all, and `POST /myplex/claim?token=` on the RUNNING server reaches plex.tv (a wrong code → 500 after plex.tv's 401). So the box can pull + start first and ask for the code after: no race. A real claim needs the household's code — the morning step on the throwaway `plex-k6` on Tester 1. Narrowed 2026-10-14 night. **PROVEN 2026-10-15 morning (`K6-2-claim-after-start.txt`):** the operator's real code, given to the RUNNING unclaimed `plex-k6` → 200, `claimed` 0→1, `PlexOnlineToken` written, and the open door closed (no token → 401). `ADVERTISE_IP` was unset and the claim did not need it; whether clients find the server without it was not measurable (the throwaway had no published port). **BUILT 2026-10-15 (v0.313.0):** `after_start` field — not asked at install; the Plex page asks for the code once Plex runs and claims the running server; the gate opens only when Plex says claimed. Proven up to the code on 9201 (G5). | **NARROWED** — the dashboard claim with a real code is the last step | a household Plex code | Claim through the Plex page with a real code (G5c); then close | CC |
|
||||
| **R-1029** | Apps & catalog | P3 | **Plex's folder picker shows the drives as identical „media” folders.** With one bind per drive at the same path (`/mnt/felhom-drives/<drive>/userdata/media`), the leaf of every library folder is `media`; the drive's name sits two levels up, so in Plex's picker the household sees several folders all called „media” (reported by the operator during the 2026-10-14 test on Tester 1). It conflicts with `15` §2 reason 2 (the same path everywhere). | **READY** — filed 2026-10-14 | the layout decision (STATUS D9) | Decide in `15` §2: the players (`:ro`, never linked) may get a readable path per drive (e.g. `/media/<drive label>`) without breaking the hardlink rule, which binds only the download/library apps | CC |
|
||||
| **R-1071** | Apps & catalog | P3 | **Jellyfin cannot use films that sit on a NAS share (the pilot's OMV), and the pilot's wish is exactly that.** MEASURED 2026-10-15: an OMV SMB share added as „Hálózati tárhely” mounts and verifies (`/mnt/felhom-drives/omv-filmek`) — but only with a user name AND a WRITABLE share (read-only → refused „az alkalmazások nem tudnak rá írni”) — and the household's „Újraindítás most” binds only local drives into Jellyfin (`per-drive binds 1 [/mnt/felhom-drives/filmlemez/userdata/media]`). By design: R-108 (a NAS is browse-only, never an app's namespace); `media_library.go` skips `Kind == "network"`. A design decision, not a defect (R-370). Evidence: `audits/omv-vm-2026-10-15/B6-omv-share.txt`. | WAITING-ON-OPERATOR | Operator: should a media app read a NAS share read-only (a new bind kind), or is the answer „put the films on a drive the box owns”? | Operator rules; if yes, a design note in `15-media-library.md` first. | Operator |
|
||||
@@ -142,9 +141,9 @@ stopping line that lies.
|
||||
| **R-995** | Apps & catalog | P4 | **Removing an app through the product can leave unnamed Docker volumes and an empty import folder behind.** MEASURED 2026-10-13 on demo-felhom after 61 removals (`documentation/audits/sizing-2026-10-13/METHOD.md` „Teardown”, `evidence/leftover-volumes-after-bench.txt`): 8 unnamed volumes (4–72 KB; immich 1, romm 2, privatebin 3, onlyoffice 2 — volumes the images declare themselves) and `userdata/import/{paperless,calibre,grimmory}` (empty, created by the deploy). The removal answered 200 and its own leftover check was clean, because it looks only at labelled volumes. No data risk; disk and tidiness only. Removed by name on the bench. | **READY** — filed 2026-10-13 | — | Make the removal account for the containers' unnamed volumes (and an empty import folder it created), with a test that deploys an image that declares a VOLUME | CC |
|
||||
| **R-1067** | Apps & catalog | P4 | **The catalog still carries `pi_compatible:` on every template, and nothing reads it any more.** controller v0.315.0 removed the „Pi kompatibilis" badge and the „csak x86" tag from every page (Raspberry Pi is no longer supported, so one was noise and the other was a warning on almost every app). The `ResourceHints.PiCompatible` struct field and the catalog key were deliberately LEFT: a box still on 0.314.0 reads the key, and removing it would make that box show „csak x86" on every app. **Gated on the floor:** once the fleet floor is at or above 0.315.0, drop `pi_compatible:` from the templates and the field from `stacks.ResourceHints` in the same pair of commits. Harmless until then — an unread key costs nothing. | **BLOCKED** — filed 2026-10-15. Checked 2026-10-16 (apps burndown): the floor is 0.321.0 and demo-hp, Tester 1 and the N100 run 0.321.0, but **Tester-2 still reports 0.292.0** (its newest report 2026-10-04; hub DB with WAL) — so nothing removed. | every box on ≥ 0.315.0 — today Tester-2 | Drop the catalog key and the struct field; grep the hub and the website for Pi wording at the same time (none found 2026-10-15) | CC |
|
||||
| **R-1043** | Apps & catalog | P4 | **Sonarr, Radarr and Prowlarr build their login redirect as an absolute `http://` URL.** They trust forwarded headers only from their „Trusted networks” (the household set `192.168.0.0/24`; traefik is `172.18.x`), so they ignore `X-Forwarded-Proto: https` and answer `Location: http://<name>/login` to an https visitor. Since controller v0.312.0 that hop is sent back to https by traefik (a GET, no password), so it is cosmetic; a template could seed traefik's network as trusted. (`audits/https-only-2026-10-14/``1A-3-4.txt`) | **OPEN** — filed 2026-10-14 | — | Seed the docker network as a trusted proxy in the *arr templates (with the R-1030 seed), measure the redirect | CC |
|
||||
| **R-1100** | Apps & catalog | P3 | **One sign-in for the dashboard and FileBrowser (proxy auth) — spike first.** Since controller 0.320.0 every app with a folder has „Fájlok ↗", and the box can show FileBrowser's password — but FileBrowser still asks for its OWN login the first time, so the household signs in twice. FileBrowser Quantum has a proxy-auth method (`auth.methods.proxyAuth`, header-trusted) that could take the dashboard session through traefik; nobody has measured it. Design: `architecture/02-controller-module-map.md` „Logins and folders". | OPEN — filed 2026-10-16 (task logins-folders). | — | Spike on demo-hp: proxy auth at the pinned tag (source + measured), a forwardAuth in traefik that answers only for a dashboard session, what a stranger reaching `files.<domain>` with a forged header gets, and what happens to FileBrowser's shares. | CC |
|
||||
| **R-1100** | Apps & catalog | P3 | **One sign-in for the dashboard and FileBrowser (proxy auth) — spike first.** Since controller 0.320.0 every app with a folder has „Fájlok ↗", and the box can show FileBrowser's password — but FileBrowser still asks for its OWN login the first time, so the household signs in twice. FileBrowser Quantum has a proxy-auth method (`auth.methods.proxyAuth`, header-trusted) that could take the dashboard session through traefik; nobody has measured it. Design: `architecture/02-controller-module-map.md` „Logins and folders". | **READY** — spike done 2026-10-18 (`audits/proxyauth-spike-2026-10-18/DESIGN-NOTE.md`): Traefik forwardAuth with `authResponseHeaders` strips a client's header (any case, twice) and its own value wins — measured; the config key is `auth.methods.proxy` (not `proxyAuth`); **but an app container reaching `filebrowser:80` directly logs in with a forged header (measured 200)**, and the dashboard cookie is host-only. Build = a files-only ticket cookie + a per-box secret header name. Filed 2026-10-16. | — | Build per the design note (controller only, ~one release), or decide not to | CC |
|
||||
| **R-1099** | Apps & catalog | P4 | **The N100 may hold RomM games in the old upload place, now shown under a fake „roms” platform.** Until 2026-10-16 RomM's own upload wrote to the drive's `roms/roms/<platform>/`; after the mapping fix (catalog `fb18471`, R-1089) those files are at RomM's `library/roms/roms/…`. Measured on 9201: such a game showed as missing in its platform and again under a platform „roms” (`audits/romm-ghost-2026-10-16/B/b-after.txt`). Files put in with the file manager are not affected — they are now found. The N100 runs RomM (hub `/apps/romm`); CC does not log in to the lived-in box. | OPEN — filed 2026-10-16 | Operator: look at `roms/roms/` on the N100's drive in the file manager | If it holds platform folders: move them up one level into `roms/`, then run „Scan” in RomM; if it is empty, close | Operator |
|
||||
| **R-1117** | Apps & catalog | P4 | **Immich's and Nextcloud's Redis keep a volume they do not need** (upstream runs both without one; queues and caches only). Deferred from R-1096 on purpose: removing a volume from the compose leaves the old volume on installed boxes, and the nightly dump / an old backup's restore for a volume the compose no longer declares is unchecked. The RDB switch (catalog `5d9450b`) already removed the full-disk danger. | DEFERRED — filed 2026-10-17 | — | Measure the dump and an old-backup restore without the volume on 9202; then the template change + an orphan-volume note | CC |
|
||||
| **R-1153** | Apps & catalog | P4 | **FileBrowser's preview of a .docx answered 500 while the OnlyOffice link was on** (`/api/resources/preview`, scratch 9202, controller 0.330.0, 2026-10-18). FileBrowser builds an office preview through OnlyOffice's converter from `server.internalUrl` when it is set (`backend/http/preview.go:243` at 1.5.6-stable) and from the request's host when not — so 0.331.0's `internalUrl` (R-1018) may already fix it. Not re-checked: OnlyOffice was removed after the walk. | OPEN — filed 2026-10-18 | — | One preview request on a box with 0.331.0 and the link on | CC |
|
||||
|
||||
|
||||
## App updates — 4 rows (P3 4)
|
||||
@@ -156,12 +155,12 @@ stopping line that lies.
|
||||
| **R-683** | App updates | P3 | **[P3-LOW] Watch: after a power cut during an update's health check, the hold named an HOUR-OLD second-drive copy, not the one the update's own backup should have just made.** 2026-09-24 chaos round 3 (nextcloud, `backup_max_age: 1m`): no `backing-up` phase was seen and the hold named Tier 2 at 13:04 for an update pressed at 14:04; the pre-cut controller log was lost with the container (the runner now saves it at arm time — R-320). Round 11, the same action without a power cut, named a fresh 14:34 copy and logged the Tier-2 copy. The sentence was TRUE (it named the copy it offered); the question is why the update did not back up first. Not reproduced; watch the next power-cut drill. `audits/night-2026-09-24/E/round-03*.json`, `E/round-11-controller-pre.log` | **OPEN — P3; owner: CC (watch)** | — | — | CC |
|
||||
| **R-785** | App updates | P3 | **[P3-LOW] SparkyFitness is pinned 11 releases and a major behind upstream (v0.17.3; upstream v1.7.3, v1.6.0 dated 2026-07-24).** READ 2026-10-01 (`audits/visitors-2026-10-01/C/bench/C1-previous-tag.txt`). **Needs:** an update walk 0.17 → 1.x through the ladder (bench + box), after R-784 is decided. | **OPEN — rank P3-LOW; owner: CC (after R-784)** | — | — | CC |
|
||||
|
||||
## Backup & restore — 28 rows (P2 3, P3 14, P4 11)
|
||||
## Backup & restore — 30 rows (P2 3, P3 16, P4 11)
|
||||
|
||||
| ID | Category | Sev | What | State | Blocked on | Next action | Owner |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| **R-978** | Backup & restore | P3 | **Starting the escrow ceremony again can make the customer's written-down recovery code useless: the new package replaces the old one before the new code is confirmed.** It can happen to an ordinary customer who starts and walks away. Fix: keep the previous package until the new code is confirmed. Security review 2026-10-12 (`audits/security-review-2026-10-12/`); detail held off-git for the operator (`/mnt/5_hdd/felhom.eu/evidence/security-review-2026-10-12-PRIVATE/`). | OPEN | — | Fix in the agent with a test | CC |
|
||||
| **R-973** | Backup & restore | P3 | **A compromised controller can make the whole-guest backup run again and again, and each run prunes older restore points: the local tier directly, and the off-site tier through the store's own nightly keep-last-2.** So `07` §8 row 10 (ransomware) is weaker than it reads. The second reader confirmed the local half; the off-site half is inferred. Fix: a cadence or daily cap on the local-API backup route, and off-site snapshots that a nightly prune cannot remove. Security review 2026-10-12 (`audits/security-review-2026-10-12/`); detail held off-git for the operator (`/mnt/5_hdd/felhom.eu/evidence/security-review-2026-10-12-PRIVATE/`). | OPEN | — | Add the cap with a test; decide the off-site retention shape | CC |
|
||||
| **R-978** | Backup & restore | P3 | **Starting the escrow ceremony again can make the customer's written-down recovery code useless: the new package replaces the old one before the new code is confirmed.** It can happen to an ordinary customer who starts and walks away. Fix: keep the previous package until the new code is confirmed. Security review 2026-10-12 (`audits/security-review-2026-10-12/`); detail held off-git for the operator (`/mnt/5_hdd/felhom.eu/evidence/security-review-2026-10-12-PRIVATE/`). | **OPEN — the fix is in agent `main`, unreleased** (`8561dc8`, 2026-10-18): the new package reaches the hub only when its code is claimed; an abandoned second ceremony leaves the first code opening the hub's package (tests + four red-proofs). Residual R-1151. Rides the next vouched agent release. | — | Release with the operator's yes; then close | CC |
|
||||
| **R-973** | Backup & restore | P3 | **A compromised controller can make the whole-guest backup run again and again, and each run prunes older restore points: the local tier directly, and the off-site tier through the store's own nightly keep-last-2.** So `07` §8 row 10 (ransomware) is weaker than it reads. The second reader confirmed the local half; the off-site half is inferred. Fix: a cadence or daily cap on the local-API backup route, and off-site snapshots that a nightly prune cannot remove. Security review 2026-10-12 (`audits/security-review-2026-10-12/`); detail held off-git for the operator (`/mnt/5_hdd/felhom.eu/evidence/security-review-2026-10-12-PRIVATE/`). | **OPEN — the local-half fix is in agent `main`, unreleased** (`f17e197`, 2026-10-18): a cadence cap on `POST /backup` — daily tier ≤ 3 successful runs per rolling 24 h, the weekly off-site tier one per 6 days; a refusal is 429, logged, persisted across a restart, and a CRITICAL degraded status for 24 h (operator-only `agent_capability_degraded`, wording R-1150). *Decided by CC unattended — operator may reverse:* the numbers. **Off-site half, design line (not built):** ep0 prunes the namespace keep-last 2 — a count, so each extra run pushes out the oldest; with the cap a pre-compromise copy survives ≥ ~6 days and each extra attempt alarms; to make it unfloodable, ep0's prune moves to time buckets (e.g. keep-weekly 2) — an ep0/hub prune-job change, the operator's; check that DooPlex's pull copy of ep0 is keep-weekly, not keep-last. Local keep-last 3: R-1149. | — | Release with the operator's yes; decide the off-site prune shape | CC |
|
||||
| **R-232** | Backup & restore | P2 | **DooPlex's backup makes every copy inside the same box — and nothing tells anyone when it fails.** Surveyed read-only 2026-08-06 (`audits/RECON-dooplex-backup-2026-08-06.md`). **What works:** five sets, 14/14 successful runs in 14 days; a file was restored from the `data` repo and matched the live original **byte for byte**; every set except two is cross-disk; k3s is integrity-checked on every run. **What the matrix exposes, ranked:** (a) **`notify_failure` is a no-op** — `NOTIFY_ON_FAILURE=true` but `NOTIFY_WEBHOOK_URL` is commented out, so a failed backup notifies **nobody**; the project already has a working Resend path that CI uses. Cheapest item, and it makes every other failure visible. (b) **Nothing leaves the box** — no rclone, no remote repo, no off-site target anywhere; Longhorn's target is `nfs://192.168.0.180:` pointing at DooPlex itself, and the only outbound-looking cron pulls *inbound* from Hetzner for a different project. The machine that runs the hub managing the customers' off-site chain has no off-site copy of its own. (c) **The backup tree is a single writable path** and the restic repos are not append-only — one bad script or ransomware destroys every copy at once. (d) **Two same-disk sets**: `.claude-memory` and the PostgreSQL dumps, whose source directory sits *inside* the backup tree. (e) **Longhorn `retain=1`** — one generation per volume, so a corruption noticed a day late has no earlier copy. (f) **`/opt/backup/docs/BACKUP-RESTORE.md` does not exist** though the systemd unit advertises it. (g) **`secrets/restic-repo` has never held a snapshot** — `backup-secrets.sh` contains no `restic` call; the secrets are GPG files on `sda1` only. (h) **No restore has ever been run** beyond today's single-file probe — the matrix's "ever demonstrated?" column is otherwise entirely empty. **Not a finding:** the restic passphrase. The on-box copy is on `sdb1`, a different disk from the backups, and the **operator holds an offline copy out of band** — so a disk loss is recoverable. The narrow residual is that it is operator-held rather than system-held, unlike the customer case's hub-vaulted escrow, so it should be confirmed current and findable by someone else. **Nothing was changed by the recon.** | **NARROWED 2026-10-08 — owner Viktor.** (a) DONE with the operator's yes in chat: `notify_failure` now mails admin@felhom.eu through Resend; proven by one test mail that reached the inbox (`audits/day-2026-10-08/r232/`; no backup was started). (b) partly: the hub database leaves DooPlex nightly to ep0 (R-173); everything else stays on the box. (c)–(h) unchanged. **2026-10-09 (operator yes in chat): (b) and (h) DONE for Gitea and the secrets.** Nightly 00:20 `felhom-dooplex-offsite` pushes Gitea (614 MB of repositories, the `gitea` database dump taken first, `app.ini`) and the nightly GPG secrets export, encrypted with a NEW key, to ep0 `operator` (`host/dooplex-gitea`) on the hub-DB write-only token — no change on ep0; first push 57 s; restore test weekly (manifest, `git fsck` every repo, `pg_restore --list`), run once: 27 805 files, 10 repos OK; `DooplexGiteaOffsiteStale`/`DooplexGiteaRestoreTestStale` in force (`absent()` included, promtool + 2 red-proofs); every backup unit now mails on failure (`OnFailure=`, dry run reached the inbox). **First real Gitea restore:** into a throwaway on the bench, 10/10 repos, product `main` = live, a file byte for byte, a login (`runbooks/gitea-restore.md`). The hub came back too (R-173, closed). **The registry is left out on purpose:** its images rebuild from the code. **Correction to the recon §7:** the API mirror holds ONE repo (`homelab-manifests`), not all — Gitea had a single copy, on DooPlex. `audits/dooplex-survival-2026-10-09/` **LEFT:** (c) append-only for the on-box tree, (d)–(g) as written; the paper copy of the new key (operator). **READY** for the rest | — | — | operator |
|
||||
| **R-893** | Backup & restore | P3 | **After a failed OFF-SITE replay, the rollback pours the NEWER pre-restore copy over the OLDER volume just put back.** Read in source 2026-10-06 (R-638 option A, not measured): `internal/backup/offbox_reconstitute.go` writes the undo copy from the live (newer) database, replaces the volumes with the snapshot's older tars, then — when the replay fails — `rollbackSafetyDump` loads that newer dump over the older database volume. The loader only drops what the dump knows, so tables the newer migration removed stay; and when the snapshot's older definition was written, the rollback branch does not put the newer definition back, so the older app starts on rolled-back data; non-database volumes stay at the snapshot's state. An order change cannot fix it (the only undo is a logical dump, and its volume was replaced). Known limit in `07` §6.3. **-- 2026-10-08 design:** `audits/day-2026-10-08/design-R-893.md` — re-verified; and the screen's „your data is back as it was" (`err.backup.db_restore_failed_rolled_back`) is false for files, other volumes and the app version. Question D8 on STATUS's decision sheet. **-- 2026-10-08 14:16 operator ruling D8 (`09` §3 decision 192):** yes, both, in that order — first the app stays stopped for support (option C), next „put back exactly as it was" (option A). | **NARROWED — 2026-10-09: the first half DELIVERED in controller 0.304.0. Slice 0 (the live measurement) NOT run: 9202 has no off-site, and a dump cannot be cut by hand inside an encrypted off-site copy on Tester 1; it needs a scratch off-site repo — a plan, not a finding.** **NARROWED 2026-10-08 — the first half (D8 option C, the app held stopped) is built on controller main and ships tomorrow; what stays open is the second half, „put back exactly as it was” (option A: a pre-restore copy of the app's volumes and placed files, a fit check, disk for one copy; the slice-0 measurement on 9202 first).** **OPEN — filed 2026-10-06** **2026-10-06 night: verified in source, no code** — `offbox_reconstitute.go:758` (undo dump from the live DB), `:778-860` (files and volumes from the snapshot), `:804` (the snapshot's definition is written when its version differs), `:898` (the rollback loads the newer dump over the older volume; nothing writes the live definition back). Not a reorder fix: it needs R-638 option B (a rebuilding loader) or a pre-restore volume copy (disk cost; R-685's class). Which state a household gets after a failed off-site replay is the operator's call. Next: the 9202 measurement, then the design. | a design: R-638 option B (a loader that rebuilds instead of overlays) or a pre-restore volume copy | Measure it once on 9202 (a forced replay failure after an off-site restore over a migrated app); then a design for the operator | CC |
|
||||
| **R-895** | Backup & restore | P2 | **The hub's clean-up-window check trusts the snapshot counts the box sends, so a broken-into box (or past-dated fakes added through the add-only key) can shrink the real off-site history without an alarm.** READ 2026-10-06 night in source (R-822's design): the before/after comparison uses counts the box itself reports (`hub/internal/offsitekeys/service.go:284`, `:343`); new fakes keep the count level. Decision 68 already accepts a box-trusted count. | **NARROWED 2026-10-18 — slice 2 built, not deployed** (hub v0.158.0): the 07:10 audit compares the `snapshots/` listing with yesterday's by name — files gone that no clean-up window explains → `offsite_snapshots_vanished` (error), files no reported box run explains → `offsite_snapshots_unexplained` (warning); operator-only, hub-origin (not in `allowedEventTypes`). First listing = baseline; a move-aside or RESET drops it; vanishing judged on a pinned tier only; files younger than 1 h judged the next day. Red-proofed twice (`snapshot_watch_test.go`). **Not built:** the window's before/after on the hub's own count (design §3.1) and refusing a window while unexplained files exist (§3.2). Unmeasured: the provider shell's `ls -l` time zone (± 3 h slack until then). **Deployed 2026-10-18 12:23 (hub v0.158.0).** The first listing (tomorrow 07:10) is a baseline; the first verdict is 2026-10-20 07:10. **Known limit (independent review 2026-10-18, F2):** `_unexplained` is explained by the box's OWN reported `offsite.last_run`, so whoever holds a box's key can silence it; `_vanished` is judged on the hub's own listing and is not affected. | the operator's yes to deploy hub v0.158.0 | Deploy; read the first 07:10 lines (newest upload vs the box's last run = the time-zone calibration), shrink the slack; then decide §3.1/§3.2 | CC |
|
||||
@@ -188,13 +187,15 @@ stopping line that lies.
|
||||
| **R-878** | Backup & restore | P4 | **A catch-up (R-871) runs the database-dump leg in the DAY, and that leg stops an app with a volume for its copy — the household may notice the stop, and a large volume makes it longer.** MEASURED 2026-10-05 on demo-felhom: the catch-up at 08:25:02 stopped opengist, copied 182.5 KB, started it again — about 1 s, then a few seconds of `health: starting`; the night does exactly the same, unseen. Nothing measured for a large volume. Fix direction (if it matters): skip the volume copy of a running app in a DAYTIME catch-up and leave it to the next night, or warn. `audits/catchup-2026-10-05/partA/live-demo-felhom.txt` | **READY — owner: CC** **2026-10-05 (burn-down night): NEEDS A LIVE MEASUREMENT** — a timed catch-up on 9202 for an app with a multi-GB volume, then the skip-or-warn choice. | — | measure a large volume first | CC |
|
||||
| **R-32** | Backup & restore | P4 | **[P2-HIGH] RESET must purge the customer base dir; the orphan card must stay honest; unattributed bytes must be visible.** The rehearsal's S7 said in advance that an orphan card would BE a finding — and one appeared (16:58:14). Cause: RESET's `"hetzner":"ok"` leg destroys the sub-account, but **a Hetzner sub-account is an access-control object, not a data object** — its directory survives, so re-enabling offsite recreated an account over the previous lifecycle's ciphertext, encrypted under a key that same RESET had destroyed. **MIGRATED FROM `ROADMAP.md` 2026-08-22 (R-369) — originally filed 2026-07-21, size M, roadmap state `idea`.** Moved verbatim; nothing added or reinterpreted. The roadmap keeps its copy as history, marked moved. | **OPEN — migrated from ROADMAP 2026-08-22, rank unchanged** **2026-10-06 night: design written** (`audits/night-burndown-2026-10-06/design-R-32.md`) — pick A: purge the repo + set-aside copies through the sub-account's own password login BEFORE deleting it (no main-account credential); the „repo data destroyed" comment at `hub/internal/offsite/offsite.go:276-280` is false for the shared tier; part (3) waits on a read-only measurement (`du` on port 23). Operator: agree to route A. **2026-10-07 07:58: `09` §3 decision 167 — option A yes, and delete the old RESET leftovers once.** **2026-10-07 (Part D step 2, read only):** the hub shows bx11 holding 3.3 GB of data while the four live customers account for ~0.6 GB → **~2.7 GB belongs to no live customer**; no live customer shows a set-aside folder, so the leftovers are in homes whose sub-account is gone, reachable only by the pool box's MAIN account. **The one-time clean-up is STOPPED:** no main-account login for bx11 is held on DooPlex (the API token in the credentials file sees no box). Needs: the main-account login, or the operator lists and deletes in the Hetzner console (`audits/day-2026-10-07/D/D2-listing-readonly.txt`). **2026-10-07: option A DELIVERED (hub 0.142.0):** RESET purges the repo and every `.orphaned-*` through the sub-account's own login before deleting it, and keeps the sub-account if the purge fails (red-proved, `audits/day-2026-10-07/D/`). Not yet seen live (no scratch RESET today). **The one-time clean-up of the ~2.7 GB of leftovers is STOPPED:** it needs the pool box's main-account login (see above). **2026-10-07 14:43 — `09` §3 decision 173: leave the ~2.7 GB of old leftovers for now; re-ranked P2 → P4.** Option A itself is delivered (hub 0.142.0). What stays open is ONLY the leftovers: they were computed (box total minus live customers), never listed, and deleting them needs the pool box's MAIN login, which reaches every customer's data. **Blocker: a safe main-account path.** | — | **Ruling from the run (three parts, deliberately separate):** (1) because RESET destroys custody, the ciphertext it leaves behind is unrecoverable **BY DESIGN** → RESET gains a **main-account purge of the customer base dir** (the existing operator ack already covers it); (2) the **move-aside guard STAYS** for reinstall-*without*-RESET — there custody survives and the card's "history recoverable" promise is true (R-26 depends on exactly that); (3) the operator **Restic tab shows per-customer directory bytes vs attributed snapshot bytes**, so dead data cannot hide. Measured on the pool box that night: **49 M attributed** (2 snapshots, 48.717 MiB) against **1.4 G + 3.0 M unattributed** across TWO `.orphaned-*` dirs. Evidence `restic-and-pool.txt` | CC |
|
||||
| **R-1087** | Backup & restore | P4 | **Whole-box copies left on the OLD local target are never pruned after the target moves.** MEASURED 2026-10-15 (storage-identity build B space check): the N100's host root (`local`) holds three copies from 2026-07-28 (5.5–5.7 GB each, ≈ 16.8 GB); its agent's `local_backup_target` is now `felhom-backup` (the USB disk, retention 3), and retention prunes only the current target. On a small-root box such leftovers would starve the local copies (R-548). Evidence: `audits/storage-identity-buildB-2026-10-15/space/felhom-pve-9201.txt`. | OPEN | — | Decide whether the agent prunes its own `vzdump-lxc-<vmid>-*` files on a former target (after a newer copy exists on the new one), or the target move asks; until then, the N100's three July files can be removed by hand with the operator's word. | CC |
|
||||
| **R-1149** | Backup & restore | P3 | **The local whole-guest tier prunes keep-last 3, so even with the R-973 cadence cap a compromised controller can replace all three restore points within one day.** A time-bucketed prune (`--prune-backups keep-daily=3`) has the same disk cost and cannot be flooded within a day. It changes `07` §6's retention, so it is the operator's call. Found by the R-973 helper 2026-10-18. | OPEN — filed 2026-10-18 | — | Operator decides keep-last vs keep-daily; then the agent's vzdump argument | operator + CC |
|
||||
| **R-1151** | Backup & restore | P3 | **The new escrow package now uploads when its code is claimed (R-978), but the wizard's typed-back check is the browser's alone:** a household that opens the new code and closes the page without writing it down has still replaced the hub's package, and the first code no longer opens it. Gate the upload on the typed-back confirmation with a new controller → agent call. Found 2026-10-18. | OPEN — filed 2026-10-18 | — | Controller + agent: the confirm call; red-proof the abandoned-after-claim case | CC |
|
||||
|
||||
## Storage & devices — 10 rows (P3 4, P4 6)
|
||||
|
||||
| ID | Category | Sev | What | State | Blocked on | Next action | Owner |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| **R-1073** | Storage & devices | P3 | **Felhom in a virtual machine sees no disk health: SMART is UNKNOWN for its own system disk and any passed disk.** MEASURED 2026-10-15 on the OMV-VM copy: every storage target reported `smart.health = UNKNOWN`, all counters null (the guest sees QEMU disks). On the pilot's box the NVMe and the „old, not stable” Toshiba would be watched by OMV only, and Felhom's drive-health alarms (R-330/R-332's whole point) cannot fire. Evidence: the host report in `audits/omv-vm-2026-10-15/README.md` (memory/disk section) and `C-hub-events-Tester4.txt`. | WAITING-ON-OPERATOR | Operator: is a VM install a supported shape, and if so, is „OMV watches SMART” acceptable or must Felhom say on the dashboard that it cannot watch the disks? | Operator rules; then CC adds a household line on the Meghajtók page when SMART is unavailable. | Operator |
|
||||
| **R-1093** | Storage & devices | P3 | **An EMPTY partition asks for the typed „data will be erased" confirmation.** MEASURED 2026-10-16 (build B, scratch VM, agent 0.158.1): `POST /api/storage/init` for a blank part → `needs_confirmation`, reason „device has a partition table" — `lsblk` reports the PARENT disk's table (PTTYPE) for a partition, so the agent's probe reads an empty part as data-bearing. Safe direction (more friction, never less), but `16` §6 says an empty part is self-serve and the wizard shows „adatot tartalmaz". Evidence: `audits/storage-identity-buildB-2026-10-15/live/12-partitions-format.txt`. | OPEN | — | In the agent's device probe, ignore a PTTYPE that a partition inherits from its parent (judge only the part's own signature and children); a red-proved test with a blank part fixture. Next agent release. | CC |
|
||||
| **R-1093** | Storage & devices | P3 | **An EMPTY partition asks for the typed „data will be erased" confirmation.** MEASURED 2026-10-16 (build B, scratch VM, agent 0.158.1): `POST /api/storage/init` for a blank part → `needs_confirmation`, reason „device has a partition table" — `lsblk` reports the PARENT disk's table (PTTYPE) for a partition, so the agent's probe reads an empty part as data-bearing. Safe direction (more friction, never less), but `16` §6 says an empty part is self-serve and the wizard shows „adatot tartalmaz". Evidence: `audits/storage-identity-buildB-2026-10-15/live/12-partitions-format.txt`. | **OPEN — the fix is in agent `main`, unreleased** (`f7b7308` + `0963bae`, 2026-10-18): lsblk's inherited PTTYPE is ignored only when `blkid -p` on the partition exits 0 with `PART_ENTRY_NUMBER` and the same scheme; every error path stays data-bearing (14-row test, three red-proofs; no sudoers change). Live on demo-hp (a loop file in `/root`, removed): the empty GPT part → blkid rc 0 with only `PART_ENTRY_*`; ext4 → `TYPE`; a table inside → `PTTYPE=dos` (`audits/agent-fixes-2026-10-18/r1093-loop-check.txt`). Not walked: `/api/storage/init` on a VM disk. | — | Release with the operator's yes; one init call on a VM with a spare disk | CC |
|
||||
| **R-1062** | Storage & devices | P4 | **The N100's USB drive cannot be split in two without losing the box's whole-guest backup target.** The operator asked (2026-10-15 16:31) for two halves for multi-drive tests. Read first (`audits/n100-lived-in-2026-10-15/B/`): no family files — 2.7 GB nightly vzdumps of guest 9201 (`dump/`, PVE storage `felhom-backup` = the whole-guest backup target), 3.1 GB app backups + an off-site restore copy, 422 MB leftover app data of removed apps, empty media folders. The agent refuses a decommission of a backup target (`refuseIfBackupTarget`) and the N100 has no other drive to move that role to. Nothing changed. | **WAITING-ON-OPERATOR** — filed 2026-10-15 | operator decision | Keep one drive (CC's pick: the lived-in box keeps its local whole-guest backup), or give a word to drop that backup leg / add a second drive first | operator |
|
||||
| **R-1064** | Storage & devices | P4 | **Removing a storage path does not mention the cross-drive backup copies it holds.** demo-hp 2026-10-15: `/mnt/sizing-bench` removed through `/settings/storage/remove` (302, „Adattároló eltávolítva") while calibre-web's and paperless-ngx's `cross_drive.destination_path` pointed at it; the copies stay on disk, the records still name the removed path. The in-use check counts HDD_PATH only (`appsUsingPath`). | **OPEN** — filed 2026-10-15 | — | Read what tonight's tier-2 run (01:30) does with the two records; then: name the copies in the removal's confirmation (or count them as in use) | CC |
|
||||
| **R-330** | Storage & devices | P3 | **Disk health Phase 2 — the three SMART attributes the wire does not carry.** The failing drive's most telling counter was **187 `Reported_Uncorrect`**, sitting at normalized **1** against threshold **0** with a raw count of **1001** — one point from failing and structurally unable to get there. Also wanted: **199 `UDMA_CRC_Error_Count`** (cabling) and **188 `Command_Timeout`**. None are on the agent→controller wire today, so v0.215.0's ladder could not use them. Phase 2 also persists periodic SMART **samples** (the right home is `metrics.MetricsStore`, NOT the Phase-1 state file, which is one record per disk and must stay that way). **This is a declared WIRE change, so under the G-1 gate the hub must model the new fields in the SAME session** — that is precisely why it was kept out of Phase 1, where it would have turned a one-word severity fix into a three-repo change | **READY** — **2026-10-12: the wire half is CLOSED in this row** — released in agent v0.150.0 (boxes run 0.154.0), controller and hub decode 187/188/199. **Only open work:** the verdict slice (use the three counters in the disk verdict and the household's words) and persisting samples. **READY (M) — NEW 2026-08-14** **2026-10-06 night: the wire half BUILT on main (unreleased):** the agent sends SMART 187 `reported_uncorrect`, 188 `command_timeout` (raw, vendor-packed on some drives — carried as reported), 199 `udma_crc_errors` (pointer + omitempty: unknown is absent, never 0); the controller decodes them and the hub's host-report mirror models them (G-1 wire gate green; it convicted the agent half alone). **Carried only** — no verdict, banner, mail or alarm reads them (`TestR330_CountersChangeNoVerdictYet`); using them changes what a household is told and is the next slice, with persisting samples. Red-proofs `audits/night-burndown-2026-10-06/r330/`. Ships with agent v0.150.0 and the next controller and hub releases. | R-328 (closed) | The verdict slice: 187/188/199 into the disk verdict with tests; then persist samples | CC |
|
||||
@@ -247,7 +248,7 @@ stopping line that lies.
|
||||
| **R-1061** | Box system & updates | P4 | **A per-customer floor reached Tester 1 only at its next 15-minute report, not in seconds.** Set 2026-10-15 12:55:51 UTC for tester-1, demo-hp and demo-felhom together (hub 0.153.0, CC's token): demo-hp and demo-felhom ran 0.314.0 within 13 s (the hub's wake); Tester 1 moved at 13:09:35, its regular report (`audits/gate-restore-2026-10-15/release/03-boxes-move.txt`). It had restarted at 12:54:29 onto rc.1 and logged „hub wait channel active". Why the wake did not reach it is not read (the wake line is DEBUG). No harm: the floor arrived. | **OPEN** — filed 2026-10-15 | — | Read Tester 1's DEBUG ring at the next floor change; compare with the hub's bump log | CC |
|
||||
|
||||
|
||||
## Monitoring & notifications — 11 rows (P2 1, P3 7, P4 3)
|
||||
## Monitoring & notifications — 12 rows (P2 1, P3 7, P4 4)
|
||||
|
||||
| ID | Category | Sev | What | State | Blocked on | Next action | Owner |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
@@ -262,6 +263,7 @@ stopping line that lies.
|
||||
| **R-285** | Monitoring & notifications | P4 | **A planned, supervised reinstall pages the operator as if the machine had died — there is no notion of expected downtime anywhere.** During the 2026-08-09 rehearsal the hub sent, all `status: sent` to the operator channel: `host_stale` 08:58 UTC, `node_stale` 09:00, **`host_down` 09:28 (error)**, **`node_down` 09:30 (error)**, `host_leaf_changed` 09:31, `host_recovered` 09:31, `node_recovered` 09:34, `offsite_delivery_stuck` 09:34 — eight operator mails for work that was deliberate, attended and announced. **This is the OPPOSITE gap from the one R-281 filed:** the alarms are not missing, they are indiscriminate. `host_stale` at 30 min and `host_down` at 60 min (`monitor/host_staleness.go:22-23`, `downAfter = 2 * threshold`) cannot distinguish a wiped-on-purpose box from a dead one, and `host_leaf_changed` firing on a reinstall is correct-but-expected. **Note the interaction with the mute used on 2026-08-09 evening:** blocking a customer silences everything, so today the only two settings are *page me for planned work* and *tell me nothing at all*. **What is owed is a middle:** a maintenance window, or an operator-set expected-downtime flag, that suppresses staleness and leaf-change while leaving genuine faults audible | **READY (M) — NEW 2026-08-09** | — | The evidence is the operator's mailbox plus `events`/`notification_log` for 2026-08-09 | CC |
|
||||
| **R-1005** | Monitoring & notifications | P4 | **A recovery mail says how long the box was „stale", not how long it was gone: after a 51-minute power cut demo-hp's `host_recovered` read „was stale for 8m" and `node_recovered` „23m".** An operator reading only the recovery mail under-reads the outage (`audits/night-power-2026-10-13/` R6). Same kind, the other way: the off-site mail of 2026-10-14 02:18 was titled „3 of 3 apps failed to back up (offsite run)" while 11 other apps were backed up in that run — only the 3 flagged apps were counted. | **READY** — filed 2026-10-14 | — | CC: add the time since the last report before the gap to both recovery messages, and count every app of the run in the off-site subject, with a test that asserts each text; next hub release | CC |
|
||||
| **R-1049** | Monitoring & notifications | P3 | **A pulled drive mails the operator „unexpectedly disconnected” (error), and nothing ever mails that it came back.** MEASURED 2026-10-14 night (`audits/media-drive-events-2026-10-14/K8-hub-mails.txt`, the hub DB with its WAL from the running pod, and the mailbox as the second channel): two pulls → two `storage_disconnected` mails; the hub RECEIVED both `storage_reconnected` and both `health_recovered` (severity info) and logged no notification for any of them — they are not in `recoveredPairedDownTypes` (`hub/internal/notify/dispatcher.go`), so the severity gate drops them. The R-97a shape: told it broke, never told it healed — the operator cannot tell a returned drive from one still gone without opening the hub. **BUILT 2026-10-15 — hub v0.151.0 (`12552aca`): both pairs in `recoveredPairedDownTypes`, cooldowns ended by the next loss (`TestR1049_DriveBackMailsBothEdges`, red-proved). Not deployed: the operator's „yes” first (decision 162); the image build waits for it too.** | **WAITING-ON-OPERATOR** — hub v0.151.0 code pushed, not built, not deployed | the operator's „yes” to the hub deploy | On the „yes”: build + deploy hub v0.151.0; read a „came back” mail on the next pull | CC |
|
||||
| **R-1150** | Monitoring & notifications | P4 | **A refused whole-guest backup (the R-973 cap) reaches the operator as `agent_capability_degraded`, whose text reads „agent privileged capability degraded”.** True but misleading for a refused backup; on the night path the controller may also mail its own backup failure. Add a dedicated operator-only type (`whole_guest_backup_refused`: `allowedEventTypes` + `operatorOnlyEvents`) or adjust the text, in the next hub release (a two-repo change: the agent's status name stays). Found 2026-10-18. | OPEN — filed 2026-10-18 | — | Next hub release | CC |
|
||||
|
||||
|
||||
## Hub & operator — 12 rows (P3 6, P4 6)
|
||||
@@ -301,7 +303,7 @@ stopping line that lies.
|
||||
| **R-990** | Business & legal | P4 | **Operator: read `documentation/audits/forum-research-2026-10-12/FELHOM.md` §5 and rule on each recommendation.** READ-ONLY research 2026-10-12: three years (#7943–#11092, 2,908 posts, 195 people, 82 beginner threads) of the public PROHARDVER! topic „Home server / házi szerver építése”, tagged and counted by person (`FINDINGS.md`, method and spot-check in `METHOD.md`; no username or post text in git). Ten ranked recommendations, none acted on; the top three: a „which machine works” page (8 of 33 beginners who name their hardware own only a Synology, a router disk or a Raspberry Pi — the installer cannot run there; the same gap is R-989's first recommendation from a different source, and the shipped-page half is R-991), „you do not choose an operating system — we do” (the biggest beginner question, 20 of 82 threads), and „bring your old office PC” as the cheap entry. Catalog gap (≥ 3 people): a DLNA server (7), an ad/DNS blocker (5), a camera recorder (5). | **WAITING-ON-OPERATOR** — filed 2026-10-12 | the operator's reading | Operator: rule on each of the ten (yes / no / later). If nothing is done: nothing changes; the research ages | operator |
|
||||
| **R-1035** | Business & legal | P4 | **Did the „felhő tárhely” page earn anything? Two dated checks.** **Baseline (reported by the operator from Search Console, 2026-10-14, last 3 months):** 1 click, 48 impressions, CTR 2.1 %, average position **73.6**; „felhő tárhely” 19 impressions, „felhőtárhely” 5, „felhő alapú tárhely” 3. CC measured separately that the phrase „felhő tárhely” appeared **0 times** on the home, FAQ and backup pages — Google was showing the home page for those queries by accident. `/sajat-felho-tarhely` + `/en/own-cloud-storage` published **2026-10-14**; full baseline, price sources and the power provenance in `audits/seo-2026-10-14/BASELINE.md`. **What to measure, both times:** impressions and average position for „felhő tárhely” and „saját felhő tárhely”, plus the new page's own impressions, clicks and position. **A null result at +4 weeks means nothing on its own** — a new page on a site at position 73.6 takes weeks to months to move, and „felhő tárhely” alone is held by Microsoft, Google and the big list sites; the page aims at the two longer phrases. That is why there are two checks and not one, and why neither should be read as a verdict on the copy. **There is no Search Console connector in this project, so CC cannot read these numbers** — the operator reads them, CC records and compares. **Blocked until the operator asks for indexing** (Search Console → URL-ellenőrzés → the URL → „Indexelés kérése”): until that happens, or Google finds the page itself, both checks read zero for a reason that is not the page. **This is a slow channel and does not replace talking to people.** | **WATCHING** | operator: request indexing, then supply the Search Console read | 2026-11-11: first read (+4 weeks); 2026-12-09: second (+8 weeks), compared with the first | operator + CC |
|
||||
|
||||
## Process & tooling — 33 rows (P3 9, P4 24)
|
||||
## Process & tooling — 31 rows (P3 9, P4 22)
|
||||
|
||||
| ID | Category | Sev | What | State | Blocked on | Next action | Owner |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
@@ -335,9 +337,7 @@ stopping line that lies.
|
||||
| **R-1127** | Process & tooling | P4 | **The controller's and the catalog's decoy suites run only when someone starts a full gate run by hand** — neither the pre-push hook nor CI (`--fast`) runs them (too slow, or they edit real files), so a decoy that goes quiet is again seen only by chance (R-1115's shape). The agent's runs in CI. Found by the R-1116 helper 2026-10-17. | OPEN — filed 2026-10-17 | — | A scheduled full run (CI cron or the nightly), with a mail on failure | CC |
|
||||
| **R-1128** | Process & tooling | P4 | **The controller's decoy suite edits real tracked files (`handler_debug.go`, `update.go`, the locale bundles, templates) and plants `.go` files, with no lock** — a full run while another session edits the controller tree can overwrite that session's edits. felhom.eu's suite took an `flock` after the 2026-10-08 clobber. Found by the R-1116 helper 2026-10-17. | OPEN — filed 2026-10-17 | — | Run the suite in a scratch copy of the tree, or take the felhom.eu-style lock | CC |
|
||||
| **R-1123** | Process & tooling | P4 | **`reuse_refs_check.py` treats every git directory beside the repo as a sibling repo**, including stray clones in the workspace (`felhom.eu-media`, `felhom.eu-sizing-push`, `felhom.eu-wt-sizing`), so a citation can resolve against a stale clone and pass. Found by the R-1116 helper 2026-10-17 (it also makes the agent suite 20 s here vs 5 s in CI). | OPEN — filed 2026-10-17 | — | Search only the four known repo names (or the main clones), with a decoy | CC |
|
||||
| **R-1135** | Process & tooling | P4 | **CI's failure mail links the run at the in-cluster Gitea address** (`http://gitea.gitea-system.svc.cluster.local:3000/...`), which a reader outside the cluster cannot open. Seen in the forced test mail 2026-10-17 (job 1997). | OPEN — filed 2026-10-17 | — | Use the public Gitea URL in the alarm step's link | CC |
|
||||
| **R-1136** | Process & tooling | P4 | **felhom.eu CI runs take ~6.9 minutes and complete, so the 5-minute job timeout R-265 assumes does not hold there** (runs 1996, 1997; `script-tests` alone ~356 s). Found by the R-1129 proof helper 2026-10-17. | OPEN — filed 2026-10-17 | — | Measure the runner's real timeout; correct R-265's assumption or speed up `script-tests` | CC |
|
||||
| **R-1137** | Process & tooling | P4 | **A CI rerun may overwrite a failed job's log** — failure mails arrived for controller commits `c02c9d83` and `29a3ce8c` while their jobs (1944, 1959) now read green. UNPROVEN (a reading, 2026-10-17). | OPEN — filed 2026-10-17 | — | Confirm with one deliberate red + rerun on a scratch repo; if true, the CI check must read the first attempt | CC |
|
||||
| **R-1152** | Process & tooling | P4 | **felhom CI's 5-minute job timeout is inert** (act_runner 0.6.1 does not enforce a job-level `timeout-minutes`, R-1136), so a hung step runs up to the runner's 3 h. Move it to a step-level `timeout-minutes` on each workflow's gate step (e.g. 20 min for felhom.eu, which runs ~7 min), measured first: a step timeout fails the job AND the `if: failure()` alarm still runs. Needs a scratch repo (the admin token cannot create one — the operator, or a token with `write:user`). Found 2026-10-18. | OPEN — filed 2026-10-18 | — | Measure on a scratch repo, then the four workflows | CC |
|
||||
|
||||
<!-- DUE-CHECKS-BEGIN — machine-readable. Parsed by scripts/due_checks_gate.py.
|
||||
One row per dated check. The R-number must have a row above. Dates are UTC.
|
||||
|
||||
@@ -203,6 +203,10 @@ produced a false "unreachable" verdict (standing rule 2).
|
||||
Create and destroy freely **on a Tier 0 host**. Two exceptions: **`drill-r50`** is a fixture, not
|
||||
scratch; and scratch **customers** outlive their VMs in the hub — delete those too, or they accumulate
|
||||
(`sess-c` and `sess-d` were both left behind before anyone noticed).
|
||||
**One standing exception (operator, 2026-10-18): customer `Tester4` is KEPT** as the fresh-install customer
|
||||
(domain `tester4.test`, mail `tester4@felhom.eu` — the Gmail catch-all). Before each new install with it, the operator
|
||||
deletes its OLD host record on the Hosts page: an enroll that finds the customer's old host reuses it and skips the claim
|
||||
and off-site re-issues in silence.
|
||||
|
||||
**The hub-side teardown waits for the stale threshold — plan for it (R-599).** After the VM is destroyed
|
||||
the hub still reads its host as ONLINE until the last report is older than `alerting.stale_threshold`
|
||||
|
||||
@@ -0,0 +1,14 @@
|
||||
== 2026-10-18T20:52:00Z revert to virgin + cold boot
|
||||
revert rc=0
|
||||
qemu started
|
||||
ssh up after ~18s
|
||||
pve-manager/9.2.2/b9984c6d90a4bd80 (running kernel: 7.0.2-6-pve)
|
||||
update successful
|
||||
system debian-13-standard_13.6-1_amd64.tar.zst
|
||||
calculating checksum...OK, checksum verified
|
||||
download of 'http://download.proxmox.com/images/system/debian-13-standard_13.6-1_amd64.tar.zst' to '/var/lib/vz/template/cache/debian-13-standard_13.6-1_amd64.tar.zst' finished
|
||||
build-golden.sh sha256 645b3b659cbadebe from felhom-agent 1693da4
|
||||
Running as unit: golden-bake.service; invocation ID: b26a0833ce6744f584a73ad589beb99f
|
||||
token in unit props: 0
|
||||
[golden] build-golden.sh v3.2.0 — baking controller gitea.dooplex.hu/admin/felhom-controller:0.331.0
|
||||
[golden] creating build LXC 9100 (nesting=1,keyctl=1, unprivileged; rootfs 32G + ONE data volume 24G @ /var/lib/felhom, backup=1) …
|
||||
@@ -0,0 +1,3 @@
|
||||
## round trip 2026-10-18T20:59:13Z
|
||||
sha256 3881de62b0986ccbe5e55580e3f1bb9765d65b8d898f67c6339bac105244f84f bytes 637197198
|
||||
bake said 3881de62b0986ccbe5e55580e3f1bb9765d65b8d898f67c6339bac105244f84f
|
||||
@@ -0,0 +1,10 @@
|
||||
== teardown 2026-10-18T20:59:29Z
|
||||
purging CT 9100 from related configurations..
|
||||
0
|
||||
ls: cannot access '/root/.gitea-token': No such file or directory
|
||||
ls: cannot access '/root/bake-run.sh': No such file or directory
|
||||
ls: cannot access '/root/bake.log': No such file or directory
|
||||
qemu running: 0
|
||||
revert rc=0
|
||||
ID TAG VM_SIZE DATE VM_CLOCK ICOUNT
|
||||
1 virgin 0 B 2026-07-03 14:12:15 0000:00:00.000 0
|
||||
@@ -0,0 +1,43 @@
|
||||
# Golden 0.331.0 — bake + publish, 2026-10-18 (evening; the floor's version)
|
||||
|
||||
Why: the evening brief of 2026-10-18 (Part 7) asks for a golden at the floor that stands, as the LAST release step of the
|
||||
night. The floor was raised to 0.331.0 (MinAgent 0.158.1) at 20:48:56 UTC and does not move again tonight. Procedure:
|
||||
`documentation/runbooks/RUNBOOK-manual-build.md` §4.0 and §4.1 steps 1–4, in the drill VM on DooPlex. The script came
|
||||
from CC's `felhom-agent` clone at `origin/main` (`1693da4`; `build-golden.sh` sha256 `645b3b65…`, the same file as the
|
||||
0.329.0 bake).
|
||||
|
||||
| | Previous (`../golden-0.329.0-2026-10-18/`) | This bake |
|
||||
|---|---|---|
|
||||
| `build-golden.sh` | v3.2.0 | same file |
|
||||
| Controller | `felhom-controller:0.329.0` (MinAgent 0.158.1) | **`felhom-controller:0.331.0`** (MinAgent **0.158.1**) |
|
||||
| Docker engine | the approved set (six pairs) | the same six pairs — the hub's newest approved Docker release `os-docker-20261004-142842` (read from the hub DB tonight) |
|
||||
| Template | `debian-13-standard_13.6-1_amd64.tar.zst` | same (after `pveam update`; `_amd64` filtered) |
|
||||
|
||||
## Pass markers (from `bake.log`, this folder)
|
||||
|
||||
```
|
||||
[golden] build-golden.sh v3.2.0 — baking controller gitea.dooplex.hu/admin/felhom-controller:0.331.0
|
||||
[golden] Docker engine set PINNED to the approved release: containerd.io=2.3.6-1~debian.13~trixie … docker-compose-plugin=5.6.0-1~debian.13~trixie
|
||||
docker OK (overlay2; data-root /var/lib/docker)
|
||||
INFO: including mount point rootfs ('/') in backup
|
||||
INFO: including mount point mp0 ('/var/lib/felhom') in backup
|
||||
[golden] upload OK (HTTP 201)
|
||||
GOLDEN_VERSION=0.331.0
|
||||
GOLDEN_SHA256=3881de62b0986ccbe5e55580e3f1bb9765d65b8d898f67c6339bac105244f84f
|
||||
```
|
||||
|
||||
No `excluding`, no `FATAL`, no `GOLDEN_DOCKER_PKGS not set`; the unit `Result=success`. Round trip: the registry's file
|
||||
downloaded and hashed = the bake's sha, 637,197,198 bytes (`02-round-trip.txt`). Token: the unit's properties grep = 0
|
||||
(`00-boot.txt`); the committed log grep = 0, with a working control (= 1 on a copy with the token appended, the copy
|
||||
shredded). Teardown: `03-teardown.txt` (CT 9100 purged, token/script/log shredded in the VM, VM off, disk back to
|
||||
`virgin`). No controller push after the bake.
|
||||
|
||||
## Vouch — the operator's act (never CC's)
|
||||
|
||||
Three fields together (runbook §4.1 step 5), Configuration → Day-0 artifacts:
|
||||
|
||||
- golden **0.331.0**, sha256 `3881de62b0986ccbe5e55580e3f1bb9765d65b8d898f67c6339bac105244f84f`
|
||||
- agent **0.159.2** (already vouched, unchanged), sha256 `4d98868e922d6c10665a6c0da9323e8627ea9c589094e81d27d741e625d9d8f9`
|
||||
- min_agent **0.158.1** (0.331.0's MinAgent)
|
||||
|
||||
The R-120 gate accepts it: the golden equals the newest controller the fleet reports (0.331.0 on all three test boxes).
|
||||
@@ -0,0 +1,342 @@
|
||||
[golden] build-golden.sh v3.2.0 — baking controller gitea.dooplex.hu/admin/felhom-controller:0.331.0
|
||||
[golden] creating build LXC 9100 (nesting=1,keyctl=1, unprivileged; rootfs 32G + ONE data volume 24G @ /var/lib/felhom, backup=1) …
|
||||
Logical volume "vm-9100-disk-0" created.
|
||||
Logical volume pve/vm-9100-disk-0 changed.
|
||||
Creating filesystem with 8388608 4k blocks and 2097152 inodes
|
||||
Filesystem UUID: 1eb5b0d8-5a90-4c9c-b0e0-955a2de54a99
|
||||
Superblock backups stored on blocks:
|
||||
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
|
||||
4096000, 7962624
|
||||
Logical volume "vm-9100-disk-1" created.
|
||||
Logical volume pve/vm-9100-disk-1 changed.
|
||||
Creating filesystem with 6291456 4k blocks and 1572864 inodes
|
||||
Filesystem UUID: 1e406149-1c42-4d19-bf3f-898513746062
|
||||
Superblock backups stored on blocks:
|
||||
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
|
||||
extracting archive '/var/lib/vz/template/cache/debian-13-standard_13.6-1_amd64.tar.zst'
|
||||
Total bytes read: 553512960 (528MiB, 137MiB/s)
|
||||
Detected container architecture: amd64
|
||||
Creating SSH host key 'ssh_host_rsa_key' - this may take some time ...
|
||||
done: SHA256:MPjcumton7btXPX/TTneLuTimHFNXPUVyhANi1WnHnQ root@felhom-golden
|
||||
Creating SSH host key 'ssh_host_ecdsa_key' - this may take some time ...
|
||||
done: SHA256:npjwkW6W8zF5CLbKHXK7Rp8RZMec+z23dtRbV/P8wKU root@felhom-golden
|
||||
Creating SSH host key 'ssh_host_ed25519_key' - this may take some time ...
|
||||
done: SHA256:5BbaIK8KpPL9/JLpxPAqgbRmbXL1Tm4yzPjp4Wl7eB4 root@felhom-golden
|
||||
[golden] starting + installing Docker (official repo, trixie channel) …
|
||||
[golden] Docker engine set PINNED to the approved release: containerd.io=2.3.6-1~debian.13~trixie docker-buildx-plugin=0.37.1-1~debian.13~trixie docker-ce=5:29.8.2-1~debian.13~trixie docker-ce-cli=5:29.8.2-1~debian.13~trixie docker-ce-rootless-extras=5:29.8.2-1~debian.13~trixie docker-compose-plugin=5.6.0-1~debian.13~trixie
|
||||
apt-listchanges: Can't set locale; make sure $LC_* and $LANG are correct!
|
||||
perl: warning: Setting locale failed.
|
||||
perl: warning: Please check that your locale settings:
|
||||
LANGUAGE = (unset),
|
||||
LC_ALL = (unset),
|
||||
LC_CTYPE = (unset),
|
||||
LC_NUMERIC = (unset),
|
||||
LC_COLLATE = (unset),
|
||||
LC_TIME = (unset),
|
||||
LC_MESSAGES = (unset),
|
||||
LC_MONETARY = (unset),
|
||||
LC_ADDRESS = (unset),
|
||||
LC_IDENTIFICATION = (unset),
|
||||
LC_MEASUREMENT = (unset),
|
||||
LC_PAPER = (unset),
|
||||
LC_TELEPHONE = (unset),
|
||||
LC_NAME = (unset),
|
||||
LANG = "en_US.UTF-8"
|
||||
are supported and installed on your system.
|
||||
perl: warning: Falling back to the standard locale ("C").
|
||||
locale: Cannot set LC_CTYPE to default locale: No such file or directory
|
||||
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
|
||||
locale: Cannot set LC_ALL to default locale: No such file or directory
|
||||
apt-listchanges: Can't set locale; make sure $LC_* and $LANG are correct!
|
||||
perl: warning: Setting locale failed.
|
||||
perl: warning: Please check that your locale settings:
|
||||
LANGUAGE = (unset),
|
||||
LC_ALL = (unset),
|
||||
LC_CTYPE = (unset),
|
||||
LC_NUMERIC = (unset),
|
||||
LC_COLLATE = (unset),
|
||||
LC_TIME = (unset),
|
||||
LC_MESSAGES = (unset),
|
||||
LC_MONETARY = (unset),
|
||||
LC_ADDRESS = (unset),
|
||||
LC_IDENTIFICATION = (unset),
|
||||
LC_MEASUREMENT = (unset),
|
||||
LC_PAPER = (unset),
|
||||
LC_TELEPHONE = (unset),
|
||||
LC_NAME = (unset),
|
||||
LANG = "en_US.UTF-8"
|
||||
are supported and installed on your system.
|
||||
perl: warning: Falling back to the standard locale ("C").
|
||||
locale: Cannot set LC_CTYPE to default locale: No such file or directory
|
||||
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
|
||||
locale: Cannot set LC_ALL to default locale: No such file or directory
|
||||
installed: containerd.io 2.3.6-1~debian.13~trixie
|
||||
installed: docker-buildx-plugin 0.37.1-1~debian.13~trixie
|
||||
installed: docker-ce 5:29.8.2-1~debian.13~trixie
|
||||
installed: docker-ce-cli 5:29.8.2-1~debian.13~trixie
|
||||
installed: docker-ce-rootless-extras 5:29.8.2-1~debian.13~trixie
|
||||
installed: docker-compose-plugin 5.6.0-1~debian.13~trixie
|
||||
[golden] no approved guest release given - the template versions stay; first-night count vs an approved release: n/a
|
||||
[golden] pending Debian upgrades in the baked guest (what a FUTURE approval may bring): 56
|
||||
[golden] baking daemon.json: classic overlay2 driver (containerd-snapshotter OFF) + log rotation …
|
||||
[golden] wiring the single data volume (R-165 variant V-c): /var/lib/felhom/{docker,sys_drive} -> binds …
|
||||
[golden] verifying Docker works in the build guest (storage driver should be overlay2 on the ext4 data volume) …
|
||||
Unable to find image 'hello-world:latest' locally
|
||||
latest: Pulling from library/hello-world
|
||||
4f55086f7dd0: Pulling fs layer
|
||||
4f55086f7dd0: Verifying Checksum
|
||||
4f55086f7dd0: Download complete
|
||||
4f55086f7dd0: Pull complete
|
||||
Digest: sha256:11a3eb1597e864c51b3f56733cb2e8a90bf0c284bfc05f9444d81c558bc848aa
|
||||
Status: Downloaded newer image for hello-world:latest
|
||||
docker OK (overlay2; data-root /var/lib/docker)
|
||||
live-restore: on
|
||||
/var/lib/docker is a real mount: /dev/mapper/pve-vm--9100--disk--1[/docker] ext4
|
||||
/mnt/sys_drive is a real mount: /dev/mapper/pve-vm--9100--disk--1[/sys_drive] ext4
|
||||
both paths are ONE filesystem: /dev/mapper/pve-vm--9100--disk--1 23317576
|
||||
[golden] baking the in-guest controller image gitea.dooplex.hu/admin/felhom-controller:0.331.0 (no registry cred at deploy) …
|
||||
|
||||
WARNING! Your credentials are stored unencrypted in '/root/.docker/config.json'.
|
||||
Configure a credential helper to remove this warning. See
|
||||
https://docs.docker.com/go/credential-store/
|
||||
|
||||
0.331.0: Pulling from admin/felhom-controller
|
||||
ecc510c1e359: Pulling fs layer
|
||||
c3f83389ce1b: Pulling fs layer
|
||||
e81c1fce9826: Pulling fs layer
|
||||
63a1406b1961: Pulling fs layer
|
||||
d747c9b6922d: Pulling fs layer
|
||||
2eabf5e8786f: Pulling fs layer
|
||||
4659a8fea0f6: Pulling fs layer
|
||||
d747c9b6922d: Waiting
|
||||
63a1406b1961: Waiting
|
||||
2eabf5e8786f: Waiting
|
||||
4659a8fea0f6: Waiting
|
||||
e81c1fce9826: Verifying Checksum
|
||||
e81c1fce9826: Download complete
|
||||
63a1406b1961: Verifying Checksum
|
||||
63a1406b1961: Download complete
|
||||
d747c9b6922d: Verifying Checksum
|
||||
d747c9b6922d: Download complete
|
||||
c3f83389ce1b: Verifying Checksum
|
||||
c3f83389ce1b: Download complete
|
||||
4659a8fea0f6: Verifying Checksum
|
||||
4659a8fea0f6: Download complete
|
||||
2eabf5e8786f: Verifying Checksum
|
||||
2eabf5e8786f: Download complete
|
||||
ecc510c1e359: Verifying Checksum
|
||||
ecc510c1e359: Download complete
|
||||
ecc510c1e359: Pull complete
|
||||
c3f83389ce1b: Pull complete
|
||||
e81c1fce9826: Pull complete
|
||||
63a1406b1961: Pull complete
|
||||
d747c9b6922d: Pull complete
|
||||
2eabf5e8786f: Pull complete
|
||||
4659a8fea0f6: Pull complete
|
||||
Digest: sha256:1b11d25ac8074f6bbdc073ea8564e8e9a91a6e245f9d91ce49791011efb73dbf
|
||||
Status: Downloaded newer image for gitea.dooplex.hu/admin/felhom-controller:0.331.0
|
||||
gitea.dooplex.hu/admin/felhom-controller:0.331.0
|
||||
[golden] asking the controller which infra images it manages …
|
||||
[golden] baking infra images (4): traefik:v3.7.13 cloudflare/cloudflared:2026.9.3 gtstef/filebrowser:1.5.6-stable gitea.dooplex.hu/admin/felhom-samba:1.2.0 …
|
||||
v3.7.13: Pulling from library/traefik
|
||||
e2de96513ba9: Pulling fs layer
|
||||
b686a4f73445: Pulling fs layer
|
||||
78cb21c375ca: Pulling fs layer
|
||||
acb2f33459b1: Pulling fs layer
|
||||
acb2f33459b1: Waiting
|
||||
e2de96513ba9: Verifying Checksum
|
||||
e2de96513ba9: Download complete
|
||||
b686a4f73445: Verifying Checksum
|
||||
b686a4f73445: Download complete
|
||||
acb2f33459b1: Verifying Checksum
|
||||
acb2f33459b1: Download complete
|
||||
e2de96513ba9: Pull complete
|
||||
78cb21c375ca: Verifying Checksum
|
||||
78cb21c375ca: Download complete
|
||||
b686a4f73445: Pull complete
|
||||
78cb21c375ca: Pull complete
|
||||
acb2f33459b1: Pull complete
|
||||
Digest: sha256:24841fe2de7304c149343d877d2923b4c8800a38ba015dea9174c23b20e344a0
|
||||
Status: Downloaded newer image for traefik:v3.7.13
|
||||
docker.io/library/traefik:v3.7.13
|
||||
2026.9.3: Pulling from cloudflare/cloudflared
|
||||
2cc7ee286bf3: Pulling fs layer
|
||||
c172f21841df: Pulling fs layer
|
||||
218cf840d0d9: Pulling fs layer
|
||||
f6069939f718: Pulling fs layer
|
||||
d6b1b89eccac: Pulling fs layer
|
||||
2780920e5dbf: Pulling fs layer
|
||||
7c12895b777b: Pulling fs layer
|
||||
3214acf345c0: Pulling fs layer
|
||||
52630fc75a18: Pulling fs layer
|
||||
dd64bf2dd177: Pulling fs layer
|
||||
b839dfae01f6: Pulling fs layer
|
||||
ebddc55facdc: Pulling fs layer
|
||||
c4bc6f35ff5e: Pulling fs layer
|
||||
b96fe2995f90: Pulling fs layer
|
||||
58c0c263dc73: Pulling fs layer
|
||||
bd8962e29291: Pulling fs layer
|
||||
cac2ae0193cb: Pulling fs layer
|
||||
f0383d5ebc47: Pulling fs layer
|
||||
dd64bf2dd177: Waiting
|
||||
f6069939f718: Waiting
|
||||
d6b1b89eccac: Waiting
|
||||
2780920e5dbf: Waiting
|
||||
7c12895b777b: Waiting
|
||||
3214acf345c0: Waiting
|
||||
52630fc75a18: Waiting
|
||||
b839dfae01f6: Waiting
|
||||
ebddc55facdc: Waiting
|
||||
c4bc6f35ff5e: Waiting
|
||||
b96fe2995f90: Waiting
|
||||
58c0c263dc73: Waiting
|
||||
bd8962e29291: Waiting
|
||||
cac2ae0193cb: Waiting
|
||||
f0383d5ebc47: Waiting
|
||||
2cc7ee286bf3: Verifying Checksum
|
||||
2cc7ee286bf3: Download complete
|
||||
218cf840d0d9: Verifying Checksum
|
||||
218cf840d0d9: Download complete
|
||||
2cc7ee286bf3: Pull complete
|
||||
c172f21841df: Verifying Checksum
|
||||
c172f21841df: Download complete
|
||||
f6069939f718: Verifying Checksum
|
||||
f6069939f718: Download complete
|
||||
d6b1b89eccac: Verifying Checksum
|
||||
d6b1b89eccac: Download complete
|
||||
2780920e5dbf: Verifying Checksum
|
||||
2780920e5dbf: Download complete
|
||||
c172f21841df: Pull complete
|
||||
7c12895b777b: Verifying Checksum
|
||||
7c12895b777b: Download complete
|
||||
3214acf345c0: Verifying Checksum
|
||||
3214acf345c0: Download complete
|
||||
52630fc75a18: Verifying Checksum
|
||||
52630fc75a18: Download complete
|
||||
dd64bf2dd177: Verifying Checksum
|
||||
dd64bf2dd177: Download complete
|
||||
b839dfae01f6: Verifying Checksum
|
||||
b839dfae01f6: Download complete
|
||||
ebddc55facdc: Verifying Checksum
|
||||
ebddc55facdc: Download complete
|
||||
c4bc6f35ff5e: Verifying Checksum
|
||||
c4bc6f35ff5e: Download complete
|
||||
58c0c263dc73: Verifying Checksum
|
||||
58c0c263dc73: Download complete
|
||||
bd8962e29291: Verifying Checksum
|
||||
bd8962e29291: Download complete
|
||||
cac2ae0193cb: Download complete
|
||||
218cf840d0d9: Pull complete
|
||||
f0383d5ebc47: Verifying Checksum
|
||||
f0383d5ebc47: Download complete
|
||||
b96fe2995f90: Verifying Checksum
|
||||
b96fe2995f90: Download complete
|
||||
f6069939f718: Pull complete
|
||||
d6b1b89eccac: Pull complete
|
||||
2780920e5dbf: Pull complete
|
||||
7c12895b777b: Pull complete
|
||||
3214acf345c0: Pull complete
|
||||
52630fc75a18: Pull complete
|
||||
dd64bf2dd177: Pull complete
|
||||
b839dfae01f6: Pull complete
|
||||
ebddc55facdc: Pull complete
|
||||
c4bc6f35ff5e: Pull complete
|
||||
b96fe2995f90: Pull complete
|
||||
58c0c263dc73: Pull complete
|
||||
bd8962e29291: Pull complete
|
||||
cac2ae0193cb: Pull complete
|
||||
f0383d5ebc47: Pull complete
|
||||
Digest: sha256:072c067d25ccbe61d46e18f0d0723255f2bb5304f7317caa95b27031520ff92c
|
||||
Status: Downloaded newer image for cloudflare/cloudflared:2026.9.3
|
||||
docker.io/cloudflare/cloudflared:2026.9.3
|
||||
1.5.6-stable: Pulling from gtstef/filebrowser
|
||||
55afa1ecc21d: Pulling fs layer
|
||||
8ed8f35f8d4f: Pulling fs layer
|
||||
989b226a579c: Pulling fs layer
|
||||
660aeead31d5: Pulling fs layer
|
||||
4f4fb700ef54: Pulling fs layer
|
||||
adce24567e4c: Pulling fs layer
|
||||
f17ea56b313b: Pulling fs layer
|
||||
6b6f3b3efe88: Pulling fs layer
|
||||
4ed1ca4f3fce: Pulling fs layer
|
||||
e6fc9c6a5757: Pulling fs layer
|
||||
d47782d1182a: Pulling fs layer
|
||||
6b6f3b3efe88: Waiting
|
||||
4ed1ca4f3fce: Waiting
|
||||
e6fc9c6a5757: Waiting
|
||||
d47782d1182a: Waiting
|
||||
660aeead31d5: Waiting
|
||||
4f4fb700ef54: Waiting
|
||||
adce24567e4c: Waiting
|
||||
f17ea56b313b: Waiting
|
||||
55afa1ecc21d: Verifying Checksum
|
||||
55afa1ecc21d: Download complete
|
||||
660aeead31d5: Verifying Checksum
|
||||
660aeead31d5: Download complete
|
||||
4f4fb700ef54: Verifying Checksum
|
||||
4f4fb700ef54: Download complete
|
||||
55afa1ecc21d: Pull complete
|
||||
8ed8f35f8d4f: Verifying Checksum
|
||||
8ed8f35f8d4f: Download complete
|
||||
989b226a579c: Verifying Checksum
|
||||
989b226a579c: Download complete
|
||||
6b6f3b3efe88: Verifying Checksum
|
||||
6b6f3b3efe88: Download complete
|
||||
f17ea56b313b: Verifying Checksum
|
||||
f17ea56b313b: Download complete
|
||||
4ed1ca4f3fce: Verifying Checksum
|
||||
4ed1ca4f3fce: Download complete
|
||||
e6fc9c6a5757: Download complete
|
||||
adce24567e4c: Verifying Checksum
|
||||
adce24567e4c: Download complete
|
||||
d47782d1182a: Download complete
|
||||
8ed8f35f8d4f: Pull complete
|
||||
989b226a579c: Pull complete
|
||||
660aeead31d5: Pull complete
|
||||
4f4fb700ef54: Pull complete
|
||||
adce24567e4c: Pull complete
|
||||
f17ea56b313b: Pull complete
|
||||
6b6f3b3efe88: Pull complete
|
||||
4ed1ca4f3fce: Pull complete
|
||||
e6fc9c6a5757: Pull complete
|
||||
d47782d1182a: Pull complete
|
||||
Digest: sha256:7c5d7ac8ffda31294d278063cf9d2e04303b39e6dce1f4c691342240ca7703b8
|
||||
Status: Downloaded newer image for gtstef/filebrowser:1.5.6-stable
|
||||
docker.io/gtstef/filebrowser:1.5.6-stable
|
||||
1.2.0: Pulling from admin/felhom-samba
|
||||
e2de96513ba9: Already exists
|
||||
0ddbc9f3a6b6: Pulling fs layer
|
||||
ce57a3f93416: Pulling fs layer
|
||||
fb94eeec2fe1: Pulling fs layer
|
||||
fb94eeec2fe1: Verifying Checksum
|
||||
fb94eeec2fe1: Download complete
|
||||
ce57a3f93416: Verifying Checksum
|
||||
ce57a3f93416: Download complete
|
||||
0ddbc9f3a6b6: Verifying Checksum
|
||||
0ddbc9f3a6b6: Download complete
|
||||
0ddbc9f3a6b6: Pull complete
|
||||
ce57a3f93416: Pull complete
|
||||
fb94eeec2fe1: Pull complete
|
||||
Digest: sha256:da96f25033827fbb77d0ee398e86fe5b744e2f29be9de898994533340c97acf9
|
||||
Status: Downloaded newer image for gitea.dooplex.hu/admin/felhom-samba:1.2.0
|
||||
gitea.dooplex.hu/admin/felhom-samba:1.2.0
|
||||
[golden] baking the controller-bootstrap unit (deploys the BAKED controller from the config mount) …
|
||||
Created symlink '/etc/systemd/system/multi-user.target.wants/felhom-controller-bootstrap.service' → '/etc/systemd/system/felhom-controller-bootstrap.service'.
|
||||
[golden] baking the controller-bootstrap PATH unit (starts the service on bootstrap-mount hot-plug — B1) …
|
||||
Created symlink '/etc/systemd/system/multi-user.target.wants/felhom-controller-bootstrap.path' → '/etc/systemd/system/felhom-controller-bootstrap.path'.
|
||||
[golden] baking the first-boot SSH host-key regeneration unit (F3) …
|
||||
Created symlink '/etc/systemd/system/multi-user.target.wants/felhom-regen-hostkeys.service' → '/etc/systemd/system/felhom-regen-hostkeys.service'.
|
||||
[golden] identity-clean + minimize …
|
||||
[golden] stop + archive …
|
||||
INFO: including mount point rootfs ('/') in backup
|
||||
INFO: including mount point mp0 ('/var/lib/felhom') in backup
|
||||
INFO: archive file size: 607MB
|
||||
INFO: Finished Backup of VM 9100 (00:00:29)
|
||||
[golden] DONE. golden archive volid: local:backup/vzdump-lxc-9100-2026_10_18-22_57_23.tar.zst (rootfs 32G + ONE data volume 24G @ /var/lib/felhom, all in the archive)
|
||||
[golden] publishing golden (637197198 bytes, sha256 3881de62b0986ccb…) → https://gitea.dooplex.hu/api/packages/admin/generic/felhom-golden/0.331.0/golden.tar.zst
|
||||
[golden] pre-delete existing: HTTP 404 (404/204 expected)
|
||||
[golden] upload OK (HTTP 201)
|
||||
GOLDEN_VERSION=0.331.0
|
||||
GOLDEN_SHA256=3881de62b0986ccbe5e55580e3f1bb9765d65b8d898f67c6339bac105244f84f
|
||||
[golden] Record in the hub operator UI (Configs → Day-0 artifacts): golden 0.331.0 / 3881de62b0986ccbe5e55580e3f1bb9765d65b8d898f67c6339bac105244f84f
|
||||
[golden] (the build guest 9100 is stopped; destroy it with: pct destroy 9100 --purge)
|
||||
Reference in New Issue
Block a user