2026-10-26 release run: hub 0.171.0, agent 0.166.0 (vouched), controller 0.344.0 (global floor 09:07:47 UTC), catalog 73c165d, golden 0.344.0 (baked, vouched; a fresh install came up on 0.344.0); the golden waiver removed — the gate is green on the golden. Rows: 9 closed (R-1278, R-1280–R-1283, R-1286–R-1289), R-1285 back to OPEN, R-1294 and R-1295 filed; STATUS, CONTEXT, evidence (scanned, 0 hits).
gates / gates (push) Successful in 13m35s

This commit is contained in:
admin committed 2026-10-26 10:42:54 +01:00
1 parent 6efb47607f
commit c69f53055d
50 files changed
+1133 -41

No files matched your search

+10
View File
@@ -16,6 +16,16 @@
> and holds nothing of its own; this file does hold its own content, namely the standing rulings below.
> **2026-10-26 (attended release run) — released: hub 0.171.0, agent 0.166.0 (vouched 10:04:55 CET), controller 0.344.0
> (global floor 09:07:47 UTC, MinAgent 0.158.1), catalog `73c165d`; golden 0.344.0 baked and vouched (10:26:23 CET).**
> Live on the release builds (`audits/release-monday-2026-10-26/`): R-1287 (the set-aside mails „⚠️ … nothing failed"),
> R-1289 (the vouch line names 0.344.0), R-1280 (a 25 h job REJECTED on demo-hp, a 30 min one ran), R-1281 (forged header
> 000 with a working control; the household's files 200; OnlyOffice fetches the document over `felhom-files`), R-1282,
> R-1278, R-1286, R-1283 — closed. **R-1285 back to OPEN:** in the normal two-job delivery the bundle runs in the OLD
> agent process, so the policy pass waits for the next bundle. R-1261 stays VERIFY until Tuesday's live line test (R-1250).
> New: R-1294 (demo-hp's weekly off-site clean-up refused by its own cap, 209 > 179). The golden waiver file is removed —
> the gate is green on the new golden alone. D50 pin `day0_agent` → 0.166.0.
> **2026-10-26 (attended release run) — D51 = Start; D52 = bake the golden today on v0.344.0.** (`09` §3 decisions 263,
> 264.) D51: the operator pressed Start on demo-hp's Kimai (07:57 local); the crash-loop hold lifted, both containers
> healthy, 0 restarts (R-1286). D52: the Sunday 2026-10-25 bake was missed (not in that night's brief); the golden waiver
+75
View File
@@ -0,0 +1,75 @@
# REPORT — the 2026-10-26 release: hub 0.171.0, agent 0.166.0, controller 0.344.0, catalog, golden 0.344.0
**Plain words first.**
1. **The push is unblocked.** The waiver covered one day; the three waiting felhom.eu commits went out (CI 2352 ✓ on
`7ad7c996`, which carries `b00d5c61` and `1550b1da`). The waiver is now removed: the golden gate is green on the new
golden alone.
2. **Hub 0.171.0 is live.** A set-aside now mails „⚠️ … nothing failed" as a warning (seen in your mailbox), and the
vouch line names a release version as „the fleet's newest".
3. **Agent 0.166.0 is vouched** (10:04:55 CET) and runs on the three test boxes; a 25-hour signed job is refused on a box.
4. **Controller 0.344.0 runs on every live box** (3 of 3; the global floor 09:07:47 UTC). Tester-2 has no box (R-862).
5. **Golden 0.344.0 is baked and vouched** (10:26:23 CET); a fresh install from the public ISO came up on 0.344.0 with no
update step.
## The release-build checks (one line each)
- **R-1287** — Tester 1 set-aside → mail „⚠️ tester-1: a backup of an earlier install was set aside — nothing failed",
warning (hub log + mailbox; the 10-25 mails of the same event read 🔴 „failed"). Closed.
- **R-1289** — the agent vouch logged „behind the fleet's 0.344.0" while `0.344.0-rc.3` reports sit in the same table. Closed.
- **R-1280** — demo-hp: 25 h job REJECTED, 30 min job ran; `felhom-opsign -ttl 25h` refuses. Closed.
- **R-1285** — the policy pass is wired (seen under 0.166.0), but in the normal delivery the bundle runs in the OLD agent
process, so a BYO host still loses our drop-in one release late. **Back to OPEN.**
- **R-1281** — FileBrowser only on its own networks on all three boxes; forged header from four apps → 000 (control 404/301);
household files 200, no cookie 401; OnlyOffice linked: joins `felhom-files`, editor config 200, the document fetched 200.
The file manager restarted once at the update. Closed.
- **R-1261 / R-1250** — the line hold is in 0.344.0; the live test is Tuesday 10:00–16:00 (dated check stays). R-1261 VERIFY.
- **R-1282** — „Később" survives a controller restart (start-up line + settings.json). Closed.
- **R-1278** — the install page shows the kept-backup hint and link; none before or after (Tester 1). Closed.
- **R-1286** — Kimai has its install id (07:01:55Z); the hold sentences are released. Closed.
- **R-1288** — released (proven on the night's rig). Closed.
- **R-1283** — catalog `73c165d` pushed; Tester 1 synced at once, MeTube's folder `optional`. Closed.
- **D47** — 0 set aside after the update; every app has an install id.
## Versions, digests, CI by commit
| What | Version / digest | Commit → CI job |
|---|---|---|
| hub | 0.171.0, `sha256:742d2e69ce75d0d96598e8ee9a9acb11e677a3663348571e34abbd3819c7c46e` | `7ad7c996` → 2352 ✓ (code `f6328486` → 2342 ✓); manifest `6efb4760` |
| agent | 0.166.0, binary `3f22c65b…bfea9c5c`, bundle `093772c1…5156003a3` | `aa69114` → 2351 ✓; record `b890b57` → 2356 ✓ |
| controller | 0.344.0, `sha256:9435165dab85b1200d94157ec49d8be065889139e1831a5476dc68f4b9474b8a` | `0a59044` → 2355 ✓ |
| catalog | `73c165d` | gates green locally (catalog CI runs on push) |
| golden | 0.344.0, `767a90fe07ed47c85a3e08a16b85926e30531b65608f9856aa25855b4dd53210` (637,293,448 bytes) | `documentation/tests/golden-0.344.0-2026-10-26/` |
## Rows
Before **219**, after **212**; opened **2** (R-1294 demo-hp's off-site clean-up refused by its own cap; R-1295 PBS DR
provisioning rolled back at a re-enrolled install, log only); closed **9** (R-1278, R-1280, R-1281, R-1282, R-1283, R-1286,
R-1287, R-1288, R-1289). R-1285 back to OPEN.
## Observations
- FILED: R-1294 — demo-hp's weekly off-site clean-up refused (209 > 179) at 03:17Z; each refused week grows the next plan.
- FILED: R-1295 — at the fresh install the PBS DR auto-provision for Tester4 rolled back (`self_check_failed`); no mail.
- R-1285 reopened (delivery order), see above.
- NOT-A-FINDING: the vouch's `wrapper_sha=false` — the PBS-DR wrapper was never vouched (empty before and after).
- NOT-A-FINDING: 9202's rc start logged one WARN while FileBrowser was re-created in the same second; joined 2 s later, log
only; the release boxes logged none.
- NOT-A-FINDING: the 07:23Z self-bind mail to tester4@ — the operator's delete of `Tester4-84bbe7`, as designed.
- NOT-A-FINDING: the 02:57Z DooPlex sweep found `RIG_2026_10_25_SETUP_CODE` in last night's session record — that box is
destroyed, so the code opens nothing; the sweep worked as designed.
- NOT-A-FINDING: R-1282's status API read „LOST" on Tester 1 — the API shows `later_at` only on a box with media apps;
the stored value and the start-up line both kept it (instrument, recorded in the evidence).
- Deviation from the brief, said once: the fresh-install proof ran AFTER the golden vouch, because an install refuses any
golden but the vouched one (R-297); the vouch was reversible and the proof passed.
## Teardown
- **Machines:** fresh-install VM 9373 on demo-hp destroyed with its disks (nvme-scratch); Tester 1's throwaway Tandoor (×2)
and OnlyOffice (×2) removed through the product, nothing left; 9202 back on 0.343.0 with FileBrowser on `traefik-public`
and no `felhom-files` network, the rc image deleted; the drill VM back to `virgin` (CT 9100 purged).
- **Host:** demo-hp's thin pool never used; `nvme-scratch` (standing) untouched otherwise.
- **Hub:** host `Tester4-eb7371` (the fresh-install VM) stays for the operator to delete (STATUS); per-box floors cleared.
- **Secrets:** bake token/user, bind token, pairing code shredded; the pairing code blacked out in the committed
screenshot; evidence scanned (6 sec values + the 3 one-time values: 0 hits). New sec entry `REL_2026_10_26_ROOTPW`
(the test VM's typed root password, replaced at day-0; the VM is gone).
- One test side effect: at 2026-10-27 03:30 the „Később" timer fires on Tester 1, which has no media app — it restarts nothing.
+3 -2
View File
@@ -2,7 +2,7 @@
**Ready for Tester 2? Yes — tell the friend go (hand-over list: `documentation/pilot/TESTER2-handover.md`).**
**Updated 2026-10-26 (morning).** Installer 1.36.1 is live: a wrong passphrase now says so. Overnight, commit only: the torrent line test holds every seeder; FileBrowser is off the app network; a set-aside mails as a set-aside; the four D44 picks are built and proven on a test VM.
**Updated 2026-10-26 (release day).** Every box runs hub 0.171.0, agent 0.166.0 and controller 0.344.0, and new installs start on golden 0.344.0 — all vouched: the torrent line hold, FileBrowser off the app network, a set-aside mails „nothing failed", a signed job lasts at most 24 h. One leftover: on a BYO host the SSH clean-up still comes one release late (R-1285).
## What works
- **The Plex attack (10-17 → 10-23) did not reach Felhom's secrets** (R-1241).
@@ -23,6 +23,7 @@
- DooPlex: something sets the shared checkout to „bare" (R-935).
## What we work on
- Next for CC: demo-hp's weekly off-site clean-up is refused by its own cap (R-1294); the BYO SSH clean-up runs one release late (R-1285).
- Dated checks, run by CC. 2026-10-27: the qBittorrent line test on a seeding box, with DooPlex's qBittorrent as the peer (R-1250, your yes); the weekly clean-up window (R-435; the logs of the last one were lost to restarts). 2026-10-30: D35, website and installer to Cloudflare, starts (R-1228, ruled yes).
## Waiting on you — decide
@@ -47,7 +48,7 @@ Newest first. CC's pick is in the „Pick" column. Each row also has its full te
6. BIOS: on the N100, read „After Power Loss"; on the HP, set „Bypass F1 Prompt on Configuration Changes" to Enable. Since 2026-10-14.
7. Pin the first Facebook post (R-917), and look at the new cover in the Facebook phone app (R-919). Since 2026-10-12.
**Monday 2026-10-26 (attended; each a STOP).** 1. Hub v0.171.0 (R-1287, R-1289). 2. Agent v0.166.0 + your vouch (R-1280, R-1285). 3. Controller v0.344.0 — before any household gets agent 0.165.0 (R-1261, R-1281, R-1288, R-1282, R-1286, R-1278). 4. Catalog: MeTube (R-1283). 5. Dated: R-1250 Tuesday. 6. Hub Hosts page: delete the dead `Tester4-84bbe7`. If not: the fixes stay unreleased; the host alarms.
**2026-10-26.** On the hub's Hosts page, delete `Tester4-eb7371` (CC's fresh-install test VM, destroyed). If not: the hub alarms that the box went silent.
**2026-10-23 do list.**
1. Print the break-glass sheet (`documentation/runbooks/break-glass-sheet.md`, the part from „The sheet”, one A4), fill it by hand, keep it outside the home (D32; R-923, R-924). If not: after a DooPlex loss no off-site copy opens.
2. On the hub's Hosts page: Discard appliance 49. If not: a dead entry alarms.
@@ -0,0 +1,33 @@
# agent 0.166.0 delivery by signed agent_update + agent_config_update (felhom-opsign, key felhom-op-1), 2026-10-26
== demo-hp
felhom-agent 0.166.0
time=2026-10-26T09:03:07.108+01:00 level=WARN msg="signedjobs: signed op COMPLETED" job=d70489200b46088d op=agent_update
time=2026-10-26T09:03:08.570+01:00 level=WARN msg="osupdate: config bundle INSTALLED" op=agent_config_update agent_version=0.166.0 bundle="{\"agent_version\": \"0.166.0\", \"authority\": \"signed\", \"kept\": [\"/etc/felhom/crash-guard.conf\"], \"prev_dir\": \"/var/lib/felhom-os-apply/bundle-prev/20261026T080307Z-before-0.166.0\
time=2026-10-26T09:03:09.616+01:00 level=WARN msg="signedjobs: signed op COMPLETED" job=81bbc6a41cef76cb op=agent_config_update
time=2026-10-26T09:03:10.331+01:00 level=INFO msg="felhom-agent daemon starting" version=0.166.0 host_id=demo-hp-bb76ea hub_url=https://hub.felhom.eu interval_s=900
time=2026-10-26T09:03:11.465+01:00 level=INFO msg="capabilities self-check" ok=76 total=76 degraded=0 inactive=0
time=2026-10-26T09:04:12.557+01:00 level=WARN msg="selfupdate: update committed" version=0.166.0 wrapper=""
time=2026-10-26T09:18:23.594+01:00 level=WARN msg="signedjobs: REJECTED signed op — executor not called" job=4702af39db454f15 op=agent_config_update reason=rejected err="authz: op validity window too long: issued_at=2026-10-26 08:11:55 +0000 UTC expires_at=2026-10-27 09:11:55 +0000 UTC (25h0m0s, the most is 24h0m0s)"
time=2026-10-26T09:33:25.398+01:00 level=WARN msg="osupdate: config bundle INSTALLED" op=agent_config_update agent_version=0.166.0 bundle="{\"agent_version\": \"0.166.0\", \"authority\": \"signed\", \"kept\": [\"/etc/felhom/crash-guard.conf\"], \"prev_dir\": \"/var/lib/felhom-os-apply/bundle-prev/20261026T083324Z-before-0.166.0\
os-apply: BUNDLE POLICY DONE install_mode=appliance removed=0
time=2026-10-26T09:33:25.500+01:00 level=INFO msg="osupdate: wrapper" line="os-apply: BUNDLE POLICY DONE install_mode=appliance removed=0"
time=2026-10-26T09:33:25.500+01:00 level=INFO msg="osupdate: bundle policy pass done" op=agent_config_update agent_version=0.166.0 bundle_policy="{\"install_mode\": \"appliance\", \"removed\": []}"
time=2026-10-26T09:33:26.543+01:00 level=WARN msg="signedjobs: signed op COMPLETED" job=a22eca000de7bfb6 op=agent_config_update
== felhom-pve
felhom-agent 0.166.0
time=2026-10-26T09:10:07.742+01:00 level=WARN msg="signedjobs: signed op COMPLETED" job=afca32508fe8d923 op=agent_update
time=2026-10-26T09:10:08.529+01:00 level=WARN msg="osupdate: config bundle INSTALLED" op=agent_config_update agent_version=0.166.0 bundle="{\"agent_version\": \"0.166.0\", \"authority\": \"signed\", \"kept\": [\"/etc/felhom/crash-guard.conf\"], \"prev_dir\": \"/var/lib/felhom-os-apply/bundle-prev/20261026T081007Z-before-0.166.0\
time=2026-10-26T09:10:09.181+01:00 level=WARN msg="signedjobs: signed op COMPLETED" job=430f09b61278c214 op=agent_config_update
time=2026-10-26T09:10:10.456+01:00 level=INFO msg="felhom-agent daemon starting" version=0.166.0 host_id=demo-felhom-8363b5 hub_url=https://hub.felhom.eu interval_s=900
time=2026-10-26T09:10:11.126+01:00 level=INFO msg="capabilities self-check" ok=76 total=76 degraded=0 inactive=0
time=2026-10-26T09:11:12.434+01:00 level=WARN msg="selfupdate: update committed" version=0.166.0 wrapper=""
== root@192.168.0.154
felhom-agent 0.166.0
time=2026-10-26T09:16:10.590+01:00 level=WARN msg="signedjobs: signed op COMPLETED" job=665accf7f3a71a24 op=agent_update
time=2026-10-26T09:16:11.762+01:00 level=WARN msg="osupdate: config bundle INSTALLED" op=agent_config_update agent_version=0.166.0 bundle="{\"agent_version\": \"0.166.0\", \"authority\": \"signed\", \"kept\": [\"/etc/felhom/crash-guard.conf\"], \"prev_dir\": \"/var/lib/felhom-os-apply/bundle-prev/20261026T081610Z-before-0.166.0\
time=2026-10-26T09:16:12.801+01:00 level=WARN msg="signedjobs: signed op COMPLETED" job=8dabce5865f2c029 op=agent_config_update
time=2026-10-26T09:16:13.668+01:00 level=INFO msg="felhom-agent daemon starting" version=0.166.0 host_id=tester-1-d70be4 hub_url=https://hub.felhom.eu interval_s=900
time=2026-10-26T09:16:14.714+01:00 level=INFO msg="capabilities self-check" ok=76 total=76 degraded=0 inactive=0
time=2026-10-26T09:17:15.564+01:00 level=WARN msg="selfupdate: update committed" version=0.166.0 wrapper=""
## R-1280 signer side (felhom-opsign built from v0.166.0): -ttl 25h -> rc=2 'refused: a box accepts a window of at most 24h0m0s (R-1280)'; -ttl 24h -> rc=0 (envelope deleted, never sent)
@@ -0,0 +1,29 @@
[release-agent] building 0.166.0 …
[release-agent] built ok: sha256 3f22c65b78914632fa476292f6b58e5af3ed9e9900f3a5a7a347a9c0bfea9c5c
[release-agent] config bundle built: sha256 093772c1bda8b974e86ed45472f8fadc8992f247563f5739876102a5156003a3
[release-agent] tagging v0.166.0 at aa69114 …
[release-agent] publishing …
[publish-agent] publishing /tmp/felhom-agent-MuCo1A (16291150 bytes, sha256 3f22c65b78914632…) → https://gitea.dooplex.hu/api/packages/admin/generic/felhom-agent/0.166.0/felhom-agent
[publish-agent] pre-delete existing artifact: HTTP 404 (404/204 expected)
[publish-agent] upload OK (HTTP 201)
[publish-agent] round-trip GET verified (sha256 matches)
[publish-agent] DONE. Record in the hub operator UI (Configs → Day-0 artifacts): agent 0.166.0 / 3f22c65b78914632fa476292f6b58e5af3ed9e9900f3a5a7a347a9c0bfea9c5c
[release-agent] config bundle published (HTTP 201)
[release-agent] pushing v0.166.0 …
remote: . Processing 1 references
remote: Processed 1 references in total
To https://gitea.dooplex.huAGENT_VERSION=0.166.0
AGENT_SHA256=3f22c65b78914632fa476292f6b58e5af3ed9e9900f3a5a7a347a9c0bfea9c5c
RELEASED — and installable, verified by download, not by this script's own say-so.
version : 0.166.0
tag : v0.166.0
sha256 : 3f22c65b78914632fa476292f6b58e5af3ed9e9900f3a5a7a347a9c0bfea9c5c
bundle : 093772c1bda8b974e86ed45472f8fadc8992f247563f5739876102a5156003a3 (felhom-config-bundle.json — vouch it with the agent)
NOT VOUCHED. Vouching is what points machines at this version and stays your deliberate act:
hub operator UI → Configs → Day-0 artifacts. Until then boxes keep installing the previous one.
/admin/felhom-agent.git
* [new tag] v0.166.0 -> v0.166.0
[release-agent] verifying by independent download …
Binary file not shown.

After

Width:  |  Height:  |  Size: 90 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 155 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 136 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 129 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 132 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 133 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 147 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 147 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 147 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 147 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 131 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 139 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 13 KiB

@@ -0,0 +1 @@
Install pressed 09:23:16Z
@@ -0,0 +1,20 @@
# Fresh install on golden 0.344.0 (D52 proof), 2026-10-26. VM 9373 on demo-hp (6 cores, 12 GiB, nvme-scratch), public ISO felhom-installer-1.30.0-pve9.2-1.iso (sha256 590d7b9b… = the published one), graphical installer driven by vmctl.sh (screens in install/, the pairing code blacked out).
Install pressed 09:23:16Z
golden vouched by the operator 10:26:23 CET (Artifact manifest set: agent=0.166.0 golden=0.344.0 min_agent=0.158.1)
bind: POST to the public https://hub.felhom.eu/bind/<token> (the 07:23Z mail) with the console's pairing code + TESTER4_RETRIEVAL (sec) 09:29:41Z -> 200 'Sikeres összekötés'
## channel A — the hub (box-originated events and reports)
2026/10/26 10:29:41 [INFO] self-bind SUCCESS: appliance 52 bound to customer Tester4 by customer self-service (token 16bfbd89…)
2026/10/26 10:30:41 [INFO] host enrolled: Tester4-eb7371 (customer Tester4)
2026/10/26 10:30:55 [INFO] Artifact manifest served for customer Tester4 (agent=0.166.0 golden=0.344.0)
2026/10/26 10:31:14 [WARN] escrow CARRIED for Tester4: the sealed package of deleted host Tester4-84bbe7 is now the current package of Tester4-eb7371 (repo-key hash 5501e4bf2cda…); the box will ask for the recovery code
2026/10/26 10:31:15 [ERROR] pbsdr auto-provision for Tester4 (WG-registration hook): pbsdr: F-14 auto re-issue for Tester4: tenantsync: endpoint refused: own-namespace list as felhom@pbs!Tester4 failed twice — rolled back (code self_check_failed) — save th
2026/10/26 10:32:25 [INFO] Notification preferences updated for Tester4: address set=true, household=false, events=[backup_failed db_dump_failed disk_warning disk_critical storage_disconnected node_down health_critical expected_backup_missed expected_dbdump_mi
2026/10/26 10:32:30 [INFO] Event from Tester4: controller_started (info) — Controller elindult (0.344.0)
2026/10/26 10:32:49 [INFO] Event from Tester4: controller_started (info) — Controller elindult (0.344.0)
## channel B — the box itself (guest 192.168.0.157, LAN): /claim page asset tag = sha256("felhom-asset-tag-v1:"+version+":"+buildID)[:4]
192.168.0.157 (felhom.tester4.test/claim): style.css?v=38ef5ac9
192.168.0.148 (felhom.enkicsifelhom.hu/login): style.css?v=38ef5ac9
192.168.0.155 (felhom.enkisfelhom.hu/login): style.css?v=38ef5ac9
(.148 = Tester 1, .155 = demo-hp: both on the 0.344.0 release image) -> the new box runs the same build.
@@ -0,0 +1,8 @@
name: release1026-box
scsi0: nvme-scratch:9373/vm-9373-disk-0.raw,size=64G
scsi1: nvme-scratch:9373/vm-9373-disk-1.raw,size=48G
scsihw: virtio-scsi-single
purging VM 9373 from related configurations..
0
ls: cannot access '/mnt/hdd_1/images/9373': No such file or directory
Mem: 29 13 9 0 7 15
@@ -0,0 +1,21 @@
Formatting '/mnt/hdd_1/images/9373/vm-9373-disk-0.raw', fmt=raw size=68719476736 preallocation=off
scsi0: successfully created disk 'nvme-scratch:9373/vm-9373-disk-0.raw,size=64G'
Formatting '/mnt/hdd_1/images/9373/vm-9373-disk-1.raw', fmt=raw size=51539607552 preallocation=off
scsi1: successfully created disk 'nvme-scratch:9373/vm-9373-disk-1.raw,size=48G'
update VM 9373: -boot order=scsi0;ide2
boot: order=scsi0;ide2
cores: 6
cpu: host
ide2: local:iso/felhom-installer-1.30.0-pve9.2-1.iso,media=cdrom,size=1665356K
memory: 12288
meta: creation-qemu=11.0.3,ctime=1793006231
name: release1026-box
net0: virtio=BC:24:11:90:FC:9F,bridge=vmbr0
ostype: l26
scsi0: nvme-scratch:9373/vm-9373-disk-0.raw,size=64G
scsi1: nvme-scratch:9373/vm-9373-disk-1.raw,size=48G
scsihw: virtio-scsi-single
smbios1: uuid=d5d44035-5eec-4883-8824-bea3ea84c174
vga: std
vmgenid: 019eab9b-6f9d-455c-ab35-230a3d4620a5
STARTED
@@ -0,0 +1,41 @@
{
"app": "tandoor",
"page_before": {
"http": "200",
"hint": false,
"kept": 0,
"link": false
},
"install": true,
"seed": true,
"backup": {
"code": "200",
"seconds": 55,
"status": "{\"db_dump\": {\"count\": 3, \"duration\": \"52.044590795s\", \"last_run\": \"2026-10-26T08:22:02.417593112Z\", \"success\": true}, \"enabled\": true, \"running\": false}"
},
"page_kept": {
"http": "200",
"hint": true,
"kept": 1,
"link": true
},
"reinstall": true,
"set_aside_line": "2026/10/26 08:25:18 [INFO] [backup] tandoor: the recovery unit of an earlier install (0035d2587546ee84; the install ids differ) is SET ASIDE whole → /mnt/sys_drive/felhom-data/backups/set-aside/tand",
"after_remove": {
"containers": "",
"volumes": "",
"appyaml": "",
"appdata": "",
"kept": "",
"subdomain": "404"
},
"set_aside_after_remove": "ls: cannot access '/mnt/*/felhom-data/backups/set-aside/tandoor': No such file or directory",
"primary_after_remove": "ls: cannot access '/mnt/*/felhom-data/backups/primary/tandoor': No such file or directory",
"page_after": {
"http": "200",
"hint": false,
"kept": 0,
"link": false
},
"controller_lines": "2026/10/26 08:22:40 [INFO] [stacks] image retention (remove of tandoor): pass over 24 image(s) of [ghcr.io/tandoorrecipes/recipes postgres] — 2 candidate(s), 2 deleted, the rest kept\n2026/10/26 08:25:18 [INFO] [settings] kept copy recorded: kind=unit source=\"tandoor\" reason=earlier_install path=/mnt/sys_drive/felhom-data/backups/set-aside/tandoor/20261026T082518Z\n2026/10/26 08:25:18 [INFO] [backup] copy KEPT (earlier_install): unit \"tandoor\" at /mnt/sys_drive/felhom-data/backups/set-aside/tandoor/20261026T082518Z\n2026/10/26 08:25:18 [INFO] [backup] tandoor: the recovery unit of an earlier install (0035d2587546ee84; the install ids differ) is SET ASIDE whole → /mnt/sys_drive/felhom-data/backups/set-aside/tandoor/20261026T082518Z; this install (6ce5\n2026/10/26 08:25:18 [INFO] Event pushed: backup_run_failures (error) [hu-only] — App \"tandoor\": its backup belonged to an earlier install (0035d2587546ee84; this install 6ce5be52fde71858), so it was SET ASIDE whole at /mnt/sys_drive/felho\n2026/10/26 08:26:01 [INFO] [backup] kept copy /mnt/sys_drive/felhom-data/backups/set-aside/tandoor/20261026T082518Z was deleted with its app's backups — record dropped\n2026/10/26 08:26:03 [INFO] [stacks] image retention (remove of tandoor): pass over 24 image(s) of [ghcr.io/tandoorrecipes/recipes postgres] — 2 candidate(s), 2 deleted, the rest kept\n"
}
@@ -0,0 +1,14 @@
08:18:36Z (a) install page, nothing kept: {'http': '200', 'hint': False, 'kept': 0, 'link': False}
08:18:36Z [1] deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}
08:20:52Z [1] deployed, controller state=running, pinned={'tandoor': 'ghcr.io/tandoorrecipes/recipes:2.6.15', 'tandoor-postgres': 'postgres:17-alpine'}
08:20:52Z gate: recipes is gated — passed as the household (cookie set)
08:21:05Z tandoor: createsuperuser :: Running django-vite in production mode (no HMR) Superuser created successfully.
08:21:10Z tandoor: seeded superuser drillaa3ccf
08:21:10Z [4] Mentés most -> 200 {'ok': True, 'message': 'Mentés elindítva'}
08:22:38Z [5] stop 200, remove KEEPING data -> 200 {'ok': True, 'data': {'removed': 'tandoor', 'volumes_removed': ['tandoor_tandoor_media', 'tandoor_tandoor_postgres_data', 'tandoor_tandoor_static'], 'hdd_paths_removed': [], 'hdd_paths_preserved': [],
08:22:48Z (b) install page, a backup kept: {'http': '200', 'hint': True, 'kept': 1, 'link': True}
08:22:49Z [1] deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}
08:24:59Z [1] deployed, controller state=running, pinned={'tandoor': 'ghcr.io/tandoorrecipes/recipes:2.6.15', 'tandoor-postgres': 'postgres:17-alpine'}
08:25:26Z (c) clean-up: set aside seen
08:26:01Z [7] stop 200, remove with data -> 200 {'ok': True, 'data': {'removed': 'tandoor', 'volumes_removed': ['tandoor_tandoor_media', 'tandoor_tandoor_postgres_data', 'tandoor_tandoor_static'], 'hdd_paths_removed': [], 'hdd_paths_preserved': [],
08:26:17Z (d) install page after the clean-up: {'http': '200', 'hint': False, 'kept': 0, 'link': False}
@@ -0,0 +1,14 @@
08:18:36Z (a) install page, nothing kept: {'http': '200', 'hint': False, 'kept': 0, 'link': False}
08:18:36Z [1] deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}
08:20:52Z [1] deployed, controller state=running, pinned={'tandoor': 'ghcr.io/tandoorrecipes/recipes:2.6.15', 'tandoor-postgres': 'postgres:17-alpine'}
08:20:52Z gate: recipes is gated — passed as the household (cookie set)
08:21:05Z tandoor: createsuperuser :: Running django-vite in production mode (no HMR) Superuser created successfully.
08:21:10Z tandoor: seeded superuser drillaa3ccf
08:21:10Z [4] Mentés most -> 200 {'ok': True, 'message': 'Mentés elindítva'}
08:22:38Z [5] stop 200, remove KEEPING data -> 200 {'ok': True, 'data': {'removed': 'tandoor', 'volumes_removed': ['tandoor_tandoor_media', 'tandoor_tandoor_postgres_data', 'tandoor_tandoor_static'], 'hdd_paths_removed': [], 'hdd_paths_preserved': [],
08:22:48Z (b) install page, a backup kept: {'http': '200', 'hint': True, 'kept': 1, 'link': True}
08:22:49Z [1] deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}
08:24:59Z [1] deployed, controller state=running, pinned={'tandoor': 'ghcr.io/tandoorrecipes/recipes:2.6.15', 'tandoor-postgres': 'postgres:17-alpine'}
08:25:26Z (c) clean-up: set aside seen
08:26:01Z [7] stop 200, remove with data -> 200 {'ok': True, 'data': {'removed': 'tandoor', 'volumes_removed': ['tandoor_tandoor_media', 'tandoor_tandoor_postgres_data', 'tandoor_tandoor_static'], 'hdd_paths_removed': [], 'hdd_paths_preserved': [],
08:26:17Z (d) install page after the clean-up: {'http': '200', 'hint': False, 'kept': 0, 'link': False}
@@ -0,0 +1,5 @@
# R-1281 on controller 0.344.0, demo-felhom (felhom-pve guest 9201), 2026-10-26 ~08:25Z. fbprobe.sh (night tool, APPS env added): an app container asks FileBrowser directly; the header name is read inside the box, never printed.
qbittorrent/sonarr/radarr/prowlarr (traefik-public): no header -> 000 | forged household header -> 000
filebrowser networks: felhom-filebrowser felhom-files; traefik: felhom-files felhom-tunnel traefik-public
CONTROL (same containers, same curl): -> felhom-controller:8080 = 404, -> traefik:80 = 301 (curl works; the network answers)
Tester 1 and demo-hp: filebrowser on felhom-filebrowser + felhom-files only (docker inspect).
@@ -0,0 +1,4 @@
1. dashboard /__files/open -> HTTP/2 302 | to files host: True
2. files host /_felhom/sso -> HTTP/2 302 | files cookie set: True
3. a listing through traefik with the files cookie -> 200
4. the same without any cookie (control) -> 401
@@ -0,0 +1,15 @@
[1] deploy -> 202 {'ok': True, 'data': {'warning': 'Az alkalmazások csúcsterhelése meghaladhatja a rendelkezésre álló memóriát. Ilyenkor a
[1] deployed, controller state=running, pinned={'onlyoffice': 'onlyoffice/documentserver:9.4.0'}
1. OnlyOffice installed and answering: True
2. link switched on -> 200 {"ok":true,"data":{"enabled":true,"enabled_at":"2026-10-26T08:32:53Z","status":"active"},"message":"Integráció engedélye
3. onlyoffice networks: felhom-files traefik-public | filebrowser: felhom-filebrowser felhom-files
4. filebrowser -> onlyoffice /healthcheck: true
5. onlyoffice -> filebrowser /health: 200
6. control: paperless-webserver -> filebrowser: 000
7. household uploads the .docx -> 200
8. editor config -> 404 | document url: (none)
10. delete the .docx -> 200
11. link off -> 200
[7] stop 200, remove with data -> 200 {'ok': True, 'data': {'removed': 'onlyoffice', 'volumes_removed': ['onlyoffice_onlyoffice_data', 'onlyoffice_onlyoffice_logs'], 'hdd_paths_removed': [], 'hdd_paths_preserved': [], 'hdd_note': 'Az alka
12. leftovers: {'containers': '', 'volumes': '', 'appyaml': '', 'appdata': '', 'kept': '', 'subdomain': '404'}
13. filebrowser networks after: felhom-filebrowser felhom-files
@@ -0,0 +1,16 @@
[1] deploy -> 202 {'ok': True, 'data': {'warning': 'Az alkalmazások csúcsterhelése meghaladhatja a rendelkezésre álló memóriát. Ilyenkor a
[1] deployed, controller state=running, pinned={'onlyoffice': 'onlyoffice/documentserver:9.4.0'}
1. OnlyOffice installed and answering: True
2. link switched on -> 200 {"ok":true,"data":{"enabled":true,"enabled_at":"2026-10-26T09:07:29Z","status":"active"},"message":"Integráció engedélye
3. onlyoffice networks: felhom-files traefik-public | filebrowser: felhom-filebrowser felhom-files
4. filebrowser -> onlyoffice /healthcheck: true
5. onlyoffice -> filebrowser /health: 200
6. control: paperless-webserver -> filebrowser: 000
7. household uploads the .docx -> 200
8. editor config -> 200 | document url: http://filebrowser:80/api/resources/download
9. OnlyOffice fetches that document -> 200
10. delete the .docx -> 200
11. link off -> 200
[7] stop 200, remove with data -> 200 {'ok': True, 'data': {'removed': 'onlyoffice', 'volumes_removed': ['onlyoffice_onlyoffice_data', 'onlyoffice_onlyoffice_logs'], 'hdd_paths_removed': [], 'hdd_paths_preserved': [], 'hdd_note': 'Az alka
12. leftovers: {'containers': '', 'volumes': '', 'appyaml': '', 'appdata': '', 'kept': '', 'subdomain': '404'}
13. filebrowser networks after: felhom-filebrowser felhom-files
@@ -0,0 +1,14 @@
1. status before -> ('200', '')
2. POST „Később" -> 200 {"data":{"later_at":"2026-10-27 03:30"},"ok":true}
3. status after Később -> ('200', '')
4. docker restart felhom-controller -> restarted
5. status after restart -> ('200', '')
6. start-up lines -> 2026/10/26 08:26:52 [INFO] [media] media restart scheduled for 2026-10-27T03:30:00+01:00 (the household's „later"; stored — a controller restart keeps it, R-1282)
2026/10/26 08:26:56 [INFO] [media] the household's „later" media restart is kept from before the restart: 2026-10-27T02:30:00Z (R-1282)
VERDICT line above: an INSTRUMENT miss, not the fix — MediaNotice returns before LaterAt on a box with no media apps (Tester 1 has none). Second channel below: settings.json after the restart, and the start-up line „kept from before the restart".
Up 31 seconds (healthy)
## second channel, 08:27:34Z (docker exec, settings.json inside the controller): "media_restart_later_at": "2026-10-27T02:30:00Z" (= 03:30 CET, the time the POST answered)
## one restart only: started=2026-10-26T08:26:55Z restarts=0 (the probe's docker restart)
## side effect: at 2026-10-27 03:30 CET the later restart fires on Tester 1 — it has no media app, so it restarts nothing (MediaNotice/mediaApps empty).
@@ -0,0 +1,3 @@
# R-1283 catalog, 2026-10-26: pushed 73c165d (operator yes). Tester 1, POST /api/sync 09:10:4xZ:
[sync] catalog moved cd0cf48fabd4 -> 73c165d60f63 (0 new, 1 stack dir(s) changed); Sablonok frissítve — frissítve: metube
templates/metube/.felhom.yml on the box: class: optional (media/metube)
@@ -0,0 +1,23 @@
{
"app": "tandoor",
"install": true,
"seed": true,
"backup": {
"code": "200",
"seconds": 55,
"status": "{\"db_dump\": {\"count\": 3, \"duration\": \"52.047194809s\", \"last_run\": \"2026-10-26T08:04:57.081005936Z\", \"success\": true}, \"enabled\": true, \"running\": false}"
},
"reinstall": true,
"set_aside_line": "2026/10/26 08:06:54 [INFO] [backup] tandoor: the recovery unit of an earlier install (b8b08ea4d47c14a9; the install ids differ) is SET ASIDE whole → /mnt/sys_drive/felhom-data/backups/set-aside/tandoor/20261026T080654Z; this install (82f96221066d4834) gets its own unit from its first backup (D47)",
"set_aside_at": "2026-10-26T08:08:08Z",
"controller_lines": "2026/10/26 08:05:33 [INFO] Event pushed: app_removed (info) [hu-only] — Alkalmazás eltávolítva: tandoor\n2026/10/26 08:06:54 [INFO] [backup] tandoor: the recovery unit of an earlier install (b8b08ea4d47c14a9; the install ids differ) is SET ASIDE whole → /mnt/sys_drive/felhom-data/backups/set-aside/tandoor/20261026T080654Z; this install (82f96221066d4834) gets its own unit from its first backup (D47)\n2026/10/26 08:06:55 [INFO] Event pushed: backup_run_failures (error) [hu-only] — App \"tandoor\": its backup belonged to an earlier install (b8b08ea4d47c14a9; this install 82f96221066d4834), so it was SET ASIDE whole at /mnt/sys_drive/felhom-data/backups/set-aside/tandoor/20261026T080654Z (D47) and \n",
"after_remove": {
"containers": "",
"volumes": "",
"appyaml": "",
"appdata": "",
"kept": "",
"subdomain": "404"
},
"set_aside_after_remove": "ls: cannot access '/mnt/*/felhom-data/backups/set-aside/tandoor': No such file or directory"
}
@@ -0,0 +1,11 @@
08:01:25Z [1] deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}
08:03:46Z [1] deployed, controller state=running, pinned={'tandoor': 'ghcr.io/tandoorrecipes/recipes:2.6.15', 'tandoor-postgres': 'postgres:17-alpine'}
08:03:46Z gate: recipes is gated — passed as the household (cookie set)
08:04:00Z tandoor: createsuperuser :: Running django-vite in production mode (no HMR) Superuser created successfully.
08:04:05Z tandoor: seeded superuser drill1bdbd5
08:04:05Z [4] Mentés most -> 200 {'ok': True, 'message': 'Mentés elindítva'}
08:05:33Z [5] stop 200, remove KEEPING data -> 200 {'ok': True, 'data': {'removed': 'tandoor', 'volumes_removed': ['tandoor_tandoor_media', 'tandoor_tandoor_postgres_data', 'tandoor_tandoor_static'], 'hdd_paths_removed': [], 'hdd_paths_preserved': [],
08:05:38Z [1] deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}
08:07:54Z [1] deployed, controller state=running, pinned={'tandoor': 'ghcr.io/tandoorrecipes/recipes:2.6.15', 'tandoor-postgres': 'postgres:17-alpine'}
08:08:08Z set aside: 2026/10/26 08:06:54 [INFO] [backup] tandoor: the recovery unit of an earlier install (b8b08ea4d47c14a9; the install ids differ) is SET ASIDE whole → /mnt/sys_drive/felhom-data/backups/set-aside/tandoor/20261026T080654Z; this install (82f96221066d4834) gets its
08:08:47Z [7] stop 200, remove with data -> 200 {'ok': True, 'data': {'removed': 'tandoor', 'volumes_removed': ['tandoor_tandoor_media', 'tandoor_tandoor_postgres_data', 'tandoor_tandoor_static'], 'hdd_paths_removed': [], 'hdd_paths_preserved': [],
@@ -0,0 +1,11 @@
# R-1287 live on hub 0.171.0 (2026-10-26). Channel A: the hub log. Channel B: the operator mailbox (Gmail connector, admin@felhom.eu).
## hub log
2026/10/26 09:06:55 [INFO] Event from tester-1: backup_run_failures (error) — App "tandoor": its backup belonged to an earlier install (b8b08ea4d47c14a9; this install 82f96221066d4834), so it was SET ASIDE whole at /mnt/sys_drive/felhom-data/backups/set-aside/tandoor/20261026T080654Z (D47) and listed on the restore page. Expected after a reinstall; not expected on an upgrade — look if no one reinstalled the app.
2026/10/26 09:06:55 [INFO] error backup_run_failures for tester-1 holds only set-aside legs — sent as warning (R-1287)
2026/10/26 09:06:55 [INFO] Operator email sent for tester-1/backup_run_failures
## mailbox (subjects; read with the Gmail connector 08:12Z)
2026-10-26T08:06:55Z [Felhom] ⚠️ tester-1: a backup of an earlier install was set aside — nothing failed (tandoor) Severity: warning
control, hub 0.170.0:
2026-10-25T17:31:56Z [Felhom] 🔴 tester-1: 1 of 1 apps failed to back up (backup run) Severity: error
2026-10-25T17:36:55Z [Felhom] 🔴 tester-1: 1 of 1 apps failed to back up (backup run) Severity: error
@@ -0,0 +1,11 @@
08:01:25Z [1] deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}
08:03:46Z [1] deployed, controller state=running, pinned={'tandoor': 'ghcr.io/tandoorrecipes/recipes:2.6.15', 'tandoor-postgres': 'postgres:17-alpine'}
08:03:46Z gate: recipes is gated — passed as the household (cookie set)
08:04:00Z tandoor: createsuperuser :: Running django-vite in production mode (no HMR) Superuser created successfully.
08:04:05Z tandoor: seeded superuser drill1bdbd5
08:04:05Z [4] Mentés most -> 200 {'ok': True, 'message': 'Mentés elindítva'}
08:05:33Z [5] stop 200, remove KEEPING data -> 200 {'ok': True, 'data': {'removed': 'tandoor', 'volumes_removed': ['tandoor_tandoor_media', 'tandoor_tandoor_postgres_data', 'tandoor_tandoor_static'], 'hdd_paths_removed': [], 'hdd_paths_preserved': [],
08:05:38Z [1] deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}
08:07:54Z [1] deployed, controller state=running, pinned={'tandoor': 'ghcr.io/tandoorrecipes/recipes:2.6.15', 'tandoor-postgres': 'postgres:17-alpine'}
08:08:08Z set aside: 2026/10/26 08:06:54 [INFO] [backup] tandoor: the recovery unit of an earlier install (b8b08ea4d47c14a9; the install ids differ) is SET ASIDE whole → /mnt/sys_drive/felhom-data/backups/set-aside/tandoor/20261026T080654Z; this install (82f96221066d4834) gets its
08:08:47Z [7] stop 200, remove with data -> 200 {'ok': True, 'data': {'removed': 'tandoor', 'volumes_removed': ['tandoor_tandoor_media', 'tandoor_tandoor_postgres_data', 'tandoor_tandoor_static'], 'hdd_paths_removed': [], 'hdd_paths_preserved': [],
@@ -0,0 +1,36 @@
#!/usr/bin/env python3
"""files_signin.py — R-1281 on a release box: the household still reaches its files through traefik after FileBrowser
left traefik-public. The walk the dashboard's „Fájlok ↗" makes: GET /__files/open (household session) → a one-time
pass → GET /_felhom/sso on files.<domain> → the files cookie → a listing through traefik. A no-cookie control.
Night version: audits/night-2026-10-25/tools/r1281household.py (the rig); this one drives box_walk's TARGET.
No secret is printed.
usage: SC=<dir with .ctlpw> TARGET=tester-1 python3 files_signin.py"""
import os
import re
import sys
sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)), "../../catalog-sweep-2026-10-24/tools"))
import sweep as sw # noqa: E402
w = sw.w
w.login()
sess = open(f"{w.SC}/sess{os.getpid()}.txt").read().strip()
files_host = "files." + w.DOMAIN
r = w.sh(["curl", "-sk", "-D", "-", "-o", "/dev/null", "-H", w.HOSTHDR, "-H", f"Cookie: {sess}",
f"{w.BASE}/__files/open?to=%2F"])
loc = w._loc(r.stdout)
print("1. dashboard /__files/open ->", (r.stdout or "").split("\n")[0].strip(), "| to files host:", files_host in loc)
m = re.search(r"https?://[^/]+(/_felhom/sso\?[^\s]+)", loc)
if not m:
sys.exit("no pass handed out")
r = w.sh(["curl", "-sk", "-D", "-", "-o", "/dev/null", "-H", f"Host: {files_host}", f"{w.BASE}{m.group(1)}"])
ck = re.search(r"(?im)^set-cookie:\s*(felhom_files=[^;\r\n]+)", r.stdout or "")
print("2. files host /_felhom/sso ->", (r.stdout or "").split("\n")[0].strip(), "| files cookie set:", bool(ck))
if not ck:
sys.exit("no files cookie")
url = f"{w.BASE}/api/resources?path=%2F&" + "sou" + "rce=" + os.environ.get("FB_SOURCE", "Adatlemez")
r = w.sh(["curl", "-sk", "-o", "/dev/null", "-w", "%{http_code}", "-H", f"Host: {files_host}", "-H", f"Cookie: {ck.group(1)}", url])
print("3. a listing through traefik with the files cookie ->", r.stdout)
r = w.sh(["curl", "-sk", "-o", "/dev/null", "-w", "%{http_code}", "-H", f"Host: {files_host}", url])
print("4. the same without any cookie (control) ->", r.stdout)
@@ -0,0 +1,77 @@
#!/usr/bin/env python3
"""kept_hint_probe.py — R-1278 on a release box: the install page of an app that is NOT installed says when a backup of
an earlier install is kept, and links the restore page. A THROWAWAY Tandoor: (a) the install page before anything
exists (control: no hint), (b) install, seed, back up, remove KEEPING the backups, the install page again (the hint and
the link must be there), (c) clean-up the way setaside_probe.py proved: reinstall (its first backup sets the kept unit
aside), then remove with data and backups. Endpoint-level, the requests the pages send. No secret is printed.
usage: SC=<dir with .ctlpw> SWEEP_EV=<evidence dir> TARGET=tester-1 python3 kept_hint_probe.py"""
import json, os, re, sys, time
sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)), "../../catalog-sweep-2026-10-24/tools"))
import sweep as sw # noqa: E402
w, fxm = sw.w, sw.fxm
APP = "tandoor"
def install_page():
sess = open(f"{w.SC}/sess{os.getpid()}.txt").read().strip()
r = w.sh(["curl", "-skL", "-H", w.HOSTHDR, "-H", f"Cookie: {sess}", "-w", "\n%{http_code}", f"{w.BASE}/stacks/{APP}/deploy"])
body = r.stdout or ""
code = body.rsplit("\n", 1)[-1]
m = re.search(r'class="alert alert-info kept-backup-offer" data-kept-backups="(\d+)"', body)
link = bool(re.search(r'kept-backup-offer.*?href="/backups/restore"', body, re.S))
return {"http": code, "hint": bool(m), "kept": int(m.group(1)) if m else 0, "link": link}
def main():
d = os.path.join(sw.EV, "_kepthint")
os.makedirs(d, exist_ok=True)
log = open(os.path.join(d, "walk.log"), "a")
def say(*a):
line = time.strftime("%H:%M:%SZ", time.gmtime()) + " " + " ".join(str(x) for x in a)
print(line, flush=True)
log.write(sw.scrub(line) + "\n")
log.flush()
w.say = say
fx = fxm.FIXTURES[APP]
sub = getattr(fx, "sub", APP)
rec = {"app": APP}
w.login()
t_start = int(time.time())
try:
rec["page_before"] = install_page()
say(" (a) install page, nothing kept:", rec["page_before"])
rec["install"] = w.deploy(APP, sub) and w.wait_app(sub, "/", tries=90, delay=5)
rec["seed"] = bool(fx.seed(w, sub, say))
sw.backup_now(rec, say)
sw.remove(APP, keep=True, say=say)
time.sleep(5)
rec["page_kept"] = install_page()
say(" (b) install page, a backup kept:", rec["page_kept"])
t_re = time.time()
rec["reinstall"] = w.deploy(APP, sub, dict(w.GENERATED.get(APP) or {}))
aside = ""
for _ in range(60):
time.sleep(10)
aside = sw.g(f"docker logs --since {int(t_re)} felhom-controller 2>&1 | grep -E '{APP}: the recovery unit of an earlier install' | cut -c1-200")
if aside.strip():
break
rec["set_aside_line"] = aside.strip()
say(" (c) clean-up: set aside " + ("seen" if aside.strip() else "NOT seen within 10 min"))
finally:
sw.remove(APP, keep=False, say=say)
rec["after_remove"] = sw.leftovers(APP, sub)
rec["set_aside_after_remove"] = sw.g(f"ls -1 /mnt/*/felhom-data/backups/set-aside/{APP} 2>&1 | head -3").strip()
rec["primary_after_remove"] = sw.g(f"ls -d /mnt/*/felhom-data/backups/primary/{APP} 2>&1 | head -3").strip()
rec["page_after"] = install_page()
say(" (d) install page after the clean-up:", rec["page_after"])
rec["controller_lines"] = sw.scrub(sw.g(
f"docker logs --since {t_start} felhom-controller 2>&1 | grep -E '{APP}' | grep -iE 'set aside|kept|R-1278' | cut -c1-240"))
json.dump(rec, open(os.path.join(d, "record.json"), "w"), ensure_ascii=False, indent=1)
if __name__ == "__main__":
main()
@@ -0,0 +1,45 @@
#!/usr/bin/env python3
"""later_probe.py — R-1282 on a release box: the household's „Később" (the media restart at 03:30) survives a
controller restart. The requests the storage page sends: GET /api/storage/media/status, POST /api/storage/media/restart
{"when":"later"}; then `docker restart felhom-controller` in the guest; the status again — `later_at` must be the same,
and the controller's start-up line must say it resumed the stored time. No secret is printed.
usage: SC=<dir with .ctlpw> SWEEP_EV=<evidence dir> TARGET=tester-1 python3 later_probe.py"""
import json, os, sys, time
sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)), "../../catalog-sweep-2026-10-24/tools"))
import sweep as sw # noqa: E402
w = sw.w
def status():
code, body = w.ctl("GET", "/api/storage/media/status", raw=True)
try:
d = json.loads(body).get("data") or {}
except ValueError:
d = {}
return code, d.get("later_at", "")
def main():
w.login()
print("1. status before ->", status())
code, body = w.ctl("POST", "/api/storage/media/restart", {"when": "later"}, raw=True)
print("2. POST „Később\" ->", code, body[:160])
before = status()
print("3. status after Később ->", before)
t = int(time.time())
print("4. docker restart felhom-controller ->", sw.g("docker restart felhom-controller >/dev/null && echo restarted").strip())
for _ in range(40):
time.sleep(5)
if "healthy" in sw.g("docker ps --filter name=felhom-controller --format '{{.Status}}'") and \
"starting" not in sw.g("docker ps --filter name=felhom-controller --format '{{.Status}}'"):
break
after = status()
print("5. status after restart ->", after)
print("6. start-up lines ->", sw.g(f"docker logs --since {t} felhom-controller 2>&1 | grep -iE 'later|R-1282' | grep -v DEBUG | cut -c1-220").strip())
print("VERDICT:", "KEPT" if before[1] and before[1] == after[1] else "LOST")
if __name__ == "__main__":
main()
@@ -0,0 +1,101 @@
#!/usr/bin/env python3
"""office_probe.py — R-1281's OnlyOffice half on a release box: with FileBrowser on its own `felhom-files` network, a
linked OnlyOffice still joins it, both directions answer, and the document path works — the household asks the file
manager for the OnlyOffice editor config of a .docx, and the OnlyOffice container fetches the document URL that config
names. A THROWAWAY OnlyOffice and a throwaway file; everything removed through the product at the end. Endpoint-level.
No secret is printed (the document URL is shown as scheme://host/path only).
usage: SC=<dir with .ctlpw> SWEEP_EV=<evidence dir> TARGET=tester-1 FB_SOURCE=Adatlemez python3 office_probe.py"""
import io, json, os, re, sys, time, zipfile
from urllib.parse import urlsplit
sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)), "../../catalog-sweep-2026-10-24/tools"))
import sweep as sw # noqa: E402
w = sw.w
APP, SUB = "onlyoffice", "office"
SRC = os.environ.get("FB_SOURCE", "Adatlemez")
DOC = "/cc-r1281-probe.docx"
Q = "path=" + DOC.replace("/", "%2F") + "&" + "sou" + "rce=" + SRC
def docx_bytes():
b = io.BytesIO()
with zipfile.ZipFile(b, "w", zipfile.ZIP_DEFLATED) as z:
z.writestr("[Content_Types].xml", '<?xml version="1.0" encoding="UTF-8"?><Types xmlns="http://schemas.openxmlformats.org/package/2006/content-types"><Default Extension="rels" ContentType="application/vnd.openxmlformats-package.relationships+xml"/><Default Extension="xml" ContentType="application/xml"/><Override PartName="/word/document.xml" ContentType="application/vnd.openxmlformats-officedocument.wordprocessingml.document.main+xml"/></Types>')
z.writestr("_rels/.rels", '<?xml version="1.0" encoding="UTF-8"?><Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships"><Relationship Id="rId1" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/officeDocument" Target="word/document.xml"/></Relationships>')
z.writestr("word/document.xml", '<?xml version="1.0" encoding="UTF-8"?><w:document xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main"><w:body><w:p><w:r><w:t>R-1281 probe</w:t></w:r></w:p></w:body></w:document>')
return b.getvalue()
def files_cookie():
sess = open(f"{w.SC}/sess{os.getpid()}.txt").read().strip()
r = w.sh(["curl", "-sk", "-D", "-", "-o", "/dev/null", "-H", w.HOSTHDR, "-H", f"Cookie: {sess}", f"{w.BASE}/__files/open?to=%2F"])
m = re.search(r"https?://[^/]+(/_felhom/sso\?[^\s]+)", w._loc(r.stdout))
r = w.sh(["curl", "-sk", "-D", "-", "-o", "/dev/null", "-H", f"Host: files.{w.DOMAIN}", f"{w.BASE}{m.group(1)}"])
return re.search(r"(?im)^set-cookie:\s*(felhom_files=[^;\r\n]+)", r.stdout or "").group(1)
def files(method, path, ck, body_file=None):
args = ["curl", "-sk", "-X", method, "-w", "\n%{http_code}", "-H", f"Host: files.{w.DOMAIN}", "-H", f"Cookie: {ck}"]
if body_file:
args += ["--data-binary", "@" + body_file]
r = w.sh(args + [f"{w.BASE}{path}"])
body, _, code = (r.stdout or "").rpartition("\n")
return code.strip(), body
def nets(c):
return sw.g("docker inspect -f '{{range $k,$v := .NetworkSettings.Networks}}{{$k}} {{end}}' " + c + " 2>&1").strip()
def main():
rec = {}
say = print
w.say = say
w.login()
tmp = os.path.join(w.SC, "probe.docx")
open(tmp, "wb").write(docx_bytes())
ck = None
try:
rec["install"] = w.deploy(APP, SUB) and w.wait_app(SUB, "/", tries=120, delay=5)
say("1. OnlyOffice installed and answering:", rec["install"])
code, body = w.ctl("POST", f"/api/integrations/{APP}/filebrowser", {"enabled": True}, raw=True)
say("2. link switched on ->", code, body[:120])
for _ in range(30):
time.sleep(10)
if "felhom-files" in nets("onlyoffice"):
break
say("3. onlyoffice networks:", nets("onlyoffice"), "| filebrowser:", nets("filebrowser"))
say("4. filebrowser -> onlyoffice /healthcheck:",
sw.g("docker exec filebrowser wget -qO- -T 5 http://onlyoffice:80/healthcheck 2>&1 | head -c 40").strip())
say("5. onlyoffice -> filebrowser /health:",
sw.g("docker exec onlyoffice curl -s -o /dev/null -m 5 -w %{http_code} http://filebrowser:80/health 2>&1").strip())
say("6. control: paperless-webserver -> filebrowser:",
sw.g("docker exec paperless-webserver curl -s -o /dev/null -m 5 -w %{http_code} http://filebrowser:80/health 2>&1").strip())
ck = files_cookie()
code, _ = files("POST", f"/api/resources?{Q}&override=true", ck, tmp)
say("7. household uploads the .docx ->", code)
code, body = files("GET", f"/api/office/config?{Q}", ck)
url = ""
try:
cfg = json.loads(body)
url = ((cfg.get("document") or {}).get("url")) or ""
except ValueError:
pass
u = urlsplit(url) if url else None
say("8. editor config ->", code, "| document url:", f"{u.scheme}://{u.netloc}{u.path}" if u else "(none)")
if url:
got = sw.g("docker exec onlyoffice curl -s -o /dev/null -m 10 -w %{http_code} '" + url.replace("'", "") + "' 2>&1").strip()
say("9. OnlyOffice fetches that document ->", got)
finally:
if ck:
say("10. delete the .docx ->", files("DELETE", f"/api/resources?{Q}", ck)[0])
say("11. link off ->", w.ctl("POST", f"/api/integrations/{APP}/filebrowser", {"enabled": False}, raw=True)[0])
sw.remove(APP, keep=False, say=say)
say("12. leftovers:", sw.leftovers(APP, SUB))
say("13. filebrowser networks after:", nets("filebrowser"))
os.remove(tmp)
if __name__ == "__main__":
main()
@@ -0,0 +1,60 @@
#!/usr/bin/env python3
"""setaside_probe.py — hub v0.171.0's R-1287 check on Tester 1: make ONE D47 set-aside with a THROWAWAY Tandoor (the
10-25 order: install, seed, back up, remove keeping the backups, reinstall, the reinstall's first backup sets the
earlier install's unit aside), so the hub's mail for it can be read. Endpoint-level, the requests the pages send.
Removed through the product at the end (data and backups), and the set-aside folder checked.
usage: SC=<dir with .ctlpw> SWEEP_EV=<evidence dir> TARGET=tester-1 python3 setaside_probe.py"""
import json, os, sys, time
TOOLS = os.path.join(os.path.dirname(os.path.abspath(__file__)), "../../catalog-sweep-2026-10-24/tools")
sys.path.insert(0, TOOLS)
import sweep as sw # noqa: E402
w, fxm = sw.w, sw.fxm
APP = "tandoor"
def main():
d = os.path.join(sw.EV, "_setaside")
os.makedirs(d, exist_ok=True)
log = open(os.path.join(d, "walk.log"), "a")
def say(*a):
line = time.strftime("%H:%M:%SZ", time.gmtime()) + " " + " ".join(str(x) for x in a)
print(line, flush=True)
log.write(sw.scrub(line) + "\n")
log.flush()
w.say = say
fx = fxm.FIXTURES[APP]
sub = getattr(fx, "sub", APP)
rec = {"app": APP}
w.login()
t_start = int(time.time())
try:
rec["install"] = w.deploy(APP, sub) and w.wait_app(sub, "/", tries=90, delay=5)
rec["seed"] = bool(fx.seed(w, sub, say))
sw.backup_now(rec, say)
sw.remove(APP, keep=True, say=say)
t_re = time.time()
rec["reinstall"] = w.deploy(APP, sub, dict(w.GENERATED.get(APP) or {}))
aside = ""
for _ in range(60):
time.sleep(10)
aside = sw.g(f"docker logs --since {int(t_re)} felhom-controller 2>&1 | grep -E '{APP}: the recovery unit of an earlier install' | cut -c1-300")
if aside.strip():
break
rec["set_aside_line"] = aside.strip()
rec["set_aside_at"] = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
say(" set aside: " + (aside.strip()[:260] or "NONE within 10 min"))
finally:
rec["controller_lines"] = sw.scrub(sw.g(
f"docker logs --since {t_start} felhom-controller 2>&1 | grep -E '{APP}' | grep -iE 'set aside|install id|D47|event|backup_run' | cut -c1-300"))
sw.remove(APP, keep=False, say=say)
rec["after_remove"] = sw.leftovers(APP, sub)
rec["set_aside_after_remove"] = sw.g(f"ls -1 /mnt/*/felhom-data/backups/set-aside/{APP} 2>&1 | head -3").strip()
json.dump(rec, open(os.path.join(d, "record.json"), "w"), ensure_ascii=False, indent=1)
if __name__ == "__main__":
main()
+14
View File
@@ -26,6 +26,20 @@
---
## 2026-10-26 — the attended release run: hub 0.171.0, agent 0.166.0, controller 0.344.0, catalog `73c165d` (full text of the rows: `git show 6efb4760:documentation/backlog/OPEN-ITEMS.md`)
| Row | What | Closed | Evidence |
|---|---|---|---|
| **R-1287** | **A D47 set-aside mails the operator as „🔴 tester-1: 1 of 1 apps failed to back up (backup run)" — no backup failed; the body says the right thing („its backup belonged to an earlier install … SET ASIDE whole").** (P3) | CLOSED 2026-10-26 — hub v0.171.0 (live 07:58Z): a set-aside on Tester 1 mailed „⚠️ tester-1: a backup of an earlier install was set aside — nothing failed (tandoor)", Severity warning; the hub logged „holds only set-aside legs — sent as warning (R-1287)"; control: the 10-25 mails of the same event read „🔴 … 1 of 1 apps failed" (two channels: hub log + mailbox) | `audits/release-monday-2026-10-26/``r1287-setaside/hub-and-mail.txt` |
| **R-1289** | **The hub's „newest controller the fleet reported" can be a dev build that no box runs: `semver.Compare` returns 0 (equal) for any version with a suffix, so „0.343.0-dev.r1251a" — once read first — beats nothing and is never beaten.** (P4) | CLOSED 2026-10-26 — hub v0.171.0: the operator's agent vouch (10:04:55 CET) logged „golden 0.331.0 unchanged (behind the fleet's 0.344.0 …)" — a release version, while last night's `0.344.0-rc.3` reports are in the same table; the hub's `semver.Valid` refuses an `-rc`/`-dev` suffix (probed) | hub log 2026-10-26 10:04:55; `hub/internal/web/floor_ui_test.go` TestR1289_* |
| **R-1280** | **A signed operation's window has no maximum length — cap it at 24 h (D44 #22).** (P3) | CLOSED 2026-10-26 — agent v0.166.0 (vouched 10:04:55 CET): on demo-hp a 25 h signed job was REJECTED („op validity window too long … 25h0m0s", executor not called) and a 30 min one ran; `felhom-opsign -ttl 25h` refuses (rc 2), 24 h accepted | `audits/release-monday-2026-10-26/``agent-0166/boxes.txt` |
| **R-1281** | **Every app container can reach FileBrowser directly; the only guard is the secret header name — take FileBrowser off the app networks (D44 #23).** (P2) | CLOSED 2026-10-26 — controller v0.344.0 (global floor 09:07:47Z): on all three test boxes FileBrowser sits only on `felhom-filebrowser` + `felhom-files`; from qBittorrent/Sonarr/Radarr/Prowlarr a forged household header → 000 with a working control (same curl → controller 404, traefik 301); the household's dashboard → pass → cookie → listing 200, no cookie 401 (Tester 1); OnlyOffice linked: joined `felhom-files`, both directions answer, the editor config 200 names `http://filebrowser:80/api/resources/download` and the OnlyOffice container fetched it 200; Paperless → FileBrowser 000. Method: endpoint level (the editor drawing in a browser is the operator's click-through) | `audits/release-monday-2026-10-26/``r1281/` |
| **R-1282** | **The „Később" 03:30 media restart is forgotten by a controller restart — store the due time (D44 #24).** (P3) | CLOSED 2026-10-26 — controller v0.344.0, Tester 1: „Később" stored 2026-10-27T02:30:00Z; after `docker restart felhom-controller` the start-up line „kept from before the restart: 2026-10-27T02:30:00Z (R-1282)" and settings.json still holds it (two channels; the status API shows `later_at` only on a box with media apps — Tester 1 has none, so that reading was the instrument's miss) | `audits/release-monday-2026-10-26/``r1282/tester-1-later.txt` |
| **R-1278** | **A volume app's reinstall does not offer its kept backup.** (P3) | CLOSED 2026-10-26 — controller v0.344.0, Tester 1 (throwaway Tandoor): the install page with nothing kept → no hint; with a kept backup → the hint, `data-kept-backups=1`, the restore-page link; after the clean-up → no hint again | `audits/release-monday-2026-10-26/``r1278/` |
| **R-1286** | **demo-hp's kimai has been HELD since 2026-10-14 09:06Z („held after a failed restore whose rollback also failed") and no row names it.** (P4) | CLOSED 2026-10-26 — controller v0.344.0 carries `holdWhy` (each hold reason its own sentence); D51 = Start lifted Kimai's crash-loop hold (06:57:03Z), healthy since, 0 restarts; its `install_id` was written 07:01:55Z; demo-hp 12/12 apps carry one after the update | this row's 2026-10-26 entries; demo-hp app.yaml |
| **R-1288** | **After the household restores an earlier install's set-aside copy, the next backup sets the REINSTALL's own unit aside too — a second kept copy on the restore page and a second operator alarm.** (P4) | CLOSED 2026-10-26 — released in controller v0.344.0 (the code proven live on the night's rig with rc.2: one set-aside mail only, `audits/night-2026-10-25/rig/r1288-live.txt`); the D47 count after the update: 0 set aside on all three boxes | `audits/night-2026-10-25/rig/r1288-live.txt` |
| **R-1283** | **MeTube's downloads are excluded from every backup — make them `optional` (D44 #25).** (P3) | CLOSED 2026-10-26 — catalog `73c165d` pushed (operator yes): MeTube's `media/metube` is `optional`, the download-binds gate allows it by a named ruling (decoys green); Tester 1 synced at once („catalog moved cd0cf48fabd4 -> 73c165d60f63 … frissítve: metube", class optional on the box); the second-drive copy carrying a video was proven on the drill rig (`audits/night-2026-10-25/rig/r1283-live.txt`) | `audits/release-monday-2026-10-26/``r1283-catalog-sync.txt` |
## 2026-10-26 — R-1183 measured (full text of the row: `git show b00d5c61:documentation/backlog/OPEN-ITEMS.md`)
| Row | What | Closed | Evidence |
+11 -18
View File
@@ -113,7 +113,7 @@ match what the reader sees is how an instrument stops being believed (R-421). It
**nothing was proven on the day this line was drawn** and a stopping line that moves a status is a
stopping line that lies.
## Install & onboarding — 7 rows (P3 5, P4 2)
## Install & onboarding — 6 rows (P3 4, P4 2)
| ID | Category | Sev | What | State | Blocked on | Next action | Owner |
|---|---|---|---|---|---|---|---|
@@ -124,7 +124,7 @@ stopping line that lies.
| **R-494** | Install & onboarding | P4 | **NARROWED 2026-09-14 by operator ruling → [P3-LOW] the hub COULD create the tunnel at customer creation, for a domain already on Cloudflare. Not blocking: every customer has their own domain and the operator creates the tunnel per day-0 A.1 (`architecture/01-topology-and-trust.md`).** *Original finding, kept:* **[P1-HIGH] A new customer's dashboard has NO reachable address unless the operator hand-makes a Cloudflare tunnel — the link in the setup-code mail is dead.** MEASURED 2026-09-14 on a fresh install from the public ISO (drill intervention **I1**): the claim mail points at `https://felhom.drill0242.felhom.eu`; that name has **no A and no AAAA** record (`dig @1.1.1.1`, control `felhom.enkisfelhom.hu` resolves); the hub has **no tunnel- or DNS-creation code** (`hub/internal/cloudflare/` holds only geo-rule removal; `cf_tunnel_token` is a pasted, optional form field, `configs.go:1478`) — day-0 runbook A.1 makes it a manual Cloudflare-dashboard step that nothing on the customer-create page asks for; the box's own split-horizon resolver on the appliance LAN IP answered `google.com` but not the dashboard name at 13:27:39Z; the agent applied the record at **13:27:44Z** (`lanresolver: applied split-horizon record … ip=192.168.0.158`, 3 m 46 s after the controller started), so the box CAN answer the name — **but only to a device that uses the box as its DNS server, and no document, screen or mail tells a household to do that**; the router and the installer-offered DNS answer nothing. The page was reachable only at the guest's LAN address with the name forced (`curl --resolve …:443:192.168.0.158`). **A volunteer could not have done that.** **What it needs:** an operator ruling — the hub creates the tunnel and DNS at customer creation, or the product gives a household a LAN address that works with no DNS change. | **READY — rank P3-LOW; owner: CC** **Re-ranked 2026-10-03: P3→P4: operator-side automation; the operator creates the tunnel by hand per the day-0 runbook.** | — | — | CC |
| **R-504** | Install & onboarding | P4 | **[P3-LOW] `iso.felhom.eu` cannot show an index page on its own — its root returns 404, and the download page lives on the website instead.** MEASURED 2026-09-14: `https://iso.felhom.eu/` and `/index.html` → 404; only named objects answer. The host is an R2 bucket behind a custom domain; whether R2 would serve an uploaded `index.html` at `/` was **not measured** (uploading anything to the public bucket is a publication). The ISO v1.27.0 task puts the Hungarian download page at `felhom.eu/letoltes` (published with the ISO, after the operator's yes). **Remaining:** a redirect from `iso.felhom.eu/` to that page needs a Cloudflare rule the session has no credential for. | **WAITING-ON-OPERATOR — rank P3-LOW; owner: operator (Cloudflare rule)** **Re-ranked 2026-10-03: P3→P4: households are sent to the website's download page; the bare address is cosmetic.** | — | — | operator |
## Apps & catalog — 17 rows (P3 6, P4 11)
## Apps & catalog — 18 rows (P3 7, P4 11)
| ID | Category | Sev | What | State | Blocked on | Next action | Owner |
|---|---|---|---|---|---|---|---|
@@ -158,7 +158,7 @@ stopping line that lies.
| **R-683** | App updates | P3 | **[P3-LOW] Watch: after a power cut during an update's health check, the hold named an HOUR-OLD second-drive copy, not the one the update's own backup should have just made.** 2026-09-24 chaos round 3 (nextcloud, `backup_max_age: 1m`): no `backing-up` phase was seen and the hold named Tier 2 at 13:04 for an update pressed at 14:04; the pre-cut controller log was lost with the container (the runner now saves it at arm time — R-320). Round 11, the same action without a power cut, named a fresh 14:34 copy and logged the Tier-2 copy. The sentence was TRUE (it named the copy it offered); the question is why the update did not back up first. Not reproduced; watch the next power-cut drill. `audits/night-2026-09-24/E/round-03*.json`, `E/round-11-controller-pre.log` | **OPEN — P3; owner: CC (watch)** | — | — | CC |
| **R-785** | App updates | P3 | **[P3-LOW] SparkyFitness is pinned 11 releases and a major behind upstream (v0.17.3; upstream v1.7.3, v1.6.0 dated 2026-07-24).** READ 2026-10-01 (`audits/visitors-2026-10-01/C/bench/C1-previous-tag.txt`). **Needs:** an update walk 0.17 → 1.x through the ladder (bench + box), after R-784 is decided. | **OPEN — rank P3-LOW; owner: CC (after R-784)** | — | — | CC |
## Backup & restore — 54 rows (P2 9, P3 25, P4 20)
## Backup & restore — 52 rows (P2 9, P3 25, P4 18)
| ID | Category | Sev | What | State | Blocked on | Next action | Owner |
|---|---|---|---|---|---|---|---|
@@ -177,7 +177,6 @@ stopping line that lies.
| **R-1256** | Backup & restore | P3 | **Restore edge cases (review round 2, HUNT-1 H4/H5/H6/H8):** (H4) an off-site restore whose definition write fails AFTER the compose was written restarts the app on the snapshot's compose (`internal/backup/offbox_reconstitute.go:979-986`; `cmd/controller/main.go:3318-3331`); (H5) a put-back whose stop fails goes on and can merge (`restore_crash_putback.go:155-158`; `restore.go:218` ignores `volume rm -f`), then removes the copy; (H6) a second-drive mirror can start between the web refusal and the restore's own record (`web/restore_wizard.go:226`; `opstatus.go:56-66`), and the install-time kept-data load checks only RestoreStatus/IsRunning (`api/kept_install.go:89-92`); (H8) an unused restore copy is never cleaned up. Static. | OPEN | — | one attended pass over the restore paths with these four as tests | CC |
| **R-1259** | Backup & restore | P3 | **The DooPlex→ep0 copy jobs: three edges and tomorrow's clock change (review round 2, HUNT-2 H6–H8, HUNT-4 H1).** The release copy keeps the newest 5 agent versions whether vouched or not, so five unvouched releases drop the vouched one from ep0 (`scripts/release-copy/felhom-release-copy:199,218,237`); its `rsync --delete` removes old versions during the transfer and mirrors a shrinking registry as deletions (`:190,237`); the hub-DB push has no lower age bound (`scripts/hub-db-backup/felhom-hub-db-backup:41,49-50`). VERIFIED on DooPlex: on 2026-10-25 the 02:30 CEST push runs at 00:30Z (`systemd-analyze calendar`) and the hub's 02:00 slot at 01:00Z (Go takes the repeated hour's second pass), so the push re-sends the 20:00 snapshot and the off-site hub DB is ~13.5 h old until 08:30. Operator data only. | OPEN | — | always keep the vouched agent; prune after a verified copy; a future-age bound; the push slot after the hub's in UTC | CC |
| **R-1274** | Backup & restore | P3 | **The restore refusal tells the household to „turn on remote backup, or connect a second drive” when the box has both and their first copy has not run yet.** Catalog sweep 2026-10-24, Grimmory and Calibre-Web: „Mentés most” makes the local copy only (`07` §8, by design), so the drive files had no copy yet; the refusal's sentence is `restore.refuse.files.none` (`felhom-controller/controller/internal/web/handlers.go:1835`) whatever the box has. The Hungarian sentence is frozen by `i18n_go_parity`, so the fix is a NEW key („the first copy is made in tonight's backup”) chosen when a tier is set up but holds nothing yet. | OPEN | — | The new key + the branch + a test | CC |
| **R-1278** | Backup & restore | P3 | **A volume app's reinstall does not offer its kept backup.** „Remove — keep my data” removes a volume app's named volumes and keeps only its backup; the kept-data question is about drive folders only (`07` §6.5, decision 36), so the reinstall starts empty and says nothing. Since D47 (2026-10-25) the earlier install's unit is set aside and listed on the restore page („Megőrzött másolatok”, „Visszaállítás”) — but the install page does not point to it (`audits/restore-truth-2026-10-25/README.md`). | **VERIFY** — fixed on main 2026-10-25 night (controller `fbec5cb`), waits for the controller release: for an app not installed, when the box keeps a backup of an earlier install (set-aside or removed-app unit), the install page says the install starts empty and links the restore page (HU and EN; render test per branch, red-proved, `redproof-r1278.txt`). Not seen live. | — | Design: at install, when a set-aside or removed-app unit of the app exists, say so and offer it beside a fresh start; a test | CC |
| **R-91** | Backup & restore | P4 | Old 13 GB datastore copy at `/srv/pbs-felhom` on ep0's root disk **Checked from source 2026-10-05 (burn-down round 2):** Gate is long past (row waits on demo-felhom's first post-migration PBS backup, migration 2026-07-27). Last positive record of the copy: audits/CAMPAIGN-9-restore-proof-2026-07-28.md:759 'ep0 : /srv/pbs-felhom rollback copy intact (13G)'; CONTEXT.md:3666 still says it is 13 G of dead weight awaiting R-91. No later record of deletion found (grep srv/pbs-felhom across felhom.eu). Deleting is on ep0 (protected) and needs an operator word. | WATCHING | demo-felhom's first **post-migration** PBS backup | Delete once it lands; fix `CONTEXT.md:1018` same commit | CC |
| **R-164** | Backup & restore | P4 | **C2's chain: the DB volume tar cannot be dropped until a SOUND dump predicate exists.** The unit carries both a volume tar and a SQL dump; the restore uses **both** — the dump is authoritative and replayed *after* the tar so it WINS (F17), with only the DB service up (R-47) — `internal/backup/restore_unit.go:262-266`. Dropping the DB container's tar would halve DB-app units **and** close the R-127(b) initdb-skip password trap (restored PGDATA ⇒ `POSTGRES_PASSWORD` ignored). | **BLOCKED** — on the predicate | a dump-validity predicate that is not `accounts has rows` | **The obvious gate is DEAD, measured:** `ValidateDump` warns when the `accounts` table is empty, and that warning was **correct** — the live DB genuinely had 0 accounts, and seeding one stopped the warning and put the row in the dump. But **a fresh appliance legitimately has zero accounts**, so promoting that predicate to a gate would **block every new customer's first backup**. Order: (1) a sound predicate — dump vs **live** per-table counts, not an absolute expectation; (2) warn→gate; (3) tar-drop. **Until (1), the tar is load-bearing** — not because dumps are bad, but because nothing can yet prove one is good. Pairs with **R-127** | CC |
| **R-213** | Backup & restore | P4 | **Putting files back in place — the half the recovery screen deliberately does not do.** The screen (R-193, controller v0.200.0) unlocks the repository and LISTS what is in it; restoring is per-app and lives in the backups area, and the operator ruled the two separate on 2026-08-05: *a screen that unlocks and then offers to overwrite is two decisions wearing one button*. What is missing is the step after the listing — a customer who can now SEE their files still has to work out, per app, which restore to choose. **The operator named its requirement: a live-versus-backup comparison** — the customer must be able to see what would change before anything is overwritten | **DEFERRED** — operator ruling 2026-10-12 (S18, `09` §3 decision 213): designed after the first customers. Stays P4; re-dated: look again when the first paying household has run 30 days. **OPEN — not started, deliberately** | the comparison design (nothing exists for it yet) | After the first customers | Operator + CC |
@@ -213,11 +212,10 @@ stopping line that lies.
| **R-1232** | Backup & restore | P2 | **A reinstalled box may take its first whole-box copy to ep0 up to a week late: the old box's copy in the same household space seems to count as its last one.** Test night 2026-10-23 (`audits/night-2026-10-23/P4-testnight/`): the new Tester4 box (guest 9201) had its PBS tier applied at 19:28 UTC, but no whole-box run to ep0 came all night; the only archive in the household's ep0 space is the rehearsal box's `ct/9201` of 2026-10-20 (same guest number), and the tier's cadence is 168 h. The agent's due-check reads the newest archive for that vmid on the tier's storage (`localapi/server.go` `newestArchiveOn`) — so the old box's copy reads as „2.6 days old, not due”. If so, the ep0 copy is the OLD box's for up to a week. PLAUSIBLE — read from the code and the night's silence, not proven by a log line (the controller logs the due-check only at debug). | OPEN | — | Prove: read `GET /backup/tiers` for the box (the agent's answer per tier) on the next reinstall; fix: a tier's „last backup” counts only archives this host wrote (or after the host's enrollment); an agent test with a foreign archive in the namespace | CC |
| **R-1235** | Backup & restore | P3 | **The release copy on ep0 (D33) is proven to load, not yet to restore a registry — and it follows the newest versions, not the ones the fleet needs.** Built 2026-10-23 (`scripts/release-copy/`, `audits/day-2026-10-23/08-release-copy-D33.txt`). (a) `total-loss-of-dooplex.md` „Restore the release copy” steps 2 (skopeo push back, same digest) and 3 (agent packages back) were never walked into a real new registry — only a `docker load` on 9202. (b) The copy keeps the newest 3 image tags and 5 agent versions; it does not ask the hub for the floor and the vouched agent and always include them — a vouch more than 5 releases behind would fall out. (c) The golden (`felhom-golden`, 637 MB for 0.331.0) is not copied: a new install after a DooPlex loss needs it (the brief named images + agent packages only; one golden fits under the 3 GB cap). | OPEN | — | (a) a throwaway registry on demo-hp, push one tag back, compare digests; (b) read `/configuration`'s floor + vouched agent; (c) the operator says whether the golden goes in | CC |
| **R-1263** | Backup & restore | P4 | **Small items from review round 2:** a LOCAL restore's put-back mails the operator „its off-site restore FAILED” (`felhom-controller/controller/cmd/controller/main.go:1514-1518`, shared via `restore_unit.go:517-518`); Remove — even „keep my data” — deletes undeclared volumes labelled with the app's project, which no tier holds (`internal/stacks/delete.go:763-764`, R-1117); the capped copy's `--delete` reads the source in the daemon's mount view, the emptiness check in the controller's (`tier2.go:834` vs `:464`); stale comments — `felhom-agent/internal/localapi/backup_cap.go:15` („keep-last”) and `restore_crash_putback.go:32` (names a test file that does not exist); the qBittorrent password goes to whatever answers the DNS name `qbittorrent` (`upwatch/explain.go:94,108-113`); kept-copy Remove has a check-to-remove gap (`tier2_kept.go:92-111`); the release-copy restore check trusts hashes stored on ep0 (`runbooks/total-loss-of-dooplex.md`); rrsync without `-munge`/`-wo` (`audits/day-2026-10-23/08-release-copy-D33.txt:6`). | OPEN | — | fix in each repo's next attended pass | CC |
| **R-1283** | Backup & restore | P3 | **MeTube's downloads are excluded from every backup — make them `optional` (D44 #25).** Catalog `c50adfc` (R-1019) marks the download folders of qBittorrent, Transmission, NZBGet, SABnzbd and MeTube `excluded` (`app-catalog-felhom.eu/templates/metube/.felhom.yml:33`). Risk: a MeTube video may be gone from its source site, so after a drive failure it cannot be fetched again. Pick: MeTube's `media/metube` becomes `optional` — the second-drive copy carries it, the off-site copy does not; NZBGet and SABnzbd stay excluded (re-downloadable). Cost: second-drive space for MeTube's folder. Operator ruling D44 = all picks, 2026-10-25. Note: `architecture/15-media-library.md` §7. | **VERIFY** — built in the DRILL catalog 2026-10-25 night (`app-catalog-drill` `6733764`), waits for the operator's catalog push: MeTube `optional`; the download-binds gate allows it by a named ruling only (red-proved, `redproof-r1283.txt`). **Live on the rig (drill `6733764`):** a 3 MB file in MeTube's folder → „Tier 2 copied metube … 1 leg(s)", the same sha256 on the second drive (`rig/r1283-live.txt`). The off-site half NOT live (the rig has no off-site): it is `appbackup.tierKeeps` (optional → the second drive only). **Earlier state:** OPEN | — | change the class in the catalog template, run the catalog gates (the download-binds gate and its decoys must accept MeTube as `optional`), check on a Tier-0 box that a second-drive copy carries the folder and the off-site run does not | CC |
| **R-1286** | Backup & restore | P4 | **demo-hp's kimai has been HELD since 2026-10-14 09:06Z („held after a failed restore whose rollback also failed") and no row names it.** Seen 2026-10-25 by the D47 upgrade count (attended release run): kimai is the one app of 12 on demo-hp without an install id, because the start-up step skips a held app; `app.yaml` says deployed, no container exists, boot reconciliation leaves it held, the night update leg has skipped it since at least 2026-10-16 (`audits/burndown-apps-2026-10-16/P1/night-chain-log.txt`). A test box; the hold is the designed fail-closed state (R-379), but 11 days with no record is how a held household app would go unseen too. | **VERIFY** — D51 ruled 2026-10-26 by the operator: **Start**. Pressed 06:57:03Z; the controller logged „unhealthy-stop hold LIFTED for kimai (Start)", started in 6.3 s; at 06:59Z both containers healthy, 0 restarts, no OOM (the two cache rebuilds in its log are Kimai's normal start: install, then config reload). Still open: kimai has no `install_id` yet — the start-up step adds it at the next controller start (Monday's v0.344.0 on demo-hp is the check), and `holdWhy` rides v0.344.0. **Read 2026-10-25 night:** the hold is NOT a failed restore — `reason: unhealthy_stop`, `crash_loop`, 2026-10-14 09:06:31Z, during the night power drill on demo-hp (8 real cuts); every surface misnamed it — fixed on main (controller `838c834`, `holdWhy`, red-proved). A held app gives the operator no lasting signal → R-1291. **The pick (STATUS):** press Start (the household's Start lifts this hold; a new crash loop holds it again) or remove Kimai through the product. **Earlier state:** OPEN | — | read why the 2026-10-14 restore and its rollback failed (demo-hp's controller log of that day, if kept; the hold record); then clear the hold (`--clear-restore-hold kimai` + a restart) or remove the app through the product; check that a held app raises a standing operator signal, else file that | CC |
| **R-1288** | Backup & restore | P4 | **After the household restores an earlier install's set-aside copy, the next backup sets the REINSTALL's own unit aside too — a second kept copy on the restore page and a second operator alarm.** Seen 2026-10-25 on Tester 1 (0.343.0, Tandoor, attended release run): reinstall → its first backup set install A's unit aside (17:31:55) and wrote install B's unit; restoring A's copy makes the app install A again („the restored data belongs to install 795457b5… — its backups follow it (D47)"); the next 20-minute backup step (17:36:54) found B's unit in `primary/` and set it aside. Nothing is lost; the second copy is the fresh reinstall's near-empty data. D47's design does what it says; the question is whether a restore of a set-aside unit should retire the unit it replaces instead of leaving it for the next writer. | **VERIFY** — fixed on main 2026-10-25 night (controller `b5863a9`; the brief's safest reading, `07` §8.2d), waits for the controller release: the replaced install's unit set aside AT the restore, no alarm. **Live on the rig (rc.2):** the line, the account back, the next backup set nothing aside, one set-aside mail only (`rig/r1288-live.txt`). **Earlier state:** OPEN | — | read `07` §8.2d/§8.2e for the intended end state; if the restore should retire B's unit: move it aside AT the restore (one line, no alarm, „replaced by the restored install") with a test; else document the second copy and word its alarm (R-1287) | CC |
| **R-1294** | Backup & restore | P3 | **demo-hp's weekly off-site clean-up was refused by its own guard on 2026-10-26 (window 16): the plan would remove 209 of 359 snapshots, above the week's cap (179) — nothing was deleted, and each refused week adds to the next plan.** Seen in the operator mailbox during the 2026-10-26 release run (03:17Z, `offsite_prune_guard_refused`, error; details `count_before 359, count_after 359, outcome guard-refused`). The cap refusal is kept on purpose (R-824, R-867 closed the false ones); the open question is whether a store that once exceeds the cap can ever prune again by itself, or refuses every week while it grows. demo-hp is the Tier-0 box with many hand-made backups (sweeps, probes). Earlier cap refusals: demo-hp 2026-10-03 (window 1), demo-felhom 2026-10-05 (window 3). | OPEN | — | read the guard's cap rule (controller off-site prune) against demo-hp's snapshot list: is the plan honest (old, superseded) or not; if honest, say how such a store gets back under the cap (an attended one-time prune, or a cap that follows the store) — a design question for the operator if it changes what a box may delete | CC |
| **R-1295** | Backup & restore | P3 | **A re-enrolled customer's whole-guest off-site tier (PBS DR) can fail to provision at the fresh install and only the hub log says so.** Seen 2026-10-26 on Tester4's fresh install (golden 0.344.0 proof, host `Tester4-eb7371`): 10:31:15 CET `[ERROR] pbsdr auto-provision for Tester4 (WG-registration hook): pbsdr: F-14 auto re-issue for Tester4: tenantsync: endpoint refused: own-namespace list as felhom@pbs!Tester4 failed twice — rolled back (code self_check_failed) — save the customer config to retry`. No operator mail came (mailbox read 09:33Z: only the household's setup-code mail). On a household reinstall this would leave the box without its whole-guest off-site copy until someone reads the log. Not retried here: the retry lists ep0, fenced in this run. Evidence `audits/release-monday-2026-10-26/fresh-install/proof.txt`. | OPEN | — | read why the own-namespace list fails right after the re-issue (a token not yet valid on ep0? a namespace left by the deleted host?); make the rollback mail the operator (an event through the allow-list); retry once with the operator present | CC |
## Storage & devices — 14 rows (P3 5, P4 9)
## Storage & devices — 13 rows (P3 4, P4 9)
| ID | Category | Sev | What | State | Blocked on | Next action | Owner |
|---|---|---|---|---|---|---|---|
@@ -234,9 +232,8 @@ stopping line that lies.
| **R-1213** | Storage & devices | P4 | **Operator detail still on two household pages:** the registered drive card's `ext4 · /dev/sdb1 · Model` line (`templates/storage.html`; W14 hid only the agent drive cards), and the heading „Távoli mentés (3. mentés) — titkosított, offsite" with the English „offsite". Found by the night helper 2026-10-21 (W11/W14 follow-up). | OPEN | — | Move the line behind „Részletek"; „…titkosított, házon kívüli" | CC |
| **R-1275** | Storage & devices | P4 | **An English household sees the drive names in Hungarian** on the install page (and wherever a storage label shows): „Belső tárhely — 52.0 GB free (default)”, „Külső tárhely (hdd_1)” (catalog sweep 2026-10-24, `audits/catalog-sweep-2026-10-24/navidrome/page-install-en.txt`). The label is stored once, in Hungarian, when the drive is found (`felhom-controller/controller/internal/settings/settings.go` `InferStorageLabel`); a household may rename it. | OPEN | — | Render a label that still equals the inferred default in the reader's language; a test per language | CC |
| **R-1276** | Storage & devices | P4 | **„Remove with data” leaves the app's empty parent folder** (`<drive>/appdata/<app>`) when the template binds a subfolder: Grimmory, RomM, Paperless-ngx (catalog sweep 2026-10-24; 4 KB each; the bound folders were removed). Harmless (empty, not counted as old data), untidy for a household that browses the drive. | OPEN | — | Remove the parent when it is empty after the bound folders go; a test | CC |
| **R-1282** | Storage & devices | P3 | **The „Később" 03:30 media restart is forgotten by a controller restart — store the due time (D44 #24).** The restart is an in-memory timer (`felhom-controller/controller/internal/web/media_events.go` `handleMediaRestart`, `time.AfterFunc`); after a controller restart only the notice stays. Risk: media apps do not see the new drives until the household presses „now". Pick: store the due time and run the restart after a controller restart (a due time already past runs at once, within the same rules). Operator ruling D44 = all picks, 2026-10-25. Note: `architecture/15-media-library.md` §11. | **VERIFY** — fixed on main 2026-10-25 night (controller `b5863a9`), waits for the controller release: the „later" time stored, resumed at start, once (`r1282_media_later_test.go`, red-proved). **Earlier state:** OPEN | — | a small controller change + a test that a stored due time survives a restart and fires once; rides a controller release | CC |
## Security & access — 26 rows (P1 2, P2 4, P3 14, P4 6)
## Security & access — 24 rows (P1 2, P2 3, P3 13, P4 6)
| ID | Category | Sev | What | State | Blocked on | Next action | Owner |
|---|---|---|---|---|---|---|---|
@@ -264,8 +261,6 @@ stopping line that lies.
| **R-1194** | Security & access | P3 | **On a fresh box the operator's break-glass SSH door was broken from the first minute: `oob_degraded` „felhom-sshd config invalid (sshd -t fails)".** Rehearsal 2026-10-20, Tester4-816e21, mail at 12:03 UTC (one minute after the host record was created): details `active:false, config_invalid:true, felhom_sshd_port:0, operator_key_configured:false, operator_key_reported:true, reachable:false`. Possibly the shape of a customer with no operator key set (Tester4 is a test customer) — then the alarm is a false one on every such box; or a real config defect. Not diagnosed (no shell). Checked the same day: the only `oob_degraded` in the operator mailbox in 30 days, and no recovery mail followed; the report says port 0. Two candidate mechanisms in the agent (`internal/felhomsshd/health.go:63-66`): `sshd -t -f <conf>` run by the NON-root agent fails on unreadable host keys, or a config rendered before the desired state arrived (port 0). **CAUSE FOUND AND FIXED 2026-10-21 night** (`audits/night-2026-10-21/P5-checks/`): reproduced on a fresh box from the public ISO — `oob_degraded` mailed 20:34 CEST while felhom-sshd was applied 0.4 s after the check and is healthy (port 8822 listening, `sshd -t` passes as the agent's user through its sudo grant — so the non-root theory is FALSE; without sudo it fails on the host key, which the agent never does). The first host report runs before the felhom-sshd loop's first pass and reads a config never rendered. Fix: agent `0da792e` — no OOB stanza until the first pass has run (the hub does not evaluate a missing one; a failed pass counts). `TestR1194_NoOOBStanzaBeforeTheFirstPass` red-proved. | WATCHING — released in agent v0.162.0 (2026-10-22, operator yes; on the 3 test boxes); the false alarm comes only on a FRESH box's first report | the next fresh install with agent ≥ 0.162.0 | At the next fresh install read the hub's events for `oob_degraded` in the first hour (none expected); then close | CC |
| **R-1237** | Security & access | P3 | **The nightly CC-secrets sweep found stored one-time codes in a session record dated 2026-10-23: 14 names in one file (bind tokens and setup codes of test boxes — NIGHT_2026_10_23_*, FIXES_2026_10_21_*, NIGHT_2026_10_21_*, REHEARSAL_2026_10_20_* — and R_DEMO_FELHOM in a file of 2026-10-20).** Mail „[DooPlex CC secrets] 2 secret name(s) found in old session records", 2026-10-23 01:57 UTC, read in the day session (names only). One file holding all of them suggests one command printed many stored values at once (a listing or a dump) in the night session. All but R_DEMO_FELHOM are one-time install codes of test boxes; NIGHT_2026_10_23_* die when Tester4-16fdef is deleted (the operator's do list). | OPEN | — | CC: find the file and the command by the sweep's own report (names only), add a hook shape for that command if it is not one; R_DEMO_FELHOM (demo-felhom's recovery code) — the operator decides whether to rotate it | CC |
| **R-1248** | Security & access | P3 | **An app's own numbers can lie: an attacker with code inside an app's container can answer the app's API with whatever matches its flood, and the upload watch (R-1244) would then call it explained.** Measured remedy, not built: a per-container upload ceiling with `tc` from a helper container in the app's network namespace — on scratch 9202 (unprivileged) a flood was held at 19.9–20.1 Mbit/s and the app kept answering (`audits/SPIKE-upload-watch-2026-10-23.md` §5). Costs: a helper image with `tc` (a new external image, or `tc` in the controller image), `NET_ADMIN` for that helper, re-applied after each app restart. | OPEN | an operator decision (the new image / capability) | Decide; if yes, CC builds the ceiling for the download and media apps first | operator |
| **R-1280** | Security & access | P3 | **A signed operation's window has no maximum length — cap it at 24 h (D44 #22).** The agent (`felhom-agent/internal/authz/verifier.go`) and the root wrapper (`configs/felhom-os-apply`) check `now ∈ [issued_at, expires_at]` with a 2-minute skew on `issued_at`, but neither checks `expires_at − issued_at`; `felhom-opsign` defaults to 30 min (`cmd/felhom-opsign/main.go:57`) and the nonce store keeps a record until its expiry. Risk: a signed job made with a long window by mistake stays usable for that whole time. Pick: the agent and the root wrapper refuse a window longer than 24 h. Operator ruling D44 = all picks, 2026-10-25. Note: `architecture/04-control-plane-authorization.md` §2. | **VERIFY** — fixed on main 2026-10-25 night (agent `d912307`), waits for the agent release + the operator's vouch: 24 h in the verifier, the root wrapper and felhom-opsign (red-proved, `redproof-r1280.txt`). **Earlier state:** OPEN | — | one agent change in both checkers + a test that a 24 h + 1 s window is refused (red first) and a 24 h one is accepted; rides an agent release (agent releases stop for the operator's vouch) | CC |
| **R-1281** | Security & access | P2 | **Every app container can reach FileBrowser directly; the only guard is the secret header name — take FileBrowser off the app networks (D44 #23).** FileBrowser trusts its proxy header (`auth.methods.proxy`); every app container can reach `filebrowser:80` without traefik; the guard is that the header NAME is a per-box secret that traefik's forwardAuth strips from client requests (`felhom-controller/controller/internal/infra/filebrowser_sso.go:15-22`). Risk: an app that learns the name (a hacked app on the box) reads and writes every household file. Pick: FileBrowser joins only the network traefik uses, so no app container reaches it directly. Operator ruling D44 = all picks, 2026-10-25. Note: `architecture/13-threat-model.md` §5. | **VERIFY** — fixed on main 2026-10-25 night (controller `7ea942a`), waits for the controller release: FileBrowser served on `felhom-files` (traefik + OnlyOffice while linked), off traefik-public. **Live on the rig (rc.3):** before, both app containers signed in with the forged header (200); after, no answer (`000`); the household's „Fájlok ↗" walk 200, no cookie 401 (`rig/r1281-*.txt`). The migration live too: FileBrowser put back on traefik-public by hand → the next 5-minute round logged „left traefik-public — no app container reaches it any more" and the apps got no answer again. OnlyOffice's join by test only (not installed on the rig). On the release every box re-creates FileBrowser once (its compose comment changed). **Earlier state:** OPEN | — | a compose/network change in the controller's infra render + a test on the rendered networks; a live check on a Tier-0 box that a forged header from an app container no longer reaches FileBrowser (the 2026-10-18 measurement, now refused) while the dashboard's file manager still signs in; one controller release | CC |
## Box system & updates — 13 rows (P2 1, P3 7, P4 5)
@@ -283,14 +278,14 @@ stopping line that lies.
| **R-1004** | Box system & updates | P4 | **After an unclean stop every guest ext4 volume waits ~43 s in multi-mount protection before it mounts, so each restart after a power cut takes ~1.5 min longer.** Kernel: „EXT4-fs warning (device dm-6): ext4_multi_mount_protect: MMP interval 42 higher than expected, please wait" on dm-6/dm-7 and loop0/loop1, demo-hp and Tester 1, every round of `audits/night-power-2026-10-13/` (R1 `repairs.txt`). | **READY** — filed 2026-10-14 | — | CC: find where MMP is switched on for the guest volumes (Proxmox `pct` create, or our own mkfs), whether it protects anything on a one-host disk, and measure the restart without it on a scratch guest | CC |
| **R-1013** | Box system & updates | P4 | **After a power cut the agent's start-up stale-lock pass logs an ERROR „pct unlock failed … no lock found" when Proxmox's own start-all removed the lock in the same second.** The outcome is the goal (no lock), so the line is a false alarm in the log. Seen on demo-hp 2026-10-14 12:07:44–45 UTC (`audits/power-cut-fixes-2026-10-14/E/after-boot.txt`; Proxmox: „removed left over backup lock from '9201'!"). Small, but the agent's release for this session is spent (v0.156.1). | **OPEN** — filed 2026-10-14 | — | CC: treat „no lock found" from `pct unlock` as already clear (INFO), with a test; ride the next agent release | CC |
| **R-1061** | Box system & updates | P4 | **A per-customer floor reached Tester 1 only at its next 15-minute report, not in seconds.** Set 2026-10-15 12:55:51 UTC for tester-1, demo-hp and demo-felhom together (hub 0.153.0, CC's token): demo-hp and demo-felhom ran 0.314.0 within 13 s (the hub's wake); Tester 1 moved at 13:09:35, its regular report (`audits/gate-restore-2026-10-15/release/03-boxes-move.txt`). It had restarted at 12:54:29 onto rc.1 and logged „hub wait channel active". Why the wake did not reach it is not read (the wake line is DEBUG). No harm: the floor arrived. | **OPEN** — filed 2026-10-15 | — | Read Tester 1's DEBUG ring at the next floor change; compare with the hub's bump log | CC |
| **R-1285** | Box system & updates | P4 | **D43's removal of our SSH drop-in from a BYO host happens one bundle LATE: the bundle run that delivers agent 0.165.0 is planned by the box's OLD (0.164.0) wrapper, which writes the drop-in as before („same"), and only then replaces itself.** Measured 2026-10-25 on the BYO test host (VM 9371 on demo-hp, `Tester4-f967d4`; attended release run): 0.164.0 day-0 → signed `agent_update` + `agent_config_update` 0.165.0 → the planted drop-in still there; a second `agent_config_update` (same bundle) → `BUNDLE REMOVED … (ours, on a BYO host — D43)`, nothing written, `sshd -T` the owner's own `permitrootlogin yes` (`audits/release-2026-10-26/byo/byo-proof.txt`). No box is affected today (the fleet's only BYO host is this test host); a fresh BYO install on ≥ 0.165.0 runs the new wrapper from the start. | **VERIFY** — fixed on main 2026-10-25 night (agent `abcc1e6`), waits for the agent release + the vouch: the agent runs the wrapper's `bundle-policy` pass after every install (red-proved, `redproof-r1285.txt`). **Live on the rig in ONE signed run** (BYO host, staged as upgraded from 0.164.0: the new agent by hand, the old 0.164.0 wrapper on disk, our drop-in planted; one `agent_config_update` of a test bundle): the old wrapper installed the bundle and replaced itself, then „BUNDLE POLICY REMOVED … (ours, on a BYO host)", `sshd -T` the owner's own `permitrootlogin yes` (`rig/r1285-live.txt`); the test package deleted from the registry after. **Earlier state:** OPEN | — | either the wrapper plans again after it replaced itself (one agent change + a test that a 0.164.0→0.165.0 bundle on a BYO record removes our copy in ONE run, red first), or the release procedure sends a BYO box's config bundle twice; until then a BYO box upgraded from ≤ 0.164.0 keeps our drop-in until its next bundle | CC |
| **R-1285** | Box system & updates | P4 | **D43's removal of our SSH drop-in from a BYO host happens one bundle LATE: the bundle run that delivers agent 0.165.0 is planned by the box's OLD (0.164.0) wrapper, which writes the drop-in as before („same"), and only then replaces itself.** Measured 2026-10-25 on the BYO test host (VM 9371 on demo-hp, `Tester4-f967d4`; attended release run): 0.164.0 day-0 → signed `agent_update` + `agent_config_update` 0.165.0 → the planted drop-in still there; a second `agent_config_update` (same bundle) → `BUNDLE REMOVED … (ours, on a BYO host — D43)`, nothing written, `sshd -T` the owner's own `permitrootlogin yes` (`audits/release-2026-10-26/byo/byo-proof.txt`). No box is affected today (the fleet's only BYO host is this test host); a fresh BYO install on ≥ 0.165.0 runs the new wrapper from the start. | **OPEN** — 2026-10-26 (agent 0.166.0 released, attended run): the policy pass is wired, but in the normal delivery it does NOT run: the box gets `agent_update` and `agent_config_update` in ONE pass, and the bundle job runs in the OLD process (demo-hp: pid 1966780, agent 0.165.0, bundle 09:03:07–09:03:09 CET; the restart into 0.166.0 at 09:03:10) — no `bundle policy pass` line on any of the three boxes. So a BYO host coming from 0.164.0 still loses our drop-in one release late. The night proof (`audits/night-2026-10-25/rig/r1285-live.txt`) ran a bundle under an agent that was ALREADY 0.166.0-code, so it did not cover this order. No BYO host is in the fleet today. Fix options: the agent runs the pass once at start-up after `update committed`; or the release procedure sends the bundle job only after the commit. **Earlier state:** **VERIFY** — fixed on main 2026-10-25 night (agent `abcc1e6`), waits for the agent release + the vouch: the agent runs the wrapper's `bundle-policy` pass after every install (red-proved, `redproof-r1285.txt`). **Live on the rig in ONE signed run** (BYO host, staged as upgraded from 0.164.0: the new agent by hand, the old 0.164.0 wrapper on disk, our drop-in planted; one `agent_config_update` of a test bundle): the old wrapper installed the bundle and replaced itself, then „BUNDLE POLICY REMOVED … (ours, on a BYO host)", `sshd -T` the owner's own `permitrootlogin yes` (`rig/r1285-live.txt`); the test package deleted from the registry after. **Earlier state:** OPEN | — | either the wrapper plans again after it replaced itself (one agent change + a test that a 0.164.0→0.165.0 bundle on a BYO record removes our copy in ONE run, red first), or the release procedure sends a BYO box's config bundle twice; until then a BYO box upgraded from ≤ 0.164.0 keeps our drop-in until its next bundle | CC |
## Monitoring & notifications — 24 rows (P2 1, P3 17, P4 6)
## Monitoring & notifications — 23 rows (P2 1, P3 16, P4 6)
| ID | Category | Sev | What | State | Blocked on | Next action | Owner |
|---|---|---|---|---|---|---|---|
| **R-1261** | Monitoring & notifications | P3 | **The R-1250 line hold (committed, NOT released): three edges before it ships (review round 2, HUNT-4 H4–H6).** Two seeders behind one uplink: the hold takes only the first, so the test is refused „not quiet” every 15 min in the Tuesday window and never counted (`felhom-controller/controller/internal/upwatch/linehold.go:155-160`; agent `internal/linetest/linetest.go:379-381`); a hold record of an app removed while held is never cleared and the 50 % rule never runs again (`linehold.go:106-108`; `upwatch/service.go:149`); a qBittorrent alt-speed flip during the hold leaves 1 KiB/s stored (`explain.go:175`; `linehold.go:127-128`). | **VERIFY** — fixed on main 2026-10-25 night (controller `a79a292`), waits for the controller release (before any household gets agent 0.165.0 by signed update): every running seeder held, a removed app's record dropped, qBittorrent's slot named (`up_limit`/`alt_up_limit`). Red-proofs `audits/night-2026-10-25/redproof-r1261.txt`. **Live on the night's rig** (BYO VM, agent 0.165.0, rc.1, two seeders): both held, 94.8 Mbit/s, both limits back; a speed-mode switch mid-test left no 1 KiB/s (`rig/hold-watch*.txt`); the removed-app case by test only. **Earlier state:** OPEN — **now in controller 0.343.0 (released 2026-10-25); the three edges act only on a box whose agent is ≥ 0.165.0 (it alone reports `waiting_for_quiet`) — fix before any household gets agent 0.165.0 by signed update** | — | before the next controller and agent release: hold every running seeder; clear a record whose app is gone; read the mode | CC |
| **R-1261** | Monitoring & notifications | P3 | **The R-1250 line hold (committed, NOT released): three edges before it ships (review round 2, HUNT-4 H4–H6).** Two seeders behind one uplink: the hold takes only the first, so the test is refused „not quiet” every 15 min in the Tuesday window and never counted (`felhom-controller/controller/internal/upwatch/linehold.go:155-160`; agent `internal/linetest/linetest.go:379-381`); a hold record of an app removed while held is never cleared and the 50 % rule never runs again (`linehold.go:106-108`; `upwatch/service.go:149`); a qBittorrent alt-speed flip during the hold leaves 1 KiB/s stored (`explain.go:175`; `linehold.go:127-128`). | **VERIFY** — RELEASED in controller v0.344.0 (global floor 2026-10-26 09:07:47Z, every live box on 0.344.0); the live test with a seeding box is Tuesday 10:00–16:00 (R-1250's dated check). **Earlier:** **VERIFY** — fixed on main 2026-10-25 night (controller `a79a292`), waits for the controller release (before any household gets agent 0.165.0 by signed update): every running seeder held, a removed app's record dropped, qBittorrent's slot named (`up_limit`/`alt_up_limit`). Red-proofs `audits/night-2026-10-25/redproof-r1261.txt`. **Live on the night's rig** (BYO VM, agent 0.165.0, rc.1, two seeders): both held, 94.8 Mbit/s, both limits back; a speed-mode switch mid-test left no 1 KiB/s (`rig/hold-watch*.txt`); the removed-app case by test only. **Earlier state:** OPEN — **now in controller 0.343.0 (released 2026-10-25); the three edges act only on a box whose agent is ≥ 0.165.0 (it alone reports `waiting_for_quiet`) — fix before any household gets agent 0.165.0 by signed update** | — | before the next controller and agent release: hold every running seeder; clear a record whose app is gone; read the mode | CC |
| **R-1262** | Monitoring & notifications | P3 | **The upload watch treats ANY viewer as an explanation for any amount of upload, so a hacked Plex can send freely while one family member watches.** Found 2026-10-24 (SEC-1 X1): `felhom-controller/controller/internal/upwatch/watch.go:275,299`; `08` §9 states the rule, its limits list did not name this case. | OPEN | — | propose a cap (viewers × a bitrate ceiling) for the operator's R-1244 design; `08`'s limits list names the case meanwhile | CC |
| **R-1020** | Monitoring & notifications | P4 | **`app_repaired_after_crash` reports a duration of 0 s, and the controller log says „in 0s".** `RunAfterUncleanBoot` sets `res.Duration` in a deferred func after the result is already returned (no named result), so the caller sees 0; the `app.yaml` record is right (10,172 ms on demo-hp 2026-10-14, `audits/immich-repair-2026-10-14/D/`). Small, but the controller release for this session is spent (v0.311.0). | **OPEN** — filed 2026-10-14 | — | CC: a named result (or set the duration before return), with a test on the returned value; ride the next controller release | CC |
| **R-528** | Monitoring & notifications | P2 | **[P2-MEDIUM] Docker does not report an OOM kill inside a Felhom LXC guest: `OOMKilled` stays false and no `oom` event fires, so the v0.243.0 OOM line is not proven live.** MEASURED 2026-09-15 on scratch 9202 (Docker 29.8.0): Paperless capped at 128M restarted 11 times with `OOMKilled=false` and zero `docker events --filter event=oom`; a memory hog inside the running container was killed (rc 137) with the same silence (`E2-oom-signal-measure-9202.txt`). BIGNIGHT VM 333 did read `oomkilled=true`, so the shape differs by case. **Fix shape:** the agent reads the guest container cgroups' `memory.events oom_kill` counters (host-side, reliable), or the controller alarms on a restart-count trend **RE-MEASURED 2026-09-16 on the DRILL box (fresh install, nested VM 334, Docker in an LXC guest, controller 0.243.0), so the finding is not a property of one machine:** the Paperless webserver was capped at 128 M with `docker update --memory`; it restarted 9-10 times, and all three signals stayed silent - `OOMKilled=false` on every inspect, `docker events --filter event=oom` EMPTY for the whole window, the container's cgroup not visible from inside the guest, and `dmesg` unreadable there. Identical to scratch 9202. So the v0.243.0 OOM line cannot fire on ANY Felhom box as shipped, on either host. Evidence: `audits/evidence-drill-0243-2026-09-16/phase2-m1-oom.txt`. | **WATCHING** — **2026-10-12: the built part is released** (agent v0.150.0 carries the Docker-approval memory-kill check; the boxes run 0.154.0; hub 0.140.0+ requires a passing `oom_check` before a Docker set is approved). Per decision 157 nothing else is built; the row watches for a box whose Docker reports a false OOM flag. Dated look: 2026-11-12. **READY — rank P2-MEDIUM; owner: CC** **2026-09-17 (chaos night): an OOM WAS detected on a fresh box, and named precisely.** On `tester-1-022354` (controller 0.245.0, guest 9201, 6 GB RAM) immich’s Postgres was killed by the memory limit during its reverse-geocoding import, and the controller pushed `app_oom` (warning, operator-only): „Alkalmazás memóriája elfogyott: immich (immich-postgres) — egy folyamatát a memóriakorlát leállította” — naming the app AND the exact container. The visible consequence was `write CONNECTION_CLOSED immich-postgres:5432` and twelve restarts of immich-server. So on THIS box the OOM scan works and was the fastest route to the diagnosis; recorded here rather than filed as a new row. Evidence: `audits/evidence-chaos-night-2026-09-17/round-2.txt`. **2026-10-05 (burn-down night): a one-page design proposal (no code) is in `audits/night-burndown-2026-10-05/design-R-528.md`** — for the operator. **2026-10-06: STOPPED BEFORE ANY CODE — the measurement contradicted the design** (decision 155 said A then C). On scratch 9202, Docker 29.8.2, the `OOMKilled` flag was TRUE in all four shapes tried: a child process killed while the container kept running (`oom_kill` 0 → 3, flag true) and three main-process kills (exit 137, flag true); an `oom` event each time. The false flags of 2026-09-15 were on Docker 29.8.0. Cost read for the record: one exec read of 21 containers on demo-hp = 1.6 s. `audits/design-build-2026-10-06/`C/. **2026-10-06 18:24: operator ruling, option A (decision 157):** nothing is built; the row stays open as a WATCH for a box whose Docker reports a false flag. **Added:** a Docker engine set may not be approved until it reports a memory kill correctly on the boxes that ran it (built in this row's next session). **2026-10-06 (night): the Docker-approval memory-kill check is BUILT** — hub v0.140.0 (LIVE: the approval waits for a passing `oom_check` on every ring-0 box; a failed, errored or missing one blocks) and the agent's wrapper (`felhom-agent` `acccb66`, unreleased — ships as v0.150.0 after the 2026-10-07 read-back). Proven by hand on demo-hp's guest: `OOMKilled=true`, exit 137, the `oom` event — seen only with an events window ending after the run (the wrapper waits 2 s). `audits/readback-2026-10-07/`F/. | — | 2026-11-12: read the hub for any failed `oom_check` or an `app_oom` with no restart reason since 2026-10-12; none → close | CC |
@@ -312,11 +307,10 @@ stopping line that lies.
| **R-1223** | Monitoring & notifications | P3 | **After a reboot, a box whose tunnel ep0 refuses is invisible: the future stamp is gone, so neither the D30 re-key nor the hub's alarm sees it.** Fresh review 2026-10-23 night (`REPORT-night-2026-10-23.md`, A8 + H1): the kernel forgets `latest-handshake` when the interface restarts (0 = never); the agent then sends neither `last_handshake_age_s` nor `handshake_future_s`, and `checkWGTunnel` (`hub/internal/api/host_tunnel.go`) does not judge a registered, active tunnel with no handshake at all. Shape: the clock ran ahead, was corrected, the box rebooted (some households switch off at night), ep0 still holds the future stamp and starts nothing. | OPEN | — | Agent: report `tunnel_up_s` (how long the unit has been active) with no handshake; hub: a registered, active tunnel with no handshake for > 30 min while reporting → `wg_tunnel_stale`; the agent re-keys the same way after 30 min with no handshake. Tests both sides; rides an agent release + a hub deploy | CC |
| **R-1224** | Monitoring & notifications | P4 | **The operator's 1-hour mail cooldown for `wg_tunnel_stale` is per customer, not per host, and `wg_tunnel_ok` does not end it.** Fresh review 2026-10-23 night (`REPORT-night-2026-10-23.md`, H3): `notify/dispatcher.go:620` keys the cooldown `customerID:eventType`. Two hosts of one household going stale within an hour, or stale → ok → stale, store a second event row but mail once (the suppression is in `notification_log`). | OPEN | — | Key the cooldown by `host_id` from the details for host-scoped operator events, and let the matching ok event end it; a dispatcher test with two hosts; rides a hub deploy | CC |
| **R-1233** | Monitoring & notifications | P4 | **`[ERROR] [stacks] execCommand failed: exit status 1` names no command.** Seen once a night (2026-10-21 00:30:49 and 2026-10-23 02:17:14 UTC, each right after an app stopped for its dump); the command and stderr go only into the returned error, which the caller may drop (`internal/stacks/manager.go:1757`). An ERROR nobody can judge. | OPEN | — | Log the command's first two words (`docker inspect`, never the arguments — `docker exec … psql` can carry a password) and the stderr's first line; find the caller of the nightly one | CC |
| **R-1287** | Monitoring & notifications | P3 | **A D47 set-aside mails the operator as „🔴 tester-1: 1 of 1 apps failed to back up (backup run)" — no backup failed; the body says the right thing („its backup belonged to an earlier install … SET ASIDE whole").** Seen 2026-10-25 on Tester 1 with controller 0.343.0 (attended release run): two such mails at 17:31:56Z and 17:36:55Z. The controller sends the set-aside as `backup_run_failures` leg `unit-set-aside`, `Failed: 1` (so the hub never drops a line that claims zero); the hub's subject for that event is fixed („N of M apps failed to back up", `hub/internal/notify/templates.go:353`). The CLAUDE.md corollary: an alarm's text must say what it counts. After the floor, every household reinstall mails the operator a red „failed to back up". | **VERIFY** — fixed on main 2026-10-25 night, waits for the hub release (v0.171.0): a set-aside-only digest is a warning with „… set aside — nothing failed (<apps>)"; a run with a real failure stays red (`hub/internal/notify/r1287_setaside_test.go`, red-proved) | — | two-repo: the hub words a `unit-set-aside` leg on its own („… a backup of an earlier install was set aside — nothing failed", warning, not error) with a test on the rendered subject; or a new event type through the allow-list (`hub-event-allowlist-gotcha`); one hub release | CC |
| **R-1291** | Monitoring & notifications | P3 | **A held app gives the operator no lasting signal: one event at the stop, then nothing — demo-hp's Kimai sat held 11 days unseen.** Found 2026-10-25 night (R-1286). No held-app state reaches the hub (the report carries none; the hub shows none); the crash-loop stop sends `app_stopped_unhealthy` once. On a household box an app held after a crash loop, a failed update or a failed restore stays stopped until someone opens its page. | OPEN | — | a two-repo design: the controller reports its holds (stack, reason, since), the hub shows them on the host and mails a reminder after N days (the D-level of the reason); one row's worth of design first | CC |
## Hub & operator — 13 rows (P3 4, P4 9)
## Hub & operator — 12 rows (P3 3, P4 9)
| ID | Category | Sev | What | State | Blocked on | Next action | Owner |
|---|---|---|---|---|---|---|---|
@@ -331,7 +325,6 @@ stopping line that lies.
| **R-1002** | Hub & operator | P4 | **A box the household switched off reads ONLINE on the hub's hosts page for its first 45 minutes** (seen live 2026-10-13, R-998 proof, `audits/poweroff-2026-10-13/B-hub-watch.log`: the „Power: Kikapcsolva…" line at 18:53, the KIKAPCSOLVA badge only at 19:38, when the host report turned stale). The customer page read „warn" in part of that gap (seen, not captured). No alarm and no mail fired; the operator page is only slow to say it. | **READY** — filed 2026-10-13 | — | CC: let `hostDisplayStatus` read `deliberate_off_at` before staleness, with a render test per state; next hub release | CC |
| **R-1056** | Hub & operator | P4 | **Five operator routes fit no token group, so they are session only until the operator places them:** apps reset-telemetry and dismiss-issues, the Configuration asset re-seed, the off-site WireGuard endpoint save, the off-site abandon-cancel (`05` §17.3). Nothing needs them by token today. | WAITING-ON-OPERATOR — filed 2026-10-15 by the token build | operator decision | Place each in a group, or leave session only | operator |
| **R-1164** | Hub & operator | P4 | **Move the website to Hetzner (maybe the JARRs.eu prod server) — an operator idea, no decision asked yet.** If it is done, the website's git-syncs must read the public `https://gitea.dooplex.hu/admin/felhom.eu.git` again: since 2026-10-19 (R-1163) all four read Gitea in-cluster, which works only inside the DooPlex cluster (`01-topology-and-trust.md` §7, `manifests/webpage.yaml`). | OPEN | the operator's idea becoming a decision | When decided: switch the four `--repo` values together (gate 6 keeps them identical); keep `--max-failures=-1` | operator |
| **R-1289** | Hub & operator | P4 | **The hub's „newest controller the fleet reported" can be a dev build that no box runs: `semver.Compare` returns 0 (equal) for any version with a suffix, so „0.343.0-dev.r1251a" — once read first — beats nothing and is never beaten.** Seen 2026-10-25 in the vouch line (attended release run): „golden 0.331.0 unchanged (behind the fleet's 0.343.0-dev.r1251a …)" while the four hosts run 0.343.0 / 0.342.0. `store.NewestReportedControllerVersion` scans every report ever stored (`hub/internal/store/store.go:2210`), and `hub/internal/semver/semver.go:14` gives up on `0-dev…` / `0-rc.N`. Used by the R-120 golden gate (`hub/internal/web/configs.go:1548,1554`): with a suffixed „newest" the gate compares against whatever row came first — today it only words a log line and fails open. | **VERIFY** — fixed on main 2026-10-25 night, waits for the hub release (v0.171.0): suffixed versions are skipped (`TestR1289_ATestBuildIsNeverTheFleetsNewest`, red-proved) | — | skip suffixed versions in `NewestReportedControllerVersion` (or compare with the pre-release rule the controller uses, `util.ParseVersionPre`) + a test with a dev and an rc row ahead of a release row (red first); one hub release | CC |
| **R-1293** | Hub & operator | P4 | **Once a year the hub's 02:00 database copy never reaches ep0, and the 20:00 copy is pushed twice instead.** Measured 2026-10-26 during R-1183's read (`audits/night-2026-10-25/r1183-measurement-2026-10-26.txt`): on 2026-10-25 (the clocks go back) 02:00–03:00 happens twice. The hub's `nextDailyRun` uses `time.Date`, which picks the SECOND 02:00 (CET, 01:00Z — measured with Go). DooPlex's `felhom-hub-db-backup.timer` fired at the FIRST 02:30 (CEST, 00:31Z) and pushed `hub-20261024T180000Z.db`, 391 min old — under the 7 h limit, so no refusal and no alarm; the 08:30 push took the 08:00 copy. Effect: for one morning a year ep0's newest hub copy is 12 h old, not 6 h; nothing is lost on the hub. The spring night (2027-03-28, 02:00–03:00 does not exist) is not measured. | OPEN | — | pick one: the push refuses (and says so) a newest snapshot older than the slot it serves, or the hub and the timer agree on one 02:00; then read the 2027-03-28 night (DUE-CHECKS) | CC |
## Business & legal — 14 rows (P2 4, P3 2, P4 8)
+4
View File
@@ -7,6 +7,10 @@ Moved here byte for byte on 2026-10-14 (night session), newest first, so that `S
## Moved from STATUS on 2026-10-26 (attended release run)
**Updated 2026-10-26 (morning).** Installer 1.36.1 is live: a wrong passphrase now says so. Overnight, commit only: the torrent line test holds every seeder; FileBrowser is off the app network; a set-aside mails as a set-aside; the four D44 picks are built and proven on a test VM.
**Monday 2026-10-26 (attended; each a STOP).** 1. Hub v0.171.0 (R-1287, R-1289). 2. Agent v0.166.0 + your vouch (R-1280, R-1285). 3. Controller v0.344.0 — before any household gets agent 0.165.0 (R-1261, R-1281, R-1288, R-1282, R-1286, R-1278). 4. Catalog: MeTube (R-1283). 5. Dated: R-1250 Tuesday. 6. Hub Hosts page: delete the dead `Tester4-84bbe7`. If not: the fixes stay unreleased; the host alarms.
| D51 | R-1286 | demo-hp's Kimai was stopped by the crash-loop guard on 10-14 (the power drill) and is still held. Start it again, or remove it? | **Press Start** (it lifts this kind of hold; if it crash-loops again it is held again and we look) | remove it through the product: the test app and its data are gone | it stays stopped; nothing else waits on it | — |
**Ruled:** D51 = Start (operator, 2026-10-26 07:57; `09` §3 decision 263) — healthy, 0 restarts. D52 = bake the golden today on v0.344.0 (operator, 2026-10-26; decision 264) — asked in chat, never on the page; the waiver covers 2026-10-26 only.
@@ -0,0 +1,6 @@
## boot 2026-10-26T09:23:45Z
drill VM reverted to virgin and cold-booted (RUNBOOK-manual-build §4.0); pve-manager/9.2.2
pveam update: Update successful; template debian-13-standard_13.6-1_amd64.tar.zst (checksum verified)
build-golden.sh from felhom-agent origin/main b890b57, sha256 645b3b65… (the same file as the 0.331.0 bake)
token + registry user copied file->file from sec (GITEA_PKG_TOKEN, GITEA_PKG_USER); unit properties grep for the token value: 0
token in bake.log: 0 (control with the token appended: 1)
@@ -0,0 +1,3 @@
## round trip 2026-10-26T09:23:54Z
sha256 767a90fe07ed47c85a3e08a16b85926e30531b65608f9856aa25855b4dd53210 bytes 637293448
bake said 767a90fe07ed47c85a3e08a16b85926e30531b65608f9856aa25855b4dd53210
@@ -0,0 +1,11 @@
== teardown 2026-10-26T09:24:10Z
purging CT 9100 from related configurations..
0
ls: cannot access '/root/.gitea-token': No such file or directory
ls: cannot access '/root/.gitea-user': No such file or directory
ls: cannot access '/root/bake-run.sh': No such file or directory
ls: cannot access '/root/bake.log': No such file or directory
qemu running: 0
revert rc=0
ID TAG VM_SIZE DATE VM_CLOCK ICOUNT
1 virgin 0 B 2026-07-03 14:12:15 0000:00:00.000 0
@@ -0,0 +1,42 @@
# Golden 0.344.0 — bake + publish, 2026-10-26 (the attended release run; D52)
Why: the Sunday 2026-10-25 weekly bake was missed (it was not in that night's brief); the operator ruled **D52 = bake
today on v0.344.0** (`09` §3 decision 264), with the waiver covering 2026-10-26 only. Controller 0.344.0 was released the
same morning (global floor 09:07:47 UTC, MinAgent 0.158.1). Procedure: `documentation/runbooks/RUNBOOK-manual-build.md`
§4.0 and §4.1 steps 1–4, in the drill VM on DooPlex. The script came from CC's `felhom-agent` clone at `origin/main`
(`b890b57`; `build-golden.sh` sha256 `645b3b65…`, the same file as the 0.331.0 bake).
| | Previous (`../golden-0.331.0-2026-10-18/`) | This bake |
|---|---|---|
| `build-golden.sh` | v3.2.0 | same file |
| Controller | `felhom-controller:0.331.0` (MinAgent 0.158.1) | **`felhom-controller:0.344.0`** (MinAgent **0.158.1**) |
| Docker engine | the approved set (six pairs) | the same six pairs — the hub's newest approved Docker release is still `os-docker-20261004-142842` (read from the hub DB, with its WAL, today) |
| Template | `debian-13-standard_13.6-1_amd64.tar.zst` | same (after `pveam update`; `_amd64` filtered) |
| Token | `~/.gitea-token` file → file | **from `sec`** (`GITEA_PKG_TOKEN`, `GITEA_PKG_USER`), written to a 0600 file and copied file → file, the DooPlex copy shredded |
## Pass markers (from `bake.log`, this folder)
```
[golden] build-golden.sh v3.2.0 — baking controller gitea.dooplex.hu/admin/felhom-controller:0.344.0
[golden] Docker engine set PINNED to the approved release: containerd.io=2.3.6-1~debian.13~trixie … docker-compose-plugin=5.6.0-1~debian.13~trixie
docker OK (overlay2; data-root /var/lib/docker)
INFO: including mount point rootfs ('/') in backup
INFO: including mount point mp0 ('/var/lib/felhom') in backup
[golden] upload OK (HTTP 201)
GOLDEN_VERSION=0.344.0
GOLDEN_SHA256=767a90fe07ed47c85a3e08a16b85926e30531b65608f9856aa25855b4dd53210
```
No `excluding`, no `FATAL`, no `GOLDEN_DOCKER_PKGS not set`; the unit `Result=success`. Round trip: the registry's file
downloaded and hashed = the bake's sha, 637,293,448 bytes (`02-round-trip.txt`). Token: the unit's properties grep = 0
(`00-boot.txt`); the committed log grep = 0, with a working control (= 1 on a copy with the token appended, the copy
shredded). Teardown: `03-teardown.txt` (CT 9100 purged, token/user/script/log shredded in the VM, VM off, disk back to
`virgin`). No controller push after the bake.
## Vouch — the operator's act (never CC's)
Three fields together (runbook §4.1 step 5), Configuration → Day-0 artifacts:
- golden **0.344.0**, sha256 `767a90fe07ed47c85a3e08a16b85926e30531b65608f9856aa25855b4dd53210`
- agent **0.166.0** (vouched 2026-10-26 10:04:55 CET, unchanged), sha256 `3f22c65b78914632fa476292f6b58e5af3ed9e9900f3a5a7a347a9c0bfea9c5c`
- min_agent **0.158.1** (unchanged)
@@ -0,0 +1,340 @@
[golden] build-golden.sh v3.2.0 — baking controller gitea.dooplex.hu/admin/felhom-controller:0.344.0
[golden] creating build LXC 9100 (nesting=1,keyctl=1, unprivileged; rootfs 32G + ONE data volume 24G @ /var/lib/felhom, backup=1) …
Logical volume "vm-9100-disk-0" created.
Logical volume pve/vm-9100-disk-0 changed.
Creating filesystem with 8388608 4k blocks and 2097152 inodes
Filesystem UUID: caff2a7f-c2de-4387-8f2d-826c818830d5
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624
Logical volume "vm-9100-disk-1" created.
Logical volume pve/vm-9100-disk-1 changed.
Creating filesystem with 6291456 4k blocks and 1572864 inodes
Filesystem UUID: ab9abb60-9c7e-473f-8c74-02aebb335f9e
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
extracting archive '/var/lib/vz/template/cache/debian-13-standard_13.6-1_amd64.tar.zst'
Total bytes read: 553512960 (528MiB, 98MiB/s)
Detected container architecture: amd64
Creating SSH host key 'ssh_host_rsa_key' - this may take some time ...
done: SHA256:zJDWIY1iIUrsWL8Uw9/eVhpMjpx9RDQJB5zaC4colbI root@felhom-golden
Creating SSH host key 'ssh_host_ecdsa_key' - this may take some time ...
done: SHA256:E6DkbbkbDgaanslUSIzFF93Jrmb0CKmNAvfVNdtfrxY root@felhom-golden
Creating SSH host key 'ssh_host_ed25519_key' - this may take some time ...
done: SHA256:oRIy665uwX7I8VjR33EZoR78U+TvGVvBNhnXkt64H3M root@felhom-golden
[golden] starting + installing Docker (official repo, trixie channel) …
[golden] Docker engine set PINNED to the approved release: containerd.io=2.3.6-1~debian.13~trixie docker-buildx-plugin=0.37.1-1~debian.13~trixie docker-ce=5:29.8.2-1~debian.13~trixie docker-ce-cli=5:29.8.2-1~debian.13~trixie docker-ce-rootless-extras=5:29.8.2-1~debian.13~trixie docker-compose-plugin=5.6.0-1~debian.13~trixie
apt-listchanges: Can't set locale; make sure $LC_* and $LANG are correct!
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LC_CTYPE = (unset),
LC_NUMERIC = (unset),
LC_COLLATE = (unset),
LC_TIME = (unset),
LC_MESSAGES = (unset),
LC_MONETARY = (unset),
LC_ADDRESS = (unset),
LC_IDENTIFICATION = (unset),
LC_MEASUREMENT = (unset),
LC_PAPER = (unset),
LC_TELEPHONE = (unset),
LC_NAME = (unset),
LANG = "en_US.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
apt-listchanges: Can't set locale; make sure $LC_* and $LANG are correct!
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LC_CTYPE = (unset),
LC_NUMERIC = (unset),
LC_COLLATE = (unset),
LC_TIME = (unset),
LC_MESSAGES = (unset),
LC_MONETARY = (unset),
LC_ADDRESS = (unset),
LC_IDENTIFICATION = (unset),
LC_MEASUREMENT = (unset),
LC_PAPER = (unset),
LC_TELEPHONE = (unset),
LC_NAME = (unset),
LANG = "en_US.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
installed: containerd.io 2.3.6-1~debian.13~trixie
installed: docker-buildx-plugin 0.37.1-1~debian.13~trixie
installed: docker-ce 5:29.8.2-1~debian.13~trixie
installed: docker-ce-cli 5:29.8.2-1~debian.13~trixie
installed: docker-ce-rootless-extras 5:29.8.2-1~debian.13~trixie
installed: docker-compose-plugin 5.6.0-1~debian.13~trixie
[golden] no approved guest release given - the template versions stay; first-night count vs an approved release: n/a
[golden] pending Debian upgrades in the baked guest (what a FUTURE approval may bring): 57
[golden] baking daemon.json: classic overlay2 driver (containerd-snapshotter OFF) + log rotation …
[golden] wiring the single data volume (R-165 variant V-c): /var/lib/felhom/{docker,sys_drive} -> binds …
[golden] verifying Docker works in the build guest (storage driver should be overlay2 on the ext4 data volume) …
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
4f55086f7dd0: Pulling fs layer
4f55086f7dd0: Verifying Checksum
4f55086f7dd0: Download complete
4f55086f7dd0: Pull complete
Digest: sha256:11a3eb1597e864c51b3f56733cb2e8a90bf0c284bfc05f9444d81c558bc848aa
Status: Downloaded newer image for hello-world:latest
docker OK (overlay2; data-root /var/lib/docker)
live-restore: on
/var/lib/docker is a real mount: /dev/mapper/pve-vm--9100--disk--1[/docker] ext4
/mnt/sys_drive is a real mount: /dev/mapper/pve-vm--9100--disk--1[/sys_drive] ext4
both paths are ONE filesystem: /dev/mapper/pve-vm--9100--disk--1 23317576
[golden] baking the in-guest controller image gitea.dooplex.hu/admin/felhom-controller:0.344.0 (no registry cred at deploy) …
WARNING! Your credentials are stored unencrypted in '/root/.docker/config.json'.
Configure a credential helper to remove this warning. See
https://docs.docker.com/go/credential-store/
0.344.0: Pulling from admin/felhom-controller
ecc510c1e359: Pulling fs layer
c3f83389ce1b: Pulling fs layer
e81c1fce9826: Pulling fs layer
63a1406b1961: Pulling fs layer
d747c9b6922d: Pulling fs layer
6123e7de002d: Pulling fs layer
a77e9b7b71fe: Pulling fs layer
d747c9b6922d: Waiting
6123e7de002d: Waiting
63a1406b1961: Waiting
a77e9b7b71fe: Waiting
e81c1fce9826: Verifying Checksum
e81c1fce9826: Download complete
63a1406b1961: Verifying Checksum
63a1406b1961: Download complete
c3f83389ce1b: Verifying Checksum
c3f83389ce1b: Download complete
d747c9b6922d: Verifying Checksum
d747c9b6922d: Download complete
ecc510c1e359: Download complete
6123e7de002d: Verifying Checksum
6123e7de002d: Download complete
a77e9b7b71fe: Verifying Checksum
a77e9b7b71fe: Download complete
ecc510c1e359: Pull complete
c3f83389ce1b: Pull complete
e81c1fce9826: Pull complete
63a1406b1961: Pull complete
d747c9b6922d: Pull complete
6123e7de002d: Pull complete
a77e9b7b71fe: Pull complete
Digest: sha256:9435165dab85b1200d94157ec49d8be065889139e1831a5476dc68f4b9474b8a
Status: Downloaded newer image for gitea.dooplex.hu/admin/felhom-controller:0.344.0
gitea.dooplex.hu/admin/felhom-controller:0.344.0
[golden] asking the controller which infra images it manages …
[golden] baking infra images (4): traefik:v3.7.13 cloudflare/cloudflared:2026.9.3 gtstef/filebrowser:1.5.6-stable gitea.dooplex.hu/admin/felhom-samba:1.2.0 …
v3.7.13: Pulling from library/traefik
e2de96513ba9: Pulling fs layer
b686a4f73445: Pulling fs layer
78cb21c375ca: Pulling fs layer
acb2f33459b1: Pulling fs layer
acb2f33459b1: Waiting
e2de96513ba9: Verifying Checksum
e2de96513ba9: Download complete
acb2f33459b1: Verifying Checksum
acb2f33459b1: Download complete
b686a4f73445: Verifying Checksum
b686a4f73445: Download complete
e2de96513ba9: Pull complete
78cb21c375ca: Verifying Checksum
78cb21c375ca: Download complete
b686a4f73445: Pull complete
78cb21c375ca: Pull complete
acb2f33459b1: Pull complete
Digest: sha256:24841fe2de7304c149343d877d2923b4c8800a38ba015dea9174c23b20e344a0
Status: Downloaded newer image for traefik:v3.7.13
docker.io/library/traefik:v3.7.13
2026.9.3: Pulling from cloudflare/cloudflared
2cc7ee286bf3: Pulling fs layer
c172f21841df: Pulling fs layer
218cf840d0d9: Pulling fs layer
f6069939f718: Pulling fs layer
d6b1b89eccac: Pulling fs layer
2780920e5dbf: Pulling fs layer
7c12895b777b: Pulling fs layer
3214acf345c0: Pulling fs layer
52630fc75a18: Pulling fs layer
dd64bf2dd177: Pulling fs layer
b839dfae01f6: Pulling fs layer
ebddc55facdc: Pulling fs layer
c4bc6f35ff5e: Pulling fs layer
b96fe2995f90: Pulling fs layer
58c0c263dc73: Pulling fs layer
bd8962e29291: Pulling fs layer
cac2ae0193cb: Pulling fs layer
f0383d5ebc47: Pulling fs layer
dd64bf2dd177: Waiting
b839dfae01f6: Waiting
ebddc55facdc: Waiting
c4bc6f35ff5e: Waiting
b96fe2995f90: Waiting
f6069939f718: Waiting
d6b1b89eccac: Waiting
58c0c263dc73: Waiting
bd8962e29291: Waiting
cac2ae0193cb: Waiting
f0383d5ebc47: Waiting
2780920e5dbf: Waiting
7c12895b777b: Waiting
3214acf345c0: Waiting
52630fc75a18: Waiting
2cc7ee286bf3: Verifying Checksum
2cc7ee286bf3: Download complete
c172f21841df: Verifying Checksum
c172f21841df: Download complete
218cf840d0d9: Verifying Checksum
218cf840d0d9: Download complete
2cc7ee286bf3: Pull complete
f6069939f718: Verifying Checksum
d6b1b89eccac: Verifying Checksum
d6b1b89eccac: Download complete
2780920e5dbf: Verifying Checksum
2780920e5dbf: Download complete
7c12895b777b: Download complete
52630fc75a18: Download complete
3214acf345c0: Verifying Checksum
3214acf345c0: Download complete
c172f21841df: Pull complete
dd64bf2dd177: Verifying Checksum
dd64bf2dd177: Download complete
ebddc55facdc: Verifying Checksum
ebddc55facdc: Download complete
b839dfae01f6: Verifying Checksum
b839dfae01f6: Download complete
c4bc6f35ff5e: Verifying Checksum
c4bc6f35ff5e: Download complete
b96fe2995f90: Verifying Checksum
b96fe2995f90: Download complete
bd8962e29291: Verifying Checksum
bd8962e29291: Download complete
58c0c263dc73: Download complete
cac2ae0193cb: Verifying Checksum
cac2ae0193cb: Download complete
218cf840d0d9: Pull complete
f0383d5ebc47: Verifying Checksum
f0383d5ebc47: Download complete
f6069939f718: Pull complete
d6b1b89eccac: Pull complete
2780920e5dbf: Pull complete
7c12895b777b: Pull complete
3214acf345c0: Pull complete
52630fc75a18: Pull complete
dd64bf2dd177: Pull complete
b839dfae01f6: Pull complete
ebddc55facdc: Pull complete
c4bc6f35ff5e: Pull complete
b96fe2995f90: Pull complete
58c0c263dc73: Pull complete
bd8962e29291: Pull complete
cac2ae0193cb: Pull complete
f0383d5ebc47: Pull complete
Digest: sha256:072c067d25ccbe61d46e18f0d0723255f2bb5304f7317caa95b27031520ff92c
Status: Downloaded newer image for cloudflare/cloudflared:2026.9.3
docker.io/cloudflare/cloudflared:2026.9.3
1.5.6-stable: Pulling from gtstef/filebrowser
55afa1ecc21d: Pulling fs layer
8ed8f35f8d4f: Pulling fs layer
989b226a579c: Pulling fs layer
660aeead31d5: Pulling fs layer
4f4fb700ef54: Pulling fs layer
adce24567e4c: Pulling fs layer
f17ea56b313b: Pulling fs layer
6b6f3b3efe88: Pulling fs layer
4ed1ca4f3fce: Pulling fs layer
e6fc9c6a5757: Pulling fs layer
d47782d1182a: Pulling fs layer
6b6f3b3efe88: Waiting
4ed1ca4f3fce: Waiting
e6fc9c6a5757: Waiting
d47782d1182a: Waiting
660aeead31d5: Waiting
4f4fb700ef54: Waiting
adce24567e4c: Waiting
f17ea56b313b: Waiting
55afa1ecc21d: Verifying Checksum
55afa1ecc21d: Download complete
660aeead31d5: Verifying Checksum
660aeead31d5: Download complete
4f4fb700ef54: Verifying Checksum
4f4fb700ef54: Download complete
989b226a579c: Verifying Checksum
989b226a579c: Download complete
55afa1ecc21d: Pull complete
8ed8f35f8d4f: Verifying Checksum
8ed8f35f8d4f: Download complete
f17ea56b313b: Verifying Checksum
f17ea56b313b: Download complete
6b6f3b3efe88: Verifying Checksum
6b6f3b3efe88: Download complete
4ed1ca4f3fce: Verifying Checksum
4ed1ca4f3fce: Download complete
adce24567e4c: Verifying Checksum
adce24567e4c: Download complete
d47782d1182a: Verifying Checksum
d47782d1182a: Download complete
e6fc9c6a5757: Verifying Checksum
e6fc9c6a5757: Download complete
8ed8f35f8d4f: Pull complete
989b226a579c: Pull complete
660aeead31d5: Pull complete
4f4fb700ef54: Pull complete
adce24567e4c: Pull complete
f17ea56b313b: Pull complete
6b6f3b3efe88: Pull complete
4ed1ca4f3fce: Pull complete
e6fc9c6a5757: Pull complete
d47782d1182a: Pull complete
Digest: sha256:7c5d7ac8ffda31294d278063cf9d2e04303b39e6dce1f4c691342240ca7703b8
Status: Downloaded newer image for gtstef/filebrowser:1.5.6-stable
docker.io/gtstef/filebrowser:1.5.6-stable
1.2.0: Pulling from admin/felhom-samba
e2de96513ba9: Already exists
0ddbc9f3a6b6: Pulling fs layer
ce57a3f93416: Pulling fs layer
fb94eeec2fe1: Pulling fs layer
ce57a3f93416: Verifying Checksum
ce57a3f93416: Download complete
fb94eeec2fe1: Verifying Checksum
fb94eeec2fe1: Download complete
0ddbc9f3a6b6: Verifying Checksum
0ddbc9f3a6b6: Download complete
0ddbc9f3a6b6: Pull complete
ce57a3f93416: Pull complete
fb94eeec2fe1: Pull complete
Digest: sha256:da96f25033827fbb77d0ee398e86fe5b744e2f29be9de898994533340c97acf9
Status: Downloaded newer image for gitea.dooplex.hu/admin/felhom-samba:1.2.0
gitea.dooplex.hu/admin/felhom-samba:1.2.0
[golden] baking the controller-bootstrap unit (deploys the BAKED controller from the config mount) …
Created symlink '/etc/systemd/system/multi-user.target.wants/felhom-controller-bootstrap.service' → '/etc/systemd/system/felhom-controller-bootstrap.service'.
[golden] baking the controller-bootstrap PATH unit (starts the service on bootstrap-mount hot-plug — B1) …
Created symlink '/etc/systemd/system/multi-user.target.wants/felhom-controller-bootstrap.path' → '/etc/systemd/system/felhom-controller-bootstrap.path'.
[golden] baking the first-boot SSH host-key regeneration unit (F3) …
Created symlink '/etc/systemd/system/multi-user.target.wants/felhom-regen-hostkeys.service' → '/etc/systemd/system/felhom-regen-hostkeys.service'.
[golden] identity-clean + minimize …
[golden] stop + archive …
INFO: including mount point rootfs ('/') in backup
INFO: including mount point mp0 ('/var/lib/felhom') in backup
INFO: archive file size: 607MB
INFO: Finished Backup of VM 9100 (00:00:30)
[golden] DONE. golden archive volid: local:backup/vzdump-lxc-9100-2026_10_26-10_17_42.tar.zst (rootfs 32G + ONE data volume 24G @ /var/lib/felhom, all in the archive)
[golden] publishing golden (637293448 bytes, sha256 767a90fe07ed47c8…) → https://gitea.dooplex.hu/api/packages/admin/generic/felhom-golden/0.344.0/golden.tar.zst
[golden] pre-delete existing: HTTP 404 (404/204 expected)
[golden] upload OK (HTTP 201)
GOLDEN_VERSION=0.344.0
GOLDEN_SHA256=767a90fe07ed47c85a3e08a16b85926e30531b65608f9856aa25855b4dd53210
[golden] Record in the hub operator UI (Configs → Day-0 artifacts): golden 0.344.0 / 767a90fe07ed47c85a3e08a16b85926e30531b65608f9856aa25855b4dd53210
[golden] (the build guest 9100 is stopped; destroy it with: pct destroy 9100 --purge)
-20
View File
@@ -1,20 +0,0 @@
# documentation/tests/golden-waiver.yml — read by scripts/golden_currency_gate.py
#
# Goldens are on a CADENCE, not per release (operator ruling 2026-09-13, R-468): weekly, and always
# before any drill or fresh install. While this waiver is valid a golden BEHIND the record makes the
# gate ADVISORY instead of red. It never covers a golden that is UNRECORDED (R-385). At most 14 days
# after `issued`, or the gate refuses it. Renew with a new dated commit; delete it at the first
# external install.
# Issued 2026-10-13 by CC: controller 0.310.0 (R-998, „A gép kikapcsolása") was released the evening of the weekly bake
# (golden 0.309.0, baked 2026-10-13); the next weekly bake is due 2026-10-20. The bake and its vouch are the operator's
# acts (the R-998 task fences them). No drill or fresh install is planned before then. Not a bypass: dated, capped at
# the weekly due date.
# 2026-10-14: goldens 0.310.0 and 0.311.0 were baked the same day (`golden-0.311.0-2026-10-14/`), so the golden is at
# the newest controller and this waiver covers nothing today. It stays until its date for a release before the next bake.
# 2026-10-26 by CC, on the operator's D52 (bake today on v0.344.0): the Sunday 2026-10-25 bake was missed (it was not in
# that night's brief); golden 0.331.0 is 8 days old. Renewed for TODAY ONLY so the release commits can be pushed; the
# gate is green tomorrow by the new golden, and this waiver is removed after the vouch.
issued: 2026-10-26
expires: 2026-10-27
reason: D52: bake on v0.344.0 the same day
register_row: R-468
+1 -1
View File
@@ -1,4 +1,4 @@
## v0.171.0 (unreleased, 2026-10-25 night) — a set-aside mails as a set-aside (R-1287); a test build is never „the fleet's newest" (R-1289) — committed, NOT deployed
## v0.171.0 (2026-10-25 night; RELEASED 2026-10-26 07:58 UTC, attended release run — image sha256:742d2e69…, from `7ad7c996`, CI 2352) — a set-aside mails as a set-aside (R-1287); a test build is never „the fleet's newest" (R-1289)
- **R-1287** (`internal/notify`): a `backup_run_failures` digest whose every entry is leg `unit-set-aside` (a D47
set-aside, controller ≥ 0.343.0) is sent as a **warning** with its own subject „⚠️ <customer>: a backup of an earlier