69 Commits

Author SHA1 Message Date
admin 97f088ec02 night evidence: both Nextcloud upload trees 3031/3033 byte-identical against the night manifest (the 2 missing are the uploader's by-design skips)
gates / gates (push) Successful in 15m49s
2026-10-22 01:12:36 +02:00
admin dee26da3fa STATUS-HISTORY: the 2026-10-21 evening Updated paragraph moved here (STATUS rule)
gates / gates (push) Successful in 15m39s
2026-10-22 00:34:26 +02:00
admin c2c130ecbb R-1209 closed: the decoy suite gets 600 s in script-tests (the others keep 300 s; a time-out still fails) — it timed out on every CI push tonight
gates / gates (push) Successful in 15m40s
2026-10-22 00:26:53 +02:00
admin f576d7f7ba register: R-1211 (the household backup_failed mail names no app and no next step), R-1212 (two backup texts ignore the box language), R-1213 (operator detail left on two pages) — the night helper's findings; 10-localisation: W25 narrows the verbatim rule on the off-site page; 180 -> 183
gates / gates (push) Failing after 15m3s
2026-10-22 00:07:57 +02:00
admin 307d25b174 STATUS: the morning list after the night run (controller, agent, hub, Tester 2's install, the publishes); D22 and D26 out of the decide table (decided), D27 in (host swap on a no-swap host, R-1207)
gates / gates (push) Failing after 15m3s
2026-10-21 23:59:18 +02:00
admin 57c06ba99d CONTEXT + 08: tonight's rulings (D22, D26) and the CC-unattended R-1199 decision (the box's page is the list the hub mails by); evidence: rc.2 smoke on 9202, 9202 back as found, no-swap test tools 2026-10-21 23:45:05 +02:00
admin 9d3bf76def register: R-1177 R-1195 R-1178 R-1180 fixed on main and proven (D22), R-1194 cause found and fixed in the agent — all wait for the morning releases; R-1188 (a) fixed; new R-1207 (host swap on a no-swap host), R-1208 (node_down after an operator delete), R-1209 (CI decoy suite near its timeout), R-1210 (first-hour words waiting for a publish); 176 -> 180 2026-10-21 23:18:37 +02:00
admin 2293b252d5 docs: 07 §6.3 the put-back covers every restore and a power cut (D22, as built and proven on 9202); 03 guest swap = RAM and the host's swap behind it; capability map D16; soak provenance
gates / gates (push) Failing after 14m51s
2026-10-21 23:00:45 +02:00
admin 6b0d23c29c night 2026-10-21 evidence so far: soak box install + fixture agent (swap raised live), D22 proof on 9202 (put-back on a damaged unit and after a controller kill), R-1194 reproduced, R-1199 read, red-proofs, the W2 website patch (not applied)
gates / gates (push) Failing after 15m10s
2026-10-21 22:39:25 +02:00
admin 2fa5b8f3d8 ISO source (NOT built, NOT published): boot text in the te-form (W1); the linked banner clears the screen and its scroll-back so the spent pairing code is gone (W7); test with a decoy
gates / gates (push) Failing after 14m52s
2026-10-21 22:32:14 +02:00
admin f90e22293a hub 0.164.0 (unreleased): bind mail true when no box exists (W3), tries left on a wrong try (W4), use the newer mail hint (W5)
gates / gates (push) Failing after 14m55s
2026-10-21 22:18:38 +02:00
admin 3c620f8d2e hub 0.164.0 (unreleased): host_swap_short / host_swap_ok operator alarm from the agent's host and guest swap numbers (R-1192 follow-up); wire gate allowlists guests.spec.swap_bytes
gates / gates (push) Failing after 14m53s
2026-10-21 21:55:15 +02:00
admin dbfe5dac06 fixes 2026-10-21: teardown (demo-hp, hub, ep0 read-only), final evidence
gates / gates (push) Successful in 13m45s
2026-10-21 18:50:23 +02:00
admin 47fa1e621e fixes 2026-10-21: register (7 closed, R-1205/R-1206 opened, R-1183 VERIFY, R-1192 diagnosed), STATUS (not ready: one step — the agent swap fix), A12/A13 proven live, CONTEXT, report, evidence
gates / gates (push) Successful in 13m51s
2026-10-21 17:57:32 +02:00
admin ef4dbcf984 fixes 2026-10-21: evidence so far (R-1176 live proof, R-1192 freeze measured, R-1191 on 9202); design notes 03 §8a / 07 §7.0, §8.2a (D25) / 05 §16.5 (DST)
gates / gates (push) Successful in 13m34s
2026-10-21 16:00:36 +02:00
admin cd3d05981a manifests: hub 0.163.0 (operator yes, 2026-10-21)
gates / gates (push) Successful in 13m21s
2026-10-21 13:42:42 +02:00
admin 9688e76efd hub v0.163.0: a rebuilt box gets the household's sealed package back (R-1176); one run per daily job on clock-change days (R-1183); a scoped abandonment purge (R-1182); the new-link page says the old box is still linked (R-1203)
gates / gates (push) Successful in 13m23s
2026-10-21 13:36:45 +02:00
admin e07b077ac2 system poster refreshed 2026-10-21: one-file HTML + A1 PDF from Claude Design (data area, https only); R-1201 closed
gates / gates (push) Successful in 13m7s
Third export passed all Part A checks (documentation/audits/poster-refresh-2026-10-21/CHECKS.md).
STATUS poster line removed; facts file History line. Register 179 -> 178.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012qRErfCoiTkvDK9N5XHbzb
2026-10-21 13:25:43 +02:00
admin 7ad17be6bd rehearsal 2026-10-20: final report, STATUS (morning list, D24), capability map A12/A13 → PARTIAL (B7), stuck point 10
gates / gates (push) Successful in 13m35s
2026-10-21 11:47:40 +02:00
admin eaff4d890c teardown: secret count 24 2026-10-21 11:41:58 +02:00
admin 7a287e71fc rehearsal 2026-10-20: Part D teardown (machine done; hub host for the operator; ep0/Storage Box listed, nothing removed) 2026-10-21 11:41:51 +02:00
admin 6559aa6d2e rehearsal 2026-10-20: B7 — re-linked after the operator's press; the household cannot restore from off-site (R-1176 proven live)
gates / gates (push) Successful in 13m11s
2026-10-21 11:36:48 +02:00
admin a2d2569719 register: the ep0 WireGuard row is R-1204 (R-1201 was taken by a parallel session; the earlier commit message says R-1201)
gates / gates (push) Successful in 13m37s
2026-10-21 11:03:32 +02:00
admin ebf142d921 rehearsal 2026-10-20: B7 — box destroyed, reinstalled; the forgotten press is a dead end for the household (R-1203, P2) 2026-10-21 10:59:24 +02:00
admin 6d73110cd0 rehearsal 2026-10-20: B6 #10 + #15 — PASS 2026-10-21 10:59:24 +02:00
admin ff06ab52da rehearsal 2026-10-20: B6 #8 wrong clock — R-1201 (ep0 tunnel dead after a clock ahead, P2), R-1202 (clock behind: silent app stop, blames the hub) 2026-10-21 10:59:24 +02:00
admin a3be7525cd rehearsal 2026-10-20: B6 #7 DST live — PASS (one database copy at the second 02:30); #8/#10 expectations written before 2026-10-21 10:59:24 +02:00
admin 3decdf94a8 poster refresh 2026-10-21: export NOT taken in (6-file folder, 4 unasked text rewrites -> R-1201); OMV guide matches Ferenc's PDF (installer >= 1.33.0)
gates / gates (push) Successful in 13m21s
Part A checks + label diff + PNG crops in documentation/audits/poster-refresh-2026-10-21/.
Guide: HDD_PATH trouble item and "remote data area" sentence removed, first-days line added,
operator note dated. Register 178 -> 179.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012qRErfCoiTkvDK9N5XHbzb
2026-10-21 10:00:54 +02:00
admin 7a773207f8 STATUS: dated checks of 2026-10-21 done (R-895 read, R-1019 closed), R-435 moved
gates / gates (push) Successful in 13m28s
2026-10-21 07:54:41 +02:00
admin 19064ef1a7 register: R-1200 state word 2026-10-21 07:41:59 +02:00
admin 31d975c997 rehearsal 2026-10-20: B5 night (db/second drive/off-site PASS, data identical; whole-box not due → R-1200; host reboot = harness); due checks R-1019 closed, R-895 read, R-435 moved 2026-10-21 07:38:10 +02:00
admin 0d3f72e463 rehearsal 2026-10-20: journal — push held by due-checks gate 2026-10-21 02:58:05 +02:00
admin c4262a6f6c rehearsal 2026-10-20: night db copy OK; watcher logs recovered after an autostash froze them (harness slip) 2026-10-21 02:44:16 +02:00
admin a600d68d50 rehearsal 2026-10-20: night log copier added before the night (read-only)
gates / gates (push) Successful in 13m39s
2026-10-20 23:39:50 +02:00
admin 830ed9714e rehearsal 2026-10-20: report draft (night/morning parts pending); W26 marked fixed (unreleased)
gates / gates (push) Successful in 12m56s
2026-10-20 23:27:47 +02:00
admin 16188df64c rehearsal 2026-10-20: 44 more screenshots off the browser container, gallery rebuilt; A4 browser spike result
gates / gates (push) Successful in 12m48s
2026-10-20 23:16:20 +02:00
admin 0d4bfe7ca6 rehearsal 2026-10-20: pre-night household snapshot (byte compare vs manifest), B7 plan written before
gates / gates (push) Successful in 12m56s
2026-10-20 23:09:29 +02:00
admin 938afb7416 rehearsal 2026-10-20: B10 ten stuck points; capability map A15 updated (still PARTIAL: R-1192); clock tool for #7/#8/#10
gates / gates (push) Successful in 12m57s
2026-10-20 22:41:46 +02:00
admin c0b70b15ac rehearsal 2026-10-20: post-night scenario plan written before any result
gates / gates (push) Successful in 12m51s
2026-10-20 22:24:21 +02:00
admin ce8cf2c477 rehearsal 2026-10-20: B6 #1 README, journal to 20:14, B5 night expectations written before the night
gates / gates (push) Successful in 12m55s
2026-10-20 22:18:53 +02:00
admin 4a557f54a5 rehearsal 2026-10-20: B6 #1 power cut mid install, #6 off-site blocked, #14 kept data — evidence
gates / gates (push) Successful in 12m58s
2026-10-20 22:13:50 +02:00
admin 92de2d324e Rehearsal: B6 #12 DNS dead an hour (as expected; household told nothing); R-1199 (P2 VERIFY) the household's ticked alarms reached only the operator (1 of 9 mails); R-1189 (b) demo-hp journal: three skipped nights
gates / gates (push) Successful in 13m31s
2026-10-20 21:50:47 +02:00
admin 12090df2d9 Rehearsal: R-1197 (Nextcloud 413 over ~1 GB by WebDAV), R-1198 (a later backup disk is never offered as the whole-box target); R-1194 facts
gates / gates (push) Successful in 13m23s
2026-10-20 20:50:36 +02:00
admin 434fc7cc02 Rehearsal: B6 #3 disk full (as expected: warned at 89 %, refused before writing, old copies kept, healed); #12 DNS dead started
gates / gates (push) Successful in 13m23s
2026-10-20 20:43:29 +02:00
admin 53d962d95b Volunteer guide (HU+EN): the console's „linked" text replaces the stale „known bug: the code stays on screen" note; „Mentés most" takes ~2 min with eight apps (measured in the 2026-10-20 rehearsal)
gates / gates (push) Successful in 13m37s
2026-10-20 20:12:20 +02:00
admin 836b47f1ec Rehearsal: B6 #16 English pass, #9 two-at-once (as expected), #13 second drive (not offered as backup target), #4 drive pulled (handled well; last uploads 0 bytes → R-1196 P3); B9 W24-W29
gates / gates (push) Successful in 13m21s
2026-10-20 20:07:30 +02:00
admin bb187f3503 Rehearsal: B8 all seven restore paths pass (NFC/NFD: both names return); B6 #2 power cut mid-restore → R-1195 (P2, proven live: the app runs on the backup's data, the page says only interrupted); B6 #5 slow line; register 173 -> 174
gates / gates (push) Successful in 13m31s
2026-10-20 19:43:31 +02:00
admin 9db8aacdad Rehearsal: R-1193 corrected and narrowed to P3 (the operator mailbox shows host_stale at 13:33 and app_oom; the household was told nothing); R-1192 second freeze + the app_oom hint; R-1194 (P3 VERIFY) fresh box's felhom-sshd config invalid; B6 #5 slow-line result
gates / gates (push) Successful in 13m17s
2026-10-20 19:10:42 +02:00
admin 78278502ac Rehearsal 2026-10-20: second freeze and plug pull, memory watch, Mentés most, B6 #5 started; B9 words W7-W23; tools
gates / gates (push) Successful in 13m11s
2026-10-20 18:32:20 +02:00
admin 2228b9f6b4 Rehearsal: R-1192 (P1) the 8 GB minimum box froze ~1 h on the family's first afternoon, plug pulled to recover; R-1193 (P2) the hub showed it OK throughout; register 170 -> 172; STATUS P1 line and a note on D5
gates / gates (push) Successful in 13m16s
2026-10-20 16:38:27 +02:00
admin ee743477b4 Rehearsal 2026-10-20 B2-B4: bound (operator passphrase), claimed, recovery code, family (accented names refused), second disk initialised, 8 of 9 apps (Jellyfin refused on memory), first uploads froze the box ~1 h (hub said OK); screenshots gallery
gates / gates (push) Successful in 13m8s
2026-10-20 16:34:30 +02:00
admin ad6397308c Rehearsal 2026-10-20: guide §3 quotes the console the box really shows; bind-page screenshots (codes never shown); app upload and gallery tools; R-1189 (d) (e) confirmed in code
gates / gates (push) Successful in 13m17s
2026-10-20 13:17:29 +02:00
admin d750a90d0e Data-loss hunt: R-1188 (c) proven — a leftover .felhom-restoring blocks every retry of the second drive's whole restore
gates / gates (push) Successful in 13m33s
2026-10-20 13:11:32 +02:00
admin a942ae90d6 Data-loss hunt: R-1185 proven (an invalid dump replaces the good one, no error); R-1182 confirmed in code; DST replay of the controller scheduler (once each on 25 Oct; spring 02:30 dump collides with the 03:30 copy)
gates / gates (push) Successful in 13m44s
2026-10-20 13:07:39 +02:00
admin 20a77d367b Data-loss hunt: R-1180 proven against production code (the nightly rsync --delete mirror runs while a restore holds the backup flag and deletes the files being restored)
gates / gates (push) Successful in 13m20s
2026-10-20 13:02:50 +02:00
admin a57a8363df Data-loss hunt: R-1191 (P1, proven live on 9202) — a .fab import makes a family-only app public (stranger 401 -> 200); R-1187 narrowed; register 169 -> 170; STATUS P1 line
gates / gates (push) Successful in 12m52s
2026-10-20 12:58:30 +02:00
admin bffe2d9dea Data-loss hunt 2026-10-20 C4: register 153 -> 169 (opened R-1175..R-1190: two P1 — shares second copy pruned by another drive's group, box rebuilt after the host delete cannot reach its off-site history; R-1177 proven LIVE on 9202 — a failed local restore destroys live data with no copy); STATUS: the P1 line and decisions D22, D23
gates / gates (push) Successful in 13m20s
2026-10-20 12:31:34 +02:00
admin 242584af4e Status view report: CI jobs 2131-2133 green, by head_sha
Claude-Session: https://claude.ai/code/session_012qRErfCoiTkvDK9N5XHbzb
2026-10-20 12:28:00 +02:00
admin 43e5a5496d Data-loss hunt 2026-10-20: four reviewers' candidates, C2 verdicts, proofs (D1 shares second copy pruned, K1 escrow unreachable after host delete, R5 whole restore leaves app stopped while saying untouched, T9 hub jobs run twice on 25 Oct); rehearsal A5 manifest, B1 notes, B9 words, tools
gates / gates (push) Successful in 12m41s
2026-10-20 12:13:19 +02:00
admin c701b2530b Status view: session report (REPORT-status-view-2026-10-20.md); STATUS points to the rehearsal's screenshot folder
gates / gates (push) Successful in 12m53s
Claude-Session: https://claude.ai/code/session_012qRErfCoiTkvDK9N5XHbzb
2026-10-20 12:06:50 +02:00
admin 54d1ee8210 Status view, part 4 (the instruction): CLAUDE.md end-of-session checklist — how to update STATUS.md in five lines (replace the Updated paragraph, never prepend; a dated morning list in do; remove what you finished; the decide-row rule; the gate checks it, R-1174). A format rule under PROMPT-TEMPLATE §9 rule 9 (decision 150); no safety rule loosened
gates / gates (push) Successful in 12m43s
Claude-Session: https://claude.ai/code/session_012qRErfCoiTkvDK9N5XHbzb
2026-10-20 11:59:54 +02:00
admin 1d7071e8fd Status view, part 3 (the gate): scripts/status_view_gate.py in repo_gates.py (fast: pre-push + CI) — size outside decide, one Updated paragraph, English outside quotes, cited rows open, no decided row in decide, the P1 and every operator-owned row on the page; WARN-FIRST until 2026-10-27, then it refuses; 39 decoy cases (red-proved per check and on both sides of the date); filed R-1174 (P4, Process & tooling) + its due check 2026-10-27; R-919 owner cell -> operator (the upload and the look); register 152 -> 153
Claude-Session: https://claude.ai/code/session_012qRErfCoiTkvDK9N5XHbzb
2026-10-20 11:59:48 +02:00
admin 9ec5f76a88 Status view, part 2 (the page): STATUS.md rebuilt from the register — 107 lines / 31,695 bytes -> one screen (50 lines, about 5,400 characters outside the decide table); decide table: 8 open decisions (13 decided rows out, rulings checked in CLOSED-ITEMS); do list checked against the hub and DooPlex (7 done items out, 7 'still needed? yes/no'); old page byte for byte in STATUS-HISTORY.md
Claude-Session: https://claude.ai/code/session_012qRErfCoiTkvDK9N5XHbzb
2026-10-20 11:59:41 +02:00
admin f84f94d987 Rehearsal 2026-10-20 (Tester4) + data-loss hunt: baselines, B1 install evidence, A6 drawn order, tools, first two reviewers' candidates
No release, deploy or publish. Off-site is OFF for Tester4 and the owner passphrase is needed from the operator (bind blocked after 2 of 5 tries).
2026-10-20 11:57:55 +02:00
admin 2954b0be3a Status view, part 1 (register pass): register 156 -> 152 (closed R-1121 file gone, R-1118 retirement done, R-231 in git + green run; R-993 merged into R-991); R-132 narrowed (rotation done, hash field left), R-1049 -> WATCHING (fix live since hub 0.151.0); hub + local read evidence
gates / gates (push) Successful in 12m40s
Claude-Session: https://claude.ai/code/session_012qRErfCoiTkvDK9N5XHbzb
2026-10-20 11:43:58 +02:00
admin 5c487313fa Night brief part 2 (run by day 2026-10-20): register 160 -> 156 (closed R-1001, R-1011, R-1167, R-1070, R-1010, R-893, R-1169; opened R-1171..R-1173; R-1019/R-462 narrowed, R-49 READY -> D21, R-1083 re-stated); 07 §6.3 option A as the design; STATUS morning list; report part 2; P4 Immich plan, P7 release evidence
gates / gates (push) Successful in 12m57s
2026-10-20 11:06:00 +02:00
admin bcaa9576b2 Night 2 evidence: P2 R-893 option A live proof on 9202 (forced failure put back exactly; normal restore OK; teardown); P5 R-462 coverage and currency list
gates / gates (push) Successful in 12m48s
2026-10-20 10:47:37 +02:00
admin 301f4e9e92 Night 2 (2026-10-20) evidence: P1 hub 0.162.0 deploy, installer 1.35.0 publish, agent 0.161.0 release+delivery+host_busy live; P2 R-893 red-proofs; P3 R-1019 classifier before/after
gates / gates (push) Successful in 13m14s
2026-10-20 09:32:53 +02:00
admin b0ae2c574a installer 1.35.0 published (operator yes, 2026-10-20): both webpage git-sync pins -> installer-v1.35.0 (R-1167)
gates / gates (push) Successful in 13m6s
2026-10-20 08:13:57 +02:00
599 changed files with 79242 additions and 565 deletions
+8 -1
View File
@@ -138,7 +138,14 @@ something, not only sessions that touch `documentation/` — which is why it is
and the eleven names head `OPEN-ITEMS.md`, and `register_shape_gate.py` refuses anything else. and the eleven names head `OPEN-ITEMS.md`, and `register_shape_gate.py` refuses anything else.
- **Root `STATUS.md`** — at the end of every session in which something shipped, broke or was - **Root `STATUS.md`** — at the end of every session in which something shipped, broke or was
decided. It is a **view** of `OPEN-ITEMS.md`; nothing may exist only there. One screen, written for decided. It is a **view** of `OPEN-ITEMS.md`; nothing may exist only there. One screen, written for
the operator in plain language, and deliberately **not** `CONTEXT.md`. the operator in plain language (STE English), and deliberately **not** `CONTEXT.md`. How to update it:
1. **Replace** the „Updated" paragraph (at most 3 sentences, the newest change only) — never prepend; the old one
moves to `documentation/backlog/STATUS-HISTORY.md` under a dated heading.
2. A morning list goes under a dated sub-heading in „Waiting on you — do".
3. Remove what you finished: a done „do" item, a fixed „broken" line, a decided row (its ruling is in its row first).
4. A decide row: the question in one answerable sentence, at most 2 options, CC's pick first with the reason, the
other option's cost, what happens if the operator does nothing, and who is blocked.
5. `scripts/status_view_gate.py` checks it on every push (warn-first until 2026-10-27, then it refuses; R-1174).
- **The golden, weekly** (operator ruling 2026-10-15 11:26, `09` §3 decision 250; decision 7 stands): **a controller - **The golden, weekly** (operator ruling 2026-10-15 11:26, `09` §3 decision 250; decision 7 stands): **a controller
release needs no golden** — the floor carries it, MinAgent declared, after the three test boxes (`09` §3 decision 250 release needs no golden** — the floor carries it, MinAgent declared, after the three test boxes (`09` §3 decision 250
item 2). The golden is baked **every Sunday night** when the floor is above it (`runbooks/nightly-rotation.md`) and item 2). The golden is baked **every Sunday night** when the floor is above it (`runbooks/nightly-rotation.md`) and
+35
View File
@@ -16,6 +16,41 @@
> and holds nothing of its own; this file does hold its own content, namely the standing rulings below. > and holds nothing of its own; this file does hold its own content, namely the standing rulings below.
> **2026-10-21 night (unattended; nothing released, deployed, published or vouched).** Operator rulings recorded:
> **D22 = yes (20:05)** — D20's put-back extended to the local unit restore, the second-drive restores and a restore cut by
> a power loss (`07` §6.3, the 2026-10-21 `[DESIGN]` paragraph; controller `d582feb`, proven on 9202 with 0.338.0-rc.1:
> a damaged unit → the live data back and running; a controller killed after the volume step → put back on start);
> **D26 = A (20:03)** — release and vouch the agent with the swap fix in the morning; a host-swap warning follows (agent
> `0da792e` reports `host_swap_mb`/`guest_swap_mb`, one WARN; hub 0.164.0 built, not deployed: operator-only
> `host_swap_short`; controller install-page sentence). **Decided by CC unattended — operator may reverse (R-1199):** the
> box's notification page is the source the hub mails the household by: a box whose household never saved a choice now
> pushes the default list its page shows at start (no address; the hub's no-clobber guard keeps the stored one) —
> `08` (the household decides; „the box's page is the source"). Measured: Tester4's hub list was saved 2026-10-18 and
> lacked disk warning, drive disconnected and second-drive failure, so the household got 1 alarm of 9. Built tonight:
> controller `0.338.0-rc.2` (`sha256:ae52c93e…`, all of tonight's controller work, smoke-tested on 9202, 9202 back on
> 0.337.0), hub `0.164.0` (`sha256:931594e3…`, manifest still 0.163.0), agent candidate `0.162.0` from `0da792e`
> (sha256 `5fd7a23b…`, not published). The 8 GB soak with the swap fix: `audits/night-2026-10-21/P1-soak/`.
> **2026-10-21 (day brief, before Tester 2) — hub 0.163.0 DEPLOYED (operator yes, 11:46 UTC); controller 0.337.0 released
> (three test boxes 14:06 UTC, global floor after the freeze proof); agent `286a7e3` committed, NOT released.** Operator rulings
> today: **D24 = yes** (the hub's DST double run fixed: `nextDailyRun`, calendar day+1, `05` §16.5); **D25** (a second-drive
> copy whose SOURCE is gone is kept until the household releases it — `07` §8.2a; controller `tier2_kept.go`, the restore
> page's „Megőrzött másolatok"); the **R-1176 shape** = the hub CARRIES the household's retained sealed package to the
> rebuilt host (`store.ReattachRetainedEscrow`: one live host, no package held, the newest escrow-ACKED deletion; moved in
> one tx at the agent's first report and the controller's report; never opened/logged; two live hosts → refused + operator
> mail) — `03` §8a. Proven live on demo-hp (Tester4 from the public ISO): no fresh key minted, recovery offered, wrong code
> refused, right code opened the history, Nextcloud 3 031/3 033 files byte-identical, off-site appended 63 → 64
> (`audits/fixes-2026-10-21/P1-live/`). R-1182: retained rows carry `repo_tag` (NULL legacy / 'live' / the set-aside path
> tagged at the hub's move-aside); the abandonment purges only rows tagged with a path the hub DELETED; the ACK says
> `abandon_purge_done`. R-1203: the new-link page tells the holder of a real spent link that the old box is still linked;
> operator mail `bind_old_host_linked`. **R-1192 diagnosed with numbers:** Proxmox's LXC default swap cap (512 MB) — the
> agent never set `swap` — turned Immich's model load (+1.15 GB) into a code-page refault thrash at the 4096 MB guest limit
> (150 000–600 000 refaults and ~1 GB of reads per 10 s, memory „full" 92–95 %, load 155, dashboard dead 6½ min, 0 OOM
> kills); `pct set --swap 4096` live ended it within a minute. Agent fix (swap = RAM at bring-up, at RAM resize, once at
> start) committed; its release is the operator's. The controller's „Normál használat mellett ez nem okoz problémát" is
> gone (0.337.0). Observed and filed: R-1205 (PBS re-issue after an un-acked second install), R-1206 (future-dated
> snapshots). Report: `REPORT-fixes-2026-10-21.md`.
> **2026-10-19 (night) — controller 0.335.0 is the floor (MinAgent 0.158.1, 18:39 UTC); hub 0.162.0 BUILT, not deployed; > **2026-10-19 (night) — controller 0.335.0 is the floor (MinAgent 0.158.1, 18:39 UTC); hub 0.162.0 BUILT, not deployed;
> agent `56aabc7` and installer 1.35.0 committed, not released/published.** R-921 (operator answer „add one field"): the > agent `56aabc7` and installer 1.35.0 committed, not released/published.** R-921 (operator answer „add one field"): the
> agent serves its host-wide heavy-operation gate as `host_busy` on `GET /backup/status`; the controller reads it before > agent serves its host-wide heavy-operation gate as `host_busy` on `GET /backup/status`; the controller reads it before
+131
View File
@@ -0,0 +1,131 @@
# REPORT — the day brief of 2026-10-21: fix the four serious problems from the rehearsal, and the Sunday clock change
Separate from the shared `REPORT.md` on purpose: another session works in this clone (repo rule: a second session writes
`REPORT-<topic>.md`). Evidence: `documentation/audits/fixes-2026-10-21/`. Design read first: `03` §8a, `07` §7.0/§6/§8.2,
`05` (scheduler, escrow store), `04` (self-scoped reads), `09` §3 decisions D19, 78, R-241.
## 1. Five lines
1. **R-1176 (a rebuilt box gets its off-site backups back) — FIXED AND LIVE.** Hub 0.163.0 (operator yes, 11:46 UTC);
proven end to end on demo-hp: no fresh key, recovery offered, wrong code refused, right code opened the history,
Nextcloud 3 031 of 3 033 files byte-identical, the next off-site runs appended (63 → 64).
2. **R-1175 (shares on two drives) — FIXED AND LIVE.** Controller 0.337.0 (global floor 15:54 UTC), D25: a gone source
keeps its copy until the household removes it.
3. **R-1191 (an imported app keeps its family lock) — FIXED AND LIVE.** Controller 0.337.0; proven on 9202: a stranger
gets 401 after export → remove → import (it was 200).
4. **R-1192 (the 8 GB freeze) — CAUSE MEASURED, FIX COMMITTED, NOT RELEASED.** The guest's swap was capped at Proxmox's
512 MB default; the agent fix (swap = RAM) is committed (`286a7e3`); its release and vouch are the operator's (D26).
Proven by applying the same setting by hand for 2 hours of the same uploads.
5. **Ready for Tester 2: NO — one step left:** the agent release with the swap fix. Everything else on the P1 list is live.
## 2. R-1192 — the mechanism, the numbers, the fix
**One sentence:** the guest's swap is capped at 512 MB (Proxmox's LXC default — the agent never set `swap`), so when
Immich loaded its analysis models (+1.15 GB) on top of 8 apps, the 4096 MB guest could evict only program code, and
~25 containers re-read their code from disk without end.
| | the freeze (13:26–13:34 UTC, cap 512 MB) | the proof (13:43–15:43 UTC, cap = RAM by hand) |
|---|---|---|
| guest private memory (anon), max | 3 585 MB | 2 782 MB |
| guest swap used, max | 511 MB of 512 (full) | 2 555 MB |
| refaults per 10 s, max | 2 741 774 | 187 462 |
| disk reads per 10 s, max | 12 GB | 0.9 GB |
| memory „full" pressure (avg10), max | 96 % | 12 % |
| IO pressure, max | 97 % | 53 % |
| load, max | 162 | 33 |
| guest „limit hit" events per 10 s, max | 3 889 | 0 |
| OOM kills | 0 | 0 |
| dashboard (DooPlex probe every 10 s) | dead 13:28:18–13:34:48 | every probe answered, slowest 1.26 s |
The experiment between them: `pct set 9201 --swap 4096` at 13:34:06, uploads still running → memory „full" 96 % → 1.7 %
within 2 minutes, the dashboard back 42 s after the change. The host had 2.2 GB RAM and 7.4 GB swap free throughout.
Load for the proof: the same uploads as fresh work, repeated — Paperless 110 cycles × 53 PDFs (emptied each time), Immich
111 new users × 306 photos (305 accepted each time; one photo is refused every cycle), two Nextcloud trees (WebDAV small
files, ~20 files/min under this load — slow, nothing lost). Method: a recorder on the box (`tools/rec.py`, 10 s, copied to
DooPlex every minute) and the second channel on DooPlex (`tools/outside.sh`: the dashboard's `/api/health` and QEMU's
disk counters from demo-hp).
**Fix:** agent `286a7e3` (commit only): `swap` = memory at bring-up, at a RAM resize, and once at agent start for every
pool guest whose swap is below its RAM (live, no restart, never lowers). Tests red-proved. The controller half (0.337.0):
„Normál használat mellett ez nem okoz problémát." is gone; the install page now says **„Az alkalmazások csúcsterhelése
meghaladhatja a rendelkezésre álló memóriát. Ilyenkor a doboz nagy feltöltések alatt lelassulhat, és akár percekig sem
válaszol. Ha ez gyakran előfordul, távolíts el egy alkalmazást."** No catalog change and no admission-ceiling change:
the measured cause is the swap cap, and the order of preference (catalog → controller → agent) gave way to the proof.
R-1193 (an earlier, plainer alarm): a design note in its row; build it only if a freeze is seen after the agent release.
## 3. R-1176 — the shape, the order check, the live proof
**Shape (operator ruling):** `store.ReattachRetainedEscrow` moves the newest retained row of the customer's newest
escrow-ACKED host deletion into `host_escrow` for its one live host, in one transaction (no other live host, no live
package, the deleted host not live again, an identity blob present). Called on the agent's host report and before the
controller ACK's escrow status. Never opened, re-sealed or logged; audit `escrow_carried` (host ids + hash); two live
hosts → `escrow_carry_refused` mailed once. Self-scope unchanged: the new host reads only rows attached to it (its
retained GET for the old host is 403).
**Order check (measured, not assumed):** the hub carried at the agent's FIRST report (12:04:21 UTC); the controller's
settle gate went GO at 12:06:25 after its first ACK; at 12:06:28 it logged „NOT minting a repository password: the hub
holds a sealed recovery package". So no controller change was needed; 0.337.0 still records the escrow before
`SetFloor` (a pinned belt for a hub that carries later).
**Live proof (demo-hp VM 9321, Tester4, public ISO 1.29.0, no operator press before the install):** bind (one OCR misread
of the pairing code cost one of five tries — harness), claim, the dashboard offered „A korábbi, házon kívüli mentéseid
visszaszerezhetők"; a wrong code: „A megadott helyreállítási kódot nem fogadtuk el … Semmi nem változott"; the right code
unlocked 9 groups; Nextcloud reinstalled („előbb telepítsd újra" is the page's own order), „Teljes visszaállítás" with
the placement confirm → „3113 fájl és 2 adatkötet és az adatbázis visszaállítva"; **sha256 through Nextcloud's own WebDAV
against the rehearsal's A5 manifest: 3 031 of 3 033 byte-identical** — the 2 missing never reached the old box (the 1.5 GB
file Nextcloud refused, a symlink rclone skips); 4 extra files are the rehearsal's later uploads. Off-site: two runs, 63 → 64
snapshots in the same 1.4 GB store, `orphaned: false`; the hub's full log has no set-aside and no key change.
## 4. Versions, commits, tests
| Component | Version | State | Commits |
|---|---|---|---|
| hub | 0.163.0 | **deployed** 11:46 UTC (operator yes), Synced/Healthy | `9688e76e`, manifest `cd3d0598` |
| controller | 0.337.0 (`sha256:264af313…`) | **released**: rc.1 on 9202; per-box floors 13:59 → three test boxes on 0.337.0 by 14:06; box walk green; **global floor 0.337.0 / MinAgent 0.158.1 at 15:54:29 UTC**, read back; per-box floors cleared | `8f5bea0`, `5929b8e`, `7848d4a` |
| agent | (0.162.0 when released) | **committed, not released** | `286a7e3` |
| catalog, installer | — | unchanged | — |
Tests (all pass; each red-proved — seen failing first): hub `TestR1183_*` (3), `TestR1176_*` (store 7, api 3),
`TestR1182_*` (store 2, api 1), `TestR1203_*`; controller `TestR1175_*` (3), `TestR1181_*` (2), `TestD25_*` (3),
`TestR1179_*`, `TestR1188c_*`, `TestR1191_*` (2), `TestR1176_EscrowIsRecordedBeforeTheFloorIsKnown`, `TestR1182_*` (2);
agent `TestR1192_*` (2) + the resize test's swap assertion. CI: green by job id for every push (see §5).
## 5. Morning list, teardown, rows, observations
**Morning list (operator):** D26 — release + vouch the agent with the swap fix; delete host **Tester4-f6b67c** (asked
today); R-1205 (the whole-box tier waits for „Re-issue PBS credentials" after a failed second install); the
`REHEARSAL_2026_10_20_*` / `FIXES_2026_10_21_*` names in sec.
**Operator acts today:** the hub-deploy yes; the root credential reveal for the freeze test (~2 min); the host delete
(done 16:49 UTC — the delete appears 6 minutes after a box goes off, R-30's rule).
**Teardown (three layers):**
- **Machine (demo-hp, Tier 0):** VM 9321 `rh1021-box2` and helper CT 9331 `rh1021-loader` destroyed with `--purge` at
16:27:40 UTC (names checked first); no image left on `nvme-scratch`; 341, 9201, 9202, 9401 untouched. 9202 moved from
`0.337.0-rc.1` to `0.337.0` by hand (it does not take the hub's floor). The public ISO 1.29.0 was only read.
- **Hub:** the operator deleted host Tester4-f6b67c (16:49:37 UTC, escrow acknowledged): no Tester4 host, no current
package, the household's package retained under Tester4-f6b67c with the same hash, `repo_tag` 'live' — a future
rebuild carries it again; the WireGuard peer gone (`P6-release/05-teardown-hub.txt`).
- **ep0 / Storage Box (read only, nothing removed):** Tester4's off-site store gained 2 snapshots (63 → 64, normal
traffic); the hub's registrar pinned the rebuilt box's key and removed the dead box's line (normal enrollment). No PBS
descriptor was made (R-1205), so nothing new on ep0's PBS.
**Rows:** before 182 (181 after a parallel session closed R-1201); after 176; opened 2 (R-1205, R-1206); closed 7 (R-1176,
R-1203, R-1175, R-1179, R-1181, R-1182, R-1191). R-1183 → VERIFY (due 2026-10-26); R-1192 → WAITING-ON-OPERATOR;
R-1188 (c) fixed. Capability map A12, A13 → PROVEN-LIVE. `unproven.py --summary`: unchanged, 35 of 55 not walked.
**Observations:**
- PBS auto re-issue refused for the rebuilt Tester4 (latest deletion un-acked) — **FILED: R-1205**.
- Future-dated snapshots from the clock test show as „Legutóbbi mentés: 2026. 10. 28." — **FILED: R-1206**.
- The pairing-code OCR misread one character — **NOT-A-FINDING:** harness (a human reads the screen); corrected against the hub's record.
- Immich's first admin needs the household's dashboard session (the setup gate) — **NOT-A-FINDING:** designed.
- The full restore asks to confirm a new place (backup on `hdd_1`, now the internal disk) — **NOT-A-FINDING:** designed (R-351).
- Nextcloud's small-file WebDAV upload ~20 files/min under heavy concurrent load — **NOT-A-FINDING:** slow, not lost; the new install-page sentence says big uploads can be slow.
- `kubectl logs -l` reads only 10 lines — **NOT-A-FINDING** (a tooling trap; saved to memory, two checks redone over the full log).
- Tester 2 last reported 2026-10-04 on 0.292.0; the floor reaches it only when it reports — **NOT-A-FINDING** (R-1139 covers the box).
**Secrets:** names only — `FIXES_2026_10_21_PAIRING_CODE`, `FIXES_2026_10_21_BIND_TOKEN`, `FIXES_2026_10_21_SETUP_CODE`
(one-time, spent), `FIXES_2026_10_21_ROOTPW` (the operator's reveal; the box is torn down). The bind link and the setup code
were read from the mailbox by the mail connector, so both one-time values entered this session's record (spent within
minutes, as in the rehearsal). A scan of all 45 evidence files for every secret used today: 0 hits. Nothing secret in any
commit.
+126 -1
View File
@@ -1,4 +1,129 @@
# REPORT — evening/night brief 2026-10-19 (one stop less per night, no false tunnel error, R-893 measured, hub and agent fixes ready, a scanner on every push, ep0's first weekly clean-up) # REPORT — the night briefs of 2026-10-19 and 2026-10-20 (part 2 run by day): one stop less per night, the safe undo for a failed off-site restore, hub 0.162.0 / agent 0.161.0 / installer 1.35.0 delivered, downloads out of the backups, a scanner on every push
# PART 2 — the night brief part 2 (run by day, 2026-10-20): your four answers, the safe undo, downloads out of the backups, Immich, the update test
**Safest readings taken alone (written first, as the brief asks):**
1. **R-1083 is not in agent 0.161.0.** An agent key would make 9202's controller book the agent's whole-guest backups,
one of which writes to ep0 — a change on ep0 the brief forbids; 9202 is also outside the `felhom` pool.
2. **All five downloaders' folders are `excluded`, not only the two named** (qBittorrent, Transmission; also NZBGet,
SABnzbd, MeTube): the brief's gate rule — a download folder is never optional or mandatory — covers all five.
3. **No update-test edge was run:** the bench (guest 9401) and the drill catalog are outside the brief's machines and
repos. The list done and the list left are written; the ask is on the morning list.
4. The brief was written for a night run and ran by day; its „stop by 05:00" was read as a time box, not a clock.
## Part 2 — in plain words
- **Morning list:** vouch agent 0.161.0 · decide D21 (Immich) · MeTube's videos out of the second-drive copy — keep? ·
the bench and the drill catalog for the update test — yes?
- **Hub 0.162.0 is live.** Its own deploy was the real restart test: Tester-2 (down since 4 October) got no repeat mail
— read in the hub and, separately, in your mailbox.
- **Agent 0.161.0 runs on demo-hp, Tester 1 and demo-felhom.** On demo-hp the agent said „busy" during a backup and
„free" before and after. **Not vouched — that is yours.**
- **Installer 1.35.0 is published.** The served file equals the tag's file.
- **A failed off-site restore now puts the app back exactly as it was** (your D20 = A, controller 0.336.0, the floor).
Proven on the scratch box: after a forced failure the database matched its fingerprints from before, the app ran,
nothing was held; a normal restore then worked.
- **Downloads are out of the backups** (torrent, usenet, MeTube): the controller's own check copies 0 of them now.
- **Immich's off-site copy:** 1.33 GB, of which 102 MB must survive. Two options; pick B (leave out the AI model files).
- **ep0's first weekly clean-up:** done this morning (Part 10 of part 1 below).
## Part 2 — each part
| Part | Done? | Why / how |
|---|---|---|
| 1a hub deploy | **done** | `manifests/hub.yaml` → 0.162.0 (`e1d349f1`), only Deployment/hub synced (the only OutOfSync); image `sha256:69285e64…` = the build; Synced/Healthy; reports from all three boxes; floor read 0.335.0; Day-0 agent list has no `-step1`. R-1001 + R-1011 closed. |
| 1b agent release | **done** | R-1083 left out (above). `release-agent.sh 0.161.0` from a clean clone; signed `agent_update` per box: demo-hp 06:38, Tester 1 06:54, demo-felhom 06:55 UTC, each COMPLETED, 76/76, committed. `host_busy` live; tunnel `absent` = the fixture row (9202's real Docker 29 answer); R-1010 by its tests. R-1070 + R-1010 closed. |
| 1b R-921 live | **not done** | The no-stop case happens only on the night schedule (a press backs up the local tier only). Row R-1173 with a dated check after demo-hp's next off-site night. |
| 1c installer | **done** | tag `installer-v1.35.0`, both webpage pins (`b0ae2c57`), webpage Deployment synced alone; served sha256 `f7a196fd…` = tag. R-1167 closed. |
| 2 safe undo | **done** | below. R-893 + R-1169 closed. |
| 3 downloads | **done, one read left** | catalog `c50adfc`, gate `download-binds`; classifier proof. demo-felhom (qBittorrent) runs its next second-drive copy 2026-10-21 03:30 — dated check on R-1019 (NARROWED). |
| 4 Immich | **done (plan only)** | below. R-49 → READY, D21. |
| 5 update test | **not done** | fixture side complete for every installed app; the edges need the bench / drill catalog (above). R-462 NARROWED. |
| 6 ep0 clean-up | **done** | part 1, Part 10. |
| 7 controller release | **done** | 0.336.0 below. |
## Part 2 — the safe undo, the 9202 proof (`07` §6.3; `audits/night2-2026-10-20/P2/`)
- Code `93fb251`: the fit check before the stop, the copy right after it (`backups/restore-undo/<app>/`), the put-back on
every failure branch, the hold only when the put-back fails, the copy removed on success. 11 tests; nine mutations red.
- **9202, rc.1** (a helper with the fences in full; Rallly + postgres 18; the same forced failure as on 19 October):
| | Live before | After the forced failure |
|---|---|---|
| public tables | 36 | 36 |
| markers | A, B, C, D | A, B, C, D |
| `r893_only_old` (backup only) | absent | **absent** (on 19 October: present) |
| schema / data / row-count md5 | `05828c77` / `e5c1a72f` / `d8bcbc4b` | **equal** |
| app | running | **running**, no hold |
| the copy | — | 69 MB in 0.9 s, removed after |
The sentence: „…visszaállítása sikertelen. Nem változott semmi: az alkalmazás adatai visszakerültek a visszaállítás
előtti állapotba, és az alkalmazás újra fut". Then a normal restore of the same snapshot: the snapshot's content
(A–C, the backup-only table present), the app running, the copy removed. 9202 back on 0.331.0 as found.
- Not measured live (each pinned by a red-proved test): the version-change branch, the `_files` sentence, the refusals
before the stop, a failing put-back, the operator event (9202 has no hub).
## Part 2 — download binds (`15` §7; `audits/night2-2026-10-20/P3/`)
| App | Bind (userdata) | Class now | Copied by a tier |
|---|---|---|---|
| qBittorrent | media/torrents | excluded (was optional) | none (was the second drive) |
| Transmission | media/transmission, media/transmission/watch | excluded, excluded | none |
| NZBGet | media/nzbget | excluded (was optional) | none |
| SABnzbd | media/usenet | excluded (was optional) | none |
| MeTube | media/metube | excluded (was optional) | none |
| Sonarr, Radarr | media (rw) | excluded | none |
| Jellyfin, Plex, Emby | media (:ro) | excluded | none |
| Prowlarr | no userdata bind | — | — |
## Part 2 — Immich (`audits/night2-2026-10-20/P4/immich-offsite-plan.md`)
Model cache 824 MB (62 %) · raw database folder 283 MB (21 %) · Immich's own backups 115 MB (9 %) · logical dump 54 MB
· originals 49 MB · thumbnails 2 MB. **A** (leave out all three re-fetchable parts, −92 %) needs a never-run restore
from the dump alone on the vectorchord image; **B** (only the model cache, −62 %) needs one restore with an empty cache.
**Pick B now**, A's other parts after their proof.
## Part 2 — update-test edges
None run (above). Installed apps: 16; with a fixture: 15; without: bentopdf (keeps no data). Upstream within a major,
waiting for a tested step: PrivateBin 2.0.6→2.1.0, Home Assistant 2026.9.4→2026.10.2, RomM 5.3.1→5.4.0, Plex
1.41.4→1.43.4, Kimai 2.67.0→2.70.0 (`audits/night2-2026-10-20/P5/r462-list.txt`).
## Part 2 — versions, commits, rows
- Hub 0.162.0 deployed (`e1d349f1`). Agent 0.161.0 released (`5152123`, tag `v0.161.0`; notes `4e1621c`). Installer
1.35.0 published (`b0ae2c57`). Controller 0.336.0 the floor (`93fb251`; notes `aa9e09e`, `7ca59e2`), image
`sha256:93cd67d8…8d73`. Catalog `c50adfc`. Evidence `301f4e9e`, `bcaa9576`.
- i18n: no page template changed; seven Go-side keys (HU + EN), one rewritten (R-1169); no fixture changed.
- **System poster:** no listed fact changed — none of the three.
- **Register 160 before, 156 after. Closed 7:** R-1001, R-1011, R-1167, R-1070, R-1010, R-893, R-1169. **Opened 3:**
R-1171 (demo-hp's local copy never restore-tested), R-1172 (the put-back logs at ERROR; an empty folder stays),
R-1173 (R-921's no-stop not yet seen live). Re-stated: R-1019 (NARROWED, dated 2026-10-21), R-1083, R-462 (NARROWED),
R-49 (READY, D21). **Both briefs together: 161 before, 156 after; closed 11, opened 6.**
## Observations — part 2
1. demo-hp's local whole-guest copy is never restore-tested: one copy kept, a 24 h settle, replaced at 24 h. FILED: R-1171
2. A successful put-back is logged at ERROR, and an empty `backups/restore-undo/` folder stays after a restore. FILED: R-1172
3. R-921's no-stop branch has no live observation yet (it needs a night schedule). FILED: R-1173
4. R-462's „only three apps are tested" was out of date: 58 apps have fixtures on both venues. NOT-A-FINDING: the row is re-stated with today's list.
5. The hub deploy at 06:03 UTC removed the old pod's log, which held the 07:10 Budapest off-site audit lines of today (R-895). NOT-A-FINDING: R-895's dated check is 2026-10-21; that run happens in the new pod.
6. In step 4 of the 9202 drill the data-only md5 differed from a read taken before snapshot 2, while the schema and every row count were equal. NOT-A-FINDING: the two reads were at different moments (the read, then the snapshot's own dump); the claim under test — before and after the forced failure — matched on all three fingerprints.
7. The helper's brief expected 37 tables after the normal restore; snapshot 2 has 36. NOT-A-FINDING: last night's 37 was the mixed state, not the snapshot (my arithmetic, corrected).
8. The first background wait loop for demo-hp's agent update printed a time and stopped before the box had updated. NOT-A-FINDING: the version was read directly and a foreground loop used instead.
9. A Python edit of the controller CHANGELOG failed on a quote character, and the commit carried only REPORT.md. NOT-A-FINDING: fixed by the next commit (`7ca59e2`).
## Part 2 — teardown
- Machine: 9202 back on 0.331.0 as found (Rallly removed through the product, the SFTP target, images and files removed
by name, settings and the off-box folder put back); demo-hp 9201's probe script removed.
- Host: demo-hp `/tmp` copies removed; nothing on `pve/data`; no whole-guest backup of 9202.
- Hub: per-box floors set and cleared; the global floor raised; nothing else.
- ep0: read only.
---
# PART 1 — the evening/night brief of 2026-10-19
## In plain words ## In plain words
+48
View File
@@ -0,0 +1,48 @@
# REPORT — system poster refresh check + OMV guide fixes (2026-10-21)
Baseline `main` = `7a773207`. Own clone; documentation only; no machine, box or release touched.
Binding: `documentation/architecture/felhom-system-poster.facts.md`.
## Plain words
- **The new poster is in the repository**, the web page and the PDF. It took three exports.
- The first export was a folder of 6 files, and Claude Design had changed four texts. Nothing was committed then
(commit `3decdf94`, with the OMV guide fixes and R-1201).
- The second export was one file; two texts were still short. The third export passed every check.
- The STATUS line „System poster needs a refresh" is gone. R-1201 is closed.
## Part A — checks
Full table, all three rounds: `documentation/audits/poster-refresh-2026-10-21/CHECKS.md`.
Round 3: file PASS · shape: plain static HTML (better than the old loader) · two changes PASS · nothing lost PASS ·
secrets PASS · look PASS (PDF; HTML render not checked, no browser on DooPlex).
## Part B — committed (second commit)
- `documentation/architecture/felhom-system-poster.html` replaced; `felhom-system-poster.pdf` added (6.25 MB, A1 size).
- Facts file: one History line.
- STATUS: the poster line removed. R-1201 moved to `CLOSED-ITEMS.md`.
- Evidence: round-3 hashes, text diff, three PNG crops.
## Part C — `documentation/pilot/GUIDE-omv-kvm-hu.md` (first commit, `3decdf94`)
1. „Ha valami nem megy" item 7 (HDD_PATH error) removed; 8 → 7.
2. Jellyfin: „Ha a géped egy korábbi telepítővel…" sentence removed.
3. Memory bullet: „Az első napokban kevés alkalmazással indulunk; a nagy fotógyűjtemény feltöltését együtt csináljuk." added.
4. Operator note: dated line, point (4) obsolete since installer 1.33.0; history kept.
Re-checked: `scripts/iso/felhom-bootstrap.sh:165` fetches `https://felhom.eu/scripts/felhom-host-install.sh`, which
serves `SCRIPT_VERSION="1.35.0"` (≥ 1.33.0). `https://felhom.eu/letoltes` lists `felhom-installer-1.29.0-pve9.2-1.iso`
and `dceacae5…4829`, the same as the guide's step 3.
## Rows
Opened R-1201 (first commit), closed R-1201 (second commit). Register 178 → 179 → 178.
## Claims in the task that were wrong
- The PDF is not A2: 2400×1696.5 pt ≈ A1 size.
- `claude/DESIGN-poster-refresh-2026-10-21.md` is not in this repository: `claude/` is the planning project's folder
(as for `claude/MONTHLY-security-retest.md`).
- felhom.eu has no root `CHANGELOG.md`; this repo records sessions in `REPORT-<topic>.md` files, so this is one.
## Notes
- The pre-push hook refuses a clone outside `/mnt/5_hdd/felhom.eu`; the clones were moved under it. No `--no-verify`.
## Teardown
Provisioned nothing. Scratch copies and clones deleted. `~/temp/` untouched.
+158
View File
@@ -0,0 +1,158 @@
# REPORT — Tester-2 rehearsal with a pretend household + an independent data-loss hunt (2026-10-20 → 21)
> Run 2026-10-20 09:30 UTC → 2026-10-21 ~09:50 UTC. Drafted before the night, finished in the morning. This repo's
> shared `REPORT.md` was not touched: a parallel session works in the same clone, and the repo rule says the second
> session writes `REPORT-<topic>.md` only (the brief asked for both; this line is why one was skipped).
> Evidence: `documentation/audits/rehearsal-2026-10-20/` (journal, A0–B10, screens) and
> `documentation/audits/dataloss-hunt-2026-10-20/` (C1–C3). Architecture read for the claims: `07-backup-architecture.md`,
> `10-localisation.md`, `00-capability-map.md` (A15).
## Five sentences
A pretend household installed a box from the public ISO, linked it, claimed it, put eight apps and about 2 GB of family
data on it, used it for a day, and every customer restore path brought its data back. The first night's backups, the
clock change, three days off and power cuts in the middle of installs and catch-ups all worked, and the morning data was
byte-identical to the evening's. But the 8 GB minimum box froze twice on its first afternoon and only a plug pull helped
(R-1192), and the household's warning mails went to the operator instead (R-1199). When the box died, the household could
not even start again without the operator, and after the operator's press the rebuilt box could not reach its off-site
history at all — proven live (R-1176, P1). Nothing was released; 29 new register rows carry the rest.
## The ten places a household gets stuck
`documentation/audits/rehearsal-2026-10-20/B10-stuck/README.md`. In short:
1. The box froze twice on the first afternoon (≈1 h and ≈35 min); only pulling the plug helped (R-1192, P1).
2. The bind page needs an owner passphrase the household does not have — 2 h 4 min waiting for the operator.
3. A power cut during a full off-site restore leaves the app on the backup's older data; the page only says „interrupted" (R-1195).
4. A pulled drive breaks the last uploads (0 bytes or gone) although each was confirmed (R-1196).
5. The family's warning mails went to the operator, not the family (R-1199).
6. „Ödön" and „Írisz" are refused as family names (W12).
7. A new drive needs a „mount name" nobody knows; the button silently does nothing (W13).
8. Sharing waits for ever for a password the page never asks for (W21).
9. Errors arrive as raw technical English (W22, W25).
10. After the box died, the household was promised a mail that never came, and after the operator's press the rebuilt box
could not reach the family's off-site data (R-1203, R-1176). (Close behind: three apps' wrong „first steps", W17–W19.)
## Data-loss findings, ranked
| row | Sev | harm | reproduced | design / defect |
|---|---|---|---|---|
| R-1175 | P1 | shares on two drives sharing one second drive: every night one drive's share copy is pruned; the night a drive dies its copy is deleted | unit test on production code (`C3-D1-shares-tier2/`) | defect |
| R-1176 | P1 | a box rebuilt after „delete the old host" cannot reach its off-site history: no recovery code asked, a new key, the old history set aside on night one | hub store unit test (`C3-K1-…`) + **LIVE in B7** (2026-10-21) | defect |
| R-1191 | P1 | a `.fab` import makes a family-only app open to anyone | LIVE on scratch 9202 | defect (security) |
| R-1192 | P1 | the 8 GB box froze under first-day use; two plug pulls | LIVE on Tester4 (twice) | defect — cause not diagnosed (no shell on the box) |
| R-1177 | P2 | a failed local restore destroys the live data, keeps no copy | LIVE on 9202 | design question (D22 in STATUS) |
| R-1195 | P2 | a power cut mid full off-site restore: app on the backup's data, page says only „interrupted" | LIVE on Tester4 (B6 #2) | defect |
| R-1178 | P2 | second-drive „whole restore" can change files, refuse the DB, leave the app stopped and say „untouched" | unit test | defect |
| R-1180 | P2 | the nightly mirror (`rsync --delete`) can run during a restore from it | unit test | defect |
| R-1181 | P2 | a missing app folder deletes its second-drive copy in one night (the only copy for optional folders) | unit test | defect (D23 in STATUS) |
| R-1179 | P2 | a drive marked „away" can still be the second-drive target | code + unit test | defect |
| R-1182 | P2 | abandoning one set-aside off-site copy deletes every retained key package | code | defect |
| R-1199 | P2 | warning mails reach the operator, not the household | LIVE (two mailboxes) | defect |
| R-1204 | P2 | a box that ran once with its clock ahead loses its WireGuard tunnel to ep0 (PBS copies + operator door) until that date | LIVE (B6 #8, ep0 `wg show`) | defect |
| R-1203 | P2 | after a box death, the household is promised a link mail that never comes until the operator deletes the old host | LIVE (B7) | defect |
| R-1185 | P3 | a dump that fails its own check has already replaced the good one | unit test (docker stub) | defect (order) |
| R-1186 | P3 | the nightly off-site copy is skipped silently when another backup holds the flag | code | defect |
| R-1184 | P3 | a cut-off shares restore can be reported restored with files missing | code | defect |
| R-1196 | P3 | uploads confirmed just before a drive pull are 0 bytes or gone | LIVE (B6 #4) | defect |
| R-1183 | P3 | on 2026-10-25 every hub daily job runs twice (DB snapshot keep-2 pruned a day early) | replay of the hub's arithmetic | defect (D24 in STATUS) |
| R-1200 | P3 | an afternoon install gets no whole-box backup on night one | LIVE (B5) | design question |
| R-1202 | P3 | a box with its clock behind stops an app silently and only says „Hub nem elérhető" | LIVE (B6 #8) | defect |
| R-1187 … R-1190 | P3 | the unproven candidates of the four reviewers, grouped | code reading | mixed |
## Failure scenarios (B6)
| # | scenario | result | evidence |
|---|---|---|---|
| ★1 | power cut mid app install | PASS — clean rollback, honest event, reinstall in 15 s | `B6-failures/s1-…` |
| ★2 | power cut mid off-site restore | 6 s: safe; 14 s: app on the backup's data, page „interrupted" → R-1195 | `s2-…` |
| ★3 | data disk 100 % during backup | PASS — warned at 89 %, refused before writing, healed; SMB error unclear; system disk 97 % not run | `s3-…` |
| ★4 | data drive pulled, put back | apps stopped in 42/34 s, back by themselves; last uploads 0 B/missing → R-1196 | `s4-…` |
| ★5 | slow lossy line, first big off-site | run failed under 1 Mbit + 5 % loss, raw English; second press OK 3 m 38 s | `s5-…` |
| ★6 | off-site blocked | failed in 2.5 min, raw English; catch-up OK 2 m 37 s (shortened, not a night) | `s6-…` |
| ★7 | DST | PASS — the database copy ran once, at the second 02:30; W30 (no summer/winter mark in the repeated hour) | `s7-dst/` |
| ★8 | wrong clock at boot ±2 y | FINDINGS — −2 y became −6 months (the OS floor): „Hub nem elérhető" only, Paperless stopped silently (R-1202); +2 y: a whole-box archive dated 2028 (R-1189 a), then the ep0 tunnel dead (R-1204) | `s8-wrong-clock/` |
| ★9 | two actions at once | PASS — second action refused with a clear sentence | `s9-…` |
| ★10 | box off 3 days (+#15 restart in the catch-up) | PASS — ONE catch-up, a clear note to the household, the interrupted catch-up recovered | `s10-three-days-off/` |
| 11 | hard reset mid app update | NOT RUN — no update was offered to the box (nothing was released during the run) | — |
| 12 | DNS dead 1 h | PASS for data; operator told after ~37 min, household never | `s12-…` |
| 13 | second drive added later | not offered as the backup place → R-1198; safe removal OK; blank texts (W29) | `s13-…` |
| 14 | remove keeping data → reinstall „use kept data" | PASS — 47/47 documents, 47/47 originals | `s14-…` |
| 16 | English browser / English dashboard | follows the household's setting (by design); a few Hungarian strings → fixed W26 (unreleased) | `s16-…` |
## Restore paths (B8)
All seven customer paths PASS on the live box (`B8-restores/README.md`): A.1.1 Mealie unit 17 s; A.1.2 Nextcloud from
the second drive 3/3 sha256; A.1.3 Radicale verify copy 5 s; A.1.4 Nextcloud missing-only 3/3 sha256 (NFC/NFD: both
names come back on the share; Nextcloud itself folds NFD into NFC); A.1.5 BookStack full off-site 30 s; A.1.6 shares
2 files; A.1.7 `.fab` Mealie. Gaps: the messages do not say which point in time or how many files.
## Night chain and DST
**The first night (B5), no hands:** database copy 00:30Z PASS (3 m 27 s); second-drive copy 01:30Z PASS (8 apps + the
share, 58 s); off-site 02:15Z PASS (44 snapshots, 3 m 6 s, nothing deleted; the hub's own listing agrees: 44 files,
newest 02:18Z); whole-box backup NOT RUN — not due under the 24 h rule, because the install's own backup at 12:09Z
started the clock (R-1200, P3: the family's first day reaches a whole-box copy only on the second night). At 02:46Z
demo-hp's own kernel update rebooted the test host and shut the rehearsal VM down (harness, not a Tester4 defect; a BYO
host skips that step by design). Health 200 all night before that; memory steady. **The morning data snapshot is
identical to the evening one, byte for byte** (`B5-night/README.md`).
**DST (#7), live:** the box was started with its clock at 2026-10-25 02:20 CEST and left alone: no database copy at the first 02:30, one at the second (01:30:00Z, 1 m 55 s). The code replay agrees for the box; the hub runs every daily job twice on that day (R-1183, D24) and spring 2027 puts the 02:30 dump on the 03:30 copy (`B6-failures/dst-replay/`).
## Interventions (operator-only acts)
| # | when (UTC) | what | minutes the household waited |
|---|---|---|---|
| 1 | 09:57 → 12:00:57 | the owner passphrase handed over (by design, out of band) | 124 |
| 2 | asked 09:35, done 12:01–12:15 | off-site switched on for Tester4 | 0 (done before the household needed it) |
| 3 | asked 08:59, done ~09:11 (2026-10-21) | delete host Tester4-816e21 before the reinstall (B7) | ~12 (and 17 min of a dead end before it, R-1203) |
Not counted as interventions: the two plug pulls (14:11, 16:13) — a household can do that, and did.
## Versions
controller 0.336.0 (`7ca59e2`, floor), agent 0.161.0 (`4e1621c`), hub 0.162.0, golden 0.331.0, min_agent 0.158.1,
installer 1.35.0, ISO 1.29.0 (sha256 `dceacae5…4829`), catalog `c50adfc6` (65 templates). Nothing was released, deployed,
vouched, baked or published during the run. One text fix was committed, not released: controller `01c967e` (W26).
## Browser spike (A4)
Works: Chromium 155 + Playwright 1.52 in a throwaway LXC on demo-hp, phone and computer, HU and EN, in ~12 minutes.
121 screenshots. Limits: no real DNS/TLS for the household's address; not a human. `A4-browser/README.md`.
## Teardown
Machine: VM 9320, CT 9330 and the ISO removed from demo-hp by their own IDs; no drill firewall rules left; memory back
to the start. DooPlex: three watchers stopped by PID. Hub: **host Tester4-648b80 is left for the operator to delete**
(the customer Tester4 stays). ep0: one PBS snapshot `ns/Tester4/ct/9201/2026-10-20T12:17:26Z` and the WireGuard peer
10.77.0.6 remain; the Storage Box keeps the household's restic history — listed, nothing removed (`D-teardown/`).
## Secrets (names only)
All in `sec`, prefix `REHEARSAL_2026_10_20_`: `9202_PW`, `9202_RESETCODE`, `APP_BOOKSTACK`, `APP_RADICALE`,
`APP_VAULTADMIN`, `BIND_TOKEN`, `BOOKSTACK_TOKEN`, `DASH_PW`, `FAMILY_IRISZ`, `FAMILY_KATA`, `FAMILY_ODON`,
`INSTALL_ROOTPW`, `IRISZ_PW`, `KATA_PW`, `ODON_PW`, `OLD_BIND_TOKEN`, `PAIRING_CODE`, `PASSPHRASE`, `RECOVERY_CODE`,
`SETUP_CODE`, `SMB_PW`. Kept (the operator decides); none written anywhere else. Evidence scanned for their values:
0 hits.
## Register
Before 153 open rows; now 182. Opened by this run: 29 (R-1175 … R-1200, R-1202, R-1203, R-1204). Closed: R-1019 (a dated
check read on demo-felhom, read-only). One row (R-1201) came from a parallel session. Updated: R-1176 (proven live),
R-1189 (a half-live), R-895 (first verdict), R-435 (moved). Capability map: A15 still PARTIAL with new evidence; A12 and
A13 PROVEN-LIVE → PARTIAL (B7). `unproven.py --summary`: unchanged (35 of 55 not walked).
## Observations
FILED: everything in the tables above. NOT-A-FINDING: the dashboard ignores an English browser (by design: the
household's setting wins); Nextcloud folding NFD into NFC (upstream behaviour; both forms survive on the share); the
films' off-site size (CBR filler compresses; nothing missing); demo-hp's own kernel reboot at 02:46 (a harness effect: a
BYO host skips that step by design, `osupdate/leg.go:674`). FILED: everything in the tables above.
**Harness slips (mine), each recorded in the journal:** a screenshot step blanked a form before submit (15 min lost); the
two watcher logs froze at 20:24 when `git pull --autostash` replaced their files (recovered in full from the open files);
my first #10 start used a date the #7 records had made meaningless (restarted, recorded before any result). **Secrets:**
the nightly DooPlex sweep found three rehearsal secret values in my session record (the old and new bind tokens and a
setup code — I had read their mails in full and typed one into a `sec put`); in B7 a second bind link and setup code
entered the record the same way. All are one-time values, spent within minutes; nothing secret is in any committed file
(323 files scanned, 0 hits; pictures checked by eye; the B7 pairing code was blacked out).
+120
View File
@@ -0,0 +1,120 @@
# REPORT — the status page back to one screen, and a gate that keeps it there (2026-10-20, day)
## In plain words
The status page was 107 lines and 31,695 bytes; now it is 50 lines and about 5,500 characters outside the decide table,
which holds the 8 decisions that are still open. The „do" list went from about 25 items to 7 „still needed? (yes/no)"
questions, 2 morning-list items and 5 other lines (plus one „Later" line of row ids); CC checked the rest in the hub
and on DooPlex, and 7 were already done. **You answer:** the „Still needed? (yes/no)" list below, one word per line.
## Still needed? (yes/no) — copied from STATUS.md
1. Look at `roms/roms/` on the N100's drive in the file manager (R-1099). Since 2026-10-16.
2. The N100 sheet, about 30 minutes: Plex, the admins in Prowlarr, Sonarr and Radarr, your download sources
(`documentation/audits/n100-lived-in-2026-10-15/SHEET.md`). Since 2026-10-15.
3. Look at these pages on demo-hp when you have time: Calibre-Web („Adatok és mentés"), the file manager, AdventureLog,
Plex, the app page, the Plex and Sonarr guides („Útmutató"). Since 2026-10-16.
4. Remove the test server „plex-k6" from your Plex account (plex.tv → Settings → Authorized Devices). Since 2026-10-15.
5. Ask Google to index `https://felhom.eu/sajat-felho-tarhely` in Search Console (R-1035). Since 2026-10-14.
6. BIOS: on the N100, read „After Power Loss"; on the HP, set „Bypass F1 Prompt on Configuration Changes" to Enable.
Since 2026-10-14.
7. Pin the first Facebook post (R-917), and look at the new cover in the Facebook phone app (R-919). Since 2026-10-12.
## What CC checked, and how (method: the hub's GET pages with CC's scoped token, from the running pod; read-only local checks on DooPlex)
| „Do" item | Verdict | Evidence |
|---|---|---|
| Vouch agent 0.161.0 | **done** — the Day-0 manifest names 0.161.0 with the asked sha256 | `documentation/audits/status-view-2026-10-20/hub-reads.txt` |
| Delete the test customer Tester3 | **done** — not in the customer overview | same |
| Discard registrations 40 and 41 | **done** — the Hosts page has no „Unclaimed appliances" section (only `registered`/`bound` rows render) | same |
| Delete Tester4's old host `Tester4-31fa0e` | **done** — not on the Hosts page | same |
| Say „tear it down" for Tester 1's media tests | **done** — R-1022 closed 2026-10-17 | `CLOSED-ITEMS.md` |
| Hub master key into the password manager, then shred the file | **done** — `new.key` is gone; its folder is there (control) | `…/local-checks.txt` |
| Hub 0.153.0 „no token can …" | not a task — a statement; moved to history | — |
| Pull the failed disk Z6A07P2G | **not done** — still in DooPlex (lsblk; its twin is the control) | `…/local-checks.txt` |
| The Facebook `first-post/` leftovers | **not done** — 15 files still there | same |
| The `deny`-list edit in homelab-manifests | **not done** — still uncommitted | same |
| Old „broken" line: a Longhorn volume at 91 % | **gone** — Alertmanager lists only the two R-934 disk alarms | same |
| Old „broken" line: CI jobs get lost (R-887) | **gone** — R-887 closed 2026-10-19 (381 jobs, 0 lost) | `CLOSED-ITEMS.md` |
Every decided row (D1, D2, D10–D20) left the decide table only after its ruling was found in its closed row in
`CLOSED-ITEMS.md`. The whole old page is in `STATUS-HISTORY.md` under „Moved from STATUS on 2026-10-20" (headings
moved down two levels, otherwise unchanged).
## Register
- **Four numbers:** rows before **156**, after **153**, opened **1** (R-1174), closed **4**.
- **Closed:** R-1121 (the rotation file is gone; control `~/.bashrc`), R-1118 (its last part, R-1134, closed
2026-10-17; `sec doctor` all OK), R-231 (scripts in git and installed from it, the first run green; its same-disk
facts live on in R-232) — evidence in `CLOSED-ITEMS.md`, `audits/status-view-2026-10-20/local-checks.txt`,
`audits/answers-2026-10-12/s20-backup-scripts.txt`.
- **Merged:** R-993 into R-991 (one decision, D5; the proposal and the „no host floor at install" fact copied into R-991).
- **Narrowed:** R-132 (the hub password was rotated 2026-10-15; left: the break-glass hash field in the public repo);
R-1049 → WATCHING (the fix is live since hub 0.151.0; it no longer waits on the operator).
- **Owner cell fixed:** R-919 `CC` → `operator (upload + the look), then CC (close)` — its next action is the operator's.
- **Operator-owned rows (43), still waiting on the operator?** yes for all, except: R-933 — **CC first** (the size
measurement), then the operator's go; R-388 — **partly** (slice 1 is built; the scope decision is open);
R-920 — parked by the operator's own choice (c). On the page: decide D3–D9, D21; do (still needed, the rest);
the „Later" line for the remaining 23.
- **Rows over 4.5 KB (not rewritten, §N.7):** R-336 7.6 KB, R-516 7.0, R-462 6.8, R-433 6.7, R-264 5.8, R-919 5.0,
R-440 4.6, R-925 9.0.
## The gate — `scripts/status_view_gate.py` (R-1174)
- **Checks:** size outside the decide table (≤ 50 lines, ≤ 6,000 characters); one „Updated" paragraph (≤ 3 sentences,
no „Earlier" chain); no Hungarian letter outside a quote; every cited R-n open; no decided/closed/done row in the
decide table; the P1 row under „What is broken" or „decide"; every open row whose Owner names the operator under
„decide" or „do". Exit 0 clean, 1 convicted, 2 inconclusive (no sections, missing section, no or empty register).
- **Switch:** warn-first until **2026-10-27** (`SWITCH_DATE`), then a refusal; the date is also a due check for
R-1174, so the next push on or after that date names it.
- **Tests** (`scripts/test_status_view_gate.py`, 39 cases, all PASS): the genuine fixture before and after the switch;
the clean run's „what it looked at" line; per check, decoys that warn before and refuse from the switch — size (50
more lines; few lines and too many characters), updated (a prepended paragraph, an Earlier chain, four sentences, no
paragraph), english (a Hungarian do-line), open-ids (a closed row cited), decide (a DECIDED row; a „done" pick), p1
(the P1 line deleted; moved to What works), operator (the Later line deleted; a row only in „broken"); genuine
controls that must pass (a long decide table, version numbers in a sentence, Hungarian inside a quote); four
inconclusive cases; and the real `STATUS.md` against the real register.
- **Red-proof:** each check removed in turn → exactly its own decoys FAIL (size 4, updated 8, english 2, open-ids 2,
decide 4, p1 4, operator 4), then put back → 39/39. The switch moved to 2099 → the 14 „refuses" cases FAIL; moved
one day earlier → the 14 „warns on 10-26" cases FAIL. Put back → 39/39.
- **Run against the old page** (as it was at `2954b0be`): 75 convictions across all seven checks (81 lines, 25,472
characters outside decide; 13 decided rows; 24 operator rows missing; the P1 only in „do").
- Wired: `repo_gates.py` (fast set, docstring entry 19), `test_gate_decoys.py` (COVERS + runs the suite), found by
`script_tests_gate.py`; `decoy_coverage_gate.py` reports it COVERED.
## The CLAUDE.md edit (repo `felhom.eu/CLAUDE.md`, end-of-session checklist, line 139–141)
- **Before:** „… One screen, written for the operator in plain language, and deliberately **not** `CONTEXT.md`."
- **After:** the same, plus „(STE English) … How to update it:" and five numbered lines — replace the „Updated"
paragraph (never prepend; the old one to `STATUS-HISTORY.md`); a morning list under a dated sub-heading in „do";
remove what you finished; the decide-row rule; the gate checks it (R-1174).
- **Why allowed:** a format rule, not a loosened safety rule (`PROMPT-TEMPLATE.md` §9 rule 9, decision 150).
## Commits and the running session
`2954b0be` register pass · `9ec5f76a` the page · `1d7071e8` the gate · `54d1ee82` the instruction · this report.
**The rehearsal session shares this clone.** Its commit `f84f94d9` (audit evidence only) landed between my parts 1
and 2; it touched neither `STATUS.md` nor the register, so there was nothing to merge. **Its `git push` carried my
parts 2–4 to `origin` with its own** (the shared branch); its hook ran with `--scope=docs`. My own `--fast` run of the
same content was green (rc 0), and `scripts/push_scope.py` classifies the full range correctly as CODE. Its decoy
suite briefly planted changes in `website/gyik.html` and `scripts/felhom-host-install.sh` while I worked (R-928's
shape); I staged only my own paths, and both files were clean again afterwards. `unproven.py --summary`: unchanged
(NOT WALKED 35 of 55).
**Gate runs.** Every push: `repo_gates.py --fast`, rc 0 (the last one on the final tree, this report included). **Not
followed: „the full run before the last push".** The full run adds exactly one gate, `iso-bootstrap`, which starts a
container from an ISO image; on DooPlex that is a Docker run on production (workspace rule, and a standing feedback
memory), and it checks only `scripts/iso/`, which this session did not touch.
## Observations
- „My commits were pushed by the other session's push, under a docs-scope hook" — **NOT-A-FINDING:** CI re-runs the
whole fast set on every push whatever the hook's scope, and CI was green for every push of this session (by `head_sha` on the `jobs` endpoint, attempt 1 each): `2954b0be` job 2131, `54d1ee82` job 2132 (it carried parts 2–4), `c701b253` job 2133 — all `success`; the
shared-clone behaviour is already recorded (R-928, the shared-worktree rule).
- Tester4 shows DOWN with no host on the hub — **NOT-A-FINDING:** the rehearsal session reinstalls Tester4 today.
- „A Longhorn volume at 91 %" had no row — **NOT-A-FINDING:** the alarm is no longer active (Alertmanager, with the two
R-934 alarms as the control).
## Teardown
Provisioned nothing: no machine, no host, no hub write (GET only, CC's scoped token).
+51 -84
View File
File diff suppressed because one or more lines are too long
@@ -85,10 +85,10 @@
| <a id="a9"></a>[A9](capability-map-detail/A.md#a9) | One name per secret on every surface — „Beállító kód" (the setup code) and „Tulajdonosi jelmondat" (the owner passphrase) — and the re-enrol mail points at a page a rebuilt box shows | controller v0.211.0, hub v0.104.0 + v0.105.0 (R-295, R-323) | **IMPLEMENTED** | written BUILT: source-read; no customer has typed the passphrase | R-295; R-323 | | <a id="a9"></a>[A9](capability-map-detail/A.md#a9) | One name per secret on every surface — „Beállító kód" (the setup code) and „Tulajdonosi jelmondat" (the owner passphrase) — and the re-enrol mail points at a page a rebuilt box shows | controller v0.211.0, hub v0.104.0 + v0.105.0 (R-295, R-323) | **IMPLEMENTED** | written BUILT: source-read; no customer has typed the passphrase | R-295; R-323 |
| <a id="a10"></a>[A10](capability-map-detail/A.md#a10) | The offsite repository password can be RECOVERED from the sealed escrow with the customer's recovery code | hub v0.94.0, agent v0.125.0, controller v0.195.0 | **PROVEN-LIVE** | demo-felhom, real endpoints | `audits/DRILL-r201-offsite-recovery-2026-08-04.md` (2026-08-04); `audits/RECON-offsite-dr-chain-2026-08-04.md` (2026-08-04) | | <a id="a10"></a>[A10](capability-map-detail/A.md#a10) | The offsite repository password can be RECOVERED from the sealed escrow with the customer's recovery code | hub v0.94.0, agent v0.125.0, controller v0.195.0 | **PROVEN-LIVE** | demo-felhom, real endpoints | `audits/DRILL-r201-offsite-recovery-2026-08-04.md` (2026-08-04); `audits/RECON-offsite-dr-chain-2026-08-04.md` (2026-08-04) |
| <a id="a11"></a>[A11](capability-map-detail/A.md#a11) | DR tier by default: PBS + WireGuard base infra on every install, hub-controlled activation | installer v1.15, agent v0.86, hub v0.51 | **PROVEN-LIVE** | demo/drill fleet only | `audits/DRILL-day0-take2-2026-07-12.md` (2026-07-12) | | <a id="a11"></a>[A11](capability-map-detail/A.md#a11) | DR tier by default: PBS + WireGuard base infra on every install, hub-controlled activation | installer v1.15, agent v0.86, hub v0.51 | **PROVEN-LIVE** | demo/drill fleet only | `audits/DRILL-day0-take2-2026-07-12.md` (2026-07-12) |
| <a id="a12"></a>[A12](capability-map-detail/A.md#a12) | A customer's file survives a machine rebuild and comes back — the whole off-site story, end to end | hub v0.94.0, agent v0.125.0, controller v0.197.0 | **PROVEN-LIVE** | the success path; the failure path is not claimed (R-379, R-380) | `audits/DRILL-r201-night-run-2026-08-04.md` (2026-08-04); `audits/DRILL-r356-hot-only-restore-2026-08-22/` (2026-08-22); `audits/DRILL-r356b-driveless-db-restore-2026-08-22/` (2026-08-22) | | <a id="a12"></a>[A12](capability-map-detail/A.md#a12) | A customer's file survives a machine rebuild and comes back — the whole off-site story, end to end | hub v0.94.0, agent v0.125.0, controller v0.197.0 | **PROVEN-LIVE** | 2026-10-21 (fix brief, hub 0.163.0): PASSES — the rebuilt Tester4 box (public ISO) got the household's sealed package from the hub, asked for the recovery code, opened the history; Nextcloud came back with 3 031 of 3 033 manifest files byte-identical (the 2 never reached the old box) and the next off-site runs appended (63 → 64). Earlier the same day it FAILED (R-1176, now closed); the failure paths are not claimed (R-379, R-380) | `audits/fixes-2026-10-21/P1-live/` (2026-10-21); `audits/rehearsal-2026-10-20/B7-died/README.md` (2026-10-21); `audits/DRILL-r201-night-run-2026-08-04.md` (2026-08-04); `audits/DRILL-r356-hot-only-restore-2026-08-22/` (2026-08-22); `audits/DRILL-r356b-driveless-db-restore-2026-08-22/` (2026-08-22) |
| <a id="a13"></a>[A13](capability-map-detail/A.md#a13) | The customer's own UNAIDED recovery journey, end to end | controller v0.206.0, hub v0.98.0, agent v0.127.0 | **PROVEN-LIVE** | fifth walk, scoped — read what it does not claim | `tests/walk5-r201-2026-08-07/journal.md` (2026-08-07); `audits/CAMPAIGN-11-recovery-journey-2026-08-05.md` (2026-08-05) | | <a id="a13"></a>[A13](capability-map-detail/A.md#a13) | The customer's own UNAIDED recovery journey, end to end | controller v0.206.0, hub v0.98.0, agent v0.127.0 | **PROVEN-LIVE** | 2026-10-21 (fix brief, hub 0.163.0): PASSES with ONE operator act by design (D19: delete the dead host) — install, bind, claim, the recovery offer, a wrong code refused, the right code, „Teljes visszaállítás", all through the household's own pages; the new-link page now tells a household whose old box is still linked to write to Felhom (R-1203). Earlier the same day it stopped at the rebuilt dashboard (R-1176) | `audits/fixes-2026-10-21/P1-live/` (2026-10-21); `audits/rehearsal-2026-10-20/B7-died/README.md` (2026-10-21); `tests/walk5-r201-2026-08-07/journal.md` (2026-08-07); `audits/CAMPAIGN-11-recovery-journey-2026-08-05.md` (2026-08-05) |
| <a id="a14"></a>[A14](capability-map-detail/A.md#a14) | A random night of household actions while random things go wrong — twelve rounds, unattended | controller v0.245.0, agent v0.131.0, hub v0.116.0, golden 0.245.0, ISO 1.28.0 | **PROVEN-LIVE** | nested box, twelve chaos rounds | `audits/DRILL-chaos-night-2026-09-17.md` (2026-09-17) | | <a id="a14"></a>[A14](capability-map-detail/A.md#a14) | A random night of household actions while random things go wrong — twelve rounds, unattended | controller v0.245.0, agent v0.131.0, hub v0.116.0, golden 0.245.0, ISO 1.28.0 | **PROVEN-LIVE** | nested box, twelve chaos rounds | `audits/DRILL-chaos-night-2026-09-17.md` (2026-09-17) |
| <a id="a15"></a>[A15](capability-map-detail/A.md#a15) | A stranger's FIRST HOUR on a fresh install — download → install → claim → two apps → use → backup → remove → restore, plus a power cut and a mistyped code | controller v0.242.0, agent v0.130.0, hub v0.112.0, golden 0.242.0, installer ISO 1.26.1 | **PARTIAL** | every mechanism passed; the unaided journey failed before the first app | `audits/DRILL-fresh-install-0242-2026-09-14.md` (2026-09-14); `audits/DRILL-new-household-2026-09-30.md` (2026-09-30) | | <a id="a15"></a>[A15](capability-map-detail/A.md#a15) | A stranger's FIRST HOUR on a fresh install — download → install → claim → two apps → use → backup → remove → restore, plus a power cut and a mistyped code | controller v0.336.0, agent v0.161.0, hub v0.162.0, golden 0.331.0, installer 1.35.0, ISO 1.29.0 | **PARTIAL** | 2026-10-20: download → install → bind → claim → 8 apps → use → backup → remove → every restore path → power cut → mistyped codes all worked; the 8 GB box FROZE twice in the first afternoon and only a plug pull brought it back (R-1192), and the owner passphrase needed the operator | `audits/rehearsal-2026-10-20/journal.md` (2026-10-20); `audits/DRILL-fresh-install-0242-2026-09-14.md` (2026-09-14); `audits/DRILL-new-household-2026-09-30.md` (2026-09-30) |
| <a id="a16"></a>[A16](capability-map-detail/A.md#a16) | Off-site app-data capture covers the paths an app declares MANDATORY — on BOTH drive layouts | controller v0.197.0 | **PROVEN-LIVE** | both drive layouts | `audits/DRILL-r201-offsite-recovery-2026-08-04.md` (2026-08-04) | | <a id="a16"></a>[A16](capability-map-detail/A.md#a16) | Off-site app-data capture covers the paths an app declares MANDATORY — on BOTH drive layouts | controller v0.197.0 | **PROVEN-LIVE** | both drive layouts | `audits/DRILL-r201-offsite-recovery-2026-08-04.md` (2026-08-04) |
| <a id="a17"></a>[A17](capability-map-detail/A.md#a17) | Recurring offsite (PBS) whole-guest backups actually LAND, and RESTORE — local daily + offsite weekly as scheduled work | agent v0.97–0.103, controller v0.174/0.175, hub v0.76.0, host-install 1.20.0 | **PROVEN-LIVE** | demo-hp | `audits/SPIKE-r82-phase0-2026-07-26.md` (2026-07-26) | | <a id="a17"></a>[A17](capability-map-detail/A.md#a17) | Recurring offsite (PBS) whole-guest backups actually LAND, and RESTORE — local daily + offsite weekly as scheduled work | agent v0.97–0.103, controller v0.174/0.175, hub v0.76.0, host-install 1.20.0 | **PROVEN-LIVE** | demo-hp | `audits/SPIKE-r82-phase0-2026-07-26.md` (2026-07-26) |
| <a id="a18"></a>[A18](capability-map-detail/A.md#a18) | Restore-proof is UNATTENDED — the scheduler covers EVERY tier, follows the BACKUP rather than the clock, and a failure is heard | agent v0.104.0 → v0.121.0, hub v0.77.0 → v0.91.0 | **PROVEN-LIVE** | — | *(evidence not on file — not re-verifiable)*: the row cites per-repo CHANGELOGs and the spec `backlog/SPEC-r85-phase4-5-2026-07-26.md` — a spec, not a run record | | <a id="a18"></a>[A18](capability-map-detail/A.md#a18) | Restore-proof is UNATTENDED — the scheduler covers EVERY tier, follows the BACKUP rather than the clock, and a failure is heard | agent v0.104.0 → v0.121.0, hub v0.77.0 → v0.91.0 | **PROVEN-LIVE** | — | *(evidence not on file — not re-verifiable)*: the row cites per-repo CHANGELOGs and the spec `backlog/SPEC-r85-phase4-5-2026-07-26.md` — a spec, not a run record |
@@ -181,6 +181,7 @@
| <a id="d13"></a>[D13](capability-map-detail/D.md#d13) | Media apps see EVERY drive; a drive added, pulled, back or safely removed is handled per app (notice, real restart, wiring, torrents paused/resumed) | controller v0.313.0, catalog `0188430`, hub v0.151.0 | **PROVEN-LIVE** | Tier-0 boxes (Tester 1, demo-hp 9201); the fleet with golden 0.313.0 + the floor | `audits/media-build-2026-10-15/G/` (2026-10-15); `audits/media-drive-events-2026-10-14/` (2026-10-14) | | <a id="d13"></a>[D13](capability-map-detail/D.md#d13) | Media apps see EVERY drive; a drive added, pulled, back or safely removed is handled per app (notice, real restart, wiring, torrents paused/resumed) | controller v0.313.0, catalog `0188430`, hub v0.151.0 | **PROVEN-LIVE** | Tier-0 boxes (Tester 1, demo-hp 9201); the fleet with golden 0.313.0 + the floor | `audits/media-build-2026-10-15/G/` (2026-10-15); `audits/media-drive-events-2026-10-14/` (2026-10-14) |
| <a id="d14"></a>[D14](capability-map-detail/D.md#d14) | A second local copy COUNTS only on a proven different physical disk: the agent knows each place's disks; a VM box makes the copy but says it cannot check; a same-disk copy is not made; a NAS never hides the one-copy notice; a box with no drive refuses a data-folder app in words | agent v0.157.0, controller v0.317.0, hub v0.154.0 | **PARTIAL** | live: VM `unknown` (Tester 1) and two real-disk boxes `proven` (demo-hp, N100); unit only: the same-disk skip, the NAS notice, the no-drive sentence; `usb_bridge` never seen on a device | `audits/storage-identity-buildA-2026-10-15/README.md` (2026-10-15); `architecture/16-storage-physical-identity.md` §13 | | <a id="d14"></a>[D14](capability-map-detail/D.md#d14) | A second local copy COUNTS only on a proven different physical disk: the agent knows each place's disks; a VM box makes the copy but says it cannot check; a same-disk copy is not made; a NAS never hides the one-copy notice; a box with no drive refuses a data-folder app in words | agent v0.157.0, controller v0.317.0, hub v0.154.0 | **PARTIAL** | live: VM `unknown` (Tester 1) and two real-disk boxes `proven` (demo-hp, N100); unit only: the same-disk skip, the NAS notice, the no-drive sentence; `usb_bridge` never seen on a device | `audits/storage-identity-buildA-2026-10-15/README.md` (2026-10-15); `architecture/16-storage-physical-identity.md` §13 |
| <a id="d15"></a>[D15](capability-map-detail/D.md#d15) | A one-disk box gets its own data area („Belső tárhely") outside every whole-box copy, registered as the default drive with no press, so media and data-folder apps install with no extra step; a free disk's partitions can be formatted or used one by one | installer 1.33.0, agent v0.158.1, controller v0.318.0, golden 0.318.0 | **PROVEN-LIVE** | fresh 512 GB one-disk VM on demo-hp; partitions on a scratch disk; the old-box signed job and the D4 sentence unit-proven only; an empty part asks for confirmation (R-1093) | `audits/storage-identity-buildB-2026-10-15/README.md` (2026-10-16) | | <a id="d15"></a>[D15](capability-map-detail/D.md#d15) | A one-disk box gets its own data area („Belső tárhely") outside every whole-box copy, registered as the default drive with no press, so media and data-folder apps install with no extra step; a free disk's partitions can be formatted or used one by one | installer 1.33.0, agent v0.158.1, controller v0.318.0, golden 0.318.0 | **PROVEN-LIVE** | fresh 512 GB one-disk VM on demo-hp; partitions on a scratch disk; the old-box signed job and the D4 sentence unit-proven only; an empty part asks for confirmation (R-1093) | `audits/storage-identity-buildB-2026-10-15/README.md` (2026-10-16) |
| <a id="d16"></a>[D16](capability-map-detail/D.md#d16) | A failed or cut restore puts the household's data back EXACTLY as it was and starts the app — local unit, second drive, kept load and off-site, and after a power cut or a killed controller (D22) | controller `main` (0.338.0-rc.1), unreleased | **IMPLEMENTED** | proven live on 9202 (rc.1): a damaged unit and a controller killed between the volume step and the replay; the second-drive pre-check (R-1178), the mirror skip (R-1180) and the off-site power cut by test only; a database outside the replaced volumes is not covered | `audits/night-2026-10-21/P3-restore/02-run.log` (2026-10-21) |
## E. Access, networking & household use ## E. Access, networking & household use
@@ -481,6 +481,18 @@ box. The escrow is how `K` comes back **without** Felhom ever being able to read
(hub v0.8.0, `PUT /api/v1/hosts/{host_id}/escrow`). Validated end-to-end on a throwaway in (hub v0.8.0, `PUT /api/v1/hosts/{host_id}/escrow`). Validated end-to-end on a throwaway in
`documentation/tests/slice7-escrow-spike-findings.md`. Restore-mode *serving/consumption* is slice 10. `documentation/tests/slice7-escrow-spike-findings.md`. Restore-mode *serving/consumption* is slice 10.
#### The rebuilt box gets the household's package back — the CARRY [DESIGN, operator ruling 2026-10-21, hub v0.163.0, R-1176]
After the operator deletes a dead box's host (escrow acknowledged), the package is demoted to retained custody under
the OLD host id. The hub now **re-attaches** it: when the customer has exactly one live host, no live host holds a
package, and its newest escrow-acknowledged deletion left a retained row with an identity blob, that row becomes the new
host's current escrow row — moved, in one transaction, on the agent's first report (and again on the controller's
report, idempotently). **Zero-knowledge is unchanged:** the hub moves the same ciphertext from one row to another; it
never opens, re-seals or logs it; the audit (`escrow_carried`) carries host ids and the non-reversible hash. Two live
hosts → no carry, `escrow_carry_refused` to the operator. The box then sees „identity present", its mint guard (R-241)
holds, and it offers the recovery screen. Proven live 2026-10-21 (`audits/fixes-2026-10-21/P1-live/`: the right code
opened the household's history, Nextcloud's 3 031 files came back byte-identical, the next off-site run appended).
#### The separation principle (the rule that governs every posture) #### The separation principle (the rule that governs every posture)
Reading customer data needs **BOTH** the encrypted chunks **AND** a usable key. **Zero-knowledge Reading customer data needs **BOTH** the encrypted chunks **AND** a usable key. **Zero-knowledge
holds for exactly as long as Felhom never holds both at once.** Every posture below is just a holds for exactly as long as Felhom never holds both at once.** Every posture below is just a
@@ -643,6 +655,20 @@ run as a **journaled reconcile job**; a mid-flight failure is compensating-rolle
the just-restored guest — allowed unsigned per §4, same-transaction provenance). They diverge in the just-restored guest — allowed unsigned per §4, same-transaction provenance). They diverge in
the *archive* and the *back half*, **and in identity policy** (below). the *archive* and the *back half*, **and in identity policy** (below).
**Guest swap = guest RAM, and the host's swap behind it (R-1192; agent `main`, unreleased — the release is D26).**
`[FACT]` 2026-10-21 (`audits/fixes-2026-10-21/P4-freeze/`): Proxmox gives an LXC a 512 MB swap cap when nothing sets
`swap`, and the agent never set it; on the 8 GB minimum machine (guest 4096 MB) the household's first uploads filled the
guest with its swap full, the kernel could evict only program code, and the box froze (memory „full" pressure 96 %, load
162, the dashboard dead 6½ minutes). `[DESIGN]` The size step sets `swap` = the guest's memory: at bring-up, at a RAM
resize (`POST /guest/memory`), and once a minute after agent start for every pool LXC whose swap is below its RAM (live,
no restart, never lowers). Proven on a fresh box the night of 2026-10-21 (`audits/night-2026-10-21/P1-soak/`: 512 →
4096 one minute after the fixture agent started; the 2-hour household load then answered every 10-second probe). **The
host must have swap behind it:** our installer gives an 8 GB / 256 GB machine 7.75 GB (measured the same night); a BYO
host may have none. The agent reports `host.host_swap_mb` and `host.guest_swap_mb` on every host report, logs one WARN
when the first is below the second, and the hub mails the operator once (`host_swap_short`, operator-only); the
controller's install page adds „A gépen nincs elég cserehely (swap)…" when memory is tight and the agent says so. The
agent never creates or enlarges host swap — changing a host its owner set up is a decision (register).
**Identity reset is scenario-specific — this is a correctness boundary, not a detail.** "Reset **Identity reset is scenario-specific — this is a correctness boundary, not a detail.** "Reset
identity" is shorthand for two different operations: identity" is shorthand for two different operations:
@@ -575,6 +575,14 @@ are deleted at start-up and at each new day (`TestAccessLog_Keeps90DaysThenDelet
ingress log: ≈ 1,900 hub requests a day at ≈ 110 bytes a line → ≈ 0.2 MB a day, ≈ 19 MB for 90 days. A logging ingress log: ≈ 1,900 hub requests a day at ≈ 110 bytes a line → ≈ 0.2 MB a day, ≈ 19 MB for 90 days. A logging
failure never fails or delays a request: lines go through a non-blocking buffer to one writer (`internal/accesslog`). failure never fails or delays a request: lines go through a non-blocking buffer to one writer (`internal/accesslog`).
### 16.5 Daily jobs and the clock change [DESIGN — hub v0.163.0, operator ruling D24 2026-10-21, R-1183]
Every daily hub job (`scheduleDaily`: db-snapshot 02:00, prune, deadline-check 05:00, offsite-key-audit 07:10) runs at
its Budapest wall-clock time **once per calendar day**: the next run is the calendar day after (`time.Date`, day+1),
never „now + 24 h", and the slot that ran last never runs again. On a 25-hour day (the last Sunday of October) the old
arithmetic ran every job after 02:00 twice; on a 23-hour day it ran them an hour late. Pinned by `TestR1183_*` for
2026-10-25 and 2027-03-28.
## 17. Operator access tokens [DESIGN — hub v0.152.0, operator ruling 2026-10-15 11:38 (`09` §3 decision 251)] ## 17. Operator access tokens [DESIGN — hub v0.152.0, operator ruling 2026-10-15 11:38 (`09` §3 decision 251)]
**The problem.** Scripts — Claude Code's floor step first — drove the hub with the operator's own password (HTTP Basic, **The problem.** Scripts — Claude Code's floor step first — drove the hub with the operator's own password (HTTP Basic,
@@ -736,6 +736,42 @@ stayed (37 tables, the live app had 36); the definition did not move (same versi
answered 409. The restore page's „the database is back as it was" is false for it. Design note with the two options and answered 409. The restore page's „the database is back as it was" is false for it. Design note with the two options and
the pick (A: a copy of the app's live volumes, put back on failure — one copy's disk for the restore's length, a fit the pick (A: a copy of the app's live volumes, put back on failure — one copy's disk for the restore's length, a fit
check before the stop): `audits/night-2026-10-19/P3/design-R-893.md`; the decision is on `STATUS.md`. check before the stop): `audits/night-2026-10-19/P3/design-R-893.md`; the decision is on `STATUS.md`.
**`[DESIGN]` 2026-10-20 — „put back exactly as it was" (R-893 option A, operator decision D20 = A, `09` §3 decision 192
second half; controller 0.336.0 is the floor).** Before the stop, every named volume the snapshot will replace is measured
and its copy is added to the restore's own free-space need (R-357's exact rule); too little room or an unmeasurable volume
refuses with nothing stopped. Right after the stop the volumes are tarred into `<namespace>/backups/restore-undo/<app>/`
(outside every capture set). On ANY failure after a volume or the definition moved, the copies are poured back, a volume
the snapshot added is removed, the live definition is written back and the app is STARTED; the household reads that the
restore failed and nothing changed (with placed files: its database and volumes are back, the copied files stay); the
operator gets `backup_run_failures` leg `restore-put-back`. The hold (`restore_mixed`) remains only for a put-back that
fails — then the copy is kept for support — and the double failure (replay AND rollback) holds as before. On success the
copy is removed. **`[FACT]` 2026-10-20 (9202, controller 0.336.0-rc.1, `audits/night2-2026-10-20/P2/live/`):** the same
forced failure as on 2026-10-19 → 36 tables, the backup-only table absent, schema/data/row-count md5 equal to before,
the app running, no hold, the copy (69 MB, 0.9 s) gone; a normal restore of the same snapshot then worked. The R-893
known limit above is closed for named volumes; placed FILES are still not taken back (the plain-files mix, as ruled).
**`[DESIGN]` 2026-10-21 — the put-back covers EVERY restore, and a power cut (operator ruling D22 = yes, 2026-10-21
20:05; controller `main`, unreleased).** (1) **The local unit restore** — the app's own unit, the second drive's unit,
the second drive's „Teljes visszaállítás" and a kept load all run `RestoreFromRecoveryUnitAtWith` — now takes the same
measure → copy → put-back (`offbox_restore_undo.go`, one implementation, no second copy of the logic): the copy must fit
beside the unit before anything stops; ANY failure after the stop (volume replay, definition write, DB-only start, DB
replay, start) pours the live volumes back, writes the live definition back and starts the app; a put-back that fails
holds and keeps the copy; with no copy (a unit with no volume archive) a half-done DB replay HOLDS (the R-379 ruling)
instead of starting on it. The window carries the app-stop marker (`unit_restore`). (2) **A restore cut by a power loss
or a controller kill** (R-1195): once the copy is whole the restore writes `undo.json` and the live definition files
beside it and records the folder in the app-stop marker; at start the crash recovery pours it back BEFORE it restarts the
app, and the household reads „A visszaállítás félbeszakadt (áramszünet). Az alkalmazás adatai a visszaállítás előtti
állapotban vannak." A cut before the copy was whole restarts as before (nothing had moved). Local and off-site alike.
(3) **The second drive's „Teljes visszaállítás" asks the unit's refusals before its file half** (R-1178), so a refusal
moves nothing and its „érintetlenek" is true; a later refusal starts the app and names the changed files. (4) **The
night's second-drive mirror skips an app while a restore of it runs** (R-1180). **`[FACT]` 2026-10-21 night (9202,
controller 0.338.0-rc.1, `audits/night-2026-10-21/P3-restore/`):** Rallly + Postgres; a damaged unit volume → the live
volume poured back, the newest row still there (A,C), the app running, „…Nem változott semmi…" (on 0.331.0 the same case
crash-looped Postgres and lost the row, C3 2026-10-20); the controller killed (kill -9) right after the volume step →
on start „interrupted restore is put back … 1 volume(s) poured back, definition=true", markers A,C,D (D = the row newer
than the unit), the app running, the record reads the D22 sentence. Known limit kept: a database whose data is NOT in a
named volume the unit replaces (none in the catalog today — a drive-file app's unit restore is refused by R-538) is not
covered by the volume copy.
**[DESIGN] 2026-08-22 — the failure ladder of a database restore: replay → rollback → hold.** **[DESIGN] 2026-08-22 — the failure ladder of a database restore: replay → rollback → hold.**
Recorded here rather than only in a closed register row, because a decision that survives only inside Recorded here rather than only in a closed register row, because a decision that survives only inside
@@ -1038,6 +1074,14 @@ here — §8 has rows where it is the actual state.
### 7.0 What a customer can and cannot do ALONE — the four steps the drill found ### 7.0 What a customer can and cannot do ALONE — the four steps the drill found
> **[DESIGN] 2026-10-21 — the box death, alone (operator ruling, hub v0.163.0, R-1176).** After the operator's one press
> (delete the dead host — D19), the household reinstalls, binds, claims, and the rebuilt box offers „A korábbi, házon
> kívüli mentéseid visszaszerezhetők" and asks for the recovery code: the hub carries the sealed package to the new
> host (03 §8a, „the CARRY"). A wrong code is refused and changes nothing; the right code unlocks the history; each
> app is reinstalled and brought back with „Teljes visszaállítás"; the night's off-site run appends to the same
> history. Measured end to end 2026-10-21 (`audits/fixes-2026-10-21/P1-live/`). Before this ruling the rebuilt box saw
> no package, minted a fresh key and set the history aside on its first night (rehearsal 2026-10-20 B7).
> **[FACT] Added 2026-08-05.** The 2026-08-04 R-201 drill (`audits/DRILL-r201-night-run-2026-08-04.md`) > **[FACT] Added 2026-08-05.** The 2026-08-04 R-201 drill (`audits/DRILL-r201-night-run-2026-08-04.md`)
> is the first end-to-end proof that a customer's file survives a machine rebuild and comes back > is the first end-to-end proof that a customer's file survives a machine rebuild and comes back
> byte-identical. **It passed with an operator present, and four manual interventions stood between > byte-identical. **It passed with an operator present, and four manual interventions stood between
@@ -1531,6 +1575,19 @@ absent or hollow primary unit from the mirror **inside the call, before returnin
observe the hollow state. **The capture itself is deliberately not guarded:** a capture that describes observe the hollow state. **The capture itself is deliberately not guarded:** a capture that describes
an empty drive as empty is correct, and guarding it would make the manifest lie. an empty drive as empty is correct, and guarding it would make the manifest lie.
### 8.2a A second-drive copy whose SOURCE is gone is kept until the household releases it [DESIGN, operator ruling D25, 2026-10-21]
**[DESIGN]** The derived-copy rule (row 5) rebuilds the secondary from the source. When the SOURCE is gone — the drive
is away or decommissioned, a registered share's folder or an app's declared folder is missing — there is nothing to
rebuild from, and the copy may be the only one left. **D25:** such a copy is kept, recorded, and listed on the restore
page („Megőrzött másolatok") with „Eltávolítás"; no night job removes it. This is a different case from „data legs are
not guarded" for a leg that LEAVES the class set (the household's or the catalog's own choice). A share the household
deleted or renamed on a present drive is still pruned. An EMPTIED present folder is not mirrored over a copy that
holds files (the run warns). Removal is by the record (path inside the copy tree, same device + inode), never by size
or pattern. **Why:** the data-loss hunt of 2026-10-20 proved two nights that deleted a household's only second copy
(R-1175: one share group's reconcile removed the other groups' copies, and a dead drive's copy; R-1181: a missing or
emptied folder took its copy within one night). Controller: `internal/backup/tier2_kept.go`.
### 8.1 The blank cells, listed explicitly ### 8.1 The blank cells, listed explicitly
Per the rule that a blank is a finding, here they are: Per the rule that a blank is a finding, here they are:
@@ -499,6 +499,13 @@ app-mail settings).*
**Pick: A.** It follows the direction (the household decides three things, not 18), keeps every existing **Pick: A.** It follows the direction (the household decides three things, not 18), keeps every existing
mechanism, and each slice is reversible on its own. B can be added later as a fourth choice if households ask. mechanism, and each slice is reversible on its own. B can be added later as a fourth choice if households ask.
**`[DESIGN]` 2026-10-21 night — the box's page is the list the hub mails by (R-1199; decided by CC unattended — operator
may reverse).** Measured: a fresh box shows its default list ticked, but pushed nothing until the household saved the
page, so the hub kept mailing by an older stored list (Tester4: saved 2026-10-18, without disk warning, drive
disconnected and second-drive failure — 1 household alarm of 9 on the rehearsal day). Now a box whose household never
saved a choice pushes the default list at start (no address — the hub keeps the stored one); a saved choice is pushed
only as before (R-922). Controller `ba22f9b`, unreleased.
**What the operator still gets:** everything, unchanged — the operator channel never reads the household's switches **What the operator still gets:** everything, unchanged — the operator channel never reads the household's switches
(`processOperator` is independent of `enabled_events`). (`processOperator` is independent of `enabled_events`).
@@ -367,6 +367,8 @@ here; every sentence this product writes is display, and a decision reads a sign
a sentinel for `errors.Is` — never `fmt.Errorf("%w: …")`, which would prepend the sentinel's own text a sentinel for `errors.Is` — never `fmt.Errorf("%w: …")`, which would prepend the sentinel's own text
to the customer's sentence. to the customer's sentence.
**2026-10-21 night (W25, controller `5776d54`, unreleased):** the off-site page's „Utolsó hiba" no longer shows the run's raw text to the household. It is built from the classifier's class and the run's own result (the failed apps by name, „the other apps are fine" when true, „the box tries again the next night" when a retry can help). The raw text — restic's included — stays verbatim in the log and, sanitised, in the operator's event; the „foreign error prints verbatim" rule below now holds for those two places only on this page.
**[FACT] One exception, with an end date:** a box upgraded to 0.251.0 carries the OLD persisted note **[FACT] One exception, with an end date:** a box upgraded to 0.251.0 carries the OLD persisted note
with no kind until its next off-site run, so `offboxWarningDisplay` keeps the substring test for with no kind until its next off-site run, so `offboxWarningDisplay` keeps the substring test for
`kind == ""` only. **Slice 2 (R-557) must not translate that producer until R-570 closes.** `kind == ""` only. **Slice 2 (R-557) must not translate that producer until R-570 closes.**
File diff suppressed because one or more lines are too long
@@ -294,3 +294,29 @@ part next to a mounted part can only be used, not formatted (the root wrapper tr
--- ---
## D16 — A failed or cut restore puts the data back exactly (D22)
**What it is:** a restore that fails after it began changing an app — or is cut by a power loss or a killed controller
— leaves the app on the household's own data as it was before the restore, running, and says so. Before (2026-10-20,
C3 on 9202): a damaged local unit removed the live database volume, Postgres crash-looped, the newest row was gone and
nothing could put it back; a power cut mid off-site restore left the app on the backup's data (rehearsal s2).
**Components:** controller `main` (`d582feb`, unreleased): `restore_unit.go` reuses `offbox_restore_undo.go` (measure,
fit, copy, put back, hold on a double failure); `restore_crash_putback.go` + the app-stop marker's `undo` record and
`Recover`'s put-back; `tier2_whole.go` pre-check (R-1178); `RunAllTier2` skip during a restore (R-1180). Design: `07` §6.3
(the 2026-10-21 `[DESIGN]` paragraph), operator ruling D22.
**Status:** **IMPLEMENTED** — proven live on scratch 9202 with 0.338.0-rc.1 (2026-10-21): a damaged unit volume → the
live volume poured back, the newest row kept (A,C), the app running, „…Nem változott semmi…"; the controller killed
(kill -9) right after the volume step → on start the copy poured back (A,C,D — D newer than the unit), the app running,
„A visszaállítás félbeszakadt (áramszünet). Az alkalmazás adatai a visszaállítás előtti állapotban vannak." A normal
restore is unchanged (the control). PROVEN-LIVE when the release reaches the fleet.
**Unit only:** the off-site power cut (`TestR1195_AKilledOffsiteRestoreIsPutBackOnStart`), the second drive's pre-check
(`TestR1178_*`), the mirror skip (`TestR1180_*`), a failed put-back at start (`TestR1195_AFailedPutBackOnStart…`).
**Known gap:** a database whose data is not in a named volume the unit replaces is not covered by the copy (none in the
catalog today; a drive-file app's unit restore is refused by R-538).
**Evidence:** `audits/night-2026-10-21/P3-restore/` (the run log, the red-proofs).
---
@@ -16,6 +16,8 @@
> renamed here the same day, with the operator's three Claude Design fixes folded in (marked *fix > renamed here the same day, with the operator's three Claude Design fixes folded in (marked *fix
> (a)/(b)/(c)* below) and one measured correction: felhom.eu is served from DooPlex with > (a)/(b)/(c)* below) and one measured correction: felhom.eu is served from DooPlex with
> **Cloudflare DNS only**, not through Cloudflare. > **Cloudflare DNS only**, not through Cloudflare.
> 2026-10-21: poster regenerated in Claude Design (one-disk data area, https-only); PDF print version added next to it
> (`felhom-system-poster.pdf`).
--- ---
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -0,0 +1,14 @@
# C1 reviewer 2 — delete and retention (fresh, read-only; clones at controller 7ca59e2)
| # | Sev guess | harm | mechanism |
|---|---|---|---|
| D1 | P1 (mech high, reach medium) | Shares second-drive copy: one drive group's reconcile deletes ANOTHER drive's share mirrors (on the night a drive is decommissioned/away; or every night when two groups target one drive, then a full re-copy) | `internal/backup/tier2_shares.go:135` groups by source; `:201-202` own destBase/legRels; `:230` `tier2ReconcileRoots(destBase, sharesTier2DestRoots(destBase), legRels)`; `:266-267` ReadDir lists every drive key on disk; `tier2.go:299-329` removes any child not in this group's legs; `shares_payload.go:233-251` away/decommissioned drives drop out |
| D2 | P1 optional legs / P2 mandatory; design question with a defect corner | A missing or emptied app folder on the drive deletes its second-drive copy within one night; optional legs (audiobooks, comics, ROMs, Immich external library) are in no other tier | `tier2_capture.go:55-61` missing leg skipped (optional silently) → `tier2.go:455` reconcile deletes its mirror; run recorded success `:465`; emptied → `rsync -a --delete` `tier2.go:751` via `:443`; catalog optional legs `audiobookshelf/.felhom.yml:24-26`, komga, romm, immich `:32-34`; 07 §8.2 „data legs are NOT guarded" covers a leg leaving the class set, not a declared-but-missing leg |
| D3 | P2 high (mech) | A host clock ahead during one whole-guest backup silences that tier until real time passes it; the hub reads the same clock and says OK | agent `internal/localapi/server.go:1178-1190` negative age never due; `internal/backup/runner.go:607-629` newest ctime unbounded; hub `monitor/deadline_tiers.go:155`, `deadline.go:243`; R-973 cap counts future entries `backup_cap.go:61`; restic: `offbox_window.go:133-136` pinned guard refuses every window → quota fills; NAS tier no guard `:209-216`; `offbox_abandon.go:248,287` set-aside rm on box clock |
| D4 | P2 | A failed LOCAL unit restore leaves the app running on mixed data, no undo copy (same as R1 of reviewer 1 — independent agreement) | `restore_unit.go:430-489`; `writeSafetyDump` only in `offbox_reconstitute.go`; 07 §6.3 ladder not scoped to off-site |
| D5 | P2 order defect / depth design | A dump that fails its own content check has ALREADY replaced last night's good dump; Tier-2 mirrors it an hour later | `appbackup/dbdump.go:388-397` size-only check, `:399` rename, `:419` `ValidateDump` after; `backup.go:688-690`; volume tars promoted with no content check `backup.go:902` |
| D6 | P3 med | „Megőrzött adatok" can list a RUNNING app's live folder as deletable leftover if its app.yaml is unreadable | `stacks/manager.go:672`; `kept.go:220-235` skips `!Deployed || AppConfig==nil`; `ListKept :330-358`; `DeleteKept :385-402` RemoveAll; `deploy.go:1180-1190` write+rename without fsync |
| D7 | P3 low-med | A unit restore can point the app back at an OLD drive; F5 then deletes the current drive's unit | `restore_unit.go:90-111, 507`; `cmd/controller/main.go:3252`; `deploy.go:820`; `backup.go:1517-1574` `pruneStalePrimaryDirs` even when this run's capture failed (`:726-733`) |
| D8 | P3/P4 design | local whole-guest keep=1 prunes the only good archive before the new one is proven | agent `runner.go:229`, `:139-161`, restore-test settle 24 h `:462` |
**Checked SAFE (reviewer):** hub off-site deletions by exact identity (`offsitekeys.go:366-515`); pinned off-site retention guard refuses future/too-new snapshots and big plans (`offbox_window.go:132-161`); dump and tar writes crash-safe (tmp+fsync+rename); R-403 hollow-unit guard; drive migration deletes source only after verify (`stacks/migrate.go:614-667`); app removal refusals and protected folders (`delete.go:136-153, 241-301, 839-846`); mirror/undo clean-ups allow-listed/prefix-guarded; agent prune never on PBS unless opted in; restore-test skips unsettled/future/foreign-key archives; `ForgetRetiredDrives` mount units only; SMB share removal deletes no folder. Not reviewed: agent `vzpartial` janitor, tenantsync/ep0 prune config, hub RESET beyond `PurgeRepos`.
@@ -0,0 +1,14 @@
# C1 reviewer 3 — keys that gate recovery (fresh, read-only; controller 7ca59e2, hub 5c48731, agent 4e1621c)
| # | Sev guess | harm | mechanism |
|---|---|---|---|
| K1 | P2 (P1 with K2 / a RESET) high | **A reinstall through „delete the old host first" (D19/R-1154) cuts the new box off from the household's escrow: the recovery code is never asked, a NEW key is minted, and the old off-site history is set aside by itself on night one; typing the old code answers „wrong code"** | hub `api/passphrase_scope.go:58` 409 forces the delete; `store.go:3204-3210` delete demotes escrow under the OLD host_id; `handler.go:1261` new host_id; `GetEscrowStatusForCustomer` `store.go:3500-3527` joins on `hosts` → nil (superseded join on deleted host also gone) → `identity_blob_present`/`superseded_present` false; controller `offbox.go:542-555` mints new pw, `:1788` no recovery offer, `:905-906` firstNight → `resetOrphanedRepo`; `handleHostEscrowRetainedGet` 404 for a deleted host (`handler.go:1691`); contrast purge `store.go:4185-4190` DOES include `host_deletions` |
| K2 | P2 high | Abandoning ONE set-aside copy deletes EVERY retained recovery package of the customer — other set-aside copies become ciphertext nobody can open | hub `handler.go:626-627` → `PurgeSupersededEscrowForCustomer` `store.go:4185-4190` deletes all rows for the customer (incl. deleted hosts); the comment at `:4176-4179` states the invariant it breaks |
| K3 | P3 med-high | A ceremony while off-site is not configured seals a package with NO repo password and supersedes the one that had it | controller `escrow_handlers.go:217` re-stage only when configured; agent `main.go:2975-2981`; hub `store.go:3392`; `escrow_confirm.go:222-258` stale card local only |
| K4 | P3 med | The global „create your recovery code" bar links to the ceremony the backups page hides while a recovery offer is outstanding | `backups_remote.html:144-160` vs `escrow_banner.go:23-33` + hu.json:1682; no `recoveryOffer()` refusal in the wizard/start handlers; settle gate `main.go:876`, `offsiteapply.go:84` |
| K5 | P3 med-high | After a reinstall recovered on the reuse path, the NEW PBS key K is never escrowed and nothing compares fingerprints | `felhom-pbs-apply:99` autogen; `escrow_confirm.go:182-205` auto-confirm on hash; no compare of `host_escrow.key_fingerprint` vs live key (agent `internal/backup/runner.go:407-429` only) |
| K6 | P3/P4 design | Whole-guest off-site (PBS) copies run before any escrow exists; host lost before the ceremony → unreadable archives | controller gates Tier 3 (`offbox.go:781`); agent `internal/backup`, `internal/pbsdr` have no escrow gate |
| K7 | P4 low | escrow PUT split in two statements; a failure leaves a row whose hash claims a key its blob lacks | hub `handler.go:1388, 1412-1420`; `store.go:3402-3408` |
| K8 | P4 | operator off-site re-issue resets the provider password before saving it | hub `offsite.go:217-245` |
**Checked SAFE (reviewer):** mint guard while the hub holds a package (`offbox.go:542-555, 659-720`); orphaned repo moved aside, never re-inited/deleted (`offbox.go:886-943, 332-397`); supersession retains both blobs (`store.go:3343-3416`); host delete demotes not destroys (`store.go:3164-3229`); ceremony two-word confirm; `escrow.Create` self-verifies; recovery refuses to replace a different local key; `ClearOffboxTarget`; PBS wrapper has no delete verb; hub abandon delete limited to one path after a delay (`offsitekeys/service.go:464-496`).
@@ -0,0 +1,24 @@
# C1 reviewer 1 — restore and rollback (fresh, read-only; clones at controller 7ca59e2 / v0.336.0)
Raw candidates as returned by the reviewer, condensed; each is UNVERIFIED until C2 marks it. File:line are in the controller unless named.
| # | Sev guess | harm | mechanism |
|---|---|---|---|
| R1 | P2 high | A local / second-drive unit restore that fails halfway restarts the app on half-replaced data, no undo copy; the next night's capture then overwrites the only restore point with the broken state | `internal/backup/restore_unit.go:429-431` failed stop only warns; `restore.go:217-218` `docker volume rm -f` error ignored before untar (10-min limit `:237-241`); `restore_unit.go:466-486` DB replay failure → app started anyway; one restore point per app, promoted in place (`atomicPromoteTar`) |
| R2 | P2 high | A power cut / controller restart during a restore starts the app on a mixed state; the undo copy is never used at boot | `offbox_reconstitute.go:942` marker → `appstop_marker.go:307` StartStack with no restore check; `offbox_restore_undo.go:137` undo dir only dropped, never read at start; unit restore and .fab import set no marker → `bootrecon/bootrecon.go:210-213,269` starts the app; updates have a journal (`stacks/update.go:1420-1489`), restores do not |
| R3 | P2 high | A cut-off shared-folder restore can be „placed" and reported as restored while files are missing; scratch deleted | `shares_restore.go:138` readiness = `rErr == nil && len(entries) > 0` (R-358 class, not fixed for shares); 30-min ctx `web/offbox_handlers.go:716`; place reports N files and deletes scratch `:301`; free-space gate fails open and fixed 2 GiB `:107` |
| R4 | P2 high | A `.fab` export silently drops a whole drive folder that contains a symlink; import never sets owners | `appexport/fabplan.go:292` `tar.FileInfoHeader(info,"")` then `os.Open`+`io.Copy` on a symlink; `export.go:714` warn-and-continue; `assertBundleDataComplete` refuses only when no data at all; `restore.go:1157-1176` extracts only dirs/regular files, no chown |
| R5 | P2 med-high | Second-drive whole restore swaps files, then refuses the DB half, leaves the app STOPPED and says „the app is untouched" | `tier2_whole.go` stops + merges first, then `RestoreFromRecoveryUnitAtWith` whose refusals (`restore_unit.go:366-371, 386-389, 413-416, 420-423`) assume nothing happened; en.json:2667 „the restore did not start. The app is untouched" |
| R6 | P2 med | `.fab` import into a STOPPED app merges into existing volumes; the default export (stop box ticked) has no SQL dump | `appexport/restore.go:361-374` volumes removed only if running; `docker cp` merge `:721`; `templates/app_data.html:221` stop default; `export.go:262` stop before `dumpDatabase` `:366`; `appbackup/dbdump.go:142` discovers only running containers |
| R7 | P2/P3 high | `.fab` import has no concurrency guard, no undo, no record | `restore.go:181-186`; `web/handler_export.go:312-353` no `restoreOpBlocked()`; `RemoveStackVolumes` before anything else, no copy |
| R8 | P2 med | `.fab` import drops the family gate, sign-up lock and version pin → a family-only app opens to anyone | `cmd/controller/main.go:3517-3530` bare `AppConfig`; `stacks/family_gate.go:173-190` removes the gate file when `FamilyGate == nil`; R-773 fix only on the unit path (`stacks/deploy.go:840-870`) |
| R9 | P3 med | `.fab` import trusts the bundle's HDD_PATH; empty → relative path; older userdata overwrites newer | `restore.go:589-631` |
| R10 | P3 high | `.fab` DB import: plain psql without ON_ERROR_STOP; postgis not classified | `restore.go:1052-1053`, `:928-937` |
| R11 | P3 med | volume replace never checks the old volume is gone | `restore.go:218` |
| R12 | P3 high | interrupted second-drive whole restore leaves `<dst>.felhom-restoring` (O_EXCL) → every retry fails „file exists" | `tier2_whole.go:86` |
| R13 | P3 design | full off-site download capped at 30 min per press; household not told to retry | `web/offbox_handlers.go:455` |
| R14 | P4 med | place/full restore re-read the LATEST snapshot instead of the downloaded one | `offbox_restore.go:675-679`, `offbox_reconstitute.go:657-661` |
**Inventory drift (July → now):** three more customer restore entries exist: the Tier-2 unit restore (R-102), the Tier-2 whole restore (decision 26, `tier2_whole.go`), the kept-data Load (`kept_load.go`).
**Checked SAFE (reviewer):** `.fab` path confinement (`appexport/restore.go:1104-1107,1152-1155`; `validate.go:28-66`; `web/handler_export.go:385-394`); off-site placement map refuses `..`/outside/`backups/` (`offbox_restore.go:603-640`); R-358 marker after restic success; full off-site restore fit/safety-dump/undo checks before stop, fail closed (`offbox_reconstitute.go:719-818, 909-927`); Tier-1 refusals before change; main loader `ON_ERROR_STOP=1 --single-transaction` (`appbackup/dbdump.go:775-784`); Tier-2 file restore `rsync --ignore-existing`, never `--delete`; whole restore newer-wins + Lchown; **NFC/NFD: no normalisation anywhere — every lane compares bytes, so the missing-only compare sees TWO names (consistent on one filesystem)**; agent RestoreLXC force only when asked. Not reviewed: `kept_load.go`, hub operator restore paths.
@@ -0,0 +1,40 @@
# C1 reviewer 4 — time and concurrency (fresh, read-only; controller 7ca59e2, agent 4e1621c, felhom.eu 5c48731)
## (A) Schedulers and what each does on Sunday 2026-10-25 (03:00 CEST → 02:00 CET; 02:00–02:59 happens twice)
| owner | job | schedule | zone | file:line | 2026-10-25 | after 3 days off |
|---|---|---|---|---|---|---|
| controller | db-dump | daily W (default 02:30) | Budapest (`time.Date`, day+1) | main.go:1262, scheduler.go:499-518 | once, at the SECOND 02:30 (CET = 01:30Z) — Go normalises the ambiguous time to the later offset | not replayed by the scheduler; the catch-up runs it once 15 min after start (nightchain.go:277-345) |
| controller | tier2-backup | W+60m | Budapest | main.go:1352 | once, 03:30 CET | once, by the catch-up |
| controller | offbox-backup + app-update leg | W+105m | Budapest | main.go:1519 | once, 04:15 CET | catch-up runs the off-site leg only |
| controller | offsite-abandon-sweep 05:10, offsite-proof 05:30, offsite-integrity 06:00 | fixed | Budapest | main.go:1561, 1613, 1591 | once | **no catch-up** (finding T6) |
| controller | metrics-prune 04:00, fill-watch 03:30, asset-sync 05:00, selfupdate-auto 04:30 (off by default) | fixed | Budapest | main.go:1621,1671,1768,1741 | once | floor update runs on any hub reply |
| controller | whole-guest quiesce loop | poll 5 min; gate [W+2h, W+6h) by wall minute; valve when age > cadence+24h | Budapest wall | quiesce.go:150, 191-205, 790-818 | gate 04:30–08:30 CET (4 h real) | valve fires ~5 min after start, any time of day |
| controller | ~20 `Every` tickers | monotonic | n/a | main.go:898-2146 | unaffected | restart at boot |
| agent | whole-guest due | age ≥ cadence | UTC | localapi/server.go:1182-1192 | unaffected | due at once |
| agent | restore-test | 30 min after start, then 6 h | UTC | backup/schedule.go:148-177 | unaffected | after 30 min |
| agent | OS leg (+ kernel reboot if hub „Tonight") | after any non-manual primary backup +90 s, 20 h min gap | no wall clock | server.go:982-986, main.go:1272-1280, leg.go:655-664, kernel.go:101-114 | follows the backup | follows the valve backup = daytime (T8) |
| agent | fstrim | hourly; due Wed 10:00, only 10:00–20:59 | host local | fstrim/fstrim.go:39-50,110 | n/a | next eligible hour |
| hub | scheduleDaily: db-snapshot 02:00 (keep 2), prune 04:30, deadline-check 05:00, offsite-key-audit 07:10 | `next.Add(24*time.Hour)` | Budapest | hub cmd/hub/main.go:1049-1072 | **every job runs TWICE** (T9) | n/a |
| host timers | mgmt-watchdog, crash-guard-check | monotonic | n/a | felhom-agent/configs/*.timer | unaffected | after boot |
| DooPlex timers | hub-db-backup 02:30; dooplex-offsite 00:20; hub-db-restore-test Sun 04:30; offsite-restore-test Sun 05:30; ep0-copy-gc 08:00 | OnCalendar local | DooPlex | felhom.eu/scripts/*/*.timer | Oct 25 is a Sunday: both weekly restore tests run on the DST day; 02:30 is in the repeated hour (systemd behaviour not verified) | Persistent=true |
## (B) Candidates
| # | Sev guess | harm | mechanism |
|---|---|---|---|
| T1 | P2 med-high | The nightly second-drive mirror (`rsync --delete`) can run DURING a restore from that same mirror and delete the files not yet copied back | `tier2.go:479-520` RunAllTier2 checks held/version/disconnected only; `:443` mirror, `:744-751` `--delete`; restore `tier2_whole.go:301` takes only `acquireRunning`; Tier-2 leg takes only `LegLock` (main.go:1348-1352); `update_guard.go:595-601` isHeld ignores restores |
| T2 | P2 if true; low-med | The Debian 13 controller image may lack tzdata → every schedule on UTC silently | `Dockerfile:77-100` debian:13.7-slim, no tzdata, no `time/tzdata` import; fallbacks scheduler.go:19-23 (logs), quiesce.go:749-751 (silent), funcmap.go:62-66 |
| T3 | P3 med | Whole-guest start creeps ~5 min/night until a night is skipped silently | agent `server.go:1184` `age >= Cadence` no slack; age from archive start; controller poll 5 min `quiesce.go:150`; gate refusal `:807-818` |
| T4 | P3 med-high | The whole-guest quiesce stops apps in the middle of a restore or an update | `quiesce.go:213-275` no `IsRunning()`/restore check; `RunningAppStacks` (stacks/manager.go:620-636); `TriggerNow` (:419-445) same; `DumpAppVolumesSafe` (backup.go:1001-1045) can restart an app quiesce stopped |
| T5 | P3 high | The nightly off-site copy is skipped SILENTLY when anything else holds the backup flag, and the catch-up never retries | `offbox.go:981-996` INFO „skipped — another backup is running" return nil; `withLeg` MarkEnded unconditionally (main.go:1246); proof 05:30 / integrity 06:00 take the same flag |
| T6 | P3 high | Off-site check and proof never run on a box off at 05:30–06:00; the code comment says the opposite | main.go:1575-1577 comment vs 1591-1619; `catchUpLegs` nightchain.go:17-18; no hub alarm on integrity/proof age |
| T7 | P3 high | Hub raises a false „no DB dump" error every morning for W after ~04:45 or evening W | hub `monitor/deadline.go:416-419` midnight Budapest; `:556-565` → `expected_dbdump_missed`; hub has no idea of W |
| T8 | P3 design | The OS leg and kernel reboot follow ANY scheduled backup, including the daytime valve after power-on | agent server.go:982-986; main.go:1279 labels „night"; kernel.go:101-114; hub osupdates/kernel.go:46-48,181-185 |
| T9 | P4 high | Hub daily jobs run TWICE on 25 October (db-snapshot keep-2 prunes a day early; duplicate deadline alarms); once an hour late in spring | hub cmd/hub/main.go:1058-1062 `next.Add(24*time.Hour)`; the controller fixed this (scheduler.go:511-516) |
| T10 | P4 high | Spring forward 2027-03-28: db-dump and tier2 fire at the same instant; tier2 may mirror yesterday's unit | scheduler normalisation; `LegLock` main.go:1237 plain mutex |
| T11 | P4 design | A clock ahead stops whole-guest backups silently (same as D3) | server.go:1182-1192; quiesce.go:816-817; deadline.go:244; backup_cap.go:61 |
| T12 | P4 high | self-update busy check before the image pull, swap after it with no re-check | updater.go:532-545, :764 |
| T13 | P4 med | double-clicking a restore shows it as failed while it runs | handlers.go:1853, :1874 `BeginRestoreOp` after the check; second POST's `EndRestoreOp(false)` |
**Checked SAFE (reviewer):** controller DST day+1 arithmetic and late-fire skip (scheduler.go:342-348, 511-516); night ledger persisted, seeded, one catch-up for many nights; quiesce marker persisted and recovered; in-memory backup flag (no stale lock); flock-based OS-leg locks; restic stale locks self-heal; two restores blocked by `m.running`; Remove refused during restore/quiesce; catch-up and whole-guest wait for each other; agent due/cap UTC durations; off-site prune guard refuses future snapshots; host power-off re-checks; no cron/OnCalendar in the installer; hub staleness uses the hub's own receive clock.
@@ -0,0 +1,22 @@
# C2 register search, 2026-10-20T10:11:13Z, OPEN-ITEMS.md + CLOSED-ITEMS.md at c701b253; grep -ciE
1 open / 2 closed : R-893
0 open / 0 closed : tier2_shares|_shares.*tier|shares.*second.drive|Megoszt.*2\. ment
0 open / 2 closed : host_deletions|retained.*delet|deleted host|escrow.*reinstall
2 open / 7 closed : scheduleDaily|DST|daylight|oraallit|time.?zone
1 open / 8 closed : restore-undo|undo copy|safety dump|unit restore.*fail
1 open / 0 closed : ValidateDump|dump.*replac.*valid
0 open / 0 closed : SharesScratchReady|shares.*scratch|shares restore
0 open / 4 closed : tier2_whole|whole restore|Teljes vissza
0 open / 3 closed : optional leg|missing on disk|--delete
0 open / 0 closed : PurgeSuperseded|abandon.*purge
0 open / 1 closed : future.dated|clock ahead|clock.*future|negative age
0 open / 0 closed : another backup is running
1 open / 2 closed : expected_dbdump_missed
2 open / 1 closed : symlink
0 open / 1 closed : family.gate.*fab|fab.*family|\.fab
0 open / 0 closed : felhom-restoring
0 open / 0 closed : quiesce.*restore|restore.*quiesce
1 open / 4 closed : offsite-proof|integrity.*catch|proof.*catch
0 open / 0 closed : double.click|dupla
0 open / 1 closed : disconnected.*target|away.*target
Every non-zero hit was read: none describes the candidate (positive control R-893 found in both).
@@ -0,0 +1,51 @@
# C2 — the main session's verdict on every candidate (2026-10-20)
Method: each candidate's `file:line` read in LIVE source (controller `7ca59e2` = v0.336.0, agent `4e1621c`, felhom.eu
`5c48731`+); then (a) the register searched, open AND closed, with ASCII fragments — positive control `R-893` found in
both; (b) the design test against `07-backup-architecture.md` and `CONTEXT.md`. Verdict words:
- **CONFIRMED** — the mechanism read true in live source AND a proof ran (unit test against production code, or live).
- **CONFIRMED-CODE** — the mechanism read true in live source; no proof ran yet.
- **PLAUSIBLE** — the reviewer's reading, spot-checked only.
- **NOT-A-FINDING** — with the reason.
Register: none of the candidates below is in OPEN-ITEMS or CLOSED-ITEMS (searches in `C2-register-search.txt`).
| id | verdict | design or defect | note |
|---|---|---|---|
| **D1** shares second-drive copy pruned by another drive's group | **CONFIRMED** (unit test, production code, fake mirror seam — `C3-D1-shares-tier2/`) | defect | **Worse than reported:** with shares on two drives that pick the same target, EVERY night the second group's reconcile deletes the first group's copy (`removed stale dest dir hdd_2/b_share`) — one drive's shares never have a second copy while the run says ok. And the night a drive goes away, its copy is deleted (`removed stale dest dir hdd_3/csaladi_fotok`). |
| D1b an AWAY drive is still chosen as the second-drive target | CONFIRMED-CODE (+ the same unit test: `hdd_3/iskola => hdd_1/...` after hdd_1 was marked disconnected) | defect | `tier2.go:151-180` loops `GetSchedulableStoragePaths()` (`settings.go:1593-1603` excludes only Decommissioned); no Disconnected check. On a real box the away drive's empty mount point is on the guest's root disk |
| **K1** reinstall after „delete the old host" loses the route to the old escrow | **CONFIRMED** at the hub store (unit test against production code, `C3-K1-escrow-after-host-delete/`); the controller half (new key minted, history set aside on night one) CONFIRMED-CODE | defect (03 §8a / 07 §7.0 promise the route; D19+R-1154 removed it unrecorded) | hub `store.go:3164-3229` demotes into `host_escrow_superseded` under the OLD host_id and deletes the `hosts` row; `GetEscrowStatusForCustomer` `store.go:3499-3527` joins on `hosts` → nil for the new box; `customer_reset.go:48,155` and `store.go:4189` DO include `host_deletions`. Live proof needs Tester4 off-site ON (B7) |
| K2 abandoning one set-aside copy purges every retained package | CONFIRMED-CODE (`store.go:4185-4190`) → R-1182 | defect (breaks `store.go:4176-4179`'s own invariant) | `store.go:4185-4190` customer-wide delete read true |
| **R1/D4** a failed LOCAL unit restore: no copy of the live state, app started anyway | **CONFIRMED LIVE** on 9202 (`C3-R1-local-restore-9202/`): live row destroyed, DB crash-loops, no undo copy, raw English Docker text to the household | **design question** for the operator: decision 154 (R-638 A) chose „start anyway"; D20 (R-893 A) gave „put back exactly" to the OFF-SITE path only | `restore_unit.go:428-489` |
| R2 a crash mid-restore starts the app on a mixed state; the undo copy is never read at start | PLAUSIBLE | defect | `appstop_marker.go:307`, `offbox_restore_undo.go:137` |
| R3 a cut-off shares restore can be placed and reported restored | CONFIRMED-CODE | defect (R-358 class, fixed for apps, not shares) | `shares_restore.go:131-140` readiness = non-empty dir |
| R4 `.fab` export drops a drive folder holding a symlink | PLAUSIBLE | defect | `appexport/fabplan.go:292`, `export.go:714` |
| R5 second-drive whole restore: files swapped, then the unit half refuses → app left STOPPED, text says „untouched" | **CONFIRMED** (unit test, production code: `C3-R5-whole-restore/` — 1 file restored + 1 replaced, app stopped, no start, message „az alkalmazás és az adatai érintetlenek") | defect | `tier2_whole.go:301-340`: stop + merge, then `RestoreFromRecoveryUnitAtWith` whose early refusals return before its own stop/start; `hu.json` `err.backup.unit_versions_mixed` „Az alkalmazás érintetlen" |
| R6 `.fab` import into a stopped app merges volumes | PLAUSIBLE | defect | `appexport/restore.go:361-374` |
| R7 `.fab` import has no guard / undo / record | PLAUSIBLE | defect | |
| R8 `.fab` import drops the family gate | **CONFIRMED LIVE** on 9202 (`C3-R1187e-fab-family-gate/`: stranger 401 before, 200 after the import) → **R-1191 (P1)** | defect | `cmd/controller/main.go:3517-3530`, `stacks/family_gate.go:173-190` |
| R9–R11, R13, R14 | PLAUSIBLE | defect / design (R13) | low |
| R12 interrupted whole restore leaves `*.felhom-restoring`, every retry fails | **CONFIRMED** (unit test: `C3-R12-leftover-temp/`) → R-1188 (c) | defect | `tier2_whole.go:86-87` O_EXCL; removed only on an in-process error |
| **D2** a missing / emptied app folder deletes its second-drive copy within one night | **CONFIRMED** for a MISSING folder (unit test, production code + real copy seam: `C3-D2-missing-leg/` — mirror of `photos` deleted, run „ok", the warning says only „nem kerültek a másodlagos mentésbe"); the EMPTIED case CONFIRMED-CODE (`rsyncMirror` = `rsync -a --delete`, `tier2.go:744-751`; the test seam does not delete) | defect-shaped corner of a choice: 07 §8.2 „data legs are NOT guarded" is about a leg LEAVING the class set; this is a declared leg that is MISSING, which the code itself logs as a gap (`tier2_capture.go:55-61`) and then reconciles away (`tier2.go:454`) | P1 for optional legs (no other tier holds them) |
| D3 / T11 a host clock ahead silences the whole-guest tier; the hub reads the same clock | PLAUSIBLE (B6 #8 on the rehearsal VM is its live test) | defect | agent `localapi/server.go:1178-1190` |
| D5 a dump that fails its own check has already replaced the good one | **CONFIRMED** (unit test with a docker stub: `C3-D5-dump-replaced/`) → R-1185 | defect for the ORDER | `appbackup/dbdump.go:388-422`: size check, rename (`:399`), then `ValidateDump` |
| D6 kept-data page can list a running app's folder | PLAUSIBLE | defect | needs an unreadable app.yaml |
| D7 unit restore can point the app at an old drive | PLAUSIBLE (low) | defect | |
| D8 local whole-guest keep=1 | design (07 §6.1 R-684 chose it) | design | **NOT-A-FINDING: the operator chose keep=1 for demo-hp; recorded in 07** |
| T1 the nightly mirror (`rsync --delete`) can run during a restore from that mirror | **CONFIRMED** (unit test, production rsync mirror: `C3-T1-mirror-during-restore/`) → R-1180 | defect | `RunAllTier2` (`tier2.go:479-520`) takes no backup flag; the whole restore holds only `acquireRunning` (`tier2_whole.go:301`) |
| T2 controller image without tzdata → UTC schedules | **NOT-A-FINDING:** measured on 9202 (controller 0.331.0, Debian 13 base): `/usr/share/zoneinfo/Europe/Budapest` present, scheduler logs „scheduled for … CEST" | — | |
| T3 whole-guest start creeps ~5 min/night | PLAUSIBLE | defect | needs the agent journal over weeks |
| T4 whole-guest quiesce stops apps mid-restore | PLAUSIBLE | defect | `quiesce.go:213-275` |
| T5 nightly off-site copy skipped SILENTLY when the flag is held; catch-up never retries | CONFIRMED-CODE → R-1186 | defect (the code chose nil for the nightly path; the catch-up gap is not chosen) | `offbox.go:981-996`, `main.go:1246` |
| T6 proof / integrity never run on a box off at 05:30–06:00 | PLAUSIBLE | defect (comment `main.go:1575-1577` says the opposite) | |
| T7 hub false „no DB dump" for late or evening W | PLAUSIBLE | defect | hub `monitor/deadline.go:416-419, 556-565` |
| T8 the OS leg follows a daytime valve backup | design question | design | 07 §6.1 / 11 §8.1 set no time bound |
| **T9** hub daily jobs run TWICE on 2026-10-25 | **CONFIRMED** (replay of the hub's exact arithmetic, `C3-T9-hub-dst/replay.txt`) | defect | deadline-check fires 04:00 CET AND 05:00 CET; db-snapshot twice (keep-2 → the previous day's copy pruned a day early). **In five days, on the live hub.** |
| T10 spring forward collision | PLAUSIBLE | defect (P4) | |
| T12, T13 | PLAUSIBLE | defect (P4) | |
| K3–K8 | PLAUSIBLE | K6 design | |
**NFC/NFD (the brief's question for A.1.4):** reviewer 1 found no normalisation anywhere — rsync, restic and the Go
merge compare names byte for byte, so the missing-only compare sees **two names** for the NFC/NFD pair. The household
data carries the pair; B8 measures it live.
@@ -0,0 +1,26 @@
# go test at controller main 7ca59e2 (v0.336.0), scratch clone, fake mirror seam (no rsync/docker); 2026-10-20T10:04:31Z
=== RUN TestRehearsalD1_DeadDriveShareCopyIsPruned
zz_rehearsal_d1_test.go:46: drive keys: map[hdd_1:hdd_1 hdd_2:hdd_2 hdd_3:hdd_3]
zz_rehearsal_d1_test.go:47: POSITIVE CONTROL — copies after run 1 (both drives present): [hdd_1: hdd_3/iskola hdd_2: hdd_1/csaladi_fotok]
zz_rehearsal_d1_test.go:48: mirror calls run 1: [hdd_1/csaladi_fotok=>hdd_2/backups/secondary/_shares/hdd_1/csaladi_fotok data/shares-payload=>hdd_2/backups/secondary/_shares/_payload hdd_3/iskola=>hdd_1/backups/secondary/_shares/hdd_3/iskola data/shares-payload=>hdd_1/backups/secondary/_shares/_payload]
zz_rehearsal_d1_test.go:56: FAULT — copies after run 2 (hdd_1 away): [hdd_1: hdd_3/iskola hdd_2: hdd_1/csaladi_fotok]
zz_rehearsal_d1_test.go:57: mirror calls run 2: [hdd_3/iskola=>hdd_1/backups/secondary/_shares/hdd_3/iskola data/shares-payload=>hdd_1/backups/secondary/_shares/_payload]
--- PASS: TestRehearsalD1_DeadDriveShareCopyIsPruned (0.02s)
=== RUN TestRehearsalD1_TwoGroupsOneTarget
zz_rehearsal_d1_test.go:73: copies after two runs: [hdd_1: hdd_3/c_share hdd_2: hdd_1/a_share]
zz_rehearsal_d1_test.go:75: LOG [INFO] [shares] tier-2 copied 1 share(s) from hdd_1 → hdd_2/backups/secondary/_shares (18 B)
zz_rehearsal_d1_test.go:75: LOG [INFO] [shares] tier-2 copied 1 share(s) from hdd_3 → hdd_1/backups/secondary/_shares (36 B)
zz_rehearsal_d1_test.go:75: LOG [INFO] [shares] tier-2 copied 1 share(s) from hdd_1 → hdd_2/backups/secondary/_shares (18 B)
zz_rehearsal_d1_test.go:75: LOG [INFO] [shares] tier-2 copied 1 share(s) from hdd_3 → hdd_1/backups/secondary/_shares (36 B)
--- PASS: TestRehearsalD1_TwoGroupsOneTarget (0.01s)
=== RUN TestRehearsalD1_SameTargetThenDriveDies
zz_rehearsal_d1_test.go:89: POSITIVE CONTROL run 1: [hdd_1: hdd_3/csaladi_fotok]
zz_rehearsal_d1_test.go:91: LOG [INFO] [shares] tier-2 copied 1 share(s) from hdd_2 → hdd_1/backups/secondary/_shares (18 B)
zz_rehearsal_d1_test.go:91: LOG [INFO] [backup] Tier 2 reconcile: removed stale dest dir hdd_2/b_share
zz_rehearsal_d1_test.go:91: LOG [INFO] [shares] tier-2 copied 1 share(s) from hdd_3 → hdd_1/backups/secondary/_shares (42 B)
zz_rehearsal_d1_test.go:100: FAULT run 2 (hdd_3 away): [hdd_1: hdd_2/b_share]
zz_rehearsal_d1_test.go:102: LOG [INFO] [backup] Tier 2 reconcile: removed stale dest dir hdd_3/csaladi_fotok
zz_rehearsal_d1_test.go:102: LOG [INFO] [shares] tier-2 copied 1 share(s) from hdd_2 → hdd_1/backups/secondary/_shares (18 B)
--- PASS: TestRehearsalD1_SameTargetThenDriveDies (0.01s)
PASS
ok gitea.dooplex.hu/admin/felhom-controller/internal/backup 0.049s
@@ -0,0 +1,104 @@
package backup
// THROWAWAY proof for the 2026-10-20 data-loss hunt (candidate D1). Never committed.
import (
"os"
"path/filepath"
"sort"
"testing"
)
func listShareCopies(env *sharesEnv) []string {
var out []string
for name, d := range env.drives {
base := filepath.Join(NamespaceRoot(d, true), "backups", "secondary", SharesPseudoStack)
keys, _ := os.ReadDir(base)
for _, k := range keys {
if !k.IsDir() || k.Name() == filepath.Base(sharesPayloadDestRel) {
continue
}
subs, _ := os.ReadDir(filepath.Join(base, k.Name()))
for _, s := range subs {
out = append(out, name+": "+k.Name()+"/"+s.Name())
}
}
}
sort.Strings(out)
return out
}
func keyOwner(env *sharesEnv) map[string]string {
m := map[string]string{}
for name, d := range env.drives {
m[sharesDriveKey(d)] = name
}
return m
}
// A drive whose shares were mirrored goes away; another share group's run lands on the same target.
func TestRehearsalD1_DeadDriveShareCopyIsPruned(t *testing.T) {
env := newSharesEnv(t, "hdd_1", "hdd_2", "hdd_3")
env.addShare(t, "hdd_1", "csaladi_fotok", true)
env.addShare(t, "hdd_3", "iskola", true)
if err := env.m.RunSharesTier2(); err != nil {
t.Fatal(err)
}
t.Logf("drive keys: %v", keyOwner(env))
t.Logf("POSITIVE CONTROL — copies after run 1 (both drives present): %v", listShareCopies(env))
t.Logf("mirror calls run 1: %v", env.mirrored)
if err := env.sett.SetDisconnected(env.drives["hdd_1"], true, nil); err != nil {
t.Fatal(err)
}
env.mirrored = nil
if err := env.m.RunSharesTier2(); err != nil {
t.Fatal(err)
}
t.Logf("FAULT — copies after run 2 (hdd_1 away): %v", listShareCopies(env))
t.Logf("mirror calls run 2: %v", env.mirrored)
}
// Shares on two drives that both pick the SAME target: does group A's reconcile remove group B's copy?
func TestRehearsalD1_TwoGroupsOneTarget(t *testing.T) {
env := newSharesEnv(t, "hdd_1", "hdd_2", "hdd_3")
env.addShare(t, "hdd_1", "a_share", true)
env.addShare(t, "hdd_3", "c_share", true)
var removed []string
env.m.logger = newCaptureLogger(&removed)
if err := env.m.RunSharesTier2(); err != nil {
t.Fatal(err)
}
if err := env.m.RunSharesTier2(); err != nil {
t.Fatal(err)
}
t.Logf("copies after two runs: %v", listShareCopies(env))
for _, l := range removed {
t.Logf("LOG %s", l)
}
}
// Shares on hdd_2 and hdd_3 (both pick hdd_1 as target). Then hdd_3 dies.
func TestRehearsalD1_SameTargetThenDriveDies(t *testing.T) {
env := newSharesEnv(t, "hdd_1", "hdd_2", "hdd_3")
env.addShare(t, "hdd_2", "b_share", true)
env.addShare(t, "hdd_3", "csaladi_fotok", true)
var logs []string
env.m.logger = newCaptureLogger(&logs)
if err := env.m.RunSharesTier2(); err != nil {
t.Fatal(err)
}
t.Logf("POSITIVE CONTROL run 1: %v", listShareCopies(env))
for _, l := range logs {
t.Logf(" LOG %s", l)
}
logs = nil
if err := env.sett.SetDisconnected(env.drives["hdd_3"], true, nil); err != nil {
t.Fatal(err)
}
if err := env.m.RunSharesTier2(); err != nil {
t.Fatal(err)
}
t.Logf("FAULT run 2 (hdd_3 away): %v", listShareCopies(env))
for _, l := range logs {
t.Logf(" LOG %s", l)
}
}
@@ -0,0 +1,18 @@
package backup
import (
"log"
"strings"
)
type capW struct{ dst *[]string }
func (c capW) Write(p []byte) (int, error) {
s := string(p)
if strings.Contains(s, "reconcile") || strings.Contains(s, "tier-2 copied") {
*c.dst = append(*c.dst, strings.TrimSpace(s))
}
return len(p), nil
}
func newCaptureLogger(dst *[]string) *log.Logger { return log.New(capW{dst}, "", 0) }
@@ -0,0 +1,8 @@
# go test at controller 7ca59e2 (v0.336.0), scratch clone; the real copyTree mirror seam (rsync --delete stands in by copyTree for the emptied case — see note); 2026-10-20T10:19:17Z
=== RUN TestRehearsalD2_MissingAndEmptiedLegDeleteTheirMirror
zz_rehearsal_d2_test.go:22: POSITIVE CONTROL after run 1: mirror photo=true doc=true
zz_rehearsal_d2_test.go:33: run 2 err=<nil>; status="ok" warning="Figyelmeztetés: a(z) app alkalmazás egyes adatmappái nem kerültek a másodlagos mentésbe: appdata/app/photos."
zz_rehearsal_d2_test.go:34: FAULT after run 2: mirror photo=false doc=true (the second-drive copy is the household's only local copy of these files)
--- PASS: TestRehearsalD2_MissingAndEmptiedLegDeleteTheirMirror (0.02s)
PASS
ok gitea.dooplex.hu/admin/felhom-controller/internal/backup 0.030s
@@ -0,0 +1,35 @@
package backup
// THROWAWAY proof for the 2026-10-20 data-loss hunt (candidate D2). Never committed.
import (
"os"
"path/filepath"
"testing"
)
func TestRehearsalD2_MissingAndEmptiedLegDeleteTheirMirror(t *testing.T) {
m, src, target, prov := newTier2V2(t, "app")
prov.has["app"] = true
prov.binds["app"] = []ClassifiedBind{mHDD("appdata/app/photos"), mHDD("appdata/app/docs")}
mustWrite(t, filepath.Join(src, "appdata", "app", "photos", "family.jpg"), "PHOTO")
mustWrite(t, filepath.Join(src, "appdata", "app", "docs", "contract.pdf"), "PDF")
if err := m.RunTier2("app"); err != nil {
t.Fatal(err)
}
destBase := filepath.Join(target, "backups", "secondary", "app")
photo := filepath.Join(destBase, "hdd", "appdata", "app", "photos", "family.jpg")
doc := filepath.Join(destBase, "hdd", "appdata", "app", "docs", "contract.pdf")
t.Logf("POSITIVE CONTROL after run 1: mirror photo=%v doc=%v", exists(photo), exists(doc))
// The household (or an app bug, or fsck → lost+found) loses the live photos folder; the docs folder is emptied.
if err := os.RemoveAll(filepath.Join(src, "appdata", "app", "photos")); err != nil {
t.Fatal(err)
}
if err := os.Remove(filepath.Join(src, "appdata", "app", "docs", "contract.pdf")); err != nil {
t.Fatal(err)
}
err := m.RunTier2("app")
cd := m.settings.GetCrossDriveConfig("app")
t.Logf("run 2 err=%v; status=%q warning=%q", err, cd.LastStatus, cd.LastWarning)
t.Logf("FAULT after run 2: mirror photo=%v doc=%v (the second-drive copy is the household's only local copy of these files)", exists(photo), exists(doc))
}
@@ -0,0 +1,9 @@
# go test at controller 7ca59e2 (v0.336.0), scratch clone; a docker STUB on PATH (no real docker reached); 2026-10-20T11:07:38Z
=== RUN TestRehearsalD5_InvalidDumpReplacesTheGoodOne
zz_rehearsal_d5_test.go:26: POSITIVE CONTROL: last night's dump, 152 B, ValidateDump={Valid:true TableCount:1 Error: FileSize:152 ModTime:2026-10-20 13:07:39.022566077 +0200 CEST UserTableFound:false UserRows:0 LooksEmpty:false}
2026/10/20 13:07:39 [WARN] [backup] ValidateDump FAIL: /tmp/TestRehearsalD5_InvalidDumpReplacesTheGoodOne2769942191/002/unit/app-postgres.sql — dump file too small (< 100 bytes)
zz_rehearsal_d5_test.go:31: tonight's dump: err=<nil> validation={Valid:false TableCount:0 Error:dump file too small (< 100 bytes) FileSize:99 ModTime:2026-10-20 13:07:39.026566071 +0200 CEST UserTableFound:false UserRows:0 LooksEmpty:false}
zz_rehearsal_d5_test.go:32: FAULT: the file on disk now = 99 B, still holds CREATE TABLE photos: false
--- PASS: TestRehearsalD5_InvalidDumpReplacesTheGoodOne (0.00s)
PASS
ok gitea.dooplex.hu/admin/felhom-controller/internal/appbackup 0.008s
@@ -0,0 +1,35 @@
package appbackup
// THROWAWAY proof for the 2026-10-20 data-loss hunt (candidate D5 / R-1185). Never committed. A docker STUB on PATH
// (no real docker is reached) prints a header-only dump, as a database that was re-initialised empty would.
import (
"context"
"io"
"log"
"os"
"path/filepath"
"testing"
)
func TestRehearsalD5_InvalidDumpReplacesTheGoodOne(t *testing.T) {
bin := t.TempDir()
stub := "#!/bin/sh\ncase \"$1\" in\n inspect) echo true ;;\n exec) printf -- '-- PostgreSQL database dump\\nSET statement_timeout = 0;\\n\\n--\\n-- PostgreSQL database dump complete\\n--\\n' ;;\nesac\n"
if err := os.WriteFile(filepath.Join(bin, "docker"), []byte(stub), 0o755); err != nil {
t.Fatal(err)
}
t.Setenv("PATH", bin)
dumpDir := filepath.Join(t.TempDir(), "unit")
final := filepath.Join(dumpDir, "app-postgres.sql")
good := "-- PostgreSQL database dump\nCREATE TABLE public.photos (id int);\nCOPY public.photos (id) FROM stdin;\n1\n2\n\\.\n\n--\n-- PostgreSQL database dump complete\n--\n"
os.MkdirAll(dumpDir, 0o755)
os.WriteFile(final, []byte(good), 0o644)
t.Logf("POSITIVE CONTROL: last night's dump, %d B, ValidateDump=%+v", len(good), ValidateDump(final, DBTypePostgres))
db := DiscoveredDB{ContainerName: "app-db", ContainerID: "0123456789abcdef", DBType: DBTypePostgres, DBUser: "u", DBName: "d", StackName: "app"}
res := DumpOneTo(context.Background(), db, final, log.New(io.Discard, "", 0), false)
b, _ := os.ReadFile(final)
t.Logf("tonight's dump: err=%v validation=%+v", res.Error, res.Validation)
t.Logf("FAULT: the file on disk now = %d B, still holds CREATE TABLE photos: %v", len(b), contains(string(b), "CREATE TABLE public.photos"))
}
func contains(s, sub string) bool { return len(s) >= len(sub) && (func() bool { for i := 0; i+len(sub) <= len(s); i++ { if s[i:i+len(sub)] == sub { return true } }; return false })() }
@@ -0,0 +1,10 @@
# go test, hub store at felhom.eu 5c48731 (scratch clone), in-memory SQLite; 2026-10-20T10:12:12Z
=== RUN TestRehearsalK1_NewBoxAfterHostDeleteSeesNoEscrow
zz_rehearsal_k1_test.go:20: POSITIVE CONTROL (old host live): status=&{IdentityBlobPresent:true ResticPwSHA256:SHA_OLD CreatedAt:2026-10-18T00:00:00Z SupersededPresent:false SupersededAt:} err=<nil>
zz_rehearsal_k1_test.go:27: retained rows under the DELETED host_id: 1 (identity blob present: true)
zz_rehearsal_k1_test.go:34: FAULT (new box, what its ACK carries): status=<nil> err=<nil>
zz_rehearsal_k1_test.go:36: => GetEscrowStatusForCustomer returns nil: identity_blob_present=false AND superseded_present=false for the new box
zz_rehearsal_k1_test.go:39: retained rows reachable under the NEW host_id: 0
--- PASS: TestRehearsalK1_NewBoxAfterHostDeleteSeesNoEscrow (0.04s)
PASS
ok gitea.dooplex.hu/admin/felhom-hub/internal/store 0.048s
@@ -0,0 +1,40 @@
package store
// THROWAWAY proof for the 2026-10-20 data-loss hunt (candidate K1). Never committed.
import "testing"
func TestRehearsalK1_NewBoxAfterHostDeleteSeesNoEscrow(t *testing.T) {
s := newTestStore(t)
const cust = "Tester4"
// The household's first box, with an escrowed package (K blob + the identity blob that opens the off-site repo).
if err := s.UpsertHost(&Host{HostID: "Tester4-old", CustomerID: cust, APIKey: "k1"}); err != nil {
t.Fatal(err)
}
if _, _, err := s.SaveHostEscrow("Tester4-old", []byte("k-blob"), "fp", "zk", "2026-10-18T00:00:00Z", "SHA_OLD"); err != nil {
t.Fatal(err)
}
if err := s.SaveHostDRBundle("Tester4-old", []byte("identity-opening-the-old-repo")); err != nil {
t.Fatal(err)
}
st, err := s.GetEscrowStatusForCustomer(cust)
t.Logf("POSITIVE CONTROL (old host live): status=%+v err=%v", st, err)
// The operator deletes the old host (the installer's 409 tells them to), escrow acknowledged → demoted.
if err := s.DeleteHost("Tester4-old", true); err != nil {
t.Fatal(err)
}
ret, _ := s.ListSupersededEscrow("Tester4-old")
t.Logf("retained rows under the DELETED host_id: %d (identity blob present: %v)", len(ret), len(ret) == 1 && len(ret[0].IdentityBlob) > 0)
// The reinstalled box enrolls under a NEW host_id for the same customer.
if err := s.UpsertHost(&Host{HostID: "Tester4-new", CustomerID: cust, APIKey: "k2"}); err != nil {
t.Fatal(err)
}
st, err = s.GetEscrowStatusForCustomer(cust)
t.Logf("FAULT (new box, what its ACK carries): status=%+v err=%v", st, err)
if st == nil {
t.Logf("=> GetEscrowStatusForCustomer returns nil: identity_blob_present=false AND superseded_present=false for the new box")
}
ret2, _ := s.ListSupersededEscrow("Tester4-new")
t.Logf("retained rows reachable under the NEW host_id: %d", len(ret2))
}
@@ -0,0 +1,306 @@
Tue Oct 20 10:09:14 UTC 2026
filebrowser Up About an hour (healthy)
felhom-controller Up About an hour (healthy)
paperless-webserver Up 3 hours (healthy)
paperless-postgres Up 3 hours (healthy)
paperless-redis Up 3 hours (healthy)
immich-server Up 3 hours (healthy)
immich-postgres Up 3 hours (healthy)
immich-machine-learning Up 3 hours (healthy)
immich-redis Up 3 hours (healthy)
traefik Up 4 days
0a4fb1a71c72cf704d1afd44e1ab24a8a68e11b8d82ef1dcb71a75d61a9dc1a3
0ce36c2e6db54e085f71a1436c4c89a9807a930c373198f451fa1665f90f0cab
0e2ca5e8738b95134285a666b1a7622bfcabc4728fb9269cc972dfaccbd59d45
0f4002ab75695fb312775a8203cc5213b256f1b01d255210cb5cdd0e50cd9070
1a8fe48409be3defe01713e2f621091630b965e1805cbb5872d1738043efd35c
1afe3420067df65e0d3e53baaec3debc39d2d42d333c7a3c266e4f6ff98e04f7
1b41940af5c4fa82af59320d6c81ba622d97f2e43d8ec51c50dcf65c2fdcb1e8
1f5120663962c7b03539bb993ab736be4800096e3d1f7147eb922d44b308ae6a
01feef94eb7e38a3679921b66cfe206e08adfd9b360ed4a60414cde606de1643
2a71e17dfa1e930ffe0c213cd1615c8d1adcbbf8d61615a38064d331c265c02e
2aaab84bf843817d9e6f32b673277fd1e0c2c8c2d038b8126df44927b1b41006
2ac49fc9cc2839f55c8abae76039950d15a5c301ea5b3918fa3e5d8715a15f42
2bbdb463b29493d60d352bafefccbe57856cf80bd2cc193b61f9bcc1310fb45e
2bc95b104191ffd31bdfdbe1e038d284d0624c04735094494c2c77d957e8e905
2dcb4bd2c3c618d71f582d33d5b1243e6ab075d3b00abc238d4430668a2b6f55
2e4c79db25c47436b44245376b1f6ff1991208ba081fddce1938c3897fc63c07
2ef8994a563ae88024ba29db4e349d26e6e3889e0683d30d1fb258208c9a10b4
2f08e882c5d23373e2df5486dce28d4728330f9f0131a2735ebc938bfb4d7b3a
02f42b55ee403ceef63a472c3624da0b48fab9e6aa6dc0c7800ec68b2f2041ef
3a3d7bc33a1ba21ae4a39c8cb345254cf66cd1dc60afae64e2299513b094b93b
3ac9820f84d0a50f233c63a62ee1b63b92acac026d68c60a8227f370b0be00b6
3d9a93769342e83a3c512d517b99cd64f8e794eb305c009e25c73a6f6627ad29
3d9c1f23abbcef771a4c38a75d75865a12d76a4359fe83c1828652dba3f91710
3f015f020f49d4967d2ac71b94060e6eaec86bc8852cf9a0b5f3dd8133dd528c
3fa030697dbaffb6809991bb69a335dc4e9accfafd4ed5ee953ed676673912e6
4a9ed2aed1fe75a0b02f688e29a20acc2bf7e081ba7518a7d75451be30fdc4e1
4bbaae5ae8cc8cc3330a73bb872b16e1420805197fa9bad903f7db9a1f358041
4c54a4cda3c5280d5b13facf3089f8a0710c4570876aaeb679025aabf92df99f
4e052b4ef5602a68eddcea049bbc4e79b306012cf8b7f0c9cfc3e6c39af64fce
4ea780981a54c3e87aa72eb80436e02f3c6b3aaeab96ecf869602f5f5b0b546e
4f2ebb779645de5cf9d24e93c6fbd9d95d358b390dec94eed6f0223016785e42
4f2266bb0b8a9b902eaa439ae9225963d6fe4359fe30569c2d8afa926b582585
004ae946f5997471f4554ab7158a3b982b72391b59c59be40ea79b501eb1dab9
5b84763ca63a28ae79c1393d700c534e7dd6e610f890f540981e49972e3854e9
5c7dd57f445d87a39eea1e7b299e1a0acbbc946d6bfc92ba165c4af8933b322a
5d47a8950cfd56492e0f8b4ed6bfbf470a2e22811206ac0da0b79e520890b41e
5def44013bf255d4cd97c3829ac7ae895c607333a3ee586a4090bda128d379fd
5ec44fe3223976727132521b3055187e88783716a1f5fd07d1625205acff9888
6a84846fd8e476d206eafb8f5a529ecbdf6b1021daab890429077e32ef4c2b6b
6aef4d0808bef53a15fc6d19614b641a5e36c941f2729572cfef8cfc8e36bbfd
6b5a55f08cc63514e70a93a3eb5d294e9b109e990521bb0ddcea4818a8df1456
6b8acdd640ae067c2ee5235ddea8ad4c0e1eec2aaef1277b426923f56303e07d
6bcaa587af8bf282f83605713213c20275b1c8178c28de97b154af25bb6a674f
6bdf1f89b902e5a7134a87e0269229531ec6c8391256ea1bf466d2d6b66dd8c9
6bf7012e27730a047bff7d69ceadef9c06433783d0b9582faefa9c9d0f966c86
6ce965b12082d21ed8ade00884de6477faea9f97cf7a85e4b5ab3302d3517fce
6ddb2a450b19b940661c677b20936e92e8de54b8bc76f8dbacc56398d7d13a38
6e21c6398b13ebac045a11b136fbb2d289e6abe99401eed9108b5464754a4c87
6e0521e4314af444375b56e348a5e3d34aceb88d2d5e2f48b6986c27250e395e
6f19aa563c4e3e2000ed997157bc7f5bc8a45ee1d77c2cdcd1fd239117a62705
7b5530f7f440ff6477f2ee5885b37ff9cfbf933d7908225998875b233b4d2a2a
7bfe063b0155297f5bca22c7ed02ae7ef9ce1b968670607415f7720e1e8649f0
7ef24d6bf9fb190db2572505b489e45731a326c5c9c4ae6e36ffd7874560bbab
7fc41ddc47424ea11d4940f7062cdae69b92af645e6759f8759f47b689a19cb7
8b44a3824395c930d0db784e5650ba7b58663090c3bd72a44f9dc716008a9642
8bdcbc26d645ae3bc60742debe32650510bcfb7c69ad4416688af5b1815e5645
8c2a7cf3f809218d20576368d4d419a63a07a89c3b12cc389fb2f3424fb45b73
8c4d45c90f336c0ffd5d619765d8d2a45e802d619548f187bb3f2672b3534969
8cae3c834dd9284d1d0069de042c2d50e49a138a78d93a59eb752272e4a5e4f4
8d2ab3e77a72ebd708630e22f2eefc0a66f73cd94ed2abbfc4f666c504a3f172
008dfd2a2e98c6b6673ee471d72b9e150cfe02859eb3a3bbd31a3c582b5a3a46
9a2e6958ba1ea4443ade7619d326c84c50dbc444810b47904ff625969725a091
9a86729a66d00d8be95f93062381b9bd91b16f94b654636681553fec6c7c7985
9abdd18e6a7b07800067237c93d48448d49d9ad2388a91d35653323c2bed4389
9c59fdcefb7ed258e615f8cb688334a69aebdb8e38fd1075a60d6fd98167065a
9c033896f0b00023816a49cea882a342c9b62ca4528640969a0d22c21f507505
9db05859a329f9544f5a48cbd987f02fd00e48f160dcf6ee8331d8accfb423b4
9e9fa98cb2344c5eda9bae7417557f2bc2503de1eed8219fdc2d2d378e1883fe
9e06437718bd8f9a27240eff7b425406aae50f061ea47a6991b9830ee2af9f59
9ee6f261789b97ea0887a6a905bf5bba61a7070cc91ff64cda0091923d9cdf25
09ea67268ce8c0442357ba316f6431cd96143da6bb54008d3c342be8b6ddb169
13b744ec13f00bd3e5e2ecaa37bf5385af38cf0ff94fd05e660cfed3339a0c05
13d76d014354ffb4e5aabfe00eeed5206b98c502dcfd515545fdc7210316bf61
18e51d43a77ac282e108fd8a8e55207282cdabd2219af1e47c2c214117179f51
20ee459190ea3442ab7cf0c288182033fb84b75a15bdfc14540e80ab60dec542
22ab14c474cdb6cf9f873dcb6c8ce85dfc588abb46e2bd0925cdeff384437c46
24a30fa240cec28f9a8b6d02a47ad1fda39674f094ae095b0d6ec3cbf099ed44
24da0da42dc2eb9b6d35dcb7cd513b1e5c7d2ee38b6d59d9a1d51f0d01f6f920
27a56be2953f040882ecd07fd8b91937f8cb17c25d1ceed56656ed2075518f86
31f3b426743b1cff7952948995915586480486ed7fd06eef3379cf54b8fb3d9a
38ba44b0ff38f8df925512a477b8b5b4cba216a8ec0bd51b7463a86a88c42fea
43d6928bc4bb7a1e097122424e35f22393a0ccf2e03863d7dc3cce44163c8ae8
49d0cebfc3199cef395ae27a6ffa479340c1c2f69da9248a777d7bff9b6b9ca9
49dbe15539d7170822958bb0699d140c4d493215236b32679340588710d5452b
054cfa9b87316799ecd24ecfb480a38aa04b5f85a478a3db95c41706e5b63523
57e9dc8d409d939d877a513023c95146400dd10078919ab9402d8144d5f30ca0
62e86d5d6aded4a8ba536696c0df0f0af89f2cd7b4b893a7383e3abc00eeadc5
63ddb5c60e6cf39dea412903cf1d6c0164a1146bcf4ab98ee94c9b1f9e45c6bb
63e2655474549ef03aaf7d9d741502287d2f01170c5ca008d7b09b8e00b1be66
67eff4c5c60d493dc2a9f7841f7f138b610dc389c32793a27a62ddc13d417a1c
68d18a8d3519b6ea6928bbca40f9db7500b700f66e6eab236e7c32a214de2cc7
68f83c9613471fe0e046a1f9252b71e86aa50d36315c675e92dc44a7cdd0267e
71cb6156444948894aec343c9a2c0c7674aca84d292d090c7f786d13146709ed
72ed18f5962e48dd5fb54e7d91d2fd0479017c27b2c60fc58a5d6b6e55242757
74c6b39b7b926dd56db16f175026c2bf9e80a912916f273c122227672f181d03
075b39bc7ae8c56a6d99923e4938ddbbb4174df6f0e653abf9c99b420b401a31
78c01e39d1997eb54344f02f14dc02c7230cfb14ba58abc1a11feab2e9af85be
78e7acb9228c1df4ca4c706d459b97c8e79e8399fe06476ce72bd919a3b2b77f
83a8287d1e8700aa7e207b2c617a68557ce6f83f373f2c3c6d1443ae50032826
83bc388f3ae110e72ba51a8f0b68f7bbdec3685907c1ff08095b63a1759502af
85d6f7309436e59cce9e373b0aa1626fa5e12d8d81163773f5f2c45da9d56543
88c6573974e81f1fff96999bf54b80c66bc6f1e9af6746b3cd6801616198e0fd
92e2e73e3aef6e5330d2ce8aed3c58e7e3632a6f191f2b03c997ccabd245bb65
93c2b9d0a1f54c2e6355d206d93bf2309f81076628aae9d868406c288b0f6bf4
98a80f5f7903ef45f92f74a91f34dc01d527f8a2fe98b4fbe76711f5259416a5
99f7aaac2646c66dd0eeeebc157b8563b56299c27ab4bf64e13920d089ca7156
104e16e4796c52fe212c04b06417161e0bab20fc01632bfcf71421b73b979fbc
122fe66bacc9f04b3f3e4f6d0a3b15abdef8321eed36637a1284e3bd145463b8
158c590779b033e48b5efcf895fd0bad4c6f277303d72b963cd0802ecac1f440
218a186ab13a38d514cf9f523e5798452ae4e5ec76ecf26fe1a46ae22b47a508
283e9d53742a13c471e5f59f53a57c179d461f0017ad5f8db27881cdd994cd04
305b75817cf5c06105b19dc8c56ff0d796dc6e267ae5bcfc6018aa3d12037744
325ec2aa2a6c307bb82d4dcb600fa3ff127aa6d1f35dc683ab61124ef5f0f5b3
338f7535ae9ef91754e76604238ca7652273ffce9e6f0a1300de0b1edb61da77
343d41a3b38b5a7eac10750eb0213b81780edd3075e9a1fdf3a2c479f4d2474c
346eaddf98e91d6d6e7b8c526074c27847e4a18be01b41027bec93abc2b7b2fb
355ba69bab94a95e00e87c6c4808587881b9675699083e087917ec8492bebc65
426ccc08685def1c9025c94d343de0e59713073984634ee226e69634d6a18fb8
552a4b340eb6be03c188cb17628aee0f916b544142465f8e52e7e9d2325206ab
556c805068afd3f6784bda2eb59642d7dc03756580e959e1c75262711a30d4d9
0642eb9eee222e235de9310b9bd86b42493e5f4d71d430619b5832484dd09cd6
645d70ab53013564fd243eae61db209e3793b826a9eeffd77395b18e27458789
670e6447721f667df5222cb06a68c2682eb92803184a44faead8aea0dff44df6
0717b5f7615c345fe47ab8212c06f506e08585551d93c6a8083b56a216adb71b
734e6b63f955d0b508dea39a8637755e75fbb353d54430956a0010f607eb727d
748a1560ef6216c8bf80545dbeadd29f99fd5c984b6f34bb678febf8b369adcc
748cce6aa136bdc0b3111c73a7c9434508200f8c231933b925d42b37b9cca17c
760cd2f9203dc3dc3f933e4a34bbaae857dee0afed66e727846a4c7f750ab5a8
762e15e05b2f84f035a7011e10c64558dc6637c9fdde1846dc77f4056f3962d2
787dc06078f3718da6faaba82269a7d21b213482c9e45025315bdb6cf59010b5
866fc46ac37407bbf74761cecd21fa6322c4e7621f3dfbe963a85671d5753e5c
921d92a1c692ea0fd42cbe500bbdb1f3defcd5fb9b51df216451aa260d95ad45
932bae2149e79ef113eb4929614a895c642960160d5f5cb2f2e9afac7dfa6af2
968a5d7a54fb1d8ef4163f287e128cbd541ab46a2cc54db721dd3eafe1397b65
991a5d344112325fb2c40700cd0ee96a784ce2150cba0a7b0be3ed6eb2411878
2469a8c88a452ed226dd6d1082f0cbfbe3107d76f919b930b6bc7ce3b839ed90
2634a60c3c37d5eba4be4827c08c0e99a951b934604c29d116fcddc27808e480
2788da1d8f917a9d5b8acab2c7fe7336b6a7810f1c7edab63ed327a6022ceddf
3000c7863b2e80a72329ca1fa0dd03d2fd3994a2391e82450a8c92199cbb56b6
3461d48cd9e88caf5363b0e4d96e799f6759ab8b8bc6ba4c148492a1637c01f0
4681b8c67ad20fdd3ad04db895084d37a9296ac76b74ea7bc9465bb3ac93df8f
4690cfe7c61625190732c028ad4b918930a65e295005171d48114aa8f27b567b
5206b60d49110c01e05dc24034e12b767efa739e2ad21c252ab83db47e56d526
5234fcace55ee7e44631499ed1f95fd0289918a84bd9f5cf6d9f761d43c34f17
07402fbe22b572302f3317958f8cc4c6ca30317e0ee5bbbe9ff51c60b3da5a05
7458b4c4b975debbd2bb705e9575d2d1ff5e9308ed9e70c302f5643136cd0cc0
8097f97f4ae00b654b1c0b4408ad585f5bfce7999e482f244bd698268e1f6061
8158fe1f757d26cadb484ed9690ad4d20d4cb382eeec7c9b0627b365afd6576f
9917d608ac0ad05856d31873432d641828ef519825a3f451e6a1ab2519dbf0d1
13175cc97740c834e2404c11ba9fa9418d0c390d27a87e6e869977d6a86c20c0
36571aa758bc335acc463c3ea074422a5123c309a37e1a7fc818a2069941c493
59438a5f6378d218a73d3106ea1984d1c160fe4fcbbb7ec1177067f17430dcb3
60496cd3a1e1ad1dcaed30427b70a57b8b14a122bae7f2cd882f8154a4549978
61544bee8e2c1cd65e999121bb63011a9c1b4139b20b04fa32e3a850ca8c82b7
84046f48a922cd337284bc2448a5d2682ea6a2e3479e71531122aff9ee4cbc93
86022b0cd1043facf7b1c933c3b6a6ff36277a76d50bdb2da6a5690d57381b9e
98129d711a1e807ab7c3ae4c97187ab8505cebf1012218e8530d0829a2ad436f
177451abc7ed0606468250a7793e018725e8c7f370ee8937d4941a11571dcb9d
0217869eb1be5181b7053b993443dac73d9cd0804cd5bedb604edeb43f911702
219118eb01fa7be03fcce2124b74708cacafe66aea617fd2bafa32f0b2d4fd80
394985d0c3db0791b8c7569fccabd36959d00deb285075f9852568c4c81d32eb
722380f945884d4dea921ffb958513e02a4802483ec0886712e5ed3fa28d751c
774878c58b6a8cae6fbecdf656f2c1f09f37a89190068663e923763dbd6969ce
925226a74657c7ed0fcdbcfa9b504a47f7e21d0982c44f7ed9576140fb417146
963079a9efb903a575b363b27e15927312569528b9066511b4c282e01e1ec931
3830913f0478ec59a7a31d538f858f8ef612e1b2ce3bb9058a2d18f7b43abe9c
8579968bcd362a1aa9dcf14e22956b2fd15fd74a2fc2e7369ffb1457c2ce6693
8678479fa55ea3871598eb6914eb8428ba599da4c619c62f231e6931f0cb1f54
13694424a3895e8e73cc1e06af29aae0c3179c3cb5587b9daadeadcdb29b7cc7
67711784cb23efbc0762e83f03365b837ec5216c38f3ba682a82eaff6a7863db
92236992ea73ecd9befe3330e88e96a7aedacd9cbca0bb8c32c31d0e2c914e5b
134995217d29dc47fb0ca1a274ceb5ea74225d36a98c4bbe6e07bf8c8d6898c7
481706314b835ab042583c3107e168932844a162a2adf3b9ef2fa27433a7fd1d
4098005900d38c71b152787ce4bdedfb8e2b2b8b061f87645b2b4a14e0004335
4699188665aa6eb965b7bb9f4938187f377e7a4ad55fc97e44802ea76f3612e9
04762469942ae24c353f929fced777f245bb9f2f888a9f2044ff27fc45933665
a6c297b9282b39caca7e513724219af096a3cf6fd453c3736ee7c4c833ef417d
a22cb584ba3444bc1dbd90b3723b1ad9424352251f2218a92b1884ef74e63b9f
a68fad1bcb1f1030af3811590cc18d74eb1da3c67190af4459fa69118aa8f035
a74b9b2333ee936c17edd0525b49fec1337768ccab68f5576a39486c875ff968
a74b41238ed9ded9e866679ab00f85813d800f92e4bb2642d3f9de28a1655b81
a84ff849188b2bab211fefe8bc61a70f58f48f6a2a6ca25c2645e0bb5c384cbf
a2815ec8af59e0a137aa0cdf2baa48ca6d1b86e291c5728fab9449280d15fa75
aa5cf2d08c68b8d3dfe3330861935fd97ff8fb9169645f6c632d970ad71b575c
aadb8f0be78890f1e08630b7a762a82853dc1402326bd16cd2a2844fa63fef1d
ac48d92438e8d19cf546764cc051172af82092223daad8ccfbfb4cb05100f5d6
ac78e0a5ac10c7d0239e0e1ba3b604cd6fd418d87a6f32b66f3ce98c674164d8
ad79ebffc7d8225cfbee6ad5fec4092ba7376f38dca912d6df8ea932f1eba20b
ad7084f002b17a1a27f70c796e3d7424a064b2ef5b71eadb7b3b08dcc9f59042
ae2a94eac2960e95f944c9a4a0649c94c405fb0e518adfba2e6fa0f80f2a5af2
ae9d5c15cba1adf881044a4449eaf72f5f2ed753c301edc1e43bcfa4560e8d31
ae5499adb73000b6f460e63d6e045fe492702bd73187f868d9988690cb72a453
b0e341cb8fb7f2fb42f7a8512a9b922591cfa40bd3522f31a422a9fdb4586008
b4bc9ff2d9b533e896dfc5082ba865a4ea7676d71259d73aa33d515a5a60dd91
b8d2fbe7e6afc931581bf5f1edacaefb9071bcc9f709e294a3f5ead104f73890
b8dd8697ad52f98dc44ed3c940752aa52e16d6ab7563d36b1c9e221db22d1e11
b21ec6b73a9f1f51623355f2bf8229b5f6f9da1484e1ce105cf8e0363475a5b8
b081c3329ed821e22b0ee9cecbc2f52a5e9be70be2b1cb83bb262c21c84640d1
b274ece5f6e1d506ca0646077964797c3e7b4413c9a9f1e453d9257e6b4d09f9
b676fcf4f5272015b942c8589135a18fa16204c5aec457f1d0898ce6c84638eb
b1591e2a2604b6404d0d18e4b2b7599aafbcfb2bbacba06b5de5ff14abf14be3
b7000ea518cd327f7dfb42858434fdb80c451aecf0a0325b6ccbff1f30545803
beb17a9d330ce361923600aac8e13f8b10e62d0b4067380ad7a95c4d77f0110e
bfb91f5666855fd57228801ed0a4294981bd2a6665a93d1093bf3c4a0e89cb2c
c0fb7358aabef2c38e6921e2ed06b7fdddddecd29d530814545968323eba964e
c1d4fc53203c4b65abb1ff76f9505a46175c57afcd9c95cf445c6a3c443839fb
c3fbcb924a928bca859b69c75e4df275eb5e0abf03029785247313b0fa1cdd27
c4bccf1c99f4990b0a23848b6fa58977dea0bdbf6ab69b7c0c2b90f7f0a2f5c3
c5e875c34f4e4f1fd4edb1c6dbf4862a6543e26387ee480eeba0f5e3f44af7b6
c8bb03639a4d7c41c233fa28d5a1c06dee874fe00614f764df9db3df14da86b5
c16aa92f949bb89a034c6ef22ea8ece859883ab309086a204e43cedced8dd059
c31f2070f92b36708a62c1d53a7aea495fd997440f2c01c851ad83c66357e379
c35e0b961694a882dcc4c7bdb8e657d4b0685687cb5c4469559ce5078c95302a
c56e78b468dfe3991fc787ab7fe30b4282a73d7fd2e79e4dfc9305bb064685ea
c67a888239bc5e11ed44ca31307f3b46f7f76698d706ffc764024110d1a69ceb
c424d546895a59e0f88e2296a665c03f4618ac304ef19742b6f1c543665b9bb3
c716ebc8ba8fa1556e1dddfaae98c7019618d39ece0e78dbd1b9f58dd0760d94
c797d99e68b0efce9b1337dc0e3e9cbf42806bef03021ade362cb4e4d1de4222
c5708e0a13ababf73203b1e297b3b73229cd6370e3fec1653b7cf1dc143a0d0a
c52417cfa3d923ce7168b7c2b082eb3ab31bc1750ff3f3edfc2c1cd59fb74f3d
c53655b36c7abc28db3020511a0ebdffe8fa37d86412753fca86f4caa7ae7a6b
c782734d8be184e0721d0a984b6ac3e8e0052206a9e46d22188a7fb2f923face
cb71ab036375acb25c69afd915261acf65f919008fbd59417989f6ebef110c7a
cb613ceec1bf86b718059ef3e864dce7ed60e5b6a7bd5ab9f7c633350df48f3a
cb10732ef556d77ec0a0a5d7b29c0789b47a9757b67bdb3fa9d5be22809debd0
cf799a4b43f87d63eba6042a73b9eba3f17b354be1fd2d94dfac2048e13f3fce
d3c941cd28b5a692c2532f1381afdc57740d603fbba5eddbe3539f561f26a0ca
d5bd7d95fbb1f56d7617d96a5a6deb65417b4448692cb45f4a0bedd824165fc1
d6a74fc39944848c79012bb06d915c609b92c1099881b90b4cda59074bd1124f
d58b42f643175b6272bb579d354a9510bc34fc88e4e70dda08346997ba3cec13
d256b9490d623a136093e459b9f466dedadaa0d00235d6934044e6591ee7e2c5
d352bb11678cf6cb47a16740ee9b79cfbf4c19d4da23f4fb95621bbdea6b91b2
d053441be51f569ac393c26c220b6369ae3455ae76f3627eba223e8b4e40eacd
daa717bf6363a3afe8dbcd3ab863a81ca737e0999242bd99c618daa0e4fa085b
db0c37c3ce9df2f2090c622eb4194e6860e26846398d4b4a092b9c4c21441af4
db2434b8bc6fb7858aa5eea51b23eb3fe17ed8e8ac025596ebb4f857d00f9ba9
db6910df9d6429a8ecffced74f22c26f45a11d48efea51e48e8097bf43331540
dbcfeebe1162c457fdf6f7cbfff66c971dfaca8a1a8ae2fb5472fb9007f83ddc
dc5c95c9a8f2f01a8eab63c7d591a7d008dbaf5f4b6157380faeeaeab168cd59
de81766fb8e806f93c97771ae1a0ff529831bc5e7f50e2187d41dd606b8e23f1
e0e25d3ed1797b0092c897ad904a4ac7a2bd77e12b96e1790a939aa009851803
e0ee75c206f1b9be41e608c6de05f8fad8532e3bda240cfad0640c673ae46395
e1ae949c5339eca43334dd6720b49df19082036b8f53151b8dfac97f2d2c0b6d
e2ca0579185aa6d7324f144357323b4d5521c1e94e8dc7929f0ffa9a73e1e4cd
e5f9a9a8f7e4b6d097aef1b0493180a547dbf19006920433e7b80aa8aac0026b
e6a40ba40f9ac5f43ba2276b3cb6110bb4c0e20003e866a55ca28c954ddd0e9a
e9e1b0601616187e8934c7212e2fde5b590b50308aa9c8c5dd12e419c6bebc18
e45c71e0680d2856e2945881fd01ad6320868e0a36427c6bca637c53dfecc625
e62af03dd8d599dfbdbecb8c6516d95e582ee803f2ddd3798479fbc769229030
e72bfb25201770edb1889f678f5a844a57c9f5308acd57a86c5f4936b800d441
e311fa91eca24a97e0c558e91d06dca353aa9348a7ea4b111e694e181868fc27
e339a495e0c048e30483e4fd3bef66186712975f3c33c90ef63a3c9686da3160
e668b1c1f44dae28ca34f530e313c1e544751b63bf0c480300be8e242683fecb
e4638cbb5b12548a76a2cb6e87c7f2ada75d61d3c7d1062535ef0bf2b3cde05d
e7945baeafe0ac6d9a1e6761786c35fddad76c81d86e544d369188c186a0389b
ea35c704202ee36415d4f7a0873bcb254d30268450d7c9ed2dc611c1e2f4d5d1
eaeb108b23d455d01ec0c9e44d39ff395522942794d26f3c149e29615be9bf1f
eb420ae07f9c459abc101c1264fa8cf779002ef20748ef97abe7c3764c24f032
ec30ca07d0b0af16afe60f217dedf15573045ed9adc45e21c3592ee6716fbe39
ed02b9c56e48d34910bc97ebff834f1cb082c23ebcae97b0f51f3a2633664466
edcc77bb3d279e3dfe7d687fe30e22f95f5d9391b9908e61b2206a6cf309ea91
eea963e8499939d36d09e42d8d5cd86798e81e264c4d4c012191e42926e508b8
f1b01ee5efb2117033ddcad5f5eafe8e6f65ad6fc3e933902bfc494c13a7d3c0
f01b151ee48f9a3a631457069ece3a2ac3e111df28110f3c06f221ee7c54eff8
f2ec30f9f836ed26cb13fdb80064d1dcabf953d1952fea2d85bb6410f4cb2271
f8a87cab6bc0890c7a2e36f73e533eac1a1cabb13b886b8ed71afa4c8b7d90be
f9b0fe025aedb49e94d80119a8a8dc0f85800bc592e568dc95b0f1e5439c54fc
f56a1ab986e83e118ff174e9805f6151e47ff693c17a2f475fcedd83cb98e051
f82a956615af08264b64e051e09c1441152d92371d4d2db8d5c0ac4ab47a7182
f287b03a768126d72af210dceab12e329573779e7aad56c123ac0dbb14fc4f4e
f347e779fb8c09daba2a757dfc3ee7c415f8f5a2eaace19535d363cc3ab3171a
f547c232f5e1785aa11200ac03dcfda6f056400bf1456a3388db6d73a8158ca0
f6822a239614fc82462535db5b2806dd616696c9552e1d469922f4b7cd2bf56e
fa453f6fb451a564d31e704bbca0bcbcc9ad1109a25a8e5ad5c704789a424c4b
fb5ad60698efb6ee9f45fd440e56c060ba01ba03a854f556e42f8090b2057c7e
fbc407de7b12a4975b60dac518ac287f076659213cce708871162342419ab383
fbf633eb758b7b66ab761c34eeebc759791e2f73020e839b3cfc2d692c9ba21f
fc1bddcbef8c02474db71bf11dace70e1e51d94f3c3fe6d9d45f4a669ac4a9fd
fd453dd47dc82ff7625bd4093c62140fe14a348814f094b43b519f7f59293af7
fdc05b3bfda3101bec7d66b5e0cfb589e06f1f4b4d6b2f9d84dac470b0abf53b
fdebde6f8bcdd5281c50bcb4669c45dc54e33563bec872f8d23d76d2fe3ebfdf
fe503d3e9a34b113d4c56b59c5986e13c44b6e401b6e6681eb62712d24368329
felhom-controller-data
filebrowser_filebrowser_data
immich_immich_ml_cache
immich_immich_postgres_data
immich_immich_redis_data
paperless-ngx_paperless_data
paperless-ngx_paperless_postgres_data
paperless-ngx_paperless_redis_data
privatebin_privatebin_data
ls: cannot access '/mnt/felhom-drives/scratch_hdd/felhom/backups/primary/': No such file or directory
Filesystem Size Used Avail Use% Mounted on
/dev/loop1 69G 11G 54G 17% /var/lib/felhom
/dev/nvme0n1 938G 210G 681G 24% /mnt/felhom-drives/scratch_hdd
@@ -0,0 +1,104 @@
10:10:26Z == 1 deploy
10:10:27Z deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}
10:11:12Z state: running
10:11:42Z == 2 marker A + backup
10:11:46Z CREATE TABLE
INSERT 0 1
10:11:49Z [2] DB markers: 'A'
10:11:49Z POST /api/backup/run -> 200 {'ok': True, 'message': 'Mentés elindítva'}
10:12:59Z backup finished: {"db_dump": {"count": 3, "duration": "1m9.055678483s", "last_run": "2026-10-20T10:12:58.690358712Z", "success": true}, "enabled": true, "running": false}
10:13:04Z unit on disk:
UNIT /mnt/sys_drive/felhom-data/backups/primary/rallly
2026-10-20 10:12 487 /mnt/sys_drive/felhom-data/backups/primary/rallly/compose/app.yaml
2026-10-20 10:12 4071 /mnt/sys_drive/felhom-data/backups/primary/rallly/compose/docker-compose.yml
2026-10-20 10:12 11894 /mnt/sys_drive/felhom-data/backups/primary/rallly/compose/.felhom.yml
2026-10-20 10:12 52228608 /mnt/sys_drive/felhom-data/backups/primary/rallly/volume-dumps/rallly_rallly_postgres_data.tar
2026-10-20 10:11 111371 /mnt/sys_drive/felhom-data/backups/primary/rallly/db-dumps/rallly-postgres.sql
2026-10-20 10:12 3195 /mnt/sys_drive/felhom-data/backups/primary/rallly/manifest.json
2026-10-20 10:12 1191 /mnt/sys_drive/felhom-data/backups/primary/rallly/data-stamps.json
10:13:04Z == 3 POSITIVE CONTROL: marker B (live) then a normal unit restore
10:13:09Z INSERT 0 1
10:13:12Z [3-before] DB markers: 'A,B'
10:13:12Z [3] restore points offered: [{"time": "2026-10-20T10:11:50Z", "short_id": "helyi", "tier": 1, "drive_label": "Belső SSD (rendszer)"}]
10:13:12Z [3] POST /backup/restore -> HTTP/2 302 ['location: /backups/restore?flash=flash.restore.started']
10:13:12Z [3] restore-status: {"running": true, "op": "restore", "stack": "rallly", "started_at": "2026-10-20T10:13:12.817369857Z", "last": {"op": "offbox-reconstitute", "stack": "rallly", "ok": true, "message": "A(z) rallly: 0 fájl és 1 adatkötet és az adatbázis visszaállítva (mentés: 2026. 10. 20. 09:30) — az alkalmazás újraindult.", "finished_at": "2026-10-20T07:36:57.852528534Z"}}
10:13:40Z [3] restore-status: {"running": false, "op": "restore", "stack": "rallly", "started_at": "2026-10-20T10:13:12.817369857Z", "last": {"op": "restore", "stack": "rallly", "ok": true, "message": "A(z) rallly: 1 adatkötet és az adatbázis visszaállítva — az alkalmazás újraindult.", "finished_at": "2026-10-20T10:13:39.161573115Z"}, "last_recent": true}
10:14:00Z [3] after: state=running hold=None
10:14:18Z [3-after] DB markers: 'A'
10:14:18Z == 4 marker C (the newest live row), then THE FAULT: the unit's DB volume tar truncated to half
10:14:22Z INSERT 0 1
10:14:25Z [4-live] DB markers: 'A,C'
10:14:29Z TRUNCATED /mnt/sys_drive/felhom-data/backups/primary/rallly/volume-dumps/rallly_rallly_postgres_data.tar 52228608 -> 26114304
10:14:29Z == 5 the unit restore over the damaged copy
10:14:29Z [5] restore points offered: [{"time": "2026-10-20T10:11:50Z", "short_id": "helyi", "tier": 1, "drive_label": "Belső SSD (rendszer)"}]
10:14:29Z [5] POST /backup/restore -> HTTP/2 302 ['location: /backups/restore?flash=flash.restore.started']
10:14:29Z [5] restore-status: {"running": true, "op": "restore", "stack": "rallly", "started_at": "2026-10-20T10:14:29.288204925Z", "last": {"op": "restore", "stack": "rallly", "ok": true, "message": "A(z) rallly: 1 adatkötet és az adatbázis visszaállítva — az alkalmazás újraindult.", "finished_at": "2026-10-20T10:13:39.161573115Z"}, "last_recent": true}
10:14:44Z [5] restore-status: {"running": false, "op": "restore", "stack": "rallly", "started_at": "2026-10-20T10:14:29.288204925Z", "last": {"op": "restore", "stack": "rallly", "ok": false, "message": "Visszaállítás sikertelen: starting rallly after restore from unit: starting stack rallly: exit code 1\nstderr: ...twork rallly_rallly-internal Creating \n Network rallly_rallly-internal Created \n Network rallly_rallly-internal Created \n Container rallly-postgres Creating \n Container rallly-postgres Created \n Container rallly Creating \n Container rallly Created \n Container rallly-postgres Starting \n Container rallly-p
10:15:04Z [5] after: state=degraded hold=None
10:15:38Z [5-after] DB markers: 'Error response from daemon: Container 6012fe2936d0bbea71dd1226f64d872b9d017ae5f5062a2ea234d84f1522029c is restarting, wait until the container is running\nError response from daemon: Container 6012fe2936d0bbea71dd1226f64d872b9d017ae5f5062a2ea234d84f1522029c is restarting, wait until the container is running\nError response from daemon: Container 6012fe2936d0bbea71dd1226f64d872b9d017ae5f5062a2ea234d84f1522029c is restarting, wait until the container is running'
10:15:41Z containers:
rallly Created
rallly-postgres Restarting (1) 5 seconds ago
10:15:44Z db container log tail:
Data page checksums are enabled.
initdb: error: directory "/var/lib/postgresql/18/docker" exists but is not empty
initdb: hint: If you want to create a new database system, either remove or empty the directory "/var/lib/postgresql/18/docker" or run initdb with an argument other than "/var/lib/postgresql/18/docker".
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are enabled.
initdb: error: directory "/var/lib/postgresql/18/docker" exists but is not empty
initdb: hint: If you want to create a new database system, either remove or empty the directory "/var/lib/postgresql/18/docker" or run initdb with an argument other than "/var/lib/postgresql/18/docker".
10:15:47Z any undo / safety copy of the live state on disk?
/mnt/sizing-bench/backups/secondary/paperless-ngx/recovery-unit/db-dumps/pre-restore-20260927T094411Z-paperless-ngx-postgres.sql
/mnt/sys_drive/felhom-data/backups/primary/rallly
/mnt/sys_drive/felhom-data/backups/primary/rallly/volume-dumps/rallly_rallly_postgres_data.tar
/mnt/sys_drive/felhom-data/backups/primary/rallly/db-dumps/rallly-postgres.sql
10:15:51Z controller log (restore lines):
Container rallly Created
Container rallly-postgres Starting
Container rallly-postgres Started
Container rallly-postgres Waiting
Container rallly-postgres Error dependency rallly-postgres failed to start
dependency failed to start: container rallly-postgres is unhealthy
2026/10/20 10:14:42 manager.go:1362: [ERROR] [stacks] Stack rallly start failed after 1.1s: exit code 1
stderr: ...twork rallly_rallly-internal Creating
Network rallly_rallly-internal Created
Network rallly_rallly-internal Created
Container rallly-postgres Creating
Container rallly-postgres Created
Container rallly Creating
Container rallly Created
Container rallly-postgres Starting
Container rallly-postgres Started
Container rallly-postgres Waiting
Container rallly-postgres Error dependency rallly-postgres failed to start
dependency failed to start: container rallly-postgres is unhealthy
2026/10/20 10:14:42 handlers.go:1881: [ERROR] [web] Restore failed (async): stack=rallly: starting rallly after restore from unit: starting stack rallly: exit code 1
stderr: ...twork rallly_rallly-internal Creating
Network rallly_rallly-internal Created
Network rallly_rallly-internal Created
Container rallly-postgres Creating
Container rallly-postgres Created
Container rallly Creating
Container rallly Created
Container rallly-postgres Starting
Container rallly-postgres Started
Container rallly-postgres Waiting
Container rallly-postgres Error dependency rallly-postgres failed to start
dependency failed to start: container rallly-postgres is unhealthy
2026/10/20 10:14:49 manager.go:920: [DEBUG] [stacks] restart-policy of down member "rallly" = "unless-stopped"
2026/10/20 10:14:59 healthprobe.go:105: [DEBUG] [stacks] RunHealthProbes: rallly has a health check but no container to probe — candidates: [rallly(stopped) rallly-postgres(restarting)]
2026/10/20 10:15:09 healthprobe.go:105: [DEBUG] [stacks] RunHealthProbes: rallly has a health check but no container to probe — candidates: [rallly(stopped) rallly-postgres(restarting)]
2026/10/20 10:15:09 main.go:4054: [INFO] [deadapp] rallly crossed the crash_loop threshold (8 in 10m0s) while a backup/quiesce/update holds it — NOT stopped
2026/10/20 10:15:19 healthprobe.go:105: [DEBUG] [stacks] RunHealthProbes: rallly has a health check but no container to probe — candidates: [rallly(stopped) rallly-postgres(restarting)]
2026/10/20 10:15:29 healthprobe.go:105: [DEBUG] [stacks] RunHealthProbes: rallly has a health check but no container to probe — candidates: [rallly(stopped) rallly-postgres(restarting)]
2026/10/20 10:15:39 healthprobe.go:105: [DEBUG] [stacks] RunHealthProbes: rallly has a health check but no container to probe — candidates: [rallly(stopped) rallly-postgres(restarting)]
2026/10/20 10:15:49 healthprobe.go:105: [DEBUG] [stacks] RunHealthProbes: rallly has a health check but no container to probe — candidates: [rallly(stopped) rallly-postgres(restarting)]
exit 0
@@ -0,0 +1,18 @@
2026/10/20 10:14:29 handlers.go:1873: [WARN] [web] Restore requested (async): stack=rallly, snapshot=helyi from 172.18.0.3:36322
2026/10/20 10:14:29 restore_dbcreds.go:114: [INFO] [backup] Restore rallly: database password(s) [DB_PASSWORD] come from the backup, [] stay as installed (older than the data) — values not logged (R-1106)
2026/10/20 10:14:29 restore_unit.go:393: [INFO] [backup] Restoring rallly from recovery unit /mnt/sys_drive/felhom-data/backups/primary/rallly: images=2, secrets recovered=2/2, data_keys=0
2026/10/20 10:14:29 manager.go:1440: [INFO] [stacks] Stopping stack: rallly
2026/10/20 10:14:40 manager.go:1449: [INFO] [stacks] Stack rallly stopped successfully (took 10.8s)
2026/10/20 10:14:40 restore.go:215: [INFO] [backup] Restoring Docker volume rallly_rallly_postgres_data for rallly
2026/10/20 10:14:40 restore.go:246: [ERROR] [backup] Failed to populate volume rallly_rallly_postgres_data: tar: short read — exit status 1
2026/10/20 10:14:40 restore_unit.go:448: [ERROR] [backup] volume restore for rallly: failed to restore 1 volume(s): [rallly_rallly_postgres_data]
2026/10/20 10:14:40 [INFO] [stacks] SaveAppConfig: saved config for rallly
2026/10/20 10:14:40 deploy.go:883: [INFO] [stacks] Redeploying rallly from recovery unit with 4 env vars
2026/10/20 10:14:40 pin.go:93: [INFO] [stacks] pin rallly: rallly=lukevella/rallly:4.15.3, rallly-postgres=postgres:18-alpine
2026/10/20 10:14:40 [INFO] [stacks] SaveAppConfig: saved config for rallly
2026/10/20 10:14:40 restore_unit.go:467: [ERROR] [backup] rallly: NOT replaying the database copy — the volume restore failed, so the database is not known to be the copy's own
2026/10/20 10:14:40 manager.go:1351: [INFO] [stacks] Starting stack: rallly
2026/10/20 10:14:42 manager.go:1704: [ERROR] [stacks] Command failed: docker compose up -d (in /opt/docker/stacks/rallly) — exit code 1 (took 1.1s)
2026/10/20 10:14:42 manager.go:1712: [ERROR] [stacks] stderr (last part): ...twork rallly_rallly-internal Creating
2026/10/20 10:14:42 manager.go:1362: [ERROR] [stacks] Stack rallly start failed after 1.1s: exit code 1
2026/10/20 10:14:42 handlers.go:1881: [ERROR] [web] Restore failed (async): stack=rallly: starting rallly after restore from unit: starting stack rallly: exit code 1
@@ -0,0 +1,28 @@
200
{
"ok": true,
"message": "Stack rallly stop completed"
}
200
{
"ok": true,
"data": {
"removed": "rallly",
"volumes_removed": [
"rallly_rallly_postgres_data"
],
"hdd_paths_removed": [],
"hdd_paths_preserved": [],
"hdd_note": "Az alkalmazás nem tárolt saját adatot külső meghajtón, így ott nem volt mit törölni.",
"userdata_kept": [],
"backup_paths_removed": [
"/mnt/sys_drive/felhom-data/backups/primary/rallly (26M)"
],
"verified": true
},
"message": "Stack rallly removed"
}
0
0
ls: cannot access '/mnt/sys_drive/felhom-data/backups/primary/rallly': No such file or directory
/opt/docker/stacks/rallly
@@ -0,0 +1,25 @@
# C3 — R1/D4 proven live: a failed LOCAL restore leaves a broken app and no copy of what it replaced
Scratch guest 9202 on demo-hp, controller **0.331.0** (the local-restore code is unchanged since v0.328.0 —
`git log` on `restore_unit.go`/`restore.go`; 0.336.0 runs the same lines). Hub off on 9202. The dashboard password was
reset with the controller's own escape hatch (`--print-reset-code` → `/claim`), held in `sec` only.
Architecture read: `07` §6.3 (the ladder „replay → rollback → hold"), §8; `09` §3 decision 154 (R-638 A), D20 (R-893 A).
| step (UTC) | act | observed |
|---|---|---|
| 10:10 | Rallly 4.15.3 + postgres:18 deployed through `POST /api/stacks/rallly/deploy` | running |
| 10:11 | marker A (psql in the app's own DB container); „Mentés most" (`POST /api/backup/run`) | unit written 10:12 (`rallly_rallly_postgres_data.tar` 52 MB + `rallly-postgres.sql`) |
| 10:13 | **positive control:** marker B, then the household's restore (`POST /backup/restore`, point „helyi") | „1 adatkötet és az adatbázis visszaállítva — az alkalmazás újraindult"; markers **A** (B rewound, as designed) |
| 10:14:22 | marker C — the household's newest row | live: **A,C** |
| 10:14:29 | **fault:** the unit's DB volume tar truncated to half (a torn / damaged copy) | 52 228 608 → 26 114 304 B |
| 10:14:29 | the same restore | stop 10.8 s → `volume rm` → `tar: short read` → `[ERROR] volume restore … failed` → definition rewritten → „NOT replaying the database copy" → start → **postgres crash-loops** (`initdb: directory … exists but is not empty`) |
| 10:14:44 | what the household reads | „Visszaállítás sikertelen: starting rallly after restore from unit: starting stack rallly: exit code 1 stderr: ...twork rallly_rallly-internal Creating …" — **raw English Docker output** |
| 10:15:47 | any copy of the live state (marker C)? | **none** — no undo dump, no live-volume copy for Rallly (`find` of `*restore-undo*`/`pre-restore-*` finds only a September paperless file) |
| 10:15 | the dead-app guard | `rallly crossed the crash_loop threshold … while a backup/quiesce/update holds it — NOT stopped` |
**So:** the household's live data (marker C) was destroyed by the restore itself, the app does not start, and nothing
on the box can put the pre-restore state back. The off-site restore got exactly this protection in v0.336.0 (D20 = A:
„put back exactly as it was"); the LOCAL path — 07 §3's main customer restore — did not. Logs: `01-run.log`,
`02-controller-log-fault-window.txt`. Teardown through the product (`03-teardown.txt`): Rallly removed with its data
and backups; no container, no volume, no unit (the `/opt/docker/stacks/rallly` folder is the catalog template every
app has).
@@ -0,0 +1,153 @@
#!/usr/bin/env python3
"""r1_proof.py — C3 proof of candidate R1/D4 on scratch guest 9202 (data-loss hunt 2026-10-20).
Claim under test: a LOCAL (Tier-1 unit) restore that fails after it has started replacing the app's volumes
restarts the app on whatever is left, and keeps NO copy of the live state it replaced — so the household's newer
data is gone and nothing can put it back (unlike the off-site path, which got „put back exactly" in v0.336.0).
Method (mechanism proof — markers are written with psql inside the app's own database container, as the
2026-10-19 R-893 measurement did; the household-facing acts are the dashboard's own endpoints):
1 deploy Rallly (throwaway) through POST /api/stacks/rallly/deploy
2 marker A; „Mentés most" = POST /api/backup/run (whole box, scratch); wait
3 POSITIVE CONTROL: marker B (live), unit restore (POST /backup/restore) -> A present, B gone (expected rewind)
4 marker C (the household's newest live row); THE FAULT: truncate the unit's DB volume tar to half
5 unit restore again -> observe: the answer, the app's state, what the DB holds, any undo/safety copy on disk
6 teardown: remove Rallly with its data and backups through the product; verify nothing left
Env: CTL_BASE, CTL_DOMAIN, CTL_PW_ENV (see tools/ctl.py). Prints a timestamped log (no secrets).
"""
import json, os, re, subprocess, sys, time
sys.path.insert(0, "/mnt/5_hdd/felhom.eu/git/felhom.eu/documentation/audits/rehearsal-2026-10-20/tools")
import ctl # noqa: E402
APP = "rallly"
DOMAIN = os.environ["CTL_DOMAIN"]
def say(*a):
print(time.strftime("%H:%M:%S", time.gmtime()) + "Z " + " ".join(str(x) for x in a), flush=True)
def guest(script, timeout=600):
r = subprocess.run(["ssh", "demo-hp", "export LC_ALL=C; cat > /tmp/r1p.sh; pct push 9202 /tmp/r1p.sh /tmp/r1p.sh;"
" pct exec 9202 -- bash /tmp/r1p.sh; pct exec 9202 -- rm -f /tmp/r1p.sh; rm -f /tmp/r1p.sh"],
input=script, capture_output=True, text=True, timeout=timeout)
return (r.stdout + r.stderr).strip()
def psql(sql):
return guest(f"""C=$(docker ps --format '{{{{.Names}}}}' | grep -E '^rallly.*(postgres|db)' | head -1)
U=$(docker exec $C printenv POSTGRES_USER); D=$(docker exec $C printenv POSTGRES_DB)
docker exec -i $C psql -v ON_ERROR_STOP=1 -U "$U" -d "$D" -tA <<'SQL'
{sql}
SQL""")
def stack():
code, d = ctl.call("GET", f"/api/stacks/{APP}")
return (d.get("data") or {}) if isinstance(d, dict) else {}
def wait_backup(t_cap=1200):
t0 = time.time()
time.sleep(5)
while time.time() - t0 < t_cap:
code, d = ctl.call("GET", "/api/backup/status")
dd = (d.get("data") or {}) if isinstance(d, dict) else {}
if not dd.get("running"):
return dd
time.sleep(5)
return {"timeout": True}
def restore(tag):
code, d = ctl.call("GET", f"/api/backup/snapshots?stack={APP}")
data = d.get("data") if isinstance(d, dict) else None
snaps = data if isinstance(data, list) else next((data[k] for k in ("snapshots", "items", "restore_points")
if isinstance(data, dict) and isinstance(data.get(k), list)), [])
say(f"[{tag}] restore points offered: {json.dumps(snaps, ensure_ascii=False)[:400]}")
sid = snaps[0].get("id") or snaps[0].get("snapshot_id") or snaps[0].get("short_id")
out = subprocess.run(["curl", "-sk", "-D", "-", "-o", "/dev/null", "-H", ctl.HOST, "-H", "Cookie: " + ctl._SESS,
"-X", "POST", "--data-urlencode", "_csrf=" + ctl._CSRF, "--data-urlencode", "stack_name=" + APP,
"--data-urlencode", "snapshot_id=" + sid, ctl.BASE + "/backup/restore"],
capture_output=True, text=True).stdout
say(f"[{tag}] POST /backup/restore -> {out.splitlines()[0] if out else '?'} "
f"{[l for l in out.splitlines() if l.lower().startswith('location')][:1]}")
t0, last = time.time(), None
while time.time() - t0 < 900:
code, d = ctl.call("GET", "/api/backup/restore-status")
dd = (d.get("data") or {}) if isinstance(d, dict) else {}
cur = json.dumps(dd, ensure_ascii=False)[:600]
if cur != last:
say(f"[{tag}] restore-status: {cur}")
last = cur
if not dd.get("running") and time.time() - t0 > 8:
break
time.sleep(3)
time.sleep(20)
st = stack()
say(f"[{tag}] after: state={st.get('state')} hold={st.get('hold_reason')!r}")
def markers(tag):
say(f"[{tag}] DB markers: {psql('SELECT string_agg(m, \',\' ORDER BY m) FROM r1_marker;')!r}")
def main():
say("== 1 deploy")
if not stack().get("deployed"):
code, d = ctl.call("GET", f"/api/stacks/{APP}/deploy-fields")
fields = (((d.get("data") or {}).get("metadata") or {}).get("deploy_fields")) or []
vals = {"DOMAIN": DOMAIN, "SUBDOMAIN": "r1poll"}
for f in fields:
if f.get("env_var") in vals:
continue
if f.get("type") == "password":
import secrets
vals[f["env_var"]] = "R1-" + secrets.token_hex(10)
elif f.get("required") and f.get("default"):
vals[f["env_var"]] = f["default"]
code, d = ctl.call("POST", f"/api/stacks/{APP}/deploy", {"values": vals})
say("deploy ->", code, str(d)[:160])
for _ in range(90):
time.sleep(5)
st = stack()
if st.get("deployed") and (st.get("app_config") or {}).get("pinned_images") and st.get("state") in ("running", "unhealthy", "degraded"):
break
say("state:", stack().get("state"))
time.sleep(30)
say("== 2 marker A + backup")
say(psql("CREATE TABLE IF NOT EXISTS r1_marker(m text primary key); INSERT INTO r1_marker VALUES ('A') ON CONFLICT DO NOTHING;"))
markers("2")
code, d = ctl.call("POST", "/api/backup/run")
say("POST /api/backup/run ->", code, str(d)[:200])
say("backup finished:", json.dumps(wait_backup(), ensure_ascii=False)[:500])
unit = guest(f"for d in /mnt/felhom-drives/*/backups/primary/{APP} /mnt/sys_drive/felhom-data/backups/primary/{APP}; do "
f"[ -d $d ] && echo UNIT $d && find $d -maxdepth 3 -type f -printf '%TY-%Tm-%Td %TH:%TM %s %p\\n'; done")
say("unit on disk:\n" + unit)
say("== 3 POSITIVE CONTROL: marker B (live) then a normal unit restore")
say(psql("INSERT INTO r1_marker VALUES ('B');"))
markers("3-before")
restore("3")
time.sleep(15)
markers("3-after")
say("== 4 marker C (the newest live row), then THE FAULT: the unit's DB volume tar truncated to half")
say(psql("INSERT INTO r1_marker VALUES ('C');"))
markers("4-live")
say(guest(f"for d in /mnt/felhom-drives/*/backups/primary/{APP} /mnt/sys_drive/felhom-data/backups/primary/{APP}; do "
f"[ -d $d ] || continue; f=$(find $d -name '*.tar' -path '*volume*' | grep -iE 'postgres|db' | head -1); "
f"[ -n \"$f\" ] || f=$(find $d -name '*.tar' | head -1); s=$(stat -c %s \"$f\"); "
f"truncate -s $((s/2)) \"$f\"; echo \"TRUNCATED $f $s -> $(stat -c %s \"$f\")\"; done"))
say("== 5 the unit restore over the damaged copy")
restore("5")
time.sleep(30)
markers("5-after")
say("containers:\n" + guest("docker ps -a --format '{{.Names}} {{.Status}}' | grep -i rallly"))
say("db container log tail:\n" + guest("C=$(docker ps -a --format '{{.Names}}' | grep -E '^rallly.*(postgres|db)' | head -1); docker logs --tail 15 $C 2>&1"))
say("any undo / safety copy of the live state on disk?\n" + guest(
"find / -xdev \\( -path /proc -o -path /sys \\) -prune -o \\( -iname '*restore-undo*' -o -iname 'pre-restore-*' \\) -print 2>/dev/null | head; "
"for d in /mnt/felhom-drives/*/backups /mnt/sys_drive/felhom-data/backups; do find $d -maxdepth 4 -iname '*rallly*' -newer /etc/hostname 2>/dev/null | head; done"))
say("controller log (restore lines):\n" + guest("docker logs --since 30m felhom-controller 2>&1 | grep -iE 'restore|rallly' | grep -vi 'GET /api' | tail -40"))
if __name__ == "__main__":
main()
@@ -0,0 +1,36 @@
10:48:52Z == 1 deploy glance
10:48:52Z deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}
10:49:12Z state: running family_gate in app_config: True
10:49:34Z [1 POSITIVE CONTROL after install] gate file: /opt/docker/stacks/traefik/dynamic/family-gate-glance.yml
10:49:34Z [1 POSITIVE CONTROL after install] a stranger's GET https://r1187glance.enkisfelhom.hu/ -> 401
10:49:34Z == 2 export to the data drive
10:49:34Z export start -> 200 {'ok': True}
10:49:34Z /api/export/status {"display_name": "Glance", "done": false, "error": "", "job_type": "export", "ok": true, "output_path": "", "output_size": "", "running": true, "stack_name": "glance", "steps": [{"label": "Előkészítés", "status": "running"}, {"label": "Konfiguráció mentése", "status": "pending"}, {"label": "Adatbázis mentése", "status": "pending"}, {"label": "Felhasználói adatok", "status": "pending"}, {"label": "
10:49:37Z /api/export/status {"display_name": "Glance", "done": true, "error": "", "job_type": "export", "ok": true, "output_path": "/mnt/felhom-drives/scratch_hdd/exports/glance_20261020-104935.fab", "output_size": "5.6 KB", "running": false, "stack_name": "glance", "steps": [{"label": "Előkészítés", "status": "done"}, {"label": "Konfiguráció mentése", "status": "done"}, {"label": "Adatbázis mentése", "status": "done"}, {"la
10:49:37Z bundles for glance: []
10:49:37Z == 3 remove glance (keep nothing)
10:50:09Z remove -> 200 {'ok': True, 'data': {'removed': 'glance', 'volumes_removed': ['glance_glance_config'], 'hdd_paths_removed': [], 'hdd_paths_preserved': [], 'hdd_note': 'Az alka
10:50:21Z [3 after remove] gate file: /opt/docker/stacks/traefik/dynamic/family-gate-glance.yml
10:50:21Z [3 after remove] a stranger's GET https://r1187glance.enkisfelhom.hu/ -> 404
10:50:21Z == 4 import the bundle
10:50:21Z import start -> 400 {'error': 'Missing path', 'ok': False}
10:50:21Z /api/export/import/status {"display_name": "Glance", "done": true, "error": "", "job_type": "export", "ok": true, "output_path": "/mnt/felhom-drives/scratch_hdd/exports/glance_20261020-104935.fab", "output_size": "5.6 KB", "running": false, "stack_name": "glance", "steps": [{"label": "Előkészítés", "status": "done"}, {"label": "Konfiguráció mentése", "status": "done"}, {"label": "Adatbázis mentése", "status": "done"}, {"la
10:53:42Z state after import: not_deployed family_gate in app_config: False app_config keys: []
10:54:14Z [4 FAULT after import] gate file: (none)
10:54:14Z [4 FAULT after import] a stranger's GET https://r1187glance.enkisfelhom.hu/ -> 404
10:54:15Z controller log: 2026/10/20 10:48:52 manager.go:1686: [DEBUG] Env vars for compose: [PATH, HOSTNAME, FELHOM_BOOTSTRAP_PATH, HOME, DOMAIN, SUBDOMAIN, DOMAIN, IMPORT_PATH] (8 app + 0 system)
2026/10/20 10:48:58 manager.go:1686: [DEBUG] Env vars for compose: [PATH, HOSTNAME, FELHOM_BOOTSTRAP_PATH, HOME, DOMAIN, SUBDOMAIN, DOMAIN, IMPORT_PATH] (8 app + 0 system)
2026/10/20 10:49:34 family_gate.go:325: [DEBUG] [web] family gate glance: GET / without a pass — 401 (visitor 192.168.0.180)
2026/10/20 10:49:35 manager.go:1686: [DEBUG] Env vars for compose: [PATH, HOSTNAME, FELHOM_BOOTSTRAP_PATH, HOME, DOMAIN, DOMAIN, SUBDOMAIN, IMPORT_PATH] (8 app + 0 system)
2026/10/20 10:49:35 manager.go:1686: [DEBUG] Env vars for compose: [PATH, HOSTNAME, FELHOM_BOOTSTRAP_PATH, HOME, DOMAIN, SUBDOMAIN, DOMAIN, IMPORT_PATH] (8 app + 0 system)
2026/10/20 10:49:37 manager.go:1686: [DEBUG] Env vars for compose: [PATH, HOSTNAME, FELHOM_BOOTSTRAP_PATH, HOME, DOMAIN, DOMAIN, SUBDOMAIN, IMPORT_PATH] (8 app + 0 system)
2026/10/20 10:49:39 manager.go:1686: [DEBUG] Env vars for compose: [PATH, HOSTNAME, FELHOM_BOOTSTRAP_PATH, HOME, DOMAIN, SUBDOMAIN, DOMAIN, IMPORT_PATH] (8 app + 0 system)
2026/10/20 10:49:43 manager.go:1686: [DEBUG] Env vars for compose: [PATH, HOSTNAME, FELHOM_BOOTSTRAP_PATH, HOME, DOMAIN, DOMAIN, SUBDOMAIN, IMPORT_PATH] (8 app + 0 system)
2026/10/20 10:50:09 manager.go:675: [DEBUG] [stacks] ScanStacks: found stack "recipe-importer" deployed=false composePath=/opt/docker/stacks/recipe-importer/docker-compose.yml
2026/10/20 10:50:09 manager.go:675: [DEBUG] [stacks] ScanStacks: found stack "recipe-importer" deployed=false composePath=/opt/docker/stacks/recipe-importer/docker-compose.yml
2026/10/20 10:50:21 auth.go:144: [DEBUG] [web] auth: valid session for POST /api/export/import
2026/10/20 10:50:21 handler_export.go:328: [DEBUG] [web] apiImportStart: path="" hasPassword=false
2026/10/20 10:50:29 family_gate.go:190: [INFO] [stacks] glance: removed the family-gate file of an app that is not installed
2026/10/20 10:52:09 manager.go:675: [DEBUG] [stacks] ScanStacks: found stack "recipe-importer" deployed=false composePath=/opt/docker/stacks/recipe-importer/docker-compose.yml
2026/10/20 10:54:09 manager.go:675: [DEBUG] [stacks] ScanStacks: found stack "recipe-importer" deployed=false composePath=/opt/docker/stacks/recipe-importer/docker-compose.yml
exit 0
@@ -0,0 +1,24 @@
10:54:26Z == 4 import the bundle /mnt/felhom-drives/scratch_hdd/exports/glance_20261020-104935.fab
10:54:26Z import start -> 200 {'ok': True}
10:54:26Z /api/export/import/status {"display_name": "Glance", "done": false, "error": "", "job_type": "import", "ok": true, "output_path": "", "output_size": "", "running": true, "stack_name": "glance", "steps": [{"label": "Csomag megnyitása", "status": "done"}, {"label": "Alkalmazás előkészítése", "status": "done"}, {"label": "Konfiguráció visszaállítása", "status": "running"}, {"label": "Felhasználói adatok visszaállítása", "stat
10:54:29Z /api/export/import/status {"display_name": "Glance", "done": false, "error": "", "job_type": "import", "ok": true, "output_path": "", "output_size": "", "running": true, "stack_name": "glance", "steps": [{"label": "Csomag megnyitása", "status": "done"}, {"label": "Alkalmazás előkészítése", "status": "done"}, {"label": "Konfiguráció visszaállítása", "status": "done"}, {"label": "Felhasználói adatok visszaállítása", "status"
10:54:32Z /api/export/import/status {"display_name": "Glance", "done": true, "error": "", "job_type": "import", "ok": true, "output_path": "", "output_size": "", "running": false, "stack_name": "glance", "steps": [{"label": "Csomag megnyitása", "status": "done"}, {"label": "Alkalmazás előkészítése", "status": "done"}, {"label": "Konfiguráció visszaállítása", "status": "done"}, {"label": "Felhasználói adatok visszaállítása", "status"
10:56:13Z state after import: running family_gate in app_config: False app_config keys: ['deployed', 'deployed_at', 'desired_state', 'env', 'installed_images', 'locked_fields']
10:56:55Z [4 FAULT after import] gate file: (none)
10:56:55Z [4 FAULT after import] a stranger's GET https://r1187glance.enkisfelhom.hu/ -> 200
10:56:56Z controller log: 2026/10/20 10:54:33 manager.go:1814: [INFO] [stacks] glance glanceapp/glance:v0.8.5 running Up 3 seconds (health: starting)
2026/10/20 10:54:39 healthprobe.go:187: [DEBUG] Health probe glance: HTTP GET :8080/ → 200 (1ms)
2026/10/20 10:54:49 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping glance — last check 10s ago, effective interval 5m0s, healthy=true
2026/10/20 10:54:59 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping glance — last check 20s ago, effective interval 5m0s, healthy=true
2026/10/20 10:55:09 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping glance — last check 30s ago, effective interval 5m0s, healthy=true
2026/10/20 10:55:19 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping glance — last check 40s ago, effective interval 5m0s, healthy=true
2026/10/20 10:55:29 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping glance — last check 50s ago, effective interval 5m0s, healthy=true
2026/10/20 10:55:39 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping glance — last check 1m0s ago, effective interval 5m0s, healthy=true
2026/10/20 10:55:49 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping glance — last check 1m10s ago, effective interval 5m0s, healthy=true
2026/10/20 10:55:59 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping glance — last check 1m20s ago, effective interval 5m0s, healthy=true
2026/10/20 10:56:09 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping glance — last check 1m30s ago, effective interval 5m0s, healthy=true
2026/10/20 10:56:19 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping glance — last check 1m40s ago, effective interval 5m0s, healthy=true
2026/10/20 10:56:29 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping glance — last check 1m50s ago, effective interval 5m0s, healthy=true
2026/10/20 10:56:39 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping glance — last check 2m0s ago, effective interval 5m0s, healthy=true
2026/10/20 10:56:49 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping glance — last check 2m10s ago, effective interval 5m0s, healthy=true
exit 0
@@ -0,0 +1,24 @@
200
{
200
{
"ok": true,
Exception in thread Thread-1 (pump):
Traceback (most recent call last):
File "/usr/lib/python3.13/threading.py", line 1043, in _bootstrap_inner
self.run()
~~~~~~~~^^
File "/usr/lib/python3.13/threading.py", line 994, in run
self._target(*self._args, **self._kwargs)
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/5_hdd/felhom.eu/git/cc-secrets/ccsec/cli.py", line 153, in pump
dst.write(m.close()); dst.flush()
~~~~~~~~~^^
BrokenPipeError: [Errno 32] Broken pipe
Exception ignored on flushing sys.stdout:
BrokenPipeError: [Errno 32] Broken pipe
removed '/mnt/felhom-drives/scratch_hdd/exports/glance_20261020-104935.fab'
0
0
0
stranger GET after teardown: 404
@@ -0,0 +1,23 @@
# C3 — proven live: a `.fab` export → import makes a family-only app public
Scratch guest 9202, controller 0.331.0 (the `.fab` import code — `cmd/controller/main.go:3517-3530`
`SaveEncryptedAppConfig`, `stacks/family_gate.go:173-190` — read the same at 0.336.0, `7ca59e2`). Architecture: `09` §3
decisions 63/64 (the family gate), 221 (Glance only behind it), `07` §6.6 („never open by default").
| UTC | act (the dashboard's own endpoints) | observed (a stranger's GET through traefik; the gate file on disk) |
|---|---|---|
| 10:48:52 | `POST /api/stacks/glance/deploy` | running, `app_config.family_gate` set |
| 10:49:34 | **control** | gate file `traefik/dynamic/family-gate-glance.yml` present; stranger → **401** |
| 10:49:35 | `POST /api/export/start` (stop ticked, no password) | `glance_20261020-104935.fab`, 5.6 KB |
| 10:50:09 | stop + `POST /api/stacks/glance/remove` (data + backups) | removed; 10:50:29 „removed the family-gate file of an app that is not installed" |
| 10:54:26 | `POST /api/export/import` (the household's „Importálás") | import done in 6 s; app running |
| 10:56:13 | the app's record | `app_config` keys: deployed, deployed_at, desired_state, env, installed_images, locked_fields — **no family_gate** |
| 10:56:55 | **fault** | gate file **none**; stranger → **200** (Glance answers anyone who reaches the address) |
The first run's bundle-list parse was a harness slip (`01-run.log`, step 4 sent an empty path → 400, nothing done);
step 4 re-ran alone with the path the export's own status gave (`02-import.log`, `step4.py`). Teardown
(`03-teardown.txt`): Glance removed through the product, the bundle deleted (own file), no container, volume or gate
file left; the address answers 404.
On a household box the address is reachable from the internet through the tunnel. Family-gated apps in the catalog:
Glance, Grimmory, MeTube (the last two hold the family's books and videos).
@@ -0,0 +1,96 @@
#!/usr/bin/env python3
"""fab_gate_proof.py — C3 for R-1187 (e) on scratch 9202: does a `.fab` export → remove → import drop the family gate?
Glance has no login of its own; the catalog puts it ONLY behind the family gate (decision 221). Observables, each
read from a DIFFERENT channel than the act: (1) the gate file `family-gate-glance.yml` on the guest's disk; (2) what a
stranger's request to the app's host gets back from traefik (302 to the family page vs 200 from Glance).
Acts are the dashboard's own endpoints: deploy, /api/export/start, stop+remove, /api/export/import.
Env: CTL_BASE, CTL_DOMAIN, CTL_PW_ENV (tools/ctl.py).
"""
import json, os, subprocess, sys, time
sys.path.insert(0, "/mnt/5_hdd/felhom.eu/git/felhom.eu/documentation/audits/rehearsal-2026-10-20/tools")
import ctl # noqa: E402
APP, SUB = "glance", "r1187glance"
DOM = os.environ["CTL_DOMAIN"]
def say(*a):
print(time.strftime("%H:%M:%SZ", time.gmtime()), *a, flush=True)
def guest(cmd):
r = subprocess.run(["ssh", "demo-hp", f"export LC_ALL=C; pct exec 9202 -- bash -c {json.dumps(cmd)}"],
capture_output=True, text=True, timeout=300)
return (r.stdout + r.stderr).strip()
def observe(tag):
say(f"[{tag}] gate file:", guest("find / -xdev -name 'family-gate-glance.yml' 2>/dev/null | head -3; true") or "(none)")
out = subprocess.run(["curl", "-sk", "-o", "/dev/null", "-w", "%{http_code} %{redirect_url}",
"-H", f"Host: {SUB}.{DOM}", ctl.BASE + "/"], capture_output=True, text=True).stdout
say(f"[{tag}] a stranger's GET https://{SUB}.{DOM}/ ->", out)
def wait_stack(pred, n=60):
for _ in range(n):
code, d = ctl.call("GET", f"/api/stacks/{APP}")
st = (d.get("data") or {}) if isinstance(d, dict) else {}
if pred(st):
return st
time.sleep(5)
return st
def poll(path, n=120):
last = None
for _ in range(n):
code, d = ctl.call("GET", path)
dd = (d.get("data") or d) if isinstance(d, dict) else {}
s = json.dumps(dd, ensure_ascii=False)[:400]
if s != last:
say(" ", path, s)
last = s
if isinstance(dd, dict) and dd.get("status") in ("completed", "failed", "done", "error") or (
isinstance(dd, dict) and dd.get("running") is False and dd.get("status") not in ("running", "pending")):
return dd
time.sleep(3)
return dd
say("== 1 deploy glance")
code, d = ctl.call("POST", f"/api/stacks/{APP}/deploy", {"values": {"DOMAIN": DOM, "SUBDOMAIN": SUB}})
say("deploy ->", code, str(d)[:150])
st = wait_stack(lambda s: s.get("deployed") and s.get("state") in ("running", "unhealthy", "degraded"))
say("state:", st.get("state"), "family_gate in app_config:", bool((st.get("app_config") or {}).get("family_gate")))
time.sleep(20)
observe("1 POSITIVE CONTROL after install")
say("== 2 export to the data drive")
drive = "/mnt/felhom-drives/scratch_hdd"
code, d = ctl.call("POST", "/api/export/start", {"stack_name": APP, "dest_drive": drive, "password": "", "stop_app": True})
say("export start ->", code, str(d)[:200])
poll("/api/export/status")
code, d = ctl.call("GET", "/api/export/bundles")
bundles = [b for b in ((d.get("data") or {}).get("bundles") or d.get("data") or []) if isinstance(b, dict) and APP in json.dumps(b)]
say("bundles for glance:", json.dumps(bundles, ensure_ascii=False)[:500])
path = (bundles[-1].get("path") if bundles else None)
say("== 3 remove glance (keep nothing)")
ctl.call("POST", f"/api/stacks/{APP}/stop")
time.sleep(5)
code, d = ctl.call("POST", f"/api/stacks/{APP}/remove", {"remove_hdd_data": True, "remove_backups": True})
say("remove ->", code, str(d)[:160])
time.sleep(10)
observe("3 after remove")
say("== 4 import the bundle")
code, d = ctl.call("POST", "/api/export/import", {"path": path, "password": ""})
say("import start ->", code, str(d)[:200])
poll("/api/export/import/status")
st = wait_stack(lambda s: s.get("deployed") and s.get("state") in ("running", "unhealthy", "degraded"), n=40)
say("state after import:", st.get("state"), "family_gate in app_config:", bool((st.get("app_config") or {}).get("family_gate")),
"app_config keys:", sorted((st.get("app_config") or {}).keys()))
time.sleep(30)
observe("4 FAULT after import")
say("controller log:", guest("docker logs --since 15m felhom-controller 2>&1 | grep -iE 'family|import' | grep -v 'GET /api' | tail -15"))
@@ -0,0 +1,17 @@
#!/usr/bin/env python3
"""Step 4 of fab_gate_proof.py re-run alone: the first run's bundle-list parse found nothing (harness slip); the
export's own status gave the path."""
import sys
sys.argv = ["x"]
exec(open("fab_gate_proof.py").read().split('say("== 1 deploy glance")')[0])
path = "/mnt/felhom-drives/scratch_hdd/exports/glance_20261020-104935.fab"
say("== 4 import the bundle", path)
code, d = ctl.call("POST", "/api/export/import", {"path": path, "password": ""})
say("import start ->", code, str(d)[:200])
poll("/api/export/import/status")
st = wait_stack(lambda s: s.get("deployed") and s.get("state") in ("running", "unhealthy", "degraded"), n=40)
say("state after import:", st.get("state"), "family_gate in app_config:", bool((st.get("app_config") or {}).get("family_gate")),
"app_config keys:", sorted((st.get("app_config") or {}).keys()))
time.sleep(40)
observe("4 FAULT after import")
say("controller log:", guest("docker logs --since 6m felhom-controller 2>&1 | grep -iE 'family|import|glance' | grep -vE 'GET /api|ScanStacks|Env vars' | tail -15"))
@@ -0,0 +1,7 @@
# go test at controller 7ca59e2 (v0.336.0), scratch clone; 2026-10-20T11:11:31Z
=== RUN TestRehearsalR12_LeftoverTempBlocksEveryRetry
zz_rehearsal_r12_test.go:21: retry 1: err=fájlmásolás sikertelen: restoring appdata/app/p1.jpg: open /tmp/TestRehearsalR12_LeftoverTempBlocksEveryRetry3138873193/001/usb/appdata/app/p1.jpg.felhom-restoring: file exists; p1.jpg back=false; app restarted=true
zz_rehearsal_r12_test.go:21: retry 2: err=fájlmásolás sikertelen: restoring appdata/app/p1.jpg: open /tmp/TestRehearsalR12_LeftoverTempBlocksEveryRetry3138873193/001/usb/appdata/app/p1.jpg.felhom-restoring: file exists; p1.jpg back=false; app restarted=true
--- PASS: TestRehearsalR12_LeftoverTempBlocksEveryRetry (0.01s)
PASS
ok gitea.dooplex.hu/admin/felhom-controller/internal/backup 0.020s
@@ -0,0 +1,23 @@
package backup
// THROWAWAY proof for the 2026-10-20 data-loss hunt (R-1188 c). Never committed. A crash during the whole restore's
// file copy leaves `<file>.felhom-restoring` behind (the copy writes it with O_EXCL); the retry then meets it.
import (
"os"
"path/filepath"
"testing"
)
func TestRehearsalR12_LeftoverTempBlocksEveryRetry(t *testing.T) {
f, _, liveLeg := wholeFixture(t)
// what a power cut in the middle of copying p1.jpg leaves on the live drive
if err := os.MkdirAll(liveLeg, 0o755); err != nil {
t.Fatal(err)
}
os.WriteFile(filepath.Join(liveLeg, "p1.jpg.felhom-restoring"), []byte("half"), 0o644)
for i := 1; i <= 2; i++ {
_, err := f.m.RestoreTier2Whole("app")
_, statErr := os.Stat(filepath.Join(liveLeg, "p1.jpg"))
t.Logf("retry %d: err=%v; p1.jpg back=%v; app restarted=%v", i, err, statErr == nil, f.fake.fullStarted)
}
}
@@ -0,0 +1,10 @@
# go test at controller 7ca59e2 (v0.336.0), scratch clone, fake provider seams; 2026-10-20T10:13:05Z
=== RUN TestRehearsalR5_WholeRestoreUnitRefusalAfterFileHalf
zz_rehearsal_r5_test.go:22: live files BEFORE: map[p2.jpg:P2 live older]
zz_rehearsal_r5_test.go:25: RestoreTier2Whole err = A(z) app adatbázis-másolata csonka: nem ér véget, ahogy egy teljes másolat. A visszaállítás biztonsági okból nem indult el — az alkalmazás és az adatai érintetlenek.
zz_rehearsal_r5_test.go:26: file counts: {Restored:1 Replaced:1 KeptNewer:0 Unchanged:0 BytesTotal:17}
zz_rehearsal_r5_test.go:27: live files AFTER: map[p1.jpg:P1 p2.jpg:P2 mirror NEWER p2.jpg.felhom-20261020T101306Z:P2 live older]
zz_rehearsal_r5_test.go:28: app stopped=true, full start happened=false, provider calls=[stop]
--- PASS: TestRehearsalR5_WholeRestoreUnitRefusalAfterFileHalf (0.01s)
PASS
ok gitea.dooplex.hu/admin/felhom-controller/internal/backup 0.019s
@@ -0,0 +1,29 @@
package backup
// THROWAWAY proof for the 2026-10-20 data-loss hunt (candidate R5). Never committed.
import (
"path/filepath"
"strings"
"testing"
"time"
)
func TestRehearsalR5_WholeRestoreUnitRefusalAfterFileHalf(t *testing.T) {
cut := strings.Replace(pgDump(1), "PostgreSQL database dump complete", "", 1) // a cut-off copy (R-640)
f := r102Tier2Fixture(t, []string{"app_data.tar"}, cut)
f.m.stackProvider = fileAppProvider{f.fake}
mid := time.Date(2026, 9, 22, 10, 0, 0, 0, time.UTC)
mirrorLeg := filepath.Join(f.destBase, "hdd", "appdata", "app")
liveLeg := filepath.Join(f.liveDrive, "appdata", "app")
writeAt(t, filepath.Join(mirrorLeg, "p1.jpg"), "P1", mid)
writeAt(t, filepath.Join(mirrorLeg, "p2.jpg"), "P2 mirror NEWER", mid.Add(72*time.Hour))
writeAt(t, filepath.Join(liveLeg, "p2.jpg"), "P2 live older", mid.Add(48*time.Hour))
before := fingerprint(t, liveLeg)
t.Logf("live files BEFORE: %v", before)
res, err := f.m.RestoreTier2Whole("app")
t.Logf("RestoreTier2Whole err = %v", err)
t.Logf("file counts: %+v", res.Files)
t.Logf("live files AFTER: %v", fingerprint(t, liveLeg))
t.Logf("app stopped=%v, full start happened=%v, provider calls=%v", f.fake.stopped, f.fake.fullStarted, f.fake.calls)
}
@@ -0,0 +1,8 @@
# go test at controller 7ca59e2 (v0.336.0), scratch clone; the PRODUCTION rsyncMirror (rsync -a --delete) on local temp dirs; 2026-10-20T11:02:49Z
=== RUN TestRehearsalT1_NightlyMirrorRunsDuringARestore
zz_rehearsal_t1_test.go:26: POSITIVE CONTROL: mirror after the night = [a.jpg b.jpg c.jpg d.jpg]
zz_rehearsal_t1_test.go:34: restore in progress: backup flag held = true
zz_rehearsal_t1_test.go:37: FAULT: mirror DURING the restore = [a.jpg b.jpg] (c.jpg, d.jpg — the files the restore is bringing back — are gone from the only copy)
--- PASS: TestRehearsalT1_NightlyMirrorRunsDuringARestore (0.22s)
PASS
ok gitea.dooplex.hu/admin/felhom-controller/internal/backup 0.230s
@@ -0,0 +1,39 @@
package backup
// THROWAWAY proof for the 2026-10-20 data-loss hunt (candidate T1 / R-1180). Never committed.
import (
"os"
"path/filepath"
"testing"
)
type t1Provider struct{ *t2v2Provider }
func (p t1Provider) ListDeployedStacks() []StackSummary { return []StackSummary{{Name: "app"}} }
func TestRehearsalT1_NightlyMirrorRunsDuringARestore(t *testing.T) {
m, src, target, prov := newTier2V2(t, "app")
prov.has["app"] = true
prov.binds["app"] = []ClassifiedBind{mHDD("appdata/app/photos")}
m.stackProvider = t1Provider{prov}
m.tier2Mirror = rsyncMirror // the PRODUCTION mirror (rsync -a --delete), local temp dirs only
for _, n := range []string{"a.jpg", "b.jpg", "c.jpg", "d.jpg"} {
mustWrite(t, filepath.Join(src, "appdata", "app", "photos", n), n)
}
m.RunAllTier2()
mir := filepath.Join(target, "backups", "secondary", "app", "hdd", "appdata", "app", "photos")
ls := func(d string) []string { e, _ := os.ReadDir(d); var o []string; for _, x := range e { o = append(o, x.Name()) }; return o }
t.Logf("POSITIVE CONTROL: mirror after the night = %v", ls(mir))
// The household deleted half the photos and starts the second drive's restore; the restore holds the backup flag.
os.Remove(filepath.Join(src, "appdata", "app", "photos", "c.jpg"))
os.Remove(filepath.Join(src, "appdata", "app", "photos", "d.jpg"))
if err := m.acquireRunning(); err != nil {
t.Fatal(err)
}
t.Logf("restore in progress: backup flag held = %v", m.acquireRunning() != nil)
// 03:30 — the nightly second-drive leg fires while the restore is still copying.
m.RunAllTier2()
t.Logf("FAULT: mirror DURING the restore = %v (c.jpg, d.jpg — the files the restore is bringing back — are gone from the only copy)", ls(mir))
m.releaseRunning()
}
@@ -0,0 +1,57 @@
// Replays hub cmd/hub/main.go scheduleDaily's arithmetic (felhom.eu 5c48731, lines 1057-1062) over
// 2026-10-23..27 with a simulated clock, for each hub daily job. Prints every fire time.
package main
import (
"fmt"
"time"
)
func main() {
bud, _ := time.LoadLocation("Europe/Budapest")
jobs := []struct{ name string; h, m int }{{"db-snapshot", 2, 0}, {"prune", 4, 30}, {"deadline-check", 5, 0}, {"offsite-key-audit", 7, 10}}
for _, j := range jobs {
now := time.Date(2026, 10, 23, 12, 0, 0, 0, bud)
end := time.Date(2026, 10, 27, 12, 0, 0, 0, bud)
fmt.Printf("%s (%02d:%02d):\n", j.name, j.h, j.m)
for now.Before(end) {
n := now.In(bud)
next := time.Date(n.Year(), n.Month(), n.Day(), j.h, j.m, 0, 0, bud)
if !next.After(n) {
next = next.Add(24 * time.Hour)
}
fmt.Printf(" fires %s (%s)\n", next.In(bud).Format("2006-01-02 15:04 MST"), next.UTC().Format("15:04Z"))
now = next // fn() runs instantly; loop re-reads the clock at the fire instant
}
}
}
--- go run, go version go1.26.0 linux/amd64
db-snapshot (02:00):
fires 2026-10-24 02:00 CEST (00:00Z)
fires 2026-10-25 02:00 CEST (00:00Z)
fires 2026-10-25 02:00 CET (01:00Z)
fires 2026-10-26 02:00 CET (01:00Z)
fires 2026-10-27 02:00 CET (01:00Z)
fires 2026-10-28 02:00 CET (01:00Z)
prune (04:30):
fires 2026-10-24 04:30 CEST (02:30Z)
fires 2026-10-25 03:30 CET (02:30Z)
fires 2026-10-25 04:30 CET (03:30Z)
fires 2026-10-26 04:30 CET (03:30Z)
fires 2026-10-27 04:30 CET (03:30Z)
fires 2026-10-28 04:30 CET (03:30Z)
deadline-check (05:00):
fires 2026-10-24 05:00 CEST (03:00Z)
fires 2026-10-25 04:00 CET (03:00Z)
fires 2026-10-25 05:00 CET (04:00Z)
fires 2026-10-26 05:00 CET (04:00Z)
fires 2026-10-27 05:00 CET (04:00Z)
fires 2026-10-28 05:00 CET (04:00Z)
offsite-key-audit (07:10):
fires 2026-10-24 07:10 CEST (05:10Z)
fires 2026-10-25 06:10 CET (05:10Z)
fires 2026-10-25 07:10 CET (06:10Z)
fires 2026-10-26 07:10 CET (06:10Z)
fires 2026-10-27 07:10 CET (06:10Z)
fires 2026-10-28 07:10 CET (06:10Z)
@@ -0,0 +1,63 @@
# R-1019 — demo-felhom's second-drive run 2026-10-21 01:30 UTC (03:30 CEST), read 02:36 UTC by the rehearsal session, read-only
# channel 1: the controller's own log (docker logs felhom-controller, 01:25–01:45Z)
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LC_CTYPE = "UTF-8",
LC_NUMERIC = (unset),
LC_COLLATE = (unset),
LC_TIME = (unset),
LC_MESSAGES = (unset),
LC_MONETARY = (unset),
LC_ADDRESS = (unset),
LC_IDENTIFICATION = (unset),
LC_MEASUREMENT = (unset),
LC_PAPER = (unset),
LC_TELEPHONE = (unset),
LC_NAME = (unset),
LANG = "en_US.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to a fallback locale ("en_US.UTF-8").
2026/10/21 01:29:59 [INFO] [stacks] Status refresh: 8 containers across 68 stacks
2026/10/21 01:30:00 [INFO] [scheduler] Running job: tier2-backup
2026/10/21 01:30:00 [INFO] [scheduler] Running job: fill-watch
2026/10/21 01:30:00 [INFO] [fillwatch] checked 3 filesystem(s), 0 unreadable/skipped, 0 notification(s); bands: all ok
2026/10/21 01:30:00 [INFO] [scheduler] Job fill-watch completed (took 0s)
2026/10/21 01:30:00 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/opengist/docker-compose.yml
2026/10/21 01:30:00 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/opengist/docker-compose.yml
2026/10/21 01:30:01 [INFO] [settings] Settings saved
2026/10/21 01:30:01 [INFO] [backup] Tier 2 copied opengist → /mnt/felhom-drives/hdd_1/backups/secondary/opengist (190.8 KB, 0 leg(s), 0s)
2026/10/21 01:30:01 [INFO] Event pushed: crossdrive_completed (info) [hu-only] — Másodlagos mentés elkészült: opengist
2026/10/21 01:30:03 [INFO] [settings] Settings saved
2026/10/21 01:30:03 [INFO] [backup] Tier 2 copied prowlarr → /mnt/felhom-drives/hdd_1/backups/secondary/prowlarr (5.0 MB, 0 leg(s), 0s)
2026/10/21 01:30:03 [INFO] Event pushed: crossdrive_completed (info) [hu-only] — Másodlagos mentés elkészült: prowlarr
2026/10/21 01:30:05 [INFO] [settings] Settings saved
2026/10/21 01:30:05 [INFO] [backup] Tier 2 copied radarr → /mnt/sys_drive/felhom-data/backups/secondary/radarr (4.1 MB, 0 leg(s), 0s) [SSD: state-only]
2026/10/21 01:30:05 [INFO] Event pushed: crossdrive_completed (info) [hu-only] — Másodlagos mentés elkészült: radarr
2026/10/21 01:30:07 [INFO] [settings] Settings saved
2026/10/21 01:30:07 [INFO] [backup] Tier 2 copied sonarr → /mnt/sys_drive/felhom-data/backups/secondary/sonarr (8.8 MB, 0 leg(s), 0s) [SSD: state-only]
2026/10/21 01:30:07 [INFO] Event pushed: crossdrive_completed (info) [hu-only] — Másodlagos mentés elkészült: sonarr
2026/10/21 01:30:08 [INFO] [settings] Settings saved
2026/10/21 01:30:08 [INFO] [backup] Tier 2 copied qbittorrent → /mnt/sys_drive/felhom-data/backups/secondary/qbittorrent (8.3 MB, 0 leg(s), 0s) [SSD: state-only]
2026/10/21 01:30:08 [INFO] [backup] Tier 2 run complete: 5 app(s) processed (incl. volume-only — F6)
2026/10/21 01:30:08 [INFO] [scheduler] Job tier2-backup completed (took 8.954s)
2026/10/21 01:30:08 [INFO] Event pushed: crossdrive_completed (info) [hu-only] — Másodlagos mentés elkészült: qbittorrent
2026/10/21 01:30:09 [INFO] [stacks] Status refresh: 8 containers across 68 stacks
2026/10/21 01:30:09 [INFO] [scheduler] Running job: agent-channel-health
# channel 2: the second-drive folder itself (find + du, and a search for *torrent* under every secondary root)
/mnt/sys_drive/felhom-data/backups/secondary/qbittorrent
/mnt/sys_drive/felhom-data/backups/secondary/qbittorrent/recovery-unit
/mnt/sys_drive/felhom-data/backups/secondary/qbittorrent/recovery-unit/compose
/mnt/sys_drive/felhom-data/backups/secondary/qbittorrent/recovery-unit/compose/app.yaml
/mnt/sys_drive/felhom-data/backups/secondary/qbittorrent/recovery-unit/compose/docker-compose.yml
/mnt/sys_drive/felhom-data/backups/secondary/qbittorrent/recovery-unit/compose/.felhom.yml
/mnt/sys_drive/felhom-data/backups/secondary/qbittorrent/recovery-unit/volume-dumps
/mnt/sys_drive/felhom-data/backups/secondary/qbittorrent/recovery-unit/volume-dumps/qbittorrent_qbittorrent_config.tar
/mnt/sys_drive/felhom-data/backups/secondary/qbittorrent/recovery-unit/manifest.json
/mnt/sys_drive/felhom-data/backups/secondary/qbittorrent/recovery-unit/data-stamps.json
/mnt/sys_drive/felhom-data/backups/secondary/qbittorrent/.felhom-tier2-layout
8.3M /mnt/sys_drive/felhom-data/backups/secondary/qbittorrent
search:
/mnt/sys_drive/felhom-data/backups/secondary/qbittorrent
/mnt/sys_drive/felhom-data/backups/secondary/qbittorrent/recovery-unit/volume-dumps/qbittorrent_qbittorrent_config.tar
@@ -0,0 +1,19 @@
# R-895 — the hub's 07:10 CEST off-site snapshot check, 2026-10-21 (the first verdict day after the baseline). Read 05:27 UTC from the running hub pod's log (sudo kubectl logs, pod/hub-8c44cf5cb-srth9), read-only.
2026/10/21 07:10:00 [INFO] offsitekeys: audit Tester-2: 1 line(s), all pinned append-only
2026/10/21 07:10:01 [WARN] offsitekeys: snapshots Tester-2: the hub's own listing failed: offsitekeys: list snapshots: "ls -l /home/felhom-repo/snapshots": Process exited with status 2: /usr/bin/ls: cannot access '/home/felhom-repo/snapshots': No such file or directory
2026/10/21 07:10:01 [INFO] offsitekeys: audit Tester4: 1 line(s), all pinned append-only
2026/10/21 07:10:02 [INFO] offsitekeys: snapshots Tester4: the hub's own listing — 44 snapshot file(s), 0 other line(s), newest 2026-10-21T02:18:00Z, oldest 2026-10-20T16:33:00Z; box's last reported run 2026-10-21T02:18:11Z (R-895)
2026/10/21 07:10:02 [INFO] offsitekeys: snapshots Tester4: first listing stored as the baseline (44 file(s), 0 too new to judge) — no verdict
2026/10/21 07:10:03 [INFO] offsitekeys: audit demo-felhom: 1 line(s), all pinned append-only
2026/10/21 07:10:05 [INFO] offsitekeys: snapshots demo-felhom: the hub's own listing — 38 snapshot file(s), 0 other line(s), newest 2026-10-21T02:15:00Z, oldest 2026-08-12T15:15:00Z; box's last reported run 2026-10-21T02:15:35Z (R-895)
2026/10/21 07:10:05 [INFO] offsitekeys: snapshots demo-felhom: 5 new snapshot file(s), each inside a reported box run
2026/10/21 07:10:06 [INFO] offsitekeys: audit demo-hp: 1 line(s), all pinned append-only
2026/10/21 07:10:07 [INFO] offsitekeys: snapshots demo-hp: the hub's own listing — 289 snapshot file(s), 0 other line(s), newest 2026-10-21T02:18:00Z, oldest 2026-08-31T21:29:00Z; box's last reported run 2026-10-21T02:18:21Z (R-895)
2026/10/21 07:10:07 [INFO] offsitekeys: snapshots demo-hp: 14 new snapshot file(s), each inside a reported box run
2026/10/21 07:10:08 [INFO] offsitekeys: audit tester-1: 1 line(s), all pinned append-only
2026/10/21 07:10:09 [INFO] offsitekeys: snapshots tester-1: the hub's own listing — 58 snapshot file(s), 0 other line(s), newest 2026-10-21T02:15:00Z, oldest 2026-10-06T02:15:00Z; box's last reported run 2026-10-21T02:16:05Z (R-895)
2026/10/21 07:10:09 [INFO] offsitekeys: snapshots tester-1: 4 new snapshot file(s), each inside a reported box run
2026/10/21 07:10:09 [INFO] off-site key check: 5 sub-account(s) read in 9.268s
2026/10/21 07:10:09 [INFO] offsite-key-audit: next run at 2026-10-22 07:10 CEST (in 23h59m51s)
2026/10/21 07:10:43 [INFO] osupdates: guest candidate 5d6cfda60716db81 (272 packages, first seen 2026-10-21T03:19:43Z): waiting — healthy for 1h51m0s of 24h0m0s
2026/10/21 07:10:43 [INFO] osupdates: host candidate 8989bb243244a4f4 (607 packages, first seen 2026-10-21T03:19:43Z): waiting — healthy for 1h51m0s of 24h0m0s
@@ -0,0 +1,6 @@
GET 200
POST 200
answer: Felhom — Doboz összekötése Felhom doboz összekötése Kösd össze a most telepített Felhom dobozodat a fiókoddal. Add meg a doboz képernyőjén látható párosító kódot és a tulajdonosi jelmondatodat. A megadott adatok nem megfelelőek. Ellenőrizd a párosító kódot és a tulajdonosi jelmondatot, majd próbáld újra. Párosító kód A doboz monitorán jelenik meg, a telepítés után. Tulajdonosi jelmondat Az öt szób
GET 200
POST 200
answer: Felhom — Doboz összekötése Felhom doboz összekötése Sikeres összekötés. A doboz kb. egy percen belül folytatja a telepítést. Ezt az oldalt bezárhatod — a beállítás a háttérben befejeződik, és a vezérlőpultod hamarosan elérhető lesz. Felhom.eu
@@ -0,0 +1,3 @@
GET /claim HTTP/2 200
csrf_len 64 cookie_present yes
POST /claim HTTP/2 302 location: /
@@ -0,0 +1,4 @@
=== /backups/remote (21160 bytes)
Biztonsági mentés — Távoli mentés — Felhom.eu Indítópult Vezérlőpult Alkalmazások Tárhely Meghajtók Hálózati tárhely Biztonsági mentés Áttekintés Távoli mentés Alkalmazások Visszaállítás Megosztás Hálózati megosztás Rendszermonitor Hibakeresés Beállítások Rendszer Értesítések Biztonság és hozzáférés 0.336.0 Magyar English Kijelentkezés ↗ A korábbi távoli mentéseid megvannak, de ehhez a géphez a helyreállítási kódod szükséges. Megnézem Most nem Ne emlékeztessen újra Biztonsági mentés — Távoli mentés tester4.test A korábbi, házon kívüli mentéseid visszaszerezhetők. Ezt a gépet újratelepítették, és a Felhom központi rendszere őriz hozzá egy lezárt csomagot. A helyreállítási kódoddal feloldhatod a korábbi mentéseidet, és megnézheted, mi van bennük. Adatok visszaszerzése Felhom offsite tárhely kiépítve — a beállítás automatikus, folyamatban. Ha egy napon belül nem áll be, jelezd az üzemeltetőnek. Távoli mentés (3. mentés) — titkosított, offsite Az alkalmazás-mentések titkosított másolata egy távoli tárolóra — saját NAS vagy Felhom offsite tárhely — restic + SFTP kapcsolaton. A tároló csak titkosított adatot lát. Ez a 3-2-1 szabály „1 off-site" lába — független a helyi másodpéldánytól és a teljes rendszermentéstől. – Utolsó távoli mentés – Tároló méret · a pillanatképek száma még ismeretlen u629488-sub5@u629488-sub5.your-storagebox.de /home/felhom-repo Távoli mentési cél beállítása Cél címe (IP vagy hosztnév — NAS vagy SFTP-kiszolgáló) SSH port Felhasználó Tároló útvonala a célgépen Napi automatikus távoli mentés engedélyezve SSH privát kulcs A kulcsot 0600-as fájlba írjuk; sosem naplózzuk és nem tároljuk a beállításokban. A célgép ismert-host sora (known_hosts) A host-kulcs rögzítése (no blind TOFU). Lekérdezhető: ssh-keyscan -p <port> <host> Mentés
=== / (4642 bytes)
Adatok visszaszerzése — Felhom Adatok visszaszerzése Tester4 Ezt a gépet újratelepítették. A korábbi, házon kívüli mentéseid megvannak — a Felhom központi rendszere őriz hozzájuk egy lezárt csomagot, amelyet 2026. 10. 20. 14:16 zártunk le. A csomagot csak a te helyreállítási kódoddal lehet kinyitni. A helyreállítási kódot senki nem tudja pótolni — sem a Felhom, sem az ügyfélszolgálat, sem az üzemeltető. Ez szándékos: így a mentéseidet rajtad kívül senki nem tudja megnyitni. Ha a kód elveszett, a korábbi mentések nem nyithatók meg többé. Ha megadod a kódot, feloldjuk a mentéseid zárolását és megmutatjuk, mi van bennük — melyik alkalmazás, mikorról, mekkora. Ebben a lépésben semmit nem állítunk vissza és semmi nem változik. A visszaállítást utána, alkalmazásonként külön választhatod. Helyreállítási kód (tíz szó) Mentések feloldása Most nem A „Most nem” csak azt jelenti, hogy nem zavarunk vele többet a kezdőlapon. A mentéseid ettől megmaradnak, és ez az oldal a Biztonsági mentés → Távoli mentés oldalról bármikor újra elérhető. Ha a helyreállítási kódod véglegesen elveszett, és tudomásul veszed, hogy a korábbi mentések így nem nyithatók meg többé: ez a lehetőség akkor válik elérhetővé, ha a gép már újra kapcsolódott a házon kívüli tárhelyhez. Addig a mentéseid érintetlenül megmaradnak. Magyar English
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,2 @@
=== /recovery (4642 bytes)
Adatok visszaszerzése — Felhom Adatok visszaszerzése Tester4 Ezt a gépet újratelepítették. A korábbi, házon kívüli mentéseid megvannak — a Felhom központi rendszere őriz hozzájuk egy lezárt csomagot, amelyet 2026. 10. 20. 14:16 zártunk le. A csomagot csak a te helyreállítási kódoddal lehet kinyitni. A helyreállítási kódot senki nem tudja pótolni — sem a Felhom, sem az ügyfélszolgálat, sem az üzemeltető. Ez szándékos: így a mentéseidet rajtad kívül senki nem tudja megnyitni. Ha a kód elveszett, a korábbi mentések nem nyithatók meg többé. Ha megadod a kódot, feloldjuk a mentéseid zárolását és megmutatjuk, mi van bennük — melyik alkalmazás, mikorról, mekkora. Ebben a lépésben semmit nem állítunk vissza és semmi nem változik. A visszaállítást utána, alkalmazásonként külön választhatod. Helyreállítási kód (tíz szó) Mentések feloldása Most nem A „Most nem” csak azt jelenti, hogy nem zavarunk vele többet a kezdőlapon. A mentéseid ettől megmaradnak, és ez az oldal a Biztonsági mentés → Távoli mentés oldalról bármikor újra elérhető. Ha a helyreállítási kódod véglegesen elveszett, és tudomásul veszed, hogy a korábbi mentések így nem nyithatók meg többé: ez a lehetőség akkor válik elérhetővé, ha a gép már újra kapcsolódott a házon kívüli tárhelyhez. Addig a mentéseid érintetlenül megmaradnak. Magyar English
@@ -0,0 +1,2 @@
status: ['HTTP/2 200 ']
page: Adatok visszaszerzése — Felhom Adatok visszaszerzése Tester4 A megadott helyreállítási kódot nem fogadtuk el. Ellenőrizd, hogy mind a tíz szót pontosan, szóközökkel elválasztva írtad be — a kis- és nagybetűk nem számítanak. Semmi nem változott, nyugodtan próbáld újra. Ezt a gépet újratelepítették. A korábbi, házon kívüli mentéseid megvannak — a Felhom központi rendszere őriz hozzájuk egy lezárt csomagot, amelyet 2026. 10. 20. 14:16 zártunk le. A csomagot csak a te helyreállítási kódoddal lehet kinyitni. A helyreállítási kódot senki nem tudja pótolni — sem a Felhom, sem az ügyfélszolgálat, sem az üzemeltető. Ez szándékos: így a mentéseidet rajtad kívül senki nem tudja megnyitni. Ha a kód elveszett, a korábbi mentések nem nyithatók meg többé. Ha megadod a kódot, feloldjuk a mentéseid zárolását és megmutatjuk, mi van bennük — melyik alkalmazás, mikorról, mekkora. Ebben a lépésben semmit nem állítunk vissza és semmi nem változik. A visszaállítást utána, alkalmazásonként külön választhatod. Helyreállítási kód (tíz szó) Mentések feloldása Most nem A „Most nem” csak azt jelenti, hogy nem zavarunk vele többet a kezdőlapon. A mentéseid ettől megmaradnak, és ez az oldal a Biztonsági mentés → Távoli mentés oldalról bármikor újra elérhető. Ha a helyreállítási kódod véglegesen elveszett, és tudomásul veszed, hogy a korábbi mentések így nem nyithatók meg többé: ez a lehetőség akkor válik elérhetővé, ha a gép már újra kapcsolódott a házon kívüli tárhelyhez. Addig a mentéseid érintetlenül me
@@ -0,0 +1,2 @@
status: ['HTTP/2 200 ']
page: Adatok visszaszerzése — Felhom A mentéseid elérhetők A mentéseid zárolása feloldva. Az alábbiakat találtuk a tárolóban — semmit nem állítottunk vissza. Ezek a te mentéseid, a lent jelzett időpontokból. Nézd át, hogy tényleg azt találod-e itt, amire számítasz — semmit nem állítottunk vissza és semmi nem változott. Alkalmazás Legutóbbi mentés Méret _shares 2026. 10. 28. 07:39 1,8 GB actualbudget 2026. 10. 28. 07:39 524,9 KB bookstack 2026. 10. 28. 07:39 154,1 MB immich 2026. 10. 28. 07:39 1,2 GB mealie 2026. 10. 28. 07:39 1,7 MB nextcloud 2026. 10. 28. 07:39 1,0 GB paperless-ngx 2026. 10. 28. 07:39 74,4 MB radicale 2026. 10. 28. 07:39 34,0 KB vaultwarden 2026. 10. 28. 07:39 391,3 KB A visszaállítás alkalmazásonként történik, a Biztonsági mentés → Visszaállítás oldalon. Ott választhatod ki, melyik alkalmazás mit hozzon vissza. Tovább a visszaállításhoz Vissza a kezdőlapra Magyar English
@@ -0,0 +1,2 @@
=== /backups/restore (25021 bytes)
Biztonsági mentés — Visszaállítás — Felhom.eu Indítópult Vezérlőpult Alkalmazások Tárhely Meghajtók Hálózati tárhely Biztonsági mentés Áttekintés Távoli mentés Alkalmazások Visszaállítás Megosztás Hálózati megosztás Rendszermonitor Hibakeresés Beállítások Rendszer Értesítések Biztonság és hozzáférés 0.336.0 Magyar English Kijelentkezés ↗ Biztonsági mentés — Visszaállítás tester4.test Visszaállítás a távoli tárolóból Válaszd ki az alkalmazást, és a következő oldalon döntsd el, mit szeretnél: ellenőrzést külön mappába, csak a hiányzó fájlok visszahozását, vagy teljes visszaállítást. Egyik sem indul el kérdés nélkül. actualbudget Nincs telepítve — előbb telepítsd újra , utána hozhatod vissza az adatait. Visszaállítás… bookstack Nincs telepítve — előbb telepítsd újra , utána hozhatod vissza az adatait. Visszaállítás… immich Nincs telepítve — előbb telepítsd újra , utána hozhatod vissza az adatait. Visszaállítás… mealie Nincs telepítve — előbb telepítsd újra , utána hozhatod vissza az adatait. Visszaállítás… nextcloud Nincs telepítve — előbb telepítsd újra , utána hozhatod vissza az adatait. Visszaállítás… paperless-ngx Nincs telepítve — előbb telepítsd újra , utána hozhatod vissza az adatait. Visszaállítás… radicale Nincs telepítve — előbb telepítsd újra , utána hozhatod vissza az adatait. Visszaállítás… vaultwarden Nincs telepítve — előbb telepítsd újra , utána hozhatod vissza az adatait. Visszaállítás… Meglévő ellenőrző másolatok Nincs ellenőrző másolat a meghajtón. Hordozható mentéscsomag (.fab) Hordozható pillanatfelvétel — bárhol tárolhatod, és bármikor visszatöltheted egy meghajtóról. A folyamatos védelmet az 1–3. szintű mentés adja. Nincs telepített alkalmazás.
@@ -0,0 +1 @@
202 {"ok": true, "message": "Telepítés elindítva – az állapot a kártyán követhető"}
@@ -0,0 +1,4 @@
12:12:27Z ('deploying', None)
12:13:07Z ('degraded', None)
12:13:27Z ('starting', None)
12:14:27Z ('running', None)
@@ -0,0 +1,12 @@
12:15:09Z POST /backup/offbox/restore {'app': 'nextcloud', 'mode': 'full'} -> HTTP/2 302 ['location: /backups/restore/app?name=nextcloud&full_prep=nextcloud&full_size=1.0+GB']
12:15:09Z POST /backup/offbox/restore {'app': 'nextcloud', 'mode': 'full', 'confirm': '1'} -> HTTP/2 302 ['location: /backups/restore/app?name=nextcloud&flash=flash.offbox.restore_started']
12:15:09Z status: {"running": true, "op": "offbox-restore", "stack": "nextcloud", "started_at": "2026-10-21T12:15:09.946691559Z"}
12:15:50Z status: {"running": false, "op": "offbox-restore", "stack": "nextcloud", "started_at": "2026-10-21T12:15:09.946691559Z", "last": {"op": "offbox-restore", "stack": "nextcloud", "ok": true, "message": "A(z) nextcloud teljes mentése visszaállítva ellenőrző mappába: /mnt/felhom-drives/belso/backups/offsite-restore/nextcloud — a saját fájljaiddal együtt. A meglévő adatok változatlanok.", "finished_at": "2026-10-21T12:15:47.152729216Z"}, "last_recent": true}
12:15:50Z POST /backup/offbox/reconstitute {'app': 'nextcloud', 'confirm': '1'} -> HTTP/2 302 ['location: /backups/restore/app?name=nextcloud&flash=flash.offbox.full_restore_started']
12:15:50Z status: {"running": true, "op": "offbox-reconstitute", "stack": "nextcloud", "started_at": "2026-10-21T12:15:50.091899858Z", "last": {"op": "offbox-restore", "stack": "nextcloud", "ok": true, "message": "A(z) nextcloud teljes mentése visszaállítva ellenőrző mappába: /mnt/felhom-drives/belso/backups/offsite-restore/nextcloud — a saját fájljaiddal együtt. A meglévő adatok változatlanok.", "finished_at": "2026-10-21T12:15:47.152729216Z"}, "last_recent": true}
12:16:00Z status: {"running": false, "op": "offbox-reconstitute", "stack": "nextcloud", "started_at": "2026-10-21T12:15:50.091899858Z", "last": {"op": "offbox-reconstitute", "stack": "nextcloud", "ok": false, "message": "A teljes visszaállítás sikertelen: A(z) nextcloud mentése szerint az adatok korábban itt voltak: /mnt/felhom-drives/hdd_1. Most viszont ide állna vissza: /mnt/felhom-drives/belso. Ez lehet szándékos — például ha meghajtót cseréltél —, de magától nem folytatjuk. Ha így jó, erősítsd meg alább, és a visszaállítás az új helyre fut.", "finished_at": "2026-10-21T12:15:54.494276193Z"}, "last_recent":
12:16:00Z app state: running
12:18:57Z POST /backup/offbox/reconstitute {'app': 'nextcloud', 'confirm': '1', 'ack_placement': '1'} -> HTTP/2 302 ['location: /backups/restore/app?name=nextcloud&flash=flash.offbox.full_restore_started']
12:18:57Z status: {"running": true, "op": "offbox-reconstitute", "stack": "nextcloud", "started_at": "2026-10-21T12:18:57.09132257Z", "last": {"op": "offbox-reconstitute", "stack": "nextcloud", "ok": false, "message": "A teljes visszaállítás sikertelen: A(z) nextcloud mentése szerint az adatok korábban itt voltak: /mnt/felhom-drives/hdd_1. Most viszont ide állna vissza: /mnt/felhom-drives/belso. Ez lehet szándékos — például ha meghajtót cseréltél —, de magától nem folytatjuk. Ha így jó, erősítsd meg alább, és a visszaállítás az új helyre fut.", "finished_at": "2026-10-21T12:15:54.494276193Z"}, "last_recent": tr
12:20:07Z status: {"running": false, "op": "offbox-reconstitute", "stack": "nextcloud", "started_at": "2026-10-21T12:18:57.09132257Z", "last": {"op": "offbox-reconstitute", "stack": "nextcloud", "ok": true, "message": "A(z) nextcloud: 3113 fájl és 2 adatkötet és az adatbázis visszaállítva (mentés: 2026. 10. 28. 07:37) — az alkalmazás újraindult.", "finished_at": "2026-10-21T12:20:03.882952809Z"}, "last_recent": true}
12:20:07Z app state: running
@@ -0,0 +1,7 @@
nextcloud Csalad: manifest 3033 files; on the box 3035; byte-identical 3031; missing/different 2; extra 4
MISSING /Csalad/nagy_fajl_1_5GB.bin
MISSING /Csalad/Kulonleges_nevek/link_a_nagy_fajlra
EXTRA /Csalad/feltoltes_1.jpg
EXTRA /Csalad/feltoltes_2.jpg
EXTRA /Csalad/feltoltes_3.jpg
EXTRA /Csalad/feltoltes_4.jpg
@@ -0,0 +1,2 @@
status 200 {"last_duration": "", "last_error": "", "last_run": "", "orphaned": false, "progress": {"active": false, "current_app": "", "percent": 0, "bytes_done": 0, "total_bytes": 0, "done_human": "", "total_human": "", "files_done": 0, "total_files": 0, "current_file": "", "elapsed_sec": 0, "phase": ""}, "repo_size_human": "", "snapshots": 0, "status": ""}
nextcloud snapshots: 1
@@ -0,0 +1,6 @@
crash_seen:Tester4-68d1d8 {'boots': 0, 'tripped_at': '', 'rearmed_at': '', 'oops_boot': ''}
crash_seen:Tester4-31fa0e {'boots': 0, 'tripped_at': '', 'rearmed_at': '', 'oops_boot': ''}
crash_seen:Tester4-816e21 {'boots': 5, 'tripped_at': '2026-10-20T17:34:12Z', 'rearmed_at': '', 'oops_boot': ''}
offsite_snapshot_listing:Tester4 {'listed_at': '2026-10-21T05:10:01.734603053Z', 'entries': {'0035e753344c8f2b7fea0e50081d7a88afb698357a0e8ac4f6b830f85d54e34b': '2026-10-20T19:59:00Z', '0297495c6dfcc5b468c95daa2b79cfacc2c1b78de7214e8e98c4f933f70805e0': '2026-10-20T20:00:00Z', '04f7483b259464e87fcb012fa7e020ab7a2014b3907dfbe960bced33ddee3a58': '2026-10-21T02:18:00Z', '0e4825a4db9343c378ca7adecd99dbf27d82649a135fabd728370fdc2d36e17d': '2026-10-21T02:17:00Z', '20e56eda6bb024f63cd03a7672b29ce80f6100c15dcac7b9c4e15b86ec10fbe5': '2026-10-20T17:24:00Z', '226db1616dd52db34d67d22a9c89c7260e177e417e6f8eeae1ff3d0192042eb9': '2026-10-21T02:17:00Z', '2454ff01c39750f4f3772d266822c6e421665c429b22f7326cde66a8f2a63dfc': '2026-10-20T17:23:00Z', '340361eb807a2721d062a88c9ed7f447997c0e3437615fbf6ecc4d202a482da8': '2026-10-20T17:01:00Z', '34457cb6649bc68a698e97abfaa9c98d7304b179c0b481a2309646da388508f0': '2026-10-20T17:23:00Z', '35c54872377f018f9e28aa4ddfd59c6b9584dc6a66c6f83307aedf3a8a38e67e': '2026-10-20T17:23:00Z', '45a6f196bb71aa168ce7242a0858f66422a0a064ac7003f0b9cae5bd6e634080': '2026-10-20T17:01:00Z', '4cfa40bbe8257f9f2c30f82547b9826e39e9a89ee064f00355591d8a01520eb1': '2026-10-20T17:06:00Z', '4e7265c1d07dd0829e5067b9858569127c4ab11743b0a8f848fe4cd5aec38c30': '2026-10-21T02:17:00Z', '5210812729c887baf4f82b8c6e1903cf979d1f489cba3edcd8c3f992751ed8b1': '2026-10-20T17:23:00Z', '63bd7b2776b77d90e4e0c3cce9ce9fe46a3efb7dc5677b572128e3cb515f2aab': '2026-10-20T17:01:00Z', '6baefdc488918a060a01ddfa34efaa6616af3bbac912e44197a2de6c2a8e9a6b': '2026-10-20T17:01:00Z', '6d851619e3d5e084a773d482ec7a3677cc5b29bdaed02c6d59f9eddc7dc2be28': '2026-10-20T17:07:00Z', '73ddca76a1f43d06e25991415500a512d12c8ee676820bd98f3fe9166d3a032c': '2026-10-20T17:06:00Z', '7d62cf64ceda05866dbc8e11aec64a827bf0e06295eed9b85f474ff06f55a4e7': '2026-10-20T17:23:00Z', '884b7302a0a0f611872b608b9b6e19a41c5c02ec916fa5d031195ed1e2609cdb': '2026-10-21T02:17:00Z', '89672cb92aa1bf747de12605fb08a2e74633ec6f90a3166dcb94acf919285ff0': '2026-10-20T19:59:00Z', '922bd5c5ec4f109586817d69ed5f46c594b0016d45a9586810dbc3d4899e24a0': '2026-10-20T17:06:00Z', '9367210c2eab00267b3a28e1421b983951b83c4f3dd0fe0d45d39cec17dacf3d': '2026-10-20T19:59:00Z', '99abedc1beba19fefcde8d3af5a32456fa1eef7d5735adebeb8c0d5eb7422983': '2026-10-20T17:07:00Z', '9bc25446b167361cc4c2fb6759b2b99379f1a0bf63116b74a2bb08957f1ae74b': '2026-10-20T17:07:00Z', 'a99a2e5a04aca65f6ccdc3788d12dd3aa14e93f8ce9cfa4cbc9ac422cc2e3104': '2026-10-20T17:06:00Z', 'a9c7b799e0fbb225cac25ffcd03524091de13b6901c07b18fa6639750214a03f': '2026-10-20T17:23:00Z', 'ab65ba7c79c70a801aa378dd5cc136ceda948f18a74d71ffb60ce7f3f37109ed': '2026-10-20T19:59:00Z', 'b58be9dc628c6adf7093ca9c6fadcc7ce5c8313ea4e267569627d383373045c0': '2026-10-20T17:07:00Z', 'b60565ceb5bc3f1c79ff340dbfda14e5b8feb4b2a395579630d22bde0ba8530d': '2026-10-21T02:17:00Z', 'ca85e58e6d5a8bacc5aca15592c9f68d16283a8dd92b3225a27e79e34444ba2d': '2026-10-20T16:34:00Z', 'd0175c1e45d6c217ff551562b118a98a3d25dbf38a069f4f5a79b254384679d3': '2026-10-20T19:59:00Z', 'd16a10549d220016df5d48112eb5919752a5fb1c3af623224f874bf49ee80378': '2026-10-20T17:24:00Z', 'd5bc5a881f83c507794f44f3ba33e2922177dfadacad57acf65576135a1da58a': '2026-10-20T17:23:00Z', 'd7386835561d4fcb433e81bc58af2bad909779d9f03c4da81bc43009ed4047a9': '2026-10-20T16:33:00Z', 'd94b45adb044464f1e3df6983c46b4ed649df79ed40cdd49ad9d2d6f79723898': '2026-10-20T19:59:00Z', 'dac78d2c5d1893171b3ddca73e1ee14cfddaa6056546f354f372e7e7b5ea4b75': '2026-10-21T02:17:00Z', 'dd54e2e7e79e895643de0b422cbf6f202f913553b273a221df47562c98a308ce': '2026-10-20T16:48:00Z', 'de64c840545fcd0278d454504353bbc826c619e77a4d406aaf4ec3e1479aef8c': '2026-10-20T19:59:00Z', 'e12cc2d54135129d9bdf2542b54ed8e7ad83efbaaa0081f7edcbac76911d5bc3': '2026-10-21T02:18:00Z', 'ee6c61c7ed1384cbb6950295e03d4c29ebc68a5fa08e689c958bb68d91b50e09': '2026-10-20T16:33:00Z', 'f2956c651cedf6ad5d710e10d3902c20ea0f9ce542586f03a645408c474be2a9': '2026-10-20T19:59:00Z', 'f99f3e44d904886cac06945d6fb8be44bbb6fe7583a37b8fde71133ce779b119': '2026-10-21T02:17:00Z', 'fa2b1dd5f8be7cdfee1723a261d7d7e97e1e8e8047cc807a050959d0b62dea75': '2026-10-20T17:06:00Z'}}
crash_seen:Tester4-648b80 {'boots': 0, 'tripped_at': '', 'rearmed_at': '', 'oops_boot': ''}
crash_seen:Tester4-f6b67c {'boots': 0, 'tripped_at': '', 'rearmed_at': '', 'oops_boot': ''}
@@ -0,0 +1,6 @@
12:42:44Z POST /backup/offbox/run -> HTTP/2 302 ['location: /backups/remote?flash=flash.offbox.run_started']
12:42:44Z ('running', 0, '', '', 0)
12:43:45Z ('ok', 63, '', '', 0)
{"last_duration": "52s", "last_error": "", "last_run": "2026-10-21T12:43:42Z", "orphaned": false, "progress": {"active": false, "current_app": "", "percent": 0, "bytes_done": 0, "total_bytes": 0, "done_human": "", "total_human": "", "files_done": 0, "total_files": 0, "current_file": "", "elapsed_sec": 0, "phase": ""}, "repo_size_human": "1.4 GB", "snapshots": 63, "status": "ok"}
12:44:02Z run 2 -> HTTP/2 302
12:45:02Z {'status': 'ok', 'snapshots': 64, 'repo_size_human': '1.4 GB', 'last_run': '2026-10-21T12:44:47Z', 'orphaned': False, 'last_error': ''}
@@ -0,0 +1,6 @@
POST 200
answer: Felhom — Doboz összekötése Felhom doboz összekötése Kész. Ha ez egy valódi hivatkozás volt, és a dobozod még nincs összekötve, néhány percen belül új e-mailt kapsz a regisztrált címedre. Ha nem jön, szólj az ügyfélszolgálatnak. Felhom.eu
--- today's spent link:
POST 200
answer: Felhom — Doboz összekötése Felhom doboz összekötése A régi dobozod még össze van kötve a fiókoddal. Ezért most nem küldünk új linket. Írj a Felhom ügyfélszolgálatának: leválasztjuk a régi dobozt, és utána küldünk új linket. Az új dobozod addig vár; nem kell újra telepítened. Felhom.eu
2026/10/21 14:45:25 [WARN] self-bind: Tester4 asked for a new link but host Tester4-f6b67c is still linked — no link sent; the operator is told
Binary file not shown.

After

Width:  |  Height:  |  Size: 90 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 155 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 136 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 129 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 132 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 133 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 148 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 148 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 131 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.3 KiB

@@ -0,0 +1,8 @@
Install pressed 11:56:12
pairing screen seen 11:59:59
bind POST 12:01:16
bind POST 2 12:02:58
claim 12:08:22
wrong code 12:09:55
right code 12:10:13
nextcloud install 12:12:15
@@ -0,0 +1,40 @@
13:48:37Z == 1 deploy glance
13:48:37Z deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}
13:48:52Z state: running family_gate in app_config: True
13:49:14Z [1 POSITIVE CONTROL after install] gate file: /opt/docker/stacks/traefik/dynamic/family-gate-glance.yml
13:49:14Z [1 POSITIVE CONTROL after install] a stranger's GET https://r1187glance.enkisfelhom.hu/ -> 401
13:49:14Z == 2 export to the data drive
13:49:14Z export start -> 200 {'ok': True}
13:49:14Z /api/export/status {"display_name": "Glance", "done": false, "error": "", "job_type": "export", "ok": true, "output_path": "", "output_size": "", "running": true, "stack_name": "glance", "steps": [{"label": "Előkészítés", "status": "running"}, {"label": "Konfiguráció mentése", "status": "pending"}, {"label": "Adatbázis mentése", "status": "pending"}, {"label": "Felhasználói adatok", "status": "pending"}, {"label": "
13:49:17Z /api/export/status {"display_name": "Glance", "done": false, "error": "", "job_type": "export", "ok": true, "output_path": "/mnt/felhom-drives/scratch_hdd/exports/glance_20261021-134917.fab", "output_size": "5.6 KB", "running": true, "stack_name": "glance", "steps": [{"label": "Előkészítés", "status": "done"}, {"label": "Konfiguráció mentése", "status": "done"}, {"label": "Adatbázis mentése", "status": "done"}, {"la
13:49:20Z /api/export/status {"display_name": "Glance", "done": true, "error": "", "job_type": "export", "ok": true, "output_path": "/mnt/felhom-drives/scratch_hdd/exports/glance_20261021-134917.fab", "output_size": "5.6 KB", "running": false, "stack_name": "glance", "steps": [{"label": "Előkészítés", "status": "done"}, {"label": "Konfiguráció mentése", "status": "done"}, {"label": "Adatbázis mentése", "status": "done"}, {"la
13:49:20Z bundles for glance: []
13:49:20Z == 3 remove glance (keep nothing)
13:49:53Z remove -> 200 {'ok': True, 'data': {'removed': 'glance', 'volumes_removed': ['glance_glance_config'], 'hdd_paths_removed': [], 'hdd_paths_preserved': [], 'hdd_note': 'Az alka
13:50:05Z [3 after remove] gate file: (none)
13:50:05Z [3 after remove] a stranger's GET https://r1187glance.enkisfelhom.hu/ -> 404
13:50:05Z == 4 import the bundle
13:50:05Z import start -> 400 {'error': 'Missing path', 'ok': False}
13:50:05Z /api/export/import/status {"display_name": "Glance", "done": true, "error": "", "job_type": "export", "ok": true, "output_path": "/mnt/felhom-drives/scratch_hdd/exports/glance_20261021-134917.fab", "output_size": "5.6 KB", "running": false, "stack_name": "glance", "steps": [{"label": "Előkészítés", "status": "done"}, {"label": "Konfiguráció mentése", "status": "done"}, {"label": "Adatbázis mentése", "status": "done"}, {"la
13:53:26Z state after import: not_deployed family_gate in app_config: False app_config keys: []
13:53:57Z [4 AFTER import (fixed: must be 401, gate file present)] gate file: (none)
13:53:57Z [4 AFTER import (fixed: must be 401, gate file present)] a stranger's GET https://r1187glance.enkisfelhom.hu/ -> 404
13:53:59Z controller log: 2026/10/21 13:48:37 family_gate.go:145: [INFO] [stacks] glance: family gate ON before the first start — only family members reach [r1187glance.enkisfelhom.hu] (exceptions: [])
2026/10/21 13:48:37 manager.go:1686: [DEBUG] Env vars for compose: [PATH, HOSTNAME, FELHOM_BOOTSTRAP_PATH, HOME, DOMAIN, SUBDOMAIN, DOMAIN, IMPORT_PATH] (8 app + 0 system)
2026/10/21 13:48:44 manager.go:1686: [DEBUG] Env vars for compose: [PATH, HOSTNAME, FELHOM_BOOTSTRAP_PATH, HOME, DOMAIN, DOMAIN, SUBDOMAIN, IMPORT_PATH] (8 app + 0 system)
2026/10/21 13:49:14 family_gate.go:325: [DEBUG] [web] family gate glance: GET / without a pass — 401 (visitor 192.168.0.180)
2026/10/21 13:49:15 manager.go:1686: [DEBUG] Env vars for compose: [PATH, HOSTNAME, FELHOM_BOOTSTRAP_PATH, HOME, DOMAIN, DOMAIN, SUBDOMAIN, IMPORT_PATH] (8 app + 0 system)
2026/10/21 13:49:17 manager.go:1686: [DEBUG] Env vars for compose: [PATH, HOSTNAME, FELHOM_BOOTSTRAP_PATH, HOME, DOMAIN, DOMAIN, SUBDOMAIN, IMPORT_PATH] (8 app + 0 system)
2026/10/21 13:49:21 manager.go:1686: [DEBUG] Env vars for compose: [PATH, HOSTNAME, FELHOM_BOOTSTRAP_PATH, HOME, DOMAIN, DOMAIN, SUBDOMAIN, IMPORT_PATH] (8 app + 0 system)
2026/10/21 13:49:21 manager.go:1686: [DEBUG] Env vars for compose: [PATH, HOSTNAME, FELHOM_BOOTSTRAP_PATH, HOME, DOMAIN, DOMAIN, SUBDOMAIN, IMPORT_PATH] (8 app + 0 system)
2026/10/21 13:49:26 manager.go:1686: [DEBUG] Env vars for compose: [PATH, HOSTNAME, FELHOM_BOOTSTRAP_PATH, HOME, DOMAIN, DOMAIN, SUBDOMAIN, IMPORT_PATH] (8 app + 0 system)
2026/10/21 13:49:53 manager.go:675: [DEBUG] [stacks] ScanStacks: found stack "recipe-importer" deployed=false composePath=/opt/docker/stacks/recipe-importer/docker-compose.yml
2026/10/21 13:50:00 manager.go:675: [DEBUG] [stacks] ScanStacks: found stack "recipe-importer" deployed=false composePath=/opt/docker/stacks/recipe-importer/docker-compose.yml
2026/10/21 13:50:00 family_gate.go:190: [INFO] [stacks] glance: removed the family-gate file of an app that is not installed
2026/10/21 13:50:05 auth.go:144: [DEBUG] [web] auth: valid session for POST /api/export/import
2026/10/21 13:50:05 handler_export.go:328: [DEBUG] [web] apiImportStart: path="" hasPassword=false
2026/10/21 13:52:00 manager.go:675: [DEBUG] [stacks] ScanStacks: found stack "recipe-importer" deployed=false composePath=/opt/docker/stacks/recipe-importer/docker-compose.yml
13:53:59Z == 5 teardown: remove glance through the product, delete the bundle
13:54:31Z remove -> 200 {'ok': True, 'data': {'removed': 'glance', 'volumes_removed': [], 'hdd_paths_removed': [], 'hdd_paths_preserved': [], 'hdd_note': 'Az alkalmazás nem tárolt sajá
13:54:43Z [5 after teardown] gate file: (none)
13:54:43Z [5 after teardown] a stranger's GET https://r1187glance.enkisfelhom.hu/ -> 404
@@ -0,0 +1,42 @@
13:55:01Z == 1 deploy glance
13:55:02Z deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}
13:55:22Z state: running family_gate in app_config: True
13:55:44Z [1 POSITIVE CONTROL after install] gate file: /opt/docker/stacks/traefik/dynamic/family-gate-glance.yml
13:55:44Z [1 POSITIVE CONTROL after install] a stranger's GET https://r1187glance.enkisfelhom.hu/ -> 401
13:55:44Z == 2 export to the data drive
13:55:44Z export start -> 200 {'ok': True}
13:55:44Z /api/export/status {"display_name": "Glance", "done": false, "error": "", "job_type": "export", "ok": true, "output_path": "", "output_size": "", "running": true, "stack_name": "glance", "steps": [{"label": "Előkészítés", "status": "running"}, {"label": "Konfiguráció mentése", "status": "pending"}, {"label": "Adatbázis mentése", "status": "pending"}, {"label": "Felhasználói adatok", "status": "pending"}, {"label": "
13:55:47Z /api/export/status {"display_name": "Glance", "done": true, "error": "", "job_type": "export", "ok": true, "output_path": "/mnt/felhom-drives/scratch_hdd/exports/glance_20261021-135546.fab", "output_size": "5.6 KB", "running": false, "stack_name": "glance", "steps": [{"label": "Előkészítés", "status": "done"}, {"label": "Konfiguráció mentése", "status": "done"}, {"label": "Adatbázis mentése", "status": "done"}, {"la
13:55:47Z bundles for glance: []
13:55:47Z bundle path: /mnt/felhom-drives/scratch_hdd/exports/glance_20261021-135546.fab
13:55:47Z == 3 remove glance (keep nothing)
13:56:19Z remove -> 200 {'ok': True, 'data': {'removed': 'glance', 'volumes_removed': ['glance_glance_config'], 'hdd_paths_removed': [], 'hdd_paths_preserved': [], 'hdd_note': 'Az alka
13:56:31Z [3 after remove] gate file: (none)
13:56:31Z [3 after remove] a stranger's GET https://r1187glance.enkisfelhom.hu/ -> 404
13:56:31Z == 4 import the bundle
13:56:31Z import start -> 200 {'ok': True}
13:56:31Z /api/export/import/status {"display_name": "Glance", "done": false, "error": "", "job_type": "import", "ok": true, "output_path": "", "output_size": "", "running": true, "stack_name": "glance", "steps": [{"label": "Csomag megnyitása", "status": "done"}, {"label": "Alkalmazás előkészítése", "status": "done"}, {"label": "Konfiguráció visszaállítása", "status": "done"}, {"label": "Felhasználói adatok visszaállítása", "status"
13:56:37Z /api/export/import/status {"display_name": "Glance", "done": true, "error": "", "job_type": "import", "ok": true, "output_path": "", "output_size": "", "running": false, "stack_name": "glance", "steps": [{"label": "Csomag megnyitása", "status": "done"}, {"label": "Alkalmazás előkészítése", "status": "done"}, {"label": "Konfiguráció visszaállítása", "status": "done"}, {"label": "Felhasználói adatok visszaállítása", "status"
13:56:52Z state after import: running family_gate in app_config: True app_config keys: ['deployed', 'deployed_at', 'desired_state', 'env', 'family_gate', 'installed_images', 'locked_fields']
13:57:24Z [4 AFTER import (fixed: must be 401, gate file present)] gate file: /opt/docker/stacks/traefik/dynamic/family-gate-glance.yml
13:57:24Z [4 AFTER import (fixed: must be 401, gate file present)] a stranger's GET https://r1187glance.enkisfelhom.hu/ -> 401
13:57:26Z controller log: 2026/10/21 13:56:31 handler_export.go:352: [INFO] [web] Import started from /mnt/felhom-drives/scratch_hdd/exports/glance_20261021-135546.fab
2026/10/21 13:56:31 restore.go:223: [INFO] [appexport] Import started for glance_20261021-135546.fab
2026/10/21 13:56:31 export.go:154: [DEBUG] [appexport] === IMPORT START: path=/mnt/felhom-drives/scratch_hdd/exports/glance_20261021-135546.fab encrypted=false ===
2026/10/21 13:56:31 export.go:154: [DEBUG] [appexport] temp dir: /tmp/felhom-import-3760501659
2026/10/21 13:56:31 restore.go:330: [INFO] Import: opening bundle for glance (Glance)
2026/10/21 13:56:31 export.go:154: [DEBUG] [appexport] restoring config from /tmp/felhom-import-3760501659/config → /opt/docker/stacks/glance
2026/10/21 13:56:31 import_record.go:51: [INFO] [stacks] glance: import record saved (fresh=true, family gate carried=false, sign-up lock carried=false)
2026/10/21 13:56:31 restore.go:762: [INFO] Import: populating volume glance_glance_config
2026/10/21 13:56:31 family_gate.go:145: [INFO] [stacks] glance: family gate ON before the first start — only family members reach [r1187glance.enkisfelhom.hu] (exceptions: [])
2026/10/21 13:56:31 import_record.go:95: [INFO] [stacks] glance: import prepared before the start (family gate=true, sign-up lock=false)
2026/10/21 13:56:31 manager.go:1686: [DEBUG] Env vars for compose: [PATH, HOSTNAME, FELHOM_BOOTSTRAP_PATH, HOME, DOMAIN, DOMAIN, SUBDOMAIN, IMPORT_PATH] (8 app + 0 system)
2026/10/21 13:56:35 restore.go:485: [INFO] Import completed: glance (Glance) in 4.029790494s
2026/10/21 13:56:35 export.go:154: [DEBUG] [appexport] === IMPORT END: elapsed=4.030525583s ===
2026/10/21 13:56:38 manager.go:1686: [DEBUG] Env vars for compose: [PATH, HOSTNAME, FELHOM_BOOTSTRAP_PATH, HOME, DOMAIN, DOMAIN, SUBDOMAIN, IMPORT_PATH] (8 app + 0 system)
2026/10/21 13:57:24 family_gate.go:325: [DEBUG] [web] family gate glance: GET / without a pass — 401 (visitor 192.168.0.180)
13:57:26Z == 5 teardown: remove glance through the product, delete the bundle
13:57:58Z remove -> 200 {'ok': True, 'data': {'removed': 'glance', 'volumes_removed': ['glance_glance_config'], 'hdd_paths_removed': [], 'hdd_paths_preserved': [], 'hdd_note': 'Az alka
13:57:59Z bundle delete: deleted /mnt/felhom-drives/scratch_hdd/exports/glance_20261021-135546.fab
13:58:11Z [5 after teardown] gate file: (none)
13:58:11Z [5 after teardown] a stranger's GET https://r1187glance.enkisfelhom.hu/ -> 404
@@ -0,0 +1,4 @@
load started 13:26:08
exp1 swap 512->4096 at 13:34:06
proof started 13:43:45 (swap cap 4096, guest RAM 4096)
evidence copied 16:27:04
@@ -0,0 +1,15 @@
13:10:54Z immich deploy 202 Telepítés elindítva – az állapot a kártyán követhető
13:13:25Z immich → running 152 s
13:13:25Z paperless-ngx deploy 202 Telepítés elindítva – az állapot a kártyán követhető
13:15:46Z paperless-ngx → running 140 s
13:15:46Z vaultwarden deploy 202 Telepítés elindítva – az állapot a kártyán követhető
13:16:06Z vaultwarden → running 20 s
13:16:06Z mealie deploy 202 Telepítés elindítva – az állapot a kártyán követhető
13:17:36Z mealie → running 90 s
13:17:37Z actualbudget deploy 202 Telepítés elindítva – az állapot a kártyán követhető
13:18:07Z actualbudget → running 30 s
13:18:07Z radicale deploy 202 Telepítés elindítva – az állapot a kártyán követhető
13:18:27Z radicale → running 20 s
13:18:27Z bookstack deploy 202 Telepítés elindítva – az állapot a kártyán követhető
13:19:37Z bookstack → running 70 s
13:19:37Z memory: {"data": {"vmid": 9201, "allocated_mb": 4096, "usage_mb": 2550, "host_total_mb": 7939, "min_mb": 2048, "max_mb": 5891, "floor_mb": 3062, "running": true}, "ok": true}
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,312 @@
13:27:30Z LOAD START
13:27:30Z immich admin-sign-up 201
/root/load.py:42: DeprecationWarning: datetime.datetime.utcfromtimestamp() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.fromtimestamp(timestamp, datetime.UTC).
st = os.stat(f); ts = dt.datetime.utcfromtimestamp(st.st_mtime).isoformat() + "Z"
13:27:31Z immich IMG_20090000.jpg 201
13:27:31Z immich IMG_20090018.jpg 201
13:27:31Z immich IMG_20090036.jpg 201
13:27:31Z immich IMG_20090054.jpg 201
13:27:31Z immich IMG_20090072.jpg 201
13:27:31Z immich IMG_20090090.jpg 201
13:27:31Z immich IMG_20090108.jpg 201
13:27:32Z immich IMG_20090126.jpg 201
13:27:32Z immich IMG_20090144.jpg 201
13:27:32Z immich IMG_20090162.jpg 201
13:27:32Z immich IMG_20090180.jpg 201
13:27:32Z immich IMG_20090198.jpg 201
13:27:32Z immich IMG_20090216.jpg 201
13:27:32Z immich IMG_20090234.jpg 201
13:27:32Z immich IMG_20090252.jpg 201
13:27:32Z immich IMG_20090270.jpg 201
13:27:32Z immich IMG_20090288.jpg 201
13:27:32Z immich IMG_20100001.jpg 201
13:27:33Z immich IMG_20100019.jpg 201
13:27:33Z immich IMG_20100037.jpg 201
13:27:33Z immich IMG_20100055.jpg 201
13:27:33Z immich IMG_20100073.jpg 201
13:27:33Z immich IMG_20100091.jpg 201
13:27:33Z immich IMG_20100109.jpg 201
13:27:33Z immich IMG_20100127.jpg 201
13:27:33Z immich IMG_20100145.jpg 201
13:27:34Z immich IMG_20100163.jpg 201
13:27:34Z immich IMG_20100181.jpg 201
13:27:34Z immich IMG_20100199.jpg 201
13:27:34Z immich IMG_20100217.jpg 201
13:27:34Z immich IMG_20100235.jpg 201
13:27:35Z immich IMG_20100253.jpg 201
13:27:35Z immich IMG_20100271.jpg 201
13:27:35Z immich IMG_20100289.jpg 201
13:27:35Z immich IMG_20110002.jpg 201
13:27:35Z immich IMG_20110020.jpg 201
13:27:35Z immich IMG_20110038.jpg 201
13:27:35Z immich IMG_20110056.jpg 201
13:27:36Z immich IMG_20110074.jpg 201
13:27:36Z immich IMG_20110092.jpg 201
13:27:37Z immich IMG_20110110.jpg 201
13:27:37Z immich IMG_20110128.jpg 201
13:27:37Z immich IMG_20110146.jpg 201
13:27:37Z immich IMG_20110164.jpg 201
13:27:38Z immich IMG_20110182.jpg 201
13:27:38Z immich IMG_20110200.jpg 201
13:27:38Z immich IMG_20110218.jpg 201
13:27:38Z immich IMG_20110236.jpg 201
13:27:38Z immich IMG_20110254.jpg 201
13:27:38Z immich IMG_20110272.jpg 201
13:27:38Z immich IMG_20110290.jpg 201
13:27:38Z immich IMG_20120003.jpg 201
13:27:39Z immich IMG_20120021.jpg 201
13:27:39Z immich IMG_20120039.jpg 201
13:27:39Z immich IMG_20120057.jpg 201
13:27:40Z immich IMG_20120075.jpg 201
13:27:40Z immich IMG_20120093.jpg 201
13:27:40Z immich IMG_20120111.jpg 201
13:27:41Z immich IMG_20120129.jpg 201
13:27:41Z immich IMG_20120147.jpg 201
13:27:42Z immich IMG_20120165.jpg 201
13:27:42Z immich IMG_20120183.jpg 201
13:27:42Z immich IMG_20120201.jpg 201
13:27:42Z immich IMG_20120219.jpg 201
13:27:42Z immich IMG_20120237.jpg 201
13:27:42Z immich IMG_20120255.jpg 201
13:27:42Z immich IMG_20120273.jpg 201
13:27:43Z immich IMG_20120291.jpg 201
13:27:43Z immich IMG_20130004.jpg 201
13:27:43Z immich IMG_20130022.jpg 201
13:27:43Z immich IMG_20130040.jpg 201
13:27:43Z immich IMG_20130058.jpg 201
13:27:43Z immich IMG_20130076.jpg 201
13:27:44Z immich IMG_20130094.jpg 201
13:27:44Z immich IMG_20130112.jpg 201
13:27:44Z immich IMG_20130130.jpg 201
13:27:44Z immich IMG_20130148.jpg 201
13:27:44Z immich IMG_20130166.jpg 201
13:27:44Z immich IMG_20130184.jpg 201
13:27:45Z immich IMG_20130202.jpg 201
13:27:45Z immich IMG_20130220.jpg 201
13:27:45Z immich IMG_20130238.jpg 201
13:27:45Z immich IMG_20130256.jpg 201
13:27:46Z immich IMG_20130274.jpg 201
13:27:46Z immich IMG_20130292.jpg 201
13:27:46Z immich IMG_20140005.jpg 201
13:27:55Z immich IMG_20140023.jpg 201
13:27:55Z immich IMG_20140041.jpg 201
13:27:55Z immich IMG_20140059.jpg 201
13:27:56Z immich IMG_20140077.jpg 201
13:27:58Z immich IMG_20140095.jpg 201
13:27:58Z immich IMG_20140113.jpg 201
13:27:58Z immich IMG_20140131.jpg 201
13:27:58Z immich IMG_20140149.jpg 201
13:27:58Z immich IMG_20140167.jpg 201
13:27:58Z immich IMG_20140185.jpg 201
13:27:59Z immich IMG_20140203.jpg 201
13:27:59Z immich IMG_20140221.jpg 201
13:27:59Z immich IMG_20140239.jpg 201
13:28:00Z immich IMG_20140257.jpg 201
13:28:00Z immich IMG_20140275.jpg 201
13:28:00Z immich IMG_20140293.jpg 201
13:28:00Z immich IMG_20150006.jpg 201
13:28:00Z immich IMG_20150024.jpg 201
13:28:00Z immich IMG_20150042.jpg 201
13:28:00Z immich IMG_20150060.jpg 201
13:28:01Z immich IMG_20150078.jpg 201
13:28:01Z immich IMG_20150096.jpg 201
13:28:01Z immich IMG_20150114.jpg 201
13:28:01Z immich IMG_20150132.jpg 201
13:28:01Z immich IMG_20150150.jpg 201
13:28:01Z immich IMG_20150168.jpg 201
13:28:01Z immich IMG_20150186.jpg 201
13:28:02Z immich IMG_20150204.jpg 201
13:28:02Z immich IMG_20150222.jpg 201
13:28:02Z immich IMG_20150240.jpg 201
13:28:02Z immich IMG_20150258.jpg 201
13:28:02Z immich IMG_20150276.jpg 201
13:28:03Z immich IMG_20150294.jpg 201
13:28:03Z immich IMG_20160007.jpg 201
13:28:03Z immich IMG_20160025.jpg 201
13:28:03Z immich IMG_20160043.jpg 201
13:28:04Z immich IMG_20160061.jpg 201
13:28:04Z immich IMG_20160079.jpg 201
13:28:04Z immich IMG_20160097.jpg 201
13:28:04Z immich IMG_20160115.jpg 201
13:28:05Z immich IMG_20160133.jpg 201
13:28:05Z immich IMG_20160151.jpg 201
13:28:05Z immich IMG_20160169.jpg 201
13:28:05Z immich IMG_20160187.jpg 201
13:28:06Z immich IMG_20160205.jpg 201
13:28:07Z immich IMG_20160223.jpg 201
13:28:07Z immich IMG_20160241.jpg 201
13:28:08Z immich IMG_20160259.jpg 201
13:28:08Z immich IMG_20160277.jpg 201
13:28:08Z immich IMG_20160295.jpg 201
13:28:09Z immich IMG_20170008.jpg 201
13:28:09Z immich IMG_20170026.jpg 201
13:28:09Z immich IMG_20170044.jpg 201
13:28:10Z immich IMG_20170062.jpg 201
13:28:10Z immich IMG_20170080.jpg 201
13:28:10Z immich IMG_20170098.jpg 201
13:28:10Z immich IMG_20170116.jpg 201
13:28:11Z immich IMG_20170134.jpg 201
13:28:11Z immich IMG_20170152.jpg 201
13:28:11Z immich IMG_20170170.jpg 201
13:34:32Z immich IMG_20170188.jpg 201
13:34:37Z immich IMG_20170206.jpg 502
13:34:55Z immich IMG_20170224.jpg 201
13:34:56Z immich IMG_20170242.jpg 201
13:34:56Z immich IMG_20170260.jpg 201
13:34:56Z immich IMG_20170278.jpg 201
13:34:57Z immich IMG_20170296.jpg 201
13:34:57Z immich IMG_20180009.jpg 201
13:34:57Z immich IMG_20180027.jpg 201
13:34:57Z immich IMG_20180045.jpg 201
13:34:58Z immich IMG_20180063.jpg 201
13:34:58Z immich IMG_20180081.jpg 201
13:34:59Z immich IMG_20180099.jpg 201
13:34:59Z immich IMG_20180117.jpg 201
13:35:00Z immich IMG_20180135.jpg 201
13:35:00Z immich IMG_20180153.jpg 201
13:35:00Z immich IMG_20180171.jpg 201
13:35:00Z immich IMG_20180189.jpg 201
13:35:01Z immich IMG_20180207.jpg 201
13:35:01Z immich IMG_20180225.jpg 201
13:35:01Z immich IMG_20180243.jpg 201
13:35:02Z immich IMG_20180261.jpg 201
13:35:02Z immich IMG_20180279.jpg 201
13:35:02Z immich IMG_20180297.jpg 201
13:35:02Z immich IMG_20190010.jpg 201
13:35:03Z immich IMG_20190028.jpg 201
13:35:03Z immich IMG_20190046.jpg 201
13:35:03Z immich IMG_20190064.jpg 201
13:35:03Z immich IMG_20190082.jpg 201
13:35:03Z immich IMG_20190100.jpg 201
13:35:03Z immich IMG_20190118.jpg 201
13:35:03Z immich IMG_20190136.jpg 201
13:35:03Z immich IMG_20190154.jpg 201
13:35:04Z immich IMG_20190172.jpg 201
13:35:04Z immich IMG_20190190.jpg 201
13:35:04Z immich IMG_20190208.jpg 201
13:35:04Z immich IMG_20190226.jpg 201
13:35:05Z immich IMG_20190244.jpg 201
13:35:05Z immich IMG_20190262.jpg 201
13:35:05Z immich IMG_20190280.jpg 201
13:35:05Z immich IMG_20190298.jpg 201
13:35:05Z immich IMG_20200011.jpg 201
13:35:05Z immich IMG_20200029.jpg 201
13:35:06Z immich IMG_20200047.jpg 201
13:35:06Z immich IMG_20200065.jpg 201
13:35:06Z immich IMG_20200083.jpg 201
13:35:06Z immich IMG_20200101.jpg 201
13:35:06Z immich IMG_20200119.jpg 201
13:35:06Z immich IMG_20200137.jpg 201
13:35:07Z immich IMG_20200155.jpg 201
13:35:07Z immich IMG_20200173.jpg 201
13:35:07Z immich IMG_20200191.jpg 201
13:35:07Z immich IMG_20200209.jpg 201
13:35:07Z immich IMG_20200227.jpg 201
13:35:07Z immich IMG_20200245.jpg 201
13:35:08Z immich IMG_20200263.jpg 201
13:35:08Z immich IMG_20200281.jpg 201
13:35:08Z immich IMG_20200299.jpg 201
13:35:08Z immich IMG_20210012.jpg 201
13:35:08Z immich IMG_20210030.jpg 201
13:35:09Z immich IMG_20210048.jpg 201
13:35:09Z immich IMG_20210066.jpg 201
13:35:09Z immich IMG_20210084.jpg 201
13:35:09Z immich IMG_20210102.jpg 201
13:35:09Z immich IMG_20210120.jpg 201
13:35:09Z immich IMG_20210138.jpg 201
13:35:10Z immich IMG_20210156.jpg 201
13:35:10Z immich IMG_20210174.jpg 201
13:35:10Z immich IMG_20210192.jpg 201
13:35:11Z immich IMG_20210210.jpg 201
13:35:11Z immich IMG_20210228.jpg 201
13:35:11Z immich IMG_20210246.jpg 201
13:35:11Z immich IMG_20210264.jpg 201
13:35:11Z immich IMG_20210282.jpg 201
13:35:11Z immich IMG_20220013.jpg 201
13:35:11Z immich IMG_20220031.jpg 201
13:35:11Z immich IMG_20220049.jpg 201
13:35:11Z immich IMG_20220067.jpg 201
13:35:11Z immich IMG_20220085.jpg 201
13:35:12Z immich IMG_20220103.jpg 201
13:35:12Z immich IMG_20220121.jpg 201
13:35:13Z immich IMG_20220139.jpg 201
13:35:13Z immich IMG_20220157.jpg 201
13:35:13Z immich IMG_20220175.jpg 201
13:35:13Z immich IMG_20220193.jpg 201
13:35:13Z immich IMG_20220211.jpg 201
13:35:14Z immich IMG_20220229.jpg 201
13:35:14Z immich IMG_20220247.jpg 201
13:35:14Z immich IMG_20220265.jpg 201
13:35:15Z immich IMG_20220283.jpg 201
13:35:15Z immich IMG_20230014.jpg 201
13:35:16Z immich IMG_20230032.jpg 201
13:35:16Z immich IMG_20230050.jpg 201
13:35:16Z immich IMG_20230068.jpg 201
13:35:17Z immich IMG_20230086.jpg 201
13:35:17Z immich IMG_20230104.jpg 201
13:35:17Z immich IMG_20230122.jpg 201
13:35:18Z immich IMG_20230140.jpg 201
13:35:18Z immich IMG_20230158.jpg 201
13:35:18Z immich IMG_20230176.jpg 201
13:35:18Z immich IMG_20230194.jpg 201
13:35:19Z immich IMG_20230212.jpg 201
13:35:19Z immich IMG_20230230.jpg 201
13:35:19Z immich IMG_20230248.jpg 201
13:35:19Z immich IMG_20230266.jpg 201
13:35:19Z immich IMG_20230284.jpg 201
13:35:20Z immich IMG_20240015.jpg 201
13:35:20Z immich IMG_20240033.jpg 201
13:35:20Z immich IMG_20240051.jpg 201
13:35:20Z immich IMG_20240069.jpg 201
13:35:20Z immich IMG_20240087.jpg 201
13:35:20Z immich IMG_20240105.jpg 201
13:35:20Z immich IMG_20240123.jpg 201
13:35:21Z immich IMG_20240141.jpg 201
13:35:21Z immich IMG_20240159.jpg 201
13:35:22Z immich IMG_20240177.jpg 201
13:35:23Z immich IMG_20240195.jpg 201
13:35:23Z immich IMG_20240213.jpg 201
13:35:23Z immich IMG_20240231.jpg 201
13:35:24Z immich IMG_20240249.jpg 201
13:35:24Z immich IMG_20240267.jpg 201
13:35:25Z immich IMG_20240285.jpg 201
13:35:25Z immich IMG_20250016.jpg 201
13:35:25Z immich IMG_20250034.jpg 201
13:35:25Z immich IMG_20250052.jpg 201
13:35:25Z immich IMG_20250070.jpg 201
13:35:26Z immich IMG_20250088.jpg 201
13:35:26Z immich IMG_20250106.jpg 201
13:35:26Z immich IMG_20250124.jpg 201
13:35:27Z immich IMG_20250142.jpg 201
13:35:27Z immich IMG_20250160.jpg 201
13:35:27Z immich IMG_20250178.jpg 201
13:35:27Z immich IMG_20250196.jpg 201
13:35:27Z immich IMG_20250214.jpg 201
13:35:27Z immich IMG_20250232.jpg 201
13:35:27Z immich IMG_20250250.jpg 201
13:35:28Z immich IMG_20250268.jpg 201
13:35:28Z immich IMG_20250286.jpg 201
13:35:28Z immich IMG_20260017.jpg 201
13:35:28Z immich IMG_20260035.jpg 201
13:35:28Z immich IMG_20260053.jpg 201
13:35:28Z immich IMG_20260071.jpg 201
13:35:28Z immich IMG_20260089.jpg 201
13:35:29Z immich IMG_20260107.jpg 201
13:35:29Z immich IMG_20260125.jpg 201
13:35:29Z immich IMG_20260143.jpg 201
13:35:29Z immich IMG_20260161.jpg 201
13:35:29Z immich IMG_20260179.jpg 201
13:35:30Z immich IMG_20260197.jpg 201
13:35:30Z immich IMG_20260215.jpg 201
13:35:30Z immich IMG_20260233.jpg 201
13:35:30Z immich IMG_20260251.jpg 201
13:35:31Z immich IMG_20260269.jpg 201
13:35:31Z immich IMG_20260287.jpg 201
13:35:31Z immich IMG_20160007_masolat.jpg 200
13:35:31Z immich duplikatum_IMG_20160007.jpg 200
13:35:39Z immich nyaralas_2025.mp4 201
13:35:39Z immich rajz_Kata.png 201
13:35:39Z immich torott_kep.jpg 201
13:35:39Z immich ures.jpg 400
13:35:39Z immich DONE ok 304 bad 2
13:35:39Z LOAD END
@@ -0,0 +1,60 @@
13:26:05Z LOAD START
13:26:05Z immich admin-sign-up 401
13:26:05Z immich login 401
13:26:05Z nextcloud rclone START
13:26:06Z paperless jelszavas_kivonat.pdf 200
13:26:06Z paperless kezikonyv_100_oldal.pdf 200
13:26:06Z paperless szamla_2024_000.pdf 200
13:26:07Z paperless szamla_2024_003.pdf 200
13:26:07Z paperless szamla_2024_006.pdf 200
13:26:08Z paperless szamla_2024_009.pdf 200
13:26:08Z paperless szamla_2024_012.pdf 200
13:26:09Z paperless szamla_2024_015.pdf 200
13:26:09Z paperless szamla_2024_018.pdf 200
13:26:10Z paperless szamla_2024_021.pdf 200
13:26:11Z paperless szamla_2024_024.pdf 200
13:26:11Z paperless szamla_2024_027.pdf 200
13:26:12Z paperless szamla_2024_030.pdf 200
13:26:12Z paperless szamla_2024_033.pdf 200
13:26:12Z paperless szamla_2024_036.pdf 200
13:26:13Z paperless szamla_2024_039.pdf 200
13:26:14Z paperless szamla_2024_042.pdf 200
13:26:14Z paperless szamla_2024_045.pdf 200
13:26:15Z paperless szamla_2024_048.pdf 200
13:26:15Z paperless szamla_2025_001.pdf 200
13:26:16Z paperless szamla_2025_004.pdf 200
13:26:16Z paperless szamla_2025_007.pdf 200
13:26:17Z paperless szamla_2025_010.pdf 200
13:26:18Z paperless szamla_2025_013.pdf 200
13:26:18Z paperless szamla_2025_016.pdf 200
13:26:19Z paperless szamla_2025_019.pdf 200
13:26:19Z paperless szamla_2025_022.pdf 200
13:26:20Z paperless szamla_2025_025.pdf 200
13:26:21Z paperless szamla_2025_028.pdf 200
13:26:21Z paperless szamla_2025_031.pdf 200
13:26:22Z paperless szamla_2025_034.pdf 200
13:26:23Z paperless szamla_2025_037.pdf 200
13:26:23Z paperless szamla_2025_040.pdf 200
13:26:24Z paperless szamla_2025_043.pdf 200
13:26:25Z paperless szamla_2025_046.pdf 200
13:26:25Z paperless szamla_2025_049.pdf 200
13:26:25Z paperless szamla_2026_002.pdf 200
13:26:26Z paperless szamla_2026_005.pdf 200
13:26:26Z paperless szamla_2026_008.pdf 200
13:26:27Z paperless szamla_2026_011.pdf 200
13:26:27Z paperless szamla_2026_014.pdf 200
13:26:28Z paperless szamla_2026_017.pdf 200
13:26:29Z paperless szamla_2026_020.pdf 200
13:26:29Z paperless szamla_2026_023.pdf 200
13:26:30Z paperless szamla_2026_026.pdf 200
13:26:30Z paperless szamla_2026_029.pdf 200
13:26:31Z paperless szamla_2026_032.pdf 200
13:26:31Z paperless szamla_2026_035.pdf 200
13:26:32Z paperless szamla_2026_038.pdf 200
13:26:32Z paperless szamla_2026_041.pdf 200
13:26:33Z paperless szamla_2026_044.pdf 200
13:26:33Z paperless szamla_2026_047.pdf 200
13:26:34Z paperless szkennelt_nyilatkozat.pdf 200
13:26:34Z paperless DONE ok 53 bad 0
16:25:27Z nextcloud DONE rc 0
16:25:27Z LOAD END
@@ -0,0 +1,555 @@
13:43:40Z PROOF START 120 min — each stream repeats on its own until the end
13:43:42Z c1 immich new user 201
13:43:47Z c1 paperless delete 52 200
13:43:48Z c1 paperless empty trash 200
13:44:47Z c1 immich DONE ok 305 bad 1
13:44:48Z c1 paperless DONE ok 53 bad 0
13:44:48Z c2 immich new user 201
13:44:50Z c2 paperless delete 3 200
13:44:51Z c2 paperless empty trash 200
13:45:55Z c2 immich DONE ok 305 bad 1
13:45:57Z c3 immich new user 201
13:46:01Z c2 paperless DONE ok 53 bad 0
13:46:06Z c3 paperless delete 5 200
13:46:07Z c3 paperless empty trash 200
13:47:01Z c3 immich DONE ok 305 bad 1
13:47:03Z c4 immich new user 201
13:47:05Z c3 paperless DONE ok 53 bad 0
13:47:06Z c4 paperless delete 5 200
13:47:08Z c4 paperless empty trash 200
13:48:04Z c4 immich DONE ok 305 bad 1
13:48:06Z c5 immich new user 201
13:48:06Z c4 paperless DONE ok 53 bad 0
13:48:08Z c5 paperless delete 4 200
13:48:10Z c5 paperless empty trash 200
13:49:06Z c5 immich DONE ok 305 bad 1
13:49:07Z c5 paperless DONE ok 53 bad 0
13:49:08Z c6 immich new user 201
13:49:12Z c6 paperless delete 4 200
13:49:14Z c6 paperless empty trash 200
13:50:17Z c6 paperless DONE ok 53 bad 0
13:50:30Z c7 paperless delete 5 200
13:50:31Z c6 immich DONE ok 305 bad 1
13:50:31Z c7 paperless empty trash 200
13:50:32Z c7 immich new user 201
13:51:26Z c7 paperless DONE ok 53 bad 0
13:51:28Z c8 paperless delete 5 200
13:51:29Z c8 paperless empty trash 200
13:51:33Z c7 immich DONE ok 305 bad 1
13:51:34Z c8 immich new user 201
13:52:29Z c8 paperless DONE ok 53 bad 0
13:52:31Z c9 paperless delete 4 200
13:52:31Z c9 paperless empty trash 200
13:52:32Z c8 immich DONE ok 305 bad 1
13:52:33Z c9 immich new user 201
13:53:24Z c9 paperless DONE ok 53 bad 0
13:53:26Z c10 paperless delete 4 200
13:53:28Z c10 paperless empty trash 200
13:53:29Z c9 immich DONE ok 305 bad 1
13:53:31Z c10 immich new user 201
13:54:21Z c10 paperless DONE ok 53 bad 0
13:54:24Z c11 paperless delete 3 200
13:54:26Z c11 paperless empty trash 200
13:54:32Z c10 immich DONE ok 305 bad 1
13:54:34Z c11 immich new user 201
13:55:18Z c11 paperless DONE ok 53 bad 0
13:55:20Z c12 paperless delete 4 200
13:55:22Z c12 paperless empty trash 200
13:55:31Z c11 immich DONE ok 305 bad 1
13:55:33Z c12 immich new user 201
13:56:23Z c12 paperless DONE ok 53 bad 0
13:56:25Z c13 paperless delete 5 200
13:56:26Z c13 paperless empty trash 200
13:56:31Z c12 immich DONE ok 305 bad 1
13:56:33Z c13 immich new user 201
13:57:23Z c13 paperless DONE ok 53 bad 0
13:57:26Z c14 paperless delete 3 200
13:57:27Z c14 paperless empty trash 200
13:57:30Z c13 immich DONE ok 305 bad 1
13:57:32Z c14 immich new user 201
13:58:26Z c14 paperless DONE ok 53 bad 0
13:58:30Z c15 paperless delete 3 200
13:58:32Z c14 immich DONE ok 305 bad 1
13:58:32Z c15 paperless empty trash 200
13:58:33Z c15 immich new user 201
13:59:32Z c15 immich DONE ok 305 bad 1
13:59:32Z c15 paperless DONE ok 53 bad 0
13:59:33Z c16 immich new user 201
13:59:34Z c16 paperless delete 6 200
13:59:35Z c16 paperless empty trash 200
14:00:48Z c16 immich DONE ok 305 bad 1
14:00:49Z c17 immich new user 201
14:00:50Z c16 paperless DONE ok 53 bad 0
14:00:52Z c17 paperless delete 5 200
14:00:53Z c17 paperless empty trash 200
14:01:47Z c17 immich DONE ok 305 bad 1
14:01:49Z c18 immich new user 201
14:01:56Z c17 paperless DONE ok 53 bad 0
14:01:58Z c18 paperless delete 4 200
14:01:58Z c18 paperless empty trash 200
14:02:52Z c18 paperless DONE ok 53 bad 0
14:02:53Z c18 immich DONE ok 305 bad 1
14:02:55Z c19 immich new user 201
14:02:55Z c19 paperless delete 4 200
14:02:56Z c19 paperless empty trash 200
14:03:50Z c19 paperless DONE ok 53 bad 0
14:03:53Z c20 paperless delete 4 200
14:03:54Z c20 paperless empty trash 200
14:03:55Z c19 immich DONE ok 305 bad 1
14:03:57Z c20 immich new user 201
14:04:53Z c20 paperless DONE ok 53 bad 0
14:04:55Z c21 paperless delete 5 200
14:04:56Z c21 paperless empty trash 200
14:05:02Z c20 immich DONE ok 305 bad 1
14:05:04Z c21 immich new user 201
14:05:55Z c21 paperless DONE ok 53 bad 0
14:05:58Z c22 paperless delete 4 200
14:06:00Z c22 paperless empty trash 200
14:06:10Z c21 immich DONE ok 305 bad 1
14:06:12Z c22 immich new user 201
14:07:05Z c22 paperless DONE ok 53 bad 0
14:07:07Z c23 paperless delete 5 200
14:07:08Z c23 paperless empty trash 200
14:07:14Z c22 immich DONE ok 305 bad 1
14:07:15Z c23 immich new user 201
14:08:05Z c23 paperless DONE ok 53 bad 0
14:08:08Z c24 paperless delete 4 200
14:08:10Z c24 paperless empty trash 200
14:08:12Z c23 immich DONE ok 305 bad 1
14:08:13Z c24 immich new user 201
14:09:08Z c24 immich DONE ok 305 bad 1
14:09:10Z c25 immich new user 201
14:09:12Z c24 paperless DONE ok 53 bad 0
14:09:14Z c25 paperless delete 6 200
14:09:15Z c25 paperless empty trash 200
14:10:14Z c25 immich DONE ok 305 bad 1
14:10:15Z c26 immich new user 201
14:10:18Z c25 paperless DONE ok 53 bad 0
14:10:22Z c26 paperless delete 3 200
14:10:24Z c26 paperless empty trash 200
14:11:11Z c26 immich DONE ok 305 bad 1
14:11:13Z c27 immich new user 201
14:11:21Z c26 paperless DONE ok 53 bad 0
14:11:25Z c27 paperless delete 4 200
14:11:27Z c27 paperless empty trash 200
14:12:17Z c27 immich DONE ok 305 bad 1
14:12:19Z c28 immich new user 201
14:12:29Z c27 paperless DONE ok 53 bad 0
14:12:33Z c28 paperless delete 5 200
14:12:34Z c28 paperless empty trash 200
14:13:16Z c28 immich DONE ok 305 bad 1
14:13:19Z c29 immich new user 201
14:13:31Z c28 paperless DONE ok 53 bad 0
14:13:34Z c29 paperless delete 4 200
14:13:35Z c29 paperless empty trash 200
14:14:14Z c29 immich DONE ok 305 bad 1
14:14:17Z c30 immich new user 201
14:14:43Z c29 paperless DONE ok 53 bad 0
14:14:46Z c30 paperless delete 5 200
14:14:47Z c30 paperless empty trash 200
14:15:26Z c30 immich DONE ok 305 bad 1
14:15:28Z c31 immich new user 201
14:15:48Z c30 paperless DONE ok 53 bad 0
14:15:50Z c31 paperless delete 6 200
14:15:50Z c31 paperless empty trash 200
14:16:27Z c31 immich DONE ok 305 bad 1
14:16:28Z c32 immich new user 201
14:16:53Z c31 paperless DONE ok 53 bad 0
14:16:56Z c32 paperless delete 4 200
14:16:57Z c32 paperless empty trash 200
14:17:27Z c32 immich DONE ok 305 bad 1
14:17:29Z c33 immich new user 201
14:17:58Z c32 paperless DONE ok 53 bad 0
14:18:01Z c33 paperless delete 5 200
14:18:03Z c33 paperless empty trash 200
14:18:25Z c33 immich DONE ok 305 bad 1
14:18:26Z c34 immich new user 201
14:19:00Z c33 paperless DONE ok 53 bad 0
14:19:04Z c34 paperless delete 4 200
14:19:05Z c34 paperless empty trash 200
14:19:24Z c34 immich DONE ok 305 bad 1
14:19:25Z c35 immich new user 201
14:20:07Z c34 paperless DONE ok 53 bad 0
14:20:10Z c35 paperless delete 5 200
14:20:11Z c35 paperless empty trash 200
14:20:21Z c35 immich DONE ok 305 bad 1
14:20:24Z c36 immich new user 201
14:21:15Z c35 paperless DONE ok 53 bad 0
14:21:21Z c36 paperless delete 5 200
14:21:22Z c36 paperless empty trash 200
14:21:32Z c36 immich DONE ok 305 bad 1
14:21:35Z c37 immich new user 201
14:22:22Z c36 paperless DONE ok 53 bad 0
14:22:26Z c37 paperless delete 4 200
14:22:29Z c37 paperless empty trash 200
14:22:31Z c37 immich DONE ok 305 bad 1
14:22:34Z c38 immich new user 201
14:23:28Z c37 paperless DONE ok 53 bad 0
14:23:32Z c38 paperless delete 3 200
14:23:33Z c38 immich DONE ok 305 bad 1
14:23:33Z c38 paperless empty trash 200
14:23:35Z c39 immich new user 201
14:24:26Z c39 immich DONE ok 305 bad 1
14:24:28Z c40 immich new user 201
14:24:32Z c38 paperless DONE ok 53 bad 0
14:24:35Z c39 paperless delete 5 200
14:24:36Z c39 paperless empty trash 200
14:25:34Z c40 immich DONE ok 305 bad 1
14:25:35Z c41 immich new user 201
14:25:37Z c39 paperless DONE ok 53 bad 0
14:25:39Z c40 paperless delete 5 200
14:25:40Z c40 paperless empty trash 200
14:26:39Z c40 paperless DONE ok 53 bad 0
14:26:41Z c41 immich DONE ok 305 bad 1
14:26:41Z c41 paperless delete 4 200
14:26:43Z c41 paperless empty trash 200
14:26:44Z c42 immich new user 201
14:27:41Z c41 paperless DONE ok 53 bad 0
14:27:43Z c42 paperless delete 4 200
14:27:46Z c42 paperless empty trash 200
14:27:47Z c42 immich DONE ok 305 bad 1
14:27:48Z c43 immich new user 201
14:28:41Z c43 immich DONE ok 305 bad 1
14:28:42Z c44 immich new user 201
14:28:42Z c42 paperless DONE ok 53 bad 0
14:28:45Z c43 paperless delete 4 200
14:28:47Z c43 paperless empty trash 200
14:29:39Z c44 immich DONE ok 305 bad 1
14:29:40Z c45 immich new user 201
14:29:49Z c43 paperless DONE ok 53 bad 0
14:29:52Z c44 paperless delete 5 200
14:29:54Z c44 paperless empty trash 200
14:30:55Z c45 immich DONE ok 305 bad 1
14:30:57Z c46 immich new user 201
14:30:58Z c44 paperless DONE ok 53 bad 0
14:31:02Z c45 paperless delete 4 200
14:31:03Z c45 paperless empty trash 200
14:32:02Z c45 paperless DONE ok 53 bad 0
14:32:07Z c46 paperless delete 4 200
14:32:09Z c46 paperless empty trash 200
14:32:09Z c46 immich DONE ok 305 bad 1
14:32:11Z c47 immich new user 201
14:33:13Z c46 paperless DONE ok 53 bad 0
14:33:15Z c47 paperless delete 6 200
14:33:17Z c47 paperless empty trash 200
14:33:29Z c47 immich DONE ok 305 bad 1
14:33:31Z c48 immich new user 201
14:34:18Z c47 paperless DONE ok 53 bad 0
14:34:21Z c48 paperless delete 4 200
14:34:22Z c48 paperless empty trash 200
14:34:43Z c48 immich DONE ok 305 bad 1
14:34:45Z c49 immich new user 201
14:35:34Z c48 paperless DONE ok 53 bad 0
14:35:38Z c49 paperless delete 5 200
14:35:40Z c49 paperless empty trash 200
14:35:48Z c49 immich DONE ok 305 bad 1
14:35:51Z c50 immich new user 201
14:36:50Z c49 paperless DONE ok 53 bad 0
14:36:56Z c50 paperless delete 4 200
14:36:58Z c50 paperless empty trash 200
14:37:01Z c50 immich DONE ok 305 bad 1
14:37:03Z c51 immich new user 201
14:38:01Z c51 immich DONE ok 305 bad 1
14:38:02Z c50 paperless DONE ok 53 bad 0
14:38:03Z c52 immich new user 201
14:38:08Z c51 paperless delete 5 200
14:38:09Z c51 paperless empty trash 200
14:39:11Z c51 paperless DONE ok 53 bad 0
14:39:12Z c52 paperless delete 5 200
14:39:13Z c52 immich DONE ok 305 bad 1
14:39:13Z c52 paperless empty trash 200
14:39:14Z c53 immich new user 201
14:40:14Z c53 immich DONE ok 305 bad 1
14:40:16Z c54 immich new user 201
14:40:17Z c52 paperless DONE ok 53 bad 0
14:40:19Z c53 paperless delete 5 200
14:40:21Z c53 paperless empty trash 200
14:41:15Z c53 paperless DONE ok 53 bad 0
14:41:19Z c54 paperless delete 3 200
14:41:20Z c54 paperless empty trash 200
14:41:28Z c54 immich DONE ok 305 bad 1
14:41:30Z c55 immich new user 201
14:42:23Z c54 paperless DONE ok 53 bad 0
14:42:28Z c55 paperless delete 5 200
14:42:29Z c55 paperless empty trash 200
14:42:38Z c55 immich DONE ok 305 bad 1
14:42:39Z c56 immich new user 201
14:43:31Z c55 paperless DONE ok 53 bad 0
14:43:34Z c56 paperless delete 5 200
14:43:36Z c56 paperless empty trash 200
14:43:37Z c56 immich DONE ok 305 bad 1
14:43:40Z c57 immich new user 201
14:44:36Z c56 paperless DONE ok 53 bad 0
14:44:38Z c57 paperless delete 5 200
14:44:39Z c57 paperless empty trash 200
14:44:50Z c57 immich DONE ok 305 bad 1
14:44:51Z c58 immich new user 201
14:45:44Z c57 paperless DONE ok 53 bad 0
14:45:47Z c58 immich DONE ok 305 bad 1
14:45:48Z c58 paperless delete 5 200
14:45:49Z c59 immich new user 201
14:45:49Z c58 paperless empty trash 200
14:46:49Z c58 paperless DONE ok 53 bad 0
14:46:53Z c59 paperless delete 4 200
14:46:57Z c59 paperless empty trash 200
14:46:59Z c59 immich DONE ok 305 bad 1
14:47:01Z c60 immich new user 201
14:47:58Z c59 paperless DONE ok 53 bad 0
14:47:59Z c60 paperless delete 5 200
14:48:00Z c60 paperless empty trash 200
14:48:01Z c60 immich DONE ok 305 bad 1
14:48:02Z c61 immich new user 201
14:48:57Z c60 paperless DONE ok 53 bad 0
14:49:01Z c61 paperless delete 3 200
14:49:05Z c61 paperless empty trash 200
14:49:06Z c61 immich DONE ok 305 bad 1
14:49:09Z c62 immich new user 201
14:50:07Z c61 paperless DONE ok 53 bad 0
14:50:10Z c62 paperless delete 4 200
14:50:11Z c62 paperless empty trash 200
14:50:15Z c62 immich DONE ok 305 bad 1
14:50:17Z c63 immich new user 201
14:51:13Z c62 paperless DONE ok 53 bad 0
14:51:16Z c63 paperless delete 4 200
14:51:17Z c63 paperless empty trash 200
14:51:27Z c63 immich DONE ok 305 bad 1
14:51:30Z c64 immich new user 201
14:52:18Z c63 paperless DONE ok 53 bad 0
14:52:20Z c64 paperless delete 5 200
14:52:21Z c64 paperless empty trash 200
14:52:45Z c64 immich DONE ok 305 bad 1
14:52:47Z c65 immich new user 201
14:53:19Z c64 paperless DONE ok 53 bad 0
14:53:23Z c65 paperless delete 4 200
14:53:25Z c65 paperless empty trash 200
14:53:41Z c65 immich DONE ok 305 bad 1
14:53:43Z c66 immich new user 201
14:54:25Z c65 paperless DONE ok 53 bad 0
14:54:27Z c66 paperless delete 4 200
14:54:28Z c66 paperless empty trash 200
14:54:44Z c66 immich DONE ok 305 bad 1
14:54:46Z c67 immich new user 201
14:55:36Z c66 paperless DONE ok 53 bad 0
14:55:40Z c67 paperless delete 5 200
14:55:42Z c67 paperless empty trash 200
14:55:42Z c67 immich DONE ok 305 bad 1
14:55:45Z c68 immich new user 201
14:56:42Z c67 paperless DONE ok 53 bad 0
14:56:44Z c68 paperless delete 5 200
14:56:44Z c68 paperless empty trash 200
14:56:52Z c68 immich DONE ok 305 bad 1
14:56:54Z c69 immich new user 201
14:57:43Z c68 paperless DONE ok 53 bad 0
14:57:46Z c69 paperless delete 4 200
14:57:47Z c69 paperless empty trash 200
14:58:00Z c69 immich DONE ok 305 bad 1
14:58:02Z c70 immich new user 201
14:58:45Z c69 paperless DONE ok 53 bad 0
14:58:47Z c70 paperless delete 4 200
14:58:48Z c70 paperless empty trash 200
14:59:12Z c70 immich DONE ok 305 bad 1
14:59:13Z c71 immich new user 201
14:59:56Z c70 paperless DONE ok 53 bad 0
14:59:58Z c71 paperless delete 5 200
14:59:59Z c71 paperless empty trash 200
15:00:13Z c71 immich DONE ok 305 bad 1
15:00:15Z c72 immich new user 201
15:01:03Z c71 paperless DONE ok 53 bad 0
15:01:07Z c72 paperless delete 5 200
15:01:08Z c72 paperless empty trash 200
15:01:23Z c72 immich DONE ok 305 bad 1
15:01:25Z c73 immich new user 201
15:02:16Z c72 paperless DONE ok 53 bad 0
15:02:18Z c73 paperless delete 4 200
15:02:19Z c73 paperless empty trash 200
15:02:29Z c73 immich DONE ok 305 bad 1
15:02:31Z c74 immich new user 201
15:03:22Z c73 paperless DONE ok 53 bad 0
15:03:25Z c74 paperless delete 3 200
15:03:26Z c74 paperless empty trash 200
15:03:33Z c74 immich DONE ok 305 bad 1
15:03:35Z c75 immich new user 201
15:04:21Z c74 paperless DONE ok 53 bad 0
15:04:25Z c75 paperless delete 4 200
15:04:28Z c75 paperless empty trash 200
15:04:31Z c75 immich DONE ok 305 bad 1
15:04:32Z c76 immich new user 201
15:05:35Z c75 paperless DONE ok 53 bad 0
15:05:37Z c76 immich DONE ok 305 bad 1
15:05:39Z c76 paperless delete 5 200
15:05:39Z c77 immich new user 201
15:05:40Z c76 paperless empty trash 200
15:06:43Z c76 paperless DONE ok 53 bad 0
15:06:46Z c77 paperless delete 5 200
15:06:47Z c77 paperless empty trash 200
15:06:53Z c77 immich DONE ok 305 bad 1
15:06:55Z c78 immich new user 201
15:07:49Z c77 paperless DONE ok 53 bad 0
15:07:50Z c78 paperless delete 4 200
15:07:51Z c78 paperless empty trash 200
15:07:58Z c78 immich DONE ok 305 bad 1
15:08:00Z c79 immich new user 201
15:08:54Z c78 paperless DONE ok 53 bad 0
15:08:58Z c79 paperless delete 4 200
15:08:59Z c79 paperless empty trash 200
15:09:09Z c79 immich DONE ok 305 bad 1
15:09:11Z c80 immich new user 201
15:10:00Z c79 paperless DONE ok 53 bad 0
15:10:03Z c80 paperless delete 5 200
15:10:06Z c80 paperless empty trash 200
15:10:15Z c80 immich DONE ok 305 bad 1
15:10:16Z c81 immich new user 201
15:11:08Z c80 paperless DONE ok 53 bad 0
15:11:11Z c81 paperless delete 5 200
15:11:15Z c81 paperless empty trash 200
15:11:15Z c81 immich DONE ok 305 bad 1
15:11:18Z c82 immich new user 201
15:12:18Z c81 paperless DONE ok 53 bad 0
15:12:20Z c82 paperless delete 5 200
15:12:21Z c82 paperless empty trash 200
15:12:27Z c82 immich DONE ok 305 bad 1
15:12:28Z c83 immich new user 201
15:13:19Z c82 paperless DONE ok 53 bad 0
15:13:21Z c83 paperless delete 4 200
15:13:22Z c83 paperless empty trash 200
15:13:35Z c83 immich DONE ok 305 bad 1
15:13:37Z c84 immich new user 201
15:14:20Z c83 paperless DONE ok 53 bad 0
15:14:22Z c84 paperless delete 4 200
15:14:23Z c84 paperless empty trash 200
15:14:41Z c84 immich DONE ok 305 bad 1
15:14:42Z c85 immich new user 201
15:15:25Z c84 paperless DONE ok 53 bad 0
15:15:28Z c85 paperless delete 4 200
15:15:29Z c85 paperless empty trash 200
15:15:44Z c85 immich DONE ok 305 bad 1
15:15:46Z c86 immich new user 201
15:16:31Z c85 paperless DONE ok 53 bad 0
15:16:34Z c86 paperless delete 3 200
15:16:36Z c86 paperless empty trash 200
15:16:51Z c86 immich DONE ok 305 bad 1
15:16:54Z c87 immich new user 201
15:17:38Z c86 paperless DONE ok 53 bad 0
15:17:41Z c87 paperless delete 4 200
15:17:42Z c87 paperless empty trash 200
15:18:01Z c87 immich DONE ok 305 bad 1
15:18:02Z c88 immich new user 201
15:18:42Z c87 paperless DONE ok 53 bad 0
15:18:45Z c88 paperless delete 4 200
15:18:46Z c88 paperless empty trash 200
15:19:01Z c88 immich DONE ok 305 bad 1
15:19:03Z c89 immich new user 201
15:19:54Z c88 paperless DONE ok 53 bad 0
15:19:57Z c89 paperless delete 5 200
15:19:59Z c89 paperless empty trash 200
15:20:04Z c89 immich DONE ok 305 bad 1
15:20:06Z c90 immich new user 201
15:20:59Z c89 paperless DONE ok 53 bad 0
15:21:02Z c90 paperless delete 4 200
15:21:04Z c90 paperless empty trash 200
15:21:05Z c90 immich DONE ok 305 bad 1
15:21:08Z c91 immich new user 201
15:22:09Z c90 paperless DONE ok 53 bad 0
15:22:12Z c91 paperless delete 5 200
15:22:12Z c91 immich DONE ok 305 bad 1
15:22:13Z c91 paperless empty trash 200
15:22:14Z c92 immich new user 201
15:23:10Z c91 paperless DONE ok 53 bad 0
15:23:14Z c92 paperless delete 4 200
15:23:15Z c92 immich DONE ok 305 bad 1
15:23:15Z c92 paperless empty trash 200
15:23:17Z c93 immich new user 201
15:24:14Z c93 immich DONE ok 305 bad 1
15:24:15Z c92 paperless DONE ok 53 bad 0
15:24:16Z c94 immich new user 201
15:24:19Z c93 paperless delete 4 200
15:24:20Z c93 paperless empty trash 200
15:25:16Z c94 immich DONE ok 305 bad 1
15:25:18Z c95 immich new user 201
15:25:20Z c93 paperless DONE ok 53 bad 0
15:25:23Z c94 paperless delete 4 200
15:25:24Z c94 paperless empty trash 200
15:26:17Z c95 immich DONE ok 305 bad 1
15:26:20Z c96 immich new user 201
15:26:29Z c94 paperless DONE ok 53 bad 0
15:26:33Z c95 paperless delete 5 200
15:26:34Z c95 paperless empty trash 200
15:27:35Z c95 paperless DONE ok 53 bad 0
15:27:38Z c96 paperless delete 4 200
15:27:40Z c96 paperless empty trash 200
15:27:40Z c96 immich DONE ok 305 bad 1
15:27:41Z c97 immich new user 201
15:28:39Z c97 immich DONE ok 305 bad 1
15:28:40Z c98 immich new user 201
15:28:43Z c96 paperless DONE ok 53 bad 0
15:28:47Z c97 paperless delete 4 200
15:28:48Z c97 paperless empty trash 200
15:29:49Z c97 paperless DONE ok 53 bad 0
15:29:49Z c98 immich DONE ok 305 bad 1
15:29:52Z c99 immich new user 201
15:29:53Z c98 paperless delete 3 200
15:29:54Z c98 paperless empty trash 200
15:30:56Z c99 immich DONE ok 305 bad 1
15:30:58Z c100 immich new user 201
15:31:02Z c98 paperless DONE ok 53 bad 0
15:31:06Z c99 paperless delete 4 200
15:31:08Z c99 paperless empty trash 200
15:32:03Z c100 immich DONE ok 305 bad 1
15:32:05Z c101 immich new user 201
15:32:12Z c99 paperless DONE ok 53 bad 0
15:32:16Z c100 paperless delete 4 200
15:32:17Z c100 paperless empty trash 200
15:33:06Z c101 immich DONE ok 305 bad 1
15:33:07Z c102 immich new user 201
15:33:12Z c100 paperless DONE ok 53 bad 0
15:33:14Z c101 paperless delete 4 200
15:33:15Z c101 paperless empty trash 200
15:34:10Z c101 paperless DONE ok 53 bad 0
15:34:16Z c102 paperless delete 4 200
15:34:17Z c102 immich DONE ok 305 bad 1
15:34:17Z c102 paperless empty trash 200
15:34:19Z c103 immich new user 201
15:35:18Z c102 paperless DONE ok 53 bad 0
15:35:23Z c103 paperless delete 4 200
15:35:24Z c103 paperless empty trash 200
15:35:38Z c103 immich DONE ok 305 bad 1
15:35:40Z c104 immich new user 201
15:36:27Z c103 paperless DONE ok 53 bad 0
15:36:30Z c104 paperless delete 4 200
15:36:31Z c104 paperless empty trash 200
15:37:03Z c104 immich DONE ok 305 bad 1
15:37:05Z c105 immich new user 201
15:37:41Z c104 paperless DONE ok 53 bad 0
15:37:45Z c105 paperless delete 5 200
15:37:45Z c105 paperless empty trash 200
15:38:10Z c105 immich DONE ok 305 bad 1
15:38:11Z c106 immich new user 201
15:38:46Z c105 paperless DONE ok 53 bad 0
15:38:49Z c106 paperless delete 4 200
15:38:50Z c106 paperless empty trash 200
15:39:14Z c106 immich DONE ok 305 bad 1
15:39:15Z c107 immich new user 201
15:39:47Z c106 paperless DONE ok 53 bad 0
15:39:51Z c107 paperless delete 4 200
15:39:52Z c107 paperless empty trash 200
15:40:25Z c107 immich DONE ok 305 bad 1
15:40:26Z c108 immich new user 201
15:40:53Z c107 paperless DONE ok 53 bad 0
15:40:57Z c108 paperless delete 5 200
15:40:58Z c108 paperless empty trash 200
15:41:28Z c108 immich DONE ok 305 bad 1
15:41:30Z c109 immich new user 201
15:42:01Z c108 paperless DONE ok 53 bad 0
15:42:05Z c109 paperless delete 4 200
15:42:06Z c109 paperless empty trash 200
15:42:32Z c109 immich DONE ok 305 bad 1
15:42:34Z c110 immich new user 201
15:43:08Z c109 paperless DONE ok 53 bad 0
15:43:10Z c110 paperless delete 4 200
15:43:11Z c110 paperless empty trash 200
15:43:35Z c110 immich DONE ok 305 bad 1
15:43:38Z c111 immich new user 201
15:44:15Z c110 paperless DONE ok 53 bad 0
15:44:15Z paperless cycles 110
15:44:38Z c111 immich DONE ok 305 bad 1
15:44:38Z immich cycles 111
Binary file not shown.

After

Width:  |  Height:  |  Size: 2.5 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.7 KiB

@@ -0,0 +1,33 @@
control NEGATIVE (a stub answering 200 over plain http): convicted, as it must (200 -)
== tester-1 (enkicsifelhom.hu): 6 public name(s)
OK control POSITIVE http://felhom.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://felhom.enkicsifelhom.hu/r1033-check?q=1
OK http://files.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://files.enkicsifelhom.hu/r1033-check?q=1
OK http://no-such-name-r1033.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://no-such-name-r1033.enkicsifelhom.hu/r1033-check?q=1
OK http://paperless.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://paperless.enkicsifelhom.hu/r1033-check?q=1
OK http://paste.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://paste.enkicsifelhom.hu/r1033-check?q=1
OK http://wiki.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://wiki.enkicsifelhom.hu/r1033-check?q=1
== demo-hp (enkisfelhom.hu): 14 public name(s)
OK http://arcade.enkisfelhom.hu/r1033-check?q=1 -> 301 https://arcade.enkisfelhom.hu/r1033-check?q=1
OK http://books.enkisfelhom.hu/r1033-check?q=1 -> 301 https://books.enkisfelhom.hu/r1033-check?q=1
OK http://docs.enkisfelhom.hu/r1033-check?q=1 -> 301 https://docs.enkisfelhom.hu/r1033-check?q=1
OK http://felhom.enkisfelhom.hu/r1033-check?q=1 -> 301 https://felhom.enkisfelhom.hu/r1033-check?q=1
OK http://files.enkisfelhom.hu/r1033-check?q=1 -> 301 https://files.enkisfelhom.hu/r1033-check?q=1
OK http://gist.enkisfelhom.hu/r1033-check?q=1 -> 301 https://gist.enkisfelhom.hu/r1033-check?q=1
OK http://ha.enkisfelhom.hu/r1033-check?q=1 -> 301 https://ha.enkisfelhom.hu/r1033-check?q=1
OK http://no-such-name-r1033.enkisfelhom.hu/r1033-check?q=1 -> 301 https://no-such-name-r1033.enkisfelhom.hu/r1033-check?q=1
OK http://papir.enkisfelhom.hu/r1033-check?q=1 -> 301 https://papir.enkisfelhom.hu/r1033-check?q=1
OK http://paste.enkisfelhom.hu/r1033-check?q=1 -> 301 https://paste.enkisfelhom.hu/r1033-check?q=1
OK http://pdf.enkisfelhom.hu/r1033-check?q=1 -> 301 https://pdf.enkisfelhom.hu/r1033-check?q=1
OK http://plex.enkisfelhom.hu/r1033-check?q=1 -> 301 https://plex.enkisfelhom.hu/r1033-check?q=1
OK http://travel.enkisfelhom.hu/r1033-check?q=1 -> 301 https://travel.enkisfelhom.hu/r1033-check?q=1
OK http://wiki.enkisfelhom.hu/r1033-check?q=1 -> 301 https://wiki.enkisfelhom.hu/r1033-check?q=1
== demo-felhom (demo-felhom.eu): 8 public name(s)
OK http://felhom.demo-felhom.eu/r1033-check?q=1 -> 301 https://felhom.demo-felhom.eu/r1033-check?q=1
OK http://files.demo-felhom.eu/r1033-check?q=1 -> 301 https://files.demo-felhom.eu/r1033-check?q=1
OK http://gist.demo-felhom.eu/r1033-check?q=1 -> 301 https://gist.demo-felhom.eu/r1033-check?q=1
OK http://no-such-name-r1033.demo-felhom.eu/r1033-check?q=1 -> 301 https://no-such-name-r1033.demo-felhom.eu/r1033-check?q=1
OK http://prowlarr.demo-felhom.eu/r1033-check?q=1 -> 301 https://prowlarr.demo-felhom.eu/r1033-check?q=1
OK http://radarr.demo-felhom.eu/r1033-check?q=1 -> 301 https://radarr.demo-felhom.eu/r1033-check?q=1
OK http://sonarr.demo-felhom.eu/r1033-check?q=1 -> 301 https://sonarr.demo-felhom.eu/r1033-check?q=1
OK http://torrent.demo-felhom.eu/r1033-check?q=1 -> 301 https://torrent.demo-felhom.eu/r1033-check?q=1
== every public name redirects http -> https
@@ -0,0 +1,3 @@
10 0 gitea.dooplex.hu/admin/felhom-controller:0.337.0 Up 6 minutes (healthy) ← root@192.168.0.154 (containers, unhealthy/restarting, controller)
24 0 gitea.dooplex.hu/admin/felhom-controller:0.337.0 Up 6 minutes (healthy) ← hp (containers, unhealthy/restarting, controller)
9 0 gitea.dooplex.hu/admin/felhom-controller:0.337.0 Up 7 minutes (healthy) ← felhom-pve (containers, unhealthy/restarting, controller)
@@ -0,0 +1,6 @@
errors=0
← root@192.168.0.154
errors=0
← hp
errors=0
← felhom-pve
@@ -0,0 +1,9 @@
purging VM 9321 from related configurations..
purging CT 9331 from related configurations..
0
0
341 night1004-tester1 running
9201 running demo-hp
9202 running demo-hp-scratch
9401 stopped upgrade-harness
demo-hp teardown 16:27:40
@@ -0,0 +1,5 @@
live Tester4 hosts: []
deletions: [('Tester4-f6b67c', '2026-10-21 16:49:37', 1), ('Tester4-648b80', '2026-10-21 10:49:13', 0), ('Tester4-816e21', '2026-10-21 09:11:21', 1)]
retained under f6b67c: [(18, 1, '5501e4bf2cda', 'live')]
current escrow rows for Tester4 hosts: []
wg peer left: []
+16
View File
@@ -0,0 +1,16 @@
#!/bin/bash
# bind_curl.sh — the household's bind page, endpoint-level (no browser on DooPlex): GET the page, then the same POST its
# form sends. Env (from `sec run`, names passed as args): TOKEN_ENV PAIR_ENV PASS_ENV. Secrets go through 0600 files,
# never a command line; prints status lines and the page's visible text only.
set -euo pipefail
TOK=${!1}; T=$(mktemp -d); chmod 700 "$T"; trap 'rm -rf "$T"' EXIT
printf '%s' "${!2}" > "$T/pc"; printf '%s' "${!3}" > "$T/pp"
U="https://hub.felhom.eu/bind/$TOK"
curl -s -o "$T/g" -w "GET %{http_code}\n" "$U"
curl -s -o "$T/p" -w "POST %{http_code}\n" --data-urlencode "pairing_code@$T/pc" --data-urlencode "passphrase@$T/pp" "$U"
python3 - "$T/p" <<'PY'
import re,sys,html
t=open(sys.argv[1],encoding="utf-8").read()
t=re.sub(r"<(script|style)[^>]*>.*?</\1>","",t,flags=re.S)
print("answer:", re.sub(r"\s+"," ",html.unescape(re.sub(r"<[^>]+>"," ",t))).strip()[:400])
PY
@@ -0,0 +1,91 @@
#!/usr/bin/env python3
"""ctl.py — a minimal headless client for a Felhom controller dashboard (rehearsal 2026-10-20).
The same requests the dashboard's pages send (endpoint-level validation): POST /login, then JSON calls with the
session cookie and the X-CSRF-Token header. Configuration by env: CTL_BASE (https://<ip>), CTL_DOMAIN (Host:
felhom.<domain>), CTL_PW_ENV (the NAME of the env var holding the dashboard password — run under `sec run`).
Session files live in a 0700 temp dir per process; nothing secret is printed.
usage as a CLI: ctl.py GET /api/stacks
ctl.py POST /api/stacks/x/stop '{"a":1}'
"""
import json, os, re, subprocess, sys, tempfile
BASE = os.environ["CTL_BASE"]
HOST = "Host: felhom." + os.environ["CTL_DOMAIN"]
_T = tempfile.mkdtemp(prefix="ctl-")
os.chmod(_T, 0o700)
_SESS = _CSRF = None
def _curl(args, inp=None, timeout=600):
r = subprocess.run(["curl", "-sk", "-H", HOST] + args, capture_output=True, text=True, timeout=timeout, input=inp)
return r.stdout
def login():
global _SESS, _CSRF
pw = os.environ[os.environ["CTL_PW_ENV"]]
hdr = os.path.join(_T, "h")
# the password goes on stdin (curl --data-urlencode password@-), never on a command line
_curl(["-D", hdr, "-o", "/dev/null", "-X", "POST", "--data-urlencode", "password@-", BASE + "/login"], inp=pw)
m = re.search(r"felhom_session=[A-Za-z0-9._-]+", open(hdr).read(), re.I)
if not m:
sys.exit("ctl: login failed (no session cookie)")
_SESS = m.group(0)
body = _curl(["-L", "-H", "Cookie: " + _SESS, BASE + "/backups"]) # "/" now lands on /launcher, which has no csrf meta
c = re.search(r'<meta name="csrf-token" content="([^"]+)"', body)
if not c:
sys.exit("ctl: login failed (no csrf token)")
_CSRF = c.group(1)
def call(method, path, data=None, raw=False, timeout=600):
if _SESS is None:
login()
for attempt in (0, 1):
args = ["-H", "Cookie: " + _SESS, "-w", "\n%{http_code}"]
if method != "GET":
args += ["-H", "X-CSRF-Token: " + _CSRF, "-H", "Content-Type: application/json", "-X", method]
if data is not None:
args += ["--data-binary", "@-"] # the body on stdin, never on a command line (it may hold a password)
out = _curl(args + [BASE + path], inp=json.dumps(data) if data is not None and method != "GET" else None, timeout=timeout)
body, _, code = out.rpartition("\n")
if code in ("302", "401") and attempt == 0:
login()
continue
if raw:
return code, body
try:
return code, json.loads(body)
except ValueError:
return code, {"_raw": body[:800]}
def page(path):
if _SESS is None:
login()
return _curl(["-H", "Cookie: " + _SESS, BASE + path])
if __name__ == "__main__":
m, p = sys.argv[1], sys.argv[2]
d = json.loads(sys.argv[3]) if len(sys.argv) > 3 else None
code, body = call(m, p, d)
print(code)
print(json.dumps(body, ensure_ascii=False, indent=1)[:6000])
def form(path, fields, timeout=600):
"""A form POST (application/x-www-form-urlencoded) with the session and the X-CSRF-Token header."""
import urllib.parse
if _SESS is None:
login()
out = _curl(["-H", "Cookie: " + _SESS, "-H", "X-CSRF-Token: " + _CSRF, "-w", "\n%{http_code}", "-X", "POST",
"--data-binary", "@-", "-H", "Content-Type: application/x-www-form-urlencoded", BASE + path],
inp=urllib.parse.urlencode(fields), timeout=timeout)
body, _, code = out.rpartition("\n")
try:
return code, json.loads(body)
except ValueError:
return code, {"_raw": body[:800]}

Some files were not shown because too many files have changed in this diff Show More