Compare commits
9 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 87dc50743c | |||
| 4b9b0af3cb | |||
| 2c9152d912 | |||
| 52af38df6f | |||
| 34979d5d25 | |||
| 0bb57a4fee | |||
| 2aebf1bde1 | |||
| d306b8c2d7 | |||
| be7e97dc1a |
@@ -16,6 +16,31 @@
|
||||
> and holds nothing of its own; this file does hold its own content, namely the standing rulings below.
|
||||
|
||||
|
||||
> **2026-10-23 ~12:00 — operator rulings by starting the D39 brief (`09` §3 decision 255; R-1239 closed).** **D39 = B:** a
|
||||
> `security_fix` ladder step — box walk on 9202 required, bench optional, taken by the night leg. **No operator word per
|
||||
> incident:** a public advisory id (CVE or GHSA) + its https:// notice is the authority; the session that writes the step
|
||||
> reports it to the operator after the push. The design (Claude.ai's recommendation) was taken as written; one fact in the
|
||||
> brief was checked and held (Plex's box fixture exists but is a `NoRoute` stub, so `seed: "none"` is right). Built:
|
||||
> catalog `3898593` (`ladder.py` `check_security` + `engine_majors_crossed` importing `check-engine-major.py`; the move gate
|
||||
> accepts it; `check-test-record.py` WARNs on a head with no bench; `upgrade-test.py --write-ladder --security`;
|
||||
> `box_walk.py --security` + `refresh_box_mirror()`), controller **0.341.0** (`legCandidate` takes `proven` or
|
||||
> `security_fix`; global floor 11:24 UTC with MinAgent 0.158.1, the operator's yes in chat). Proof:
|
||||
> `audits/security-lane-2026-10-23/`. Older controllers skip the step at night and take it on a hand press (proven).
|
||||
|
||||
> **2026-10-23 10:56 — operator ruling: Plex to the fixed version now, without the test record (Plex2Shell).**
|
||||
> Verbatim: *„bump the version to the newest in the app template immediately (as there are no customers yet), I want the
|
||||
> demo boxes on the newest version asap."* It skips `09` §3 decision 13's test record **once, for Plex only**. Done: catalog
|
||||
> `6cc317d` (Plex 1.41.4.9463-630c9f557 → 1.43.5.11029-6bf8af3f0, CVE-2026-96652/-96654/-96655/-96656), pushed
|
||||
> `--no-verify` past `check-test-record-move.py` (its refusal is in `app-catalog-felhom.eu/REPORT.md`); the only box running
|
||||
> Plex (demo-hp 9201) was checked for the attack's signs (none) and updated by the Update press; the guide re-checked
|
||||
> (`guide.checked_version` moved). Decision 13 itself is unchanged. Follow-ups: R-1238 (Plex's test record and fixture),
|
||||
> R-1239 (a fast lane for security fixes — the operator's decision, D39). Evidence: `audits/plex-demo-boxes-2026-10-23/`.
|
||||
|
||||
> **2026-10-23 morning — the operator's „go" for the day brief: the releases, D31 as built, D32–D36 (decide rows in `STATUS-HISTORY.md`).**
|
||||
> **Releases:** controller 0.340.0 (global floor 06:41 UTC, MinAgent 0.158.1); agent 0.163.0 (binary sha256 561b5316…, bundle 3afb88d9…, vouched by the operator 07:58 UTC; delivered to the three test boxes by signed jobs through a step bundle `0.163.0-step1` — the D4 drop-in is a new path, R-880); installer 1.36.0 (tag `installer-v1.36.0` → 83388c08, both webpage pins); hub 0.167.0 (sha256:177860a9…, 08:27 UTC); the website `0bb57a4f`.
|
||||
> **D31** = as built (the re-issue reads the newest deletion of a KEY-HOLDING host; an install that got the password but no recovery package is rotated without a press). **D32** = yes: the sheet is names and places only, one A4, HU + EN; the operator prints and fills it. **D33** = yes: `scripts/release-copy/` nightly 01:40, newest 3 image tags (controller, hub, samba) + 5 agent versions, skopeo `dir:` (digest-true — why not `docker save`: a re-push re-compresses), ep0 `/srv/felhom-release-copy` under the one limited user `felhom-relcopy` (rrsync), cap 3 GB enforced by the sender, a read-only registry token of its own. **D34** = yes: hub snapshots at 02/08/14/20 (catch-up > 6 h), DooPlex push :30 after, limit 7 h + a wait for a snapshot being written (R-1234: 2 h was wrong), alarm 13 h; ep0 retention unchanged (keep-daily 14, keep-weekly 8). **D35** = yes, after the release week (R-1228, dated 2026-10-30). **D36** = yes, both sentences, published.
|
||||
> **Added on the operator's word the same day (not soaked):** D34's hub half went into 0.167.0 (asked before the build). Record: `audits/day-2026-10-23/`, `REPORT-day-2026-10-23.md`.
|
||||
|
||||
> **2026-10-22 evening — operator rulings D29, D30, D31 (CC's picks accepted; the decide rows now in `STATUS-HISTORY.md`).**
|
||||
> **D29** = no: the controller repository stays public for now (R-925). **D30** = the agent makes a new WireGuard key and
|
||||
> registers it on the existing peer path (R-1204). **D31** = the hub skips a deleted install that never made a key when it
|
||||
|
||||
@@ -0,0 +1,103 @@
|
||||
# REPORT — day 2026-10-23: the releases and the DooPlex hour (the operator's „go")
|
||||
|
||||
Design read: `07-backup-architecture.md` §6.3/§8, `09-update-architecture.md` §3 decision 250, `03-host-agent.md`,
|
||||
`05-hub-architecture.md` §16.3, `runbooks/total-loss-of-dooplex.md`, `audits/dooplex-survival-2026-10-22/PLAN.md`.
|
||||
Evidence: `documentation/audits/day-2026-10-23/01`–`14`.
|
||||
|
||||
## Five lines
|
||||
|
||||
1. Released: controller 0.340.0 (global floor), agent 0.163.0 (vouched by the operator), installer 1.36.0, hub 0.167.0,
|
||||
the website with D36's two sentences.
|
||||
2. Root SSH from the LAN is key-only on all three test boxes; new installs get it from installer 1.36.0.
|
||||
3. The hub database now leaves every 6 hours (restored back byte-identical); the released images and agent packages
|
||||
leave nightly to ep0 (pulled back, every sha256 OK, one image loaded on a scratch box).
|
||||
4. One false failure mail at 10:28 (my install of the 6-hourly copy) — filed R-1234, fixed and red-proved the same hour.
|
||||
5. Register 191 → 184; the operator: print the sheet, Hosts page clean-up, D37 + D38.
|
||||
|
||||
## Per box — versions before → after, and the SSH checks
|
||||
|
||||
| Box | controller | agent | root-file bundle | `sshd -T` root | sshd reloaded | key login | felhom-sshd | swap = RAM (9201) |
|
||||
|---|---|---|---|---|---|---|---|---|
|
||||
| Tester 1 | 0.339.0 → 0.340.0 | 0.162.0 → 0.163.0 | 0.160.0 → 0.163.0 (via 0.163.0-step1) | without-password | yes, 09:27:38 | ok | active, `sshd -t` ok | 4096/4096 |
|
||||
| demo-hp | 0.339.0 → 0.340.0 | 0.162.0 → 0.163.0 | 0.160.0 → 0.163.0 | without-password | yes, 09:19:44 | ok | active, `sshd -t` ok | 25898/25898 |
|
||||
| demo-felhom | 0.339.0 → 0.340.0 | 0.162.0 → 0.163.0 | 0.160.0 → 0.163.0 | without-password | yes, 09:24:42 | ok | active, `sshd -t` ok | 12288/12288 |
|
||||
| hub | 0.165.0 → 0.167.0 | | | | | | | |
|
||||
|
||||
Not proven (said plainly): the real root password over SSH (CC does not hold it — a dummy one is refused, but OpenSSH logs
|
||||
only „Failed password"); a felhom-sshd connection (it admits only the operator's tunnel peer); the physical console (the
|
||||
drop-in does not touch it; `passwd -S root` = P).
|
||||
|
||||
## Digests, sha256, CI by commit
|
||||
|
||||
| What | Value |
|
||||
|---|---|
|
||||
| controller 0.340.0 | `sha256:a8d1c6458da4ea9d667f928498656a291771e0b3756b20f2fcc8c1cf9d516621` — `caaa86e`, CI 2245 success |
|
||||
| agent 0.163.0 binary / bundle | `561b5316cd511de9fa9d8b8be4553da7add684f715d3bea533ca330ff7f3763c` / `3afb88d9fbbc4adc16083fa52a98cd4974583c6378a77a1a6e0ed8ae6d1b80c8` — `1109092` CI 2246, `b4721ee` CI 2249 success |
|
||||
| step bundle 0.163.0-step1 | `6cbc066902d5f3294765d5c7dc7113166c1dca25e994dbc36bfc5f139bb61fcf` |
|
||||
| installer 1.36.0 (served = tag) | `de8bf5f32cbcda44c9976ba829f04ea151e3b30634b54d5fdd9eb6c4a0351cec` — `83388c08` CI 2247 success; pins `2aebf1bd` |
|
||||
| hub 0.167.0 | `sha256:177860a996cb285d935777d6ffdf85725f6cf4489885f54fd3bf8135c68041a0` — built at `d306b8c2` (hub/ = `be7e97dc`, CI 2250 success; `d306b8c2` CI 2251 success); manifest `34979d5d` |
|
||||
| website | `0bb57a4f` |
|
||||
| D34 fix (R-1234) | `52af38df` |
|
||||
| prometheus rules | homelab-manifests `1bbef892` |
|
||||
|
||||
CI of the last commits (`2aebf1bd`, `0bb57a4f`, `34979d5d`, `52af38df` and this close-out): see the final message.
|
||||
|
||||
## Parts
|
||||
|
||||
- **1 — controller 0.340.0:** 9202 smoke, per-customer floors, three boxes green, box walk + https-only rc 0, global floor
|
||||
06:41 UTC (MinAgent 0.158.1), read back with the WAL (`01`–`06`).
|
||||
- **2 — agent + installer:** release, a step bundle because the D4 drop-in is a new path (R-880), three signed jobs per
|
||||
box (`07`); the operator's vouch read back from the DB and the live page (`10`); installer tag, both pins, a
|
||||
webpage-only sync, the served script = the tag (`11`).
|
||||
- **3 — hub 0.167.0:** built from a clean clone; on the operator's word it also carries D34's 6-hourly snapshot (asked
|
||||
before the build; not soaked); hub-only sync; 30-minute watch: 0 hub events, 0 errors, all boxes reporting, no hub mail
|
||||
(`14`).
|
||||
- **4 — website:** D3/D5 patch + D36 HU + EN; site gates; published; read back live: new text present, old gone (control:
|
||||
the previous commit), screenshots at 390 and 1366 px, no sideways scroll (`12-website-readback/`).
|
||||
- **5 — D34:** retention unchanged, server-side on ep0 (`prune-operator-hubdb` 03:45, keep-daily 14, keep-weekly 8 — the
|
||||
current day's up to 4 copies until the next prune). Hand run 501.6 MiB / 24.5 MiB uploaded in 4 s; the newest copy
|
||||
restored from ep0 with the read-only token = the pod's snapshot by sha256 `a9ff4c8a…`, integrity ok, 4 hosts, holds
|
||||
today's vouch. Alarm 13 h. Runbook updated (`13`).
|
||||
- **6 — D33:** ep0 before: `/` 14 GB free (the data disk's root IS the PBS datastore — not used). One user
|
||||
`felhom-relcopy` (locked, rrsync-confined: shell and `..` refused), one directory `/srv/felhom-release-copy`. Cap 3 GB,
|
||||
enforced by the sender (ep0 has no quota). **skopeo, not `docker save`:** `dir:` keeps the registry's bytes, so a push
|
||||
back keeps the digest; no Docker daemon. A read-only registry token of its own, minted in the Gitea pod straight into a
|
||||
0600 file. First run 676 MB, 91 s; pulled back: 141 files OK, 9 images re-verified (digests match independent records),
|
||||
controller 0.340.0 loaded on 9202 (ID = config digest). Alarm `FelhomReleaseCopyStale`. Restore steps in
|
||||
`total-loss-of-dooplex.md` (`08`).
|
||||
- **7 — D32:** the sheet is names and places only, HU + EN, one A4 (rendered: 1 page; a copy in the operator's home
|
||||
folder); every place checked to exist. demo-hp: three superseded ISOs removed by name, `/` 8.5 → 14 GB free (`09`).
|
||||
R-548: the box half done, the product fix proposed (R-921's pattern).
|
||||
- **8 — close-out:** CONTEXT rulings, STATUS (decide rows D32–D36 → history, D37/D38 new, the do list, D35 dated
|
||||
2026-10-30), REPORTs and CHANGELOGs; the system poster's facts file updated, its hub-DB label edited (text), and a new
|
||||
arrow asked for in STATUS („System poster needs a refresh").
|
||||
|
||||
## Register — 191 → 184; opened 4, closed 11
|
||||
|
||||
Opened: R-1234 (closed the same hour), R-1235, R-1236, R-1237. Closed: R-49, R-982, R-985, R-991, R-1204, R-1205,
|
||||
R-1225, R-1226, R-1227, R-1230, R-1234. Updated: R-1214 (released; dated check 2026-10-24), R-1187, R-1189, R-232,
|
||||
R-548, R-923, R-924, R-1228 (D35, dated 2026-10-30), R-1183's check text (four snapshot slots).
|
||||
|
||||
## Observations
|
||||
|
||||
- The D34 first push refused the hub's own catch-up and mailed admin@ — FILED: R-1234 (closed, fixed `52af38df`).
|
||||
- The release copy is proven to load, not to restore a registry; it follows the newest versions, not the floor/vouch; no
|
||||
golden — FILED: R-1235.
|
||||
- Gitea OOM-killed at its 1 GiB limit, the last at 06:41:10 UTC during the release — FILED: R-1236 (STATUS D37).
|
||||
- The night sweep's mail: one-time test-box codes in one session record — FILED: R-1237.
|
||||
- The hub DB snapshot is 501 MiB, not 24 MB — NOT-A-FINDING: 24 MB is the compressed upload, which is what ep0 stores.
|
||||
- felhom-sshd does not answer from the LAN — NOT-A-FINDING: its firewall admits only the operator peer (H1, by design).
|
||||
- A dummy root password logs „Failed password", not a policy refusal — NOT-A-FINDING: OpenSSH's behaviour under
|
||||
`prohibit-password`; the policy is proven by `sshd -T` and the reload.
|
||||
- The vouch left `artifact_wrapper_sha256` empty — NOT-A-FINDING: an optional field; the wrapper rides the vouched bundle.
|
||||
- Guests 9202, 9401 (demo-hp) and a transient 990000 (Tester 1) have swap ≠ RAM — NOT-A-FINDING: outside the felhom pool.
|
||||
- `known_hosts` holds an older key for 192.168.0.155 (demo-hp's guest address) — NOT-A-FINDING: guests are reached by
|
||||
`pct exec`; nothing was changed.
|
||||
- The D4 drop-in needed a step bundle — NOT-A-FINDING: the documented R-880 path for a new bundle path.
|
||||
|
||||
## Teardown — three layers
|
||||
|
||||
Machine: 9202 — the test image untagged, the tar removed. Host: demo-hp `/tmp` tar removed; three ISOs removed on purpose
|
||||
(Part 7); DooPlex: the read-back copies and the hub-DB scratch restore shredded/removed. Hub: nothing provisioned; the
|
||||
per-customer floors cleared after the global floor. Left on purpose: the ep0 user + directory, the DooPlex release-copy
|
||||
job, the 6-hourly hub-DB timer, both alarms.
|
||||
@@ -0,0 +1,42 @@
|
||||
# REPORT — Plex to the fixed version in the catalog and on the demo/test boxes (2026-10-23 day, Plex2Shell)
|
||||
|
||||
**Plain words first.**
|
||||
- The catalog now gives Plex 1.43.5.11029-6bf8af3f0 (catalog `6cc317d`, pushed without the update test, on your ruling).
|
||||
- One box ran Plex: demo-hp. It showed no sign of the attack. CC pressed Update; it runs the new Plex now.
|
||||
- You have one decision (D39): may a security fix reach the boxes at night without the full update test?
|
||||
|
||||
## Per box
|
||||
|
||||
| Box | Plex before → after | Signs check | Box walk |
|
||||
|---|---|---|---|
|
||||
| demo-hp 9201 | 1.41.4.9463-630c9f557 → 1.43.5.11029-6bf8af3f0 | clean (NOT-A-FINDING; logs reach back only to 00:32 UTC today — CPU history and files cover 10-16 → 10-22) | green: all containers healthy, 0 errors, 0 warnings since the press |
|
||||
| demo-hp 9202, demo-felhom 9201, tester-1 9201 | no Plex | — | — |
|
||||
| Tester4 | hub host DOWN, VM gone | — | — |
|
||||
| Tester 2 | no host on the hub | — | — |
|
||||
|
||||
Detail and evidence: `documentation/audits/plex-demo-boxes-2026-10-23/README.md`.
|
||||
|
||||
## Records
|
||||
|
||||
- Register 184 → 186: opened R-1238 (P2, Plex has no test record or fixture), R-1239 (P2, a fast lane for security
|
||||
fixes — the operator's decision); closed none.
|
||||
- `CONTEXT.md`: the 10:56 ruling. `STATUS.md`: decide row D39. No „broken" line: Part 2 was finished before the
|
||||
records were written, so the line the brief asked for would have been added and removed in the same commit.
|
||||
The „Updated" paragraph was left as the day session wrote it (it is that session's summary; this run adds one
|
||||
decide row only).
|
||||
- Architecture read: `09-update-architecture.md` §3 decisions 13 and 52; `13-threat-model.md` (Plex row);
|
||||
`runbooks/target-selection.md`. Decision 13 is unchanged; no poster fact changed (none of the three poster actions).
|
||||
- `--no-verify` was used ONCE, for catalog `6cc317d`, on the ruling. The `check-test-record-move.py` refusal is verbatim
|
||||
in `app-catalog-felhom.eu/REPORT.md`.
|
||||
|
||||
## Fixed without a row
|
||||
|
||||
- The memory index (`.claude-memory/MEMORY.md`, outside git) had grown to 201 lines / 25 735 bytes at 11:09 (another
|
||||
session), over the 200 / 25 600 ceiling, so `instructions_gate` failed and so did two script suites that run it.
|
||||
Lines 71–72 (two memories about the deleted credentials file, both superseded by `sec`) became one line: 200 lines,
|
||||
25 590 bytes. `repo_gates.py --fast` then OK. The index sits right at the ceiling; the next added line trips it again.
|
||||
|
||||
## Teardown
|
||||
|
||||
Machine: Plex stays on the new version; helper scripts removed from guest and host. Host: nothing provisioned.
|
||||
Hub: reads only.
|
||||
@@ -0,0 +1,17 @@
|
||||
# REPORT — D39 = B: the security-fix lane (2026-10-23, attended)
|
||||
|
||||
**Plain words first.** An app with a public security hole can now get its fix at night on every box, after one test
|
||||
on the test box. Controller 0.341.0 does it and is the floor since 13:24 (your yes). Older controllers skip such a fix
|
||||
at night and take it when a person presses Update — both seen on the test box. Nothing needs you.
|
||||
|
||||
- Design: `09-update-architecture.md` §3 decision 255 (+ a dated note on decision 13). Capability map B23.
|
||||
- Catalog `3898593` (+ a box-walk address fix after it); controller `63dcccf` = 0.341.0; drill commits `8c1d908` …
|
||||
`17e0da0` (the drill only).
|
||||
- Evidence and the full table: `documentation/audits/security-lane-2026-10-23/README.md`.
|
||||
- Register 186 → 186: closed R-1239 (D39 = B); opened R-1240 (9202's 34 saved configs with the catalog token); R-1238
|
||||
stays open, its text now says the next Plex fix can use the lane.
|
||||
- `STATUS.md`: D39 out of the decide table, one „what works" line, the „Updated" paragraph replaced (the old one is in
|
||||
`STATUS-HISTORY.md`). `CONTEXT.md`: the rulings. System poster: no fact changed (none of the three actions).
|
||||
- Instruction / fact files edited: `app-catalog-felhom.eu/CLAUDE.md` (the lane, added); `operations/nodes.md` line 221
|
||||
(9202 „same dashboard password" → its own password, measured) and line 46 (Tester 1 `.101` → DHCP, `.148` today).
|
||||
- No `--no-verify` in this run.
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
**Ready for Tester 2? Yes — tell the friend go (hand-over list: `documentation/pilot/TESTER2-handover.md`).**
|
||||
|
||||
**Updated 2026-10-23 (morning).** Your three picks are built (the tunnel repairs itself with a new key, the hub skips a keyless deleted install, the controller repository stays public), and the night copy can no longer outlive the controller. Four fresh reviews found 34 points, every real one fixed or a row; the test night PASSED (no failed check at night, every night job passed, nothing lost). Nothing was released or deployed; the morning list has the order, and the DooPlex survey asks four questions (D32–D35).
|
||||
**Updated 2026-10-23 (afternoon).** Your D39 = B is built and live: an app with a public security hole can now get its fix at night on every box, after one test on the test box — controller 0.341.0 is the floor since 13:24 (your yes). The test: Plex moved by itself at night on the test box, and an older controller safely waited for a hand press. Nothing needs you.
|
||||
|
||||
## What works
|
||||
- A box installs from nothing with the public installer, links to the hub, and moves to the floor by itself (map A1, A3, A4).
|
||||
@@ -12,6 +12,7 @@
|
||||
- After a power cut the box starts by itself, cleans up a half-written backup, and Immich repairs broken photos (map C10, F12).
|
||||
- A second drive is offered as the backup target; media apps see every drive; a one-disk box gets its own data area (map D1, D13, D15).
|
||||
- The three test boxes answer only over https; the hub's key opens nothing on a box; password guesses are counted (map E12, E13, E17).
|
||||
- An urgent security fix reaches the boxes at night after one test on the test box; the box backs up first and undoes a failed update (map B23).
|
||||
- The hub mails alarms and recoveries, and keeps one whole-box copy per week on ep0; CC uses its own limited token, not your password (map F1, F11, G15).
|
||||
|
||||
## What is broken
|
||||
@@ -21,7 +22,7 @@
|
||||
- The scratch box cannot use agent features (R-1083).
|
||||
|
||||
## What we work on
|
||||
- Dated checks, run by CC. 2026-10-23: no app stop when the host is busy (R-1173). 2026-10-26: each hub daily job ran once on the clock-change day (R-1183). 2026-10-27: the weekly clean-up window (R-435; the logs of the last one were lost to restarts).
|
||||
- Dated checks, run by CC. 2026-10-24: the 8 GB box's first night copy on 0.340.0 (R-1214). 2026-10-26: each hub daily job ran once on the clock-change day (R-1183). 2026-10-27: the weekly clean-up window (R-435; the logs of the last one were lost to restarts). 2026-10-30: D35, website and installer to Cloudflare, starts (R-1228, ruled yes).
|
||||
|
||||
## Waiting on you — decide
|
||||
|
||||
@@ -29,11 +30,8 @@ Newest first. CC's pick is in the „Pick" column. Each row also has its full te
|
||||
|
||||
| # | Row | Question | Pick (why) | The other option costs | If you do nothing | Blocked |
|
||||
|---|---|---|---|---|---|---|
|
||||
| D36 | R-1225, R-1226 | Two website sentences before the publish: „8 GB runs the basics; the photo app needs 16 GB” and „what you open from outside the home passes Cloudflare's network, readable there; the backups leave only encrypted”? | **Yes, both** (the patch's own aim is the whole truth) | — | the patch publishes with both promises a little too strong | — |
|
||||
| D35 | R-1228 | After the release week, serve the website and the installer from Cloudflare, like the ISO? | **Yes, later** (no new installs stop when DooPlex is down) | keep them home: after a DooPlex loss, no install until it is back | the same | — |
|
||||
| D34 | R-232 | Copy the hub database to ep0 every 6 hours instead of every night? | **Yes** (0 €, about 30 minutes, one DooPlex change) | keep nightly: up to a day of hub changes lost, e.g. a new box's recovery package | the same | — |
|
||||
| D33 | R-1227 | Copy the released images and agent packages to ep0 every night? | **Yes** (0 €, about 2 hours of CC, a few GB on ep0) | a timed rebuild drill instead: about 3 hours once, and rebuilt images are not the same | after a DooPlex loss no box updates, reinstalls or installs until everything is rebuilt | — |
|
||||
| D32 | R-923, R-924 | Print the break-glass sheet now (eight keys and logins + ep0's fingerprint) and keep it outside the home? | **Yes** (30 minutes; nothing else opens the off-site copies after a loss) | an encrypted USB stick outside the home: ~10 €, 1 hour, one more passphrase | after a DooPlex loss no off-site copy opens | — |
|
||||
| D38 | R-1235 | Put the newest golden (637 MB) into the nightly release copy on ep0 too? | **Yes** (a new install after a DooPlex loss needs it; it fits under the 3 GB cap) | leave it out: after a loss, no new install until a golden is rebuilt and vouched | the same | — |
|
||||
| D37 | R-1236 | Raise Gitea's memory limit from 1 GiB to 2 GiB? It was killed for memory 4 times in 17 days, once during today's release. | **Yes** (DooPlex has the RAM; a DooPlex change, so yours) | keep 1 GiB: a restart in release hours fails box updates and CI for a minute | the same | — |
|
||||
| D9 | R-1062 | Split the N100's USB drive in two, for multi-drive tests? | **No — keep one drive** (the lived-in box keeps its nightly whole-box backup) | Split: the box loses its local whole-box backup, or you buy a second drive first | it stays one drive; multi-drive tests stay on Tester 1 | — |
|
||||
| D7 | R-990, R-989 | Rule on the forum ideas and the selfh.st ideas now, or park them? | **Park them** (P4; nothing waits on them) | Rule now: about one hour of reading | nothing changes; the research gets old | — |
|
||||
|
||||
@@ -48,17 +46,14 @@ Newest first. CC's pick is in the „Pick" column. Each row also has its full te
|
||||
6. BIOS: on the N100, read „After Power Loss"; on the HP, set „Bypass F1 Prompt on Configuration Changes" to Enable. Since 2026-10-14.
|
||||
7. Pin the first Facebook post (R-917), and look at the new cover in the Facebook phone app (R-919). Since 2026-10-12.
|
||||
|
||||
**Morning, 2026-10-23 — in this order.**
|
||||
1. Hold or go: CC says go — the test night passed on a fresh 8 GB box (no failed check at night, every night job passed, nothing lost; one gap: no whole-box copy to ep0, R-1232). One question from the review: D31 as built also covers an install that got the whole-box password but no recovery package — its password is rotated without your press (R-1205). CC: go. If not: revert one hub commit before item 4.
|
||||
2. Release controller 0.340.0 (main after tonight; tested as rc.8 on the scratch box, rc.6 on the test box): the safe night copy, the catch-up fix, R-49, R-1187, R-1189, R-1214. If not: these wait.
|
||||
3. Release the agent (the tunnel repair, its tunnel field, the root-SSH bundle) and the installer tag with the root-SSH removal (R-1204, R-982); then vouch. If not: a clock-ahead box stays cut off; root keeps its password on the LAN.
|
||||
4. Build and deploy hub 0.167.0 (D31, the tunnel alarm and its clearing). If not: a dead tunnel stays silent.
|
||||
5. Publish the website patch (D3, D5), with D36's two sentences if yes. If not: the old wording stays.
|
||||
6. On the hub's Hosts page: Discard appliance 49, and delete host Tester4-16fdef (tonight's test box). If not: two dead entries alarm.
|
||||
7. The DooPlex hour: D32–D35 in the table. If not: a DooPlex loss stays as the survey describes.
|
||||
**Day, 2026-10-23 — your do list.**
|
||||
1. Print the break-glass sheet (`documentation/runbooks/break-glass-sheet.md`, the part from „The sheet”, one A4), fill it by hand, keep it outside the home (D32; R-923, R-924). If not: after a DooPlex loss no off-site copy opens.
|
||||
2. On the hub's Hosts page: Discard appliance 49, and delete host Tester4-16fdef. If not: two dead entries alarm.
|
||||
3. D37 and D38 in the table (two small yes/no). If not: Gitea may restart in release hours; no golden in the release copy.
|
||||
**The rest**
|
||||
- System poster needs a refresh: the new 01:40 „release copy” arrow from DooPlex to ep0 (D33); the hub-database label already says every 6 hours. If not: the drawing misses one copy.
|
||||
- Pull the failed disk Z6A07P2G out of DooPlex (R-934). If not: its alarms return on 11 November. Since 2026-10-12.
|
||||
- Rotate the reused homelab logins and decide the Gitea repo's visibility (R-925, P1). Since 2026-10-09.
|
||||
- Two leftovers on DooPlex: the Facebook `first-post/` folder in `leftovers/2026-10-14/` (keep, commit or delete); the `deny`-list edit in homelab-manifests' `.claude/settings.json` (commit or drop). If not: they stay. Since 2026-10-14.
|
||||
- Before the first paying customer: Tandoor's permission (R-789); a lawyer for the licence list and the terms (R-802, R-813); SparkyFitness by e-mail (R-784); print the break-glass sheet and the signing keys (R-923, R-924).
|
||||
- Before the first paying customer: Tandoor's permission (R-789); a lawyer for the licence list and the terms (R-802, R-813); SparkyFitness by e-mail (R-784);.
|
||||
- Later (yours, not urgent): R-1229 (ArgoCD's sync settings into git), R-862 (Tester 2's bootstrap, when it is back), R-504, R-1071, R-1073, R-1077, R-1042, R-132, R-908, R-779, R-904, R-232, R-433, R-933, R-213, R-832, R-388, R-882, R-883, R-814, R-1056, R-1164, R-930, R-920, R-902.
|
||||
|
||||
@@ -124,6 +124,7 @@
|
||||
| <a id="b20"></a>[B20](capability-map-detail/B.md#b20) | App crashes → customer notified (one event per transition, no flapping spam) | controller v0.120, hub v0.48 | **IMPLEMENTED** | crash→customer mail never live-confirmed | `audits/CAMPAIGN-3-2026-07-11.md` (2026-07-11) |
|
||||
| <a id="b21"></a>[B21](capability-map-detail/B.md#b21) | Post-deploy optional config (API keys etc.) with restart | controller, catalog `.felhom.yml` | **IMPLEMENTED** | config-save + restart in no campaign | `tests/CAMPAIGN-2-2026-07-07.md` (2026-07-07) |
|
||||
| <a id="b22"></a>[B22](capability-map-detail/B.md#b22) | Backup classification: 13 bind-bearing apps carry mandatory/optional/excluded classes | catalog, controller v0.132–133 | **PROVEN-LIVE** | — | `audits/SPIKE-backup-classification-2026-07-14.md` (2026-07-14); `audits/CAMPAIGN-6D-2026-07-15.md` (2026-07-15) |
|
||||
| <a id="b23"></a>[B23](capability-map-detail/B.md#b23) | An urgent security fix reaches the boxes at night — a `security_fix` ladder step (box walk required, bench optional, a CVE/GHSA notice), taken by the night update (`09` §3 decision 255, D39 = B) | catalog `3898593`, controller v0.341.0 | **PROVEN-LIVE** | scratch 9202 (drill catalog), Plex 1.43.4 → 1.43.5; an app with no box fixture is checked by health, version and a soak only (`seed: none`) | `audits/security-lane-2026-10-23/` (2026-10-23) |
|
||||
|
||||
## C. Protection & recovery (the product promise)
|
||||
|
||||
|
||||
@@ -286,6 +286,9 @@ builds them (`audits/update-rulings-2026-09-23/`).
|
||||
update night measured the proxy wrong in both directions — minor moves that held
|
||||
(adventurelog, outline) and a tag shape it cannot read at all (§3b Q3). The test record is the
|
||||
thing the proxy was guessing at.
|
||||
**Dated note 2026-10-23 (decision 255, D39 = B):** a second step kind now exists — the `security_fix` step, proven
|
||||
on the box with the bench optional, for an urgent upstream fix with a public notice. Decision 13's rule is
|
||||
unchanged for every other move: a step a box takes was still tested, on at least the box venue.
|
||||
|
||||
14. **The update ladder** (Q3). A box more than one step behind **climbs one tested step at a time, in
|
||||
order, and never jumps**. Each step is the full guarded update of §6.1. **Why:** a step was tested
|
||||
@@ -1031,6 +1034,52 @@ its length, and both fixes cost something the household would notice — operato
|
||||
Built in controller v0.314.0 (`internal/stacks/restore_gate.go`); pinned by `restore_gate_test.go` and
|
||||
`r1057_unit_gate_test.go` (ten mutations each seen failing). Proof: `audits/gate-restore-2026-10-15/9202/`.
|
||||
|
||||
### 2026-10-23 (~12:00) — D39 = B: the security-fix step (operator ruling by starting the run; R-1239)
|
||||
|
||||
255. **An urgent upstream security fix reaches the boxes at night as a `security_fix` ladder step: the box walk is
|
||||
required, the bench is optional, and a public notice is the authority.** *Operator ruling 2026-10-23 ~12:00 (D39 =
|
||||
B, R-1239), taken by starting the D39 brief; its design is Claude.ai's recommendation, taken with it.* Why: on
|
||||
2026-10-23 the only route for the Plex2Shell fix was an operator word, `git push --no-verify` past the move gate,
|
||||
and a hand press on each box — and the night leg never takes a step with no test record (R-1238). That does not
|
||||
scale past the first customers and needs the operator awake.
|
||||
1. **The entry** (`scripts/ladder.py` `check_security`, the one reader): `"verdict": "security_fix"` with `security`
|
||||
`{ids: [CVE-…|GHSA-…], advisory: "https://…", fixed_in}`, `box_evidence` (required), `seed` `read_back`|`none`,
|
||||
`memory_peak_pct` from the BOX soak with `memory_basis: "box_soak"`; `evidence` = the bench record or null — a
|
||||
bench given must be `proven` (`bench_verdict`, `harness_version` ≥ 2 ride with it). Never `backfilled`,
|
||||
`digest_from` (a same-tag fix stays decision 52's lane), `engine_conversion`, nor a database engine MAJOR (any
|
||||
engine — MariaDB included; the judgement is `check-engine-major.py`'s own `engine_of`/`major_of`, imported).
|
||||
2. **No operator word per incident.** The session that writes the step reports it to the operator after the push.
|
||||
3. **The gates:** the move gate accepts a `security_fix` move exactly as a proven one (from/to per service, digests
|
||||
the registry still serves); `check-test-record.py` WARNs on every push while a template's head is a security-fix
|
||||
step with no bench test („<app>: security-fix step <to> has no bench test (since <tested_at>)"). Seven decoys
|
||||
(`test_gate_decoys.py`), each seen failing with the rules switched off.
|
||||
4. **The box venue:** `box_walk.py --security` on 9202 through the product's Update; verdict `proven` only if the
|
||||
update ended `done` with no undo and no hold, the app runs, the four version observables are TO, the seed (if
|
||||
any) reads back, and a soak (default 600 s, cgroup `anon` against the limit, OOM kills and restarts counted)
|
||||
sees no kill and no restart. A negative control (a TO image that exits at once) comes back `failed`.
|
||||
5. **The writer:** `upgrade-test.py --write-ladder --security …`, refusing a box walk not proven, a bench given and
|
||||
not proven, a template not at the walk's FROM, a digest that moved since the walk, an engine major, a missing
|
||||
field. The compose move, `catalog_since` and the superseded step's own files are the shared `apply_entry`.
|
||||
6. **The controller (v0.341.0):** the leg takes `proven` OR `security_fix`; every other guard runs unchanged, in
|
||||
the same order; it logs `[update-leg] <app>: security fix <ids> taken (<from> → <to>)` and the hub report's
|
||||
step carries the ids. A person's press and the badge are unchanged. **A controller ≤ 0.340.0 skips the step at
|
||||
night (`not_proven`) and a hand press takes it** — proven on 9202, below.
|
||||
7. **The trade, stated.** A security-fix step skipped the bench. For an app with a box fixture the data is still
|
||||
read back after the update; for an app without one (Plex today) the step is checked only by „it starts, it is
|
||||
healthy, it runs the new version, and a 10-minute soak sees no kill or restart". `files_may_change` is a bench
|
||||
measurement: with no bench the step carries `false`. That is accepted because the leg's own guard is not the only
|
||||
net — the update takes a per-app backup first, the undo puts the pre-update data back on a failed health check
|
||||
(decision 15), and a security fix that rewrites files without breaking the health check is the case the bench
|
||||
would catch and this lane accepts the risk of. A session that knows a fix migrates files runs the bench.
|
||||
**Proof (9202, drill catalog, 2026-10-23):** `audits/security-lane-2026-10-23/` — the negative control (TO
|
||||
`busybox:1.36.1`) came back `failed` (the box undid it after 335 s) and the writer refused it; Plex 1.43.4.10903 →
|
||||
1.43.5.11029 `proven` (done in 36.9 s, the four observables at TO, digest `sha256:01bdf0ad…5c5f`, soak 600 s: anon
|
||||
peak 10.7 % of 2 GiB, 0 kills, 0 restarts, `seed: none`); the writer wrote the drill step (CVE-2026-96652, -96654,
|
||||
-96655, -96656; Plex's own notice as CISA KEV cites it, added 2026-10-20; fixed in 1.43.3.10861); on 0.340.0 the
|
||||
night leg logged `plex: skipped — not_proven` and a hand press took the step; on 0.341.0-rc.1 the night leg logged
|
||||
`plex: security fix CVE-2026-96652,CVE-2026-96654,CVE-2026-96655,CVE-2026-96656 taken` and ended `done` in 30.1 s
|
||||
after the update's second-drive copy and undo copy. Released as controller 0.341.0, global floor 11:24 UTC.
|
||||
|
||||
### 2026-10-15 (07:48–08:20) — bake first; Plex through its own key; HSTS one day (recorded before the media build)
|
||||
|
||||
247. ~~**Bake first.** A controller version is RELEASED only when its golden is baked.~~ **REVERSED by decision 250,
|
||||
|
||||
@@ -379,3 +379,28 @@
|
||||
**Gap / roadmap (as written):** Remaining ~39 apps are legacy-class by design (unit-only offsite)
|
||||
|
||||
---
|
||||
|
||||
## B23 — An urgent security fix reaches the boxes at night (the security-fix step)
|
||||
|
||||
*Map row: [B23](../00-capability-map.md#b23) · added 2026-10-23.*
|
||||
|
||||
**Scenario:** an app has a published hole (a CVE or GHSA id with its notice) and a fixed upstream version. CC walks the
|
||||
step on scratch 9202 through the product's Update (`box_walk.py --security`: install at FROM, the guarded Update, the
|
||||
four version observables at TO, a 600 s memory soak), and the only writer turns a proven walk into a `security_fix`
|
||||
ladder step. Every box on controller >= 0.341.0 takes it at night with the usual guards (backup first, undo on a
|
||||
failed health check, held / stopped / needs_person / W+5h respected).
|
||||
|
||||
**Components:** catalog `3898593` (`ladder.py check_security`, the two test-record gates, `upgrade-test.py --write-ladder
|
||||
--security`, `box_walk.py --security`); controller v0.341.0 (`legCandidate`).
|
||||
|
||||
**Status:** **PROVEN-LIVE** on 9202 with the drill catalog, 2026-10-23.
|
||||
|
||||
**Evidence:** `audits/security-lane-2026-10-23/` — the negative control (TO `busybox:1.36.1`) `failed` and the writer
|
||||
refused it; Plex 1.43.4.10903 → 1.43.5.11029 `proven`; on 0.340.0 the night leg logged `plex: skipped — not_proven` and
|
||||
a hand press took the step; on 0.341.0-rc.1 the night leg logged `plex: security fix CVE-2026-96652,… taken` and ended
|
||||
`done` after a second-drive copy and an undo copy.
|
||||
|
||||
**Gap:** an app with no box fixture (Plex today, R-1238) is checked only by health, version and the soak — no data is
|
||||
read back. `files_may_change` is a bench measurement and reads `false` when no bench ran (decision 255 item 7).
|
||||
|
||||
---
|
||||
|
||||
@@ -101,6 +101,7 @@ Footnote: "W = the household's backup window start, 02:30 by default. Tiers 1–
|
||||
- one namespace **per household** (whole-guest copies, each household's own key),
|
||||
with a NEUTRAL note under it (not a red gap box): "By design: each household holds the key to its own copies; the operator cannot open them without their recovery code." — *fix (c), 2026-10-09: this is a property of the design, not a gap, and a red box said the opposite of what it means.*
|
||||
- the **`operator`** namespace: two groups — **"hub database"** and **"DooPlex copy"** (code, password manager, secrets, signing keys).
|
||||
- outside PBS, a plain folder: the **release copy** (images + agent packages, D33).
|
||||
Prune jobs on ep0: households keep one copy per week for 2 weeks (`keep-weekly 2`, R-1162, 2026-10-19; every new household's job is made with its namespace, R-1165); `operator` keeps 14 daily + 8 weekly.
|
||||
Mark ep0 with a small shield: "**Protected** — holds the only off-premises copy of real household data."
|
||||
- **Hetzner Storage Box** (grey): the shared box for the households' per-app restic copies (Tier 3), one sub-account per household, add-only keys.
|
||||
@@ -121,8 +122,9 @@ Footnote: "W = the household's backup window start, 02:30 by default. Tiers 1–
|
||||
|
||||
Arrows from DooPlex (dashed, orange → ep0):
|
||||
- **00:20 "DooPlex copy"**: the code (614 MB), Gitea's database and settings, the **password manager**, the nightly secrets export, the **signing keys**. Encrypted on DooPlex with the **DooPlex off-site key**. The key that writes it **cannot delete** old copies. Tested back every Sunday.
|
||||
- **02:30 "Hub database"**: encrypted with the **hub-DB key**. Tested back every Sunday at 04:30.
|
||||
- **Left out on purpose:** the container registry (27.7 GB) — "rebuilt from the code".
|
||||
- **02:30, 08:30, 14:30, 20:30 "Hub database"** (every 6 hours since 2026-10-23, D34): encrypted with the **hub-DB key**. Tested back every Sunday at 04:30.
|
||||
- **01:40 "Release copy"** (since 2026-10-23, D33): the newest 3 released images of the controller, hub and samba, and the newest 5 agent packages — not encrypted (nothing secret), into a plain folder on ep0 (`/srv/felhom-release-copy`, its own limited user, outside PBS), about 0.7 GB, capped at 3 GB.
|
||||
- **Left out on purpose:** the rest of the container registry (older tags, test builds, non-Felhom images) — "rebuilt from the code".
|
||||
Arrow ep0 → DooPlex (dashed, 05:00): "**ep0-copy** pull: a second copy of ep0 at home. Keeps 8 weekly copies. A daily job removes a deleted customer's copy within 30 days."
|
||||
|
||||
**The test machines** (small, plain tiles near DooPlex, labelled "disposable"):
|
||||
|
||||
@@ -133,7 +133,7 @@ code{font-family: ui-monospace, "SFMono-Regular", Menlo, monospace; font-size: 1
|
||||
<div style="position: absolute; left: 1720px; top: 706px; width: 184px; font-size: 16px; line-height: 20px; font-weight: 400; color: #4a5670; text-align: center;">The hub never connects into a box.</div>
|
||||
<div style="position: absolute; left: 1720px; top: 784px; width: 184px; font-size: 16px; line-height: 20px; font-weight: 800; color: #a85400; text-align: center;">00:20 DooPlex copy</div>
|
||||
<div style="position: absolute; left: 1720px; top: 824px; width: 184px; font-size: 16px; line-height: 20px; font-weight: 400; color: #a85400; text-align: center;">DooPlex off-site key</div>
|
||||
<div style="position: absolute; left: 1720px; top: 872px; width: 184px; font-size: 16px; line-height: 20px; font-weight: 800; color: #a85400; text-align: center;">02:30 hub database</div>
|
||||
<div style="position: absolute; left: 1720px; top: 872px; width: 184px; font-size: 16px; line-height: 20px; font-weight: 800; color: #a85400; text-align: center;">every 6 h: hub database</div>
|
||||
<div style="position: absolute; left: 1720px; top: 912px; width: 184px; font-size: 16px; line-height: 20px; font-weight: 400; color: #a85400; text-align: center;">hub-DB key</div>
|
||||
<div style="position: absolute; left: 1720px; top: 962px; width: 184px; font-size: 16px; line-height: 20px; font-weight: 800; color: #0a3d73; text-align: center;">05:00 ep0-copy pull</div>
|
||||
<div style="position: absolute; left: 1720px; top: 1002px; width: 184px; font-size: 16px; line-height: 20px; font-weight: 400; color: #0a3d73; text-align: center;">ciphertext only</div>
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
== 9202 smoke 06:37:25
|
||||
switched to gitea.dooplex.hu/admin/felhom-controller:0.340.0 at 06:37:28
|
||||
healthy after ~10s: Up 7 seconds (healthy)
|
||||
2026/10/23 06:37:28 [INFO] [tier2] start-up: no leftover second-drive copy container
|
||||
errors, unhealthy+restarting: 0 0
|
||||
login page: 200
|
||||
@@ -0,0 +1,3 @@
|
||||
tester-1: HTTP/1.1 303 See Other Location: /customers/tester-1?flash=floor_set
|
||||
demo-hp: HTTP/1.1 303 See Other Location: /customers/demo-hp?flash=floor_set
|
||||
demo-felhom: HTTP/1.1 303 See Other Location: /customers/demo-felhom?flash=floor_set
|
||||
@@ -0,0 +1,15 @@
|
||||
### root@192.168.0.154
|
||||
containers=10 unhealthy=0 restarting=0 gitea.dooplex.hu/admin/felhom-controller:0.340.0 Up 53 seconds (healthy)
|
||||
errors=0
|
||||
copy-containers=0
|
||||
2026/10/23 06:38:21 [INFO] [tier2] start-up: no leftover second-drive copy container
|
||||
### hp
|
||||
containers=24 unhealthy=0 restarting=0 gitea.dooplex.hu/admin/felhom-controller:0.340.0 Up 54 seconds (healthy)
|
||||
errors=0
|
||||
copy-containers=0
|
||||
2026/10/23 06:38:22 [INFO] [tier2] start-up: no leftover second-drive copy container
|
||||
### felhom-pve
|
||||
containers=9 unhealthy=0 restarting=0 gitea.dooplex.hu/admin/felhom-controller:0.340.0 Up 58 seconds (healthy)
|
||||
errors=0
|
||||
copy-containers=0
|
||||
2026/10/23 06:38:05 [INFO] [tier2] start-up: no leftover second-drive copy container
|
||||
@@ -0,0 +1,33 @@
|
||||
control NEGATIVE (a stub answering 200 over plain http): convicted, as it must (200 -)
|
||||
== tester-1 (enkicsifelhom.hu): 6 public name(s)
|
||||
OK control POSITIVE http://felhom.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://felhom.enkicsifelhom.hu/r1033-check?q=1
|
||||
OK http://files.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://files.enkicsifelhom.hu/r1033-check?q=1
|
||||
OK http://no-such-name-r1033.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://no-such-name-r1033.enkicsifelhom.hu/r1033-check?q=1
|
||||
OK http://paperless.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://paperless.enkicsifelhom.hu/r1033-check?q=1
|
||||
OK http://paste.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://paste.enkicsifelhom.hu/r1033-check?q=1
|
||||
OK http://wiki.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://wiki.enkicsifelhom.hu/r1033-check?q=1
|
||||
== demo-hp (enkisfelhom.hu): 14 public name(s)
|
||||
OK http://arcade.enkisfelhom.hu/r1033-check?q=1 -> 301 https://arcade.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://books.enkisfelhom.hu/r1033-check?q=1 -> 301 https://books.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://docs.enkisfelhom.hu/r1033-check?q=1 -> 301 https://docs.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://felhom.enkisfelhom.hu/r1033-check?q=1 -> 301 https://felhom.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://files.enkisfelhom.hu/r1033-check?q=1 -> 301 https://files.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://gist.enkisfelhom.hu/r1033-check?q=1 -> 301 https://gist.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://ha.enkisfelhom.hu/r1033-check?q=1 -> 301 https://ha.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://no-such-name-r1033.enkisfelhom.hu/r1033-check?q=1 -> 301 https://no-such-name-r1033.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://papir.enkisfelhom.hu/r1033-check?q=1 -> 301 https://papir.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://paste.enkisfelhom.hu/r1033-check?q=1 -> 301 https://paste.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://pdf.enkisfelhom.hu/r1033-check?q=1 -> 301 https://pdf.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://plex.enkisfelhom.hu/r1033-check?q=1 -> 301 https://plex.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://travel.enkisfelhom.hu/r1033-check?q=1 -> 301 https://travel.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://wiki.enkisfelhom.hu/r1033-check?q=1 -> 301 https://wiki.enkisfelhom.hu/r1033-check?q=1
|
||||
== demo-felhom (demo-felhom.eu): 8 public name(s)
|
||||
OK http://felhom.demo-felhom.eu/r1033-check?q=1 -> 301 https://felhom.demo-felhom.eu/r1033-check?q=1
|
||||
OK http://files.demo-felhom.eu/r1033-check?q=1 -> 301 https://files.demo-felhom.eu/r1033-check?q=1
|
||||
OK http://gist.demo-felhom.eu/r1033-check?q=1 -> 301 https://gist.demo-felhom.eu/r1033-check?q=1
|
||||
OK http://no-such-name-r1033.demo-felhom.eu/r1033-check?q=1 -> 301 https://no-such-name-r1033.demo-felhom.eu/r1033-check?q=1
|
||||
OK http://prowlarr.demo-felhom.eu/r1033-check?q=1 -> 301 https://prowlarr.demo-felhom.eu/r1033-check?q=1
|
||||
OK http://radarr.demo-felhom.eu/r1033-check?q=1 -> 301 https://radarr.demo-felhom.eu/r1033-check?q=1
|
||||
OK http://sonarr.demo-felhom.eu/r1033-check?q=1 -> 301 https://sonarr.demo-felhom.eu/r1033-check?q=1
|
||||
OK http://torrent.demo-felhom.eu/r1033-check?q=1 -> 301 https://torrent.demo-felhom.eu/r1033-check?q=1
|
||||
== every public name redirects http -> https
|
||||
@@ -0,0 +1,13 @@
|
||||
== tester-1 (dashboard login with the household password, the press a browser makes)
|
||||
python3: can't open file '/tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/38fde0c6-e964-4ec1-b0c7-56e62d675225/scratchpad/n23/fileswalk.py': [Errno 2] No such file or directory
|
||||
== demo-hp (no session): GET /__files/open -> HTTP/2 302 location: /login
|
||||
== demo-felhom (no session): GET /__files/open -> HTTP/2 302 location: /login
|
||||
== tester-1 06:40:35
|
||||
Fájlok press: GET /__files/open -> ['HTTP/2 302'] https://files.enkicsifelhom.hu/_felhom/sso
|
||||
manual second-drive copy: 200 2. mentés elindítva
|
||||
2026/10/23 06:40:43 [INFO] [backup] Tier 2 copied bookstack → /mnt/felhom-drives/adatlemez/backups/secondary/bookstack (157.0 MB, 0 leg(s), 1s)
|
||||
2026/10/23 06:40:46 [INFO] [backup] Tier 2 copied paperless-ngx → /mnt/sys_drive/felhom-data/backups/secondary/paperless-ngx (85.9 MB, 1 leg(s), 2s) [SSD: state-only]
|
||||
2026/10/23 06:40:48 [INFO] [backup] Tier 2 copied privatebin → /mnt/felhom-drives/adatlemez/backups/secondary/privatebin (16.4 KB, 0 leg(s), 0s)
|
||||
2026/10/23 06:40:48 [INFO] [backup] Tier 2 run complete: 3 app(s) processed (incl. volume-only — F6)
|
||||
copy-containers-left=0
|
||||
[exited with code 0]
|
||||
@@ -0,0 +1,5 @@
|
||||
== global floor 06:41:04: HTTP/1.1 303 See Other Location: /configuration?flash=floor_set
|
||||
clear tester-1: HTTP/1.1 303 See Other Location: /customers/tester-1?flash=floor_set
|
||||
clear demo-hp: HTTP/1.1 303 See Other Location: /customers/demo-hp?flash=floor_set
|
||||
clear demo-felhom: HTTP/1.1 303 See Other Location: /customers/demo-felhom?flash=floor_set
|
||||
read back (with WAL): [('min_controller_version', '0.340.0', '2026-10-23 06:41:04'), ('min_controller_version_declared_min_agent', '0.340.0=0.158.1', '2026-10-23 06:41:04')]
|
||||
@@ -0,0 +1,78 @@
|
||||
# Part 2 — agent 0.163.0 (operator go 2026-10-23), read 2026-10-23T07:28:53Z
|
||||
|
||||
## release (scripts/release-agent.sh at felhom-agent 1109092, CI 2246 green; CHANGELOG b4721ee CI 2249 green)
|
||||
[release-agent] built ok: sha256 561b5316cd511de9fa9d8b8be4553da7add684f715d3bea533ca330ff7f3763c
|
||||
[release-agent] config bundle built: sha256 3afb88d9fbbc4adc16083fa52a98cd4974583c6378a77a1a6e0ed8ae6d1b80c8
|
||||
[publish-agent] upload OK (HTTP 201)
|
||||
[publish-agent] round-trip GET verified (sha256 matches)
|
||||
[release-agent] config bundle published (HTTP 201)
|
||||
sha256 : 561b5316cd511de9fa9d8b8be4553da7add684f715d3bea533ca330ff7f3763c
|
||||
bundle : 3afb88d9fbbc4adc16083fa52a98cd4974583c6378a77a1a6e0ed8ae6d1b80c8 (felhom-config-bundle.json — vouch it with the agent)
|
||||
* [new tag] v0.163.0 -> v0.163.0
|
||||
|
||||
## step bundle (R-880: the D4 drop-in is a NEW path; the boxes ran bundle 0.160.0 = 6b06090190ff609af1fe9842447ebf385fd52da4be382bc7ca708b558c8e202f)
|
||||
0.163.0-step1 = base 0.160.0 with only /usr/local/sbin/felhom-os-apply replaced; sha256 6cbc066902d5f3294765d5c7dc7113166c1dca25e994dbc36bfc5f139bb61fcf; PUT 201; read back by download: same sha
|
||||
|
||||
## delivery: three signed jobs per box (felhom-opsign, key felhom-op-1): agent_update 06:48Z, agent_config_update 0.163.0-step1 06:59Z, agent_config_update 0.163.0 07:19Z
|
||||
|
||||
## hp (192.168.0.104) 2026-10-23T07:26:29Z
|
||||
agent: felhom-agent 0.163.0
|
||||
bundle: 0.163.0 3afb88d9fbbc4adc16083fa52a98cd4974583c6378a77a1a6e0ed8ae6d1b80c8 2026-10-23T07:19:44Z
|
||||
1 BUNDLE DONE agent=0.163.0 written=1 same=30 self-check=ok
|
||||
2 BUNDLE DONE agent=0.163.0-step1 written=1 same=29 self-check=ok
|
||||
guest 9201: memory: 25898 swap: 25898
|
||||
guest 9202: memory: 25898 swap: 512
|
||||
guest 9401: memory: 10240 swap: 0
|
||||
drop-in: -rw-r--r-- 1 root root 690 Oct 23 09:19 /etc/ssh/sshd_config.d/10-felhom-root-key-only.conf
|
||||
drop-in content: PermitRootLogin prohibit-password
|
||||
sshd -T: permitrootlogin without-password
|
||||
root password set (passwd -S): P
|
||||
felhom-sshd: active; its root policy: permitrootlogin without-password
|
||||
felhom-sshd listens: 0.0.0.0:8822 [::]:8822 config test: ok; firewall admits only: elements = { 10.77.0.250 }
|
||||
main sshd reloaded after the drop-in: Oct 23 09:19:44 demo-hp sshd[1010]: Received SIGHUP; restarting. Oct 23 09:19:44 demo-hp systemd[1]: Reloaded ssh.service - OpenBSD Secure Shell server.
|
||||
pveproxy: active
|
||||
main sshd banner from DooPlex (:22): SSH-2.0-OpenSSH_10.0p2 Debian-7+deb13u4
|
||||
key login: ok
|
||||
root password attempt rc=255 (non-zero = refused)
|
||||
Oct 23 09:26:37 demo-hp sshd-session[3026730]: Failed password for root from 192.168.0.180 port 34150 ssh2
|
||||
## root@192.168.0.154 (192.168.0.154) 2026-10-23T07:28:00Z
|
||||
agent: felhom-agent 0.163.0
|
||||
bundle: 0.163.0 3afb88d9fbbc4adc16083fa52a98cd4974583c6378a77a1a6e0ed8ae6d1b80c8 2026-10-23T07:27:38Z
|
||||
2 BUNDLE DONE agent=0.163.0 written=1 same=30 self-check=ok
|
||||
2 BUNDLE DONE agent=0.163.0-step1 written=1 same=29 self-check=ok
|
||||
guest 9201: memory: 4096 swap: 4096
|
||||
guest 990000: memory: 4096 swap: 512
|
||||
drop-in: -rw-r--r-- 1 root root 690 Oct 23 09:27 /etc/ssh/sshd_config.d/10-felhom-root-key-only.conf
|
||||
drop-in content: PermitRootLogin prohibit-password
|
||||
sshd -T: permitrootlogin without-password
|
||||
root password set (passwd -S): P
|
||||
felhom-sshd: active; its root policy: permitrootlogin without-password
|
||||
felhom-sshd listens: 0.0.0.0:8822 [::]:8822 config test: ok; firewall admits only: elements = { 10.77.0.250 }
|
||||
main sshd reloaded after the drop-in: Oct 23 09:27:38 felhom sshd[890]: Received SIGHUP; restarting. Oct 23 09:27:38 felhom systemd[1]: Reloaded ssh.service - OpenBSD Secure Shell server.
|
||||
pveproxy: active
|
||||
main sshd banner from DooPlex (:22): SSH-2.0-OpenSSH_10.0p2 Debian-7+deb13u4
|
||||
key login: ok
|
||||
root password attempt rc=255 (non-zero = refused)
|
||||
Oct 23 09:28:08 felhom sshd-session[4184378]: Failed password for root from 192.168.0.180 port 33442 ssh2
|
||||
## felhom-pve (192.168.0.162) 2026-10-23T07:28:11Z
|
||||
agent: felhom-agent 0.163.0
|
||||
bundle: 0.163.0 3afb88d9fbbc4adc16083fa52a98cd4974583c6378a77a1a6e0ed8ae6d1b80c8 2026-10-23T07:24:43Z
|
||||
2 BUNDLE DONE agent=0.163.0 written=1 same=30 self-check=ok
|
||||
2 BUNDLE DONE agent=0.163.0-step1 written=1 same=29 self-check=ok
|
||||
guest 9201: memory: 12288 swap: 12288
|
||||
drop-in: -rw-r--r-- 1 root root 690 Oct 23 09:24 /etc/ssh/sshd_config.d/10-felhom-root-key-only.conf
|
||||
drop-in content: PermitRootLogin prohibit-password
|
||||
sshd -T: permitrootlogin without-password
|
||||
root password set (passwd -S): P
|
||||
felhom-sshd: active; its root policy: permitrootlogin without-password
|
||||
felhom-sshd listens: 0.0.0.0:8822 [::]:8822 config test: ok; firewall admits only: elements = { 10.77.0.250 }
|
||||
main sshd reloaded after the drop-in: Oct 23 09:24:42 demo-felhom sshd[1011]: Received SIGHUP; restarting. Oct 23 09:24:42 demo-felhom systemd[1]: Reloaded ssh.service - OpenBSD Secure Shell server
|
||||
pveproxy: active
|
||||
main sshd banner from DooPlex (:22): SSH-2.0-OpenSSH_10.0p2 Debian-7+deb13u4
|
||||
key login: ok
|
||||
root password attempt rc=255 (non-zero = refused)
|
||||
|
||||
## notes
|
||||
- felhom-sshd :8822 admits only the operator peer 10.77.0.250 over wg-felhom (H1); a connection test from DooPlex times out BY DESIGN — proven: active, listening, sshd -t ok, firewall set.
|
||||
- a dummy root password is logged as 'Failed password' (OpenSSH refuses a root password silently under prohibit-password); the policy proof is sshd -T + the SIGHUP reload after the drop-in. The real root password was not tried (CC does not hold it).
|
||||
- guests 9202, 9401 (demo-hp) and the transient 990000 (Tester 1, gone a minute later) are outside the felhom pool; swap = RAM applies to pool guests (9201 on all three: swap = memory).
|
||||
@@ -0,0 +1,53 @@
|
||||
# Part 6 — D33: the released images and agent packages to ep0 nightly (operator yes 2026-10-23), 2026-10-23T07:51:56Z
|
||||
|
||||
## ep0 — free space first, then ONE user and ONE directory (nothing else on ep0 changed)
|
||||
before: /dev/sda1 38G 23G used 14G free 63% (/mnt/pbs-datastore is the PBS datastore root — not used)
|
||||
created: user felhom-relcopy (system, uid 999, password locked 'L', shell /bin/sh for the forced command), /srv/felhom-release-copy (0750) with data/ and .ssh/authorized_keys:
|
||||
command="/usr/bin/rrsync /srv/felhom-release-copy/data",restrict ssh-ed25519 <the job key, felhom-release-copy@dooplex>
|
||||
fence test: rsync --list-only OK; a shell command → 'rrsync error: SSH_ORIGINAL_COMMAND does not run rsync'; '..' → 'do not use .. in arg'
|
||||
after the first copy: /dev/sda1 38G 24G used 13G free 65%; /srv/felhom-release-copy/data 677M
|
||||
cap: CAP_MB=3072, enforced by the sender (ep0's root fs has no quota; a quota would change more than one directory)
|
||||
|
||||
## DooPlex — the job (felhom.eu d306b8c2): /usr/local/sbin/felhom-release-copy + .service + .timer (01:40), /etc/felhom-release-copy/{env,id_ed25519,known_hosts,registry-token}
|
||||
registry token: a NEW Gitea access token for user admin, name felhom-release-copy-d33, scope read:package ONLY — minted inside the gitea pod straight into /etc/felhom-release-copy/registry-token (0600 root); never printed
|
||||
known_hosts: ep0's ed25519 host key, SHA256:NGtcS5v3K+9UjMTM6X5Ib0CoChdhqj9bezLz1hOUqgY — equal to the key already pinned for root@ep0
|
||||
skopeo, not docker save: skopeo dir: keeps the registry's manifest bytes and compressed blobs, so a push back gives the SAME digest (docker save re-compresses → a new digest); no Docker daemon needed
|
||||
|
||||
## the first run (systemctl start, under the unit's sandbox)
|
||||
2026-10-23T09:48:33+02:00 dooplex felhom-release-copy[1567269]: felhom-release-copy: felhom-controller: keeping 0.338.0, 0.339.0, 0.340.0
|
||||
2026-10-23T09:48:37+02:00 dooplex felhom-release-copy[1567269]: felhom-release-copy: felhom-hub: keeping 0.164.0, 0.165.0, 0.166.0
|
||||
2026-10-23T09:48:43+02:00 dooplex felhom-release-copy[1567269]: felhom-release-copy: felhom-samba: keeping 1.0.0, 1.1.0, 1.2.0
|
||||
2026-10-23T09:48:47+02:00 dooplex felhom-release-copy[1567269]: felhom-release-copy: felhom-agent: keeping 0.159.2, 0.160.0, 0.161.0, 0.162.0, 0.163.0
|
||||
2026-10-23T09:48:51+02:00 dooplex felhom-release-copy[1567269]: felhom-release-copy: staged and verified: 676 MB, new this run: felhom-controller:0.338.0, felhom-controller:0.339.0, felhom-controller:0.340.0, felhom-hub:0.164.0, felhom-hub:0.165.0, felhom-hub:0.166.0, felhom-samba:1.0.0, felhom-samba:1.1.0, felhom-samba:1.2.0
|
||||
2026-10-23T09:49:51+02:00 dooplex felhom-release-copy[1567269]: felhom-release-copy: pushed to ep0 and compared by checksum: no difference (91 s)
|
||||
2026-10-23T09:49:51+02:00 dooplex felhom-release-copy[1567269]: felhom-release-copy: success signal written
|
||||
2026-10-23T09:49:51+02:00 dooplex systemd[1]: felhom-release-copy.service: Consumed 30.416s CPU time, 743.1M memory peak.
|
||||
|
||||
## read back from ep0 (rsync pull with the job key → /var/lib/felhom-release-copy/readback)
|
||||
sha256sum -c SHA256SUMS: every file OK (141 files), 677 MB
|
||||
every image re-verified (top manifest = its digest, every blob = its name):
|
||||
felhom-controller 0.338.0 sha256:e8a529f395bfe8c0e7b87d5d98929a6f8e9834aa13c0ab77bd214ee9e7e53433
|
||||
felhom-controller 0.339.0 sha256:fee71aafceb4a7d5f24ae4a6a273a8d51eb26e0550fe9d12d4d86955f6a343a8
|
||||
felhom-controller 0.340.0 sha256:a8d1c6458da4ea9d667f928498656a291771e0b3756b20f2fcc8c1cf9d516621
|
||||
felhom-hub 0.164.0 sha256:931594e333a4e8a9784eff2fb144d022ab46f89dd6a865a93e6100dc28dbd9dc
|
||||
felhom-hub 0.165.0 sha256:c6180fb79a2ff7d5546a704c68b634b9207afb0a0ecd524c74b0310434532537
|
||||
felhom-hub 0.166.0 sha256:829667bb289c790bc99719e88dad8f7d08faaa9f662df1e9a06b1774915a1af6
|
||||
felhom-samba 1.0.0 sha256:0fa9fae3eaa9d48836c855a4884f0751a90ff050dc9e7d17e41d93bb434499c0
|
||||
felhom-samba 1.1.0 sha256:1c17c09422bec0366d7cf0e0fcfc1486ba6c90334a0a5d5c851073a9342f8f10
|
||||
felhom-samba 1.2.0 sha256:da96f25033827fbb77d0ee398e86fe5b744e2f29be9de898994533340c97acf9
|
||||
agent 0.159.2 binary 4d98868e922d6c10665a6c0da9323e8627ea9c589094e81d27d741e625d9d8f9 bundle 477704ea2980948433ffc34bb59f008cc11410fd9683d386bfd3e2f085d55bc9
|
||||
agent 0.160.0 binary 7ebb547f2b1ab93544ba828d3040351b60c4126327a5b85bbcb33f8f2b3dfca1 bundle 6b06090190ff609af1fe9842447ebf385fd52da4be382bc7ca708b558c8e202f
|
||||
agent 0.161.0 binary ff4e6dc52dd70234913445ecdeb88d31dea8e394d77c48704d26ff19b999d3b5 bundle 485f36c957d83f772aa22cac24e5e73f44da5cf8b957b1fd513bec2bc1d12c47
|
||||
agent 0.162.0 binary 5fd7a23bfe319cb521c4bc399612d5fb3413f1e695f79bc663c386f69c757928 bundle ac772967fd4669045e837a70bf5acca239c58d55f7f00212e472a8facd60b5a5
|
||||
agent 0.163.0 binary 561b5316cd511de9fa9d8b8be4553da7add684f715d3bea533ca330ff7f3763c bundle 3afb88d9fbbc4adc16083fa52a98cd4974583c6378a77a1a6e0ed8ae6d1b80c8
|
||||
independent controls (other channels): controller 0.340.0 = the release digest from build.sh (03-release-0340-boxes.txt); hub 0.165.0 = the running hub pod's imageID; hub 0.164.0 = hub/CHANGELOG; agent 0.163.0 = release-agent.sh today; agent 0.161.0 = night2-2026-10-20/P1/1b; bundle 0.160.0 = the three boxes' installed bundle_sha256
|
||||
|
||||
## load on a scratch machine (9202 on demo-hp, Tier 0)
|
||||
the ep0 copy of felhom-controller 0.340.0 → skopeo docker-archive (amd64) → pct push 9202 → docker load:
|
||||
Loaded image: felhom-restorecopy-test:0.340.0; ID=sha256:3f34d74519463c338422594b70c31d9c5beabed48847134c6d26b5a95ebbae2b = the copy's amd64 config digest; a container ran (ls /)
|
||||
teardown: image untagged (0 left), the tar removed on 9202, on demo-hp /tmp and on DooPlex
|
||||
|
||||
timer: Sat 2026-10-24 01:42:16 CEST 15h - - felhom-release-copy.timer felhom-release-copy.service
|
||||
|
||||
## alarm (homelab-manifests 1bbef892): FelhomReleaseCopyStale (> 26 h or absent, warning) — promtool check: 7 rules OK; promtool test: quiet at 25 h, fires at 27 h
|
||||
synced ConfigMap mon-system/prometheus-rules only; POST /-/reload 200; /api/v1/rules: FelhomReleaseCopyStale inactive ok; metric age at read ~3235 s
|
||||
@@ -0,0 +1,10 @@
|
||||
no guest references the three
|
||||
BEFORE: /dev/mapper/pve-root 39G 29G 8.5G 77% /
|
||||
Removed volume 'local:iso/felhom-installer-1.27.1-pve9.2-1.iso'
|
||||
removed local:iso/felhom-installer-1.27.1-pve9.2-1.iso
|
||||
Removed volume 'local:iso/felhom-installer-1.28.0-pve9.2-1.iso'
|
||||
removed local:iso/felhom-installer-1.28.0-pve9.2-1.iso
|
||||
Removed volume 'local:iso/felhom-installer-1.29.0-pve9.2-1.iso'
|
||||
removed local:iso/felhom-installer-1.29.0-pve9.2-1.iso
|
||||
AFTER: /dev/mapper/pve-root 39G 24G 14G 64% /
|
||||
local:iso/felhom-installer-1.30.0-pve9.2-1.iso iso iso 1705324544
|
||||
@@ -0,0 +1,16 @@
|
||||
# The vouch read-back — agent 0.163.0 (the operator vouched on the hub, answered 'Vouched now'), read 2026-10-23T08:00:42Z
|
||||
|
||||
## channel 1: hub DB (hub.db + -wal + -shm in ONE tar from the pod), hub_settings
|
||||
artifact_agent_sha256|561b5316cd511de9fa9d8b8be4553da7add684f715d3bea533ca330ff7f3763c|2026-10-23 07:58:46
|
||||
artifact_agent_version|0.163.0|2026-10-23 07:58:46
|
||||
artifact_bundle_sha256|3afb88d9fbbc4adc16083fa52a98cd4974583c6378a77a1a6e0ed8ae6d1b80c8|2026-10-23 07:58:46
|
||||
artifact_golden_sha256|3881de62b0986ccbe5e55580e3f1bb9765d65b8d898f67c6339bac105244f84f|2026-10-23 07:58:46
|
||||
artifact_golden_version|0.331.0|2026-10-23 07:58:46
|
||||
artifact_min_agent|0.158.1|2026-10-23 07:58:46
|
||||
artifact_wrapper_sha256||2026-10-23 07:58:46
|
||||
|
||||
## channel 2: the running hub's operator page GET /configuration (CC's token), the selected options
|
||||
value="0.163.0" data-sha="561b5316cd511de9fa9d8b8be4553da7add684f715d3bea533ca330ff7f3763c" selected>0.163.0
|
||||
value="0.331.0" data-sha="3881de62b0986ccbe5e55580e3f1bb9765d65b8d898f67c6339bac105244f84f" selected>0.331.0
|
||||
|
||||
agent 0.163.0 sha256 = the release (561b5316…); bundle sha256 = the release (3afb88d9…). Golden 0.331.0 unchanged. The floor's MinAgent stays 0.158.1 (artifact_min_agent 0.158.1; controller 0.340.0 declares 0.158.1).
|
||||
@@ -0,0 +1,13 @@
|
||||
# Installer 1.36.0 published (operator yes 2026-10-23), read 2026-10-23T08:16:46Z
|
||||
|
||||
1. bump: felhom.eu 83388c08 (SCRIPT_VERSION 1.36.0; scripts/test_hostinstall.py 40 passed; CI 2247 success)
|
||||
2. tag: installer-v1.36.0 (annotated d00f868c) -> 83388c08, pushed
|
||||
3. pins: manifests/webpage.yaml both --ref=installer-v1.36.0 (sidecar git-sync-scripts + init git-sync-scripts-init), commit 2aebf1bd;
|
||||
ArgoCD: only Deployment felhom-system/felhom-webpage was OutOfSync; synced that ONE resource at revision 2aebf1bd; rollout ok;
|
||||
live args: git-sync-scripts-init ['--ref=installer-v1.36.0'], git-sync-scripts ['--ref=installer-v1.36.0'], website git-sync ['--ref=main'] (unchanged)
|
||||
4. read back: curl https://felhom.eu/scripts/felhom-host-install.sh
|
||||
SCRIPT_VERSION="1.36.0" # the SINGLE version source (F-1): -h and the run banners follow it.
|
||||
sha256 served = de8bf5f32cbcda44c9976ba829f04ea151e3b30634b54d5fdd9eb6c4a0351cec
|
||||
sha256 tag file = de8bf5f32cbcda44c9976ba829f04ea151e3b30634b54d5fdd9eb6c4a0351cec
|
||||
the drop-in name 10-felhom-root-key-only appears 2 times (the uninstall's removal)
|
||||
Rollback: move the tag back to installer-v1.35.0's commit and wait ~30 s (no sync).
|
||||
@@ -0,0 +1,4 @@
|
||||
# Website publish read-back (felhom.eu 0bb57a4f, operator yes 2026-10-23)
|
||||
Method: headless Chromium (playwright 1.61.1 image) on the LIVE site; the FAQ answer opened by a click; 390x844 (phone) and 1366x900 (computer).
|
||||
No page scrolls sideways at either size (scrollWidth == viewport on all 10 shots).
|
||||
Text: see text-check.txt (new sentences present; the old 'only encrypted (the remote backup)' gone; control: HEAD~1 had it twice).
|
||||
|
After Width: | Height: | Size: 144 KiB |
|
After Width: | Height: | Size: 122 KiB |
|
After Width: | Height: | Size: 113 KiB |
|
After Width: | Height: | Size: 122 KiB |
|
After Width: | Height: | Size: 124 KiB |
|
After Width: | Height: | Size: 70 KiB |
|
After Width: | Height: | Size: 55 KiB |
|
After Width: | Height: | Size: 55 KiB |
|
After Width: | Height: | Size: 54 KiB |
|
After Width: | Height: | Size: 59 KiB |
@@ -0,0 +1,7 @@
|
||||
waited ~0 s
|
||||
NEW letoltes 'az alapokat futtatja' count 1 (want 2 on the FAQ: visible + JSON-LD; 1 on download)
|
||||
NEW en/download 'runs the basics; the photo app needs 16 GB' count 1 (want 2 on the FAQ: visible + JSON-LD; 1 on download)
|
||||
NEW gyik 'a Cloudflare h' count 4 (want 2 on the FAQ: visible + JSON-LD; 1 on download)
|
||||
NEW en/faq "passes through Cloudflare's network and is readable there" count 2 (want 2 on the FAQ: visible + JSON-LD; 1 on download)
|
||||
OLD gyik 'otthonodat (a t' live 0 (want 0); control HEAD~1 2 (want 2)
|
||||
OLD en/faq 'leaves your home only encrypted (the remote backup)' live 0 (want 0); control HEAD~1 2 (want 2)
|
||||
@@ -0,0 +1,43 @@
|
||||
# Part 5 — D34: the hub DB off-site every 6 hours (operator yes 2026-10-23), 2026-10-23T08:41:15Z
|
||||
|
||||
## what changed
|
||||
- hub 0.167.0 (deployed 08:27Z, image sha256:177860a9…): snapshots at 02:00 08:00 14:00 20:00 Budapest; start-up catch-up when the newest > 6 h. Hub log at start:
|
||||
2026/10/23 10:28:12 [INFO] db-snapshot 08:00: next run at 2026-10-24 08:00 CEST (in 21h31m48s)
|
||||
2026/10/23 10:28:12 [INFO] db-snapshot 14:00: next run at 2026-10-23 14:00 CEST (in 3h31m48s)
|
||||
2026/10/23 10:28:12 [INFO] db-snapshot 20:00: next run at 2026-10-23 20:00 CEST (in 9h31m48s)
|
||||
2026/10/23 10:28:12 [INFO] db-snapshot 02:00: next run at 2026-10-24 02:00 CEST (in 15h31m48s)
|
||||
2026/10/23 10:30:06 [INFO] db snapshot written: hub-20261023T082812Z.db (525996032 bytes, 1m54.537s); pruned 1, keeping 2
|
||||
- DooPlex felhom-hub-db-backup (felhom.eu 52af38df): timer 02:30 08:30 14:30 20:30; waits ≤ 20 min for a snapshot being written; refuses a newest snapshot older than 7 h
|
||||
- R-1234 (closed): the first version (2 h, no wait, be7e97dc) refused the timer's own catch-up at 10:28 CEST and mailed admin@ once — fixed in 52af38df
|
||||
|
||||
## retention (ep0, server-side, unchanged): prune-job prune-operator-hubdb, ns operator, 03:45, keep-daily 14, keep-weekly 8
|
||||
=> the current day's copies (up to 4) stay until the next 03:45 prune, then the newest of each day for 14 days and one per week for 8 weeks. Size per copy: 501.6 MiB raw, 24.5 MiB uploaded (compressed, dedup'd chunks).
|
||||
|
||||
## run once by hand (systemctl start felhom-hub-db-backup.service, 10:39 CEST)
|
||||
2026-10-23T10:39:33+02:00 dooplex felhom-hub-db-backup[2374380]: felhom-hub-db-backup: snapshot hub-20261023T082812Z.db, 11 min old
|
||||
2026-10-23T10:39:45+02:00 dooplex felhom-hub-db-backup[2374380]: felhom-hub-db-backup: checked: 525996032 bytes, integrity ok, 4 host(s)
|
||||
2026-10-23T10:39:48+02:00 dooplex felhom-hub-db-backup[2377041]: hubdb.pxar: had to backup 501.629 MiB of 501.629 MiB (compressed 24.471 MiB) in 2.47 s (average 203.437 MiB/s)
|
||||
2026-10-23T10:39:49+02:00 dooplex felhom-hub-db-backup[2377041]: Duration: 3.21s
|
||||
2026-10-23T10:39:49+02:00 dooplex felhom-hub-db-backup[2374380]: felhom-hub-db-backup: pushed hub-20261023T082812Z.db to ep0 (ns operator) in 4 s
|
||||
2026-10-23T10:39:49+02:00 dooplex felhom-hub-db-backup[2374380]: felhom-hub-db-backup: success signal written
|
||||
|
||||
## read back: restore the NEWEST copy from ep0 with the READ-ONLY token into a scratch dir (shredded after)
|
||||
15 copies on ep0, oldest 2026-10-10T00:30:56Z
|
||||
newest copy on ep0: host/dooplex-hub/2026-10-23T08:39:45Z
|
||||
Using encryption key from '/etc/felhom-hub-backup/enc.key'..
|
||||
Fingerprint: b2:19:bf:36:3b:97:3d:6c
|
||||
restore complete (501.629 MiB processed in 5.4s, average 92.519 MiB/s)
|
||||
restored size: 525996032 bytes
|
||||
restored sha256: a9ff4c8a0d0fe34da72411d9c165e573069f5601492d1e625578464f56adbde7
|
||||
integrity_check: ok
|
||||
SELECT COUNT(*) FROM hosts: 4
|
||||
SELECT value FROM hub_settings WHERE key='artifact_agent_version': 0.163.0
|
||||
pod snapshot hub-20261023T082812Z.db: 525996032 bytes, sha256 a9ff4c8a0d0fe34da72411d9c165e573069f5601492d1e625578464f56adbde7 (kubectl exec sha256sum — a different channel from the restore)
|
||||
=> byte-identical; it holds today's vouch (artifact_agent_version 0.163.0, set 07:58Z) — a nightly copy could not
|
||||
|
||||
## timers
|
||||
Fri 2026-10-23 14:30:08 CEST 3h 48min Fri 2026-10-23 10:28:30 CEST 12min ago felhom-hub-db-backup.timer felhom-hub-db-backup.service
|
||||
|
||||
## alarm (homelab-manifests 1bbef892): HubDBBackupStale 26 h -> 13 h (two missed 6-hourly pushes)
|
||||
promtool test: quiet at 12 h, fires at 14 h; RED-PROOF: the rule at 26 h fails the 14 h case
|
||||
in force: /api/v1/rules shows the 13 * 3600 expression, inactive ok; metric age at read ~237 s
|
||||
@@ -0,0 +1,21 @@
|
||||
# Part 3 — hub 0.167.0 deploy and the 30-minute watch (operator yes 2026-10-23), read 2026-10-23T08:56:31Z
|
||||
|
||||
build: clean clone at d306b8c2 (hub/ unchanged since be7e97dc, CI 2250 success; d306b8c2 CI 2251 success before the deploy); hub tests rc 0 (28 packages)
|
||||
image: gitea.dooplex.hu/admin/felhom-hub:0.167.0 sha256:177860a996cb285d935777d6ffdf85725f6cf4489885f54fd3bf8135c68041a0
|
||||
manifest 34979d5d; ArgoCD: only Deployment felhom-system/hub OutOfSync; synced that one resource; rollout ok 08:27:34Z; pod imageID = the digest above; Synced/Healthy
|
||||
|
||||
## watch 08:27:34Z → 08:56:31Z — channel 1: the hub DB (with WAL) + log
|
||||
events since the deploy: 0 (newest event 07:07:10Z Tester4 host_down — the expected dead test box; 161 events since 2026-10-22: the table is live)
|
||||
host report|demo-felhom-8363b5|2026-10-23 08:54:54|2
|
||||
host report|demo-hp-bb76ea|2026-10-23 08:49:41|2
|
||||
host report|tester-1-d70be4|2026-10-23 08:42:35|1
|
||||
controller report|demo-felhom|2026-10-23 08:53:20|3
|
||||
controller report|demo-hp|2026-10-23 08:53:24|3
|
||||
controller report|tester-1|2026-10-23 08:53:23|3
|
||||
hub log since deploy: 0 [ERROR]; 'osupdates: alarm check done … 0 alarm(s) sent'; 'Host staleness checker initialized: 3 ok, 1 already alarmed (seeded silently, R-1001)' — Tester4 not re-mailed
|
||||
|
||||
## channel 2: the operator mailbox (Gmail connector, to:admin@felhom.eu newer_than:1d)
|
||||
after 08:27Z: ONE mail, 08:28:32Z '[DooPlex backup] FAILED: felhom-hub-db-backup.service' — NOT from the hub: CC's install of the 6-hourly timer (R-1234, closed, fixed 52af38df). No hub mail after the deploy.
|
||||
before the deploy (control that the channel delivers): 07:07:11Z Tester4 host_down, 07:04:11Z node_down, 06:22Z/06:19Z stale — the hub's mails do arrive
|
||||
|
||||
verdict: no new false alarm from the hub in the 30 minutes after the deploy; one CC-caused DooPlex mail, filed and fixed
|
||||
@@ -0,0 +1,81 @@
|
||||
# Plex to the fixed version on the demo/test boxes — 2026-10-23 (Plex2Shell)
|
||||
|
||||
Operator ruling 2026-10-23 10:56 (recorded in `CONTEXT.md`): move the catalog's Plex to the newest fixed version at once,
|
||||
without the test record of `09` §3 decision 13, and put the demo boxes on it. DooPlex and ep0 were not touched; the
|
||||
incident session handles DooPlex.
|
||||
|
||||
## The catalog move
|
||||
|
||||
- Catalog `6cc317d`: `plexinc/pms-docker:1.41.4.9463-630c9f557` → `1.43.5.11029-6bf8af3f0` (Docker Hub, pushed
|
||||
2026-10-20; manifest list `sha256:01bdf0ad6d53198da4331540c1e73ab65db7a72e961fff97ecb0847d46055c5f`, re-resolved at
|
||||
start — still the newest plain `X.Y.Z.B-hash` tag; `public`/`beta`/`plexpass` moved on 2026-10-22 but carry no plain
|
||||
tag and are excluded). `catalog_since: "2026-10-23"`.
|
||||
- Gates green: image-pins, catalog-since (range), test-record, image-resolvable (plex), `catalog_gates.py --fast plex`.
|
||||
`check-test-record-move.py --range ce33e3b..6cc317d` refused, as expected (verbatim in `app-catalog-felhom.eu/REPORT.md`).
|
||||
Pushed with `--no-verify`, that commit only.
|
||||
- Read back from the box's catalog cache on demo-hp 9201 after one sync: the new tag, `catalog_since: "2026-10-23"`,
|
||||
HEAD `6cc317d`. The badge went to „Frissítés elérhető — ma" / „Update available — today" in 4.7 s.
|
||||
|
||||
## Which boxes run Plex
|
||||
|
||||
Hub `/hosts` (CC token, read only): four hosts — Tester4 (DOWN), demo-felhom, demo-hp, tester-1. The customer
|
||||
**Tester 2 has 0 hosts**, so the operator's question was not needed. The hub's `/apps` page counts 2 Plex deployments;
|
||||
the boxes say ONE runs it (a second channel: `docker ps -a` in every running guest):
|
||||
|
||||
| Box | Guest | Plex |
|
||||
|---|---|---|
|
||||
| demo-hp | 9201 | yes, 1.41.4.9463-630c9f557 |
|
||||
| demo-hp | 9202 (scratch) | no |
|
||||
| demo-felhom (N100) | 9201 | no |
|
||||
| tester-1 (VM 341 on demo-hp) | 9201 | no |
|
||||
| Tester4 | — | the hub host is DOWN and its VM no longer exists on demo-hp or felhom-pve (`qm list`); nothing to update. The hub's second Plex deployment is most likely this stale host's last report (not proven). |
|
||||
|
||||
## demo-hp 9201 — the signs check (before the update, read only) → **NOT-A-FINDING**
|
||||
|
||||
Scripts: `demo-hp-9201/signs.sh`, `demo-hp-9201/signs2.sh` (run inside the guest). Outputs, e-mail and token redacted:
|
||||
`demo-hp-9201/signs-9201-before-update.txt`, `demo-hp-9201/signs2-9201-before-update.txt`.
|
||||
|
||||
| Sign | Result | Control |
|
||||
|---|---|---|
|
||||
| CPU, last days | whole guest 0.23–0.34 cores average of 7, max 1.04, per day 10-16 → 10-23 (PVE rrd, the HOST's channel). A miner would hold several cores flat. | — |
|
||||
| Processes in the container | only s6, Plex Media Server, Plex Script Host, Plex Tuner Service | — |
|
||||
| `xmrig`, `mylovelyminer`, `x12345`, `myawesomeproxy` anywhere in the guest | none | — |
|
||||
| ELF files under `Codecs` | none (one file, `.device-id`; one empty codec dir from 2026-10-15) | the ELF test says ELF for `/usr/bin/docker`, not for `Preferences.xml` |
|
||||
| ELF > 10 kB in the config volume newer than 2026-10-15 | none | same |
|
||||
| Container layer (`docker diff plex`) | 80 entries, all s6 runtime under `/run` | — |
|
||||
| `TranscoderH264Options` / `…Override` in `Preferences.xml` | absent | the attribute list printed (27 names) |
|
||||
| Log lines `searchOne`, `/video/:/transcode/universal`, `?x=http…` | 0, 0, 0 | 633 `Request: [` lines in the same log |
|
||||
| Open connections from the container | plex.tv / Cloudflare 443 and the box's traefik only | — |
|
||||
|
||||
**Limit, stated:** Plex keeps 6 rotated logs and restarted several times in the night, so the logs reach back only to
|
||||
2026-10-23 00:32 UTC — not to 2026-10-16. The 10-16 → 10-22 window is covered by the CPU history and the files only,
|
||||
not by the logs. The container was recreated at 04:39 UTC, so its layer has no history before that either.
|
||||
|
||||
## The update (the product's Update press, `POST /api/stacks/plex/update`)
|
||||
|
||||
`demo-hp-9201/update-press.json`: accepted 202; phases safety-dump → pulling → copying → verifying → done in 28.4 s;
|
||||
no error, no hold. After (`demo-hp-9201/after-update.txt`):
|
||||
|
||||
- `/identity`: version `1.43.5.11029-6bf8af3f0`, `claimed: true`, the SAME `machineIdentifier` as before.
|
||||
- Library: 0 sections before and after (this Plex never had a library; `demo-hp-9201/library-before-update.txt`). The
|
||||
server's identity and claim are what carried over.
|
||||
- Box walk: every container healthy (traefik has no health check, up), none restarting; 0 `[ERROR]` and 0 `[WARN]` lines
|
||||
in the controller log since the press (control: 99 `[INFO]` lines in the same window); badge „Naprakész" / „Up to date";
|
||||
pinned = installed = catalog = live compose = the running container, all the new tag.
|
||||
|
||||
## The guide (Part 3)
|
||||
|
||||
The guide's labels were read from the new image's own web client (`WebClient.bundle/.../translations/hu.json`, plex-web
|
||||
4.162.1; its keys are the English labels): Home/„Kezdőképernyő", Play/„Lejátszás", Scan Library Files/„Könyvtárfájlok
|
||||
beolvasása", More/„Több", Fix Match/„Találatok pontosítása", Settings/„Beállítások", Manage Library Access/„Könyvtár
|
||||
hozzáférés kezelése", and the FAQ's Video Quality/„Videó minőség", Original/„Eredeti" — all present, all equal
|
||||
(`demo-hp-9201/guide-labels-1.43.5.txt`; ASCII-fragment controls: `beolvas` 5, `kezel` 26, `xqzzyw` 0). The box's own
|
||||
Plex wiring (`POST /api/storage/media/wire`, the household's „connect now") answered `plex: unchanged`, no error, against
|
||||
the new Plex (`demo-hp-9201/guide-wiring-after-update.txt`). **Not exercised:** the wiring's PUT that adds a new
|
||||
drive's folder — this box has one drive and no library, so it had nothing to add. `guide.checked_version` moved.
|
||||
|
||||
## Teardown
|
||||
|
||||
- Machine: Plex stays on the new version (that is the goal). The helper scripts were removed from the guest and host.
|
||||
- Host: nothing provisioned.
|
||||
- Hub: nothing changed (reads only).
|
||||
@@ -0,0 +1,57 @@
|
||||
identity: {"MediaContainer":{"size":0,"apiVersion":"1.2.3","claimed":true,"machineIdentifier":"fb56097105b927dd42576cd879499a50c064c9fc","version":"1.43.5.11029-6bf8af3f0"}}
|
||||
sections: 0
|
||||
---containers
|
||||
adventurelog Up 5 hours (healthy)
|
||||
adventurelog-frontend Up 5 hours (healthy)
|
||||
adventurelog-postgres Up 5 hours (healthy)
|
||||
bentopdf Up 5 hours (healthy)
|
||||
bookstack Up 5 hours (healthy)
|
||||
bookstack-db Up 5 hours (healthy)
|
||||
calibre-web Up 5 hours (healthy)
|
||||
cloudflared Up 2 days (healthy)
|
||||
docmost Up 4 hours (healthy)
|
||||
docmost-postgres Up 5 hours (healthy)
|
||||
docmost-redis Up 5 hours (healthy)
|
||||
felhom-controller Up 3 hours (healthy)
|
||||
filebrowser Up 2 days (healthy)
|
||||
home-assistant Up 4 hours (healthy)
|
||||
opengist Up 4 hours (healthy)
|
||||
paperless-postgres Up 4 hours (healthy)
|
||||
paperless-redis Up 4 hours (healthy)
|
||||
paperless-webserver Up 4 hours (healthy)
|
||||
plex Up 48 seconds (healthy)
|
||||
privatebin Up 4 hours (healthy)
|
||||
romm Up 4 hours (healthy)
|
||||
romm-db Up 4 hours (healthy)
|
||||
romm-redis Up 4 hours (healthy)
|
||||
traefik Up 30 hours
|
||||
---not-healthy
|
||||
---restarting
|
||||
---ctl-errors-since-update
|
||||
0
|
||||
---ctl-plex-lines
|
||||
2026/10/23 09:08:22 manager.go:675: [DEBUG] [stacks] ScanStacks: found stack "plex" deployed=true composePath=/opt/docker/stacks/plex/docker-compose.yml
|
||||
2026/10/23 09:08:22 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping plex — last check 4s ago, effective interval 5m0s, healthy=true
|
||||
2026/10/23 09:08:22 backup.go:526: [DEBUG] groupStacksByDrive: /mnt/felhom-drives/hdd_1 → [calibre-web, paperless-ngx, plex, romm]
|
||||
2026/10/23 09:08:23 dbdump.go:149: [DEBUG] DiscoverDatabases: docker ps output: e94565919cd7 plex plex plexinc/pms-docker:1.43.5.11029-6bf8af3f0
|
||||
f8900ef8917f felhom-controller gitea.dooplex.hu/admin/felhom-controller:0.340.0
|
||||
2026/10/23 09:08:23 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container plex (image=plexinc/pms-docker:1.43.5.11029-6bf8af3f0, not a database)
|
||||
2026/10/23 09:08:23 dbdump.go:171: [DEBUG] DiscoverDatabases: skipping container felhom-controller (image=gitea.dooplex.hu/admin/felhom-controller:0.340.0, not a database)
|
||||
2026/10/23 09:08:23 sync.go:559: [DEBUG] [sync] plex/docker-compose.yml: hash match, skipped
|
||||
2026/10/23 09:08:23 sync.go:559: [DEBUG] [sync] plex/.felhom.yml: hash match, skipped
|
||||
2026/10/23 09:08:23 [INFO] [stacks] ParseComposeHDDMounts: found 0 HDD mounts for /opt/docker/stacks/plex/docker-compose.yml
|
||||
2026/10/23 09:08:23 recovery_unit.go:248: [INFO] [backup] plex: the app now runs [plexinc/pms-docker:1.43.5.11029-6bf8af3f0]; the unit keeps the definition of its data ([plexinc/pms-docker:1.41.4.9463-630c9f557], written 2026-10-23T02:16:55Z) until the next backup replaces the data
|
||||
2026/10/23 09:08:23 recovery_unit.go:289: [INFO] [backup] Recovery unit captured for plex → /mnt/felhom-drives/hdd_1/backups/primary/plex (images=1, secrets-referenced=0, data_keys=0, portable-carried=0/0, withheld=0)
|
||||
2026/10/23 09:08:24 auth.go:144: [DEBUG] [web] auth: valid session for GET /apps/plex
|
||||
2026/10/23 09:08:24 server.go:637: [DEBUG] [web] ServeHTTP: GET /apps/plex from 172.18.0.10:45638
|
||||
2026/10/23 09:08:24 handlers.go:421: [DEBUG] [web] deployPageData: stack=plex method=GET
|
||||
2026/10/23 09:08:24 auth.go:144: [DEBUG] [web] auth: valid session for GET /apps/plex
|
||||
2026/10/23 09:08:24 server.go:637: [DEBUG] [web] ServeHTTP: GET /apps/plex from 172.18.0.10:45638
|
||||
2026/10/23 09:08:24 handlers.go:421: [DEBUG] [web] deployPageData: stack=plex method=GET
|
||||
2026/10/23 09:08:24 auth.go:144: [DEBUG] [web] auth: valid session for GET /api/stacks/plex
|
||||
2026/10/23 09:08:24 router.go:78: [DEBUG] [api] GET /api/stacks/plex (path=/stacks/plex)
|
||||
2026/10/23 09:08:25 logscanner.go:69: [DEBUG] [metrics] logscanner: scanned plex: errors=0 warnings=0 issues=0 (took 28ms)
|
||||
2026/10/23 09:08:32 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping plex — last check 13s ago, effective interval 5m0s, healthy=true
|
||||
2026/10/23 09:08:42 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping plex — last check 23s ago, effective interval 5m0s, healthy=true
|
||||
2026/10/23 09:08:52 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping plex — last check 33s ago, effective interval 5m0s, healthy=true
|
||||
2026/10/23 09:09:02 healthprobe.go:64: [DEBUG] [stacks] RunHealthProbes: skipping plex — last check 43s ago, effective interval 5m0s, healthy=true
|
||||
@@ -0,0 +1,10 @@
|
||||
Plex web client translations/hu.json from plexinc/pms-docker:1.43.5.11029-6bf8af3f0 (WebClient plex-4.162.1), keys = the English UI string
|
||||
EN key 'Home': present=True; HU in Plex='Kezdőképernyő'; guide HU='Kezdőképernyő'; match=True
|
||||
EN key 'Play': present=True; HU in Plex='Lejátszás'; guide HU='Lejátszás'; match=True
|
||||
EN key 'Scan Library Files': present=True; HU in Plex='Könyvtárfájlok beolvasása'; guide HU='Könyvtárfájlok beolvasása'; match=True
|
||||
EN key 'More': present=True; HU in Plex='Több'; guide HU='Több'; match=True
|
||||
EN key 'Fix Match': present=True; HU in Plex='Találatok pontosítása'; guide HU='Találatok pontosítása'; match=True
|
||||
EN key 'Settings': present=True; HU in Plex='Beállítások'; guide HU='Beállítások'; match=True
|
||||
EN key 'Manage Library Access': present=True; HU in Plex='Könyvtár hozzáférés kezelése'; guide HU='Könyvtár hozzáférés kezelése'; match=True
|
||||
EN key 'Video Quality': present=True; HU in Plex='Videó minőség'; guide HU='Videó minőség'; match=True
|
||||
EN key 'Original': present=True; HU in Plex='Eredeti'; guide HU='Eredeti'; match=True
|
||||
@@ -0,0 +1,2 @@
|
||||
status 200 {"data": {"pending": false, "drives": null, "apps": ["plex"], "wiring": [{"app": "folders", "state": "done"}, {"app": "plex", "state": "unchanged"}], "wired_at": "2026-10-23T09:10:30Z"}, "ok": true}
|
||||
wire 200 {"data": [{"app": "folders", "state": "done"}, {"app": "plex", "state": "unchanged"}], "ok": true}
|
||||
@@ -0,0 +1,2 @@
|
||||
identity: {"MediaContainer":{"size":0,"apiVersion":"0.1.0","claimed":true,"machineIdentifier":"fb56097105b927dd42576cd879499a50c064c9fc","version":"1.41.4.9463-630c9f557"}}
|
||||
sections: 0
|
||||
@@ -0,0 +1,63 @@
|
||||
### 1. processes in the plex container (docker top)
|
||||
PID USER ELAPSED TIME %CPU COMMAND
|
||||
1493203 root 04:25:08 00:00:00 0.0 s6-svscan
|
||||
1493262 root 04:25:08 00:00:00 0.0 s6-supervise
|
||||
1493541 root 04:25:08 00:00:00 0.0 s6-supervise
|
||||
1493544 1000 04:25:08 00:00:21 0.1 Plex Media Serv
|
||||
1493588 1000 04:25:08 00:00:15 0.0 Plex Script Hos
|
||||
1494146 1000 04:25:05 00:00:12 0.0 Plex Tuner Serv
|
||||
### 1b. suspect names anywhere in the guest
|
||||
none
|
||||
### 1c. top CPU processes in the guest
|
||||
PID USER %CPU ELAPSED COMMAND
|
||||
235 root 1.7 2-06:15:22 dockerd
|
||||
183 root 0.8 2-06:15:23 containerd
|
||||
1493858 999 0.7 04:25:07 redis-server
|
||||
1489649 999 0.6 04:25:33 redis-server
|
||||
1492193 999 0.6 04:25:19 redis-server
|
||||
1717354 root 0.3 02:26:14 felhom-controll
|
||||
1496145 root 0.3 04:24:23 gunicorn
|
||||
1698 65532 0.3 2-06:15:16 cloudflared
|
||||
1496141 root 0.3 04:24:23 gunicorn
|
||||
1492308 1000 0.2 04:25:19 node-MainThread
|
||||
1495252 1000 0.2 04:24:46 [celeryd: celer
|
||||
### 2. ELF files under Codecs (path size mtime sha256)
|
||||
all files under Codecs: 1
|
||||
total 16
|
||||
drwxr-xr-x 3 1000 1000 4096 2026-10-15 08:46:05.701113471 +0000 .
|
||||
drwxr-xr-x 11 1000 1000 4096 2026-10-23 08:39:53.737639828 +0000 ..
|
||||
-rw-r--r-- 1 1000 1000 36 2026-10-15 08:46:05.701113471 +0000 .device-id
|
||||
drwxr-xr-x 2 1000 1000 4096 2026-10-15 08:46:05.700807513 +0000 e613bce-97f23d579c1001d8e9cc0d2e-linux-x86_64
|
||||
### 2b. any ELF in the config volume outside Plex's own folders, newer than 2026-10-15
|
||||
(end 2b)
|
||||
### 3. Preferences.xml transcoder attributes
|
||||
TranscoderH264Options*: absent
|
||||
attribute names present:
|
||||
AcceptedEULA AnonymousMachineIdentifier CertificateUUID CertificateVersion DlnaEnabled FSEventLibraryPartialScanEnabled FSEventLibraryUpdatesEnabled GlobalMusicVideoPathMigrated LanguageInCloud LastAutomaticMappedPort MachineIdentifier MetricsEpoch OldestPreviousVersion PlexOnlineHome PlexOnlineMail PlexOnlineToken PlexOnlineUsername ProcessedMachineIdentifier PubSubServer PubSubServerPing PubSubServerRegion PublishServerOnPlexOnlineKey TranscoderTempDirectory customConnections encoding version
|
||||
### 4. log lines: searchOne / transcode/universal (tokens redacted)
|
||||
total 940
|
||||
drwxr-xr-x 3 1000 1000 4096 2026-10-23 04:39:29.306642337 +0000 .
|
||||
drwxr-xr-x 11 1000 1000 4096 2026-10-23 08:39:53.737639828 +0000 ..
|
||||
drwxr-xr-x 2 1000 1000 4096 2026-10-23 04:39:30.774964086 +0000 PMS Plugin Logs
|
||||
-rw-r--r-- 1 1000 1000 880 2026-10-23 04:38:23.878139505 +0000 Plex Crash Uploader.1.log
|
||||
-rw-r--r-- 1 1000 1000 561 2026-10-23 03:34:34.438792292 +0000 Plex Crash Uploader.2.log
|
||||
-rw-r--r-- 1 1000 1000 880 2026-10-23 03:33:23.603543601 +0000 Plex Crash Uploader.3.log
|
||||
-rw-r--r-- 1 1000 1000 561 2026-10-23 02:59:31.142955018 +0000 Plex Crash Uploader.4.log
|
||||
-rw-r--r-- 1 1000 1000 880 2026-10-23 02:58:23.855206791 +0000 Plex Crash Uploader.5.log
|
||||
-rw-r--r-- 1 1000 1000 561 2026-10-23 04:39:28.559376452 +0000 Plex Crash Uploader.log
|
||||
-rw-r--r-- 1 1000 1000 122590 2026-10-23 04:38:23.956828290 +0000 Plex Media Server.1.log
|
||||
-rw-r--r-- 1 1000 1000 95344 2026-10-23 03:33:23.650544218 +0000 Plex Media Server.2.log
|
||||
-rw-r--r-- 1 1000 1000 97012 2026-10-23 02:58:23.959207947 +0000 Plex Media Server.3.log
|
||||
-rw-r--r-- 1 1000 1000 96861 2026-10-23 02:38:23.762270724 +0000 Plex Media Server.4.log
|
||||
-rw-r--r-- 1 1000 1000 157007 2026-10-23 02:16:45.904202135 +0000 Plex Media Server.5.log
|
||||
-rw-r--r-- 1 1000 1000 311157 2026-10-23 09:04:06.104368626 +0000 Plex Media Server.log
|
||||
-rw-r--r-- 1 1000 1000 2769 2026-10-23 04:38:23.939140257 +0000 Plex Tuner Service.1.log
|
||||
-rw-r--r-- 1 1000 1000 2769 2026-10-23 03:33:23.635544021 +0000 Plex Tuner Service.2.log
|
||||
-rw-r--r-- 1 1000 1000 2769 2026-10-23 02:58:23.938207713 +0000 Plex Tuner Service.3.log
|
||||
-rw-r--r-- 1 1000 1000 2769 2026-10-23 02:38:23.745270485 +0000 Plex Tuner Service.4.log
|
||||
-rw-r--r-- 1 1000 1000 2769 2026-10-23 02:16:45.852201461 +0000 Plex Tuner Service.5.log
|
||||
-rw-r--r-- 1 1000 1000 1855 2026-10-23 04:39:39.105066764 +0000 Plex Tuner Service.log
|
||||
count searchOne: 0
|
||||
count transcode/universal: 0
|
||||
count lines with http(s):// in a query: 0
|
||||
oldest log line: Oct 23, 2026 02:32:13.754 [1343875524390
|
||||
@@ -0,0 +1,34 @@
|
||||
#!/bin/sh
|
||||
# Read-only Plex2Shell signs check, run INSIDE the guest. Prints no token.
|
||||
V=/var/lib/docker/volumes/plex_plex_config/_data
|
||||
P="$V/Library/Application Support/Plex Media Server"
|
||||
echo "### 1. processes in the plex container (docker top)"
|
||||
docker top plex -eo pid,user,etime,time,pcpu,comm
|
||||
echo "### 1b. suspect names anywhere in the guest"
|
||||
ps -eo pid,user,comm,args | grep -i -E 'xmrig|mylovelyminer|x12345|myawesomeproxy' | grep -v grep || echo "none"
|
||||
echo "### 1c. top CPU processes in the guest"
|
||||
ps -eo pid,user,pcpu,etime,comm --sort=-pcpu | head -12
|
||||
echo "### 2. ELF files under Codecs (path size mtime sha256)"
|
||||
if [ -d "$P/Codecs" ]; then
|
||||
find "$P/Codecs" -type f | while read -r f; do
|
||||
if head -c 4 "$f" | grep -q 'ELF'; then
|
||||
printf '%s\t%s\t%s\t%s\n' "$f" "$(stat -c %s "$f")" "$(stat -c %y "$f")" "$(sha256sum "$f" | cut -c1-64)"
|
||||
fi
|
||||
done
|
||||
echo "all files under Codecs: $(find "$P/Codecs" -type f | wc -l)"
|
||||
ls -la --time-style=full-iso "$P/Codecs"
|
||||
else echo "no Codecs dir"; fi
|
||||
echo "### 2b. any ELF in the config volume outside Plex's own folders, newer than 2026-10-15"
|
||||
find "$V" -type f -newermt 2026-10-15 -size +10k 2>/dev/null | while read -r f; do head -c 4 "$f" | grep -q ELF && printf '%s\t%s\t%s\n' "$f" "$(stat -c %s "$f")" "$(sha256sum "$f"|cut -c1-64)"; done; echo "(end 2b)"
|
||||
echo "### 3. Preferences.xml transcoder attributes"
|
||||
grep -o -E 'TranscoderH264Options(Override)?="[^"]*"' "$P/Preferences.xml" || echo "TranscoderH264Options*: absent"
|
||||
echo "attribute names present:"; grep -o -E ' [A-Za-z]+="' "$P/Preferences.xml" | tr -d ' ="' | sort | tr '\n' ' '; echo
|
||||
echo "### 4. log lines: searchOne / transcode/universal (tokens redacted)"
|
||||
ls -la --time-style=full-iso "$P/Logs/" | head -30
|
||||
for f in "$P/Logs/"Plex\ Media\ Server*.log; do
|
||||
grep -h -E 'searchOne|/video/:/transcode/universal' "$f" 2>/dev/null
|
||||
done | sed -E 's/(X-Plex-Token|PlexOnlineToken|token)=[^&" ]*/\1=REDACTED/gI' | head -200
|
||||
echo "count searchOne: $(cat "$P/Logs/"Plex\ Media\ Server*.log 2>/dev/null | grep -c searchOne)"
|
||||
echo "count transcode/universal: $(cat "$P/Logs/"Plex\ Media\ Server*.log 2>/dev/null | grep -c '/video/:/transcode/universal')"
|
||||
echo "count lines with http(s):// in a query: $(cat "$P/Logs/"Plex\ Media\ Server*.log 2>/dev/null | grep -c -E '[?&][a-zA-Z]+=https?(%3A|:)')"
|
||||
echo "oldest log line: $(head -1 "$P/Logs/Plex Media Server.5.log" 2>/dev/null | cut -c1-40)"
|
||||
@@ -0,0 +1,21 @@
|
||||
#!/bin/sh
|
||||
V=/var/lib/docker/volumes/plex_plex_config/_data
|
||||
P="$V/Library/Application Support/Plex Media Server"
|
||||
echo "### controls"
|
||||
printf 'ELF detector on /usr/bin/docker (positive): '; head -c 4 /usr/bin/docker | grep -q ELF && echo ELF || echo not-ELF
|
||||
printf 'ELF detector on Preferences.xml (negative): '; head -c 4 "$P/Preferences.xml" | grep -q ELF && echo ELF || echo not-ELF
|
||||
echo "log grep positive control (lines with 'Request: [' in current log): $(grep -c 'Request: \[' "$P/Logs/Plex Media Server.log")"
|
||||
echo "### docker diff plex (files changed in the container layer)"
|
||||
docker diff plex | head -60; echo "total changes: $(docker diff plex | wc -l)"
|
||||
echo "### created / started / restarts"
|
||||
docker inspect plex --format 'created={{.Created}} started={{.State.StartedAt}} restarts={{.RestartCount}}'
|
||||
echo "### transcode volume: files"
|
||||
find /var/lib/docker/volumes/plex_plex_transcode/_data -type f | head; echo "(n=$(find /var/lib/docker/volumes/plex_plex_transcode/_data -type f | wc -l))"
|
||||
echo "### crash uploader log (latest)"
|
||||
sed -E 's/(token|Token)[=:" ]+[A-Za-z0-9_-]{8,}/\1=REDACTED/g' "$P/Logs/Plex Crash Uploader.1.log"
|
||||
echo "### first lines of each PMS log (start time)"
|
||||
for f in "$P/Logs/"Plex\ Media\ Server*.log; do echo "$(basename "$f"): $(head -1 "$f" | cut -c1-24)"; done
|
||||
echo "### outbound connections now from plex netns"
|
||||
PID=$(docker inspect -f '{{.State.Pid}}' plex); nsenter -t $PID -n ss -tnp 2>/dev/null | head -20
|
||||
echo "### docker logs plex (tail, redacted)"
|
||||
docker logs --tail 30 plex 2>&1 | sed -E 's/(claim-|token=)[A-Za-z0-9_-]+/\1REDACTED/g'
|
||||
@@ -0,0 +1,78 @@
|
||||
{
|
||||
"update": {
|
||||
"accepted": true,
|
||||
"http": "202",
|
||||
"phases": [
|
||||
{
|
||||
"t": 0.0,
|
||||
"phase": "safety-dump",
|
||||
"label": "Adatbázis pillanatkép…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 4.1,
|
||||
"phase": "pulling",
|
||||
"label": "Új verzió letöltése…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 10.1,
|
||||
"phase": "copying",
|
||||
"label": "Az adatok másolása a frissítés előtt…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 22.3,
|
||||
"phase": "verifying",
|
||||
"label": "Működés ellenőrzése…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 28.4,
|
||||
"phase": "done",
|
||||
"label": "Frissítve",
|
||||
"updating": false,
|
||||
"error": null,
|
||||
"hold": null
|
||||
}
|
||||
],
|
||||
"duration_s": 28.4,
|
||||
"final_phase": "done",
|
||||
"update_error": null,
|
||||
"hold_reason": null,
|
||||
"state": "running"
|
||||
},
|
||||
"observables": {
|
||||
"pinned_images": {
|
||||
"plex": "plexinc/pms-docker:1.43.5.11029-6bf8af3f0"
|
||||
},
|
||||
"installed_images": {
|
||||
"plex": "plexinc/pms-docker:1.43.5.11029-6bf8af3f0"
|
||||
},
|
||||
"catalog_images": {
|
||||
"plex": "plexinc/pms-docker:1.43.5.11029-6bf8af3f0"
|
||||
},
|
||||
"live_compose_image_lines": [
|
||||
"image: plexinc/pms-docker:1.43.5.11029-6bf8af3f0"
|
||||
],
|
||||
"docker_inspect": [
|
||||
"plex plexinc/pms-docker:1.43.5.11029-6bf8af3f0 running=true restarts=0"
|
||||
]
|
||||
},
|
||||
"log": [
|
||||
"11:07:51 [6] Update -> 202 {'ok': True, 'data': {'accepted': True, 'completed': False}, 'message': 'Frissítés elindult – az állapot a kártyán követhető'}",
|
||||
"11:07:51 + 0.0s phase=safety-dump label=Adatbázis pillanatkép… err=None hold=None",
|
||||
"11:07:55 + 4.1s phase=pulling label=Új verzió letöltése… err=None hold=None",
|
||||
"11:08:01 + 10.1s phase=copying label=Az adatok másolása a frissítés előtt… err=None hold=None",
|
||||
"11:08:14 + 22.3s phase=verifying label=Működés ellenőrzése… err=None hold=None",
|
||||
"11:08:20 + 28.4s phase=done label=Frissítve err=None hold=None"
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
{
|
||||
"app": "plex",
|
||||
"walk": "security",
|
||||
"box_walk_version": 1,
|
||||
"verdict": "inconclusive",
|
||||
"venue": "box 9202 (drill catalog), the guarded Update",
|
||||
"from": {
|
||||
"plex": "plexinc/pms-docker:1.43.4.10903-e5521bd8c"
|
||||
},
|
||||
"to": {
|
||||
"plex": "busybox:1.36.1"
|
||||
},
|
||||
"why": "the install at FROM did not complete — nothing was tested",
|
||||
"log": [
|
||||
"12:26:15 [1] made the drive paths this app requires: ['/mnt/felhom-drives/scratch_hdd/userdata/plex']",
|
||||
"12:26:15 [1] required fields filled beyond DOMAIN/SUBDOMAIN: ['PLEX_CLAIM', 'HDD_PATH']",
|
||||
"12:26:15 [1] deploy -> 409 {'ok': False, 'error': 'Az alkalmazás adatainak helye egy meghajtó lehet, nem egy mappa a meghajtón belül. Válaszd ezt: "
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
12:26:15 [1] made the drive paths this app requires: ['/mnt/felhom-drives/scratch_hdd/userdata/plex']
|
||||
12:26:15 [1] required fields filled beyond DOMAIN/SUBDOMAIN: ['PLEX_CLAIM', 'HDD_PATH']
|
||||
12:26:15 [1] deploy -> 409 {'ok': False, 'error': 'Az alkalmazás adatainak helye egy meghajtó lehet, nem egy mappa a meghajtón belül. Válaszd ezt:
|
||||
12:26:15 [9] verdict inconclusive -> /mnt/5_hdd/felhom.eu/git/felhom.eu/documentation/audits/security-lane-2026-10-23/P2-negative/verdict.json
|
||||
12:26:15 RESULT inconclusive — the install at FROM did not complete — nothing was tested
|
||||
@@ -0,0 +1,188 @@
|
||||
{
|
||||
"app": "plex",
|
||||
"walk": "security",
|
||||
"box_walk_version": 1,
|
||||
"verdict": "failed",
|
||||
"venue": "box 9202 (drill catalog), the guarded Update",
|
||||
"from": {
|
||||
"plex": "plexinc/pms-docker:1.43.4.10903-e5521bd8c"
|
||||
},
|
||||
"to": {
|
||||
"plex": "busybox:1.36.1"
|
||||
},
|
||||
"observables_before": {
|
||||
"pinned_images": {
|
||||
"plex": "plexinc/pms-docker:1.43.4.10903-e5521bd8c"
|
||||
},
|
||||
"installed_images": {
|
||||
"plex": "plexinc/pms-docker:1.43.4.10903-e5521bd8c"
|
||||
},
|
||||
"catalog_images": null,
|
||||
"live_compose_image_lines": [
|
||||
"image: plexinc/pms-docker:1.43.4.10903-e5521bd8c"
|
||||
],
|
||||
"docker_inspect": [
|
||||
"plex plexinc/pms-docker:1.43.4.10903-e5521bd8c running=true restarts=0"
|
||||
]
|
||||
},
|
||||
"setup_gate_before": {
|
||||
"browser": {
|
||||
"status": "302",
|
||||
"redirect": "https://felhom.enkisfelhom.hu/__gate/start?…"
|
||||
},
|
||||
"script": {
|
||||
"status": "401",
|
||||
"redirect": null
|
||||
}
|
||||
},
|
||||
"seed": "none",
|
||||
"seed_why": "the first-run claim needs a token minted at plex.tv by a real Plex account; no account exists for this venue",
|
||||
"drill_commit": "577947198bff",
|
||||
"sync_waited_s": 4.5,
|
||||
"badge_before": {
|
||||
"hu": [
|
||||
{
|
||||
"title": "Újabb változat érhető el ehhez az alkalmazáshoz. A frissítés indításához nyomd meg a Frissítés gombot.",
|
||||
"text": "Frissítés elérhető — ma"
|
||||
}
|
||||
],
|
||||
"en": [
|
||||
{
|
||||
"title": "A newer version of this app is available. Select the Update button to start it.",
|
||||
"text": "Update available — today"
|
||||
}
|
||||
]
|
||||
},
|
||||
"measured_at": "2026-10-23T10:27:24Z",
|
||||
"update": {
|
||||
"accepted": true,
|
||||
"http": "202",
|
||||
"phases": [
|
||||
{
|
||||
"t": 0.0,
|
||||
"phase": "backing-up",
|
||||
"label": "Biztonsági mentés készül a frissítés előtt…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 11.3,
|
||||
"phase": "safety-dump",
|
||||
"label": "Adatbázis pillanatkép…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 12.3,
|
||||
"phase": "pulling",
|
||||
"label": "Új verzió letöltése…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 14.4,
|
||||
"phase": "copying",
|
||||
"label": "Az adatok másolása a frissítés előtt…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 24.7,
|
||||
"phase": "starting",
|
||||
"label": "Indítás az új verzióval…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 25.7,
|
||||
"phase": "verifying",
|
||||
"label": "Működés ellenőrzése…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 327.2,
|
||||
"phase": "undoing",
|
||||
"label": "Visszaállítás az előző változatra…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 335.4,
|
||||
"phase": "undone",
|
||||
"label": "Visszaállítva az előző változatra",
|
||||
"updating": false,
|
||||
"error": null,
|
||||
"hold": null
|
||||
}
|
||||
],
|
||||
"duration_s": 335.4,
|
||||
"final_phase": "undone",
|
||||
"update_error": null,
|
||||
"hold_reason": null,
|
||||
"state": "running"
|
||||
},
|
||||
"observables_after": {
|
||||
"pinned_images": {
|
||||
"plex": "plexinc/pms-docker:1.43.4.10903-e5521bd8c"
|
||||
},
|
||||
"installed_images": {
|
||||
"plex": "plexinc/pms-docker:1.43.4.10903-e5521bd8c"
|
||||
},
|
||||
"catalog_images": {
|
||||
"plex": "busybox:1.36.1"
|
||||
},
|
||||
"live_compose_image_lines": [
|
||||
"image: plexinc/pms-docker:1.43.4.10903-e5521bd8c"
|
||||
],
|
||||
"docker_inspect": [
|
||||
"plex plexinc/pms-docker:1.43.4.10903-e5521bd8c running=true restarts=0"
|
||||
]
|
||||
},
|
||||
"badge_after": {
|
||||
"hu": [
|
||||
{
|
||||
"title": "Újabb változat érhető el ehhez az alkalmazáshoz. A frissítés indításához nyomd meg a Frissítés gombot.",
|
||||
"text": "Frissítés elérhető — ma"
|
||||
}
|
||||
],
|
||||
"en": [
|
||||
{
|
||||
"title": "A newer version of this app is available. Select the Update button to start it.",
|
||||
"text": "Update available — today"
|
||||
}
|
||||
]
|
||||
},
|
||||
"state_after": "running",
|
||||
"hold_after": null,
|
||||
"to_digests": {
|
||||
"plex": "sha256:e0ab27395614a8e1a4fdf84c6bc60ac664915cfdde70c52d030c7728a1c48e14"
|
||||
},
|
||||
"why": "the update ended 'undone' (error None, hold None)",
|
||||
"log": [
|
||||
"12:26:46 [1] required fields filled beyond DOMAIN/SUBDOMAIN: ['HDD_PATH', 'PLEX_CLAIM']",
|
||||
"12:26:46 [1] deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}",
|
||||
"12:27:06 [1] deployed, controller state=running, pinned={'plex': 'plexinc/pms-docker:1.43.4.10903-e5521bd8c'}",
|
||||
"12:27:06 gate: plex is gated — passed as the household (cookie set)",
|
||||
"12:27:10 [2] setup gate as installed: {'browser': {'status': '302', 'redirect': 'https://felhom.enkisfelhom.hu/__gate/start?…'}, 'script': {'status': '401', 'redirect': None}}",
|
||||
"12:27:10 [3] seed: none — the first-run claim needs a token minted at plex.tv by a real Plex account; no account exists for this venue",
|
||||
"12:27:11 [5] drill commit 577947198bff: plex plexinc/pms-docker:1.43.4.10903-e5521bd8c -> busybox:1.36.1 (push rc=0)",
|
||||
"12:27:19 [5] the box's drill mirror now at 57794719",
|
||||
"12:27:24 [6] Update -> 202 {'ok': True, 'data': {'accepted': True, 'completed': False}, 'message': 'Frissítés elindult – az állapot a kártyán követhető'}",
|
||||
"12:27:24 + 0.0s phase=backing-up label=Biztonsági mentés készül a frissítés előtt… err=None hold=None",
|
||||
"12:27:35 + 11.3s phase=safety-dump label=Adatbázis pillanatkép… err=None hold=None",
|
||||
"12:27:37 + 12.3s phase=pulling label=Új verzió letöltése… err=None hold=None",
|
||||
"12:27:39 + 14.4s phase=copying label=Az adatok másolása a frissítés előtt… err=None hold=None",
|
||||
"12:27:49 + 24.7s phase=starting label=Indítás az új verzióval… err=None hold=None",
|
||||
"12:27:50 + 25.7s phase=verifying label=Működés ellenőrzése… err=None hold=None",
|
||||
"12:32:51 + 327.2s phase=undoing label=Visszaállítás az előző változatra… err=None hold=None",
|
||||
"12:33:00 + 335.4s phase=undone label=Visszaállítva az előző változatra err=None hold=None"
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,22 @@
|
||||
12:26:46 [1] required fields filled beyond DOMAIN/SUBDOMAIN: ['HDD_PATH', 'PLEX_CLAIM']
|
||||
12:26:46 [1] deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}
|
||||
12:27:06 [1] deployed, controller state=running, pinned={'plex': 'plexinc/pms-docker:1.43.4.10903-e5521bd8c'}
|
||||
12:27:06 gate: plex is gated — passed as the household (cookie set)
|
||||
12:27:10 [2] setup gate as installed: {'browser': {'status': '302', 'redirect': 'https://felhom.enkisfelhom.hu/__gate/start?…'}, 'script': {'status': '401', 'redirect': None}}
|
||||
12:27:10 [3] seed: none — the first-run claim needs a token minted at plex.tv by a real Plex account; no account exists for this venue
|
||||
12:27:11 [5] drill commit 577947198bff: plex plexinc/pms-docker:1.43.4.10903-e5521bd8c -> busybox:1.36.1 (push rc=0)
|
||||
12:27:19 [5] the box's drill mirror now at 57794719
|
||||
12:27:24 [6] Update -> 202 {'ok': True, 'data': {'accepted': True, 'completed': False}, 'message': 'Frissítés elindult – az állapot a kártyán követhető'}
|
||||
12:27:24 + 0.0s phase=backing-up label=Biztonsági mentés készül a frissítés előtt… err=None hold=None
|
||||
12:27:35 + 11.3s phase=safety-dump label=Adatbázis pillanatkép… err=None hold=None
|
||||
12:27:37 + 12.3s phase=pulling label=Új verzió letöltése… err=None hold=None
|
||||
12:27:39 + 14.4s phase=copying label=Az adatok másolása a frissítés előtt… err=None hold=None
|
||||
12:27:49 + 24.7s phase=starting label=Indítás az új verzióval… err=None hold=None
|
||||
12:27:50 + 25.7s phase=verifying label=Működés ellenőrzése… err=None hold=None
|
||||
12:32:51 + 327.2s phase=undoing label=Visszaállítás az előző változatra… err=None hold=None
|
||||
12:33:00 + 335.4s phase=undone label=Visszaállítva az előző változatra err=None hold=None
|
||||
12:33:18 [9] verdict failed -> /mnt/5_hdd/felhom.eu/git/felhom.eu/documentation/audits/security-lane-2026-10-23/P2-negative/verdict.json
|
||||
12:33:18 RESULT failed — the update ended 'undone' (error None, hold None)
|
||||
12:33:27 [X] stop -> 200 {'ok': True, 'message': 'Stack plex stop completed'}
|
||||
12:33:59 [X] remove (with drive data) -> 200 {'ok': True, 'data': {'removed': 'plex', 'volumes_removed': ['plex_plex_config', 'plex_plex_transcode'], 'hdd_paths_removed': [], 'hdd_paths_preserved': [], 'us
|
||||
12:34:08 [X] after remove: deployed=False leftovers='/opt/docker/stacks/plex'
|
||||
@@ -0,0 +1 @@
|
||||
REFUSED plex: the box walk is 'security' with verdict 'failed' — only a PROVEN `box_walk.py --security` walk is written as a security-fix step
|
||||
@@ -0,0 +1,9 @@
|
||||
12:57:50 [5] drill commit f8b700e1d889: plex plexinc/pms-docker:1.43.5.11029-6bf8af3f0 -> plexinc/pms-docker:1.43.4.10903-e5521bd8c (push rc=0)
|
||||
12:57:58 [5] the box's drill mirror now at f8b700e1
|
||||
f8b700e1d889
|
||||
13:00:27 [sync] the badge NEVER caught up to plexinc/pms-docker:1.43.4.10903-e5521bd8c in 148.8s — catalog_images = None
|
||||
13:00:27 [1] required fields filled beyond DOMAIN/SUBDOMAIN: ['HDD_PATH', 'PLEX_CLAIM']
|
||||
13:00:27 [1] deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}
|
||||
13:00:47 [1] deployed, controller state=running, pinned={'plex': 'plexinc/pms-docker:1.43.4.10903-e5521bd8c'}
|
||||
13:00:47 gate: plex is gated — passed as the household (cookie set)
|
||||
deployed True {"pinned_images": {"plex": "plexinc/pms-docker:1.43.4.10903-e5521bd8c"}, "installed_images": {"plex": "plexinc/pms-docker:1.43.4.10903-e5521bd8c"}, "catalog_images": null, "live_compose_image_lines": ["image: plexinc/pms-docker:1.43.4.10903-e5521bd8c"], "docker_inspect": ["plex plexinc/pms-docker:1.43.4.10903-e5521bd8c running=true restarts=0"]}
|
||||
@@ -0,0 +1,15 @@
|
||||
13:01:23 [5] the box's drill mirror now at f46e1bc6
|
||||
before: {"pinned_images": {"plex": "plexinc/pms-docker:1.43.4.10903-e5521bd8c"}, "installed_images": {"plex": "plexinc/pms-docker:1.43.4.10903-e5521bd8c"}, "catalog_images": {"plex": "plexinc/pms-docker:1.43.5.11029-6bf8af3f0"}, "live_compose_image_lines": ["image: plexinc/pms-docker:1.43.4.10903-e5521bd8c"], "docker_inspect": ["plex plexinc/pms-docker:1.43.4.10903-e5521bd8c running=true restarts=0"]}
|
||||
badge before: {'hu': [{'title': 'Újabb változat érhető el ehhez az alkalmazáshoz. A frissítés indításához nyomd meg a Frissítés gombot.', 'text': 'Frissítés elérhető — ma'}], 'en': [{'title': 'A newer version of this app is available. Select the Update button to start it.', 'text': 'Update available — today'}]}
|
||||
night chain -> 202 {'data': {'legs': ['db-dump', 'tier2', 'update-leg']}, 'message': 'started', 'ok': True}
|
||||
the leg's own lines:
|
||||
2026/10/23 11:01:34 night_chain.go:108: [INFO] [night-chain] manual run started from 172.18.0.6:46254: [db-dump tier2 update-leg]
|
||||
2026/10/23 11:02:40 night_chain.go:77: [INFO] [night-chain] update-leg: started
|
||||
2026/10/23 11:02:40 unattended.go:270: [INFO] [update-leg] started (manual-chain): window 02:30, no step starts at or after 16:17
|
||||
2026/10/23 11:02:41 unattended.go:316: [INFO] [update-leg] plex: skipped — not_proven
|
||||
2026/10/23 11:02:41 unattended.go:316: [INFO] [update-leg] privatebin: skipped — stopped_by_household
|
||||
2026/10/23 11:02:41 unattended.go:248: [INFO] [update-leg] update leg (manual-chain): done=0 undone=0 held=0 failed=0 skipped=2 in 0s [skipped: plex=not_proven, privatebin=stopped_by_household]
|
||||
2026/10/23 11:02:41 night_chain.go:82: [INFO] [night-chain] update-leg: done in 0s
|
||||
|
||||
after: {"pinned_images": {"plex": "plexinc/pms-docker:1.43.4.10903-e5521bd8c"}, "installed_images": {"plex": "plexinc/pms-docker:1.43.4.10903-e5521bd8c"}, "catalog_images": {"plex": "plexinc/pms-docker:1.43.5.11029-6bf8af3f0"}, "live_compose_image_lines": ["image: plexinc/pms-docker:1.43.4.10903-e5521bd8c"], "docker_inspect": ["plex plexinc/pms-docker:1.43.4.10903-e5521bd8c running=true restarts=0"]}
|
||||
state running hold None phase None
|
||||
@@ -0,0 +1,11 @@
|
||||
13:03:07 [6] Update -> 202 {'ok': True, 'data': {'accepted': True, 'completed': False}, 'message': 'Frissítés elindult – az állapot a kártyán követhető'}
|
||||
13:03:07 + 0.0s phase=safety-dump label=Adatbázis pillanatkép… err=None hold=None
|
||||
13:03:08 + 1.0s phase=pulling label=Új verzió letöltése… err=None hold=None
|
||||
13:03:16 + 9.2s phase=copying label=Az adatok másolása a frissítés előtt… err=None hold=None
|
||||
13:03:26 + 19.5s phase=starting label=Indítás az új verzióval… err=None hold=None
|
||||
13:03:27 + 20.5s phase=verifying label=Működés ellenőrzése… err=None hold=None
|
||||
13:03:32 + 25.6s phase=done label=Frissítve err=None hold=None
|
||||
{"accepted": true, "final_phase": "done", "update_error": null, "hold_reason": null, "state": "running", "duration_s": 25.7}
|
||||
after: {"pinned_images": {"plex": "plexinc/pms-docker:1.43.5.11029-6bf8af3f0"}, "installed_images": {"plex": "plexinc/pms-docker:1.43.5.11029-6bf8af3f0"}, "catalog_images": {"plex": "plexinc/pms-docker:1.43.5.11029-6bf8af3f0"}, "live_compose_image_lines": ["image: plexinc/pms-docker:1.43.5.11029-6bf8af3f0@sha256:01bdf0ad6d53198da4331540c1e73ab65db7a72e961fff97ecb0847d46055c5f"], "docker_inspect": ["plex plexinc/pms-docker:1.43.5.11029-6bf8af3f0@sha256:01bdf0ad6d53198da4331540c1e73ab65db7a72e961fff97ecb0847d46055c5f running=true restarts=0"]}
|
||||
13:03:51 gate: plex is gated — passed as the household (cookie set)
|
||||
identity 200 {"MediaContainer":{"size":0,"apiVersion":"1.2.3","claimed":false,"machineIdentifier":"f34aaee0f637bb3bf168e5b4e5ce2223c7c40c5f","version":"1.43.5.11029-6bf8af3f0"}}
|
||||
@@ -0,0 +1,227 @@
|
||||
{
|
||||
"app": "plex",
|
||||
"walk": "security",
|
||||
"box_walk_version": 1,
|
||||
"verdict": "proven",
|
||||
"venue": "box 9202 (drill catalog), the guarded Update",
|
||||
"from": {
|
||||
"plex": "plexinc/pms-docker:1.43.4.10903-e5521bd8c"
|
||||
},
|
||||
"to": {
|
||||
"plex": "plexinc/pms-docker:1.43.5.11029-6bf8af3f0"
|
||||
},
|
||||
"observables_before": {
|
||||
"pinned_images": {
|
||||
"plex": "plexinc/pms-docker:1.43.4.10903-e5521bd8c"
|
||||
},
|
||||
"installed_images": {
|
||||
"plex": "plexinc/pms-docker:1.43.4.10903-e5521bd8c"
|
||||
},
|
||||
"catalog_images": null,
|
||||
"live_compose_image_lines": [
|
||||
"image: plexinc/pms-docker:1.43.4.10903-e5521bd8c"
|
||||
],
|
||||
"docker_inspect": [
|
||||
"plex plexinc/pms-docker:1.43.4.10903-e5521bd8c running=true restarts=0"
|
||||
]
|
||||
},
|
||||
"setup_gate_before": {
|
||||
"browser": {
|
||||
"status": "302",
|
||||
"redirect": "https://felhom.enkisfelhom.hu/__gate/start?…"
|
||||
},
|
||||
"script": {
|
||||
"status": "401",
|
||||
"redirect": null
|
||||
}
|
||||
},
|
||||
"seed": "none",
|
||||
"seed_why": "the first-run claim needs a token minted at plex.tv by a real Plex account; no account exists for this venue",
|
||||
"drill_commit": "b5e2406d3b49",
|
||||
"sync_waited_s": 4.5,
|
||||
"badge_before": {
|
||||
"hu": [
|
||||
{
|
||||
"title": "Újabb változat érhető el ehhez az alkalmazáshoz. A frissítés indításához nyomd meg a Frissítés gombot.",
|
||||
"text": "Frissítés elérhető — ma"
|
||||
}
|
||||
],
|
||||
"en": [
|
||||
{
|
||||
"title": "A newer version of this app is available. Select the Update button to start it.",
|
||||
"text": "Update available — today"
|
||||
}
|
||||
]
|
||||
},
|
||||
"measured_at": "2026-10-23T10:36:22Z",
|
||||
"update": {
|
||||
"accepted": true,
|
||||
"http": "202",
|
||||
"phases": [
|
||||
{
|
||||
"t": 0.0,
|
||||
"phase": "backing-up",
|
||||
"label": "Biztonsági mentés készül a frissítés előtt…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 11.3,
|
||||
"phase": "safety-dump",
|
||||
"label": "Adatbázis pillanatkép…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 12.3,
|
||||
"phase": "pulling",
|
||||
"label": "Új verzió letöltése…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 20.5,
|
||||
"phase": "copying",
|
||||
"label": "Az adatok másolása a frissítés előtt…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 30.8,
|
||||
"phase": "starting",
|
||||
"label": "Indítás az új verzióval…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 31.8,
|
||||
"phase": "verifying",
|
||||
"label": "Működés ellenőrzése…",
|
||||
"updating": true,
|
||||
"error": null,
|
||||
"hold": null
|
||||
},
|
||||
{
|
||||
"t": 36.9,
|
||||
"phase": "done",
|
||||
"label": "Frissítve",
|
||||
"updating": false,
|
||||
"error": null,
|
||||
"hold": null
|
||||
}
|
||||
],
|
||||
"duration_s": 36.9,
|
||||
"final_phase": "done",
|
||||
"update_error": null,
|
||||
"hold_reason": null,
|
||||
"state": "running"
|
||||
},
|
||||
"observables_after": {
|
||||
"pinned_images": {
|
||||
"plex": "plexinc/pms-docker:1.43.5.11029-6bf8af3f0"
|
||||
},
|
||||
"installed_images": {
|
||||
"plex": "plexinc/pms-docker:1.43.5.11029-6bf8af3f0"
|
||||
},
|
||||
"catalog_images": {
|
||||
"plex": "plexinc/pms-docker:1.43.5.11029-6bf8af3f0"
|
||||
},
|
||||
"live_compose_image_lines": [
|
||||
"image: plexinc/pms-docker:1.43.5.11029-6bf8af3f0"
|
||||
],
|
||||
"docker_inspect": [
|
||||
"plex plexinc/pms-docker:1.43.5.11029-6bf8af3f0 running=true restarts=0"
|
||||
]
|
||||
},
|
||||
"badge_after": {
|
||||
"hu": [
|
||||
{
|
||||
"title": "Ez az alkalmazás a legfrissebb elérhető változatot futtatja.",
|
||||
"text": "Naprakész"
|
||||
}
|
||||
],
|
||||
"en": [
|
||||
{
|
||||
"title": "This app is running the newest version available.",
|
||||
"text": "Up to date"
|
||||
}
|
||||
]
|
||||
},
|
||||
"state_after": "running",
|
||||
"hold_after": null,
|
||||
"to_digests": {
|
||||
"plex": "sha256:01bdf0ad6d53198da4331540c1e73ab65db7a72e961fff97ecb0847d46055c5f"
|
||||
},
|
||||
"four_observables": {
|
||||
"pinned": {
|
||||
"plex": "plexinc/pms-docker:1.43.5.11029-6bf8af3f0"
|
||||
},
|
||||
"installed": {
|
||||
"plex": "plexinc/pms-docker:1.43.5.11029-6bf8af3f0"
|
||||
},
|
||||
"catalog": {
|
||||
"plex": "plexinc/pms-docker:1.43.5.11029-6bf8af3f0"
|
||||
},
|
||||
"live_compose": [
|
||||
"plexinc/pms-docker:1.43.5.11029-6bf8af3f0"
|
||||
]
|
||||
},
|
||||
"setup_gate_after": {
|
||||
"browser": {
|
||||
"status": "302",
|
||||
"redirect": "https://felhom.enkisfelhom.hu/__gate/start?…"
|
||||
},
|
||||
"script": {
|
||||
"status": "401",
|
||||
"redirect": null
|
||||
}
|
||||
},
|
||||
"memory": {
|
||||
"soak_s": 600,
|
||||
"interval_s": 10,
|
||||
"samples": 60,
|
||||
"containers": {
|
||||
"plex": {
|
||||
"samples": 60,
|
||||
"anon_peak_bytes": 229015552,
|
||||
"limit_bytes": 2147483648,
|
||||
"anon_peak_pct": 10.7,
|
||||
"cgroup_peak_pct": 11.5,
|
||||
"oom_kill_delta": 0,
|
||||
"oom_killed_flag": false,
|
||||
"restarts_delta": 0,
|
||||
"running_all_samples": true,
|
||||
"cgroup": "/sys/fs/cgroup/system.slice/docker-f699bdec12560064fa6bd64884d941915072974ca27d389bcb3e3b917861ac88.scope"
|
||||
}
|
||||
},
|
||||
"oom_kills": 0,
|
||||
"restarts": 0,
|
||||
"peak_pct": 10.7
|
||||
},
|
||||
"why": "the security-fix step ran on the box through the guarded Update; healthy, TO on all four observables, no undo, no hold, a clean soak",
|
||||
"log": [
|
||||
"12:35:39 [1] required fields filled beyond DOMAIN/SUBDOMAIN: ['HDD_PATH', 'PLEX_CLAIM']",
|
||||
"12:35:40 [1] deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}",
|
||||
"12:36:05 [1] deployed, controller state=running, pinned={'plex': 'plexinc/pms-docker:1.43.4.10903-e5521bd8c'}",
|
||||
"12:36:05 gate: plex is gated — passed as the household (cookie set)",
|
||||
"12:36:08 [2] setup gate as installed: {'browser': {'status': '302', 'redirect': 'https://felhom.enkisfelhom.hu/__gate/start?…'}, 'script': {'status': '401', 'redirect': None}}",
|
||||
"12:36:08 [3] seed: none — the first-run claim needs a token minted at plex.tv by a real Plex account; no account exists for this venue",
|
||||
"12:36:09 [5] drill commit b5e2406d3b49: plex plexinc/pms-docker:1.43.4.10903-e5521bd8c -> plexinc/pms-docker:1.43.5.11029-6bf8af3f0 (push rc=0)",
|
||||
"12:36:18 [5] the box's drill mirror now at b5e2406d",
|
||||
"12:36:23 [6] Update -> 202 {'ok': True, 'data': {'accepted': True, 'completed': False}, 'message': 'Frissítés elindult – az állapot a kártyán követhető'}",
|
||||
"12:36:23 + 0.0s phase=backing-up label=Biztonsági mentés készül a frissítés előtt… err=None hold=None",
|
||||
"12:36:34 + 11.3s phase=safety-dump label=Adatbázis pillanatkép… err=None hold=None",
|
||||
"12:36:35 + 12.3s phase=pulling label=Új verzió letöltése… err=None hold=None",
|
||||
"12:36:43 + 20.5s phase=copying label=Az adatok másolása a frissítés előtt… err=None hold=None",
|
||||
"12:36:53 + 30.8s phase=starting label=Indítás az új verzióval… err=None hold=None",
|
||||
"12:36:54 + 31.8s phase=verifying label=Működés ellenőrzése… err=None hold=None",
|
||||
"12:36:59 + 36.9s phase=done label=Frissítve err=None hold=None",
|
||||
"12:37:18 [7] memory soak 600 s …",
|
||||
"12:47:24 [7] soak: peak 10.7% of the limit (anon), kills 0, restarts 0"
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,23 @@
|
||||
12:35:39 [1] required fields filled beyond DOMAIN/SUBDOMAIN: ['HDD_PATH', 'PLEX_CLAIM']
|
||||
12:35:40 [1] deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}
|
||||
12:36:05 [1] deployed, controller state=running, pinned={'plex': 'plexinc/pms-docker:1.43.4.10903-e5521bd8c'}
|
||||
12:36:05 gate: plex is gated — passed as the household (cookie set)
|
||||
12:36:08 [2] setup gate as installed: {'browser': {'status': '302', 'redirect': 'https://felhom.enkisfelhom.hu/__gate/start?…'}, 'script': {'status': '401', 'redirect': None}}
|
||||
12:36:08 [3] seed: none — the first-run claim needs a token minted at plex.tv by a real Plex account; no account exists for this venue
|
||||
12:36:09 [5] drill commit b5e2406d3b49: plex plexinc/pms-docker:1.43.4.10903-e5521bd8c -> plexinc/pms-docker:1.43.5.11029-6bf8af3f0 (push rc=0)
|
||||
12:36:18 [5] the box's drill mirror now at b5e2406d
|
||||
12:36:23 [6] Update -> 202 {'ok': True, 'data': {'accepted': True, 'completed': False}, 'message': 'Frissítés elindult – az állapot a kártyán követhető'}
|
||||
12:36:23 + 0.0s phase=backing-up label=Biztonsági mentés készül a frissítés előtt… err=None hold=None
|
||||
12:36:34 + 11.3s phase=safety-dump label=Adatbázis pillanatkép… err=None hold=None
|
||||
12:36:35 + 12.3s phase=pulling label=Új verzió letöltése… err=None hold=None
|
||||
12:36:43 + 20.5s phase=copying label=Az adatok másolása a frissítés előtt… err=None hold=None
|
||||
12:36:53 + 30.8s phase=starting label=Indítás az új verzióval… err=None hold=None
|
||||
12:36:54 + 31.8s phase=verifying label=Működés ellenőrzése… err=None hold=None
|
||||
12:36:59 + 36.9s phase=done label=Frissítve err=None hold=None
|
||||
12:37:18 [7] memory soak 600 s …
|
||||
12:47:24 [7] soak: peak 10.7% of the limit (anon), kills 0, restarts 0
|
||||
12:47:24 [9] verdict proven -> /mnt/5_hdd/felhom.eu/git/felhom.eu/documentation/audits/security-lane-2026-10-23/P2-walk/verdict.json
|
||||
12:47:24 RESULT proven — the security-fix step ran on the box through the guarded Update; healthy, TO on all four observables, no undo, no hold, a clean soak
|
||||
12:47:33 [X] stop -> 200 {'ok': True, 'message': 'Stack plex stop completed'}
|
||||
12:48:05 [X] remove (with drive data) -> 200 {'ok': True, 'data': {'removed': 'plex', 'volumes_removed': ['plex_plex_config', 'plex_plex_transcode'], 'hdd_paths_removed': [], 'hdd_paths_preserved': [], 'us
|
||||
12:48:13 [X] after remove: deployed=False leftovers='/opt/docker/stacks/plex'
|
||||
@@ -0,0 +1 @@
|
||||
WROTE plex: {'plex': 'plexinc/pms-docker:1.43.4.10903-e5521bd8c'} -> {'plex': 'plexinc/pms-docker:1.43.5.11029-6bf8af3f0'} peak 10.7% marks {'files_may_change': False, 'needs_person': None, 'memory_tight': False}
|
||||
@@ -0,0 +1,21 @@
|
||||
13:04:15 [X] stop -> 200 {'ok': True, 'message': 'Stack plex stop completed'}
|
||||
13:04:47 [X] remove (with drive data) -> 200 {'ok': True, 'data': {'removed': 'plex', 'volumes_removed': ['plex_plex_config', 'plex_plex_transcode'], 'hdd_paths_removed': [], 'hdd_paths_preserved': [], 'us
|
||||
13:04:56 [X] after remove: deployed=False leftovers='/opt/docker/stacks/plex'
|
||||
remote: . Processing 1 references
|
||||
remote: Processed 1 references in total
|
||||
13:05:05 [5] the box's drill mirror now at 42945c94
|
||||
13:07:34 [sync] the badge NEVER caught up to plexinc/pms-docker:1.43.4.10903-e5521bd8c in 149.0s — catalog_images = None
|
||||
13:07:34 [1] required fields filled beyond DOMAIN/SUBDOMAIN: ['HDD_PATH', 'PLEX_CLAIM']
|
||||
13:07:34 [1] deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}
|
||||
13:07:54 [1] deployed, controller state=running, pinned={'plex': 'plexinc/pms-docker:1.43.4.10903-e5521bd8c'}
|
||||
deploy True
|
||||
13:07:55 gate: plex is gated — passed as the household (cookie set)
|
||||
remote: . Processing 1 references
|
||||
remote: Processed 1 references in total
|
||||
17e0da0 Revert "DRILL plex: compose image line back to 1.43.4 ONLY (the ladder untouched) — a fresh install at FROM for the Part 3 proof; reverted right after"
|
||||
42945c9 DRILL plex: compose image line back to 1.43.4 ONLY (the ladder untouched) — a fresh install at FROM for the Part 3 proof; reverted right after
|
||||
f46e1bc DRILL plex: security_fix step 1.43.4 -> 1.43.5 written by upgrade-test.py --write-ladder --security (D39 = B proof)
|
||||
|
||||
13:08:04 [5] the box's drill mirror now at 17e0da02
|
||||
obs {"pinned_images": {"plex": "plexinc/pms-docker:1.43.4.10903-e5521bd8c"}, "installed_images": {"plex": "plexinc/pms-docker:1.43.4.10903-e5521bd8c"}, "catalog_images": {"plex": "plexinc/pms-docker:1.43.5.11029-6bf8af3f0"}, "live_compose_image_lines": ["image: plexinc/pms-docker:1.43.4.10903-e5521bd8c"], "docker_inspect": ["plex plexinc/pms-docker:1.43.4.10903-e5521bd8c running=true restarts=0"]}
|
||||
badge {'hu': [{'title': 'Újabb változat érhető el ehhez az alkalmazáshoz. A frissítés indításához nyomd meg a Frissítés gombot.', 'text': 'Frissítés elérhető — ma'}], 'en': [{'title': 'A newer version of this app is available. Select the Update button to start it.', 'text': 'Update available — today'}]}
|
||||
@@ -0,0 +1,17 @@
|
||||
13:12:03 [5] the box's drill mirror now at 17e0da02
|
||||
before: {"pinned_images": {"plex": "plexinc/pms-docker:1.43.4.10903-e5521bd8c"}, "installed_images": {"plex": "plexinc/pms-docker:1.43.4.10903-e5521bd8c"}, "catalog_images": {"plex": "plexinc/pms-docker:1.43.5.11029-6bf8af3f0"}, "live_compose_image_lines": ["image: plexinc/pms-docker:1.43.4.10903-e5521bd8c"], "docker_inspect": ["plex plexinc/pms-docker:1.43.4.10903-e5521bd8c running=true restarts=0"]}
|
||||
badge before: {'hu': [{'title': 'Újabb változat érhető el ehhez az alkalmazáshoz. A frissítés indításához nyomd meg a Frissítés gombot.', 'text': 'Frissítés elérhető — ma'}], 'en': [{'title': 'A newer version of this app is available. Select the Update button to start it.', 'text': 'Update available — today'}]}
|
||||
night chain -> 202 {'data': {'legs': ['db-dump', 'tier2', 'update-leg']}, 'message': 'started', 'ok': True}
|
||||
the leg's own lines:
|
||||
2026/10/23 11:12:13 night_chain.go:108: [INFO] [night-chain] manual run started from 172.18.0.6:38520: [db-dump tier2 update-leg]
|
||||
2026/10/23 11:13:15 night_chain.go:77: [INFO] [night-chain] update-leg: started
|
||||
2026/10/23 11:13:15 unattended.go:279: [INFO] [update-leg] started (manual-chain): window 02:30, no step starts at or after 16:28
|
||||
2026/10/23 11:13:15 unattended.go:345: [INFO] [update-leg] plex: step pressed plex=plexinc/pms-docker:1.43.4.10903-e5521bd8c → plex=plexinc/pms-docker:1.43.5.11029-6bf8af3f0
|
||||
2026/10/23 11:13:15 unattended.go:351: [INFO] [update-leg] plex: security fix CVE-2026-96652,CVE-2026-96654,CVE-2026-96655,CVE-2026-96656 taken (plex=plexinc/pms-docker:1.43.4.10903-e5521bd8c → plex=plexinc/pms-docker:1.43.5.11029-6bf8af3f0)
|
||||
2026/10/23 11:13:45 unattended.go:359: [INFO] [update-leg] plex: step ended done after 30.1 s
|
||||
2026/10/23 11:13:45 unattended.go:325: [INFO] [update-leg] privatebin: skipped — stopped_by_household
|
||||
2026/10/23 11:13:45 unattended.go:257: [INFO] [update-leg] update leg (manual-chain): done=1 undone=0 held=0 failed=0 skipped=1 in 30s [skipped: privatebin=stopped_by_household] [security fix plex: CVE-2026-96652,CVE-2026-96654,CVE-2026-96655,CVE-2026-96656 done]
|
||||
2026/10/23 11:13:45 night_chain.go:82: [INFO] [night-chain] update-leg: done in 30s
|
||||
|
||||
after: {"pinned_images": {"plex": "plexinc/pms-docker:1.43.5.11029-6bf8af3f0"}, "installed_images": {"plex": "plexinc/pms-docker:1.43.5.11029-6bf8af3f0"}, "catalog_images": {"plex": "plexinc/pms-docker:1.43.5.11029-6bf8af3f0"}, "live_compose_image_lines": ["image: plexinc/pms-docker:1.43.5.11029-6bf8af3f0@sha256:01bdf0ad6d53198da4331540c1e73ab65db7a72e961fff97ecb0847d46055c5f"], "docker_inspect": ["plex plexinc/pms-docker:1.43.5.11029-6bf8af3f0@sha256:01bdf0ad6d53198da4331540c1e73ab65db7a72e961fff97ecb0847d46055c5f running=true restarts=0"]}
|
||||
state running hold None phase done
|
||||
@@ -0,0 +1,28 @@
|
||||
2026/10/23 11:13:15 tier2.go:507: [INFO] [backup] Tier 2 copied plex → /mnt/sizing-bench/backups/secondary/plex (2.3 MB, 0 leg(s), 0s)
|
||||
2026/10/23 11:13:15 update.go:537: [INFO] [stacks] update plex: accepted — guarded update started
|
||||
2026/10/23 11:13:15 unattended.go:345: [INFO] [update-leg] plex: step pressed plex=plexinc/pms-docker:1.43.4.10903-e5521bd8c → plex=plexinc/pms-docker:1.43.5.11029-6bf8af3f0
|
||||
2026/10/23 11:13:15 unattended.go:351: [INFO] [update-leg] plex: security fix CVE-2026-96652,CVE-2026-96654,CVE-2026-96655,CVE-2026-96656 taken (plex=plexinc/pms-docker:1.43.4.10903-e5521bd8c → plex=plexinc/pms-docker:1.43.5.11029-6bf8af3f0)
|
||||
2026/10/23 11:13:15 update.go:1378: [INFO] [stacks] update plex: phase checking
|
||||
2026/10/23 11:13:15 update.go:795: [INFO] [stacks] update plex: ladder — the last step (1 of 1) — the catalog's current definition
|
||||
2026/10/23 11:13:15 update.go:823: [INFO] [stacks] update plex: precondition met — Tier 2 (second drive) copy from 2026-10-23T11:13:04Z (0s old, limit 24h0m0s)
|
||||
2026/10/23 11:13:15 update.go:1378: [INFO] [stacks] update plex: phase safety-dump
|
||||
2026/10/23 11:13:16 update_guard.go:497: [INFO] [backup] update safety dump for plex: the app has no database — nothing to copy (no-op)
|
||||
2026/10/23 11:13:16 update.go:855: [INFO] [stacks] update plex: safety dump done (0 file(s)) []
|
||||
2026/10/23 11:13:17 undo.go:287: [INFO] [stacks] update plex: the undo copy will hold 2 named volume(s), 2.8 MiB
|
||||
2026/10/23 11:13:17 update.go:1378: [INFO] [stacks] update plex: phase pinning
|
||||
2026/10/23 11:13:17 pin.go:385: [INFO] [stacks] update plex: pin advanced to /opt/docker/felhom-controller/data/catalog-cache/templates/plex/docker-compose.yml (plex=plexinc/pms-docker:1.43.5.11029-6bf8af3f0)
|
||||
2026/10/23 11:13:17 update.go:1378: [INFO] [stacks] update plex: phase pulling
|
||||
2026/10/23 11:13:28 update.go:1378: [INFO] [stacks] update plex: phase copying
|
||||
2026/10/23 11:13:38 update.go:1378: [INFO] [stacks] update plex: phase copying
|
||||
2026/10/23 11:13:38 undo.go:437: [INFO] [stacks] update plex: copied plex_plex_config → plex_plex_config.pre-update-20261023T111338Z in 520ms
|
||||
2026/10/23 11:13:38 update.go:1378: [INFO] [stacks] update plex: phase copying
|
||||
2026/10/23 11:13:39 undo.go:437: [INFO] [stacks] update plex: copied plex_plex_transcode → plex_plex_transcode.pre-update-20261023T111338Z in 465ms
|
||||
2026/10/23 11:13:39 update.go:1378: [INFO] [stacks] update plex: phase starting
|
||||
2026/10/23 11:13:39 update.go:1378: [INFO] [stacks] update plex: phase verifying
|
||||
2026/10/23 11:13:44 update.go:1006: [INFO] [stacks] update plex: healthy after 5s (the app's health check passed)
|
||||
2026/10/23 11:13:45 update.go:1043: [INFO] [stacks] update plex: DONE in 29s
|
||||
2026/10/23 11:13:45 image_retention.go:238: [INFO] [stacks] image retention (update of plex): pass over 40 image(s) of [plexinc/pms-docker] — 0 candidate(s), 0 deleted, the rest kept
|
||||
2026/10/23 11:13:45 unattended.go:359: [INFO] [update-leg] plex: step ended done after 30.1 s
|
||||
2026/10/23 11:13:45 unattended.go:257: [INFO] [update-leg] update leg (manual-chain): done=1 undone=0 held=0 failed=0 skipped=1 in 30s [skipped: privatebin=stopped_by_household] [security fix plex: CVE-2026-96652,CVE-2026-96654,CVE-2026-96655,CVE-2026-96656 do
|
||||
identity 200 {"MediaContainer":{"size":0,"apiVersion":"1.2.3","claimed":false,"machineIdentifier":"de0d1cd3ffa436835af296505a1e2534be391d1f","version":"1.43.5.11029-6bf8af3f
|
||||
state running health None
|
||||
@@ -0,0 +1,27 @@
|
||||
### root@192.168.0.154
|
||||
containers=10 unhealthy=0 restarting=0 exited=[] gitea.dooplex.hu/admin/felhom-controller:0.341.0 Up 35 seconds (healthy)
|
||||
started=2026-10-23T11:15:27.242236608Z
|
||||
errors=0 info=126
|
||||
2026/10/23 11:15:28 [INFO] Self-update enabled (check every 6h, auto-update: false, auto-update time: 04:30)
|
||||
2026/10/23 11:15:28 [INFO] [scheduler] Registered periodic job: selfupdate-check (every 6h0m0s)
|
||||
2026/10/23 11:15:59 [INFO] [selfupdate] Current version 0.341.0 is up to date
|
||||
2026/10/23 11:15:29 [INFO] [sync] Initial sync: Sablonok frissítve — a katalógus új változata betöltve; a telepített alkalmazások fájljai nem változt
|
||||
### hp
|
||||
containers=24 unhealthy=0 restarting=0 exited=[] gitea.dooplex.hu/admin/felhom-controller:0.341.0 Up 36 seconds (healthy)
|
||||
started=2026-10-23T11:15:28.047379649Z
|
||||
errors=0 info=135
|
||||
2026/10/23 11:15:29 deploy.go:1442: [WARN] [stacks] Stack bookstack: new field ADMIN_PASSWORD (type=password) requires manual configuration
|
||||
2026/10/23 11:15:29 deploy.go:1442: [WARN] [stacks] Stack calibre-web: new field ADMIN_PASSWORD (type=password) requires manual configuration
|
||||
2026/10/23 11:15:29 deploy.go:1442: [WARN] [stacks] Stack plex: new field PLEX_CLAIM (type=text) requires manual configuration
|
||||
2026/10/23 11:15:29 main.go:833: [INFO] Self-update enabled (check every 6h, auto-update: false, auto-update time: 04:30)
|
||||
2026/10/23 11:15:29 scheduler.go:120: [INFO] [scheduler] Registered periodic job: selfupdate-check (every 6h0m0s)
|
||||
2026/10/23 11:16:01 updater.go:196: [INFO] [selfupdate] Current version 0.341.0 is up to date
|
||||
2026/10/23 11:15:30 sync.go:142: [INFO] [sync] Initial sync: Sablonok frissítve — a katalógus új változata betöltve; a telepített alkalmazások fájljai
|
||||
### felhom-pve
|
||||
containers=9 unhealthy=0 restarting=0 exited=[] gitea.dooplex.hu/admin/felhom-controller:0.341.0 Up 40 seconds (healthy)
|
||||
started=2026-10-23T11:15:11.64495822Z
|
||||
errors=0 info=141
|
||||
2026/10/23 11:15:12 [INFO] Self-update enabled (check every 6h, auto-update: false, auto-update time: 04:30)
|
||||
2026/10/23 11:15:12 [INFO] [scheduler] Registered periodic job: selfupdate-check (every 6h0m0s)
|
||||
2026/10/23 11:15:44 [INFO] [selfupdate] Current version 0.341.0 is up to date
|
||||
2026/10/23 11:15:13 [INFO] [sync] Initial sync: Sablonok frissítve — a katalógus új változata betöltve; a telepített alkalmazások fájljai nem változt
|
||||
@@ -0,0 +1,10 @@
|
||||
2026-10-23T11:24:47Z
|
||||
HTTP/1.1 303 See Other
|
||||
Location: /configuration?flash=floor_set
|
||||
Effective floor: v0.341.0 — source: DB (hub_settings) ; env fallback would be v0.120.0 Save global floor… DB override: v
|
||||
clear tester-1 Location: /customers/tester-1?flash=floor_set
|
||||
clear demo-hp Location: /customers/demo-hp?flash=floor_set
|
||||
clear demo-felhom Location: /customers/demo-felhom?flash=floor_set
|
||||
2026/10/23 13:15:19 [INFO] managed floor SERVED for tester-1: floor 0.341.0, agent requirement "0.158.1" from declared (golden 0.331.0)
|
||||
2026/10/23 13:15:20 [INFO] managed floor SERVED for demo-felhom: floor 0.341.0, agent requirement "0.158.1" from declared (golden 0.331.0)
|
||||
2026/10/23 13:15:20 [INFO] managed floor SERVED for demo-hp: floor 0.341.0, agent requirement "0.158.1" from declared (golden 0.331.0)
|
||||
@@ -0,0 +1,33 @@
|
||||
control NEGATIVE (a stub answering 200 over plain http): convicted, as it must (200 -)
|
||||
== tester-1 (enkicsifelhom.hu): 6 public name(s)
|
||||
OK control POSITIVE http://felhom.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://felhom.enkicsifelhom.hu/r1033-check?q=1
|
||||
OK http://files.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://files.enkicsifelhom.hu/r1033-check?q=1
|
||||
OK http://no-such-name-r1033.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://no-such-name-r1033.enkicsifelhom.hu/r1033-check?q=1
|
||||
OK http://paperless.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://paperless.enkicsifelhom.hu/r1033-check?q=1
|
||||
OK http://paste.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://paste.enkicsifelhom.hu/r1033-check?q=1
|
||||
OK http://wiki.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://wiki.enkicsifelhom.hu/r1033-check?q=1
|
||||
== demo-hp (enkisfelhom.hu): 14 public name(s)
|
||||
OK http://arcade.enkisfelhom.hu/r1033-check?q=1 -> 301 https://arcade.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://books.enkisfelhom.hu/r1033-check?q=1 -> 301 https://books.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://docs.enkisfelhom.hu/r1033-check?q=1 -> 301 https://docs.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://felhom.enkisfelhom.hu/r1033-check?q=1 -> 301 https://felhom.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://files.enkisfelhom.hu/r1033-check?q=1 -> 301 https://files.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://gist.enkisfelhom.hu/r1033-check?q=1 -> 301 https://gist.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://ha.enkisfelhom.hu/r1033-check?q=1 -> 301 https://ha.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://no-such-name-r1033.enkisfelhom.hu/r1033-check?q=1 -> 301 https://no-such-name-r1033.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://papir.enkisfelhom.hu/r1033-check?q=1 -> 301 https://papir.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://paste.enkisfelhom.hu/r1033-check?q=1 -> 301 https://paste.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://pdf.enkisfelhom.hu/r1033-check?q=1 -> 301 https://pdf.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://plex.enkisfelhom.hu/r1033-check?q=1 -> 301 https://plex.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://travel.enkisfelhom.hu/r1033-check?q=1 -> 301 https://travel.enkisfelhom.hu/r1033-check?q=1
|
||||
OK http://wiki.enkisfelhom.hu/r1033-check?q=1 -> 301 https://wiki.enkisfelhom.hu/r1033-check?q=1
|
||||
== demo-felhom (demo-felhom.eu): 8 public name(s)
|
||||
OK http://felhom.demo-felhom.eu/r1033-check?q=1 -> 301 https://felhom.demo-felhom.eu/r1033-check?q=1
|
||||
OK http://files.demo-felhom.eu/r1033-check?q=1 -> 301 https://files.demo-felhom.eu/r1033-check?q=1
|
||||
OK http://gist.demo-felhom.eu/r1033-check?q=1 -> 301 https://gist.demo-felhom.eu/r1033-check?q=1
|
||||
OK http://no-such-name-r1033.demo-felhom.eu/r1033-check?q=1 -> 301 https://no-such-name-r1033.demo-felhom.eu/r1033-check?q=1
|
||||
OK http://prowlarr.demo-felhom.eu/r1033-check?q=1 -> 301 https://prowlarr.demo-felhom.eu/r1033-check?q=1
|
||||
OK http://radarr.demo-felhom.eu/r1033-check?q=1 -> 301 https://radarr.demo-felhom.eu/r1033-check?q=1
|
||||
OK http://sonarr.demo-felhom.eu/r1033-check?q=1 -> 301 https://sonarr.demo-felhom.eu/r1033-check?q=1
|
||||
OK http://torrent.demo-felhom.eu/r1033-check?q=1 -> 301 https://torrent.demo-felhom.eu/r1033-check?q=1
|
||||
== every public name redirects http -> https
|
||||
@@ -0,0 +1,3 @@
|
||||
9201 login ok, /api/stacks 200
|
||||
login failed: no session cookie
|
||||
n100 /api/health 200tester-1 (BASE .148) login ok, /api/stacks 200
|
||||
@@ -0,0 +1,3 @@
|
||||
tester-1 HTTP/1.1 303 See Other Location: /customers/tester-1?flash=floor_set
|
||||
demo-hp HTTP/1.1 303 See Other Location: /customers/demo-hp?flash=floor_set
|
||||
demo-felhom HTTP/1.1 303 See Other Location: /customers/demo-felhom?flash=floor_set
|
||||
@@ -0,0 +1,74 @@
|
||||
# D39 = B — the security-fix lane, built and proven (2026-10-23)
|
||||
|
||||
Decision: `architecture/09-update-architecture.md` §3 decision 255 (the operator started the D39 brief ~12:00; R-1239
|
||||
closed). Read before the work: `09` §3 decisions 13–17, 22, 25–26, 52 and §6.4 parts 4–7; `00-capability-map.md` (the
|
||||
B rows); `13-threat-model.md`; `runbooks/target-selection.md`; `runbooks/publish-train-rules.md`.
|
||||
|
||||
Baselines at start: catalog `cde26ab`, controller `32e61d5` (0.340.0, global floor 06:41 UTC — read from the hub before
|
||||
the first controller commit), felhom.eu `4b9b0af3`. Machines: DooPlex (builds, the writer), demo-hp (9202 scratch; the
|
||||
release on 9201), Tester 1, demo-felhom (release only). Not touched: ep0, any household box.
|
||||
|
||||
## Venue
|
||||
|
||||
9202 was pointed at the drill catalog through its LOCAL mirror (`tools/repoint.py`, copied from
|
||||
`audits/burndown-3-2026-10-17/tools/`, R-992 — no catalog credential on the box): `repoint-drill.txt`. The drill was
|
||||
reset to live `main` (`cde26ab`) first. 9202's dashboard password is its own (`REHEARSAL_2026_10_20_9202_PW`; the
|
||||
demo-hp one was refused — five failed logins hit the one-minute limit; `operations/nodes.md` corrected).
|
||||
|
||||
## Part 2 — the lane on the old controller (9202 on 0.340.0)
|
||||
|
||||
| Step | Result | File |
|
||||
|---|---|---|
|
||||
| First try of the negative control | `inconclusive` — the install was refused before anything was made (Plex wants the drive itself as `HDD_PATH`); the walk gained `--value` | `P2-negative-try1-install-refused/` |
|
||||
| **Negative control**: FROM Plex 1.43.4.10903, TO `busybox:1.36.1` (exits at once) | **`failed`** — backing-up → … → verifying → undoing → **undone** after 335 s; Plex 1.43.4 back | `P2-negative/walk.txt`, `box-verdict-plex.json` |
|
||||
| The writer on that walk | **refused**: „the box walk is 'security' with verdict 'failed' — only a PROVEN `box_walk.py --security` walk is written" | `P2-negative/writer-refusal.txt` |
|
||||
| **The walk**: Plex 1.43.4.10903-e5521bd8c → 1.43.5.11029-6bf8af3f0 | **`proven`** — done in 36.9 s, no undo, no hold; pinned = installed = catalog = live compose = running = TO; digest `sha256:01bdf0ad6d53198da4331540c1e73ab65db7a72e961fff97ecb0847d46055c5f` (the registry's, re-read by the writer); soak 600 s, 60 samples, anon peak 10.7 % of 2 GiB (cgroup peak 11.5 %), 0 OOM kills, 0 restarts; `seed: none` (Plex's fixture is a `NoRoute` stub) | `P2-walk/` |
|
||||
| Setup gate (unclaimed Plex) | a browser (`Accept: text/html`) → 302 to `felhom.<domain>/__gate/start`; a script → 401; the same after the update | `P2-walk/box-verdict-plex.json` |
|
||||
| The writer, into the DRILL only | wrote the `security_fix` entry: ids CVE-2026-96652/-96654/-96655/-96656; advisory Plex's notice `https://forums.plex.tv/t/important-security-update-for-plex-media-server-v1-43-2-and-earlier/942319` (the one CISA KEV cites, all four added 2026-10-20 — `cisa-kev-plex2shell.json`, from CISA's own feed); `fixed_in` 1.43.3.10861; the gate accepts it with its WARN | `P2-walk/writer.txt`; drill `f46e1bc` |
|
||||
| Old controller, night (`POST /api/debug/backup/night-chain`) | **`[update-leg] plex: skipped — not_proven`**; Plex stays 1.43.4 (control: the same leg judged privatebin) | `P2-oldctl/2-night-chain-0.340.0.txt` |
|
||||
| Old controller, hand press | **done** in 25.7 s; Plex 1.43.5 at the step's digest; `/identity` 200 with 1.43.5 | `P2-oldctl/3-hand-press-0.340.0.txt` |
|
||||
|
||||
**How Plex was put back at 1.43.4 for Part 3:** removed through the product (with its data); drill commit `42945c9`
|
||||
moved ONLY the compose's image line back to 1.43.4 (the ladder line untouched); a fresh install pinned 1.43.4; `17e0da0`
|
||||
reverted that commit, so the step is the head again and the badge read „Frissítés elérhető" (`P3-newctl/0-…`).
|
||||
|
||||
## Part 3 — the new controller (0.341.0-rc.1 on 9202)
|
||||
|
||||
Controller `63dcccf` (CI run 2264 / #471 success), image `0.341.0-rc.1` set by hand (9202's self-update is off). The
|
||||
night chain: `[update-leg] plex: step pressed …`, **`[update-leg] plex: security fix
|
||||
CVE-2026-96652,CVE-2026-96654,CVE-2026-96655,CVE-2026-96656 taken (plex=…1.43.4… → plex=…1.43.5…)`**, `step ended done
|
||||
after 30.1 s`; the summary line ends `[security fix plex: … done]`. Backup first: the update's precondition met by a
|
||||
second-drive copy 0 s old, the undo copy of 2 named volumes (2.8 MiB), then the pin. `/identity` 200 with 1.43.5
|
||||
(`P3-newctl/1-…`, `2-…`).
|
||||
|
||||
## Part 4 — release 0.341.0
|
||||
|
||||
Image `0.341.0` (`sha256:8aa4ce60…c0bc`) from `63dcccf`; per-box floors on tester-1, demo-hp, demo-felhom (303
|
||||
`floor_set`); all three ran 0.341.0 by 11:15:31 UTC. Box walk: 0 unhealthy, 0 restarting, 0 exited, 0 `[ERROR]` lines since
|
||||
the start (126 / 135 / 141 `[INFO]` lines — the control); selfupdate „0.341.0 is up to date"; the https-only check passes
|
||||
on every public name; the dashboard login works on demo-hp and Tester 1 (`/api/stacks` 200), the N100's `/api/health`
|
||||
200 (`P4-release/`). **The operator said yes in chat; the global floor 0.341.0 / MinAgent 0.158.1 was set at 11:24:47
|
||||
UTC** (303 `floor_set`; the running hub's page reads „Effective floor: v0.341.0 — source: DB"); the per-box floors
|
||||
cleared.
|
||||
|
||||
## Teardown (three layers)
|
||||
|
||||
- **Machine (9202):** Plex removed through the product (no plex container or volume left); `controller.yaml` restored
|
||||
byte-identical (`cmp`) and the catalog cache re-cloned from the live catalog (R-615); this run's saved copy deleted
|
||||
after the `cmp`; image `0.341.0`. 34 older saved copies from earlier sessions stay — filed R-1240. The drill
|
||||
repository keeps this run's commits as the record.
|
||||
- **Host (demo-hp):** nothing provisioned.
|
||||
- **Hub:** the global floor 0.341.0 (the operator's yes); per-box floors cleared.
|
||||
|
||||
## Observations
|
||||
|
||||
- FILED: R-1240 — 9202 holds 34 saved `controller.yaml` copies with the catalog token.
|
||||
- FIXED without a row: drill commits never reached a box pointed at the drill's local mirror (`box_walk.refresh_box_mirror`);
|
||||
two stale catalog tests (9202 address, gate count + missing sibling in the shallow clone); Tester 1's guest moved
|
||||
`.101` → `.148` and the box walk had it typed (now read from the guest); `operations/nodes.md` (9202's password,
|
||||
Tester 1's address).
|
||||
- NOT-A-FINDING: the writer's own engine-major check is redundant with `check_entry` (two layers on purpose; the test
|
||||
asserts the refusal); `/opt/docker/stacks/plex` keeps the catalog's template files after a removal, as every catalog
|
||||
app's folder does (72); the demo-hp start WARNs „new field … requires manual configuration" come from code 0.341.0 did
|
||||
not change; a waiter of mine matched its own command line (a known trap, memory `pgrep-waiter-matches-itself`) and
|
||||
was stopped.
|
||||
@@ -0,0 +1,70 @@
|
||||
{
|
||||
"catalogVersion": "2026.10.21",
|
||||
"dateReleased": "2026-10-21T14:04:39.8983Z",
|
||||
"entries": [
|
||||
{
|
||||
"cveID": "CVE-2026-96656",
|
||||
"vendorProject": "Plex",
|
||||
"product": "Media Server",
|
||||
"vulnerabilityName": "Plex Media Server External Control of File Name or Path Vulnerability",
|
||||
"dateAdded": "2026-10-20",
|
||||
"shortDescription": "Plex Media Server contains an external control of file name or path vulnerability which can allow an admin user to write arbitrary files that may be executed on load. This vulnerability can be chained with CVE-2026-96652, CVE-2026-96654, and CVE-2026-96655.",
|
||||
"requiredAction": "Apply mitigations in accordance with vendor instructions, ensuring compliance with CISA\u2019s BOD 26-04 Prioritizing Security Updates Based on Risk (see URL in Notes) guidance and CISA\u2019s \u201cForensics Triage Requirements\u201d (see URL in Notes). Follow applicable BOD 26-04 guidance for cloud services or discontinue use of the product if mitigations are unavailable. Stakeholders are responsible for evaluating each asset's internet exposure and ensuring adherence to BOD 26-04 patching guidelines.",
|
||||
"dueDate": "2026-10-23",
|
||||
"knownRansomwareCampaignUse": "Unknown",
|
||||
"forensicTriage": "Yes",
|
||||
"notes": "https://forums.plex.tv/t/important-security-update-for-plex-media-server-v1-43-2-and-earlier/942319 ; ; BOD 26-04: https://www.cisa.gov/news-events/directives/bod-26-04-prioritizing-security-updates-based-risk ; Forensics Triage Requirements: https://www.cisa.gov/news-events/directives/bod-26-04-implementation-guidance-prioritizing-security-updates-based-risk ; https://nvd.nist.gov/vuln/detail/CVE-2026-96656",
|
||||
"cwes": [
|
||||
"CWE-73"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cveID": "CVE-2026-96655",
|
||||
"vendorProject": "Plex",
|
||||
"product": "Media Server",
|
||||
"vulnerabilityName": "Plex Media Server Server-Side Request Forgery (SSRF) Vulnerability",
|
||||
"dateAdded": "2026-10-20",
|
||||
"shortDescription": "Plex Media Server contains a server-side request forgery (SSRF) vulnerability that could allow an authenticated user to request arbitrary internal or external addresses via the '/video/:/transcode/universal' path parameter. This vulnerability can be chained with CVE-2026-96652, CVE-2026-96654, and CVE-2026-96656.",
|
||||
"requiredAction": "Apply mitigations in accordance with vendor instructions, ensuring compliance with CISA\u2019s BOD 26-04 Prioritizing Security Updates Based on Risk (see URL in Notes) guidance and CISA\u2019s \u201cForensics Triage Requirements\u201d (see URL in Notes). Follow applicable BOD 26-04 guidance for cloud services or discontinue use of the product if mitigations are unavailable. Stakeholders are responsible for evaluating each asset's internet exposure and ensuring adherence to BOD 26-04 patching guidelines.",
|
||||
"dueDate": "2026-11-03",
|
||||
"knownRansomwareCampaignUse": "Unknown",
|
||||
"forensicTriage": "No",
|
||||
"notes": "https://forums.plex.tv/t/important-security-update-for-plex-media-server-v1-43-2-and-earlier/942319 ; ; BOD 26-04: https://www.cisa.gov/news-events/directives/bod-26-04-prioritizing-security-updates-based-risk ; Forensics Triage Requirements: https://www.cisa.gov/news-events/directives/bod-26-04-implementation-guidance-prioritizing-security-updates-based-risk ; https://nvd.nist.gov/vuln/detail/CVE-2026-96655",
|
||||
"cwes": [
|
||||
"CWE-918"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cveID": "CVE-2026-96654",
|
||||
"vendorProject": "Plex",
|
||||
"product": "Media Server",
|
||||
"vulnerabilityName": "Plex Media Server Improper Neutralization of Encoded URI Schemes in a Web Page Vulnerability",
|
||||
"dateAdded": "2026-10-20",
|
||||
"shortDescription": "Plex Media Server contains an improper neutralization of encoded URI schemes in a web page vulnerability that could allow an attacker to call other plugins' functions and supply their own parameters. This vulnerability can be chained with CVE-2026-96652, CVE-2026-96655, and CVE-2026-96656.",
|
||||
"requiredAction": "Apply mitigations in accordance with vendor instructions, ensuring compliance with CISA\u2019s BOD 26-04 Prioritizing Security Updates Based on Risk (see URL in Notes) guidance and CISA\u2019s \u201cForensics Triage Requirements\u201d (see URL in Notes). Follow applicable BOD 26-04 guidance for cloud services or discontinue use of the product if mitigations are unavailable. Stakeholders are responsible for evaluating each asset's internet exposure and ensuring adherence to BOD 26-04 patching guidelines.",
|
||||
"dueDate": "2026-10-23",
|
||||
"knownRansomwareCampaignUse": "Unknown",
|
||||
"forensicTriage": "No",
|
||||
"notes": "https://forums.plex.tv/t/important-security-update-for-plex-media-server-v1-43-2-and-earlier/942319 ; ; BOD 26-04: https://www.cisa.gov/news-events/directives/bod-26-04-prioritizing-security-updates-based-risk ; Forensics Triage Requirements: https://www.cisa.gov/news-events/directives/bod-26-04-implementation-guidance-prioritizing-security-updates-based-risk ; https://nvd.nist.gov/vuln/detail/CVE-2026-96654",
|
||||
"cwes": [
|
||||
"CWE-84"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cveID": "CVE-2026-96652",
|
||||
"vendorProject": "Plex",
|
||||
"product": "Media Server",
|
||||
"vulnerabilityName": "Plex Media Server Server-Side Request Forgery (SSRF) Vulnerability",
|
||||
"dateAdded": "2026-10-20",
|
||||
"shortDescription": "Plex Media Server contains a server-side request forgery (SSRF) vulnerability which could allow an attacker using any X-Plex-Token value to include a full URL in the 'protocol' parameter and force the Plex server to POST to the attacker's chosen destination. This vulnerability can be chained with CVE-2026-96654, CVE-2026-96655, and CVE-2026-96656. ",
|
||||
"requiredAction": "Apply mitigations in accordance with vendor instructions, ensuring compliance with CISA\u2019s BOD 26-04 Prioritizing Security Updates Based on Risk (see URL in Notes) guidance and CISA\u2019s \u201cForensics Triage Requirements\u201d (see URL in Notes). Follow applicable BOD 26-04 guidance for cloud services or discontinue use of the product if mitigations are unavailable. Stakeholders are responsible for evaluating each asset's internet exposure and ensuring adherence to BOD 26-04 patching guidelines.",
|
||||
"dueDate": "2026-11-03",
|
||||
"knownRansomwareCampaignUse": "Unknown",
|
||||
"forensicTriage": "No",
|
||||
"notes": "https://forums.plex.tv/t/important-security-update-for-plex-media-server-v1-43-2-and-earlier/942319 ; ; BOD 26-04: https://www.cisa.gov/news-events/directives/bod-26-04-prioritizing-security-updates-based-risk ; Forensics Triage Requirements: https://www.cisa.gov/news-events/directives/bod-26-04-implementation-guidance-prioritizing-security-updates-based-risk ; https://nvd.nist.gov/vuln/detail/CVE-2026-96652",
|
||||
"cwes": [
|
||||
"CWE-918"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,33 @@
|
||||
mirror main cde26ab plex: guide checked against 1.43.5.11029-6bf8af3f0 (every HU + EN label read from the new web client, all equal); CHANGELOG + REPORT for the Plex2Shell move
|
||||
|
||||
drill repo main: cde26ab
|
||||
26: repo_url: file:///opt/docker/felhom-controller/drill-catalog.git
|
||||
26c26
|
||||
< repo_url: https://gitea.dooplex.hu/admin/app-catalog-felhom.eu.git
|
||||
---
|
||||
> repo_url: file:///opt/docker/felhom-controller/drill-catalog.git
|
||||
|
||||
Traceback (most recent call last):
|
||||
File "/mnt/5_hdd/felhom.eu/git/felhom.eu/documentation/audits/security-lane-2026-10-23/tools/repoint.py", line 86, in <module>
|
||||
{"mirror": mirror, "drill": drill, "restore": restore, "status": status}[sys.argv[1]]()
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
|
||||
File "/mnt/5_hdd/felhom.eu/git/felhom.eu/documentation/audits/security-lane-2026-10-23/tools/repoint.py", line 74, in drill
|
||||
restart_sync()
|
||||
~~~~~~~~~~~~^^
|
||||
File "/mnt/5_hdd/felhom.eu/git/felhom.eu/documentation/audits/security-lane-2026-10-23/tools/repoint.py", line 64, in restart_sync
|
||||
w.sync_rescan()
|
||||
~~~~~~~~~~~~~^^
|
||||
File "/mnt/5_hdd/felhom.eu/git/app-catalog-felhom.eu/scripts/box_walk.py", line 466, in sync_rescan
|
||||
ctl("POST", "/api/sync")
|
||||
~~~^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/mnt/5_hdd/felhom.eu/git/app-catalog-felhom.eu/scripts/box_walk.py", line 132, in ctl
|
||||
sess = open(f"{SC}/sess{os.getpid()}.txt").read().strip()
|
||||
~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/63b6c84e-0c09-4657-a160-855d2e4fe326/scratchpad/sc9202/sess3574518.txt'
|
||||
26: repo_url: file:///opt/docker/felhom-controller/drill-catalog.git
|
||||
27: sync_interval: 15m
|
||||
-rw------- 1 root root 1994 Oct 22 14:47 /var/lib/docker/volumes/felhom-controller-data/_data/controller.yaml.pre-seclane1023
|
||||
mirror main cde26ab plex: guide checked against 1.43.5.11029-6bf8af3f0 (every HU + EN label read from the new web client, all equal); CHANGELOG + REPORT for the Plex2Shell move
|
||||
cache HEAD cde26ab plex: guide checked against 1.43.5.11029-6bf8af3f0 (every HU + EN label read from the new web client, all equal); CHANGELOG + REPORT for the Plex2Shell move
|
||||
file:///opt/docker/felhom-controller/drill-catalog.git
|
||||
|
||||
@@ -0,0 +1,3 @@
|
||||
13:25:23 [X] stop -> 200 {'ok': True, 'message': 'Stack plex stop completed'}
|
||||
13:25:55 [X] remove (with drive data) -> 200 {'ok': True, 'data': {'removed': 'plex', 'volumes_removed': ['plex_plex_config', 'plex_plex_transcode'], 'hdd_paths_removed': [], 'hdd_paths_preserved': [], 'us
|
||||
13:26:04 [X] after remove: deployed=False leftovers='/opt/docker/stacks/plex'
|
||||
@@ -0,0 +1,19 @@
|
||||
RESTORED-IDENTICAL
|
||||
26: repo_url: https://gitea.dooplex.hu/admin/app-catalog-felhom.eu.git
|
||||
|
||||
2026/10/23 11:26:11 sync.go:344: [WARN] [sync] git.repo_url changed (cache was cloned from file:///opt/docker/felhom-controller/drill-catalog.git, config says https://gitea.dooplex.hu/admin/app-catalog-felhom.eu.git) — re-cloning the catalog cache from the configured repository (R-615)
|
||||
2026/10/23 11:26:11 sync.go:318: [INFO] [sync] Cloning https://gitea.dooplex.hu/admin/app-catalog-felhom.eu.git → /opt/docker/felhom-controller/data/catalog-cache
|
||||
2026/10/23 11:26:13 sync.go:511: [WARN] [sync] immich: catalog definition REFUSED by the allow-list — the app keeps its last good compose, nothing written: the catalog definition of immich asks for container rights the box does not allow: immich-server: bind /mnt/sizing-bench/appdata/immich -> /usr/src/app/upload rw; immich-server: bind /mnt/sizing-bench/userdata/media/photos -> /external/photos ro
|
||||
2026/10/23 11:26:13 sync.go:535: [INFO] [sync] Updated plex/docker-compose.yml
|
||||
2026/10/23 11:26:13 sync.go:535: [INFO] [sync] Updated plex/.felhom.yml
|
||||
2026/10/23 11:26:13 sync.go:261: [INFO] [sync] catalog moved 17e0da02fd61 -> 389859359491 (0 new, 1 stack dir(s) changed)
|
||||
2026/10/23 11:26:13 sync.go:299: [INFO] [sync] Catalog sync complete
|
||||
2026/10/23 11:26:13 sync.go:142: [INFO] [sync] Initial sync: Sablonok frissítve — frissítve: plex
|
||||
|
||||
26: repo_url: https://gitea.dooplex.hu/admin/app-catalog-felhom.eu.git
|
||||
27: sync_interval: 15m
|
||||
-rw------- 1 root root 1994 Oct 22 14:47 /var/lib/docker/volumes/felhom-controller-data/_data/controller.yaml.pre-seclane1023
|
||||
mirror main 17e0da02 Revert "DRILL plex: compose image line back to 1.43.4 ONLY (the ladder untouched) — a fresh install at FROM for the Part 3 proof; reverted right after"
|
||||
cache HEAD 3898593 D39 = B: the security-fix lane — a `security_fix` ladder step (box walk required, bench optional, CVE/GHSA notice), its gates + 8 decoys, the writer (--write-ladder --security), the box venue (box_walk.py --security, drill-mirror refresh); two stale tests fixed
|
||||
https://gitea.dooplex.hu/admin/app-catalog-felhom.eu.git
|
||||
|
||||
@@ -0,0 +1,4 @@
|
||||
felhom-controller gitea.dooplex.hu/admin/felhom-controller:0.341.0 Up 25 seconds (healthy)
|
||||
0
|
||||
removed-identical-save
|
||||
34
|
||||
@@ -0,0 +1,25 @@
|
||||
"""chain.py — press "run tonight's chain now" (POST /api/debug/backup/night-chain, R-705) on the box and quote the update
|
||||
leg's own lines; then read Plex's four observables. D39 = B proof (felhom.eu audits/security-lane-2026-10-23)."""
|
||||
import json, re, sys, time
|
||||
sys.path.insert(0, "/mnt/5_hdd/felhom.eu/git/app-catalog-felhom.eu/scripts")
|
||||
import box_walk as w
|
||||
w.LOG = []
|
||||
w.login()
|
||||
w.refresh_box_mirror()
|
||||
w.sync_rescan("plex", "plexinc/pms-docker:1.43.5.11029-6bf8af3f0", tries=20, delay=6)
|
||||
print("before:", json.dumps(w.observables("plex")))
|
||||
print("badge before:", w.badges("plex"))
|
||||
since = w.guest("date -u +%Y-%m-%dT%H:%M:%SZ").strip()
|
||||
code, d = w.ctl("POST", "/api/debug/backup/night-chain")
|
||||
print("night chain ->", code, str(d)[:200])
|
||||
leg = ""
|
||||
for _ in range(240):
|
||||
time.sleep(10)
|
||||
leg = w.guest("docker logs --since %s felhom-controller 2>&1 | grep -E 'update-leg' | grep -v DEBUG | cut -c1-400" % since)
|
||||
if re.search(r"update leg .*: done=", leg):
|
||||
break
|
||||
print("the leg's own lines:\n" + leg)
|
||||
time.sleep(10)
|
||||
print("after:", json.dumps(w.observables("plex")))
|
||||
st = w.stack("plex")
|
||||
print("state", st.get("state"), "hold", st.get("hold_reason"), "phase", st.get("update_phase"))
|
||||
@@ -0,0 +1,86 @@
|
||||
#!/usr/bin/env python3
|
||||
"""repoint.py mirror|drill|restore|status — point a box (scratch 9202; copied from audits/burndown-3-2026-10-17/tools/ for the D39 security lane, 2026-10-23) at the drill catalog WITHOUT a credential (R-992).
|
||||
|
||||
WHY A LOCAL MIRROR. `09` §6.5's route needs a catalog credential on the box: the drill repo is private (R-992). This
|
||||
session could not mint a READ-ONLY one — the Gitea credential on DooPlex is itself a token, and Gitea answers 403 to a
|
||||
token asking to list or create tokens (measured 2026-10-14) — and the night of 2026-10-06 put the admin WRITE token
|
||||
into the box's controller.yaml, which this task forbids. The cache-staging workaround of 2026-10-12 does not hold for a
|
||||
test that lasts days: every sync (and every controller start) re-applies the live catalog over the staged templates
|
||||
and over the installed apps' compose files (Syncer.copyTemplates, no deployed check).
|
||||
|
||||
So the drill catalog is MIRRORED onto the box: a bare git repository inside the controller's own data volume, and
|
||||
`git.repo_url: file:///opt/docker/felhom-controller/drill-catalog.git`. The controller's sync runs exactly as for a
|
||||
remote (clone/fetch/reset, the R-615 re-clone on a changed URL); no secret is on the box. DooPlex updates the mirror
|
||||
from the drill repo with a git bundle (`mirror`). The drill repo on Gitea stays the record.
|
||||
|
||||
SC=~/.felhom-tester1 TARGET=tester-1 python3 repoint.py mirror # (re)load the mirror from the drill repo main
|
||||
... repoint.py drill # save controller.yaml (once), set repo_url to the mirror, restart, sync, rescan
|
||||
... repoint.py restore # put the saved controller.yaml back (byte-identical), restart, sync, rescan
|
||||
... repoint.py status
|
||||
"""
|
||||
import os, subprocess, sys, time
|
||||
sys.path.insert(0, "/mnt/5_hdd/felhom.eu/git/app-catalog-felhom.eu/scripts")
|
||||
import box_walk as w
|
||||
|
||||
DRILL = "/mnt/5_hdd/felhom.eu/drill/app-catalog-drill"
|
||||
VOL = "/var/lib/docker/volumes/felhom-controller-data/_data" # guest path of the controller's data volume
|
||||
CVOL = "/opt/docker/felhom-controller" # the same, inside the controller container
|
||||
SAVE = f"{VOL}/controller.yaml.pre-seclane1023"
|
||||
MIRROR_URL = f"file://{CVOL}/drill-catalog.git"
|
||||
|
||||
|
||||
def status():
|
||||
print(w.guest(f"""grep -nE 'repo_url|sync_interval' {VOL}/controller.yaml; ls -l {SAVE} 2>&1 | tail -1
|
||||
docker exec felhom-controller sh -c 'git -C {CVOL}/drill-catalog.git log -1 --format="mirror main %h %s" main 2>&1;
|
||||
git -C {CVOL}/data/catalog-cache log -1 --format="cache HEAD %h %s" 2>&1; git -C {CVOL}/data/catalog-cache remote get-url origin'"""))
|
||||
|
||||
|
||||
def mirror():
|
||||
b = "/tmp/drill-seclane1023.bundle"
|
||||
subprocess.run(["git", "-C", DRILL, "bundle", "create", b, "main"], check=True, capture_output=True)
|
||||
head = subprocess.run(["git", "-C", DRILL, "rev-parse", "--short", "main"], capture_output=True, text=True).stdout.strip()
|
||||
ssh = w.ssh_args()
|
||||
# bundle -> PVE host -> guest -> controller volume; the bare repo is made and fetched BY the controller's own user
|
||||
subprocess.run(["scp", "-q", "-o", "StrictHostKeyChecking=accept-new"] + (["-J", w._T["jump"]] if w._T["jump"] else [])
|
||||
+ [b, f"{w._T['ssh']}:{b}"], check=True)
|
||||
subprocess.run(ssh + [f"pct push {w.GUEST} {b} {VOL}/drill-seclane1023.bundle && rm -f {b}"], check=True)
|
||||
print(w.guest(f"""set -e
|
||||
docker exec felhom-controller sh -c 'test -d {CVOL}/drill-catalog.git || git init -q --bare -b main {CVOL}/drill-catalog.git
|
||||
git -C {CVOL}/drill-catalog.git fetch -q {CVOL}/drill-seclane1023.bundle +main:main && rm -f {CVOL}/drill-seclane1023.bundle
|
||||
git -C {CVOL}/drill-catalog.git log -1 --format="mirror main %h %s" main'"""))
|
||||
print("drill repo main:", head)
|
||||
os.remove(b)
|
||||
|
||||
|
||||
def restart_sync():
|
||||
w.guest("docker restart felhom-controller >/dev/null")
|
||||
for _ in range(60):
|
||||
time.sleep(3)
|
||||
try:
|
||||
w.login()
|
||||
break
|
||||
except SystemExit:
|
||||
continue
|
||||
w.sync_rescan()
|
||||
time.sleep(3)
|
||||
print(w.guest("docker logs --since 3m felhom-controller 2>&1 | grep -E '\\[sync\\]' | grep -v DEBUG | tail -8"))
|
||||
|
||||
|
||||
def drill():
|
||||
print(w.guest(f"""set -e
|
||||
test -f {SAVE} || cp -p {VOL}/controller.yaml {SAVE}
|
||||
sed -i -E 's#^(\\s+repo_url: ).*$#\\1{MIRROR_URL}#' {VOL}/controller.yaml
|
||||
grep -n 'repo_url' {VOL}/controller.yaml; diff {SAVE} {VOL}/controller.yaml || true"""))
|
||||
restart_sync()
|
||||
status()
|
||||
|
||||
|
||||
def restore():
|
||||
print(w.guest(f"""set -e
|
||||
cp -p {SAVE} {VOL}/controller.yaml
|
||||
cmp {SAVE} {VOL}/controller.yaml && echo RESTORED-IDENTICAL; grep -n repo_url {VOL}/controller.yaml"""))
|
||||
restart_sync()
|
||||
status()
|
||||
|
||||
|
||||
{"mirror": mirror, "drill": drill, "restore": restore, "status": status}[sys.argv[1]]()
|
||||
@@ -26,6 +26,28 @@
|
||||
|
||||
---
|
||||
|
||||
## 2026-10-23 day (later) — D39 = B: the security-fix lane
|
||||
|
||||
| Row | What | Closed | Evidence |
|
||||
|---|---|---|---|
|
||||
| **R-1239** | **The catalog has no fast lane for an urgent security fix — a decision for the operator.** (P2) | CLOSED 2026-10-23 — **D39 = B** (operator ruling by starting the D39 brief; `09` §3 decision 255): a `security_fix` ladder step, box walk required, bench optional, a CVE/GHSA notice as the authority, no operator word per incident. Catalog `3898593` (format, gates, 8 decoys, writer `--write-ladder --security`, `box_walk.py --security`); controller **0.341.0** (the leg takes it; global floor 11:24 UTC, MinAgent 0.158.1). **Reasoning kept:** a security-fix step never crosses a database engine major and is never a same-tag fix (decision 52's lane); a controller ≤ 0.340.0 skips it at night and a hand press takes it. | `audits/security-lane-2026-10-23/` (negative control `failed` and refused; Plex 1.43.4 → 1.43.5 proven on 9202; 0.340.0 night `not_proven` + hand press; rc.1 night `security fix … taken`) |
|
||||
|
||||
## 2026-10-23 day — the releases and the DooPlex hour (the operator's „go")
|
||||
|
||||
| Row | What | Closed | Evidence |
|
||||
|---|---|---|---|
|
||||
| **R-49** | **[P2] The offsite capture set is ~90% cache and duplication — 1.1 GB of a 1.2 GB immich "photo backup".** (P3) | CLOSED 2026-10-23 — released in controller **0.340.0** (D21 = B: a template may exclude a named volume; Immich's model cache is out of every backup); the global floor 0.340.0 (MinAgent 0.158.1) since 06:41 UTC, all three test boxes on it | `audits/day-2026-10-23/` 03, 06 |
|
||||
| **R-1204** | **A box that runs once with its clock ahead loses its WireGuard tunnel to ep0 — the PBS whole-box copies and the operator's door — until the real date passes the wrong one, and nothing says why.** (P2) | CLOSED 2026-10-23 — the sign reported and the heal (D30, the agent re-keys on the existing peer path) in agent **0.163.0** (vouched by the operator 07:58 UTC; on Tester 1, demo-hp, demo-felhom by signed jobs), the alarm and its clearing in hub **0.167.0** (deployed 08:27 UTC). The live heal was proven on the test night (`audits/night-2026-10-23/P4-testnight/`) | `audits/day-2026-10-23/` 07, and the hub watch in 14 |
|
||||
| **R-1205** | **After a box death, a SECOND install that is deleted before it makes a key leaves the whole-box (PBS) tier waiting for the operator's re-issue.** (P2) | CLOSED 2026-10-23 — D31 in hub **0.167.0** (deployed 08:27 UTC, image sha256:177860a9…): the re-issue reads the newest deletion of a host that held a key | `audits/day-2026-10-23/` 14 |
|
||||
| **R-1230** | **A reinstalled box that waits for the household's recovery code was reported „stuck” after an hour: a false operator mail, a needless re-issue of the off-site login, and a controller restart on the box.** (P3) | CLOSED 2026-10-23 — hub **0.167.0** (deployed 08:27 UTC): the delivery checker stands down for `awaiting_recovery_key` | `audits/day-2026-10-23/` 14 |
|
||||
| **R-982** | **The Proxmox host accepts root password login over SSH from the LAN (measured on demo-hp), so anyone on the household network can try the root password.** (P3) | CLOSED 2026-10-23 — D4 delivered: the bundle drop-in in agent **0.163.0** (config bundle sha256 3afb88d9…, via a step bundle 0.163.0-step1 because it adds a path, R-880) on all three test boxes — `sshd -T` `permitrootlogin without-password`, sshd reloaded after the drop-in, a key login works, felhom-sshd active; new installs: installer **1.36.0** published (tag `installer-v1.36.0`, served sha256 de8bf5f3… = the tag). The real root password was not tried over SSH (CC does not hold it); the console is untouched by design | `audits/day-2026-10-23/` 07, 11 |
|
||||
| **R-985** | **Our zero-knowledge wording is stronger than the product: on a live box the operator can reach the off-site backup key (through the vaulted root credential and the operator tunnel) without the customer's recovery code.** (P3) | CLOSED 2026-10-23 — the D3 wording published on the website (felhom.eu `0bb57a4f`), read back live at phone and computer size | `audits/day-2026-10-23/` 12-website-readback |
|
||||
| **R-991** | **Nothing a buyer can read says which machines Felhom installs on, and a quarter of the forum beginners own only machines it cannot install on.** (P3) | CLOSED 2026-10-23 — the D5 minimum machine published (felhom.eu `0bb57a4f`), read back live | `audits/day-2026-10-23/` 12-website-readback |
|
||||
| **R-1225** | **The website's minimum machine (8 GB) cannot run the photo app, and the download page does not say so.** (P3) | CLOSED 2026-10-23 — D36: „8 GB runs the basics; the photo app needs 16 GB” on both download pages, published `0bb57a4f`, read back live | `audits/day-2026-10-23/` 12-website-readback |
|
||||
| **R-1226** | **Two FAQ answers say the household's data leaves the home only encrypted (the remote backup) — not true for data reached from outside.** (P2) | CLOSED 2026-10-23 — D36: both FAQ answers (visible + JSON-LD, HU + EN) now say what is opened from outside passes Cloudflare's network and is readable there; the backups leave only encrypted. Published `0bb57a4f`; live: the new sentence 2× per FAQ page, the old one 0× (control: the previous commit had it 2×) | `audits/day-2026-10-23/` 12-website-readback |
|
||||
| **R-1227** | **The container registry and the agent packages exist only on DooPlex — after its loss no box updates, reinstalls or installs.** (P2) | CLOSED 2026-10-23 — D33 built (felhom.eu `d306b8c2`): nightly 01:40 the newest 3 release tags of controller, hub, samba (skopeo `dir:`, digest-true) and the newest 5 agent versions to ep0 `/srv/felhom-release-copy` (one new user `felhom-relcopy`, rrsync-confined; cap 3 GB by the sender); first run 676 MB in 91 s; pulled back, every sha256 OK, every image re-verified; controller 0.340.0 from the copy loaded on 9202 (ID = config digest); alarm `FelhomReleaseCopyStale`; restore steps in `total-loss-of-dooplex.md`. Follow-ups: R-1235 | `audits/day-2026-10-23/` 08-release-copy-D33.txt |
|
||||
| **R-1234** | **D34's first push refused the hub's own catch-up and mailed a false „hub-db-backup failed" to admin@.** Installing the 6-hourly timer at 10:28 made systemd catch up the missed 08:30 slot at once (`Persistent=true`), while the freshly deployed hub 0.167.0 was still writing its start-up snapshot (`hub-*.db.tmp`); the only finished one was 02:00 (8 h), above the new 2-hour limit → „the hub stopped snapshotting", `OnFailure` mail id `01a167e0…`. The 2 h limit was also stricter than the hub's own 6-hour catch-up, so any restart could repeat it. (P3) | CLOSED 2026-10-23 day, the same session — the push waits (≤ 20 min) for a snapshot being written, and the limit is one slot gap + 1 h (7 h); tests `test_d34_waits_for_a_snapshot_being_written` (reproduces the 10:28 run when the wait is removed), `…_a_restart_within_one_slot_is_not_an_alarm`, `…_two_missed_slots_are_refused`, each red-proved | `audits/day-2026-10-23/13-hubdb-6h-D34.txt` |
|
||||
|
||||
## 2026-10-22 night — the seven decisions put into effect (full text of the rows: `git show 1100a344:documentation/backlog/OPEN-ITEMS.md`)
|
||||
|
||||
| Row | What | Closed | Evidence |
|
||||
|
||||
@@ -5,6 +5,30 @@ Moved here byte for byte on 2026-10-14 (night session), newest first, so that `S
|
||||
|
||||
**Updated 2026-10-13 (night): demo-hp, demo-felhom and Tester 1 run agent 0.155.0; demo-felhom runs controller 0.310.0, the other two 0.309.0 until the floor is raised. The open-items list is at 127. Reports: `REPORT-poweroff.md`, `REPORT-plugmeter.md`, `REPORT-usenet-2026-10-12.md`, `REPORT-newapps-2026-10-12.md`, `REPORT-disk-evening-2026-10-12.md`, `REPORT-answers-2026-10-12.md`, `REPORT-m3-2026-10-11.md`, `REPORT-redesign-2026-10-10.md`, `REPORT-testers-2026-10-10.md`, `REPORT-register-shrink-2026-10-10.md`, `REPORT-new-apps-2026-10-10.md`, `REPORT-release-2026-10-10.md`, `REPORT-break-the-circle-2026-10-09.md`, `REPORT-dooplex-survival-2026-10-09.md`, `REPORT-day4-2026-10-09.md`.**
|
||||
|
||||
## Moved from STATUS on 2026-10-23 (afternoon)
|
||||
|
||||
**Updated 2026-10-23 (day).** Released on your go: controller 0.340.0 (the floor), agent 0.163.0 (vouched; root SSH by key only on the three test boxes), installer 1.36.0, hub 0.167.0, and the website with your two sentences. The hub database now leaves every 6 hours and the released images and agent packages every night, both read back from ep0; the break-glass sheet is current — please print it. One false failure mail reached you at 10:28 (my install of the 6-hour copy, fixed in the same hour).
|
||||
|
||||
**Decided 2026-10-23 ~12:00 — D39 = B** (the operator started the D39 brief; the ruling is in R-1239 in CLOSED-ITEMS.md and in CONTEXT.md):
|
||||
|
||||
| # | Row | Question | Pick (why) | The other option costs | If you do nothing | Blocked |
|
||||
|---|---|---|---|---|---|---|
|
||||
| D39 | R-1239 | When an app has an urgent security hole, may its fix reach the boxes at night without the full update test? Today it needs your word and a hand press on each box (Plex, today). | **Yes — a marked „security fix” step** (tested on one test box only; the night update then takes it; works without you awake) | stay as today: your word and a CC session for every hole; households wait for the press | the same as today | — |
|
||||
|
||||
## Moved from STATUS on 2026-10-23 (day)
|
||||
|
||||
**Updated 2026-10-23 (morning).** Your three picks are built (the tunnel repairs itself with a new key, the hub skips a keyless deleted install, the controller repository stays public), and the night copy can no longer outlive the controller. Four fresh reviews found 34 points, every real one fixed or a row; the test night PASSED (no failed check at night, every night job passed, nothing lost). Nothing was released or deployed; the morning list has the order, and the DooPlex survey asks four questions (D32–D35).
|
||||
|
||||
**Decided 2026-10-23 morning — the operator's „go" for the day brief** (each ruling in its row and in CONTEXT.md): D32 = yes, print the sheet; D33 = yes, the nightly release copy to ep0 (built, R-1227 closed); D34 = yes, the hub DB every 6 hours (built); D35 = yes, after the release week (R-1228, dated 2026-10-30); D36 = yes, both sentences (published). The rows as they stood:
|
||||
|
||||
| # | Row | Question | Pick (why) | The other option costs | If you do nothing | Blocked |
|
||||
|---|---|---|---|---|---|---|
|
||||
| D36 | R-1225, R-1226 | Two website sentences before the publish: „8 GB runs the basics; the photo app needs 16 GB” and „what you open from outside the home passes Cloudflare's network, readable there; the backups leave only encrypted”? | **Yes, both** (the patch's own aim is the whole truth) | — | the patch publishes with both promises a little too strong | — |
|
||||
| D35 | R-1228 | After the release week, serve the website and the installer from Cloudflare, like the ISO? | **Yes, later** (no new installs stop when DooPlex is down) | keep them home: after a DooPlex loss, no install until it is back | the same | — |
|
||||
| D34 | R-232 | Copy the hub database to ep0 every 6 hours instead of every night? | **Yes** (0 €, about 30 minutes, one DooPlex change) | keep nightly: up to a day of hub changes lost, e.g. a new box's recovery package | the same | — |
|
||||
| D33 | R-1227 | Copy the released images and agent packages to ep0 every night? | **Yes** (0 €, about 2 hours of CC, a few GB on ep0) | a timed rebuild drill instead: about 3 hours once, and rebuilt images are not the same | after a DooPlex loss no box updates, reinstalls or installs until everything is rebuilt | — |
|
||||
| D32 | R-923, R-924 | Print the break-glass sheet now (eight keys and logins + ep0's fingerprint) and keep it outside the home? | **Yes** (30 minutes; nothing else opens the off-site copies after a loss) | an encrypted USB stick outside the home: ~10 €, 1 hour, one more passphrase | after a DooPlex loss no off-site copy opens | — |
|
||||
|
||||
## Moved from STATUS on 2026-10-23
|
||||
|
||||
**Updated 2026-10-22 (night).** Your seven picks of 15:47 are in effect or ready: D6 and D8 are live, D21 waits for the controller release, and D3, D5 (the website) and D4 (root SSH) are ready patches. Four controller fixes are built and proven on the scratch box, among them a gentler night copy (the 03:30 memory pressure) and no import over a stopped app. Nothing was released or deployed; the morning list below has the order.
|
||||
|
||||
@@ -43,7 +43,7 @@ DooPlex while the boxes are away (`felhom-pve-lan` → `No route to host`, 2026-
|
||||
| What it is | **QEMU VM 341 `night1004-tester1` on demo-hp** (installed from the ISO on 2026-10-04, `audits/night-2026-10-04/tester1/`); disposable (operator) |
|
||||
| PVE node name | `felhom` (its certificate: `CN=felhom.enkicsifelhom.hu`) |
|
||||
| LAN address | **DHCP** — read 2026-10-06 as `192.168.0.154` (MAC `bc:24:11:ac:e3:f9`; find it with `ip neigh` after a ping) |
|
||||
| Customer / guest | `tester-1`; guest LXC 9201 at `192.168.0.101`, dashboard `felhom.enkicsifelhom.hu` |
|
||||
| Customer / guest | `tester-1`; guest LXC 9201 by DHCP (`192.168.0.101` until 2026-10-23, then `.148` — read it: `pct exec 9201 -- ip -4 -o addr show eth0`; `box_walk.py` reads it itself), dashboard `felhom.enkicsifelhom.hu` |
|
||||
| SSH | **`ssh root@192.168.0.154` from DooPlex works BY KEY** (measured 2026-10-09: `BatchMode=yes`, used for the release read-back); the old route `ssh -J demo-hp root@<VM address>` stays the fallback; `box_walk.py` `TARGET=tester-1` holds the route |
|
||||
| Identity checked | 2026-10-06: the agent's own report `host.node=felhom` = the VM's certificate; the guest answers its domain (200) and not demo-hp's (404) — `audits/readback-2026-10-07/E1-identity-match.txt` |
|
||||
|
||||
@@ -218,7 +218,7 @@ Details, and the two hard rules that apply to any host node, in `operations/tail
|
||||
### demo-hp scratch guest — LXC 9202 `demo-hp-scratch` (R-481, built 2026-09-13)
|
||||
|
||||
- **Disposition:** the nightly rotation's throwaway host. Under the **demo-hp customer** (same domain,
|
||||
same dashboard password), **hub reporting OFF, Cloudflare tunnel OFF, agent local API OFF, off-site
|
||||
a dashboard password of its OWN — `REHEARSAL_2026_10_20_9202_PW` in the CC secret store, named for the 2026-10-20 rehearsal; the demo-hp `PASSWORD` is refused, measured 2026-10-23), **hub reporting OFF, Cloudflare tunnel OFF, agent local API OFF, off-site
|
||||
OFF, self-update OFF** (the image is set by hand in `/etc/felhom-controller-image`). **Persists across
|
||||
nights on purpose.** Apps on it are throwaways; nothing on it is a customer promise. Not in the felhom
|
||||
pool, so the hub never sees it. Tags `scratch,r481`; the same text sits in the guest at
|
||||
|
||||
@@ -93,6 +93,17 @@ itself. **Corrected:** one snapshot took 44 s on the live volume (0.63 s on a lo
|
||||
|
||||
### Step 4 — the push (on DooPlex, root; `felhom-hub-db-backup.service` + `.timer` 02:30) — DONE 2026-10-05
|
||||
|
||||
> **Since 2026-10-23 (D34, operator yes): every 6 hours.** The hub (≥ 0.167.0) writes a snapshot at 02:00, 08:00, 14:00
|
||||
> and 20:00 Budapest (`dbsnap.Slots`; start-up catch-up when the newest is older than 6 h); the timer pushes at 02:30,
|
||||
> 08:30, 14:30 and 20:30. The push waits (≤ 20 min) for a snapshot being written and refuses a newest snapshot older
|
||||
> than **7 h** (one slot gap + 1 h — a 2 h limit refused the hub's own catch-up once and mailed a false failure,
|
||||
> R-1234). Alarm `HubDBBackupStale` at **13 h** (two missed pushes). **Retention is unchanged and server-side on ep0**
|
||||
> (`prune-operator-hubdb`, 03:45, keep-daily 14, keep-weekly 8): the current day's copies (up to 4) live until the next
|
||||
> prune, then the newest of each day for 14 days and one per week for 8 weeks. One copy: ~500 MiB raw, ~25 MiB
|
||||
> uploaded. Proven 2026-10-23: a hand run, then the newest copy restored from ep0 with the read-only token was
|
||||
> byte-identical to the pod's snapshot, integrity ok, `SELECT COUNT(*) FROM hosts` = 4
|
||||
> (`audits/day-2026-10-23/13-hubdb-6h-D34.txt`). The text below is the original nightly design.
|
||||
|
||||
**The real script is `scripts/hub-db-backup/felhom-hub-db-backup`** (versioned, R-231; installed by `install.sh`;
|
||||
15 tests in `test_hub_db_backup.py`, run by hand — not in CI). It adds to the sketch below: it refuses a snapshot older
|
||||
than 26 h (the hub stopped snapshotting), a copy whose size differs from the pod's file, and a copy with no hosts; it
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# Break-glass sheet — the keys that must exist outside DooPlex (R-923)
|
||||
# Break-glass sheet — the keys that must exist outside DooPlex (R-923, D32)
|
||||
|
||||
> **A template. It holds NO key value, and none may ever be written into this file, a commit, a log or a chat.**
|
||||
> Print this page, then write or stick each value onto the paper copy only. Keep the paper away from home (DooPlex is
|
||||
@@ -33,31 +33,43 @@ captured line on some systems; strip it when typing the value back.
|
||||
**Check a print later without exposing it:** for S1/S2 the only reliable check is one restore with a key file rebuilt
|
||||
from the printed `data` field (hub-DB runbook Step 0 note) — `key show` cannot check a rebuilt key.
|
||||
|
||||
## Public fingerprints — to check you hold the right key (not secret; not on the printed page)
|
||||
|
||||
| # | Fingerprint |
|
||||
|---|---|
|
||||
| S1 | PBS key `93:03:bf:d7:1f:4c:9e:fe…` (`proxmox-backup-client key show`) |
|
||||
| S2 | PBS key `b2:19:bf:36:3b:97:3d:6c…` |
|
||||
| S6a | `SHA256:/ixgTesZqykAGJpFUUd4kLAiHFgKOkYFLNC3AQXWP+k` |
|
||||
| S6b | `SHA256:7YqN4rXO08yixTeOO+UtQ8jHyIGycICuctQgRYVGnWw` |
|
||||
| S19 | recipient `age1aw83ej37x9sck89xwt856ry0553kf5sng3w544an4et54yjdx5qq79utrv` |
|
||||
|
||||
Every place below was checked to exist on 2026-10-23 (existence only, no value read). The release copy on ep0 (D33)
|
||||
and the 6-hourly hub-DB copy (D34) add NO key: the release copy is not encrypted (it holds no secret), and the hub-DB
|
||||
copy uses S2 as before.
|
||||
|
||||
---
|
||||
|
||||
## The sheet (print from here)
|
||||
## The sheet (print from here — one A4 page)
|
||||
|
||||
**FELHOM — break-glass keys.** Printed on: ____________ Stored at: ______________________________
|
||||
**FELHOM — vészhelyzeti kulcsok / break-glass keys.** Nyomtatva / printed: ____________ Hol tárolom / kept at: ______________________
|
||||
|
||||
| # | Key | Public fingerprint (to match the right key) | Value — write or stick here |
|
||||
Csak nevek és helyek; az értéket kézzel írd be. *Names and places only; write each value by hand.* Tartsd az otthonon kívül. *Keep it outside the home.*
|
||||
|
||||
| # | Név / Name | Hol van most / Where it lives now | Érték / Value |
|
||||
|---|---|---|---|
|
||||
| S1 | DooPlex off-site key — opens Gitea, the password manager, the k8s Secrets export (ep0 `operator`, `host/dooplex-gitea`) | PBS key `93:03:bf:d7:1f:4c:9e:fe…` | `data`: ______________________________________________ |
|
||||
| S2 | Hub-DB off-site key — opens the hub database (ep0 `operator`, `host/dooplex-hub`) | PBS key `b2:19:bf:36:3b:97:3d:6c…` | `data`: ______________________________________________ |
|
||||
| S4 | Hub seal key `OFFSITE_SECRET_KEY` (64 hex) | — | ______________________________________________________ |
|
||||
| S5 | DooPlex restic / Secrets-export passphrase | — | ______________________________________________________ |
|
||||
| S6a | Signing RECOVERY key `felhom-rec-recovery` (also in the S1 copy since 2026-10-09; the paper is the copy that needs nothing else) | `SHA256:/ixgTesZqykAGJpFUUd4kLAiHFgKOkYFLNC3AQXWP+k` | (staple the printout) |
|
||||
| S6b | Signing OPERATIONAL key `felhom-op-operational` | `SHA256:7YqN4rXO08yixTeOO+UtQ8jHyIGycICuctQgRYVGnWw` | (staple the printout) |
|
||||
| S7 | ep0 read-only token `dooplex-hub@pbs!restore` | — | ______________________________________________________ |
|
||||
| S8 | Hetzner account: login e-mail · password · two-factor recovery codes | — | ______________________________________________________ |
|
||||
| S11 | Cloudflare account: login · password · two-factor recovery codes | — | ______________________________________________________ |
|
||||
| S12 | Gmail `felhom.eu@gmail.com`: password · two-factor recovery codes | — | ______________________________________________________ |
|
||||
| S19 | CC secrets age key — opens `admin/cc-secrets` (every secret Claude Code uses on DooPlex); the store is in Gitea, behind S1 | recipient `age1aw83ej37x9sck89xwt856ry0553kf5sng3w544an4et54yjdx5qq79utrv` | `AGE-SECRET-KEY-1`______________________________________ |
|
||||
| S3 | Vaultwarden master password — **in your head**; write it here only if you decide to | — | ______________________________________________________ |
|
||||
| S1 | DooPlex off-site kulcs / DooPlex off-site key (Gitea, jelszókezelő / password manager, k8s Secrets) | DooPlex `/etc/felhom-dooplex-offsite/enc.key` | `data`: ____________________________ |
|
||||
| S2 | Hub-adatbázis off-site kulcs / Hub-DB off-site key | DooPlex `/etc/felhom-hub-backup/enc.key` | `data`: ____________________________ |
|
||||
| S4 | Hub pecsétkulcs / hub seal key `OFFSITE_SECRET_KEY` | k8s Secret `felhom-system/offsite-secret-key` | __________________________________ |
|
||||
| S5 | DooPlex restic / Secrets-export jelmondat / passphrase | DooPlex `/etc/backup/restic-password` | __________________________________ |
|
||||
| S6a | Aláíró HELYREÁLLÍTÓ kulcs / signing RECOVERY key | DooPlex `/mnt/5_hdd/felhom.eu/felhom-rec-recovery` | (tűzd ide / staple) |
|
||||
| S6b | Aláíró ÜZEMI kulcs / signing OPERATIONAL key | DooPlex `/mnt/5_hdd/felhom.eu/felhom-op-operational` | (tűzd ide / staple) |
|
||||
| S7 | ep0 csak-olvasó token / read-only token `dooplex-hub@pbs!restore` | DooPlex `/etc/felhom-hub-backup/token-restore` | __________________________________ |
|
||||
| S8 | Hetzner fiók / account (e-mail · jelszó / password · 2FA kódok / codes) | Vaultwarden | __________________________________ |
|
||||
| S11 | Cloudflare fiók / account (login · jelszó / password · 2FA kódok / codes) | Vaultwarden | __________________________________ |
|
||||
| S12 | Gmail `felhom.eu@gmail.com` (jelszó / password · 2FA kódok / codes) | Vaultwarden | __________________________________ |
|
||||
| S19 | CC secrets age kulcs / key (`admin/cc-secrets`) | DooPlex `~/.config/sops/age/keys.txt`; Vaultwarden | `AGE-SECRET-KEY-1`__________________ |
|
||||
| S3 | Vaultwarden mesterjelszó / master password | a fejedben / in your head | (csak ha akarod / only if you choose) |
|
||||
|
||||
**Not secret, needed with the keys:**
|
||||
- ep0: `167.233.158.164` (Hetzner, `felhom-hetzner`); PBS datastore `felhom-offsite`, namespace `operator`;
|
||||
its certificate fingerprint `c6:07:28:3f:5b:7b:5a:41:90:28:d7:ca:4f:37:14:70:56:39:2e:2f:0b:71:e8:06:ca:60:4a:d5:56:5f:3c:fd`.
|
||||
- Restore order: `total-loss-of-dooplex.md` (in the restored Gitea, repo `felhom.eu`, `documentation/runbooks/`).
|
||||
**Print that page too** — the runbook itself is inside the copy it explains how to open.
|
||||
|
||||
**Re-print when** a key above is rotated, and once a year.
|
||||
**Nem titok, de kell / not secret, needed:** ep0 = `167.233.158.164` (Hetzner, `felhom-hetzner`), PBS `felhom-offsite`, névtér / namespace `operator`; tanúsítvány / certificate
|
||||
`c6:07:28:3f:5b:7b:5a:41:90:28:d7:ca:4f:37:14:70:56:39:2e:2f:0b:71:e8:06:ca:60:4a:d5:56:5f:3c:fd`. Kiadások másolata / release copy: ep0 `/srv/felhom-release-copy/data`.
|
||||
Sorrend / order: `total-loss-of-dooplex.md` — **nyomtasd ki azt is / print it too.** Újranyomtatás / re-print: kulcscserekor és évente / when a key rotates, and yearly.
|
||||
|
||||
@@ -19,8 +19,7 @@ build tree. **Assume all of it is gone, and the house with it.**
|
||||
| The boxes themselves | they keep running and backing up; they cannot report, update or mail until the hub is back | — |
|
||||
| The operator's workstation (if it has them) | the WireGuard peer to ep0 (`10.77.0.250`), maybe one of ep0's two root SSH keys, maybe a Bitwarden client's offline vault cache | the operator |
|
||||
|
||||
**Lost with DooPlex and NOT in any copy** (measured 2026-10-09): the container registry (rebuild the images from
|
||||
code); *(the operator signing keys were here until 2026-10-09 afternoon; they now ride the nightly copy — R-924)*; DooPlex's own SSH key (one of ep0's two root keys);
|
||||
**Lost with DooPlex and NOT in any copy** (measured 2026-10-09): *(the container registry was here until 2026-10-23: the released images and agent packages now ride a nightly copy to ep0 — D33, „Restore the release copy" below; the rest of the registry — older tags, rc builds, non-Felhom images — is still lost)*; *(the operator signing keys were here until 2026-10-09 afternoon; they now ride the nightly copy — R-924)*; DooPlex's own SSH key (one of ep0's two root keys);
|
||||
the build tree and drills (`/mnt/5_hdd/felhom.eu/{build,drills}`); `.claude-memory` (same-disk restic only); the
|
||||
local restic repos; Prometheus history; DooPlex's other homelab apps (not Felhom; Longhorn on DooPlex only).
|
||||
|
||||
@@ -66,16 +65,48 @@ local restic repos; Prometheus history; DooPlex's other homelab apps (not Felhom
|
||||
copy, use that. Every later login comes from here.
|
||||
6. **The Secrets export:** `gpg --batch --passphrase-file <S5 file> -d secrets-<stamp>.yaml.gpg > secrets.yaml` —
|
||||
every k8s Secret of the old cluster (shred it after use).
|
||||
7. **Gitea:** `gitea-restore.md` (on a normal network this time). Then rebuild the images from the code into the new
|
||||
registry: hub, controller, agent (`RUNBOOK-manual-build.md`).
|
||||
7. **Gitea:** `gitea-restore.md` (on a normal network this time). Then put the released images and agent packages
|
||||
back into its registry from ep0 — „Restore the release copy" below (same digests; no rebuild). Rebuild from code
|
||||
(`RUNBOOK-manual-build.md`) only what the copy does not hold.
|
||||
8. **The hub:** k3s, then `RUNBOOK-hub-db-offsite-backup.md` §3 with S2 and S4 — **start it with mail held**
|
||||
(§3 step 1: the `MAIL-HOLD` marker — shipped in hub 0.145.0, 2026-10-10).
|
||||
9. **DNS:** in Cloudflare (S11) point `hub.felhom.eu` (today a CNAME to `dooplex.hopto.org`) and `gitea.dooplex.hu` at
|
||||
the new place. The boxes reconnect by themselves: their API keys are in the restored hub DB.
|
||||
10. **Signing:** the keys are in the restored copy (`signing/`) and on the sheet (S6) — put them back and sign as before. Without it, no box accepts an agent update or a
|
||||
bundle until it is re-enrolled on site (`04` §4).
|
||||
11. **Backups again:** reinstall `scripts/hub-db-backup/` and `scripts/dooplex-offsite/` from the restored code, the
|
||||
tunnel and the tokens, and check the next night's copies.
|
||||
11. **Backups again:** reinstall `scripts/hub-db-backup/`, `scripts/dooplex-offsite/` and `scripts/release-copy/` from
|
||||
the restored code, the tunnel and the tokens (the release copy needs a new read-only registry token and a new SSH
|
||||
key in ep0's `felhom-relcopy` `authorized_keys`), and check the next night's copies.
|
||||
|
||||
## Restore the release copy (D33, since 2026-10-23)
|
||||
|
||||
**What it is.** Every night at 01:40 DooPlex copies the newest 3 release tags of `felhom-controller`, `felhom-hub` and
|
||||
`felhom-samba`, and the newest 5 agent versions (binary + config bundle), to ep0 `/srv/felhom-release-copy/data`
|
||||
(`scripts/release-copy/`). Images are in skopeo's `dir:` layout — the registry's own bytes — so a push back gives the
|
||||
**same digest** as the release. Nothing in it is secret; it is not encrypted. Alarm: `FelhomReleaseCopyStale`.
|
||||
|
||||
1. **Get it off ep0** (ep0 root — step 2): `rsync -a root@167.233.158.164:/srv/felhom-release-copy/data/ ./release-copy/`
|
||||
(or work on ep0 itself). Then `cd release-copy && sha256sum -c --quiet SHA256SUMS` — every line must pass.
|
||||
2. **Images back into the new registry**, one per tag (log in to the new Gitea first: `skopeo login <gitea host>`):
|
||||
```bash
|
||||
for d in images/*/*; do repo=${d#images/}; repo=${repo%%/*}; tag=${d##*/}
|
||||
skopeo copy --all --preserve-digests "dir:$d" "docker://<gitea host>/admin/$repo:$tag"
|
||||
echo "$repo:$tag $(skopeo inspect --raw "docker://<gitea host>/admin/$repo:$tag" | sha256sum | cut -c1-64) want $(cut -c8- "$d/DIGEST")"
|
||||
done
|
||||
```
|
||||
The two hashes on each line must be equal.
|
||||
3. **Agent packages back** (Gitea's generic registry, a token with package write):
|
||||
```bash
|
||||
for d in agent/*; do v=${d#agent/}; for f in felhom-agent felhom-config-bundle.json; do
|
||||
curl -fsS -u admin:"$(cat token-file)" -X PUT --upload-file "$d/$f" \
|
||||
"https://<gitea host>/api/packages/admin/generic/felhom-agent/$v/$f"; done; done
|
||||
```
|
||||
Check each download's sha256 against `agent/<v>/<file>.sha256`.
|
||||
4. **Keep the registry's name** (`gitea.dooplex.hu`, step 9's DNS) — the boxes pull `gitea.dooplex.hu/admin/…` by name.
|
||||
|
||||
**Proven 2026-10-23** (`audits/day-2026-10-23/08-release-copy-D33.txt`): the copy was pulled back from ep0, every
|
||||
sha256 matched, every image re-verified against its digest, and controller 0.340.0 from it loaded on a scratch box
|
||||
(image ID = its config digest; a container ran). **Not yet walked:** step 2 into a real new registry, and step 3.
|
||||
|
||||
## Walked on paper (2026-10-09): what still needs DooPlex if the operator has only the sheet, memory and a new machine
|
||||
|
||||
@@ -90,7 +121,7 @@ local restic repos; Prometheus history; DooPlex's other homelab apps (not Felhom
|
||||
| the hub's seal key | S4 (or S1+S5) | sheet, or via S5 |
|
||||
| the hub DB | S2 (or Vaultwarden) | sheet, or via S3 |
|
||||
| sign a box update | S6 | **sheet — today exists on DooPlex only** |
|
||||
| hub image | the registry (lost) | rebuild from restored code — needs a Go/Docker build machine, no secret |
|
||||
| hub, controller, samba images; agent packages | the release copy on ep0 (D33, nightly since 2026-10-23) | ep0 root (S8/S9) → „Restore the release copy" — no secret, no rebuild |
|
||||
| Gitea `main` → `felhom.eu` website, ISO host | Cloudflare (S11), the build tree (lost) | S11 on the sheet; the build tree is rebuilt from code |
|
||||
| Authentik (S17) | not in any copy | not needed for the business; DooPlex homelab apps that use it are out of scope |
|
||||
|
||||
|
||||
@@ -19,6 +19,7 @@
|
||||
box's controller). The checker now stands down for that declared state (the wait is the household's, it can take
|
||||
days); an undeclared consumed shape alarms as before. `TestR1230_AHeldBoxIsNotStuck`, red-proved.
|
||||
- `scripts/wire_contract_gate.py`: `wireguard.last_handshake_age_s` left the unread list (the hub reads it since 0.166.0).
|
||||
- **D34 (operator yes 2026-10-23, R-232 G5) — the database snapshot every 6 hours.** The hub writes its `VACUUM INTO` snapshot at 02:00, 08:00, 14:00 and 20:00 Budapest (`dbsnap.Slots`, keep 2 unchanged), and the start-up catch-up fires when the newest is older than 6 h (`dbsnap.CatchUpAge`, was 24 h). DooPlex's push now runs :30 after each slot (`scripts/hub-db-backup`, refuses a newest copy older than 7 h — one slot gap + 1 h — was 26 h; waits for a snapshot being written), so the off-site hub copy is at most about 6.5 h old instead of a day. Added the same day as the build, on the operator's word (not soaked). Tests `TestD34_SlotsAreEverySixHours`, `TestR173_MainSchedulesTheDBSnapshot` (now the loop), and the script's `test_schedules` (the timer read against the hub's slots) and `test_d34_a_missed_slot_is_not_pushed_again` — each red-proved.
|
||||
|
||||
## v0.166.0 (unreleased — built 2026-10-22 night, NOT deployed; the deploy is the operator's yes) — the clock-ahead tunnel alarm (R-1204)
|
||||
|
||||
|
||||
@@ -944,9 +944,9 @@ func main() {
|
||||
}
|
||||
}()
|
||||
|
||||
// Nightly DB snapshot (R-173, v0.136.0, `05` §16.3): VACUUM INTO <data>/snapshots/, keep 2. DooPlex's
|
||||
// felhom-hub-db-backup.service picks up the newest at 02:30, checks, encrypts and pushes it to ep0. A start-up
|
||||
// catch-up runs one when the newest is older than 24 h (a pod that was down at 02:00).
|
||||
// DB snapshot every 6 hours (R-173, v0.136.0, `05` §16.3; D34 v0.167.0): VACUUM INTO <data>/snapshots/, keep 2.
|
||||
// DooPlex's felhom-hub-db-backup.service picks up the newest at :30 after each slot, checks, encrypts and pushes it
|
||||
// to ep0. A start-up catch-up runs one when the newest is older than one slot gap (a pod that was down at a slot).
|
||||
snapMaker := &dbsnap.Maker{Store: dataStore, Dir: filepath.Join(cfg.Server.DataDir, "snapshots"), Logger: logger}
|
||||
runSnapshot := func() {
|
||||
sctx, cancel := context.WithTimeout(ctx, 10*time.Minute)
|
||||
@@ -955,10 +955,12 @@ func main() {
|
||||
logger.Printf("[ERROR] db snapshot failed: %v", err)
|
||||
}
|
||||
}
|
||||
if dbsnap.NeedsCatchUp(snapMaker.Dir, time.Now(), 24*time.Hour) {
|
||||
if dbsnap.NeedsCatchUp(snapMaker.Dir, time.Now(), dbsnap.CatchUpAge) {
|
||||
go runSnapshot()
|
||||
}
|
||||
go scheduleDaily(ctx, "db-snapshot", "02:00", runSnapshot, logger)
|
||||
for _, at := range dbsnap.Slots {
|
||||
go scheduleDaily(ctx, "db-snapshot "+at, at, runSnapshot, logger)
|
||||
}
|
||||
|
||||
// Backup deadline checker — runs daily at 05:00 Budapest
|
||||
go scheduleDaily(ctx, "deadline-check", "05:00", func() {
|
||||
|
||||
@@ -4,13 +4,13 @@ import (
|
||||
"go/ast"
|
||||
"go/parser"
|
||||
"go/token"
|
||||
"strconv"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// R-173 seam wiring: main() must schedule the nightly db snapshot — a scheduleDaily call named "db-snapshot" at
|
||||
// "02:00", before DooPlex's 02:30 push — and must run the start-up catch-up (dbsnap.NeedsCatchUp). Without them the
|
||||
// dbsnap package exists and DooPlex pushes nothing new. RED-PROOF: delete either call → this test fails.
|
||||
// R-173 seam wiring, D34: main() must schedule a db snapshot at every dbsnap.Slots time — a `for _, at := range
|
||||
// dbsnap.Slots` loop calling scheduleDaily(ctx, <name>, at, …) — and must run the start-up catch-up
|
||||
// (dbsnap.NeedsCatchUp with dbsnap.CatchUpAge). Without them the dbsnap package exists and DooPlex pushes nothing
|
||||
// new. RED-PROOF: delete the loop, schedule a literal time instead, or pass 24*time.Hour → this test fails.
|
||||
func TestR173_MainSchedulesTheDBSnapshot(t *testing.T) {
|
||||
f, err := parser.ParseFile(token.NewFileSet(), "main.go", nil, 0)
|
||||
if err != nil {
|
||||
@@ -18,30 +18,39 @@ func TestR173_MainSchedulesTheDBSnapshot(t *testing.T) {
|
||||
}
|
||||
scheduled, catchUp := false, false
|
||||
ast.Inspect(f, func(n ast.Node) bool {
|
||||
if r, ok := n.(*ast.RangeStmt); ok {
|
||||
sel, ok := r.X.(*ast.SelectorExpr)
|
||||
v, vok := r.Value.(*ast.Ident)
|
||||
if ok && vok && sel.Sel.Name == "Slots" {
|
||||
ast.Inspect(r.Body, func(m ast.Node) bool {
|
||||
c, ok := m.(*ast.CallExpr)
|
||||
if !ok {
|
||||
return true
|
||||
}
|
||||
if id, ok := c.Fun.(*ast.Ident); ok && id.Name == "scheduleDaily" && len(c.Args) >= 3 {
|
||||
if a, ok := c.Args[2].(*ast.Ident); ok && a.Name == v.Name {
|
||||
scheduled = true
|
||||
}
|
||||
}
|
||||
return true
|
||||
})
|
||||
}
|
||||
}
|
||||
c, ok := n.(*ast.CallExpr)
|
||||
if !ok {
|
||||
return true
|
||||
}
|
||||
if id, ok := c.Fun.(*ast.Ident); ok && id.Name == "scheduleDaily" && len(c.Args) >= 3 {
|
||||
name, _ := c.Args[1].(*ast.BasicLit)
|
||||
at, _ := c.Args[2].(*ast.BasicLit)
|
||||
if name != nil && at != nil {
|
||||
n, _ := strconv.Unquote(name.Value)
|
||||
a, _ := strconv.Unquote(at.Value)
|
||||
if n == "db-snapshot" && a == "02:00" {
|
||||
scheduled = true
|
||||
}
|
||||
if sel, ok := c.Fun.(*ast.SelectorExpr); ok && sel.Sel.Name == "NeedsCatchUp" && len(c.Args) == 3 {
|
||||
if a, ok := c.Args[2].(*ast.SelectorExpr); ok && a.Sel.Name == "CatchUpAge" {
|
||||
catchUp = true
|
||||
}
|
||||
}
|
||||
if sel, ok := c.Fun.(*ast.SelectorExpr); ok && sel.Sel.Name == "NeedsCatchUp" {
|
||||
catchUp = true
|
||||
}
|
||||
return true
|
||||
})
|
||||
if !scheduled {
|
||||
t.Error(`cmd/hub/main.go never calls scheduleDaily(ctx, "db-snapshot", "02:00", …) — no nightly snapshot`)
|
||||
t.Error("cmd/hub/main.go never schedules a snapshot at each dbsnap.Slots time — no 6-hourly snapshot (D34)")
|
||||
}
|
||||
if !catchUp {
|
||||
t.Error("cmd/hub/main.go never calls dbsnap.NeedsCatchUp — a pod down at 02:00 leaves a stale snapshot")
|
||||
t.Error("cmd/hub/main.go never calls dbsnap.NeedsCatchUp(…, dbsnap.CatchUpAge) — a pod down at a slot leaves a stale snapshot")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -26,6 +26,15 @@ import (
|
||||
// grown to 2 GiB for it, operator choice 2026-10-05).
|
||||
const Keep = 2
|
||||
|
||||
// Slots are the Budapest times the hub writes a snapshot — every 6 hours (D34, operator yes 2026-10-23, R-232 G5).
|
||||
// DooPlex's felhom-hub-db-backup.timer pushes the newest at :30 after each slot, so the off-site copy is at most
|
||||
// about 6.5 h old instead of a day. Pinned by TestD34_SlotsAreEverySixHours and cmd/hub TestR173_MainSchedulesTheDBSnapshot.
|
||||
var Slots = []string{"02:00", "08:00", "14:00", "20:00"}
|
||||
|
||||
// CatchUpAge is the start-up catch-up limit: a pod that was down at a slot writes one at start when the newest is
|
||||
// older than one slot gap. Pinned by TestD34_SlotsAreEverySixHours.
|
||||
const CatchUpAge = 6 * time.Hour
|
||||
|
||||
// Prefix and Suffix make a snapshot's file name: hub-20261005T020000Z.db.
|
||||
const (
|
||||
Prefix = "hub-"
|
||||
|
||||
@@ -206,3 +206,43 @@ func TestNeedsCatchUp(t *testing.T) {
|
||||
t.Error("25 h old: want catch-up")
|
||||
}
|
||||
}
|
||||
|
||||
// D34 (operator yes 2026-10-23): a snapshot every 6 hours. The consequence pinned: no gap between two slots — the
|
||||
// wrap past midnight included — is longer than CatchUpAge, so the off-site copy (pushed :30 after each slot) is never
|
||||
// more than one slot gap behind; 02:00 stays a slot (the nightly restore test and the runbook read it); the start-up
|
||||
// catch-up fires for a pod that missed exactly one slot. RED-PROOF: drop a slot, or set CatchUpAge to 24 h → fails.
|
||||
func TestD34_SlotsAreEverySixHours(t *testing.T) {
|
||||
if len(Slots) != 4 {
|
||||
t.Fatalf("Slots = %v, want 4 (every 6 h)", Slots)
|
||||
}
|
||||
var mins []int
|
||||
has0200 := false
|
||||
for _, s := range Slots {
|
||||
tm, err := time.Parse("15:04", s)
|
||||
if err != nil {
|
||||
t.Fatalf("slot %q: %v", s, err)
|
||||
}
|
||||
mins = append(mins, tm.Hour()*60+tm.Minute())
|
||||
has0200 = has0200 || s == "02:00"
|
||||
}
|
||||
if !has0200 {
|
||||
t.Error("02:00 is not a slot")
|
||||
}
|
||||
for i := range mins {
|
||||
next := mins[(i+1)%len(mins)]
|
||||
if i == len(mins)-1 {
|
||||
next += 24 * 60
|
||||
}
|
||||
if gap := time.Duration(next-mins[i]) * time.Minute; gap <= 0 || gap > CatchUpAge {
|
||||
t.Errorf("gap after %s is %s (want 0 < gap <= %s)", Slots[i], gap, CatchUpAge)
|
||||
}
|
||||
}
|
||||
dir := t.TempDir()
|
||||
_ = os.WriteFile(filepath.Join(dir, "hub-20261023T000000Z.db"), nil, 0o600) // 02:00 Budapest (CEST)
|
||||
if !NeedsCatchUp(dir, time.Date(2026, 10, 23, 7, 0, 1, 0, time.UTC), CatchUpAge) {
|
||||
t.Error("a pod up at 09:00 CEST that missed the 08:00 slot (newest 7 h old): want catch-up")
|
||||
}
|
||||
if NeedsCatchUp(dir, time.Date(2026, 10, 23, 5, 0, 0, 0, time.UTC), CatchUpAge) {
|
||||
t.Error("newest 5 h old: want no catch-up")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -128,7 +128,7 @@ spec:
|
||||
spec:
|
||||
containers:
|
||||
- name: hub
|
||||
image: gitea.dooplex.hu/admin/felhom-hub:0.165.0
|
||||
image: gitea.dooplex.hu/admin/felhom-hub:0.167.0
|
||||
ports:
|
||||
- containerPort: 8080
|
||||
name: http
|
||||
|
||||
@@ -344,7 +344,7 @@ spec:
|
||||
image: registry.k8s.io/git-sync/git-sync:v4.4.0
|
||||
args:
|
||||
- --repo=http://gitea.gitea-system.svc.cluster.local:3000/admin/felhom.eu.git
|
||||
- --ref=installer-v1.35.0
|
||||
- --ref=installer-v1.36.0
|
||||
- --root=/git-scripts
|
||||
- --link=current
|
||||
- --period=30s
|
||||
@@ -390,7 +390,7 @@ spec:
|
||||
image: registry.k8s.io/git-sync/git-sync:v4.4.0
|
||||
args:
|
||||
- --repo=http://gitea.gitea-system.svc.cluster.local:3000/admin/felhom.eu.git
|
||||
- --ref=installer-v1.35.0
|
||||
- --ref=installer-v1.36.0
|
||||
- --root=/git-scripts
|
||||
- --link=current
|
||||
- --one-time
|
||||
|
||||
@@ -1,3 +1,8 @@
|
||||
## 2026-10-23 — `release-copy/`: the released images and agent packages leave DooPlex nightly (D33, R-1227); `hub-db-backup/`: every 6 hours (D34)
|
||||
|
||||
- **`release-copy/felhom-release-copy`** (new, DooPlex root, timer 01:40): the newest 3 release tags of felhom-controller, felhom-hub and felhom-samba (`skopeo copy --all --preserve-digests` into `dir:` — the registry's own bytes, so a push back gives the SAME digest; a `docker save` tar would not) and the newest 5 agent versions (binary + config bundle). Before anything leaves: the top manifest hashes to the registry's digest, every blob and child manifest to its name, every named blob is present, each agent file matches the registry's sha256; the staged tree ≤ the cap (3072 MB). Then `rsync --delete` to ep0's limited user `felhom-relcopy` (rrsync, one directory), and a `--checksum --dry-run` must show no difference before the success signal. Spoiled tags (0.315.0, 0.316.3) are never copied. 13 tests (`test_release_copy.py`), four red-proved.
|
||||
- **`hub-db-backup/`** — the push runs at 02:30, 08:30, 14:30 and 20:30 (the hub snapshots every 6 hours since hub 0.167.0) and refuses a newest snapshot older than 7 h (one slot gap + 1 h; was 26 h), and waits (up to 20 min) while the hub is writing one. **Corrected the same day:** 2 h was shipped first, and its first run refused the timer's own catch-up while the hub's start-up snapshot was still being written — one false failure mail to admin@ at 10:28 (R-1234). Tests: the timer read against the hub's slots; two missed slots refused; a restart within one slot pushed; a snapshot being written waited for — each red-proved (the last reproduces the 10:28 run).
|
||||
|
||||
## 2026-10-23 — felhom-host-install.sh 1.36.0 (pushed; the tag `installer-v1.36.0` waits for the operator's yes): the uninstall removes the root-SSH drop-in (R-982, D4)
|
||||
|
||||
The agent's config bundle (next agent release) gives the host's own sshd `/etc/ssh/sshd_config.d/10-felhom-root-key-only.conf`
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
#!/bin/sh
|
||||
# felhom-hub-db-backup — push the hub's newest nightly snapshot to ep0's PBS, encrypted (R-173, decision A).
|
||||
# Runs on DooPlex as root from felhom-hub-db-backup.timer (02:30; the hub writes the snapshot at 02:00).
|
||||
# felhom-hub-db-backup — push the hub's newest snapshot to ep0's PBS, encrypted (R-173, decision A).
|
||||
# Runs on DooPlex as root from felhom-hub-db-backup.timer (02:30, 08:30, 14:30, 20:30; the hub writes a snapshot at
|
||||
# 02:00, 08:00, 14:00 and 20:00 since hub 0.167.0 — D34).
|
||||
# Runbook: documentation/runbooks/RUNBOOK-hub-db-offsite-backup.md Step 4. Pinned by test_hub_db_backup.py.
|
||||
#
|
||||
# Refuses to push — and so never writes the success signal — when: no snapshot exists; the newest is older than
|
||||
@@ -11,7 +12,13 @@ set -eu
|
||||
CONF=${FELHOM_HUBBK_CONF:-/etc/felhom-hub-backup}
|
||||
STATE=${FELHOM_HUBBK_STATE:-/var/lib/felhom-hub-backup}
|
||||
TEXTFILE_DIR=${FELHOM_HUBBK_TEXTFILE_DIR:-/var/lib/node_exporter/textfile_collector}
|
||||
MAX_AGE_H=${FELHOM_HUBBK_MAX_AGE_H:-26}
|
||||
# D34: one slot gap + 1 h. The hub writes a catch-up snapshot at start when its newest is older than 6 h
|
||||
# (dbsnap.CatchUpAge), so after any restart the newest is either < 7 h old or being written (waited for below). A hub
|
||||
# that really stopped is refused at the second missed slot. (2 h was tried first, 2026-10-23: it refused the timer's own
|
||||
# catch-up run while the hub's catch-up snapshot was still being written — a false "stopped snapshotting" mail.)
|
||||
MAX_AGE_H=${FELHOM_HUBBK_MAX_AGE_H:-7}
|
||||
WAIT_S=${FELHOM_HUBBK_WAIT_S:-1200}
|
||||
POLL_S=${FELHOM_HUBBK_POLL_S:-15}
|
||||
NOW=${FELHOM_HUBBK_NOW:-$(date +%s)}
|
||||
. "$CONF/env" # PBS_REPOSITORY_PUSH, PBS_FINGERPRINT (no secrets in this file)
|
||||
|
||||
@@ -24,6 +31,13 @@ mkdir -p "$STAGE"; chmod 700 "$STATE" "$STAGE"
|
||||
rm -f "$STAGE"/*
|
||||
trap 'if [ -f "$STAGE/hub.db" ]; then shred -u "$STAGE/hub.db" 2>/dev/null || rm -f "$STAGE/hub.db"; fi' EXIT
|
||||
|
||||
# A snapshot being written (hub-*.db.tmp) is waited for, so a push never takes the older one meanwhile (D34).
|
||||
WAITED=0
|
||||
while kubectl -n felhom-system exec deploy/hub -- sh -c 'ls /data/snapshots/hub-*.db.tmp >/dev/null 2>&1'; do
|
||||
[ "$WAITED" -ge "$WAIT_S" ] && { log "a snapshot is still being written after ${WAIT_S} s — taking the newest finished one"; break; }
|
||||
[ "$WAITED" -eq 0 ] && log "a snapshot is being written in the hub pod — waiting for it (up to ${WAIT_S} s)"
|
||||
sleep "$POLL_S"; WAITED=$((WAITED + POLL_S))
|
||||
done
|
||||
SNAP=$(kubectl -n felhom-system exec deploy/hub -- sh -c 'ls -1 /data/snapshots/hub-*.db 2>/dev/null | tail -n 1') || die "listing snapshots in the hub pod"
|
||||
[ -n "$SNAP" ] || die "no snapshot in the hub pod's /data/snapshots"
|
||||
NAME=${SNAP##*/}
|
||||
|
||||
@@ -1,9 +1,13 @@
|
||||
# Versioned in felhom.eu/scripts/hub-db-backup/ (R-231); installed by install.sh.
|
||||
[Unit]
|
||||
Description=Felhom: push the hub DB snapshot to ep0 (R-173) — schedule
|
||||
Description=Felhom: push the hub DB snapshot to ep0 (R-173, D34) — every 6 hours
|
||||
|
||||
[Timer]
|
||||
# D34 (operator yes 2026-10-23): every 6 h, :30 after each hub snapshot slot (hub dbsnap.Slots 02/08/14/20:00).
|
||||
OnCalendar=*-*-* 02:30:00
|
||||
OnCalendar=*-*-* 08:30:00
|
||||
OnCalendar=*-*-* 14:30:00
|
||||
OnCalendar=*-*-* 20:30:00
|
||||
Persistent=true
|
||||
RandomizedDelaySec=2min
|
||||
|
||||
|
||||
@@ -8,6 +8,7 @@ was written. Run: python3 scripts/hub-db-backup/test_hub_db_backup.py
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import sqlite3
|
||||
import stat
|
||||
@@ -184,6 +185,42 @@ class Push(Base):
|
||||
self.assertEqual(self.pushed(), [])
|
||||
self.assertFalse(self.signal("felhom_hub_db_backup.prom"))
|
||||
|
||||
def test_d34_two_missed_slots_are_refused(self):
|
||||
# D34: snapshots every 6 h; the limit is one slot gap + 1 h (7 h) — a newest copy 8 h old means the hub missed
|
||||
# two slots, or stopped. RED-PROOF: MAX_AGE_H back to 26 → this pushes.
|
||||
self.snapshot(age_s=8 * 3600)
|
||||
r = self.run_script(PUSH)
|
||||
self.assertNotEqual(r.returncode, 0)
|
||||
self.assertIn("stopped snapshotting", r.stderr)
|
||||
self.assertEqual(self.pushed(), [])
|
||||
self.assertFalse(self.signal("felhom_hub_db_backup.prom"))
|
||||
|
||||
def test_d34_a_restart_within_one_slot_is_not_an_alarm(self):
|
||||
# the hub's catch-up fires only past 6 h, so after a restart the newest can be up to 6 h old — that is pushed,
|
||||
# not refused (2 h refused it: the false mail of 2026-10-23 10:28). RED-PROOF: MAX_AGE_H 2 → this fails.
|
||||
self.snapshot(age_s=int(5.9 * 3600))
|
||||
r = self.run_script(PUSH)
|
||||
self.assertEqual(r.returncode, 0, r.stderr)
|
||||
self.assertEqual(len(self.pushed()), 1)
|
||||
|
||||
def test_d34_waits_for_a_snapshot_being_written(self):
|
||||
# the 2026-10-23 10:28 case: the hub's catch-up snapshot is being written (hub-*.db.tmp) and the only finished
|
||||
# one is 8 h old. The push must wait for the new one, not refuse the old. RED-PROOF: drop the wait loop → fails.
|
||||
import threading
|
||||
self.snapshot(age_s=8 * 3600)
|
||||
snaps = self.j("pod", "snapshots")
|
||||
tmp = os.path.join(snaps, "hub-%s.db.tmp" % stamp(int(time.time())))
|
||||
src = [os.path.join(snaps, n) for n in os.listdir(snaps) if n.endswith(".db")][0]
|
||||
shutil.copy(src, tmp)
|
||||
def finish():
|
||||
time.sleep(1.5)
|
||||
os.rename(tmp, tmp[: -len(".tmp")])
|
||||
threading.Thread(target=finish).start()
|
||||
r = self.run_script(PUSH, FELHOM_HUBBK_POLL_S="1", FELHOM_HUBBK_WAIT_S="20")
|
||||
self.assertEqual(r.returncode, 0, r.stderr)
|
||||
self.assertIn("being written", r.stdout)
|
||||
self.assertEqual(len(self.pushed()), 1)
|
||||
|
||||
def test_truncated_copy_is_not_pushed(self):
|
||||
self.snapshot()
|
||||
r = self.run_script(PUSH, FAKE_TRUNCATE="1")
|
||||
@@ -268,7 +305,14 @@ class Units(unittest.TestCase):
|
||||
return open(os.path.join(HERE, n)).read()
|
||||
|
||||
def test_schedules(self):
|
||||
self.assertIn("OnCalendar=*-*-* 02:30:00", self.read("felhom-hub-db-backup.timer"))
|
||||
# D34: the push runs :30 after EVERY hub snapshot slot — read from the hub's own source, so a slot moved on
|
||||
# one side only fails here (RED-PROOF: drop one OnCalendar line, or change a hub slot → fails).
|
||||
src = open(os.path.join(HERE, "..", "..", "hub", "internal", "dbsnap", "dbsnap.go")).read()
|
||||
slots = re.findall(r'"(\d\d):(\d\d)"', re.search(r"var Slots = \[\]string\{([^}]*)\}", src).group(1))
|
||||
self.assertEqual(len(slots), 4)
|
||||
want = sorted("OnCalendar=*-*-* %02d:%02d:00" % (int(h), int(m) + 30) for h, m in slots)
|
||||
got = sorted(l.strip() for l in self.read("felhom-hub-db-backup.timer").splitlines() if l.startswith("OnCalendar="))
|
||||
self.assertEqual(got, want)
|
||||
self.assertIn("OnCalendar=Sun *-*-* 04:30:00", self.read("felhom-hub-db-restore-test.timer"))
|
||||
|
||||
def test_units_run_the_installed_scripts(self):
|
||||
|
||||
@@ -0,0 +1,300 @@
|
||||
#!/usr/bin/env python3
|
||||
"""felhom-release-copy — copy the released images and agent packages off DooPlex to ep0, nightly (D33, R-1227).
|
||||
|
||||
Runs on DooPlex as root from felhom-release-copy.timer. Runbook: documentation/runbooks/total-loss-of-dooplex.md
|
||||
(„Restore the release copy"). Pinned by test_release_copy.py.
|
||||
|
||||
WHAT. The newest KEEP release tags (X.Y.Z only) of each image in REPOS, and the newest AGENT_KEEP agent versions (the
|
||||
binary and its config bundle). After a DooPlex loss these are what every box update, reinstall and new install pulls.
|
||||
|
||||
HOW, and why skopeo, not `docker save`. Each image tag is copied with `skopeo copy --all --preserve-digests` into
|
||||
skopeo's `dir:` layout: the registry's own manifest bytes and compressed layer blobs, unchanged. Pushed back into any
|
||||
registry, the image has THE SAME DIGEST as the release (a `docker save` tar is re-compressed on the next push, so the
|
||||
digest — what the „a version is pushed once" rule and the hub's spoiled-tag list are about — would differ). It also
|
||||
needs no Docker daemon on DooPlex.
|
||||
|
||||
CHECKS, before anything leaves: the sha256 of the copied top manifest equals the registry's digest; every blob and
|
||||
child manifest hashes to its own name; every blob a manifest names is present; each agent file's sha256 equals the
|
||||
registry's record. The staged tree must be ≤ CAP_MB (the cap — the sender enforces it; ep0 has no quota). Then
|
||||
`rsync --delete` to ep0 (the limited user `felhom-relcopy`, rrsync-confined to one directory), and a second
|
||||
`rsync --checksum --dry-run` must report NO difference. Only then is the success signal written (CLAUDE.md „presence
|
||||
is not success"). Versions beyond KEEP / AGENT_KEEP are deleted locally, and so on ep0 by `--delete`.
|
||||
"""
|
||||
import base64
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
import urllib.request
|
||||
|
||||
CONF = os.environ.get("FELHOM_RC_CONF", "/etc/felhom-release-copy")
|
||||
STATE = os.environ.get("FELHOM_RC_STATE", "/var/lib/felhom-release-copy")
|
||||
TEXTFILE_DIR = os.environ.get("FELHOM_RC_TEXTFILE_DIR", "/var/lib/node_exporter/textfile_collector")
|
||||
REGISTRY = "gitea.dooplex.hu"
|
||||
OWNER = "admin"
|
||||
SPOILED = {"felhom-controller": {"0.315.0", "0.316.3"}} # withdrawn builds (R-1085) — never offered, never copied
|
||||
AGENT_FILES = ("felhom-agent", "felhom-config-bundle.json")
|
||||
HEX64 = re.compile(r"^[0-9a-f]{64}$")
|
||||
RELEASE = re.compile(r"^\d+\.\d+\.\d+$")
|
||||
|
||||
|
||||
class Refused(Exception):
|
||||
pass
|
||||
|
||||
|
||||
def log(msg):
|
||||
print("felhom-release-copy: " + msg, flush=True)
|
||||
|
||||
|
||||
def load_env(path):
|
||||
env = {}
|
||||
for line in open(path, encoding="utf-8"):
|
||||
line = line.strip()
|
||||
if line and not line.startswith("#") and "=" in line:
|
||||
k, v = line.split("=", 1)
|
||||
env[k.strip()] = v.strip().strip("'\"")
|
||||
return env
|
||||
|
||||
|
||||
def newest(versions, n, spoiled=()):
|
||||
rel = [v for v in versions if RELEASE.match(v) and v not in spoiled]
|
||||
rel.sort(key=lambda v: tuple(int(x) for x in v.split(".")))
|
||||
return rel[-n:] if n > 0 else []
|
||||
|
||||
|
||||
def sha256_file(path):
|
||||
h = hashlib.sha256()
|
||||
with open(path, "rb") as f:
|
||||
for chunk in iter(lambda: f.read(1 << 20), b""):
|
||||
h.update(chunk)
|
||||
return h.hexdigest()
|
||||
|
||||
|
||||
def verify_image_dir(d, want_digest):
|
||||
"""Refuses unless the dir: copy in d is exactly the image want_digest names (see the module docstring)."""
|
||||
top = os.path.join(d, "manifest.json")
|
||||
if not os.path.isfile(top):
|
||||
raise Refused(f"{d}: no manifest.json")
|
||||
got = "sha256:" + sha256_file(top)
|
||||
if got != want_digest:
|
||||
raise Refused(f"{d}: manifest digest {got} != registry {want_digest}")
|
||||
names = set(os.listdir(d))
|
||||
manifests = [top]
|
||||
for n in sorted(names):
|
||||
p = os.path.join(d, n)
|
||||
if HEX64.match(n):
|
||||
if sha256_file(p) != n:
|
||||
raise Refused(f"{d}: blob {n} does not hash to its name")
|
||||
elif n.endswith(".manifest.json") and HEX64.match(n[: -len(".manifest.json")]):
|
||||
if sha256_file(p) != n[: -len(".manifest.json")]:
|
||||
raise Refused(f"{d}: child manifest {n} does not hash to its name")
|
||||
manifests.append(p)
|
||||
for m in manifests:
|
||||
doc = json.load(open(m, encoding="utf-8"))
|
||||
for ref in doc.get("manifests", []):
|
||||
hx = ref["digest"].split(":", 1)[1]
|
||||
if hx + ".manifest.json" not in names:
|
||||
raise Refused(f"{d}: child manifest {hx[:12]} named by the index is missing")
|
||||
for ref in ([doc["config"]] if "config" in doc else []) + doc.get("layers", []):
|
||||
hx = ref["digest"].split(":", 1)[1]
|
||||
if hx not in names:
|
||||
raise Refused(f"{d}: blob {hx[:12]} named by a manifest is missing")
|
||||
|
||||
|
||||
def tree_bytes(root):
|
||||
total = 0
|
||||
for dp, _, fs in os.walk(root):
|
||||
for f in fs:
|
||||
total += os.lstat(os.path.join(dp, f)).st_size
|
||||
return total
|
||||
|
||||
|
||||
def prune(root, keep):
|
||||
"""Removes every directory under root whose name is not in keep; returns the removed names."""
|
||||
gone = []
|
||||
if os.path.isdir(root):
|
||||
for n in sorted(os.listdir(root)):
|
||||
if n not in keep:
|
||||
shutil.rmtree(os.path.join(root, n))
|
||||
gone.append(n)
|
||||
return gone
|
||||
|
||||
|
||||
class Registry:
|
||||
def __init__(self, user, token):
|
||||
self.auth = "Basic " + base64.b64encode(f"{user}:{token}".encode()).decode()
|
||||
|
||||
def get(self, url, raw=False):
|
||||
r = urllib.request.urlopen(urllib.request.Request(url, headers={"Authorization": self.auth}), timeout=120)
|
||||
return r.read() if raw else json.load(r)
|
||||
|
||||
def agent_versions(self):
|
||||
out, page = [], 1
|
||||
while True:
|
||||
batch = self.get(f"https://{REGISTRY}/api/v1/packages/{OWNER}?type=generic&q=felhom-agent&limit=50&page={page}")
|
||||
if not batch:
|
||||
return out
|
||||
out += [p["version"] for p in batch if p.get("name") == "felhom-agent"]
|
||||
page += 1
|
||||
|
||||
def agent_files(self, ver):
|
||||
return {f["name"]: f["sha256"] for f in self.get(
|
||||
f"https://{REGISTRY}/api/v1/packages/{OWNER}/generic/felhom-agent/{ver}/files")}
|
||||
|
||||
def agent_download(self, ver, name, dest):
|
||||
with open(dest, "wb") as f:
|
||||
f.write(self.get(f"https://{REGISTRY}/api/packages/{OWNER}/generic/felhom-agent/{ver}/{name}", raw=True))
|
||||
|
||||
|
||||
def run(cmd, **kw):
|
||||
return subprocess.run(cmd, check=True, text=True, capture_output=True, **kw)
|
||||
|
||||
|
||||
def skopeo(authfile, *args):
|
||||
return run(["skopeo", *args[:1], "--authfile", authfile, *args[1:]]).stdout
|
||||
|
||||
|
||||
def copy_images(reg_authfile, stage, repos, keep):
|
||||
images = os.path.join(stage, "images")
|
||||
os.makedirs(images, exist_ok=True)
|
||||
copied = []
|
||||
for repo in repos:
|
||||
tags = json.loads(skopeo(reg_authfile, "list-tags", f"docker://{REGISTRY}/{OWNER}/{repo}"))["Tags"]
|
||||
want = newest(tags, keep, SPOILED.get(repo, ()))
|
||||
if not want:
|
||||
raise Refused(f"{repo}: no release tag")
|
||||
rdir = os.path.join(images, repo)
|
||||
os.makedirs(rdir, exist_ok=True)
|
||||
for tag in want:
|
||||
ref = f"docker://{REGISTRY}/{OWNER}/{repo}:{tag}"
|
||||
digest = "sha256:" + hashlib.sha256(
|
||||
skopeo(reg_authfile, "inspect", "--raw", ref).encode()).hexdigest()
|
||||
d = os.path.join(rdir, tag)
|
||||
if os.path.isfile(os.path.join(d, "DIGEST")) and open(os.path.join(d, "DIGEST")).read().strip() == digest:
|
||||
verify_image_dir(d, digest) # already here — re-checked every night, not trusted
|
||||
continue
|
||||
tmp = d + ".tmp"
|
||||
shutil.rmtree(tmp, ignore_errors=True)
|
||||
shutil.rmtree(d, ignore_errors=True)
|
||||
skopeo(reg_authfile, "copy", "--all", "--preserve-digests", "--quiet", ref, f"dir:{tmp}")
|
||||
verify_image_dir(tmp, digest)
|
||||
with open(os.path.join(tmp, "DIGEST"), "w") as f:
|
||||
f.write(digest + "\n")
|
||||
os.rename(tmp, d)
|
||||
copied.append(f"{repo}:{tag}")
|
||||
gone = prune(rdir, set(want))
|
||||
log(f"{repo}: keeping {', '.join(want)}" + (f"; removed {', '.join(gone)}" if gone else ""))
|
||||
prune(images, set(repos))
|
||||
return copied
|
||||
|
||||
|
||||
def copy_agents(reg, stage, keep):
|
||||
adir = os.path.join(stage, "agent")
|
||||
os.makedirs(adir, exist_ok=True)
|
||||
want = newest(reg.agent_versions(), keep)
|
||||
if not want:
|
||||
raise Refused("felhom-agent: no release version")
|
||||
for ver in want:
|
||||
d = os.path.join(adir, ver)
|
||||
os.makedirs(d, exist_ok=True)
|
||||
recorded = reg.agent_files(ver)
|
||||
for name in AGENT_FILES:
|
||||
if name not in recorded:
|
||||
raise Refused(f"felhom-agent {ver}: the registry has no {name}")
|
||||
p = os.path.join(d, name)
|
||||
if not (os.path.isfile(p) and sha256_file(p) == recorded[name]):
|
||||
reg.agent_download(ver, name, p + ".tmp")
|
||||
if sha256_file(p + ".tmp") != recorded[name]:
|
||||
os.remove(p + ".tmp")
|
||||
raise Refused(f"felhom-agent {ver}/{name}: the download does not match the registry's sha256")
|
||||
os.rename(p + ".tmp", p)
|
||||
with open(os.path.join(d, name + ".sha256"), "w") as f:
|
||||
f.write(f"{recorded[name]} {name}\n")
|
||||
gone = prune(adir, set(want))
|
||||
log(f"felhom-agent: keeping {', '.join(want)}" + (f"; removed {', '.join(gone)}" if gone else ""))
|
||||
|
||||
|
||||
def write_sums(stage):
|
||||
lines = []
|
||||
for dp, _, fs in os.walk(stage):
|
||||
for f in fs:
|
||||
if f == "SHA256SUMS":
|
||||
continue
|
||||
p = os.path.join(dp, f)
|
||||
lines.append(f"{sha256_file(p)} {os.path.relpath(p, stage)}")
|
||||
with open(os.path.join(stage, "SHA256SUMS"), "w") as f:
|
||||
f.write("\n".join(sorted(lines, key=lambda l: l[66:])) + "\n")
|
||||
|
||||
|
||||
def rsync_to(env, stage, dry_check=False):
|
||||
ssh = (f"ssh -i {CONF}/id_ed25519 -o UserKnownHostsFile={CONF}/known_hosts -o StrictHostKeyChecking=yes "
|
||||
f"-o BatchMode=yes -o ConnectTimeout=30")
|
||||
cmd = ["rsync", "-a", "--delete", "-e", ssh]
|
||||
if dry_check:
|
||||
cmd += ["--checksum", "--dry-run", "--itemize-changes"]
|
||||
return run(cmd + [stage.rstrip("/") + "/", env["REMOTE"] + ":./"]).stdout
|
||||
|
||||
|
||||
def signal(nbytes):
|
||||
tmp = os.path.join(TEXTFILE_DIR, f"felhom_release_copy.prom.{os.getpid()}")
|
||||
with open(tmp, "w") as f:
|
||||
f.write("# HELP felhom_release_copy_last_success_timestamp_seconds Last verified copy of the releases to ep0 (D33).\n"
|
||||
"# TYPE felhom_release_copy_last_success_timestamp_seconds gauge\n"
|
||||
f"felhom_release_copy_last_success_timestamp_seconds {int(time.time())}\n"
|
||||
f"felhom_release_copy_last_success_bytes {nbytes}\n")
|
||||
os.chmod(tmp, 0o644)
|
||||
os.rename(tmp, os.path.join(TEXTFILE_DIR, "felhom_release_copy.prom"))
|
||||
|
||||
|
||||
def main():
|
||||
env = load_env(os.path.join(CONF, "env"))
|
||||
keep, akeep, cap = int(env.get("KEEP", "3")), int(env.get("AGENT_KEEP", "5")), int(env["CAP_MB"])
|
||||
repos = env["REPOS"].split()
|
||||
token = open(os.path.join(CONF, "registry-token"), encoding="utf-8").read().strip()
|
||||
user = env.get("REGISTRY_USER", OWNER)
|
||||
stage = os.path.join(STATE, "stage")
|
||||
os.makedirs(stage, exist_ok=True)
|
||||
os.chmod(STATE, 0o700)
|
||||
with tempfile.TemporaryDirectory(dir=STATE) as td:
|
||||
authfile = os.path.join(td, "auth.json")
|
||||
fd = os.open(authfile, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
|
||||
with os.fdopen(fd, "w") as f:
|
||||
json.dump({"auths": {REGISTRY: {"auth": base64.b64encode(f"{user}:{token}".encode()).decode()}}}, f)
|
||||
start = time.time()
|
||||
copied = copy_images(authfile, stage, repos, keep)
|
||||
copy_agents(Registry(user, token), stage, akeep)
|
||||
for n in os.listdir(stage):
|
||||
if n not in ("images", "agent", "SHA256SUMS", "COPIED_AT"):
|
||||
p = os.path.join(stage, n)
|
||||
shutil.rmtree(p) if os.path.isdir(p) else os.remove(p)
|
||||
with open(os.path.join(stage, "COPIED_AT"), "w") as f:
|
||||
f.write(time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()) + "\n")
|
||||
write_sums(stage)
|
||||
nbytes = tree_bytes(stage)
|
||||
if nbytes > cap * 1024 * 1024:
|
||||
raise Refused(f"the copy is {nbytes // (1024 * 1024)} MB, above the cap {cap} MB — nothing sent")
|
||||
log(f"staged and verified: {nbytes // (1024 * 1024)} MB, new this run: {', '.join(copied) or 'none'}")
|
||||
rsync_to(env, stage)
|
||||
diff = rsync_to(env, stage, dry_check=True).strip()
|
||||
if diff:
|
||||
raise Refused("ep0's copy differs from the staged one after the push: " + diff.splitlines()[0])
|
||||
log(f"pushed to ep0 and compared by checksum: no difference ({int(time.time() - start)} s)")
|
||||
signal(nbytes)
|
||||
log("success signal written")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
try:
|
||||
main()
|
||||
except Refused as e:
|
||||
print(f"felhom-release-copy: FAILED: {e}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
except subprocess.CalledProcessError as e:
|
||||
print(f"felhom-release-copy: FAILED: {e.cmd[0]} {e.cmd[1] if len(e.cmd) > 1 else ''} rc={e.returncode}: "
|
||||
f"{(e.stderr or '').strip()[:300]}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
@@ -0,0 +1,19 @@
|
||||
# Versioned in felhom.eu/scripts/release-copy/ (D33, R-1227); installed by install.sh. Runbook: total-loss-of-dooplex.md.
|
||||
[Unit]
|
||||
Description=Felhom: copy the released images and agent packages to ep0 (D33)
|
||||
Wants=network-online.target
|
||||
After=network-online.target
|
||||
OnFailure=felhom-backup-failmail@%n.service
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
ExecStart=/usr/local/sbin/felhom-release-copy
|
||||
UMask=0077
|
||||
TimeoutStartSec=2h
|
||||
Nice=10
|
||||
IOSchedulingClass=idle
|
||||
PrivateTmp=yes
|
||||
ProtectSystem=strict
|
||||
ProtectHome=yes
|
||||
ReadWritePaths=/var/lib/felhom-release-copy /var/lib/node_exporter/textfile_collector
|
||||
NoNewPrivileges=yes
|
||||
@@ -0,0 +1,12 @@
|
||||
# Versioned in felhom.eu/scripts/release-copy/ (D33, R-1227); installed by install.sh.
|
||||
[Unit]
|
||||
Description=Felhom: copy the released images and agent packages to ep0 (D33) — nightly
|
||||
|
||||
[Timer]
|
||||
# 01:40: after the 00:20 off-site push, before the 02:30 hub DB push and the 03:00 local backup.
|
||||
OnCalendar=*-*-* 01:40:00
|
||||
Persistent=true
|
||||
RandomizedDelaySec=5min
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
@@ -0,0 +1,30 @@
|
||||
#!/bin/sh
|
||||
# install.sh — install the release copy job on DooPlex (D33, R-1227). Root. Idempotent.
|
||||
# Installs the script and two units, writes /etc/felhom-release-copy/env (no secrets) when absent, and does NOT enable
|
||||
# the timer — enable it by hand after the first manual run: systemctl enable --now felhom-release-copy.timer
|
||||
# The registry token (registry-token), the SSH key (id_ed25519) and ep0's host key (known_hosts) are created
|
||||
# separately, file to file, never by this script (runbook total-loss-of-dooplex.md, „The release copy").
|
||||
set -eu
|
||||
HERE=$(cd "$(dirname "$0")" && pwd)
|
||||
[ "$(id -u)" = 0 ] || { echo "install.sh: run as root" >&2; exit 1; }
|
||||
install -m 0755 "$HERE/felhom-release-copy" /usr/local/sbin/felhom-release-copy
|
||||
for u in felhom-release-copy.service felhom-release-copy.timer; do
|
||||
install -m 0644 "$HERE/$u" "/etc/systemd/system/$u"
|
||||
done
|
||||
install -d -m 0700 /etc/felhom-release-copy /var/lib/felhom-release-copy
|
||||
if [ ! -f /etc/felhom-release-copy/env ]; then
|
||||
umask 077
|
||||
cat > /etc/felhom-release-copy/env <<'ENV'
|
||||
# Not secret. The token is in registry-token (0600), the SSH key in id_ed25519 (0600).
|
||||
REMOTE='felhom-relcopy@167.233.158.164'
|
||||
REGISTRY_USER='admin'
|
||||
# the newest KEEP release tags of each image; the newest AGENT_KEEP agent versions (the vouched one can lag the newest)
|
||||
REPOS='felhom-controller felhom-hub felhom-samba'
|
||||
KEEP=3
|
||||
AGENT_KEEP=5
|
||||
# the cap: the sender refuses to send a bigger copy (ep0's / has ~14 GB free; ep0 has no quota)
|
||||
CAP_MB=3072
|
||||
ENV
|
||||
fi
|
||||
systemctl daemon-reload
|
||||
echo "install.sh: installed; timer NOT enabled (see the header)"
|
||||
@@ -0,0 +1,165 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Tests for felhom-release-copy (D33). No network, no registry, no ep0: a synthetic skopeo `dir:` copy, and the
|
||||
end-to-end refusals through main() with the copy steps replaced. Run: python3 test_release_copy.py"""
|
||||
import hashlib
|
||||
import importlib.machinery
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest import mock
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
_loader = importlib.machinery.SourceFileLoader("rc", os.path.join(HERE, "felhom-release-copy"))
|
||||
_spec = importlib.util.spec_from_loader("rc", _loader)
|
||||
rc = importlib.util.module_from_spec(_spec)
|
||||
_loader.exec_module(rc)
|
||||
|
||||
|
||||
def h(b):
|
||||
return hashlib.sha256(b).hexdigest()
|
||||
|
||||
|
||||
def make_image(d):
|
||||
"""A two-level image like gitea's: an OCI index → one platform manifest → config + one layer. Returns the digest."""
|
||||
os.makedirs(d)
|
||||
layer, config = b"layer-bytes", b'{"config":1}'
|
||||
child = json.dumps({"schemaVersion": 2, "config": {"digest": "sha256:" + h(config)},
|
||||
"layers": [{"digest": "sha256:" + h(layer)}]}).encode()
|
||||
index = json.dumps({"schemaVersion": 2, "manifests": [{"digest": "sha256:" + h(child)}]}).encode()
|
||||
for name, data in ((h(layer), layer), (h(config), config), (h(child) + ".manifest.json", child),
|
||||
("manifest.json", index), ("version", b"Directory Transport Version: 1.1\n")):
|
||||
open(os.path.join(d, name), "wb").write(data)
|
||||
return "sha256:" + h(index)
|
||||
|
||||
|
||||
class Verify(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.t = tempfile.mkdtemp()
|
||||
self.d = os.path.join(self.t, "img")
|
||||
self.digest = make_image(self.d)
|
||||
|
||||
def test_a_true_copy_passes(self):
|
||||
rc.verify_image_dir(self.d, self.digest)
|
||||
|
||||
def test_a_changed_layer_is_refused(self):
|
||||
# RED-PROOF: drop the per-blob hash check in verify_image_dir → this passes silently
|
||||
blob = [n for n in os.listdir(self.d) if rc.HEX64.match(n)][0]
|
||||
with open(os.path.join(self.d, blob), "ab") as f:
|
||||
f.write(b"x")
|
||||
with self.assertRaisesRegex(rc.Refused, "does not hash to its name"):
|
||||
rc.verify_image_dir(self.d, self.digest)
|
||||
|
||||
def test_another_image_is_refused(self):
|
||||
with self.assertRaisesRegex(rc.Refused, "!= registry"):
|
||||
rc.verify_image_dir(self.d, "sha256:" + "0" * 64)
|
||||
|
||||
def test_a_missing_layer_is_refused(self):
|
||||
# RED-PROOF: drop the „named by a manifest is missing" check → a copy without its layer passes
|
||||
layer = [n for n in os.listdir(self.d) if rc.HEX64.match(n)
|
||||
and open(os.path.join(self.d, n), "rb").read() == b"layer-bytes"][0]
|
||||
os.remove(os.path.join(self.d, layer))
|
||||
with self.assertRaisesRegex(rc.Refused, "missing"):
|
||||
rc.verify_image_dir(self.d, self.digest)
|
||||
|
||||
def test_a_missing_child_manifest_is_refused(self):
|
||||
child = [n for n in os.listdir(self.d) if n.endswith(".manifest.json")][0]
|
||||
os.remove(os.path.join(self.d, child))
|
||||
with self.assertRaisesRegex(rc.Refused, "child manifest .* missing"):
|
||||
rc.verify_image_dir(self.d, self.digest)
|
||||
|
||||
|
||||
class Select(unittest.TestCase):
|
||||
def test_newest_releases_by_number_not_text(self):
|
||||
tags = ["0.9.0", "0.10.0", "0.340.0", "0.339.0", "0.340.0-rc.8", "latest", "0.99.1"]
|
||||
self.assertEqual(rc.newest(tags, 3), ["0.99.1", "0.339.0", "0.340.0"])
|
||||
|
||||
def test_spoiled_tags_are_never_copied(self):
|
||||
self.assertEqual(rc.newest(["0.314.0", "0.315.0", "0.316.3"], 3, rc.SPOILED["felhom-controller"]), ["0.314.0"])
|
||||
|
||||
def test_prune_deletes_beyond_keep(self):
|
||||
t = tempfile.mkdtemp()
|
||||
for v in ("0.1.0", "0.2.0", "0.3.0"):
|
||||
os.makedirs(os.path.join(t, v))
|
||||
self.assertEqual(rc.prune(t, {"0.3.0"}), ["0.1.0", "0.2.0"])
|
||||
self.assertEqual(os.listdir(t), ["0.3.0"])
|
||||
|
||||
|
||||
class Main(unittest.TestCase):
|
||||
"""main() with the registry copies replaced: the cap and the post-push comparison gate the success signal."""
|
||||
|
||||
def setUp(self):
|
||||
self.t = tempfile.mkdtemp()
|
||||
self.conf, self.state, self.tf = (os.path.join(self.t, n) for n in ("conf", "state", "textfile"))
|
||||
for d in (self.conf, self.state, self.tf):
|
||||
os.makedirs(d)
|
||||
open(os.path.join(self.conf, "env"), "w").write(
|
||||
"REMOTE=felhom-relcopy@ep0\nKEEP=3\nAGENT_KEEP=5\nCAP_MB=1\nREPOS='felhom-hub'\n")
|
||||
open(os.path.join(self.conf, "registry-token"), "w").write("not-a-token\n")
|
||||
self.patches = [mock.patch.object(rc, "CONF", self.conf), mock.patch.object(rc, "STATE", self.state),
|
||||
mock.patch.object(rc, "TEXTFILE_DIR", self.tf)]
|
||||
for p in self.patches:
|
||||
p.start()
|
||||
self.addCleanup(lambda: [p.stop() for p in self.patches])
|
||||
self.pushed = []
|
||||
|
||||
def fake_images(self, size):
|
||||
def f(authfile, stage, repos, keep):
|
||||
d = os.path.join(stage, "images", "felhom-hub")
|
||||
os.makedirs(d, exist_ok=True)
|
||||
open(os.path.join(d, "blob"), "wb").write(b"x" * size)
|
||||
return ["felhom-hub:0.167.0"]
|
||||
return f
|
||||
|
||||
def fake_rsync(self, diff=""):
|
||||
def f(env, stage, dry_check=False):
|
||||
self.pushed.append(dry_check)
|
||||
return diff if dry_check else ""
|
||||
return f
|
||||
|
||||
def go(self, size, diff=""):
|
||||
with mock.patch.object(rc, "copy_images", self.fake_images(size)), \
|
||||
mock.patch.object(rc, "copy_agents", lambda reg, stage, keep: None), \
|
||||
mock.patch.object(rc, "rsync_to", self.fake_rsync(diff)):
|
||||
rc.main()
|
||||
|
||||
def signal(self):
|
||||
return os.path.exists(os.path.join(self.tf, "felhom_release_copy.prom"))
|
||||
|
||||
def test_success_pushes_compares_then_signals(self):
|
||||
self.go(1000)
|
||||
self.assertEqual(self.pushed, [False, True])
|
||||
self.assertTrue(self.signal())
|
||||
sums = open(os.path.join(self.state, "stage", "SHA256SUMS")).read()
|
||||
self.assertIn("images/felhom-hub/blob", sums)
|
||||
|
||||
def test_above_the_cap_nothing_is_sent(self):
|
||||
# RED-PROOF: drop the cap check in main() → the push happens and the signal is written
|
||||
with self.assertRaisesRegex(rc.Refused, "above the cap"):
|
||||
self.go(2 * 1024 * 1024)
|
||||
self.assertEqual(self.pushed, [])
|
||||
self.assertFalse(self.signal())
|
||||
|
||||
def test_a_difference_after_the_push_writes_no_signal(self):
|
||||
# RED-PROOF: ignore the dry-run's output → a copy ep0 does not hold reads as success
|
||||
with self.assertRaisesRegex(rc.Refused, "differs"):
|
||||
self.go(1000, diff=">fcsT...... images/felhom-hub/blob\n")
|
||||
self.assertFalse(self.signal())
|
||||
|
||||
|
||||
class Units(unittest.TestCase):
|
||||
def read(self, n):
|
||||
return open(os.path.join(HERE, n)).read()
|
||||
|
||||
def test_unit_runs_the_installed_script_and_mails_on_failure(self):
|
||||
s = self.read("felhom-release-copy.service")
|
||||
self.assertIn("ExecStart=/usr/local/sbin/felhom-release-copy\n", s)
|
||||
self.assertIn("OnFailure=felhom-backup-failmail@%n.service", s)
|
||||
|
||||
def test_nightly(self):
|
||||
self.assertIn("OnCalendar=*-*-* 01:40:00", self.read("felhom-release-copy.timer"))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main(verbosity=2)
|
||||
@@ -1,4 +1,24 @@
|
||||
## 2026-10-22 — the download pages point at installer 1.30.0 (W1, W7; operator yes)
|
||||
## 2026-10-23 — the privacy wording says what is true; the minimum machine is written down (D3 / R-985, D5 / R-991; operator rulings 2026-10-22 15:47) and D36's two sentences (R-1225, R-1226) — published on the operator's yes (day brief)
|
||||
|
||||
- **D36 (operator yes 2026-10-23, R-1225, R-1226).** `letoltes` / `en/download`: „A 8 GB az alapokat futtatja; a fotóalkalmazáshoz 16 GB kell.” / „8 GB runs the basics; the photo app needs 16 GB.” `gyik` / `en/faq` („Ti hozzáfértek az adataimhoz?”, the visible answer and its JSON-LD twin): „…csak titkosítva hagyják el az otthonodat (a távoli mentés)” became „Amit az otthonodon kívülről nyitsz meg, az a Cloudflare hálózatán halad át, és ott olvasható; a mentések csak titkosítva hagyják el az otthonodat.” / „What you open from outside your home passes through Cloudflare's network and is readable there; the backups leave your home only encrypted.” (`13-threat-model.md` §6, R-904).
|
||||
|
||||
- **D3 (R-985):** every promise that „nobody, not even us" can read the remote backup now says the whole truth.
|
||||
The box encrypts the backup with a key made on the box; the recovery code is made on the box, shown once, and we keep
|
||||
no copy; we hold only the locked key package and cannot open it; from the remote storage and the hub nobody can read
|
||||
the backup. **New, stated plainly:** while the box runs, the operator has root access for support, and with it could
|
||||
reach the data and the backup key on the box. Pages: `biztonsagimentes` / `en/backups`, `gyik` / `en/faq` („Ti
|
||||
hozzáfértek az adataimhoz?", JSON-LD too), `szolgaltatasok` / `en/services`, `sajat-felho-tarhely` /
|
||||
`en/own-cloud-storage`, `adatkezeles` (§6, §7, §8) and `feltetelek` (§6, §7: „a távoli mentésed nem nyitható ki" now
|
||||
only when the box is lost AND the code is lost). Sources: `documentation/architecture/03-host-agent.md` §8a,
|
||||
`07-backup-architecture.md` §2, `13-threat-model.md` §5. The volunteer guide (both languages) got the same wording.
|
||||
- **D5 (R-991, R-993 merged):** the minimum machine — x86-64, 4 cores, 8 GB memory, 256 GB SSD; recommended 16 GB and
|
||||
512 GB; not supported: Raspberry Pi / ARM, Synology / QNAP, routers, rented cloud servers. New block „Milyen gépre
|
||||
telepíthető?" / „Which machines can it install on?" in the „Mielőtt letöltöd" card of `letoltes` / `en/download`
|
||||
(installer version, links and checksum untouched); one paragraph in the FAQ „Milyen gépre van szükségem?" (JSON-LD
|
||||
too); the estimator page's advice list now names the memory floor (`gepigeny` / `en/sizing`). `sizing-data.json`
|
||||
unchanged. Nothing enforces the floor at install time (R-993's measurement) — that stays the operator's call.
|
||||
|
||||
## 2026-10-22 — the download pages point at installer 1.30.0 (W1, W7; operator yes)
|
||||
|
||||
- `letoltes.html` and `en/download.html`: version, both links, the side-file link, the two check commands and the SHA-256
|
||||
now name `felhom-installer-1.30.0-pve9.2-1.iso` (`590d7b9b…74d5cc1`, the same 1 705 324 544 bytes). Published and
|
||||
|
||||
@@ -250,8 +250,8 @@
|
||||
</ul>
|
||||
<p>
|
||||
<strong>A kulcsletétről, egyszerűen:</strong> a mentésed kulcsát csak lezárt formában tároljuk.
|
||||
Kinyitni csak a <strong>helyreállító kódoddal</strong> lehet, ami nálunk nincs meg. Ha elveszíted
|
||||
ezt a kódot, a távoli mentésed nem nyitható ki — mi sem tudjuk kinyitni.
|
||||
Kinyitni csak a <strong>helyreállító kódoddal</strong> lehet, ami nálunk nincs meg. Ha a szervered
|
||||
tönkremegy, és ezt a kódot is elveszíted, a távoli mentésed nem nyitható ki — mi sem tudjuk kinyitni.
|
||||
</p>
|
||||
<!-- source: documentation/architecture/07-backup-architecture.md:67-75 ("genuinely zero-knowledge… R exists in zero system copies by design") -->
|
||||
<p>
|
||||
@@ -265,8 +265,9 @@
|
||||
<h2>7. A szervered adatainak távoli mentése</h2>
|
||||
<p>
|
||||
A szervereden lévő adatokról távoli mentés készül. <strong>A mentés nálad, a szerveren
|
||||
titkosítódik, mielőtt elhagyja az otthonodat.</strong> Sem a tárhely üzemeltetője, sem mi nem
|
||||
tudjuk elolvasni a tartalmát.
|
||||
titkosítódik, mielőtt elhagyja az otthonodat.</strong> A tárhelyen lévő tartalmát sem a tárhely
|
||||
üzemeltetője, sem mi nem tudjuk elolvasni. A működő szervereden lévő kulcsot a rendszergazdai
|
||||
hozzáférésünkkel elérhetnénk (lásd a 8. pontot).
|
||||
</p>
|
||||
<!-- source: documentation/architecture/01-topology-and-trust.md:118 ("ciphertext only (operator can't read)"); 07-backup-architecture.md:378-379 -->
|
||||
<ul>
|
||||
@@ -288,7 +289,7 @@
|
||||
<p>
|
||||
Őszintén: <strong>rendszergazdai (root) hozzáférésünk van minden szerverhez, amit kezelünk.</strong>
|
||||
Ez kell a frissítésekhez, a mentések ellenőrzéséhez és a hibák javításához. Ezzel a hozzáféréssel
|
||||
a szervereden tárolt adatok elérhetők. Csak üzemeltetésre és hibaelhárításra használjuk.
|
||||
a szervereden tárolt adatok és a távoli mentés kulcsa is elérhetők. Csak üzemeltetésre és hibaelhárításra használjuk.
|
||||
</p>
|
||||
<!-- source: documentation/architecture/07-backup-architecture.md:49-52 ("The operator holds root SSH on every box… 'The operator cannot read customer data' was never the security property") -->
|
||||
<p>
|
||||
|
||||
@@ -269,8 +269,9 @@
|
||||
<p>Egy lemezen minden helyi másolat <strong>ugyanazon a lemezen</strong> van. Ez véd a véletlen törlés és a hibás fájlok ellen, de egy lemezhiba ellen nem. Lemezhiba ellen <strong>egy másik lemez</strong> vagy a <strong>távoli mentés</strong> véd. A Felhom ezt meg is mutatja: csak akkor számol második másolatot, ha biztosan tudja, hogy az egy másik lemezen van, és ha ezt nem tudja ellenőrizni (például egy virtuális gépben), azt ki is írja.</p>
|
||||
|
||||
<h3 style="margin-top: 24px;"><svg class="ico-lg" aria-hidden="true"><use href="/assets/icons.svg?v=1#i-lock"/></svg> A helyreállítási kód – amit csak te ismersz</h3>
|
||||
<p>A távoli mentésed titkosított, és a titkosítás kulcsa egy <strong>helyreállítási kód</strong>. Ez a kód pontosan egyszer jelenik meg: a te képernyődön, a beállításkor. Nálunk nem marad belőle másolat.</p>
|
||||
<p>Ez tudatos döntés, és két oldala van. Az egyik, hogy a távoli mentésedet rajtad kívül <strong>senki</strong> nem tudja elolvasni – mi sem. A másik, hogy ha a doboz megsemmisül és a kód is elveszett, akkor a távoli mentés <strong>nekünk sem visszaállítható</strong>. Ezért a kód papírra kerül, és jó helyre.</p>
|
||||
<p>A távoli mentésedet a doboz titkosítja, mielőtt az elhagyja az otthonodat. A kulcs a dobozon készül, és ott is marad. A kulcsról a doboz egy lezárt másolatot is készít, amit a <strong>helyreállítási kód</strong> nyit. Ezt a lezárt másolatot mi őrizzük, de kinyitni nem tudjuk. A kód is a dobozon készül, és pontosan egyszer jelenik meg: a te képernyődön, a beállításkor. Nálunk nem marad belőle másolat.</p>
|
||||
<p>Ez tudatos döntés, és két oldala van. Az egyik: a távoli tárhelyen csak a titkosított mentés van, nálunk pedig csak a lezárt kulcs. Ezekből <strong>senki</strong> nem tudja elolvasni a mentésedet – mi sem. A másik: ha a doboz megsemmisül és a kód is elveszett, akkor a távoli mentés <strong>nekünk sem visszaállítható</strong>. Ezért a kód papírra kerül, és jó helyre.</p>
|
||||
<p>Egy dolgot őszintén elmondunk. Amíg a dobozod működik, támogatás céljából <strong>rendszergazdai (root) hozzáférésünk</strong> van hozzá. Ezzel a dobozon lévő adataidat és a mentés kulcsát is elérhetnénk. Ezt a hozzáférést csak üzemeltetésre és hibaelhárításra használjuk. A helyreállítási kód tehát a távoli másolatot védi, nem a működő dobozt.</p>
|
||||
|
||||
<h3 style="margin-top: 24px;"><svg class="ico-lg" aria-hidden="true"><use href="/assets/icons.svg?v=1#i-chart-column"/></svg> Monitoring és értesítések</h3>
|
||||
<p>A mentéseket folyamatosan figyeljük. Ha probléma van – sikertelen mentés, betelő tárhely, kapcsolati hiba – az a felügyeleti rendszerünkben megjelenik, és e-mail értesítés is indul róla. A zárt teszt egyik célja épp az, hogy ezeket az értesítéseket éles körülmények között csiszoljuk.</p>
|
||||
|
||||
@@ -269,8 +269,9 @@
|
||||
<p>On one disk, every local copy is <strong>on the same disk</strong>. That protects against deleting something by mistake and against broken files, but not against a disk failure. A <strong>different disk</strong> or the <strong>remote backup</strong> protects against a disk failure. Felhom shows this too: it counts a second copy only when it knows for sure that the copy is on another disk, and when it cannot check that (for example inside a virtual machine), it says so.</p>
|
||||
|
||||
<h3 style="margin-top: 24px;"><svg class="ico-lg" aria-hidden="true"><use href="/assets/icons.svg?v=1#i-lock"/></svg> The recovery code – only you know it</h3>
|
||||
<p>Your remote backup is encrypted, and the key to the encryption is a <strong>recovery code</strong>. The code appears exactly once: on your screen, when it is set up. We keep no copy of it.</p>
|
||||
<p>This is a deliberate choice, and it has two sides. One: <strong>nobody</strong> but you can read your remote backup – not even us. The other: if the box is destroyed and the code is lost too, then <strong>we cannot restore</strong> the remote backup either. So the code goes on paper, and somewhere safe.</p>
|
||||
<p>The box encrypts your remote backup before it leaves your home. The key is made on the box, and it stays there. The box also makes a locked copy of the key, which the <strong>recovery code</strong> opens. We keep that locked copy, but we cannot open it. The code is made on the box too, and it appears exactly once: on your screen, when it is set up. We keep no copy of it.</p>
|
||||
<p>This is a deliberate choice, and it has two sides. One: the remote storage holds only the encrypted backup, and we hold only the locked key. From these, <strong>nobody</strong> can read your backup – not even us. The other: if the box is destroyed and the code is lost too, then <strong>we cannot restore</strong> the remote backup either. So the code goes on paper, and somewhere safe.</p>
|
||||
<p>One thing we tell you honestly. While your box runs, we have <strong>administrator (root) access</strong> to it for support. With it, we could reach the data on the box and the backup key too. We use this access only to run the service and to fix faults. So the recovery code protects the remote copy, not the running box.</p>
|
||||
|
||||
<h3 style="margin-top: 24px;"><svg class="ico-lg" aria-hidden="true"><use href="/assets/icons.svg?v=1#i-chart-column"/></svg> Monitoring and notifications</h3>
|
||||
<p>We watch the backups all the time. If there is a problem – a failed backup, storage filling up, a connection fault – it shows up in our monitoring, and an e-mail goes out about it. One aim of the closed test is to polish these notifications in real use.</p>
|
||||
|
||||
@@ -92,6 +92,11 @@
|
||||
<p>Installing <strong>erases every piece of data</strong> on the disk you select. The installer never chooses for you: it lists the disks with their size and type, and overwrites the one you pick.</p>
|
||||
<p>Unplug the external drive you use for backups before you start. If you are not certain which disk is the right one, <strong>stop and call your operator</strong>.</p>
|
||||
<p>The guide from your operator describes every step, screen by screen. Do not start without it.</p>
|
||||
<h3 style="margin-top: 24px;"><svg class="ico-lg" aria-hidden="true"><use href="/assets/icons.svg?v=1#i-cpu"/></svg> Which machines can it install on?</h3>
|
||||
<p><strong>At least:</strong> an x86-64 processor (Intel or AMD) with 4 cores, N100 class or newer; 8 GB of memory; a 256 GB SSD. 8 GB runs the basics; the photo app needs 16 GB.</p>
|
||||
<p><strong>Recommended:</strong> 16 GB of memory and a 512 GB SSD. Photos and films go on a separate data drive.</p>
|
||||
<p><strong>Not supported:</strong> Raspberry Pi and other ARM boards, Synology / QNAP, routers, rented cloud servers.</p>
|
||||
<p>To estimate how much memory your own apps need, use the <a href="/en/sizing">Which machine do I need?</a> page.</p>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||