Compare commits
32
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
e1d349f100 | ||
|
|
7d7868e5e5 | ||
|
|
189eb82eb0 | ||
|
|
f92fa5a8c8 | ||
|
|
5d0dfeac5f | ||
|
|
15f7428871 | ||
|
|
d92a712ee4 | ||
|
|
5f31da04bc | ||
|
|
a28f162c66 | ||
|
|
97f27a7528 | ||
|
|
b6a911c3f9 | ||
|
|
224fe41e06 | ||
|
|
73f3899266 | ||
|
|
e2c35007b5 | ||
|
|
fd75003e17 | ||
|
|
5d3cb5c132 | ||
|
|
a17db0c3c8 | ||
|
|
503c93fada | ||
|
|
d5e3c8b0d9 | ||
|
|
db2371c6f5 | ||
|
|
200ab6530b | ||
|
|
5622ce2735 | ||
|
|
afc939bb3c | ||
|
|
b505ce4ba0 | ||
|
|
5121536f10 | ||
|
|
e4f022a0d9 | ||
|
|
b9b56c47c0 | ||
|
|
1fe9f84893 | ||
|
|
7f7081d127 | ||
|
|
e881115c43 | ||
|
|
37cb83c166 | ||
|
|
3332eb0919 |
No files matched your search
@@ -29,5 +29,7 @@ ArgoCD's source of truth is the **manifest in git**:
|
|||||||
`main`, and `/scripts/` from the tag `installer-v<SCRIPT_VERSION>`. Changing either `--ref` changes
|
`main`, and `/scripts/` from the tag `installer-v<SCRIPT_VERSION>`. Changing either `--ref` changes
|
||||||
what real machines download — the init container **and** the sidecar both carry it, and both must
|
what real machines download — the init container **and** the sidecar both carry it, and both must
|
||||||
move together. `hostinstall_gates.py` gate 6 fails if the manifest stops naming an `installer-v…`
|
move together. `hostinstall_gates.py` gate 6 fails if the manifest stops naming an `installer-v…`
|
||||||
tag or if the website stops tracking `main`. The full publish/rollback procedure is the R-110 fence
|
tag or if the website stops tracking `main`. All four git-sync containers (two sidecars, two init containers) read
|
||||||
|
Gitea **in-cluster** (`http://gitea.gitea-system.svc.cluster.local:3000/…`, since 2026-10-19) and must name the same
|
||||||
|
repo — gate 6 fails if they differ; the sidecars carry `--max-failures=-1` (retry, never exit). The full publish/rollback procedure is the R-110 fence
|
||||||
in the repo core `CLAUDE.md`.
|
in the repo core `CLAUDE.md`.
|
||||||
@@ -184,6 +184,10 @@ jobs:
|
|||||||
# which prints at the moment a release is committed, where someone can actually act on it.
|
# which prints at the moment a release is committed, where someone can actually act on it.
|
||||||
# Those are the compensating controls that make this green honest. Every other gate still
|
# Those are the compensating controls that make this green honest. Every other gate still
|
||||||
# fails this job on any push, and golden-currency still fails it on a push touching code.
|
# fails this job on any push, and golden-currency still fails it on a push touching code.
|
||||||
|
env:
|
||||||
|
# R-986: the security-scan gate fetches the PINNED gitleaks here (sha256-checked); no Go, so govulncheck says
|
||||||
|
# NOT CHECKED in CI, out loud. Warn-first either way: the scan lists, it never fails this job.
|
||||||
|
FELHOM_SCAN_FETCH: "1"
|
||||||
run: python3 scripts/repo_gates.py --fast --scope="${PUSH_SCOPE:-code}"
|
run: python3 scripts/repo_gates.py --fast --scope="${PUSH_SCOPE:-code}"
|
||||||
|
|
||||||
- name: Manual proof - alarm key present, optional forced failure (dispatch only, R-1129)
|
- name: Manual proof - alarm key present, optional forced failure (dispatch only, R-1129)
|
||||||
@@ -228,8 +232,16 @@ jobs:
|
|||||||
|
|
||||||
repo = os.environ.get("GITHUB_REPOSITORY", "?")
|
repo = os.environ.get("GITHUB_REPOSITORY", "?")
|
||||||
sha = os.environ.get("GITHUB_SHA", "?")
|
sha = os.environ.get("GITHUB_SHA", "?")
|
||||||
run = os.environ.get("GITHUB_RUN_NUMBER", "?")
|
# R-1135: the link is for a PERSON, so it is the public address and the run ID.
|
||||||
srv = os.environ.get("GITHUB_SERVER_URL", "https://gitea.dooplex.hu")
|
# NOT GITHUB_SERVER_URL: the runner registers at the in-cluster address, so that variable
|
||||||
|
# reads http://gitea.gitea-system.svc.cluster.local:3000, which nobody outside the cluster
|
||||||
|
# can open (forced-test mail, job 1997). NOT GITHUB_RUN_NUMBER: Gitea 1.26 reads
|
||||||
|
# /actions/runs/<n> as a run id first, so a run number that is also an older run's id
|
||||||
|
# opens THAT run (measured 2026-10-18: felhom.eu /actions/runs/1116 opened the run whose
|
||||||
|
# id is 1116 = run number 613). The link is printed too, so the job log carries it.
|
||||||
|
run = os.environ.get("GITHUB_RUN_ID", "?")
|
||||||
|
srv = "https://gitea.dooplex.hu"
|
||||||
|
print("ALARM-LINK %s/%s/actions/runs/%s" % (srv, repo, run))
|
||||||
|
|
||||||
# R-265: how long the run took, so a reap is self-identifying. An honest gate failure
|
# R-265: how long the run took, so a reap is self-identifying. An honest gate failure
|
||||||
# lands in well under a minute; a multi-minute figure means the job hit its own timeout
|
# lands in well under a minute; a multi-minute figure means the job hit its own timeout
|
||||||
|
|||||||
@@ -0,0 +1,37 @@
|
|||||||
|
# .security-accepted — hits security_scan_gate.py (felhom.eu/scripts, R-986) lists as ACCEPTED, not WARN.
|
||||||
|
# One per line: <tool> <id> <where> # reason. govulncheck: OSV id + module dir; gitleaks: rule@<sha256[:12] of the line> + file.
|
||||||
|
# The first scan's verdicts (2026-10-19): felhom.eu documentation/audits/night-2026-10-19/P7/triage.md
|
||||||
|
gitleaks generic-api-key@0275c9c51957 documentation/audits/usenet-2026-10-12/sabnzbd-bench/13-compose-resolved.txt # accepted: a bench app's generated admin password and API key (LXC 9401, stopped; app removed from 9202)
|
||||||
|
gitleaks generic-api-key@0a189ae60ea4 documentation/tests/campaign2-evidence-2026-07-07/state/runner.log # accepted: the same dry-run deploy values in the campaign runner's log
|
||||||
|
gitleaks generic-api-key@2a27df76f661 documentation/audits/usenet-2026-10-12/sabnzbd-bench/02-telemetry-and-net.txt # accepted: a bench app's generated API key (LXC 9401, stopped; app removed from 9202 at teardown)
|
||||||
|
gitleaks generic-api-key@3a4c7b17e349 documentation/audits/usenet-2026-10-12/nzbget-bench/03-process-and-entrypoint.txt # accepted: a bench app's generated control password (LXC 9401, stopped; app removed from 9202 at teardown)
|
||||||
|
gitleaks generic-api-key@3c180a9f957c documentation/tests/campaign2-evidence-2026-07-07/state/runner.log # accepted: the same dry-run deploy values in the campaign runner's log
|
||||||
|
gitleaks generic-api-key@486819f82f58 documentation/audits/update-night-2026-09-21/apps/actualbudget/log.txt # accepted: a demo app's session token, demo-hp 2026-09-21 (Tier 0); no running demo app has it (2026-10-12 review)
|
||||||
|
gitleaks generic-api-key@486819f82f58 documentation/audits/update-night-2026-09-21/batchA.log # accepted: the same demo app session token as actualbudget/log.txt
|
||||||
|
gitleaks generic-api-key@5a64c6ffd6df documentation/audits/signup-lock-2026-09-29/B/wd.sh # accepted: a stranger-probe test password in an audit script (signup lock, Tier 0)
|
||||||
|
gitleaks generic-api-key@6e2e899b7d6e documentation/audits/offsite-lock-build-2026-10-03/partD/demo-hp-migration.txt # false: an SSH key FINGERPRINT in a log line, not a key
|
||||||
|
gitleaks generic-api-key@70380348263d hub/internal/web/bind_language_test.go # false: a test constant or a decoy value, no live secret
|
||||||
|
gitleaks generic-api-key@7134caf03160 documentation/audits/evidence-rehearsal-2026-08-09/GATE0-demo-hp-before.txt # false: a WireGuard PUBLIC key
|
||||||
|
gitleaks generic-api-key@7134caf03160 documentation/audits/evidence-rehearsal-2026-08-09/P1-demo-hp-after.txt # false: a WireGuard PUBLIC key
|
||||||
|
gitleaks generic-api-key@71ba98258279 hub/internal/api/r1140_accesslog_test.go # false: a test constant or a decoy value, no live secret
|
||||||
|
gitleaks generic-api-key@7aa746ddb210 documentation/audits/offsite-lock-build-2026-10-03/partD/demo-hp-migration.txt # false: an SSH key FINGERPRINT in a log line, not a key
|
||||||
|
gitleaks generic-api-key@7ce547845b1e documentation/audits/night-rulings-2026-09-30/D/D3-wanderer-routes-probe.txt # accepted: a throwaway app's session cookie (wanderer probe, 2026-09-30, Tier 0)
|
||||||
|
gitleaks generic-api-key@8044eef94360 documentation/tests/campaign2-evidence-2026-07-07/state/runner.log # accepted: the same dry-run deploy values in the campaign runner's log
|
||||||
|
gitleaks generic-api-key@85b662612937 hub/internal/store/r879_box_seal_test.go # false: a test constant or a decoy value, no live secret
|
||||||
|
gitleaks generic-api-key@9530a022fde8 documentation/audits/usenet-2026-10-12/sabnzbd-bench/13-compose-resolved.txt # accepted: a bench app's generated admin password and API key (LXC 9401, stopped; app removed from 9202)
|
||||||
|
gitleaks generic-api-key@9b3a9c2be48a documentation/tests/campaign2-evidence-2026-07-07/evidence/T-DEPLOY-SET/values_bookstack.json # accepted: dry-run deploy values of a 2026-07-07 campaign (generated, demo boxes since rebuilt)
|
||||||
|
gitleaks generic-api-key@a9538cfd7063 documentation/tests/campaign2-evidence-2026-07-07/evidence/T-DEPLOY-SET/values_paperless-ngx.json # accepted: dry-run deploy values of a 2026-07-07 campaign (generated, demo boxes since rebuilt)
|
||||||
|
gitleaks generic-api-key@b0414c6f59b9 documentation/audits/offsite-lock-build-2026-10-03/partD/demo-hp-migration.txt # false: an SSH key FINGERPRINT in a log line, not a key
|
||||||
|
gitleaks generic-api-key@b1db45e463b6 documentation/audits/family-gate-2026-10-02/tools/fg_items.py # false: test strings in an audit tool (websocket probe labels)
|
||||||
|
gitleaks generic-api-key@b1db45e463b6 documentation/audits/permanent-gate-2026-10-01/tools/items12.py # false: test strings in an audit tool (websocket probe labels)
|
||||||
|
gitleaks generic-api-key@bd228c851720 documentation/tests/walk5-r201-2026-08-07/journal.md # false: sha256 digests compared in a walk journal, not secrets
|
||||||
|
gitleaks generic-api-key@c1be98b7b703 documentation/tests/walk5-r201-2026-08-07/journal.md # false: sha256 digests compared in a walk journal, not secrets
|
||||||
|
gitleaks generic-api-key@c3b2b5665c9c documentation/audits/evidence-fixes-first-tester-2026-09-30/part0/cloudflare-dns.txt # false: a tunnel ID (a UUID), not the tunnel token
|
||||||
|
gitleaks generic-api-key@cf453385b707 documentation/tests/campaign2-evidence-2026-07-07/state/runner.log # accepted: the same dry-run deploy values in the campaign runner's log
|
||||||
|
gitleaks generic-api-key@e59eaeb4aee7 documentation/audits/new-apps-2026-10-01/tools/newapp_box.py # accepted: a throwaway test user's password in an audit tool for a fresh-app walk on a Tier-0 box
|
||||||
|
gitleaks generic-api-key@eada8358f62a hub/internal/web/render_test.go # false: a test constant or a decoy value, no live secret
|
||||||
|
gitleaks generic-api-key@efd4cc86d8d9 documentation/audits/power-cut-fixes-2026-10-14/E/demo-hp-journal-boot.txt # false: the id of HP's public UEFI certificate in a boot journal
|
||||||
|
gitleaks generic-api-key@f28c207293c1 documentation/tests/campaign2-evidence-2026-07-07/evidence/T-DEPLOY-SET/values_vaultwarden.json # accepted: dry-run deploy values of a 2026-07-07 campaign (generated, demo boxes since rebuilt)
|
||||||
|
gitleaks generic-api-key@f399eb9e3059 documentation/audits/night-2026-09-23/chaos/round-01-run.txt # accepted: the same demo app session token as round-01.log
|
||||||
|
gitleaks generic-api-key@f399eb9e3059 documentation/audits/night-2026-09-23/chaos/round-01.log # accepted: a demo app's session token, demo-hp 2026-09-23 (Tier 0); the 2026-10-12 review found no running demo app with it
|
||||||
|
gitleaks generic-api-key@fdabf6a76b48 documentation/audits/r890-instructions-2026-10-06/tools/wgerwalk.py # accepted: a throwaway wger walk's API token in an audit tool (Tier 0)
|
||||||
@@ -178,6 +178,12 @@ something, not only sessions that touch `documentation/` — which is why it is
|
|||||||
rows — a run can sit several pages earlier. **Scan every page** and match on `head_sha`; with a
|
rows — a run can sit several pages earlier. **Scan every page** and match on `head_sha`; with a
|
||||||
few hundred jobs that is a handful of requests. Guessing the page produced three consecutive
|
few hundred jobs that is a handful of requests. Guessing the page produced three consecutive
|
||||||
false "no CI job for this commit" readings in one session.
|
false "no CI job for this commit" readings in one session.
|
||||||
|
3. **A rerun REPLACES the attempt the `jobs` endpoint shows** (R-1137, measured 2026-10-18): the job
|
||||||
|
keeps its id, `run_attempt` becomes 2, `started_at`, `conclusion` and the log are the rerun's — the
|
||||||
|
red first attempt is gone from `jobs` and its log (controller jobs 1944, 1959: failure mails, now
|
||||||
|
green). **A matched job with `run_attempt` > 1 is not a plain green:** list `actions/tasks` (same
|
||||||
|
paging, match `head_sha`) — one task per attempt, the verdict in `status`, so it reads
|
||||||
|
„failure, then rerun success". The first attempt's log is not reachable by the API.
|
||||||
|
|
||||||
An unchecked green is an assumption, not an observation — and so is a green read off a field the
|
An unchecked green is an assumption, not an observation — and so is a green read off a field the
|
||||||
API never populates.
|
API never populates.
|
||||||
+56
@@ -16,6 +16,62 @@
|
|||||||
> and holds nothing of its own; this file does hold its own content, namely the standing rulings below.
|
> and holds nothing of its own; this file does hold its own content, namely the standing rulings below.
|
||||||
|
|
||||||
|
|
||||||
|
> **2026-10-19 (night) — controller 0.335.0 is the floor (MinAgent 0.158.1, 18:39 UTC); hub 0.162.0 BUILT, not deployed;
|
||||||
|
> agent `56aabc7` and installer 1.35.0 committed, not released/published.** R-921 (operator answer „add one field"): the
|
||||||
|
> agent serves its host-wide heavy-operation gate as `host_busy` on `GET /backup/status`; the controller reads it before
|
||||||
|
> any stop (held → contention, no stop). R-1070's cause: Docker 29 prints „error: no such object" in lower case, so a
|
||||||
|
> guest without cloudflared read `unknown` and failed the host OS-update rule; new state `absent` (passes when absent
|
||||||
|
> before the step too); `11` §8.2 had claimed `unknown` does not fail — corrected. R-1010: the janitor's forced snapshot
|
||||||
|
> delete is the API's `force=1`. Hub 0.162.0: R-1001 (restart seeding from the DB), R-1011 (exact-file sha). R-953 is
|
||||||
|
> gone: ArgoCD tracks none of the three Secrets now. **R-986: `scripts/security_scan_gate.py`, shared, in every repo's
|
||||||
|
> fast set and CI, WARN-FIRST**; govulncheck pin v1.1.4 → v1.8.0 (v1.1.4 cannot read Go 1.27); CI has no Go, so
|
||||||
|
> govulncheck runs only in the hook; a depth-1 CI clone scans the whole tree (the root commit), all accepted. R-893
|
||||||
|
> measured on 9202 (`audits/night-2026-10-19/P3/`) → D20. 9202 is on DHCP (.121 now).
|
||||||
|
|
||||||
|
> **2026-10-19 (late afternoon) — controller 0.334.0 is the floor (MinAgent 0.158.1, 16:31 local); hub 0.161.0 DEPLOYED;
|
||||||
|
> tenantsync v1.3.0 on ep0 (operator yes).** R-1159 (D19 „box key first, then lock"): the passphrase is used AFTER enroll
|
||||||
|
> (installer `/config` for its git login, the agent bakes it into `bootstrap.json`, the guest's first-boot pull, and —
|
||||||
|
> until 0.334.0 — every config refresh), so "keys only before a host exists" was not buildable. Now: `GET /box-config`
|
||||||
|
> with the controller key; the hub refuses passphrase keys (409, `passphrase_keys_refused`) once a ≥ 0.334.0 controller
|
||||||
|
> has reported since the host record was made — an rc (`0.334.0-rc.N`) counts as older. **A reinstall over a live box
|
||||||
|
> needs the operator's delete first.** R-1165: every new ep0 namespace gets `prune-<id>`; the hub alarms on a namespace
|
||||||
|
> with none. demo-hp's .104 clean after the router reservation. Register 163 → 161. `REPORT-day3-2026-10-19.md`.
|
||||||
|
|
||||||
|
> **2026-10-19 (afternoon) — controller 0.333.0 is the floor (MinAgent 0.158.1, 12:16 local); hub 0.160.0 DEPLOYED
|
||||||
|
> (operator yes); ep0 prune `keep-weekly 2` (operator yes, D18).** ep0 change: `prune-demo-hp`/`prune-demo-felhom` only,
|
||||||
|
> after PBS's own `--dry-run true` (API `proxmox-backup-debug api create …/prune`; it creates no task) showed nothing
|
||||||
|
> removed; households with no prune job (tester-1, Tester-2) = R-1165. R-1155: FileBrowser 1.5.6 reads its token from the
|
||||||
|
> cookie, `?auth=`, Bearer AND the Basic-auth password (`extractToken`) — the forwardAuth refusal must read all four (the
|
||||||
|
> security review caught Basic). R-1159 measured: passphrase → `/config` key and `/host-enroll` existing host key → every
|
||||||
|
> off-site key route; D19 open (CC picks A: keys only while no host exists). R-1158: the reinstall's off-site re-issue only
|
||||||
|
> rotates the hub's own registrar login; kept. gofmt gates in hub and agent runners. demo-hp's .104 is taken by another
|
||||||
|
> device (R-1166, the router). Register 167 → 163. `REPORT-day2-2026-10-19.md`.
|
||||||
|
|
||||||
|
> **2026-10-19 (day) — controller 0.332.0 is the floor (MinAgent 0.158.1, 05:24 local); agent 0.160.0 released and on the
|
||||||
|
> three test hosts, NOT vouched; hub 0.159.0 DEPLOYED (operator yes in advance).** One sign-in for the file manager (R-1100,
|
||||||
|
> D16): dashboard pass → host-only `felhom_files` cookie → traefik forwardAuth `/__felhom_gate/files` sets the per-box
|
||||||
|
> SECRET header (`<data>/filebrowser-sso-header`, never logged) naming FileBrowser user `felhom-household`; traefik makes a
|
||||||
|
> RELATIVE Location in a forwardAuth answer absolute against the auth server — always answer absolute URLs. R-1151: the
|
||||||
|
> escrow package uploads at `POST /escrow/ceremony/confirm` (two typed words), an old controller keeps the claim order.
|
||||||
|
> R-1149 (D17): local tier `keep-daily=N`. R-1154 rule (after a security review of the helper's first draft): the
|
||||||
|
> host-enroll reuse path NEVER re-issues — the passphrase must not reach the operator's re-issue powers; it mails the
|
||||||
|
> operator. R-1153 was the test file (FileBrowser previews .docx with MuPDF before OnlyOffice). R-887 closed (381 jobs, 0
|
||||||
|
> lost since S21). The old-restic check after demo-hp's first real 0.19.1 prune passed. Register 168 → 166.
|
||||||
|
> `REPORT-day-2026-10-19.md`.
|
||||||
|
|
||||||
|
> **2026-10-18 (evening) — controller 0.331.0 is the floor (MinAgent 0.158.1): OnlyOffice opens and saves from FileBrowser
|
||||||
|
> and Nextcloud, FileBrowser back at once after a restart (R-1018), a removal deletes a volume the definition dropped
|
||||||
|
> (R-1117); catalog `30eea97`: Immich/Nextcloud Redis without a volume; golden 0.331.0 baked, NOT vouched; a fresh install
|
||||||
|
> from the ISO proved installer 1.34.0 + vouched golden/agent + auto-floor (R-1148); agent `main` carries R-978, R-973,
|
||||||
|
> R-1093 unreleased. *Decided by CC unattended — operator may reverse:* the R-973 cap numbers (3 successful daily-tier runs
|
||||||
|
> per 24 h, one weekly off-site run per 6 days; `07` §6.1). Tester4 is kept as the standing fresh-install customer
|
||||||
|
> (operator). `REPORT-evening-2026-10-18.md`.**
|
||||||
|
|
||||||
|
> **2026-10-18 (day) — hub 0.158.0 DEPLOYED (access log live, agent-only vouch, R-895 first verdict 2026-10-20);
|
||||||
|
> controller 0.330.0 is the floor (R-1144); agent 0.159.2 released and on the three test hosts, NOT vouched; installer
|
||||||
|
> 1.34.0 published (vault-first step 4b, R-1145; fresh install unproven, R-1148); estimator Immich peak 3,983 MB
|
||||||
|
> (R-1147).** Independent review of the night: `audits/day-2026-10-18/P2/`. `REPORT-day-2026-10-18.md`.
|
||||||
|
|
||||||
> **2026-10-18 (night) — controller 0.329.0 is the floor (MinAgent 0.158.1); golden 0.329.0 baked, NOT vouched; hub 0.158.0
|
> **2026-10-18 (night) — controller 0.329.0 is the floor (MinAgent 0.158.1); golden 0.329.0 baked, NOT vouched; hub 0.158.0
|
||||||
> built, NOT deployed.** Each app's own `appdata/<folder>` (R-1142), empty allow-list (R-981), drive retirement asks the
|
> built, NOT deployed.** Each app's own `appdata/<folder>` (R-1142), empty allow-list (R-981), drive retirement asks the
|
||||||
> agent first (R-1063), plain no-durable-id refusal (R-1069), `:latest` for releases only (R-1143), Immich 3072M/2048M
|
> agent first (R-1063), plain no-durable-id refusal (R-1069), `:latest` for releases only (R-1143), Immich 3072M/2048M
|
||||||
|
|||||||
@@ -0,0 +1,139 @@
|
|||||||
|
# REPORT — day 2026-10-18
|
||||||
|
|
||||||
|
## In plain words
|
||||||
|
|
||||||
|
- **The hub update is live.** Its own access log works and writes no secret (tested live with a fake token). You can
|
||||||
|
now vouch an agent without a new golden. The off-site snapshot check gives its first verdict on 20 October.
|
||||||
|
- **A fresh reviewer read last night's code.** One medium finding: a stale host key could have left the emergency
|
||||||
|
password unknown. Today's installer change (save in the hub first, then set) already closes it. Three low findings:
|
||||||
|
two fixed today, one is a known design limit, now written in its row.
|
||||||
|
- **„Retire anyway":** when the host says no, the drive's apps start again. Where the host's disk list already says no,
|
||||||
|
nothing is stopped at all. Controller 0.330.0 is the floor for every box.
|
||||||
|
- **Agent 0.159.2** runs on demo-hp, Tester 1 and demo-felhom (your yes). A whole-box backup ran with no pool check.
|
||||||
|
**The vouch is yours** (STATUS).
|
||||||
|
- **Installer 1.34.0 is published** (your yes): the emergency password is saved in the hub first, then set, then
|
||||||
|
checked with a login. Proven on demo-hp. A fresh install with it has not run yet (row R-1148).
|
||||||
|
- **The calculator** shows Immich's bigger figure (your yes). Immich alone on an 8 GB machine now reads „does not fit".
|
||||||
|
- **The old-restic check has not run.** demo-hp's clean-up comes only after tonight's backup (2026-10-19 about 04:16
|
||||||
|
Budapest). The STATUS item stays open.
|
||||||
|
|
||||||
|
## 1. Hub 0.158.0 deploy
|
||||||
|
|
||||||
|
ArgoCD: only `Deployment/hub` was OutOfSync; synced that resource alone → Synced/Healthy, image `felhom-hub:0.158.0`,
|
||||||
|
startup log `felhom-hub 0.158.0 starting` + `access log: /data/access-log …, kept 90 days`. Floor read back from the hub
|
||||||
|
DB with its WAL: 0.329.0 / 0.158.1 at deploy time (`audits/day-2026-10-18/P1/hub-deploy-readback.txt`). Reports from all
|
||||||
|
three boxes in the access log within two minutes (sources .101, .149, .155).
|
||||||
|
- **Access log (R-1140, closed):** today's file `access-2026-10-18.log`, mode 0600. A fresh 48-hex fake token sent in a
|
||||||
|
query, a `/bind/<token>` path, a `session=` query, a Bearer header, a Cookie and a POST body: found 0 times; control
|
||||||
|
(the same file with the token appended) 1. Secret shapes (Bearer value, a 32+ character run, `password=`, `token=`,
|
||||||
|
`session=`): 0; control 1. `/bind/<token>` is written `/bind/[redacted]` (`P1/access-log-live-test.txt`).
|
||||||
|
- **Agent-only vouch:** read-only — `/configuration` loads with CC's token (Day-0 section, golden 0.329.0, agent 0.159.1);
|
||||||
|
the save (`/configuration/artifacts`) answers 403 to the token, by design. No manifest change by CC.
|
||||||
|
- **Off-site alarm (R-895, NARROWED):** no fixture route exists in the hub for it; the unit tests are the proof so far.
|
||||||
|
Tomorrow's 07:10 run is the baseline (no verdict by design); the first verdict is 2026-10-20 07:10 — a DUE-CHECKS row.
|
||||||
|
|
||||||
|
## 2. The independent review
|
||||||
|
|
||||||
|
A fresh subagent with no context from the night reviewed controller v0.328.0→v0.329.0, hub 0.157.0→0.158.0, the agent's
|
||||||
|
orphan-check change and the installer's `--rotate-recovery-only`. Its report was written to a file and read back
|
||||||
|
(`audits/day-2026-10-18/P2/independent-review.md`, 54 lines). Verdicts (`P2/verdicts.md`):
|
||||||
|
|
||||||
|
| # | Finding | Verdict |
|
||||||
|
|---|---|---|
|
||||||
|
| F1 medium | rotate-only: unauthenticated healthz, then chpasswd, then a vault that a stale key fails → an unvaulted password | REAL in the night's code → **fixed** by R-1145 (vault first); harness `R1145-e`, fails against HEAD |
|
||||||
|
| F2 low–medium | `offsite_snapshots_unexplained` is explained by the box's own reported run time | REAL, by design (decision 68) → written into R-895; the custody alarm `_vanished` is not affected |
|
||||||
|
| F3 low | „anyway" stops apps before the agent is asked | REAL, was R-1144 → **fixed** (controller 0.330.0) |
|
||||||
|
| F4 low | host key in curl's argv during the vault PUT | REAL, pre-existing → **fixed** (header file) |
|
||||||
|
|
||||||
|
## 3. R-1144
|
||||||
|
|
||||||
|
`decommissionAnyway`: reads the agent's disk list first (a role other than user-data, or the backup target → the plain
|
||||||
|
refusal, nothing stopped); after a refusal past the stops (policy refusal or a failed agent call) starts exactly the apps
|
||||||
|
that were running; a drive the agent let go keeps its apps stopped. Household text HU + EN. Seam `decomStacks`, so no
|
||||||
|
test reaches docker. Three tests, two red-proofs (`P3/redproof-r1144.txt`). Not pressed live: no test box has a drive
|
||||||
|
that is safe to retire.
|
||||||
|
|
||||||
|
## 4. Agent 0.159.2
|
||||||
|
|
||||||
|
Released with `release-agent.sh` (code pushed first, CHANGELOG after — the `release-complete` hook refuses the other
|
||||||
|
order): sha256 `4d98868e922d6c10665a6c0da9323e8627ea9c589094e81d27d741e625d9d8f9`, bundle
|
||||||
|
`477704ea2980948433ffc34bb59f008cc11410fd9683d386bfd3e2f085d55bc9` (read back from the registry). **demo-hp proof:** a
|
||||||
|
whole-guest backup of 9201 through the dashboard (`/api/guest-backup/trigger`, snapshot mode, completed in 7.5 min):
|
||||||
|
**0** `felhom-thin-check` sudo lines; control — the night's backup on 0.159.1 at 04:33 had **1**
|
||||||
|
(`P4/demo-hp-backup-journal.txt`). The failure path stays covered by the agent's fixture tests. **Operator: yes** →
|
||||||
|
Tester 1 and demo-felhom by signed jobs (agent_update + agent_config_update): both `signed op COMPLETED`, capability
|
||||||
|
probe 75/75, `update committed` 0.159.2, and a host report on 0.159.2 from each (`P4/`). **Not vouched** — STATUS.
|
||||||
|
|
||||||
|
## 5. The password order (R-1145, R-1146)
|
||||||
|
|
||||||
|
Step 4b (install and rotate-only): generate → PUT to the vault (200 = the hub's DB write is done) → chpasswd (one retry)
|
||||||
|
→ a PVE API login with the new value. **The states that can remain, and why each is safe or not:**
|
||||||
|
|
||||||
|
| State | When | Safe? |
|
||||||
|
|---|---|---|
|
||||||
|
| nothing changed | the vault refuses or is unreachable (a stale key included) | safe — root and the vault keep their old, matching pair |
|
||||||
|
| root = vault = new, PVE login 200 | the normal end | safe |
|
||||||
|
| vault = new, root = previous | chpasswd fails twice (or the run is killed between vault and set) | **not fully safe:** the vaulted value does not open the console; root keeps its previous password (the shadow line is fingerprinted before/after to prove it). The host key cannot read the vault back (by design), so the previous value cannot be re-vaulted; the log says the state in words, and the next run fixes it (it vaults first again). |
|
||||||
|
| root = vault = new, PVE login refused | pveproxy trouble | safe for the shell; the run dies and says the web login refused |
|
||||||
|
|
||||||
|
The old order's dangerous state (root = new, vault = old) is no longer reachable. The hub cannot tell the installer its
|
||||||
|
„last vaulted" time (no host-readable route); the installer trusts the PUT's 200, and CC checked the time moved in the
|
||||||
|
live proof. Live on demo-hp: vault `set_at` 2026-07-21 → 2026-10-18T10:43:33Z, the new value gets a PVE ticket (200),
|
||||||
|
the old one 401 (`P5/demo-hp-rotation.txt`). Harness: 52 passed / 0 failed (was 45/2); R1145-a..e red-proofed against
|
||||||
|
HEAD. Installer **1.34.0 published** on the operator's yes: tag `installer-v1.34.0`, both webpage pins, ArgoCD synced
|
||||||
|
the webpage Deployment only; the served file's sha256 = the tag's (`df722ee4…`). R-1146: SC2066 and GL8-F6 fixed, the
|
||||||
|
harness is gate `hostinstall-harness` with two decoys.
|
||||||
|
|
||||||
|
## 6. Calculator (R-1147)
|
||||||
|
|
||||||
|
Before/after (`audits/day-2026-10-18/P6-before-after.md`): Immich peak 3,290 → 3,983 MB (typical unchanged 1,557). The
|
||||||
|
one changed verdict: **Immich alone on 8 GB: fits, but tight → does not fit**. The family starter set is unchanged.
|
||||||
|
**Operator: yes** → published (cache-bust v5); live `sizing-data.json?v=5` serves 3,983. SIZING.md's table and the
|
||||||
|
pinned estimator tests moved with it. R-1036 closed (D14 = no).
|
||||||
|
|
||||||
|
## 7. The old-restic check
|
||||||
|
|
||||||
|
**Not run — the prune has not happened yet.** The hub allows demo-hp a clean-up window 6 days 12 hours after the last
|
||||||
|
one (2026-10-12 02:16:53 UTC → from 2026-10-18 14:16:53 UTC). But the box asks for a window only after its own nightly
|
||||||
|
off-site run, and that runs at 04:15 CEST. So the first real 0.19.1 prune comes **2026-10-19 at about 02:16 UTC**
|
||||||
|
(04:16 Budapest). Read at 14:33 UTC: no `[offbox]` line since the 11:09 restart, the scheduler's next run 2026-10-19
|
||||||
|
04:15 CEST, the hub's last window still id 9 (`P8/cleanup-window-not-yet.txt`, the hub DB read with its WAL). CC did not
|
||||||
|
grant a window by hand (that is the operator's). The STATUS item stays open, with the corrected time; the check script
|
||||||
|
is ready (`rx.sh` + a wrapper that lists and restores the newest, the oldest and the newest 0.14-made snapshot with both
|
||||||
|
restics and compares tree hashes).
|
||||||
|
|
||||||
|
## 8. Versions, commits, fixtures
|
||||||
|
|
||||||
|
- Hub **0.158.0** deployed (felhom.eu `4c319dde`).
|
||||||
|
- Controller **0.330.0** — `6c6bd87`; floor 0.330.0 / MinAgent 0.158.1 at 11:10:59 UTC (303 `floor_set`, read back with
|
||||||
|
the WAL); per-box floors set and cleared; box walk green on the three test boxes (0 refusals, all healthy, login and
|
||||||
|
files 200, https-only OK).
|
||||||
|
- Agent **0.159.2** — `65c1994` (code), tag `v0.159.2`, CHANGELOG `e25454c`.
|
||||||
|
- Installer **1.34.0** — felhom.eu `88aecf55` (vault-first + harness), `624bcdc3` (1.34.0, tagged), `3332eb09` (pins).
|
||||||
|
- Estimator — `37cb83c1`. Fixtures: none changed (the new texts travel in JSON).
|
||||||
|
|
||||||
|
## 9. Register — 173 before, 168 after; opened 1, closed 6
|
||||||
|
|
||||||
|
Closed: R-1140, R-1144, R-1145, R-1146, R-1147, R-1036. Opened: R-1148. Narrowed: R-895 (dated check 2026-10-20).
|
||||||
|
|
||||||
|
## 10. Observations
|
||||||
|
|
||||||
|
1. Installer 1.34.0's default path has not run in a fresh install. FILED: R-1148.
|
||||||
|
2. The review's F2 (box-reported run times explain new snapshots). NOT-A-FINDING as a new row: a known design limit,
|
||||||
|
recorded in R-895.
|
||||||
|
3. `release-agent.sh` vs the `release-complete` hook: the CHANGELOG must follow the release. NOT-A-FINDING: the documented
|
||||||
|
order (memory and skill); I tried the other order first and the hook refused it, as designed.
|
||||||
|
4. The estimator's new Immich figure is `memory.current` (with cache), not the `used` measure it normally takes.
|
||||||
|
NOT-A-FINDING: on the safe side, said in SIZING.md, and approved by the operator with the table in front of them.
|
||||||
|
5. A safety check refused a `sh -c` with an unguarded `rm "$H"`. NOT-A-FINDING: rewritten with `${H:?}` in a helper.
|
||||||
|
|
||||||
|
## Teardown — three layers
|
||||||
|
|
||||||
|
- **Machines:** demo-hp 9201 — one whole-guest backup taken (kept, like any backup); no app installed or removed today.
|
||||||
|
demo-hp host — the installer copy shredded after the rotation; root password rotated on purpose (new value in the
|
||||||
|
vault). Tester 1, demo-felhom — agent 0.159.2 and controller 0.330.0 through their normal paths; nothing else.
|
||||||
|
- **Hub:** hub 0.158.0; global floor 0.330.0 / 0.158.1; no per-box floor; signed jobs consumed. No vouch by CC.
|
||||||
|
- DooPlex: builds only; ArgoCD synced two Deployments (hub, webpage), each alone.
|
||||||
|
|
||||||
|
Instruction files: none edited. System poster: none of the three (the poster shows backup retentions; the hub's access
|
||||||
|
log is not on it). Architecture documents read: `02`, `03`, `05`, `07`, `09` §3 decision 250; `break-glass.md`.
|
||||||
@@ -0,0 +1,156 @@
|
|||||||
|
# REPORT — day 2026-10-19
|
||||||
|
|
||||||
|
The operator answered the two STOPs in advance („YES to the agent and the hub release") and earlier: golden 0.331.0
|
||||||
|
vouched; D16 (one sign-in) yes; D17 „one per day, 3 days". No question was asked during the session.
|
||||||
|
|
||||||
|
## In plain words
|
||||||
|
|
||||||
|
- **Agent 0.160.0 runs on all three test hosts.** A new recovery code replaces the old one only after the household types
|
||||||
|
two of its words back; a broken-into box cannot flood the whole-box backups; the local copies are one per day for 3
|
||||||
|
days; an empty partition does not ask „data will be erased". **It is not vouched — that is yours.**
|
||||||
|
- **One sign-in for the file manager is live on every box** (controller 0.332.0, the floor). „Fájlok ↗" opens the app's
|
||||||
|
folder with no second login. A sign-out on the dashboard asks again. No app on the box can pretend to be the household.
|
||||||
|
- **Hub 0.159.0 is live.** A refused backup has its own alarm text. A reinstall over an old host record now mails you; it
|
||||||
|
does not re-issue the codes by itself (a security review stopped the first draft that did).
|
||||||
|
- **The preview error (R-1153) was a broken test file**, not the box.
|
||||||
|
- **The old backup program still reads demo-hp's off-site copies** after the first real clean-up with the new one.
|
||||||
|
|
||||||
|
## 1. Agent 0.160.0 (R-978, R-973, R-1093, R-1149, R-1151) — `audits/day-2026-10-19/P1/`
|
||||||
|
|
||||||
|
- **Content:** R-978 (an abandoned second ceremony keeps the first code), R-973 (the cadence cap: daily tier 3 per 24 h,
|
||||||
|
off-site 1 per 6 days), R-1093 (an empty partition is not data), **R-1149** (local tier `keep-daily=N`, operator D17),
|
||||||
|
**R-1151** (the package uploads only at `POST /escrow/ceremony/confirm` with two typed words; a claim without
|
||||||
|
`defer_upload` keeps the R-978 order, so an old controller still completes a ceremony). Commits `1b6e316`, `c11e3e2`,
|
||||||
|
`124164d` (CHANGELOG), `4545819` (two texts, after the release, unreleased). Released with `release-agent.sh`: sha256
|
||||||
|
`7ebb547f2b1ab93544ba828d3040351b60c4126327a5b85bbcb33f8f2b3dfca1`, bundle
|
||||||
|
`6b06090190ff609af1fe9842447ebf385fd52da4be382bc7ca708b558c8e202f`, round trip verified, tag `v0.160.0`.
|
||||||
|
- **Red-proofs:** keep-last for the primary → „primary tier prune = "keep-last=3"" and „holds 0 copies made before the
|
||||||
|
compromise"; the claim's defer flag ignored → „a code that was shown but never typed back replaced the package: code 1
|
||||||
|
no longer opens it (uploads=2)" (all three abandon paths); the word check skipped → „one wrong word: got 200".
|
||||||
|
- **demo-hp proof** (signed jobs 01:22 UTC, running 01:23, „update committed" 01:24; probe 75/75):
|
||||||
|
- a normal „Mentés most" of 9201: green, 6.9 GB in 443 s; yesterday's copy pruned (keep-daily=1, the same copy keep-last
|
||||||
|
would prune); the cap's history on disk holds both successes;
|
||||||
|
- escrow through the wizard's own calls: ceremony 1 — one wrong word 422, the right words 200; code 1 opens the hub's
|
||||||
|
package, code 0 is a kept earlier one (422, superseded at the confirm second 01:47:47). Ceremony 2 shown and abandoned —
|
||||||
|
code 1 still opens it (200), code 2 opens nothing (400). demo-hp's stored code is now code 1 (CC's secret store);
|
||||||
|
- R-1093 on the real route: a 64 MB RAM disk (`scsi_debug`, `/dev/sdb`), one empty GPT partition → `POST /disks/format`
|
||||||
|
→ 200 formatted, no confirmation; control: the same partition, now ext4 → 403 `needs_confirmation`. RAM disk removed.
|
||||||
|
**A loop device cannot be used:** the route and the probe accept only sd/nvme/vd/hd names (by design).
|
||||||
|
- **The answer:** the operator's „yes" (in advance). Tester 1 and demo-felhom took 0.160.0 by signed jobs (01:52 UTC →
|
||||||
|
01:54 / 01:55, probe 75/75, `keep-daily=3` armed on both); the hub's host rows read 0.160.0 for all three, and fresh
|
||||||
|
reports after the hub restart. **Old controller with the new agent:** pinned by `TestEscrowCeremony_R1151_OldControllerKeepsTheClaimOrder`; **new controller with agent 0.159.2:** pinned by `TestR1151_OldAgentClaimCompletesAtOnce` and the agentapi test — no host runs 0.159.2 any more, so not walked live.
|
||||||
|
|
||||||
|
## 2. One sign-in (R-1100) — `audits/day-2026-10-19/P2/`, `P4/05-*`, `P4/06-*`
|
||||||
|
|
||||||
|
- **Built** exactly the design note's recommendation (controller `9bdd8f8`, `601bdba`): the dashboard pass, the host-only
|
||||||
|
files cookie (derived from the session fingerprint, the R-976 shape), the per-box secret header name, the forwardAuth
|
||||||
|
answer; header user `felhom-household` with the household's file rights from userDefaults; `logoutRedirectUrl` = the
|
||||||
|
dashboard; `admin` stays a password user; the data tab's „asks for its own password" note only without the sign-in.
|
||||||
|
- **Tests, each red-proved:** replayed pass, expired pass, pass for another host (two guards: the host check and the
|
||||||
|
host-bound session key — red only with both removed), wrong X-Forwarded-Host, no session → no pass, signed out → asks
|
||||||
|
again (and after a password change), cross-site → no pass, dot-segments in `to`, the absolute redirect, the config
|
||||||
|
trusts only the secret name, the header user's rights, the route asks the controller and strips the header, the note.
|
||||||
|
- **Live, found and fixed before release:** on rc.2 the pass answer's relative redirect became
|
||||||
|
`http://felhom-controller:8080/files/` — traefik resolves a forwardAuth answer's Location against the auth server. Fixed
|
||||||
|
(absolute), red-proved, rc.3. A background security review then flagged CSRF and revocation: `/__files/open` now refuses
|
||||||
|
a cross-site request and dot-segments; the revocation residual is R-1155.
|
||||||
|
- **Live proof (rc.3, 9201, headless Chrome through the public names mapped to the box):** signed in → calibre-web's
|
||||||
|
„Fájlok ↗" → FileBrowser at `NVME 1TB/media/books`, no login form, user `felhom-household` (admin false, modify true,
|
||||||
|
login method proxy); signed out → `files.<domain>` went to the dashboard login. Screenshots `browser/01-…`, `02-…`,
|
||||||
|
`03-…`. Forged headers (`forged-headers.txt`): through traefik with no cookie — the old public name, a random name and the
|
||||||
|
REAL secret name all 401; from the bookstack container straight to `filebrowser:80` — the old name and a random name 401;
|
||||||
|
control: the real name from the same container 200 (the protection is the name's secrecy). The brief named Nextcloud;
|
||||||
|
bookstack was used (an app on 9201's network with curl).
|
||||||
|
|
||||||
|
## 3. Preview check (R-1153) — `audits/day-2026-10-19/P3/`
|
||||||
|
|
||||||
|
On 9202 (controller 0.331.0): OnlyOffice installed, the link on (FileBrowser back in 4 s). The hand-made `teszt.docx`
|
||||||
|
(912 bytes) → 500 „fitz: cannot open document"; OnlyOffice's own `sample.docx` → **200, a 256×256 JPEG, 0.3 s**. Same with
|
||||||
|
the link OFF. FileBrowser 1.5.6 previews .docx with its own MuPDF before OnlyOffice (`preview.go` `determinePreviewType`).
|
||||||
|
**Closed: not a product defect.** OnlyOffice removed through the product (both volumes gone), test files deleted.
|
||||||
|
|
||||||
|
## 4. Controller release 0.332.0 — `audits/day-2026-10-19/P4/`
|
||||||
|
|
||||||
|
Image `0.332.0` from `601bdba` (digest `sha256:245266a4…`), 03:16 UTC. Per-box floors 03:16:43 → Tester 1 and demo-felhom
|
||||||
|
on 0.332.0 at 03:16:45–55, demo-hp (from rc.3) at 03:17. Box walk on all three: 0 unhealthy, 0 allow-list refusals,
|
||||||
|
FileBrowser re-published 6 s after its re-create, public pages 200; https-only 28 OK, 0 failures, both controls right.
|
||||||
|
„Fájlok ↗" with no login through the public internet on demo-hp and Tester 1 (pass → cookie → `felhom-household`; a fresh
|
||||||
|
browser 401; sign-out → back to the dashboard). demo-felhom: CC holds no dashboard password for the lived-in box, so its
|
||||||
|
check is the files (secret 0600, config and route carry it) and the refusals (forged secret name through traefik 401, old
|
||||||
|
name from outside 401, a forged pass 403). **Global floor 0.332.0 / MinAgent 0.158.1 at 03:24:18 UTC**, read back with the
|
||||||
|
WAL; per-box floors cleared (read back empty). MinAgent 0.158.1 ≤ the vouched agent 0.159.2. i18n fixtures changed (named):
|
||||||
|
`escrow_first`, `escrow_receremony`; all others byte-identical. No golden (decision 250); the floor is above golden 0.331.0
|
||||||
|
until Sunday's bake.
|
||||||
|
|
||||||
|
## 5. Hub 0.159.0 (R-1150, R-1154) — `audits/day-2026-10-19/P5/`
|
||||||
|
|
||||||
|
- Written by a helper (`5121536f`, `b505ce4b`). **R-1154's rule, and why:** the helper's first draft re-issued the claim
|
||||||
|
code and the off-site provider password automatically when the old host was stale or switched off. A background
|
||||||
|
security review flagged it; the helper then read what the retrieval passphrase already grants (`/config` → the
|
||||||
|
controller API key; the enroll → the host key) and found the re-issues are the operator's session-only powers
|
||||||
|
(decision 252). So the passphrase must not reach them: **the reuse path re-issues nothing; when the old box looks gone it
|
||||||
|
mails the operator** („delete the old host and run the installer again, or re-issue …"), a live box gets an info record.
|
||||||
|
The operator's delete stays the gate. **R-1150:** `whole_guest_backup_refused` (operator-only, hub-origin; NOT in
|
||||||
|
`allowedEventTypes`, against the brief's wording — the suspected controller must not post or pre-empt it, R-895's
|
||||||
|
shape). Tests red-proved (helper's list in `hub/CHANGELOG.md`).
|
||||||
|
- **A push blocker on the way:** R-887's dated check was due today. Measured: 381 CI jobs since the S21 change, 0 lost (the
|
||||||
|
three re-runs' first attempts ran 47–49 s; control: the known lost attempt of 2026-10-10 ran 11 min 48 s and the attempt
|
||||||
|
count finds it). Closed.
|
||||||
|
- **Deploy:** image from a clean clone (the shared checkout held my uncommitted evidence), manifest `200ab653`, ArgoCD
|
||||||
|
synced the hub Deployment only (the only OutOfSync resource), Synced/Healthy, no ERROR, the floor read back, then fresh
|
||||||
|
reports from all three hosts (02:23–02:25 UTC) and all three controllers (02:29–02:31 UTC).
|
||||||
|
|
||||||
|
## 6. The old-restic check — `audits/evening-2026-10-18/P8/old-restic-check.txt`
|
||||||
|
|
||||||
|
The night watcher ran it: window 12 at 02:18 UTC removed 63 of demo-hp's 324 copies (→ 261); the old restic (0.14.0) and
|
||||||
|
the new (0.19.1) both list 261 and restore a 0.19.1-made copy and an August copy to identical content hashes. STATUS item
|
||||||
|
closed; `REPORT-evening-2026-10-18.md` §9 filled.
|
||||||
|
|
||||||
|
## 7. Versions, commits, fixtures, rows
|
||||||
|
|
||||||
|
- Agent **0.160.0** (released, not vouched) — `1b6e316`, `c11e3e2`, `124164d`, `4545819`. Controller **0.332.0** (floor) —
|
||||||
|
`ba4a6f0`, `9bdd8f8`, `601bdba`; rc.1–rc.3 on 9201 only. Hub **0.159.0** (deployed) — `5121536f`, `b505ce4b`, manifest
|
||||||
|
`200ab653`. felhom.eu `afc939bb` (R-887), `5622ce27`, this commit. Catalog: none.
|
||||||
|
- Fixtures: `escrow_first`, `escrow_receremony` (controller); no other page changed.
|
||||||
|
- **Register: 168 before, 166 after; closed 10** (R-887, R-978, R-973, R-1093, R-1149, R-1151, R-1100, R-1153, R-1150,
|
||||||
|
R-1154), **opened 8** (R-1155 … R-1162).
|
||||||
|
- `unproven.py --summary`: not walked 35 of 55 — unchanged. Capability map: not changed. System poster: text only — the
|
||||||
|
local whole-guest „Kept" cell („one copy per day, up to 3 days") edited in both the facts file and the drawing.
|
||||||
|
|
||||||
|
## Observations
|
||||||
|
|
||||||
|
1. FileBrowser's own 2-hour token can outlive the dashboard session if only the files cookie is lost. FILED: R-1155.
|
||||||
|
2. A reinstall within 45 min of the old box's last report reads „live" (no mail). FILED: R-1156.
|
||||||
|
3. Repeated cap refusals inside the 24 h hold raise no new alarm. FILED: R-1157.
|
||||||
|
4. The off-site leg of the clean-slate re-issue may be obsolete since hub v0.127.0. FILED: R-1158.
|
||||||
|
5. The retrieval passphrase yields the controller API key, which registers and confirms off-site keys. FILED: R-1159.
|
||||||
|
6. Hub and agent have no gofmt gate (~20 and 14 files drift). FILED: R-1160.
|
||||||
|
7. The escrow wizard calls the code „live" at the reveal; since R-1151 it is live only after the typed-back words.
|
||||||
|
FILED: R-1161.
|
||||||
|
8. R-973's off-site half (ep0 prunes by count) outlives the closed row. FILED: R-1162.
|
||||||
|
9. traefik resolves a forwardAuth answer's relative Location against the auth server. NOT-A-FINDING: found on rc.2 and
|
||||||
|
fixed before the release (red-proved); recorded as a rule in CONTEXT and memory.
|
||||||
|
10. The ceremony log line and two R-973 comments were stale. NOT-A-FINDING: fixed in agent `4545819` (the size rule).
|
||||||
|
11. My first OnlyOffice link call sent `{}` and switched the link off. NOT-A-FINDING: my call; the API wants
|
||||||
|
`{"enabled": true}`; it was already off, nothing restarted.
|
||||||
|
12. The R-1093 route refuses loop devices. NOT-A-FINDING: by design (only real disk names are allowed); a RAM disk was used.
|
||||||
|
13. CC holds no dashboard password for demo-felhom, so its „Fájlok ↗" was not walked as a household. NOT-A-FINDING: the
|
||||||
|
lived-in box's household password is not CC's (decision 254); its files and refusals were checked instead.
|
||||||
|
14. A helper's commits sat unpushed in the shared felhom.eu clone for ~40 minutes (blocked by R-887's due check).
|
||||||
|
NOT-A-FINDING: the gate did its job; pushed with my R-887 commit.
|
||||||
|
15. Two shell scripts of mine were refused by Claude Code's removal check (a cleanup `rm` it could not read behind
|
||||||
|
`bash -c`). NOT-A-FINDING: rewritten without the removal; nothing was deleted.
|
||||||
|
|
||||||
|
## Teardown — three layers
|
||||||
|
|
||||||
|
- **Machines:** 9202 — OnlyOffice removed through the product (volumes gone), test documents deleted; 9201 — on 0.332.0
|
||||||
|
(the floor); the escrow test left demo-hp on code 1 (stored); demo-hp host — the RAM disk module removed, helper scripts
|
||||||
|
deleted, no file left in `/root` (an older helper, `/root/opens.sh`, removed by name at the end). Cookie jars and
|
||||||
|
password files on DooPlex shredded or emptied. One debugging sign-in on demo-hp's dashboard (curl, 03:03 UTC) was not
|
||||||
|
signed out; its cookie file is shredded, so the session expires unused.
|
||||||
|
- **Hub:** global floor 0.332.0 / 0.158.1, no per-box floor; hub 0.159.0; no vouch.
|
||||||
|
- **DooPlex:** builds (agent, controller rc.1–3 + 0.332.0, hub 0.159.0 from a clean clone) and the hub deploy; no Docker
|
||||||
|
cleanup.
|
||||||
|
|
||||||
|
Architecture read: `01`, `02` „Logins and folders", `03` §8a, `04`, `05`, `07` §6/§6.1; the design note. Documents changed:
|
||||||
|
`02`, `03` §8a, `07` §6.1, the poster facts + drawing text, `05` §8 (helper). Instruction files: none changed.
|
||||||
@@ -0,0 +1,120 @@
|
|||||||
|
# REPORT — second day brief, 2026-10-19 (ep0 one copy per week; the passphrase measured; hub 0.160.0; controller 0.333.0; gofmt gates)
|
||||||
|
|
||||||
|
## In plain words
|
||||||
|
|
||||||
|
- **ep0 keeps one whole-box copy per week, for 2 weeks** (your „yes"). Before the change, PBS's own dry run showed it
|
||||||
|
removes nothing today. The first real run is 2026-10-20 03:30 UTC; CC reads it then (row R-1162 stays open until then).
|
||||||
|
- **The household passphrase:** alone it gets the box's keys, and with them every off-site key action. Two options are
|
||||||
|
in STATUS (D19). CC picks A: the passphrase gives keys only while the customer has no box record.
|
||||||
|
- **Controller 0.333.0 is the floor for every box** (12:16 local): a dashboard sign-out now also ends the file manager's
|
||||||
|
own 2-hour pass; the recovery-code page says the code becomes live after the two words.
|
||||||
|
- **Hub 0.160.0 is live** (your „yes"): a reinstall within 45 minutes mails you too; a repeated refused backup is counted
|
||||||
|
and the alarm repeats at most once a day; the old off-site re-issue on a reinstall is kept, with true words.
|
||||||
|
- **gofmt:** the hub and the agent are formatted, and both now have the controller's gate.
|
||||||
|
- **New for you:** another device keeps taking demo-hp's network address (R-1166) — fix it in the router.
|
||||||
|
|
||||||
|
## 1. ep0 (R-1162; `07` Plane-2 row and §6.1)
|
||||||
|
|
||||||
|
- **Before** (`audits/day-2026-10-19b/P1/ep0-settings-before.txt`): `prune-demo-hp` and `prune-demo-felhom` —
|
||||||
|
`keep-last 2`, 03:30, ns-scoped, `max-depth 0`; `prune-operator-hubdb` — keep-daily 14 + keep-weekly 8, 03:45.
|
||||||
|
`tester-1` (3 copies) and `Tester-2` (1) have **no job**. ep0's clock is UTC.
|
||||||
|
- **DooPlex's copy of ep0:** `prune-ep0-copy`, `keep-weekly 8`, 07:30 (decision 71) — already by time; not changed.
|
||||||
|
- **Dry run** (PBS's own: `proxmox-backup-debug api create /admin/datastore/felhom-offsite/prune --dry-run true …`, no
|
||||||
|
task created; cross-checked with an offline model of PBS's selection, `P1/prune_sim.py`; 8 snapshots before and after):
|
||||||
|
|
||||||
|
| Box | Snapshots | keep-last 2 removes | keep-weekly 2 keeps | keep-weekly 2 removes |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| demo-hp | 2 | none | 2026-10-08, 2026-10-15 | none |
|
||||||
|
| demo-felhom | 2 | none | 2026-10-06, 2026-10-13 | none |
|
||||||
|
| tester-1 (no job) | 3 | (2026-10-04) | 2026-10-11, 2026-10-18 | (2026-10-04) |
|
||||||
|
| Tester-2 (no job) | 1 | none | 2026-10-04 | none |
|
||||||
|
|
||||||
|
- **Answer:** „Yes, the two rules" (not the tester-1 job). **After** (07:49 UTC, `P1/ep0-settings-after.txt`): both jobs
|
||||||
|
`keep-weekly 2`, schedule 03:30 unchanged, nothing run by hand. Check for 2026-10-20: the 03:30 `prunejob` tasks OK with
|
||||||
|
`keep-weekly 2` and two snapshots per namespace (the next step of R-1162).
|
||||||
|
|
||||||
|
## 2. The household passphrase (R-1159)
|
||||||
|
|
||||||
|
Measured with the hub's real router (`P2/passphrase-reach.txt`; a made-up customer, a fake off-site service). A live
|
||||||
|
measurement on Tester4 was not possible: CC holds no Tester4 passphrase, and reading it from the hub database was
|
||||||
|
refused by CC's own safety check (correctly).
|
||||||
|
|
||||||
|
- Passphrase → `GET /config/{id}` 200 with `hub.api_key` (wrong passphrase 401); `POST /host-enroll` 200 with the
|
||||||
|
EXISTING host's key.
|
||||||
|
- Controller key → `register-key`, `confirm-key` (removes every other line, the box's own key included), `move-aside`,
|
||||||
|
`window-open`, `abandon-request` (the hub deletes a set-aside copy after 7 days unless cancelled), `abandon-cancel`,
|
||||||
|
`abandon-status`: all past auth (wrong key 401 each).
|
||||||
|
- Host key → escrow GET/PUT, recovery-credential PUT, wg, desired-state, jobs, host-report: all past auth.
|
||||||
|
- The passphrase is used only at install time (the installer, the controller's setup wizard).
|
||||||
|
- **Option A (CC's pick):** keys only while the customer has no host record (`/config` leaves out the key, `/host-enroll`
|
||||||
|
refuses once a host exists). Closes the chain at its source; a reinstall needs the operator's delete first — already the
|
||||||
|
rule since R-1154. **Option B:** a second factor on the destructive off-site routes. Every reinstall waits for the
|
||||||
|
operator, and the host key stays reachable by the passphrase, so B alone does not close it. No change in this task.
|
||||||
|
|
||||||
|
## 3. Controller 0.333.0 — R-1155, R-1161, the Install chip
|
||||||
|
|
||||||
|
- **R-1155:** first measured on demo-hp (0.332.0): the household's FileBrowser token opened FileBrowser after the
|
||||||
|
dashboard sign-out (200, cookie and Bearer). Fix: the forwardAuth answer refuses a request with no valid files cookie
|
||||||
|
that carries a non-admin FileBrowser token, everywhere FileBrowser reads one (cookie, `?auth=`, Bearer, Basic
|
||||||
|
password). **The commit's security review found Basic missing** in the first version (`c809dd3`); fixed in `a6fa3c0`
|
||||||
|
before any release. Red-proofs: without the check → 200 after the sign-out; without the Basic branch → 200; with every
|
||||||
|
token refused → admin's own token 302. Live (rc.2 demo-hp, 0.333.0 Tester 1): files cookie 200; token alone 401 in all
|
||||||
|
four forms, before and after the sign-out; FileBrowser's sign-in page 200.
|
||||||
|
- **R-1161:** new key `backups_escrow.ez_a_kod_az_ellenorzes` (hu + en); the old sentence only when the claim answer says
|
||||||
|
no confirm. Fixtures re-captured: `escrow_first`, `escrow_receremony` (reason: R-1151 made the old sentence untrue);
|
||||||
|
all others byte-identical. Live on demo-hp (`P6/06-escrow-page-text.txt`): new text shown, old hidden, switch present,
|
||||||
|
negative control 0.
|
||||||
|
- **The Chrome „Install" chip: NOT-A-FINDING.** FileBrowser 1.5.6 links `/public/static/site.webmanifest` on every page;
|
||||||
|
its only PWA setting is `disablePWAInstall` (already set). Not patched (`P3/install-chip.txt`).
|
||||||
|
|
||||||
|
## 4. Hub 0.160.0
|
||||||
|
|
||||||
|
- **R-1156:** a live re-enroll is a warning mailed to the operator, same instruction; a `--resume` mails too and says it
|
||||||
|
needs nothing. **R-1157:** further refusals in one episode counted (`refusals_in_episode`), alarm at most once a day,
|
||||||
|
total on the cleared record. **R-1158:** the reinstall's off-site re-issue changes nothing a box uses (only the hub's
|
||||||
|
own registrar reads that password; no box has received it since v0.127.0); kept because it repairs a broken stored
|
||||||
|
login; comments and the `offsite_reissued` text corrected (`05` §8). Each red-proved.
|
||||||
|
- **Answer:** „Yes, deploy now". Image `0.160.0` digest `sha256:e9a360bc00d00a15014e272c4d023fef118a8b043956f3fc928dd26c00b34230`
|
||||||
|
(clean clone at `a17db0c3`; the first push hit a DNS timeout, the retry pushed it). Manifest `5d3cb5c1`; only
|
||||||
|
Deployment/hub was OutOfSync and only it was synced; Synced/Healthy; running image digest = built digest; 0 `[ERROR]`
|
||||||
|
lines; floor read back from hub.db + WAL (0.332.0 then, 0.333.0 now); host and controller reports from demo-hp, Tester 1
|
||||||
|
and demo-felhom after the restart.
|
||||||
|
|
||||||
|
## 5. gofmt (R-1160)
|
||||||
|
|
||||||
|
Hub `d5e3c8b0` (22 files) and agent `1edfd8e` (14 files), formatting only, tests green before each. Gates: `hub-gofmt`
|
||||||
|
(`scripts/hub_gofmt_gate.py`, `repo_gates.py`) and `gofmt` (felhom-agent `scripts/gofmt_gate.py`, `agent_gates.py`), fast
|
||||||
|
set, decoys on scratch trees, red-proved. No agent release.
|
||||||
|
|
||||||
|
## 6. Controller release (decision 250)
|
||||||
|
|
||||||
|
Per-box floors 0.333.0 at 09:59:38 UTC → all three on 0.333.0 within about a minute (demo-hp from rc.2 at 10:13). Box
|
||||||
|
walk (`P6/`): 0 unhealthy, 0 allow-list refusals, catalog in sync; https-only 28 OK, 0 failures; „Fájlok ↗" with no second
|
||||||
|
login on demo-hp and Tester 1, sign-out asks again. **Global floor 0.333.0 / MinAgent 0.158.1 at 10:16:22 UTC**
|
||||||
|
(303 `floor_set`, read back with the WAL); per-box floors cleared.
|
||||||
|
|
||||||
|
## 7. Versions, commits, fixtures, rows
|
||||||
|
|
||||||
|
- **Controller 0.333.0** (`c809dd3`, `a6fa3c0`, REPORT `e1a2abf`), image digest
|
||||||
|
`sha256:f150a8d8cd329f5cde9e34249e34edc59b6b5db29db1e3200900f3869a0de11b`; rc.1/rc.2 on demo-hp only. Fixtures:
|
||||||
|
`escrow_first`, `escrow_receremony`.
|
||||||
|
- **Hub 0.160.0** (`d5e3c8b0`, `503c93fa`, `a17db0c3`, manifest `5d3cb5c1`).
|
||||||
|
- **Agent:** no release (`1edfd8e`, `0cbd500`). 0.160.0 stays the vouched one.
|
||||||
|
- **Rows: 167 before, 163 after; closed 6** (R-1155, R-1156, R-1157, R-1158, R-1160, R-1161), **opened 2** (R-1165,
|
||||||
|
R-1166); R-1159 → READY (D19), R-1162 → NARROWED (one run to read).
|
||||||
|
- **System poster:** text only — `felhom-system-poster.facts.md` and the matching text in `felhom-system-poster.html`
|
||||||
|
(„last 2" → „one per week, 2 weeks").
|
||||||
|
- **Teardown:** machine — rc images only on demo-hp 9201, now on the release; no test app. Host — none. Hub — no test
|
||||||
|
customer; the per-box floors cleared. ep0 — the two prune jobs changed on the operator's yes, nothing else.
|
||||||
|
|
||||||
|
## Observations
|
||||||
|
|
||||||
|
1. Two household namespaces on ep0 have no prune job; nothing creates one for a new household. FILED: R-1165.
|
||||||
|
2. Another device takes demo-hp's address 192.168.0.104 every few minutes; demo-hp also lost its link to ep0 for 8 minutes. FILED: R-1166.
|
||||||
|
3. A config download with the passphrase is a log line only, and the host key it yields can rewrite the box's escrow and recovery credential. FILED: R-1159.
|
||||||
|
4. The first R-1155 version missed FileBrowser's Basic-auth token source (the commit's security review). NOT-A-FINDING: fixed before any release in `a6fa3c0` and red-proved (the size rule).
|
||||||
|
5. Chrome offers „Install" for the file manager. NOT-A-FINDING: FileBrowser 1.5.6 has no setting that stops its manifest; the brief forbids patching it.
|
||||||
|
6. One of CC's probe scripts on DooPlex passed the household's FileBrowser token to curl on its command line (visible to local processes for about a second; never printed or saved). NOT-A-FINDING: CC's own tooling on DooPlex, not the product; every probe now passes tokens through 0600 header files, and the token expired two hours later.
|
||||||
|
7. Two copies of CC's hub-database snapshot ran into one folder at once and one read failed („malformed"). NOT-A-FINDING: CC's helper, a local copy only; the live hub and its database were untouched, and the next read was clean.
|
||||||
|
8. The first hub image push failed on a DNS timeout at DooPlex. NOT-A-FINDING: a short DNS outage (the website session saw it too); the build's own check found the tag absent and the retry pushed it.
|
||||||
|
9. FileBrowser's `admin` password login was not walked live. NOT-A-FINDING: CC holds no FileBrowser admin password; admin's token passing is unit-pinned and the sign-in page answers 200 live.
|
||||||
@@ -0,0 +1,104 @@
|
|||||||
|
# REPORT — third day brief, 2026-10-19 (the passphrase locks once the box uses its own key; every ep0 household gets a prune job; demo-hp's address)
|
||||||
|
|
||||||
|
## In plain words
|
||||||
|
|
||||||
|
- **The household passphrase no longer gives a live box's keys** (your D19). Each box now fetches its config with its own
|
||||||
|
key. The passphrase still works for a new install. A reinstall over a live box needs you to delete the old host first.
|
||||||
|
- **The brief's first shape could not be built as written.** "Keys only before a box exists" would have broken every
|
||||||
|
install and every live box's config updates: both use the passphrase after the box record exists. CC stopped and asked;
|
||||||
|
you chose "box key first, then lock".
|
||||||
|
- **ep0:** every new household gets its weekly clean-up rule automatically, and the hub alarms on a household without
|
||||||
|
one. tester-1 and Tester-2 got theirs (your yes).
|
||||||
|
- **demo-hp's address** stayed demo-hp's for a 30-minute watch after your router change.
|
||||||
|
- **Live now:** hub 0.161.0, controller 0.334.0 (the floor), tenant-sync v1.3.0 on ep0.
|
||||||
|
|
||||||
|
## 1. The passphrase (R-1159; `05` §8 [DESIGN R-1159])
|
||||||
|
|
||||||
|
- **Install-order proof** (read before building, the brief's STOP): the passphrase is used AFTER `POST /host-enroll`:
|
||||||
|
- `scripts/felhom-host-install.sh:2631` → `:480`, and `:3525`: `step_agent_install` and `step_golden` fetch the
|
||||||
|
download login from `GET /config`;
|
||||||
|
- `felhom-agent/cmd/felhom-agent/main.go:2835`: the provision bakes the passphrase into the guest's `bootstrap.json`;
|
||||||
|
- `felhom-controller/controller/internal/bootstrap/bootstrap.go:142`: the guest's first boot pulls `GET /config` with it
|
||||||
|
(that is how the box gets its own key);
|
||||||
|
- `cmd/controller/main.go:1176` (`report.ConfigRefresher`): every live box re-pulled with it on each config change;
|
||||||
|
- `:2317`: the preflight checks it with `/config` on every run.
|
||||||
|
- **Your answer:** „Box key first, then lock".
|
||||||
|
- **Built:**
|
||||||
|
- Hub 0.161.0: `GET /box-config/{id}` (the controller's own key) and `passphraseKeysClosed`. Passphrase keys are
|
||||||
|
refused (409, no key, operator event `passphrase_keys_refused`) once a ≥ 0.334.0 controller has reported since the
|
||||||
|
host record was made. `/config`, `/recovery` and the `/host-enroll` reuse path are covered; `/artifacts` has no key
|
||||||
|
and is unchanged.
|
||||||
|
- Controller 0.334.0: the refresh pulls with its own key (`report.PullConfigPreferKey`). The passphrase is used only
|
||||||
|
against an older hub or before the box has a key, and never as a retry after a refused key.
|
||||||
|
- **Tests:**
|
||||||
|
- Hub: `TestR1159_PassphraseGivesKeysOnlyUntilTheBoxUsesItsOwn` (5 cases), `TestR1159_BoxPullsItsConfigWithItsOwnKey`,
|
||||||
|
`TestR1159_BoxConfigMinControllerIsTheReleaseThatPullsWithItsKey`.
|
||||||
|
- Controller: `TestR1159_RefreshUsesTheBoxKey`, `TestR1159_MainWiresTheKeyFirstRefresh`.
|
||||||
|
- Red-proofs: lock off → all three keys served; the old pull → 0 key pulls.
|
||||||
|
- **Live on demo-hp** (`audits/day-2026-10-19c/P4/`; the probe ran on the host from the box's own `bootstrap.json`,
|
||||||
|
printing status codes only):
|
||||||
|
- on 0.333.0, the passphrase → 200;
|
||||||
|
- after its first 0.334.0 report (14:30:13 UTC), the passphrase → **409, no key**, and `passphrase_keys_refused` was
|
||||||
|
raised (the operator got the mail — CC's probe);
|
||||||
|
- its own key → 200; a wrong key → 401.
|
||||||
|
|
||||||
|
## 2. ep0 prune jobs (R-1165; `07` Plane-2 row)
|
||||||
|
|
||||||
|
- **Automatic:** `felhom-tenantsync` v1.3.0 creates `prune-<id>` (keep-weekly 2, 03:30, max-depth 0) with a household's
|
||||||
|
namespace unless a job already names it; it removes the job with the household. Its `usage` answer lists namespaces no
|
||||||
|
job covers.
|
||||||
|
- The hub alarms on that list (`pbsdr_namespace_without_prune_job`). An unread list never reads as "all covered".
|
||||||
|
- Tests: 12 script cases (`scripts/test_tenantsync_prune.py`, run on every push), the checker and the client; each
|
||||||
|
red-proved.
|
||||||
|
- **Dry run before the question** (PBS's own, `P2/ep0-dry-run-tester.txt`; 32 snapshots before and after):
|
||||||
|
- tester-1: keep 2026-10-11, 2026-10-18; remove 2026-10-04.
|
||||||
|
- Tester-2: keep 2026-10-04.
|
||||||
|
- **Answer:** „Yes, add both". **After** (13:35 UTC, `P2/ep0-jobs-after.txt`): `prune-tester-1` and `prune-Tester-2`
|
||||||
|
(keep-weekly 2, 03:30) beside the demo-hp, demo-felhom and operator jobs.
|
||||||
|
- **Script on ep0** (14:07 UTC, the same yes): the installed v1.2.0 matched the repo byte for byte; v1.3.0 is installed
|
||||||
|
(hash = the repo's `36412c6a…`), the old copy kept as `.bak-1.2.0`. Live `usage` → `unpruned: []`; `fingerprint` still
|
||||||
|
answers.
|
||||||
|
|
||||||
|
## 3. demo-hp's address (R-1166)
|
||||||
|
|
||||||
|
- A watch every 10 s for 30 minutes (10:59–11:29 UTC): **180 samples, 0 with another MAC**, SSH open throughout.
|
||||||
|
- The host's own MAC (`cat /sys/class/net/vmbr0/address`): **`7c:d3:0a:77:d9:76`** — the same as your router entry.
|
||||||
|
- Note: your DHCP pool is .120–.160, so .104 was never in it. The other device (`b8:06:0d:60:45:58`) most likely had .104
|
||||||
|
set by hand. If it comes back, look there.
|
||||||
|
|
||||||
|
## 4. Hub deploy and the controller release
|
||||||
|
|
||||||
|
- **Hub 0.161.0**
|
||||||
|
- Image digest: `sha256:d56c96a59c86ce2d006938d0980803a12949bcbf027cea2834cabd64118838b8` (clean clone at `b6a911c3`).
|
||||||
|
- Manifest `97f27a75`; only Deployment/hub was synced. Result: Synced/Healthy; the running digest is the built one;
|
||||||
|
0 `[ERROR]` lines; the ep0 read at 14:13 UTC was OK, with no alarm (every namespace has a job).
|
||||||
|
- Checks: the floor was read back; host and controller reports from demo-hp, Tester 1 and demo-felhom arrived after
|
||||||
|
the restart.
|
||||||
|
- **Controller 0.334.0**
|
||||||
|
- Image digest: `sha256:deeabe59d932cd60f8939a37bccf94f145555101fc7be64b6c5cc15f3557031a` (from `546eef3`); rc.1 ran
|
||||||
|
on demo-hp only.
|
||||||
|
- Per-box floors at 14:16:11 UTC. Tester 1 and demo-felhom moved at once; demo-hp moved at 14:30 (from rc.1).
|
||||||
|
- Box walk (`P5/`): 0 unhealthy, 0 refusals, catalog in sync; https-only 28 OK, 0 failures; „Fájlok ↗" with no second
|
||||||
|
login on demo-hp and Tester 1.
|
||||||
|
- **Global floor 0.334.0 / MinAgent 0.158.1 at 14:31:44 UTC.** The per-box floors were cleared.
|
||||||
|
|
||||||
|
## 5. Versions, commits, rows
|
||||||
|
|
||||||
|
- **Hub 0.161.0** (`b6a911c3`, manifest `97f27a75`); **controller 0.334.0** (`546eef3`); **tenantsync v1.3.0** (in
|
||||||
|
`b6a911c3`, on ep0). No agent change. No i18n fixture changed (no page changed).
|
||||||
|
- **Rows: 163 before, 161 after; closed 3** (R-1159, R-1165, R-1166), **opened 1** (R-1167).
|
||||||
|
- **System poster:** facts file only (`felhom-system-poster.facts.md`: new households get their prune job); the drawing's
|
||||||
|
text („one per week, 2 weeks") is still true.
|
||||||
|
- **Teardown:**
|
||||||
|
- Machine: rc.1 only on demo-hp 9201, now on the release; the probe scripts removed from demo-hp.
|
||||||
|
- Host: none.
|
||||||
|
- Hub: per-box floors cleared; one `passphrase_keys_refused` event on demo-hp (CC's probe).
|
||||||
|
- ep0: two jobs added and the script replaced, both on the operator's yes; nothing else.
|
||||||
|
|
||||||
|
## Observations
|
||||||
|
|
||||||
|
1. The installer prints „unexpected hub status 409" instead of the hub's sentence when a re-run meets a live box. FILED: R-1167.
|
||||||
|
2. The brief's „keys only while no host exists" would have broken every install and every live box's config refresh. NOT-A-FINDING: measured before building and put to the operator, who chose the two-step shape (D19); recorded in `05` §8.
|
||||||
|
3. A reinstall over a live box now gets two operator mails (R-1154's reinstall warning and the refusal). NOT-A-FINDING: both are true and give the same instruction; the 1-hour cooldown limits repeats.
|
||||||
|
4. An rc build (`0.334.0-rc.N`) counts as older than 0.334.0, so the lock stays open on an rc. NOT-A-FINDING: by design (an rc runs on demo-hp only, and the floor moves it to the release); proven on rc.1 (200) and the release (409).
|
||||||
|
5. demo-hp's .104 lies outside the router's DHCP pool, yet another MAC held it this morning. NOT-A-FINDING: the router reservation fixed it (30 minutes clean); the other device's source is noted in R-1166's closing text in case it returns.
|
||||||
@@ -0,0 +1,153 @@
|
|||||||
|
# REPORT — evening → night 2026-10-18
|
||||||
|
|
||||||
|
## How I read the brief
|
||||||
|
|
||||||
|
- **Part 5, R-1135:** the brief says the mail should use the public Gitea URL — the register row says the same, and that
|
||||||
|
is what was done (the CI helper first read the brief's wording the other way round and followed the row).
|
||||||
|
- **R-1052** was already closed (2026-10-15); Part 1 proves R-1148 and re-proves R-1052's path with tonight's golden.
|
||||||
|
- **Part 6, Immich:** 9202's Immich lives on `/mnt/sizing-bench`, a hand-made bench folder outside `/mnt/felhom-drives/`,
|
||||||
|
so the allow-list refuses its definition (see Observations); the restore half of the measurement ran on Nextcloud.
|
||||||
|
- **The operator kept Tester4** (asked at 21:20): not deleted; its old host must go before its next install (R-1154).
|
||||||
|
|
||||||
|
## In plain words
|
||||||
|
|
||||||
|
- **The morning list:** vouch golden 0.331.0; say yes or no to the agent release with three fixes; two decisions (one
|
||||||
|
sign-in, local backup retention).
|
||||||
|
- **A brand-new box from nothing works end to end** with installer 1.34.0: emergency password saved in the hub first,
|
||||||
|
then set, then the web login took it; the approved golden and agent; the box moved itself to the floor.
|
||||||
|
- **OnlyOffice now really works** from FileBrowser and from Nextcloud. Before tonight FileBrowser could not open a
|
||||||
|
document and Nextcloud lost every save while saying „saved". Fixed in controller 0.331.0, now the floor for every box.
|
||||||
|
- **One sign-in for the file manager can be built safely**, but only with two safeguards — measured, design note written.
|
||||||
|
- **Three agent fixes are written and tested, not released.**
|
||||||
|
- **CI:** the failure mail's link opens from outside; the 5-minute CI timeout never worked (filed); a rerun hides the
|
||||||
|
first try's log, and the check now looks for it.
|
||||||
|
- **Redis for Immich and Nextcloud has no disk volume any more**, after a test; the test found that removing an app left
|
||||||
|
an old volume behind — fixed too.
|
||||||
|
- **Golden 0.331.0 is baked** (not vouched).
|
||||||
|
- **The old-restic check:** §9.
|
||||||
|
|
||||||
|
## 1. The morning list
|
||||||
|
|
||||||
|
1. **Vouch golden 0.331.0** — sha256 `3881de62b0986ccbe5e55580e3f1bb9765d65b8d898f67c6339bac105244f84f` (registry round
|
||||||
|
trip = the bake), agent 0.159.2 unchanged, **Min agent 0.158.1**.
|
||||||
|
2. **Agent release (R-978, R-973, R-1093) — may I?** If yes: release, prove on demo-hp, ask for the vouch.
|
||||||
|
3. **Decide D16** (R-1100: build one sign-in with the two safeguards?) and **D17** (R-1149: local whole-box backups
|
||||||
|
„last 3" or „one per day, 3 days"?). STATUS has both.
|
||||||
|
4. Before Tester4's next install: delete its old host `Tester4-31fa0e` (Hosts page).
|
||||||
|
|
||||||
|
## 2. Fresh install from nothing (R-1148, R-1052) — `audits/newbox-2026-10-18/`
|
||||||
|
|
||||||
|
| UTC | Step | Result |
|
||||||
|
|---|---|---|
|
||||||
|
| 17:42:21 | Tester4 created by the operator | bind mail sent |
|
||||||
|
| 17:43:15 | VM 9302 `newbox-r1148` on demo-hp, ISO 1.29.0 (sha = published) | 8 GB / 4 cores / 120 GB on `/mnt/hdd_1` root |
|
||||||
|
| 17:50:19 | Install pressed (graphical, Hungarian keyboard kept) | installed, rebooted, pairing code on the console |
|
||||||
|
| 18:02:58 | bind: mailed link + pairing code + owner phrase (operator's 0600 file, shredded) | „Sikeres összekötés" |
|
||||||
|
| 18:03:43 | step 4b: vault → chpasswd → PVE login | `[OK] … vaulted, set, and the PVE login takes it`; hub `set_at` same second |
|
||||||
|
| 18:03:44 | step 5: the hub manifest | agent **0.159.2**, golden **0.329.0** — the vouched pair, both sha-verified |
|
||||||
|
| 18:05:37 → 18:05:51 | first controller reports | 0.329.0, then **0.330.0** (`initiated_by: auto-floor` at 18:05:38) — 2 min 47 s after the bind |
|
||||||
|
| 18:17:14 | CC's PVE login from DooPlex | vaulted value **200**, installer value **401** |
|
||||||
|
| 18:18:32 → 18:20:05 | claim (setup code mail), BentoPDF | claimed; BentoPDF healthy |
|
||||||
|
| 18:24:15 | household „turn off" | hub `deliberate_off_at` set; VM stopped |
|
||||||
|
| — | teardown | name read twice, VM 9302 destroyed, disk gone; other VMs untouched; Tester4 kept (operator) |
|
||||||
|
|
||||||
|
## 3. OnlyOffice (R-1018) — `audits/onlyoffice-2026-10-18/`
|
||||||
|
|
||||||
|
On 0.330.0: FileBrowser → the editor loaded, the document never did (OnlyOffice fetched `https://files.<domain>` → 403);
|
||||||
|
Nextcloud → edit and „All changes saved", but the file kept the old text (`http://onlyoffice:80cache/…`). Switching the
|
||||||
|
link on/off left files.<domain> 502 for 4 min 8 s / 3 min 38 s. **Fixed (controller 0.331.0):** FileBrowser
|
||||||
|
`server.internalUrl`, Nextcloud's slash, the gate after a FileBrowser restart. On rc.1: both open, edit, save, read back
|
||||||
|
(„… RC1-FB-OK", „… RC1-NC-OK"); the stop revokes both (no editor; Nextcloud 404); the stop's outage one 2-s sample.
|
||||||
|
|
||||||
|
## 4. One sign-in spike (R-1100 → READY) — `audits/proxyauth-spike-2026-10-18/DESIGN-NOTE.md`
|
||||||
|
|
||||||
|
Traefik forwardAuth strips a client's header (any case, twice) and its own value wins — measured. But an app container
|
||||||
|
reaching FileBrowser directly logs in with a forged header (200), and the dashboard cookie never reaches `files.`. Verdict:
|
||||||
|
build only with a files-only ticket cookie and a per-box secret header name. Everything on 9202 removed (users, routes,
|
||||||
|
stand-in container, config).
|
||||||
|
|
||||||
|
## 5. Agent fixes (committed, NOT released) — felhom-agent `REPORT.md`
|
||||||
|
|
||||||
|
R-1093 `f7b7308` + `0963bae` (and a fail-open hole closed after a background review), R-978 `8561dc8`, R-973 `f17e197`;
|
||||||
|
tests and red-proofs for each; CI job 2062 success. R-1093 live: a loop disk on demo-hp showed the probe's condition holds.
|
||||||
|
|
||||||
|
## 6. CI (R-1135, R-1136, R-1137 closed)
|
||||||
|
|
||||||
|
Link public and by run id (forced mail 18:46:01Z, job 2067, one mail). The runner's real limit is 3 h; the job-level
|
||||||
|
5-minute timeout is not enforced (R-265 corrected; R-1152 for the step-level one). A rerun replaces the job's attempt; the
|
||||||
|
pull-check recipe in `CLAUDE.md` now reads `actions/tasks` (instruction-file edit by the CI helper: felhom.eu `CLAUDE.md`
|
||||||
|
CI recipe, item 3 added — a proven fact, stricter; `b9b56c47`).
|
||||||
|
|
||||||
|
## 7. Redis volumes (R-1117 closed) — `audits/redis-volumes-2026-10-18/`
|
||||||
|
|
||||||
|
(a) Both apps healthy on the new shape; Immich's metadata job ran (0 → 11, 0 failed); Nextcloud `cron.php` rc 0, WebDAV
|
||||||
|
200. (b) The dump skips the dropped volume (the old tar gone from the unit); a second-drive restore from a copy made WITH
|
||||||
|
the volume: 3/3 volumes, 1/1 database, healthy — it brings back that copy's compose (by design). (c) The old volume stays
|
||||||
|
attached under the tmpfs; **a removal left it behind** — fixed (controller 0.331.0, red-proofed). Templates changed
|
||||||
|
(catalog `30eea97`); Nextcloud on 9202 took it through the sync + restart (tmpfs, `save ""`).
|
||||||
|
|
||||||
|
## 8. Controller release and golden — `audits/evening-2026-10-18/P7/`, `tests/golden-0.331.0-2026-10-18/`
|
||||||
|
|
||||||
|
0.331.0 (digest `1b11d25a…`): per-box floors 20:44:00 → all three on 0.331.0 within 20 s; box walk green (0 refusals,
|
||||||
|
0 unhealthy, FileBrowser re-published in 5–6 s, public pages 200 after one 20-s blip each on two names, https-only 28 OK);
|
||||||
|
**global floor 0.331.0 / 0.158.1 at 20:48:56 UTC**, read back with the WAL; per-box floors cleared. Golden 0.331.0 baked
|
||||||
|
after it, round trip = bake sha, token checks 0 with a working control, teardown clean. No controller push after the bake.
|
||||||
|
|
||||||
|
## 9. The old-restic check (Part 8)
|
||||||
|
|
||||||
|
**Done 2026-10-19 by the watcher** (filled in by the day session, `audits/evening-2026-10-18/P8/old-restic-check.txt`):
|
||||||
|
demo-hp's clean-up window 12 ran at 02:17:53–02:18:24 UTC after the night run — „63 of 193 planned snapshot(s) removed,
|
||||||
|
324 -> 261" (the hub's row: `pruned`, 324 → 261). Then `rx.sh` (read-only): the new restic (0.19.1) and the OLD one (image
|
||||||
|
0.322.0, restic 0.14.0) both list **261**; both restore `c0b6299c` (made by 0.19.1, 02:17:48) to the same content hash
|
||||||
|
`d808e6537c7bcdb6` (7 files) and `05c3346a` (2026-08-31) to `8df241f4cf889265` (10 files). The old restic still reads and
|
||||||
|
restores demo-hp's store after its first real 0.19.1 prune. (Pre-check at 20:57 UTC: 310 and 310.)
|
||||||
|
|
||||||
|
## 10. Versions, commits, fixtures, rows
|
||||||
|
|
||||||
|
- Controller **0.331.0** — `772ef36`, `8ae9178` (+ `615940d` CI); rc.1 on 9202 only. Agent `main` `f7b7308` `8561dc8`
|
||||||
|
`0963bae` `f17e197` `79ba884` `1693da4` — unreleased. Catalog `c97eb1c`, `30eea97`. felhom.eu `1fe9f848`, `b9b56c47`,
|
||||||
|
this commit. Golden **0.331.0** baked. Hub unchanged (0.158.0).
|
||||||
|
- Fixtures: none changed — no page changed.
|
||||||
|
- **Register: 168 before, 168 after; closed 6** (R-1148, R-1018, R-1117, R-1135, R-1136, R-1137), **opened 6**
|
||||||
|
(R-1149, R-1150, R-1151, R-1152, R-1153, R-1154). Updated: R-1100 (READY), R-978, R-973, R-1093 (fixed in agent
|
||||||
|
`main`), R-265 (corrected).
|
||||||
|
|
||||||
|
## Observations
|
||||||
|
|
||||||
|
1. Installer password, golden and agent all proven in one fresh install. — closes R-1148 (not a finding).
|
||||||
|
2. FileBrowser's office preview answered 500 with the link on. FILED: R-1153.
|
||||||
|
3. A reinstall reuses an old host and skips two re-issues; now recurring with the kept Tester4. FILED: R-1154.
|
||||||
|
4. The R-973 cap leaves the local keep-last 3 floodable within a day. FILED: R-1149.
|
||||||
|
5. The cap's refusal reaches the operator with a misleading event text. FILED: R-1150.
|
||||||
|
6. R-978's typed-back confirmation is browser-only. FILED: R-1151.
|
||||||
|
7. The CI job timeout is inert. FILED: R-1152.
|
||||||
|
8. 9202's Immich sits on `/mnt/sizing-bench`, outside `/mnt/felhom-drives/`, so the allow-list refuses its definition every
|
||||||
|
15 minutes since 0.328.0 and a restore of it is refused. NOT-A-FINDING: the check works as designed; the folder is a
|
||||||
|
hand-made bench artifact (R-994), not a shape a real box has.
|
||||||
|
9. A restore brings back the backup's own compose (the old Redis volume with it). NOT-A-FINDING: by design (R-47: the
|
||||||
|
unit's compose becomes the live one); the older shape works.
|
||||||
|
10. My commit `7f7081d1` (day session) carries a „Co-Authored-By" trailer, against both briefs' rule 10. NOT-A-FINDING:
|
||||||
|
my slip; it is pushed and cannot be rewritten without a force-push (forbidden); no later commit has one.
|
||||||
|
11. I started the catalog's FULL gate run on DooPlex, whose third gate starts containers. NOT-A-FINDING: stopped at gate 2
|
||||||
|
(a network check), no container was made (checked), the fast run used instead.
|
||||||
|
12. One OnlyOffice „Connection is lost" on the first rc.1 attempt, after two blind clicks. NOT-A-FINDING: not reproduced;
|
||||||
|
the next attempt saved; no container restarted in between.
|
||||||
|
13. Two public names answered 000 once in the box walk. NOT-A-FINDING: 18 of 18 retries answered 200 in < 0.2 s.
|
||||||
|
14. The R-1100 row named the config key `proxyAuth`; it is `auth.methods.proxy`. NOT-A-FINDING: corrected in the row.
|
||||||
|
|
||||||
|
## Teardown — three layers
|
||||||
|
|
||||||
|
- **Machines:** VM 9302 destroyed (its disk gone). 9202: OnlyOffice and Nextcloud removed through the product (their
|
||||||
|
volumes, data and backup copies), the test documents deleted, the spike's route file, container, FileBrowser users and
|
||||||
|
proxy setting removed, helper files and the Immich scratch password shredded; Immich keeps the new Redis shape by hand
|
||||||
|
(= the template); 9202 on controller 0.331.0. `nextcloud_nextcloud_redis_data` (the orphan the pre-fix
|
||||||
|
removal left) removed by name afterwards; no Nextcloud volume remains. demo-hp host: the loop-disk file removed. Drill VM: CT 9100 purged, VM off,
|
||||||
|
disk `virgin`.
|
||||||
|
- **Hub:** global floor 0.331.0 / 0.158.1, no per-box floor; Tester4 and its host `Tester4-31fa0e` kept (operator); no
|
||||||
|
vouch, no deploy.
|
||||||
|
- **DooPlex:** builds and the golden bake only; the headless browser as a test client; no Docker cleanup.
|
||||||
|
|
||||||
|
Architecture read: `01`, `02`, `03` §8a, `04`, `05`, `07` §6/§8, `09` §3 decisions 162/250; `target-selection.md`,
|
||||||
|
`nightly-rotation.md`, `RUNBOOK-manual-build.md` §4. Documents changed: `03` §8a, `07` §6.1 and §8 row 10,
|
||||||
|
`target-selection.md` (Tester4). System poster: none of the three (no listed fact changed). `unproven.py --summary`:
|
||||||
|
not walked 35 of 55 — unchanged.
|
||||||
@@ -0,0 +1,164 @@
|
|||||||
|
# REPORT — evening/night brief 2026-10-19 (one stop less per night, no false tunnel error, R-893 measured, hub and agent fixes ready, a scanner on every push, ep0's first weekly clean-up)
|
||||||
|
|
||||||
|
## In plain words
|
||||||
|
|
||||||
|
- **Morning list:** deploy hub 0.162.0? · release agent 0.161.0? · publish installer 1.35.0? · decide D20 (R-893).
|
||||||
|
- **One app stop less per night** (R-921): the controller (0.335.0, the floor) now asks the agent „are you busy?" before
|
||||||
|
it stops the apps. The agent's answer is written but not released, so it starts working with the agent release.
|
||||||
|
- **No false error for a box without a tunnel** (R-1070): the real cause was Docker 29 writing its „not found" message
|
||||||
|
in small letters. Fixed in the agent (not released).
|
||||||
|
- **A failed off-site restore, measured** (R-893): the box puts the newer rows over the backup's older volume; a table
|
||||||
|
only the backup had stays. The app is held stopped. Two options written; pick A (save the app's volumes first).
|
||||||
|
- **Hub fixes built, not installed:** no repeat mail after a restart; the agent list offers only real agent packages.
|
||||||
|
The ArgoCD danger to three hub secrets is already gone (ArgoCD no longer tracks them).
|
||||||
|
- **Installer message written, not published:** a re-run on a live box says „delete the old host first", EN and HU.
|
||||||
|
- **A scanner on every push, in all four repos** (warn-first). First scan: no real secret, no called weakness.
|
||||||
|
- **The scratch box cannot get its agent key without an agent change** (R-1083): the exact gap is written down.
|
||||||
|
- **ep0's first weekly clean-up ran as predicted:** only tester-1's 4 October copy went; every household has a rule.
|
||||||
|
R-1162 closed.
|
||||||
|
|
||||||
|
Decisions CC took: none beyond the brief. The operator answered one question (R-921, 19:2x): „add one field".
|
||||||
|
|
||||||
|
## Part 1 — R-921 (`07` §6.4; `03` local API) — DONE (live once the agent ships)
|
||||||
|
|
||||||
|
- No existing agent answer served the host-wide gate (read: `/backup/due`, `/backup/status`, the 409 shape). Operator:
|
||||||
|
add a read-only field to the existing route, no new route.
|
||||||
|
- Agent `56aabc7`: `host_busy` `{busy, what}` on `GET /backup/status`. Controller 0.335.0: `precheckSkips` → held →
|
||||||
|
no stop, contention for each due tier (15-minute skip, one „BLOCKED" alarm after 3 h); free / older agent / failed
|
||||||
|
read → the old order; the window's own tier in flight → attach as before.
|
||||||
|
- **The race** (free at the read, busy at the start): one short stop, resumed at once — today's behaviour, pinned.
|
||||||
|
- Tests, red-proved: agent `TestBackupStatus_R921_HostBusy` (assignment removed → `free gate: host_busy = <nil>`);
|
||||||
|
controller `TestR921_NoStopWhileTheHostIsBusy` (branch removed → 3 apps stopped, the start refused),
|
||||||
|
`…SkipsStillAlarmAfterTheCeiling`, `…OtherAnswersKeepTheOldOrder` (4 rows), `TestHostBusy_ReadsTheAgentsGate`.
|
||||||
|
- Live: demo-hp's agent answers no `host_busy` (read inside 9201) — the old order, as designed. A live busy state was
|
||||||
|
not made (it needs the new agent on the box). R-921 closed per the brief.
|
||||||
|
|
||||||
|
## Part 2 — R-1070 (`11` §8.2) — DONE in the agent (not released)
|
||||||
|
|
||||||
|
- Where: `felhom-agent/internal/osupdate/leg.go` `HostHealthVerdict` (the tunnel check) and
|
||||||
|
`internal/hub/cloudflared.go` `ClassifyTunnel`. Measured on demo-hp: `docker inspect` of a missing name prints
|
||||||
|
`error: no such object: …` (Docker 29); the match wanted `No such object` → `unknown` → the host rule failed.
|
||||||
|
- Fix: case-insensitive match → new state `absent`; the host rule passes `absent` only when the reading from before
|
||||||
|
the step (Docker answered) also had no cloudflared; `not_running` and `unknown` still fail. Hub 0.162.0 shows
|
||||||
|
`absent` as „none set up". `11` §8.2 had said `unknown` does not fail — corrected (the code always failed it).
|
||||||
|
- Tests, red-proved: `TestHostHealthVerdict_R1070_NoTunnelSetUp` (7 rows), three `TestClassifyTunnel` rows,
|
||||||
|
`TestSystemRows_R1070_AbsentTunnelIsNotAWarning`.
|
||||||
|
|
||||||
|
## Part 3 — R-893: measured and designed, no restore code changed
|
||||||
|
|
||||||
|
Helper on 9202 (fences carried in full). Evidence and the note: `documentation/audits/night-2026-10-19/P3/`.
|
||||||
|
|
||||||
|
- Rallly 4.15.3 + postgres 18; a scratch SFTP off-site on 9202's own disk; one failing SQL line in the prepared copy
|
||||||
|
of the dump, so the replay failed AFTER the volume was replaced.
|
||||||
|
- **What it left:** the newest rows (the undo dump) over the backup's volume; a table only the backup had stayed
|
||||||
|
(37 tables against 36); definition unchanged (same version); app held (`restore_mixed`), a start → 409.
|
||||||
|
- The restore page's sentence is false on both halves for this app → R-1169.
|
||||||
|
- **Options:** A — copy the app's live volumes before the replay, put them back on failure, start the app (one copy's
|
||||||
|
disk for minutes; Rallly 69 MB in ~1 s; a fit check before the stop). B — empty the DB volume before the undo load
|
||||||
|
(no disk; a bad undo copy leaves an empty database; usually still a support call). **Pick: A** (the operator's D8
|
||||||
|
direction). Not measured: the version-change branch (needs a drill-catalog commit).
|
||||||
|
- Teardown: 9202 back on 0.331.0 as found, every throwaway removed (checked again by CC); host: `/tmp` only; hub: nothing.
|
||||||
|
|
||||||
|
## Part 4 — hub 0.162.0, built, NOT deployed (`05`)
|
||||||
|
|
||||||
|
- R-1001: start-up seeding from the DB for `host_stale`/`host_down`; `restore_test_stale` edge from the DB. Tests
|
||||||
|
red-proved (each restart = a new checker over the same DB).
|
||||||
|
- R-1011: exact-file sha (`ErrNoFile`); step bundles leave the list; a vouch of one → `artifact_file_missing`. Registry
|
||||||
|
read: 6 step-bundle versions, every release holds `felhom-agent`. Tests red-proved.
|
||||||
|
- R-953: **no change needed** — read back 18:21 UTC: ArgoCD tracks none of the three Secrets (no tracking mark), so the
|
||||||
|
next deploy does nothing with them. `P4/r953-readback.txt`.
|
||||||
|
- Image `sha256:69285e640a9c5e4ce59b1fbb5cd94623fa47a71798e9d86fa0767ee6c69a6ac8` from a clean clone at `d92a712e`.
|
||||||
|
|
||||||
|
## Part 5 — agent, committed, NOT released (`03`)
|
||||||
|
|
||||||
|
R-1010 (`56aabc7`): on „Failed to find logical volume …_vzdump" one retry with the API's `force=1` (`pvesh usage`
|
||||||
|
lists `--force`; the same token privilege; no sudoers line, no root command); never a second force. Red-proved. Plus
|
||||||
|
R-921's field and R-1070. CHANGELOG „Unreleased". Not exercised live (it needs a half-removed snapshot on `pve/data`).
|
||||||
|
|
||||||
|
## Part 6 — installer 1.35.0, written, NOT published
|
||||||
|
|
||||||
|
R-1167: `die_live_box` in EN and HU at the preflight, host-enroll and the controller.yaml fetch. Harness: a static case
|
||||||
|
and two behaviour cases (the real functions against a local fake hub), red-proved on a mutated copy. **CI red twice**
|
||||||
|
(2106, 2108): the CI runner has no curl; the behaviour cases now SKIP out loud there (`5d0dfeac`).
|
||||||
|
|
||||||
|
## Part 7 — the scanner (R-986), warn-first — DONE
|
||||||
|
|
||||||
|
- Shared `scripts/security_scan_gate.py` in all four runners' fast sets (hook + CI); `.security-accepted` per repo;
|
||||||
|
8 decoys in scratch repos, red-proved; CI fetches the pinned gitleaks (both sha256 checked — seen working in CI 2108).
|
||||||
|
- govulncheck pin v1.1.4 → **v1.8.0** (v1.1.4 cannot read the Go 1.27 code). CI has no Go: govulncheck NOT CHECKED
|
||||||
|
there, out loud; the hook runs it.
|
||||||
|
- **First scan of today's `main`** (`P7/triage.md`): govulncheck 0 called in all three modules. gitleaks history:
|
||||||
|
controller 12, agent 15, catalog 1 (all as on 2026-10-12), felhom.eu 50 (+7: the usenet bench 5, a boot journal 1,
|
||||||
|
a test 1). Today's tree: controller 4, felhom.eu 42, agent 15, catalog 0 — every one triaged: **0 real**; accepted =
|
||||||
|
throwaway demo/bench app values (the bench LXC 9401 stopped, the apps gone from 9202); false = test constants, public
|
||||||
|
keys, an SSH fingerprint, a tunnel ID, hashes.
|
||||||
|
|
||||||
|
## Part 8 — R-1083: the exact gap (no agent change)
|
||||||
|
|
||||||
|
The agent mints a per-guest token only inside `--selftest=provision`, for a guest it brings up from a golden in the same
|
||||||
|
call. No product way exists for an existing guest. Row stays OPEN with the gap and two options.
|
||||||
|
|
||||||
|
## Part 9 — controller 0.335.0, the floor
|
||||||
|
|
||||||
|
rc.1 on demo-hp 18:08 UTC (0 errors) → `0.335.0` `sha256:262831f9…` → per-box floors 18:22 → all three moved by
|
||||||
|
18:37 → box walk green (0 unhealthy, 0 refusals, catalog in sync; https-only 28 OK, 0 FAIL; „Fájlok ↗" one sign-in on
|
||||||
|
demo-hp and Tester 1) → **global floor 0.335.0 / MinAgent 0.158.1 at 18:39:26 UTC**, read back; per-box floors cleared.
|
||||||
|
i18n: no page changed, no fixture changed. Evidence `P9/`.
|
||||||
|
|
||||||
|
## Part 10 — ep0's first weekly clean-up (`07` §plane-2 retention) — DONE, R-1162 closed
|
||||||
|
|
||||||
|
Read only on ep0, 03:45–03:47 UTC. Evidence `P10/01-before.txt` (19:49 UTC), `P10/02-after.txt`.
|
||||||
|
|
||||||
|
| Household | Task 03:30 UTC | Removed | Copies after |
|
||||||
|
|---|---|---|---|
|
||||||
|
| demo-hp | OK, keep-weekly 2 | nothing | 2 (10-08, 10-15) |
|
||||||
|
| demo-felhom | OK, keep-weekly 2 | nothing | 2 (10-06, 10-13) |
|
||||||
|
| Tester-2 | OK, keep-weekly 2 | nothing | 1 (10-04) |
|
||||||
|
| tester-1 | OK, keep-weekly 2 | 2026-10-04 | 2 (10-11, 10-18) |
|
||||||
|
|
||||||
|
- Exactly the dry runs' prediction. The operator job (03:45, keep-daily 14 + keep-weekly 8) removed one hub copy
|
||||||
|
(10-06) — its own rule.
|
||||||
|
- **No „household without a rule" alarm.** Positive check from a second channel: ep0's read-only `usage` op, run
|
||||||
|
directly, answers `unpruned_checked: true, unpruned: []` (tenantsync v1.3.0, sha256 same as the repo file). The hub
|
||||||
|
(0.161.0) logs its ep0 read every 15 minutes (03:31 UTC the last before my read) and no „no prune job" line since its
|
||||||
|
start (2026-10-19 14:10 UTC). The alarm is operator-channel only, so it is not in the events table (none there).
|
||||||
|
|
||||||
|
## Versions, commits
|
||||||
|
|
||||||
|
- Controller 0.335.0 (`15ea002`, REPORT `c4b3b0a`, scanner `a03c25a`). Hub 0.162.0 built (`5f31da04`). Agent
|
||||||
|
`56aabc7`, `84ef581` (not released). Installer 1.35.0 (`d92a712e`, `5d0dfeac`, not published). felhom.eu scanner
|
||||||
|
`15f74288`, docs `f92fa5a8`. Catalog `87f5588`, `57ddb20`.
|
||||||
|
- **System poster:** no listed fact changed tonight — none of the three.
|
||||||
|
- **Instruction/doc corrections:** `11-os-updates.md` §8.2 (the `unknown` tunnel sentence was false); `nodes.md` and
|
||||||
|
`nightly-rotation.md` (9202 is DHCP, .121 now).
|
||||||
|
|
||||||
|
## Rows
|
||||||
|
|
||||||
|
Register **161 before, 160 after.** **Closed 4:** R-921, R-953, R-986, R-1162. **Opened 3:** R-1168 (warn →
|
||||||
|
fail), R-1169 (the false restore sentence), R-1170 (a vanished tunnel raises no alarm). Re-stated: R-1001, R-1011,
|
||||||
|
R-1010, R-1070, R-1167 (READY, waiting on the release/deploy/publish), R-893 (READY, D20), R-1083 (the gap).
|
||||||
|
**Dates moved** (the due-checks gate fired on the new date): R-895 and R-435, 2026-10-20 → 2026-10-21. Both are reads
|
||||||
|
this brief did not name; R-895's first verdict lands at 07:10 hub time, after this session. The reason is in each row.
|
||||||
|
|
||||||
|
## Observations
|
||||||
|
|
||||||
|
1. The restore page's held-mixed sentence is false for the measured app. FILED: R-1169
|
||||||
|
2. A tunnel container that disappears on a box with a tunnel configured raises no alarm. FILED: R-1170
|
||||||
|
3. The scanner warns only. FILED: R-1168
|
||||||
|
4. `11` §8.2 said an `unknown` tunnel does not fail the host rule; the code always failed it. NOT-A-FINDING: corrected in the session (the size rule); it is R-1070's cause.
|
||||||
|
5. govulncheck v1.1.4 (the 2026-10-12 pin) cannot read Go 1.27 code. NOT-A-FINDING: the pin moved to v1.8.0 tonight, recorded in `SCANNERS.md` §5.
|
||||||
|
6. The CI runner has no curl; two new harness cases failed CI 2106 and 2108. NOT-A-FINDING: fixed the same night (`5d0dfeac`), they skip out loud there.
|
||||||
|
7. R-953's tracking marks went after 2026-10-12 with no record of who removed them. NOT-A-FINDING: the danger the row named is gone, nothing to fix.
|
||||||
|
8. 9202's DHCP address moved (.114 → .121) and the walk named the old one. NOT-A-FINDING: fixed (`box_walk.py` reads it; two docs corrected).
|
||||||
|
9. The brief said the installer already prints HU and EN; it printed English only. NOT-A-FINDING: the HU lines are new, named in the scripts CHANGELOG.
|
||||||
|
10. The agent's backup cap (R-973) can also refuse a POST after the stop. NOT-A-FINDING: it answers 409, so the apps resume at once (the BUSY path), and it fires only after a flood of presses that each stopped the apps anyway.
|
||||||
|
11. The manual off-site run dumped and restarted every app, not only the one switched on. NOT-A-FINDING: by design, the coherent app-data pair (`07`).
|
||||||
|
12. On CI's depth-1 clone, gitleaks scans the whole tree (the root commit), ~100 s for felhom.eu. NOT-A-FINDING: a stronger check than the diff, all current hits accepted; recorded in `CONTEXT.md`.
|
||||||
|
13. My 19:49 before-file listed the operator job's keep-weekly but not its keep-daily 14. NOT-A-FINDING: my read printed only some fields; the job is unchanged.
|
||||||
|
|
||||||
|
## Teardown
|
||||||
|
|
||||||
|
- Machine: 9202 back as found (0.331.0, no throwaway left — checked); 9201 on the release 0.335.0; probe scripts removed.
|
||||||
|
- Host: `/tmp` copies only, removed. Nothing on `pve/data`.
|
||||||
|
- Hub: per-box floors cleared; nothing else created.
|
||||||
|
- ep0: read only (task list, task logs, snapshot list, prune-job list, the read-only `usage` op). Nothing changed.
|
||||||
@@ -0,0 +1,87 @@
|
|||||||
|
# REPORT — the website keeps syncing during an internet outage (2026-10-19)
|
||||||
|
|
||||||
|
## In plain words
|
||||||
|
|
||||||
|
1. **It worked.** The website's copy helpers now read Gitea inside the cluster, with no internet hop, and they wait
|
||||||
|
and try again instead of stopping. The new website pod runs with 0 restarts, and the site and the installer
|
||||||
|
script both answer normally.
|
||||||
|
2. **Note for later:** this ties the website to the DooPlex cluster. If it moves (for example to Hetzner), the
|
||||||
|
address must go back to the public one — register row R-1164.
|
||||||
|
|
||||||
|
## Part 1 — anonymous in-cluster fetch (before any commit)
|
||||||
|
|
||||||
|
Throwaway pod in a scratch namespace (`ccscratch-gitsync`, deleted after), no credentials:
|
||||||
|
`git ls-remote http://gitea.gitea-system.svc.cluster.local:3000/admin/felhom.eu.git main installer-v1.34.0`
|
||||||
|
|
||||||
|
| ref | in-cluster | public (anonymous) |
|
||||||
|
|---|---|---|
|
||||||
|
| `refs/heads/main` | `5d3cb5c1…` | `5d3cb5c1…` — equal |
|
||||||
|
| `refs/tags/installer-v1.34.0` | `add58eec…` (annotated tag object) | `add58eec…` — equal; peels to `624bcdc3…` |
|
||||||
|
|
||||||
|
Evidence: `audits/website-gitsync-incluster-2026-10-19/gitsync-part1-*.txt`.
|
||||||
|
|
||||||
|
## Part 2 — the change
|
||||||
|
|
||||||
|
Commit **`fd75003e`** (no Co-Authored-By, per the brief):
|
||||||
|
- `manifests/webpage.yaml`: all four git-syncs `--repo=http://gitea.gitea-system.svc.cluster.local:3000/admin/felhom.eu.git`;
|
||||||
|
the two sidecars `--max-failures=-1` (not the `--one-time` inits); one comment above the website sidecar. Refs and
|
||||||
|
`nginx-config-rev` unchanged.
|
||||||
|
- `scripts/hostinstall_gates.py` gate 6: fails unless there are four `--repo=` values and all are identical. Existing
|
||||||
|
ref checks unchanged.
|
||||||
|
|
||||||
|
Gates: `python3 scripts/repo_gates.py` → `all felhom.eu gates OK`; `python3 scripts/hostinstall_gates.py` → rc 0
|
||||||
|
(`ok: manifest: all four git-syncs read the same repo (…)`).
|
||||||
|
|
||||||
|
## Part 4 — deploy and proof
|
||||||
|
|
||||||
|
**Deploy:** hard refresh showed only `Deployment/felhom-webpage` OutOfSync; synced that resource alone at `fd75003e`
|
||||||
|
(the skill's Deployment-only sync). New pod `felhom-webpage-869ff88bf7-fswbc` ready at 09:42 UTC; the old pod
|
||||||
|
terminated. ArgoCD: Synced, operation Succeeded. (The first `rollout status` returned at once — it ran before the new
|
||||||
|
ReplicaSet existed; re-run after the sync it waited for the old replica and finished.)
|
||||||
|
|
||||||
|
**Scenario A** (`gitsync-scenarioA.txt`): restarts `0,0,0`; both sidecars started with the in-cluster `--repo` and
|
||||||
|
`--max-failures=-1`, then `updated successfully` (`main` → `fd75003e`, `installer-v1.34.0` → `624bcdc3`); both init
|
||||||
|
containers `updated successfully`; `https://felhom.eu/` **200**, `https://felhom.eu/scripts/felhom-host-install.sh`
|
||||||
|
**200**, served `SCRIPT_VERSION="1.34.0"` = manifest `--ref=installer-v1.34.0`.
|
||||||
|
|
||||||
|
**Scenarios B and C** (`gitsync-scenarioBC.txt`): scratch namespace `ccscratch-gitsync-bc` (deleted after),
|
||||||
|
`git-sync:v4.4.0 --repo=http://192.0.2.1/x.git --period=10s`:
|
||||||
|
- **B, with `--max-failures=-1`:** `error syncing repo, will retry` (`failCount:1`), restarts **0**.
|
||||||
|
- **C, without:** `too many failures, aborting`, restarts **1**.
|
||||||
|
- The first read at 95 s showed nothing for either: one fetch to an address that never answers takes **~133 s** to
|
||||||
|
time out. The re-read at 09:48 recorded both results. Instead of 90 s, a later run of this test needs about 150 s.
|
||||||
|
|
||||||
|
**Scenario D — gate red-proof:** one `--repo` changed to the public address → gate 6 `FAIL … the git-sync --repo
|
||||||
|
values differ`, rc 1; restored byte-identical → rc 0.
|
||||||
|
|
||||||
|
## Part 3 — docs changed
|
||||||
|
|
||||||
|
- `documentation/architecture/01-topology-and-trust.md` §7: a `[DESIGN]` paragraph — the website pulls `website/`
|
||||||
|
(main) and `scripts/` (installer tag) from Gitea in-cluster over plain HTTP, sidecars retry forever, tied to the
|
||||||
|
DooPlex cluster.
|
||||||
|
- `.claude/rules/manifests.md` „The two git-syncs": the fact that all four containers read Gitea in-cluster, must name
|
||||||
|
one repo (gate 6), and the sidecars carry `--max-failures=-1`.
|
||||||
|
- `scripts/CHANGELOG.md`: one entry.
|
||||||
|
|
||||||
|
## Register
|
||||||
|
|
||||||
|
Rows before **166**, after **167**. Closed **R-1163** (finding + fix, with evidence). Opened **R-1164** (P4, Hub &
|
||||||
|
operator: move the website to Hetzner; then the git-syncs must use the public address again).
|
||||||
|
|
||||||
|
## Observations
|
||||||
|
|
||||||
|
- The 95-second wait in the brief is too short for an address that never answers (~133 s TCP timeout per fetch) —
|
||||||
|
NOT-A-FINDING: recorded here as the correct wait for this test; nothing in the product depends on it.
|
||||||
|
- The skill says a whole-app sync would push git's view over de-gitted Secrets; today only the webpage Deployment was
|
||||||
|
OutOfSync — NOT-A-FINDING: the Deployment-only sync was used anyway, as the skill says.
|
||||||
|
|
||||||
|
- Around 09:50–09:55 UTC, DooPlex itself briefly could not resolve names (`Temporary failure in name resolution`;
|
||||||
|
`curl https://felhom.eu/` → `000`). By 09:56 DNS, the internet and the site (200) were back. The website pod kept
|
||||||
|
restarts `0,0,0` and synced `e2c35007` in-cluster at 09:54 with no error. NOT-A-FINDING: the same class of short
|
||||||
|
outage that triggered this task; it is not proof that the fix worked (whether the blip reached the cluster is unknown).
|
||||||
|
- CI: `fd75003e` job 2092 success, `e2c35007` job 2094 success.
|
||||||
|
|
||||||
|
## Teardown
|
||||||
|
|
||||||
|
Scratch namespaces `ccscratch-gitsync` and `ccscratch-gitsync-bc` deleted (`kubectl get ns | grep -c ccscratch` → 0).
|
||||||
|
No hub, controller, agent, customer box, demo box or ep0 touched.
|
||||||
@@ -2,7 +2,7 @@
|
|||||||
|
|
||||||
**Ready for the first real tester (Tester-2): yes. Tester 2 (a laptop) is off; nothing was sent to it.**
|
**Ready for the first real tester (Tester-2): yes. Tester 2 (a laptop) is off; nothing was sent to it.**
|
||||||
|
|
||||||
**Updated 2026-10-18 (night).** Controller 0.329.0 is the floor for every box (21:36 UTC, MinAgent 0.158.1), after the three test boxes: each app reaches only its own data folder on the drive; homepage no longer controls Docker, so no app has a special right; when the host refuses to retire a drive, nothing changes and the household reads why; a virtual disk without a serial gets a plain sentence. Immich gets 3 GB (it was killed twice at 2 GB on a 4,000-photo import). Golden 0.329.0 is baked, NOT vouched. Hub 0.158.0 is built, NOT deployed — `REPORT-night-2026-10-18.md`. **Earlier:** **Updated 2026-10-17 (evening).** Controller 0.328.0 is the floor for every box (17:22 local): the box refuses a catalog change that asks for dangerous container rights (your D2 = A). Agent 0.159.1 (a gone, decommissioned drive is forgotten; an orphan thin snapshot is named in the backup failure; new Go) — runs on demo-hp, Tester 1 and demo-felhom; **your vouch in the hub is still open**. New emergency root passwords on demo-hp, Tester 1, demo-felhom (your D1 = A; read them in the hub). Hub 0.157.0 deployed on your yes — `REPORT-agent-allowlist-rotation-2026-10-17.md`. **Earlier:** **Updated 2026-10-17 (afternoon, later).** Controller 0.327.1 is the floor for every box since 12:06 UTC: FileBrowser is published only after its default password is checked (your D11 = A), and the install check uses real memory amounts with a clear message (your D10 = A) — `REPORT-filebrowser-memory-ci-2026-10-17.md`. **Earlier:** Controller 0.326.0 is the floor for every box since 09:55 UTC: FileBrowser's default password is checked at every FileBrowser start. Tester 1 is back on the live app list (test apps and drives removed); six memory limits raised — `REPORT-tester1-limits-filebrowser-2026-10-17.md`. **Earlier today:** controller 0.325.0 was the floor since 08:21 UTC (after the three test boxes): two more password doors count a try before checking it, family sign-ins are stored as fingerprints, a file-share rename always applies; five apps' Redis saves snapshots now, which a full disk cannot damage — `REPORT-burndown-3-2026-10-17.md`. **Earlier (2026-10-17 night):** controller 0.324.0 was the floor for every box since 02:46 UTC (all three test boxes made tonight's off-site backup on it first): a restore now brings the database password back with the data (your 1A), restic 0.19.1 (your 2A), dashboard hardening — `REPORT-night-2026-10-17.md`. **Earlier (2026-10-16 evening):** 0.323.0, Debian 13 — `REPORT-debian13-restic-2026-10-16.md`. **Earlier (midday):** Controller 0.319.0 (the app page in panels, the login in one panel) is the floor, MinAgent 0.158.1 — `REPORT-app-page-tuning-2026-10-16.md`. **Earlier (morning):** Storage identity build B shipped: agent 0.158.1 and golden 0.318.0 vouched by you, controller 0.318.0 is the floor (MinAgent 0.158.1), installer 1.33.0 published — new one-disk boxes get their own data area. Open items: see `documentation/backlog/OPEN-ITEMS.md`. Older notes, day by day:
|
**Updated 2026-10-19 (night).** An off-site backup no longer stops the household's apps for nothing when the agent is busy (controller 0.335.0 is the floor since 20:39 local; it works once the next agent is released — morning list). Three agent fixes and three hub fixes are written and tested, NOT released or deployed; the installer's „delete the old host first" message is written, NOT published (morning list). A security scanner now runs on every push in all four repos (warn-first: it lists, it never refuses); the first full scan found no real secret and no called weakness. The ArgoCD danger to three live hub secrets is gone (ArgoCD no longer tracks them). A failed off-site restore was measured on the scratch box — **decide D20**. ep0's first weekly clean-up ran at 05:30 local: it removed only tester-1's 4 October copy, as predicted, and the hub sees a rule for every household. — `REPORT-night-2026-10-19.md`. **Earlier:** **Updated 2026-10-19 (late afternoon).** The household passphrase no longer gives a live box's keys (your D19): each box now fetches its config with its own key (controller 0.334.0 is the floor since 16:31 local; hub 0.161.0 is live, your yes). A reinstall over a live box needs you to delete the old host first; the installer does not word that refusal yet (row filed). Every new household on ep0 gets its weekly clean-up rule automatically, and the hub alarms on one without; tester-1 and Tester-2 got theirs (your yes; tester-1's 4 October copy goes tonight). demo-hp kept its own address for a 30-minute watch after your router change. — `REPORT-day3-2026-10-19.md`. **Earlier:** **Updated 2026-10-19 (afternoon).** ep0 now keeps one whole-box copy per week for 2 weeks (your yes; the change removed nothing today, the first run is tomorrow 03:30 UTC). Controller 0.333.0 is the floor for every box (12:16 local): signing out on the dashboard now also ends the file manager's own 2-hour pass, and the recovery-code page says the code becomes live after the two words. Hub 0.160.0 is live (your yes): a reinstall within 45 minutes also mails you; a refused backup that repeats is counted, and the alarm repeats at most once a day. The household passphrase was measured: alone it reaches the box's keys and through them every off-site key action — **decide D19**. demo-hp loses its address to another device every few minutes — **fix it in the router** (do-list). — `REPORT-day2-2026-10-19.md`. **Earlier:** **Updated 2026-10-19 (day).** One sign-in for the file manager is live on every box (your D16 = yes; controller 0.332.0 is the floor since 05:24 local): „Fájlok ↗" opens the app's folder with no second login; signing out on the dashboard asks again; no app on a box can pretend to be the household (tested from inside an app). Agent 0.160.0 runs on demo-hp, Tester 1 and demo-felhom (your yes): a new recovery code replaces the old one only after the household types two words back; a broken-into box cannot flood the whole-box backups, and the local copies are one per day for 3 days (your D17); an empty partition no longer asks „data will be erased". **Vouch agent 0.160.0** (morning list). Hub 0.159.0 is live (your yes): a refused backup has its own alarm; a reinstall over an old host now mails you instead of staying silent — a first draft that re-issued the codes by itself was stopped by a security review. The old backup program still reads demo-hp's off-site copies after the first real clean-up with the new one. The document-preview error was a broken test file, not the box — `REPORT-day-2026-10-19.md`. **Earlier:** **Updated 2026-10-18 (evening).** A brand-new box was installed from nothing on demo-hp (your test customer Tester4): the emergency password was saved in the hub first, then set, and the web login took it; the box came up on the approved golden and moved to the floor by itself in under 3 minutes; its agent was the approved one. OnlyOffice now opens AND saves documents from FileBrowser and from Nextcloud (before tonight: FileBrowser could not open one, and Nextcloud lost every save while saying „saved") — controller 0.331.0 is the floor for every box (22:48 local). Immich's and Nextcloud's Redis no longer have a disk volume. Three agent fixes are written and tested, NOT released (your yes). Golden 0.331.0 is baked, NOT vouched — `REPORT-evening-2026-10-18.md`. **Earlier:** **Updated 2026-10-18 (day).** Hub 0.158.0 is live (your yes): its own 90-day access log works and writes no secret (tested with a fake token); you can vouch an agent without a new golden; the off-site snapshot check gives its first verdict on 20 October. Controller 0.330.0 is the floor (13:10 local): if the host refuses „retire anyway", the drive's apps start again. Agent 0.159.2 runs on the three test hosts (your yes; no pool check on a good backup) — **vouch it**. Installer 1.34.0 is published (your yes): the emergency password is saved in the hub first, then set. The calculator shows Immich'Line truncated
|
||||||
`documentation/backlog/STATUS-HISTORY.md`. Reports: `REPORT-night-2026-10-14.md`, `REPORT-immich-rollout-2026-10-14.md`, `REPORT-seo-felho-tarhely-2026-10-14.md`, `REPORT-immich-repair-2026-10-14.md`, `REPORT-power-cut-fixes-2026-10-14.md`, `REPORT-night-power-2026-10-13.md`, `REPORT-poweroff.md`.
|
`documentation/backlog/STATUS-HISTORY.md`. Reports: `REPORT-night-2026-10-14.md`, `REPORT-immich-rollout-2026-10-14.md`, `REPORT-seo-felho-tarhely-2026-10-14.md`, `REPORT-immich-repair-2026-10-14.md`, `REPORT-power-cut-fixes-2026-10-14.md`, `REPORT-night-power-2026-10-13.md`, `REPORT-poweroff.md`.
|
||||||
|
|
||||||
## What works
|
## What works
|
||||||
@@ -29,7 +29,7 @@
|
|||||||
- 65 apps in the catalog. New today: Prowlarr — you enter your download sources once, and Sonarr and Radarr get them, linked by the box with no button (the website lists 63 apps — R-1036). OS updates, the kernel lane too, run at night.
|
- 65 apps in the catalog. New today: Prowlarr — you enter your download sources once, and Sonarr and Radarr get them, linked by the box with no button (the website lists 63 apps — R-1036). OS updates, the kernel lane too, run at night.
|
||||||
- The hub is closed again after the 2026-10-12 security review; one box cannot read another's data.
|
- The hub is closed again after the 2026-10-12 security review; one box cannot read another's data.
|
||||||
|
|
||||||
- Felhom runs in a virtual machine on OpenMediaVault 8, tried on an exact copy of the first outside tester's box: install, bind, claim, apps, backup, and back by itself after an OMV restart. The Hungarian guide for him is `documentation/pilot/GUIDE-omv-kvm-hu.md`; open: one-disk Jellyfin (R-1068), NAS films for Jellyfin (R-1071, yours), no disk health in a VM (R-1073, yours). Waits on you: send the guide; delete test customer Tester4 in the hub.
|
- Felhom runs in a virtual machine on OpenMediaVault 8, tried on an exact copy of the first outside tester's box: install, bind, claim, apps, backup, and back by itself after an OMV restart. The Hungarian guide for him is `documentation/pilot/GUIDE-omv-kvm-hu.md`; open: one-disk Jellyfin (R-1068), NAS films for Jellyfin (R-1071, yours), no disk health in a VM (R-1073, yours). Waits on you: send the guide.
|
||||||
- A second local copy counts only when it is on a different physical disk, and the box can prove it. Inside a virtual machine the box makes the copy but says „A Felhom nem tudja ellenőrizni…" (Tester 1). On real disks nothing changes (demo-hp, N100). A NAS never hides the „only one copy" notice. A box with no drive refuses a media app in words, with a link to the Meghajtók page. Live on the three test boxes; the NAS and no-drive parts are unit-tested only. Next: build B (the internal place on one-disk boxes, R-1079). Your rulings D1–D6 are in the design (`architecture/16-storage-physical-identity.md`).
|
- A second local copy counts only when it is on a different physical disk, and the box can prove it. Inside a virtual machine the box makes the copy but says „A Felhom nem tudja ellenőrizni…" (Tester 1). On real disks nothing changes (demo-hp, N100). A NAS never hides the „only one copy" notice. A box with no drive refuses a media app in words, with a link to the Meghajtók page. Live on the three test boxes; the NAS and no-drive parts are unit-tested only. Next: build B (the internal place on one-disk boxes, R-1079). Your rulings D1–D6 are in the design (`architecture/16-storage-physical-identity.md`).
|
||||||
- **One-disk boxes get their own data area** (2026-10-16): a new install makes a „Belső tárhely" on the disk, outside every whole-box copy, and media apps install there with no extra step (proven on a fresh 512 GB box: 213 GiB area, Jellyfin with no extra step). A disk split into parts can be used part by part. The backup offer no longer promises disk-failure safety when it cannot check the disk, and `/mnt/sys_drive` is no longer offered.
|
- **One-disk boxes get their own data area** (2026-10-16): a new install makes a „Belső tárhely" on the disk, outside every whole-box copy, and media apps install there with no extra step (proven on a fresh 512 GB box: 213 GiB area, Jellyfin with no extra step). A disk split into parts can be used part by part. The backup offer no longer promises disk-failure safety when it cannot check the disk, and `/mnt/sys_drive` is no longer offered.
|
||||||
|
|
||||||
@@ -41,7 +41,7 @@
|
|||||||
|
|
||||||
## What we work on
|
## What we work on
|
||||||
- **CC's secrets** (R-1118): done 2026-10-17 — the store, `sec`, the guards in every CC session (option B: `bash -x` and `curl -v` allowed inside `sec run`), the nightly check (linger on). The old credentials file and its three copies are deleted.
|
- **CC's secrets** (R-1118): done 2026-10-17 — the store, `sec`, the guards in every CC session (option B: `bash -x` and `curl -v` allowed inside `sec run`), the nightly check (linger on). The old credentials file and its three copies are deleted.
|
||||||
- **After 2026-10-18 14:16 UTC** (demo-hp's next clean-up window): check that the old restic (image 0.322.0) still lists and restores demo-hp's store after its first real 0.19.1 prune (`audits/night-restore-2026-10-17/tools/rx.sh`). Everything else of R-1105/R-1107 is done.
|
- **Old restic after the first real 0.19.1 clean-up — DONE 2026-10-19:** demo-hp's window 12 ran at 02:18 UTC (324 → 261 copies, 63 removed as planned); the old restic (image 0.322.0, restic 0.14.0) then listed the same 261 and restored the newest copy (made by 0.19.1) and one from August with the same content hash as the new restic (`audits/evening-2026-10-18/P8/old-restic-check.txt`). R-1105/R-1107: nothing left.
|
||||||
- The power-cut findings still open (R-1004, R-1005, R-1010, R-1013). R-1012 closed: agent 0.159.1 names an orphan thin snapshot in the backup failure.
|
- The power-cut findings still open (R-1004, R-1005, R-1010, R-1013). R-1012 closed: agent 0.159.1 names an orphan thin snapshot in the backup failure.
|
||||||
- Media apps on every drive (R-1023): live with the 0.313.0 floor — every drive in every media app, a new drive made ready by the box, a pulled drive loses nothing, a safe removal lets go first and the drive comes back by itself.
|
- Media apps on every drive (R-1023): live with the 0.313.0 floor — every drive in every media app, a new drive made ready by the box, a pulled drive loses nothing, a safe removal lets go first and the drive comes back by itself.
|
||||||
|
|
||||||
@@ -58,21 +58,27 @@
|
|||||||
| D7 | R-990, R-989 | Rule yes / no / later on the forum ideas (`audits/forum-research-2026-10-12/FELHOM.md` §5) and the selfh.st ideas (`audits/selfhst-survey-2026-10-12/FELHOM.md` §6) | — | — | nothing changes; the research ages | — |
|
| D7 | R-990, R-989 | Rule yes / no / later on the forum ideas (`audits/forum-research-2026-10-12/FELHOM.md` §5) and the selfh.st ideas (`audits/selfhst-survey-2026-10-12/FELHOM.md` §6) | — | — | nothing changes; the research ages | — |
|
||||||
| D8 | R-206 | Install DooPlex's Docker settings from git (restarts Docker), and prune the build cache weekly: yes or no? | **Yes, in a quiet window** | — | a rebuild of DooPlex still has the settings in git | — |
|
| D8 | R-206 | Install DooPlex's Docker settings from git (restarts Docker), and prune the build cache weekly: yes or no? | **Yes, in a quiet window** | — | a rebuild of DooPlex still has the settings in git | — |
|
||||||
| D13 | R-1140 | **DECIDED 2026-10-17: A** — the hub keeps its own access log, 90 days. Built in hub 0.158.0 (≈ 19 MB for 90 days: the hub serves ≈ 1,870 requests a day, not 500); it waits for your deploy yes (morning list). | done | — | — | — |
|
| D13 | R-1140 | **DECIDED 2026-10-17: A** — the hub keeps its own access log, 90 days. Built in hub 0.158.0 (≈ 19 MB for 90 days: the hub serves ≈ 1,870 requests a day, not 500); it waits for your deploy yes (morning list). | done | — | — | — |
|
||||||
| D14 | R-1036 | The apps page lacks Plant-it and wger. Both are closed to new installs (wger hidden, Plant-it abandoned by its makers): add their cards anyway? | **No — leave them off** and close the row; the site shows only what a visitor can install | Yes: first open them for install again (wger's login throttle, Plant-it has no upstream), then the cards | nothing changes; the site lists 62 catalog apps, all installable | — |
|
| D14 | R-1036 | **DECIDED 2026-10-18: no** — wger and Plant-it stay off the site; it shows only apps a visitor can install. | done | — | — | — |
|
||||||
| D15 | R-1147 | The public machine estimator sizes Immich from a 200-photo test. Tonight's 4,000-photo test needed more. May the public figures move up? | **Yes** — CC re-runs the estimator with the new figures and shows you the before/after first | No: the page keeps the old Immich figure | a buyer with a big photo library may pick too small a machine | — |
|
| D15 | R-1147 | **DECIDED 2026-10-18: yes** — the estimator's Immich peak is 3,983 MB (was 3,290); Immich alone on 8 GB now reads „does not fit". Live. | done | — | — | — |
|
||||||
| D10 | R-1125 | **DECIDED 2026-10-17: A** — the 23 memory requests now match real use (catalog `1af6ea7`); the install refusal and page say what to do (controller 0.327.0). An 8 GB box no longer installs the whole household set. | done | — | — | — |
|
| D10 | R-1125 | **DECIDED 2026-10-17: A** — the 23 memory requests now match real use (catalog `1af6ea7`); the install refusal and page say what to do (controller 0.327.0). An 8 GB box no longer installs the whole household set. | done | — | — | — |
|
||||||
| D11 | R-1126 | **DECIDED 2026-10-17: A** — FileBrowser is published only after its default password is checked; it never answers on the default (controller 0.327.1). The tunnel still runs before the claim, by design (`01` §5). | done | — | — | — |
|
| D11 | R-1126 | **DECIDED 2026-10-17: A** — FileBrowser is published only after its default password is checked; it never answers on the default (controller 0.327.1). The tunnel still runs before the claim, by design (`01` §5). | done | — | — | — |
|
||||||
| D12 | R-1129 | **CLOSED 2026-10-17: not a defect** — CC misread the CI log; the failure mail always worked. A forced test mail went to admin@felhom.eu at 10:48:54 UTC (CC saw it arrive). Your key change did no harm. | done | — | — | — |
|
| D12 | R-1129 | **CLOSED 2026-10-17: not a defect** — CC misread the CI log; the failure mail always worked. A forced test mail went to admin@felhom.eu at 10:48:54 UTC (CC saw it arrive). Your key change did no harm. | done | — | — | — |
|
||||||
| D9 | R-1062 | The N100's USB drive is also its whole-box backup drive. Split it in two anyway? | **No — keep one drive**; the lived-in box keeps its nightly whole-box backup | Split: the box loses its local whole-box backup (or add a second drive first) | it stays one drive; multi-drive tests stay on Tester 1 | — |
|
| D9 | R-1062 | The N100's USB drive is also its whole-box backup drive. Split it in two anyway? | **No — keep one drive**; the lived-in box keeps its nightly whole-box backup | Split: the box loses its local whole-box backup (or add a second drive first) | it stays one drive; multi-drive tests stay on Tester 1 | — |
|
||||||
|
| D16 | R-1100 | **DECIDED 2026-10-19: yes** — built with both safeguards and live on every box (controller 0.332.0): a one-time pass from the dashboard, and a header name only the box knows. | done | — | — | — |
|
||||||
|
| D17 | R-1149 | **DECIDED 2026-10-19: one per day, 3 days** — agent 0.160.0 on the three test hosts. | done | — | — | — |
|
||||||
|
| D18 | R-1162 | **DECIDED 2026-10-19: A** — ep0 keeps one whole-box copy per ISO week, 2 weeks (`keep-weekly 2` on demo-hp and demo-felhom; nothing removed today; the first run, 2026-10-20 03:30 UTC, removed only tester-1's 2026-10-04 copy, as predicted). tester-1's missing job is R-1165 (your pick: not by hand today). | done | — | — | — |
|
||||||
|
| D20 | R-893 | After a failed off-site restore of one app, put it back exactly as it was and start it (A), or keep holding it stopped and only make the database part exact (B)? Measured tonight on the scratch box: the box puts the newer rows over the backup's older volume and a table only the backup had stays; the app is held. Note: `audits/night-2026-10-19/P3/design-R-893.md` | **A — copy the app's volumes first** (your D8 already chose this direction): one copy's disk for the restore's minutes; a fit check refuses before any stop | B: no disk, but a bad undo copy leaves an empty database, and the household still waits for support | a failed off-site restore keeps ending in a stopped app only support can bring back | — |
|
||||||
|
| D19 | R-1159 | **DECIDED 2026-10-19: box key first, then lock** — a live box pulls its config with its own key (controller 0.334.0, the floor), and the passphrase gives no keys once the box has reported on it (hub 0.161.0). Proven on demo-hp: 409, no key, you got the alarm mail. A reinstall needs your delete of the old host first. | done | — | — | — |
|
||||||
|
|
||||||
## Waiting on you — do
|
## Waiting on you — do
|
||||||
- **Vouch golden 0.329.0 and agent 0.159.1 together** (Configuration → Day-0 artifacts): golden sha256 `58dc40df555676d87200e5cb24cdc65f39cbacb2417e15a1955013526de4f68c`, agent sha256 `de8f6ad96860ec399757a88d8bbb0191c089c9bb99db8212482c12daab41aff9` (read from the registry tonight: matches), bundle `2af3d27d686fe611f3f44e89ba3d5123a5572f51eaf87e0ac63df00a41e67cd6`, Min agent **0.158.1**. If you do not: a new box installs golden 0.318.0 with agent 0.158.1 and then updates itself to the floor.
|
- **Morning list, 2026-10-20 (from the night of 2026-10-19):**
|
||||||
- **May I deploy hub 0.158.0** (you can vouch an agent without a new golden; the alarm for missing off-site copies; the hub's own 90-day access log)? Built tonight, not installed (night rule). The next daytime session asks you. If you say no: the hub stays 0.157.0 and nothing changes.
|
1. **Deploy hub 0.162.0?** (built `sha256:69285e64…`, from `d92a712e`): a hub restart no longer re-mails a box you already know is down (R-1001); the Day-0 agent list offers only real agent packages (R-1011); a box with no tunnel reads „none set up". **What the deploy does with the three Secrets** (`gitea-creds`, `umami-config`, `healthchecks-config`): **nothing** — ArgoCD no longer tracks them (read 18:21 UTC: none in the app's 25 resources, no tracking mark), so even a whole-app sync with prune leaves them; still sync only the hub, as always (R-953 closed). If not: Tester 2's restart mails continue on every hub restart.
|
||||||
- **Sunday night's golden:** a golden at the floor's version (0.329.0) already exists, unvouched. The Sunday rotation does not bake a duplicate (runbook updated); it repeats the vouch line above if it is still open, and bakes only if the floor has moved above 0.329.0.
|
2. **Release agent 0.161.0?** (code `56aabc7`, CHANGELOG „Unreleased"): `host_busy` for the off-site stop (R-921 — without it the controller's new check has nothing to read), a box with no tunnel no longer fails its OS update (R-1070), the janitor clears a half-removed backup snapshot (R-1010). Then your vouch. If not: the extra app stop and the Tester4-type false error stay.
|
||||||
|
3. **Publish installer 1.35.0?** (tag `installer-v1.35.0` + the `webpage.yaml` ref): a re-run on a live box says „the operator deletes the old host first" in EN and HU instead of „unexpected 409" (R-1167). If not: the next person to hit it reads only „409".
|
||||||
- **Nézd meg a demo-hp-n a Calibre-Web „Adatok és mentés" fülét és a Fájlkezelőt** (`felhom.enkisfelhom.hu`, controller 0.321.0, minden dobozon kiadva). A mentés egy részben, egyszerű szavakkal; a Fájlkezelő nem kér frissítést és nem ajánl „Install app"-ot.
|
- **Nézd meg a demo-hp-n a Calibre-Web „Adatok és mentés" fülét és a Fájlkezelőt** (`felhom.enkisfelhom.hu`, controller 0.321.0, minden dobozon kiadva). A mentés egy részben, egyszerű szavakkal; a Fájlkezelő nem kér frissítést és nem ajánl „Install app"-ot.
|
||||||
- **Look at `roms/roms/` on the N100's drive in the file manager (R-1099).** RomM now scans `roms/<platform>/`. A game uploaded through RomM before today sits one folder deeper: if there are platform folders in `roms/roms/`, move them up into `roms/` and press „Scan” in RomM. If not: those games show under a platform called „roms”.
|
- **Look at `roms/roms/` on the N100's drive in the file manager (R-1099).** RomM now scans `roms/<platform>/`. A game uploaded through RomM before today sits one folder deeper: if there are platform folders in `roms/roms/`, move them up into `roms/` and press „Scan” in RomM. If not: those games show under a platform called „roms”.
|
||||||
- **Nézd meg a Calibre-Web oldalát a demo-hp-n** (`felhom.enkisfelhom.hu`, controller 0.319.0, minden dobozon kiadva). A fejléc, a fülsor és minden rész most külön panel (az „A” stílus). A belépési adatok egy panelben vannak, egymás mellett. A Calibre-Web jelszaváról a doboz nem tud, mert az alkalmazás előbb települt, mint hogy a sablonban jelszó lett volna — ezt most egy mondat mondja ki. Ha nem tetszik, írd meg, mit változtassak. Ha nem nézed meg: semmi nem romlik el.
|
- **Nézd meg a Calibre-Web oldalát a demo-hp-n** (`felhom.enkisfelhom.hu`, controller 0.319.0, minden dobozon kiadva). A fejléc, a fülsor és minden rész most külön panel (az „A” stílus). A belépési adatok egy panelben vannak, egymás mellett. A Calibre-Web jelszaváról a doboz nem tud, mert az alkalmazás előbb települt, mint hogy a sablonban jelszó lett volna — ezt most egy mondat mondja ki. Ha nem tetszik, írd meg, mit változtassak. Ha nem nézed meg: semmi nem romlik el.
|
||||||
- **Discard two unclaimed boxes and delete Tester4** (hub, browser only): the build B scratch VMs registered as unclaimed boxes — registration ids 40 (`b43afa10…`) and 41 (`9c78ea30…`); both VMs are destroyed. Tester4 (the throwaway customer) is switched off the household way. If not: two dead entries stay in the Hosts list and Tester4 stays in Customers.
|
- **Discard two unclaimed boxes** (hub, browser only): the build B scratch VMs registered as unclaimed boxes — registration ids 40 (`b43afa10…`) and 41 (`9c78ea30…`); both VMs are destroyed. If not: two dead entries stay in the Hosts list. **Tester4 stays** (your word, 2026-10-18) as the fresh-install customer; **before its next install, delete its old host `Tester4-31fa0e`** (Hosts page) — otherwise the new install reuses the old host and skips two re-issues (since hub 0.159.0 you get a mail saying so; the codes are not re-issued by themselves, on purpose).
|
||||||
- System poster needs a refresh: one-disk boxes now have a data area („Belső tárhely") on the system disk, outside the whole-guest backups (facts file updated 2026-10-16).
|
- System poster needs a refresh: one-disk boxes now have a data area („Belső tárhely") on the system disk, outside the whole-guest backups (facts file updated 2026-10-16).
|
||||||
- **Olvasd el a Plex és a Sonarr útmutatót a demo-hp-n** (az alkalmazás oldalán az „Útmutató” fül). Ha más formát szeretnél, egy menetben átírom mind a 62-t. A robot 2026-10-16-án bejelentkezve megnézte: mindkét oldal megjelenik magyarul és angolul, 4 rész és a GYIK, minden sor egyezik a katalógussal. Ha nem nézed meg: csak a forma marad emberi szemmel nem látott.
|
- **Olvasd el a Plex és a Sonarr útmutatót a demo-hp-n** (az alkalmazás oldalán az „Útmutató” fül). Ha más formát szeretnél, egy menetben átírom mind a 62-t. A robot 2026-10-16-án bejelentkezve megnézte: mindkét oldal megjelenik magyarul és angolul, 4 rész és a GYIK, minden sor egyezik a katalógussal. Ha nem nézed meg: csak a forma marad emberi szemmel nem látott.
|
||||||
- **Nézd meg az AdventureLog és a Plex oldalát a demo-hp-n, a rendes címén** (`felhom.enkisfelhom.hu`). A túl nagy keret javítva: a tartalom most 32 képpontra van a bal oszloptól és 36-ra a tetejétől (előtte 80 és 64 volt). Az alkalmazás oldala mostantól egy hasábban van: egy szélesség, vékony vonalak a szakaszok között, aláhúzott fül, és doboz csak ott, ahol az alkalmazás vár rád (kék) vagy ahol valami kockázatos (sárga). A „sikeres frissítés" sáv kis szürke sorrá vált a név alatt. Ha nem nézed meg: a felület él, csak senki nem látta még böngészőben.
|
- **Nézd meg az AdventureLog és a Plex oldalát a demo-hp-n, a rendes címén** (`felhom.enkisfelhom.hu`). A túl nagy keret javítva: a tartalom most 32 képpontra van a bal oszloptól és 36-ra a tetejétől (előtte 80 és 64 volt). Az alkalmazás oldala mostantól egy hasábban van: egy szélesség, vékony vonalak a szakaszok között, aláhúzott fül, és doboz csak ott, ahol az alkalmazás vár rád (kék) vagy ahol valami kockázatos (sárga). A „sikeres frissítés" sáv kis szürke sorrá vált a név alatt. Ha nem nézed meg: a felület él, csak senki nem látta még böngészőben.
|
||||||
|
|||||||
@@ -271,6 +271,13 @@ app. Controller down → the gated app answers an error, never the app. Measured
|
|||||||
tunnel. Since controller v0.286.0 it sits ALONE on the `felhom-tunnel` network at the fixed address `172.16.253.2`
|
tunnel. Since controller v0.286.0 it sits ALONE on the `felhom-tunnel` network at the fixed address `172.16.253.2`
|
||||||
(traefik at `.3`), so traefik can believe forwarded headers from it and from nothing else (§5). Geo-restriction WAF
|
(traefik at `.3`), so traefik can believe forwarded headers from it and from nothing else (§5). Geo-restriction WAF
|
||||||
is **hub-enforced** (the hub holds the CF API token; the controller only reports geo desired-state).
|
is **hub-enforced** (the hub holds the CF API token; the controller only reports geo desired-state).
|
||||||
|
- **[DESIGN] The website's content comes from Gitea IN-CLUSTER** (2026-10-19). The `felhom-webpage` pod pulls
|
||||||
|
`website/` (from `main`) and `scripts/` (from the `installer-v…` tag, R-110) with four git-syncs that read
|
||||||
|
`http://gitea.gitea-system.svc.cluster.local:3000/admin/felhom.eu.git` — plain HTTP, safe only because it never
|
||||||
|
leaves the cluster (the hub's own Gitea client does the same). No internet hop: the pod can start, and keep its copy
|
||||||
|
fresh, during an internet outage. The two sidecars run with `--max-failures=-1`, so an unreachable Gitea is retried,
|
||||||
|
not a restart. **This ties the website to the DooPlex cluster:** moved elsewhere, it must read the public HTTPS
|
||||||
|
address again. `scripts/hostinstall_gates.py` gate 6 keeps the four `--repo` values identical.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@@ -539,8 +539,28 @@ state (both demo boxes) the panel says „Ezt a jelszót te állítottad be a F
|
|||||||
the press. The box uses FileBrowser's own CLI (`set -u admin,<pw>`) with the server STOPPED (the CLI opens the
|
the press. The box uses FileBrowser's own CLI (`set -u admin,<pw>`) with the server STOPPED (the CLI opens the
|
||||||
database with no lock timeout), verifies new=200 and admin/admin refused, stores the value encrypted and records
|
database with no lock timeout), verifies new=200 and admin/admin refused, stores the value encrypted and records
|
||||||
`generated`; a failed verify leaves `operator`. Not the `FILEBROWSER_ADMIN_PASSWORD` env: it re-applies on every
|
`generated`; a failed verify leaves `operator`. Not the `FILEBROWSER_ADMIN_PASSWORD` env: it re-applies on every
|
||||||
start. Spike: `audits/logins-folders-2026-10-16/P1b`. FileBrowser still asks for its own login the first time; one
|
start. Spike: `audits/logins-folders-2026-10-16/P1b`. Since v0.332.0 the folder links open FileBrowser without its own
|
||||||
sign-in for both is R-1100.
|
login (the next paragraph); `admin`'s password is the fallback and the operator's way in.
|
||||||
|
|
||||||
|
**[DESIGN] One sign-in for the file manager (v0.332.0, R-1100, operator decision D16 = yes 2026-10-19; spike
|
||||||
|
`audits/proxyauth-spike-2026-10-18/DESIGN-NOTE.md`).** Every dashboard link to a FileBrowser folder goes to
|
||||||
|
`/__files/open?to=<path>` on the dashboard host, which — for a signed-in, not-cross-site request — mints a pass (60 s,
|
||||||
|
one use, bound to `files.<domain>` and to the session) and sends the browser to `files.<domain>/_felhom/sso`. traefik
|
||||||
|
asks the controller before every FileBrowser request (forwardAuth `felhom-files-sso` → `/__felhom_gate/files`, the setup
|
||||||
|
gate's handshake): the pass becomes the host-only `felhom_files` cookie (a value derived from the session's fingerprint
|
||||||
|
for that host — the R-976 shape); a valid cookie → 200 with the box's **secret header** naming FileBrowser user
|
||||||
|
`felhom-household`; no cookie → 200 without it (FileBrowser's own password); a stale cookie (sign-out, password change)
|
||||||
|
→ both cookies cleared, back to the dashboard. **Why a secret header name:** FileBrowser trusts the header blindly and
|
||||||
|
every app container reaches `filebrowser:80` directly — measured: a forged fixed-name header from an app logged in. The
|
||||||
|
name (`X-Felhom-User-<32 hex>`, per box, 0600 in the data dir) is written only into FileBrowser's config and traefik's
|
||||||
|
route (`authResponseHeaders`, which strips a client's copy); never a log, a page or a report. **Why not a parent-domain
|
||||||
|
cookie:** every app subdomain is third-party code and would receive the dashboard session. **What stays:** `admin` is a
|
||||||
|
password user (R-513's reveal and reset, the R-1126 gate unchanged); the header user is a separate FileBrowser user whose
|
||||||
|
rights come from the box's userDefaults (change, add, delete, share, download — not admin, not API); FileBrowser's logout
|
||||||
|
goes to the dashboard. Residual: FileBrowser's own 2-hour token for the header user is cleared with the files cookie,
|
||||||
|
but a browser that lost only the files cookie keeps it until it expires. Measured live on all three test boxes
|
||||||
|
(`audits/day-2026-10-19/P2/`, `P4/`): opened at the folder with no login; forged headers refused through traefik and
|
||||||
|
from an app container; a sign-out asks again.
|
||||||
|
|
||||||
**[DESIGN] A stored FileBrowser password is checked before it is offered (v0.321.0, R-1101).** In `generated`, the
|
**[DESIGN] A stored FileBrowser password is checked before it is offered (v0.321.0, R-1101).** In `generated`, the
|
||||||
page asks FileBrowser whether the stored value still logs in; only FileBrowser's own 401/403 counts as refused (a
|
page asks FileBrowser whether the stored value still logs in; only FileBrowser's own 401/403 counts as refused (a
|
||||||
@@ -562,8 +582,9 @@ role, else the first declared folder, else (a media app) the first present drive
|
|||||||
`movies`, the players both, a download client its `media_library.folder`) with NO backup sentence — the bind's class
|
`movies`, the players both, a download client its `media_library.folder`) with NO backup sentence — the bind's class
|
||||||
there means „not this app's data", which is not the class-driven „temporary, not backed up". **A folder whose drive
|
there means „not this app's data", which is not the class-driven „temporary, not backed up". **A folder whose drive
|
||||||
is not bound into FileBrowser now is text, never a link** (`drivePresent` = the same gates the FileBrowser sync
|
is not bound into FileBrowser now is text, never a link** (`drivePresent` = the same gates the FileBrowser sync
|
||||||
applies). The button never promises one click: FileBrowser's router brings a cold visitor back to the folder after its
|
applies). Since v0.332.0 the button opens the folder signed in (one sign-in, above); on a box without it, FileBrowser's
|
||||||
login (measured in the headless browser, `audits/logins-folders-2026-10-16/browser/`).
|
router brings a cold visitor back to the folder after its login (measured in the headless browser,
|
||||||
|
`audits/logins-folders-2026-10-16/browser/`).
|
||||||
|
|
||||||
**[DESIGN] „Cím és kulcsok" is rows, not the install form.** The settings tab does not re-render the
|
**[DESIGN] „Cím és kulcsok" is rows, not the install form.** The settings tab does not re-render the
|
||||||
install wizard's fields: the address is the „Cím" row, the storage is a „Tárhely: <név>" row, every
|
install wizard's fields: the address is the „Cím" row, the storage is a „Tárhely: <név>" row, every
|
||||||
|
|||||||
@@ -232,7 +232,11 @@ The controller (in its LXC) reaches the agent (on the host) over the local bridg
|
|||||||
- `POST /rollback` — roll *this* guest back to a named snapshot (post-deploy failure recovery).
|
- `POST /rollback` — roll *this* guest back to a named snapshot (post-deploy failure recovery).
|
||||||
- `POST /backup` — request a backup-now of *this* guest (enqueued; non-destructive).
|
- `POST /backup` — request a backup-now of *this* guest (enqueued; non-destructive).
|
||||||
- `GET /backup/due` — whether a policy-scheduled backup is due for *this* guest, so the controller can quiesce then call `POST /backup` (the app-consistent path, §8).
|
- `GET /backup/due` — whether a policy-scheduled backup is due for *this* guest, so the controller can quiesce then call `POST /backup` (the app-consistent path, §8).
|
||||||
- `GET /backup/status`, `GET /restore-test/status` — read-only status for the controller's UI.
|
- `GET /backup/status`, `GET /restore-test/status` — read-only status for the controller's UI. **`host_busy`
|
||||||
|
(R-921, agent Unreleased `56aabc7`):** `GET /backup/status` also says whether the host-wide heavy-operation gate is
|
||||||
|
held (`{busy, what}` — another guest's backup, the restore test, an OS step, the stale-lock sweep, a power-off), the
|
||||||
|
state in which `POST /backup` answers 409; absent when the gate is not wired. The controller (≥ 0.335.0) reads it
|
||||||
|
before it stops any app. A read, never a reservation.
|
||||||
- **Crash-boot fact (R-856, agent v0.149.0):** `GET /host/crash-guard` — the host crash guard's last-boot record
|
- **Crash-boot fact (R-856, agent v0.149.0):** `GET /host/crash-guard` — the host crash guard's last-boot record
|
||||||
(`present`, `last_boot_at`, `last_boot_unclean`, `tripped`) read from `/var/lib/felhom-crash-guard/state.json`; a missing
|
(`present`, `last_boot_at`, `last_boot_unclean`, `tripped`) read from `/var/lib/felhom-crash-guard/state.json`; a missing
|
||||||
or garbled file answers 200 `present:false`, an older agent 404 — both read as a normal boot. The controller waits
|
or garbled file answers 200 `present:false`, an older agent 404 — both read as a normal boot. The controller waits
|
||||||
@@ -399,7 +403,10 @@ per Part 1: **snapshot** (LVM-thin, transient, whole-guest rollback — not a ba
|
|||||||
> space on …" — so the hub raises `restore_test_failed`; it is never a pass and never dropped.
|
> space on …" — so the hub raises `restore_test_failed`; it is never a pass and never dropped.
|
||||||
> - **Leftovers on a timer.** A failed scratch teardown and the stale-lock sweep (R-673) run every 10
|
> - **Leftovers on a timer.** A failed scratch teardown and the stale-lock sweep (R-673) run every 10
|
||||||
> minutes, not only at agent start; the sweep holds the one-heavy-operation gate; after 3 failed
|
> minutes, not only at agent start; the sweep holds the one-heavy-operation gate; after 3 failed
|
||||||
> teardown tries the operator is told.
|
> teardown tries the operator is told. **A half-removed `vzdump` snapshot (R-1010, agent Unreleased
|
||||||
|
> `56aabc7`):** when the sweep's delete fails ONLY with „Failed to find logical volume …_vzdump" (one disk's
|
||||||
|
> snapshot already gone), it retries once with Proxmox's own `force=1` on the same API call (no sudoers line, no
|
||||||
|
> root command); any other error is returned unchanged, and a failed forced delete is never forced again.
|
||||||
> - **A thin pool ≥ 90 % requests an immediate report**; the hub judges a thin pool on the worse of data
|
> - **A thin pool ≥ 90 % requests an immediate report**; the hub judges a thin pool on the worse of data
|
||||||
> and metadata, critical at 90 %, one alarm per pool per 6 hours (`08` §6.2).
|
> and metadata, critical at 90 %, one alarm per pool per 6 hours (`08` §6.2).
|
||||||
>
|
>
|
||||||
@@ -544,6 +551,19 @@ nicety.
|
|||||||
**loses Felhom's backup guarantees** — no PBS verify / monitoring on storage we can't reach. An
|
**loses Felhom's backup guarantees** — no PBS verify / monitoring on storage we can't reach. An
|
||||||
honest signup-time tradeoff, not a hidden one.
|
honest signup-time tradeoff, not a hidden one.
|
||||||
|
|
||||||
|
**When the new package reaches the hub (R-978; agent 0.160.0, 2026-10-19):** a ceremony started again no
|
||||||
|
longer replaces the hub's package when it starts. The package travels with the pending ceremony and is uploaded when its
|
||||||
|
code is claimed, before the code is handed out (an upload failure is a 502 and the code stays claimable). An abandoned
|
||||||
|
ceremony — TTL, a new start, a restart — leaves the first code opening the hub's package. **[DESIGN] Only after the
|
||||||
|
household typed it back (R-1151; agent 0.160.0 + controller 0.332.0, 2026-10-19):** a controller ≥ 0.332.0 claims with
|
||||||
|
`{"defer_upload": true}`; the agent hands out the code with two word positions it drew, keeps the package (and only those
|
||||||
|
two words, never the code) for 30 minutes, and uploads it only when `POST /escrow/ceremony/confirm` brings the two words
|
||||||
|
back — so a code opened and not written down never replaces the package. Wrong words 422 (still waiting), window gone 410,
|
||||||
|
nothing waiting 404, hub failure 502 (still waiting). A controller that has not updated claims with `{}` and gets the
|
||||||
|
R-978 order. Proven on demo-hp 2026-10-19: ceremony 1 typed back → code 1 opens the hub's package, code 0 is a kept
|
||||||
|
earlier one (422); ceremony 2 shown and abandoned → code 1 still opens it, code 2 opens nothing
|
||||||
|
(`audits/day-2026-10-19/P1/escrow-*.txt`).
|
||||||
|
|
||||||
#### Custody across generations — what the hub retains when a blob is superseded (hub v0.60.0; **contract corrected hub v0.93.0, R-198**)
|
#### Custody across generations — what the hub retains when a blob is superseded (hub v0.60.0; **contract corrected hub v0.93.0, R-198**)
|
||||||
|
|
||||||
A ceremony that seals a *different* passphrase does not destroy the blob it replaces: the hub copies
|
A ceremony that seals a *different* passphrase does not destroy the blob it replaces: the hub copies
|
||||||
|
|||||||
@@ -90,7 +90,11 @@ Evolves the existing staleness checker (60s **cadence**, a **configured** thresh
|
|||||||
2026-09-17, **45 m** since operator ruling A on R-549; OK under it, down at 2× = 90 m; today: controller-report recency → `node_stale`/`down`/`recovered`):
|
2026-09-17, **45 m** since operator ruling A on R-549; OK under it, down at 2× = 90 m; today: controller-report recency → `node_stale`/`down`/`recovered`):
|
||||||
|
|
||||||
- **Primary = host-report recency → `host_stale` / `host_down`.** The agent heartbeat is the box's
|
- **Primary = host-report recency → `host_stale` / `host_down`.** The agent heartbeat is the box's
|
||||||
liveness signal; a silent agent = the box is gone (the critical alert).
|
liveness signal; a silent agent = the box is gone (the critical alert). **A hub restart does not re-mail a known
|
||||||
|
silence (R-1001, hub 0.162.0 — built 2026-10-19, not yet deployed):** a host whose newest `host_stale`/`host_down`
|
||||||
|
event is newer than its last report is seeded with that state at start, silently; `restore_test_stale` reads its edge
|
||||||
|
from the DB on the first sweep (a signal for the tier inside the read window and newer than its last proof). A failed
|
||||||
|
lookup lets the alarm through.
|
||||||
- **Guest up/down comes from the host report's per-guest status** — authoritative, every poll, faster
|
- **Guest up/down comes from the host report's per-guest status** — authoritative, every poll, faster
|
||||||
than waiting for a guest report to go stale.
|
than waiting for a guest report to go stale.
|
||||||
- **Guest-report recency = secondary** app-level signal.
|
- **Guest-report recency = secondary** app-level signal.
|
||||||
@@ -134,7 +138,11 @@ walk is green; the golden is baked weekly and vouched by the operator; a new box
|
|||||||
itself to the floor (R-1052: not yet proven end to end). Setting the global floor today needs the operator page with the
|
itself to the floor (R-1052: not yet proven end to end). Setting the global floor today needs the operator page with the
|
||||||
operator's password — there is no scoped route for CC (R-1051). **An agent can be vouched mid-week; the golden stays
|
operator's password — there is no scoped route for CC (R-1051). **An agent can be vouched mid-week; the golden stays
|
||||||
weekly** (hub v0.158.0): the R-120 golden guard judges only a CHANGED golden selection, so a Day-0 save that keeps the
|
weekly** (hub v0.158.0): the R-120 golden guard judges only a CHANGED golden selection, so a Day-0 save that keeps the
|
||||||
vouched golden (same version and sha) saves a new agent even while that golden is behind the fleet.
|
vouched golden (same version and sha) saves a new agent even while that golden is behind the fleet. **The agent list
|
||||||
|
offers real agent packages only (R-1011, hub 0.162.0, not yet deployed):** the sha lookup asks for exactly the
|
||||||
|
`felhom-agent` file (the golden: `golden.tar.zst`) and never falls back to another file, so a step bundle
|
||||||
|
(`0.156.1-step1`, only the config bundle) is left out of the list, and a vouch of one is refused
|
||||||
|
(`artifact_file_missing`).
|
||||||
|
|
||||||
**The controller floor and its agent requirement (hub v0.112.0, R-472).** The managed controller floor
|
**The controller floor and its agent requirement (hub v0.112.0, R-472).** The managed controller floor
|
||||||
rides the report ACK. `store.ResolveManagedFloor` decides per box whether to serve it, from three
|
rides the report ACK. `store.ResolveManagedFloor` decides per box whether to serve it, from three
|
||||||
@@ -222,6 +230,51 @@ Today: `GET /config/{id}` with an `X-Retrieval-Password` (Hungarian passphrase)
|
|||||||
The existing `configgen` deep-merge + Hungarian-passphrase machinery is the base; it grows the
|
The existing `configgen` deep-merge + Hungarian-passphrase machinery is the base; it grows the
|
||||||
agent-first + key-pinning + restore-mode steps.
|
agent-first + key-pinning + restore-mode steps.
|
||||||
|
|
||||||
|
**A reinstall over an existing host record is made loud, never acted on [DESIGN, 2026-10-19 — R-1154, brief
|
||||||
|
2026-10-19; hub v0.159.0].** `POST /host-enroll` is the installer's step 4, run on every pass (a `--resume` too), with
|
||||||
|
the customer's retrieval passphrase; an existing host record is REUSED (same id and key — the reinstalled box takes
|
||||||
|
it over). The clean-slate re-issues (a new claim code mailed to the household, the off-site provider password reset)
|
||||||
|
run only on the MINT path, i.e. after the operator deleted the old host — **the operator's delete is their gate,
|
||||||
|
and stays so.** A reinstall that kept the old record used to skip them in silence. Now the reuse path classifies the
|
||||||
|
old box and re-issues NOTHING: **gone** (no controller wait connection open now, and either `deliberate_off_at`
|
||||||
|
inside its hold or BOTH the host report and the controller report older than `alerting.stale_threshold` or absent —
|
||||||
|
the `hostStatus` threshold and comparison) → the operator-only WARNING `host_reinstall_over_old_host`, mailed to the
|
||||||
|
operator: delete the old host (Hosts page) and run the installer again, or re-issue the claim code and the off-site
|
||||||
|
login from the customer page; **live** → the operator-only `host_reenroll_while_live` — since hub v0.160.0 (R-1156) a
|
||||||
|
WARNING mailed with the same instruction (a reinstall within 45 minutes of the old box's last report reads as live; an
|
||||||
|
installer `--resume` or a repair re-run mails too, and the text says those need nothing).
|
||||||
|
**Why not re-issue automatically** (a security review of the first draft, the same day): the passphrase is a
|
||||||
|
household-held secret typed into the installer, and both re-issues are operator SESSION-ONLY actions — not even an
|
||||||
|
operator token may run them (§17.5; `09` §3 decision 252, R-1055/R-1059: an off-site re-issue „stops its backup for a
|
||||||
|
while"). Re-issuing on the reuse path would give those powers to anyone holding the passphrase whenever the box is
|
||||||
|
switched off or silent. The cost of the loud shape is two operator acts per reinstall that kept its old record. Both
|
||||||
|
event types are hub-origin only. `hub/internal/api/reenroll.go`, pinned by `hub/internal/api/r1154_reenroll_test.go`.
|
||||||
|
|
||||||
|
**What the household's retrieval passphrase can get [DESIGN, 2026-10-19 — R-1159, operator decision D19 „box key
|
||||||
|
first, then lock"; hub v0.161.0 + controller v0.334.0].** The passphrase is the household's install secret: a NEW box
|
||||||
|
needs it (the installer's preflight, download login and enroll; the guest's first boot, `bootstrap.json` → `GET
|
||||||
|
/config`, which is how the box gets its own hub key). Measured the same day, it also gave anyone who held it the box's
|
||||||
|
keys for good, and through them every off-site key action (`audits/day-2026-10-19b/P2/`). **The rule:** the passphrase
|
||||||
|
gives keys only until the box uses its own — `GET /config`, `GET /recovery` and the reuse path of `POST /host-enroll`
|
||||||
|
answer **409 with no key** (and the operator gets `passphrase_keys_refused`) once a host record exists AND a controller
|
||||||
|
report arrived since it was made AND that controller pulls its config with its own key (≥ 0.334.0, `GET /box-config`,
|
||||||
|
`boxConfigMinController`). Open for a first install, the install in progress, a reinstall after the operator deleted
|
||||||
|
the old host, and a box on an older controller (which still refreshes with the passphrase — the lock arms box by box
|
||||||
|
as the floor moves). A reinstall over a live box therefore needs the operator's delete first, which R-1154 already
|
||||||
|
made the rule. **Why this shape and not „before a host exists":** the install itself uses the passphrase AFTER enroll
|
||||||
|
(`felhom-host-install.sh` step_agent_install → `/config`; the agent bakes it into `bootstrap.json`; the guest pulls with
|
||||||
|
it), and so did every live box's config refresh — measured from the code before building, and put to the operator.
|
||||||
|
`hub/internal/api/passphrase_scope.go`; pinned by `r1159_passphrase_scope_test.go`. Proven live on demo-hp
|
||||||
|
(2026-10-19 14:30 UTC): before 0.334.0 → 200; after its first 0.334.0 report → 409, no key, the event raised; the box's
|
||||||
|
own key → 200.
|
||||||
|
|
||||||
|
**[FACT, 2026-10-19 — R-1158, hub v0.160.0] What the MINT path's off-site re-issue still does.** It resets the Storage
|
||||||
|
Box sub-account password and stores the new one — a password only the hub itself uses (`offsitekeys.TargetFor`, the key
|
||||||
|
registrar's login). No box receives it since v0.127.0 (`consume-password` answers 410; `ConsumeOneTimeSecret` has no
|
||||||
|
production caller); the fresh box registers its own key with the controller API key. So it changes nothing a box uses.
|
||||||
|
It is kept because it hands the hub a fresh, working login at each clean reinstall, which repairs a broken stored one;
|
||||||
|
the `offsite_reissued` record now says the password is the hub's and the backups do not change.
|
||||||
|
|
||||||
## 9. DR model
|
## 9. DR model
|
||||||
|
|
||||||
The headline: the **old heavy infra-backup push retires** — not because the hub authors everything
|
The headline: the **old heavy infra-backup push retires** — not because the hub authors everything
|
||||||
|
|||||||
@@ -383,8 +383,8 @@ never moved by an update or an undo, and the unit restore accepts it.
|
|||||||
| **Tier-1** recovery unit | `<nsRoot>/backups/primary/<app>/` on the app's own drive | compose + `.felhom.yml` + **secret-stripped** `app.yaml`, `db-dumps/*.sql`, `volume-dumps/*.tar`, `manifest.json` | nightly at W, plus a checksum-gated refresh on the periodic status pass | one point per app (`restore_points.go:14-18`) | no |
|
| **Tier-1** recovery unit | `<nsRoot>/backups/primary/<app>/` on the app's own drive | compose + `.felhom.yml` + **secret-stripped** `app.yaml`, `db-dumps/*.sql`, `volume-dumps/*.tar`, `manifest.json` | nightly at W, plus a checksum-gated refresh on the periodic status pass | one point per app (`restore_points.go:14-18`) | no |
|
||||||
| **Tier-2** cross-drive | `<target nsRoot>/backups/secondary/<app>/` | **always** a full mirror of the unit (`tier2.go:368-369`) **plus** `mandatory + optional` file legs, v2 layout `hdd/<rel>` + `userdata/<rel>` | nightly at W+60m | mirror (rsync) | no |
|
| **Tier-2** cross-drive | `<target nsRoot>/backups/secondary/<app>/` | **always** a full mirror of the unit (`tier2.go:368-369`) **plus** `mandatory + optional` file legs, v2 layout `hdd/<rel>` + `userdata/<rel>` | nightly at W+60m | mirror (rsync) | no |
|
||||||
| **Tier-3** restic offsite | Hetzner Storage Box over SFTP, one multi-path snapshot per app per run | the unit **plus** `mandatory` legs only; a separate `_shares` snapshot | nightly at W+105m | `--keep-daily 7 --keep-weekly 4 --keep-monthly 6` | **yes** (restic) |
|
| **Tier-3** restic offsite | Hetzner Storage Box over SFTP, one multi-path snapshot per app per run | the unit **plus** `mandatory` legs only; a separate `_shares` snapshot | nightly at W+105m | `--keep-daily 7 --keep-weekly 4 --keep-monthly 6` | **yes** (restic) |
|
||||||
| **Plane-2** whole-guest, local | `local:` → `/var/lib/vz/dump` on the host | rootfs + `mp0 /var/lib/felhom` (its `docker/` and `sys_drive/` folders, bound in the guest to `/var/lib/docker` and `/mnt/sys_drive` — §7.5.1; `mp1` no longer exists, corrected 2026-10-15) | **24 h** (`backup_cadence_seconds: 0`) | `local_backup_retention: 3` | **no** |
|
| **Plane-2** whole-guest, local | `local:` → `/var/lib/vz/dump` on the host | rootfs + `mp0 /var/lib/felhom` (its `docker/` and `sys_drive/` folders, bound in the guest to `/var/lib/docker` and `/mnt/sys_drive` — §7.5.1; `mp1` no longer exists, corrected 2026-10-15) | **24 h** (`backup_cadence_seconds: 0`) | `local_backup_retention: 3` → `--prune-backups keep-daily=3`: one copy per day, 3 days (keep-last until agent 0.160.0, R-1149) | **no** |
|
||||||
| **Plane-2** whole-guest, offsite | `felhom-pbs:` → ep0 datastore `felhom-offsite`, per-customer namespace, over WireGuard | same contents | **7 days** (`604800`) | server-side prune on ep0, `keep-last 2` at `03:30` — **and the box asks for none** (R-191) | **yes** (per-customer key) |
|
| **Plane-2** whole-guest, offsite | `felhom-pbs:` → ep0 datastore `felhom-offsite`, per-customer namespace, over WireGuard | same contents | **7 days** (`604800`) | server-side prune on ep0, **`keep-weekly 2`** at `03:30` (one copy per ISO week, 2 weeks — R-1162, operator decision STATUS D18, 2026-10-19; `keep-last 2` before) — **and the box asks for none** (R-191) | **yes** (per-customer key) |
|
||||||
|
|
||||||
> **[FACT] Tier-3 has a fifth state the table above does not show: PAUSED (R-543, controller
|
> **[FACT] Tier-3 has a fifth state the table above does not show: PAUSED (R-543, controller
|
||||||
> v0.245.0).** Off-site is ON by default from hub v0.116.0, and a run does not start until the
|
> v0.245.0).** Off-site is ON by default from hub v0.116.0, and a run does not start until the
|
||||||
@@ -479,6 +479,31 @@ upload library is a mandatory class) or uploaded again by the household — its
|
|||||||
`pct config 9201`, both hosts). So the whole-guest tiers carry the guest and **none of the customer's
|
`pct config 9201`, both hosts). So the whole-guest tiers carry the guest and **none of the customer's
|
||||||
data drives** — 916 GB on demo-felhom, 938 GB on demo-hp.
|
data drives** — 916 GB on demo-felhom, 938 GB on demo-hp.
|
||||||
|
|
||||||
|
> **The whole-guest tiers are capped per guest (R-973; agent 0.160.0, released 2026-10-19 on the operator's yes — the
|
||||||
|
> numbers were *decided by CC unattended, operator may reverse*, and the operator said yes to the release).** Because the tiers prune by COUNT, a compromised controller that runs `POST /backup` again and
|
||||||
|
> again pushes out older restore points. The agent now refuses (429, logged, kept across a restart, a CRITICAL degraded
|
||||||
|
> status for 24 h) a run beyond: the daily tier — **3 successful runs per rolling 24 h** (the night run, one „Mentés
|
||||||
|
> most", the next night: a press never costs the night, R-899); the weekly off-site tier — **one per 6 days**
|
||||||
|
> (max(cadence − 24 h, cadence / 2)). The hub turns that status into the operator-only `whole_guest_backup_refused` (hub v0.159.0, R-1150); since hub v0.160.0 a further refusal inside one degraded stretch (the agent's reason names a new time) is counted (`refusals_in_episode`) and the alarm repeats at most once a day (R-1157). The question: how many runs may a guest make before the cap refuses? Options: no cap
|
||||||
|
> (today, floodable); a per-tier count per rolling window (chosen: no ordinary day is refused); a time-bucketed prune
|
||||||
|
> instead (the durable shape, but a retention change — R-1149 for the local tier, R-973's design line for ep0). Limits:
|
||||||
|
> ep0's prune is time-bucketed since 2026-10-19: **[FACT] `prune-demo-hp` and `prune-demo-felhom` keep `keep-weekly 2`**
|
||||||
|
> (R-1162, operator decision STATUS D18, 2026-10-19 08:24; read back 07:49 UTC; the first scheduled run, 2026-10-20
|
||||||
|
> 03:30 UTC, removed only tester-1's 2026-10-04 copy, as the dry runs said — `audits/night-2026-10-19/P10/`) — the newest copy of each of the two newest
|
||||||
|
> ISO weeks, so a flood inside a week replaces only that week's copy and the previous week's survives, however many runs
|
||||||
|
> get through. Before the change, PBS's own dry run and an offline model of its selection both showed the new rule
|
||||||
|
> removing nothing on either box (`audits/day-2026-10-19b/P1/`). DooPlex's copy of ep0 already prunes by time
|
||||||
|
> (`prune-ep0-copy`, `keep-weekly 8`, decision 71). **[FACT] Every household namespace gets its prune job when it is made**
|
||||||
|
> (R-1165, 2026-10-19): `felhom-tenantsync` v1.3.0 creates `prune-<id>` (keep-weekly 2, 03:30) with the namespace and
|
||||||
|
> removes it with the household; the hub alarms on a namespace no job covers (`pbsdr_namespace_without_prune_job`,
|
||||||
|
> from the 15-minute ep0 read). `tester-1` and `Tester-2` got theirs by hand on the operator's yes (13:35 UTC; the
|
||||||
|
> dry run: tester-1's 2026-10-04 copy goes at the next run, nothing on Tester-2). **[DESIGN] The local tier keeps one copy per day (R-1149, operator decision D17,
|
||||||
|
> 2026-10-19; agent 0.160.0):** `keep-daily=3` instead of keep-last 3 — the same at most three archives, but extra runs on
|
||||||
|
> one day replace only that day's copy, so with the cap a day of flooding leaves the two earlier days' copies
|
||||||
|
> (`TestBackupCap_R1149_LocalFloodKeepsOlderDays`). Measured at the first new-rule prune on demo-hp (retention 1: the
|
||||||
|
> same single newest copy either way) and read before on Tester 1 and demo-felhom (one copy per day each: the same copy
|
||||||
|
> goes at their next night under either rule) — `audits/day-2026-10-19/P1/`.
|
||||||
|
|
||||||
### 6.1.1 A box that is not always on — the catch-up, the banner, the alarms (R-871..R-874)
|
### 6.1.1 A box that is not always on — the catch-up, the banner, the alarms (R-871..R-874)
|
||||||
|
|
||||||
> **Why this section exists (R-871):** until 2026-10-05 no architecture document covered a box that is OFF at its
|
> **Why this section exists (R-871):** until 2026-10-05 no architecture document covered a box that is OFF at its
|
||||||
@@ -704,7 +729,13 @@ definition is written or a named volume replaced — a failed placement, volume
|
|||||||
the live definition back and HOLDS the app stopped (`restore_mixed`, operator-cleared); the household reads that it
|
the live definition back and HOLDS the app stopped (`restore_mixed`, operator-cleared); the household reads that it
|
||||||
needs our help; the operator gets a `backup_run_failures` line. The hold is persisted before the stop. Placed files
|
needs our help; the operator gets a `backup_run_failures` line. The hold is persisted before the stop. Placed files
|
||||||
ALONE (no version change, no volume replaced) still restart after a successful rollback (`TestR379_ScenarioA`): that
|
ALONE (no version change, no volume replaced) still restart after a successful rollback (`TestR379_ScenarioA`): that
|
||||||
mix, and the undo of everything, is option A — „put back exactly as it was", the next slice on R-893.
|
mix, and the undo of everything, is option A — „put back exactly as it was", the next slice on R-893. **`[FACT]`
|
||||||
|
2026-10-19 (measured on 9202, controller 0.335.0, `audits/night-2026-10-19/P3/`):** a forced replay failure after the
|
||||||
|
volume was replaced left the NEWEST rows (the undo dump) on top of the backup's volume — a table only the backup had
|
||||||
|
stayed (37 tables, the live app had 36); the definition did not move (same version); the app was held and a start
|
||||||
|
answered 409. The restore page's „the database is back as it was" is false for it. Design note with the two options and
|
||||||
|
the pick (A: a copy of the app's live volumes, put back on failure — one copy's disk for the restore's length, a fit
|
||||||
|
check before the stop): `audits/night-2026-10-19/P3/design-R-893.md`; the decision is on `STATUS.md`.
|
||||||
|
|
||||||
**[DESIGN] 2026-08-22 — the failure ladder of a database restore: replay → rollback → hold.**
|
**[DESIGN] 2026-08-22 — the failure ladder of a database restore: replay → rollback → hold.**
|
||||||
Recorded here rather than only in a closed register row, because a decision that survives only inside
|
Recorded here rather than only in a closed register row, because a decision that survives only inside
|
||||||
@@ -802,8 +833,12 @@ parts above, not a measured press).
|
|||||||
demo-hp: the off-site tier stopped every app for ~1 min and the agent then refused it (BUSY: the night OS step held the
|
demo-hp: the off-site tier stopped every app for ~1 min and the agent then refused it (BUSY: the night OS step held the
|
||||||
agent's host-wide lock). Before stopping anything the controller now asks the agent which tiers have a job in flight
|
agent's host-wide lock). Before stopping anything the controller now asks the agent which tiers have a job in flight
|
||||||
(`GET /backup/status`); another tier's job running or snapshotted → no stop, the tier stays due. A refusal after the
|
(`GET /backup/status`); another tier's job running or snapshotted → no stop, the tier stays due. A refusal after the
|
||||||
stop still resumes the apps at once (pinned by test). **Not covered:** the host-wide busy lock (OS step, restore test,
|
stop still resumes the apps at once (pinned by test). **The host-wide busy lock (2026-10-19, operator answer „add one
|
||||||
fstrim) — the agent serves it on no endpoint, so the measured case needs an agent change too (R-921, LEFT).
|
field"):** the agent serves it as `host_busy` on `GET /backup/status` (agent Unreleased `56aabc7`); controller 0.335.0
|
||||||
|
(the floor) reads it before any stop — held → no app stopped, each due tier recorded as contention (retry in 15 min,
|
||||||
|
one „BLOCKED" alarm after 3 h), exactly the verdict a 409 gets. An agent without the field keeps the old order (stop,
|
||||||
|
ask, resume at once). The check is a read, not a reservation: a gate taken between the read and the start costs one
|
||||||
|
short stop, resumed at once. The measured case is closed on a box once it runs the next agent release.
|
||||||
**Measured 2026-10-05 on demo-hp (9 apps, controller v0.295.0):** „Mentés most" stopped the apps at 09:19:08Z, the
|
**Measured 2026-10-05 on demo-hp (9 apps, controller v0.295.0):** „Mentés most" stopped the apps at 09:19:08Z, the
|
||||||
local tier ran 09:19:29–09:24:09, the PBS tier was busy (the controller logged a retry in 15 min; no second stop was seen in the next 55 min), the last app was back at 09:24:55Z — the
|
local tier ran 09:19:29–09:24:09, the PBS tier was busy (the controller logged a retry in 15 min; no second stop was seen in the next 55 min), the last app was back at 09:24:55Z — the
|
||||||
longest stop **5 min 47 s**, for the local tier alone. The button text and its confirm (v0.296.0) give both
|
longest stop **5 min 47 s**, for the local tier alone. The button text and its confirm (v0.296.0) give both
|
||||||
@@ -1447,7 +1482,7 @@ crosses the line — **R-158**.
|
|||||||
| 7 | **Guest stopped and does not come back** | everything | guest-power watchdog (60 s, `onboot` as the deliberate-stop discriminator) | automatic | **120 s** | | **PROVEN** | agent v0.107.0 replay — 120 s unattended vs the incident's 587 s with a human |
|
| 7 | **Guest stopped and does not come back** | everything | guest-power watchdog (60 s, `onboot` as the deliberate-stop discriminator) | automatic | **120 s** | | **PROVEN** | agent v0.107.0 replay — 120 s unattended vs the incident's 587 s with a human |
|
||||||
| 8 | **Host dies (hardware), drives intact** | the data drives; ep0's PBS namespace; the hub's Recipe + Escrow | **install a new host, then** `--selftest=bring-up -mode dr` per guest, **then** re-attach drives by `durable_id` | **operator** (SSH) | | 7 d | **IMPLEMENTED** **`[BETA-DEFERRED]`** | bring-up code exists and has **never been executed** (`CAMPAIGN-8…:522`); the drive half of the plan is empty on every box (**R-105**) |
|
| 8 | **Host dies (hardware), drives intact** | the data drives; ep0's PBS namespace; the hub's Recipe + Escrow | **install a new host, then** `--selftest=bring-up -mode dr` per guest, **then** re-attach drives by `durable_id` | **operator** (SSH) | | 7 d | **IMPLEMENTED** **`[BETA-DEFERRED]`** | bring-up code exists and has **never been executed** (`CAMPAIGN-8…:522`); the drive half of the plan is empty on every box (**R-105**) |
|
||||||
| 9 | **Whole box lost (fire/theft) — host and drives gone** | ep0 PBS namespace; the restic repo; the hub's Recipe + Escrow | new hardware → day-0 → escrow-consume with **R** → restore guests from PBS → app data from Tier-3 | **operator + customer** (R) | | 7 d (guest) · 24 h (app data) | **IMPLEMENTED / UNPROVEN** **`[BETA-DEFERRED]`** | every leg exists; the composed path has never been run. The destructive S5 drill is operator-gated and unrun (`06-offsite-connectivity.md:327`) **`[FACT]` 2026-10-03 (decision 70): ep0's datastore now has a nightly copy on DooPlex** (PBS pull-sync `ep0-felhom-offsite`, `remove-vanished false`, weekly verify, failures mailed to the operator); first pull 201 s, 12 GB, 4 of 4 snapshots, matching ep0 namespace for namespace. Restore route: `runbooks/ep0-datastore-copy.md` — never walked. `audits/offsite-lock-build-2026-10-03/partF/`. **`[FACT]` 2026-10-04: the restore route from the DooPlex copy was WALKED** on demo-hp (scratch VMID): listed in 2 s, restored in 186 s (15 GB logical), data read by `pct mount`; the restored config carries `onboot: 1` and the host's real drive binds — strip them on a restore beside the original (R-834). The copy now keeps 8 weekly copies (decision 71). `runbooks/ep0-datastore-copy.md` route 1. |
|
| 9 | **Whole box lost (fire/theft) — host and drives gone** | ep0 PBS namespace; the restic repo; the hub's Recipe + Escrow | new hardware → day-0 → escrow-consume with **R** → restore guests from PBS → app data from Tier-3 | **operator + customer** (R) | | 7 d (guest) · 24 h (app data) | **IMPLEMENTED / UNPROVEN** **`[BETA-DEFERRED]`** | every leg exists; the composed path has never been run. The destructive S5 drill is operator-gated and unrun (`06-offsite-connectivity.md:327`) **`[FACT]` 2026-10-03 (decision 70): ep0's datastore now has a nightly copy on DooPlex** (PBS pull-sync `ep0-felhom-offsite`, `remove-vanished false`, weekly verify, failures mailed to the operator); first pull 201 s, 12 GB, 4 of 4 snapshots, matching ep0 namespace for namespace. Restore route: `runbooks/ep0-datastore-copy.md` — never walked. `audits/offsite-lock-build-2026-10-03/partF/`. **`[FACT]` 2026-10-04: the restore route from the DooPlex copy was WALKED** on demo-hp (scratch VMID): listed in 2 s, restored in 186 s (15 GB logical), data read by `pct mount`; the restored config carries `onboot: 1` and the host's real drive binds — strip them on a restore beside the original (R-834). The copy now keeps 8 weekly copies (decision 71). `runbooks/ep0-datastore-copy.md` route 1. |
|
||||||
| 10 | **Ransomware / malicious deletion inside the guest** | PBS offsite (the box **cannot** delete its own snapshots); **the restic repo is NOT protected the same way** | whole-guest restore from PBS to a point before the event | **operator** (SSH) | | 7 d | **PARTIAL** **`[BETA-DEFERRED]`** | R-89 proved the box is refused when deleting its own PBS snapshot (`CAMPAIGN-8…:514-515`). **R-95 (open, ranked #1):** the restic credential **can delete** — `readonly=False`, `forget --prune` runs from the box, and SFTP cannot express append-only (`OPEN-ITEMS.md:13`) **2026-08-30 (R-359): the store is now VERIFIED on a cadence** — a daily `offsite-integrity` job runs `restic check` when the last successful one is over 7 days old. **This is a readability check, not a restore-test (R-87 stays open).** The depth that ships ON now re-reads **100%** of the pack data (R-399 CLOSED, controller v0.228.0) — the sentence here previously said the opposite and was stale. **2026-08-31 (SPIKE R-87):** a scratch restore of every app on demo-hp was measured at **25 s for 8 snapshots / 774 MB**, against 40.3 s for one weekly check — but restic 0.14.0's `--verify` checks size and mtime, **not content**, so nothing available today can vouch for the restored BYTES. Row 4's verdict is UNCHANGED by the spike. `audits/SPIKE-restic-restore-test-2026-08-31.md`. **2026-09-01 (SPIKE R-95, `audits/SPIKE-r95-offsite-delete-2026-09-01.md`) — THIS ROW'S STATUS IS UNCHANGED, but the seven-day bound beside it is now known to be unverified.** Measured on BOTH boxes over their own SFTP credential, with controls: **no `.snapshots` is visible to either sub-account**, and the account is jailed. Either none exist or a sub-account cannot see them — and a snapshot the box cannot see is one the box cannot restore from, so recovery would be an operator act at the Hetzner panel, not a product capability (R-429; the confirming field needs the provider API, fenced by §11-D). **The PBS shape does NOT transfer**: the sub-account API has one permission axis, `readonly`, and a backup target cannot be read-only — PBS is a server that can refuse, a Storage Box is a filesystem that runs nothing. **What the spike removed as a fear:** withdrawing delete does NOT wedge the store (measured — restic treats a dead owner's lock as stale and proceeds). **What it added:** `unlock --remove-all` reports success while deleting nothing (R-430). **2026-09-01, LATER THE SAME DAY — THE PROBE ABOVE LOOKED FOR THE WRONG NAME AND THIS ROW'S SECOND CLAUSE IS NOW HALF WRONG.** It searched `.snapshots`; the vendor documents `/.zfs/snapshot`. Re-probed at the documented path on BOTH boxes, with controls, the two halves separate cleanly: **(a) the LIVE repository is deletable by the box — unchanged, R-95 stands;** **(b) the daily SNAPSHOTS of it are not writable by anything — PROVEN, not cited:** a write into `/.zfs/snapshot` is refused (`dest open …: Failure`) while the identical write to the account home succeeds. Seven daily snapshots confirmed in the panel (R-429). **So this row's "the restic repo is NOT protected the same way" is true of the repository and FALSE of its snapshots** — a deletion costs at most the day since the last snapshot, recoverable per-file (vendor). **Two limits kept honest:** a sub-account sees the snapshot directory EMPTY, so per-file recovery is operator-only today (R-432); and a panel-driven restore rolls back the WHOLE box. **The status is NOT moved** — evidence (b) is measured, but the recovery ROUTE has never been walked, which is what PARTIAL means. **Detection shipped hub v0.111.0 (R-431):** an unexplained fall in the snapshot count is noticed within a day — **but only a fall of MORE than half (R-435).** **2026-09-01, THE DRILL — CLAUSE (b) OF THE RE-SCOPE ABOVE IS WITHDRAWN; THE STATUS IS STILL NOT MOVED.** The re-scope said a deletion is *recoverable per-file*. **It is not, from the box: no snapshot is reachable by ANY name.** MEASURED on demo-hp, read-only, no delete verb issued: **777,600** exact names in the vendor form `YYYY-MM-DDTHH-MM-SS` (nine full days, second granularity) plus 126 alternative shapes — **zero hits**, against a control where the identical 600-name batch returns a path that exists (6/6). **Structural cause:** `/home` (`u629488-sub3`) is **st_dev 0,82**, `/.zfs/snapshot` is **st_dev 0,276**, and `/home/.zfs` does not exist — a snapshot under `/.zfs/snapshot` belongs to a different dataset than the one holding the repository. **Clause (a) — the box cannot WRITE into the snapshot area — is unchanged and re-confirmed.** So this row's *"recoverable per-file (vendor)"* and its limit *"per-file recovery is operator-only today (R-432)"* both overstate what exists: the remaining routes are a panel rollback of the WHOLE Storage Box and the provider API (fenced, and the hub's client has no snapshot method at all). **R-432 ANSWERED negatively; R-433 opened. RTO still blank — nothing was recoveredLine truncated
|
| 10 | **Ransomware / malicious deletion inside the guest** | PBS offsite (the box **cannot** delete its own snapshots); **the restic repo is NOT protected the same way** | whole-guest restore from PBS to a point before the event | **operator** (SSH) | | 7 d | **PARTIAL** **`[BETA-DEFERRED]`** | R-89 proved the box is refused when deleting its own PBS snapshot (`CAMPAIGN-8…:514-515`). **R-95 (open, ranked #1):** the restic credential **can delete** — `readonly=False`, `forget --prune` runs from the box, and SFTP cannot express append-only (`OPEN-ITEMS.md:13`) **2026-08-30 (R-359): the store is now VERIFIED on a cadence** — a daily `offsite-integrity` job runs `restic check` when the last successful one is over 7 days old. **This is a readability check, not a restore-test (R-87 stays open).** The depth that ships ON now re-reads **100%** of the pack data (R-399 CLOSED, controller v0.228.0) — the sentence here previously said the opposite and was stale. **2026-08-31 (SPIKE R-87):** a scratch restore of every app on demo-hp was measured at **25 s for 8 snapshots / 774 MB**, against 40.3 s for one weekly check — but restic 0.14.0's `--verify` checks size and mtime, **not content**, so nothing available today can vouch for the restored BYTES. Row 4's verdict is UNCHANGED by the spike. `audits/SPIKE-restic-restore-test-2026-08-31.md`. **2026-09-01 (SPIKE R-95, `audits/SPIKE-r95-offsite-delete-2026-09-01.md`) — THIS ROW'S STATUS IS UNCHANGED, but the seven-day bound beside it is now known to be unverified.** Measured on BOTH boxes over their own SFTP credential, with controls: **no `.snapshots` is visible to either sub-account**, and the account is jailed. Either none exist or a sub-account cannot see them — and a snapshot the box cannot see is one the box cannot restore from, so recovery would be an operator act at the Hetzner panel, not a product capability (R-429; the confirming field needs the provider API, fenced by §11-D). **The PBS shape does NOT transfer**: the sub-account API has one permission axis, `readonly`, and a backup target cannot be read-only — PBS is a server that can refuse, a Storage Box is a filesystem that runs nothing. **What the spike removed as a fear:** withdrawing delete does NOT wedge the store (measured — restic treats a dead owner's lock as stale and proceeds). **What it added:** `unlock --remove-all` reports success while deleting nothing (R-430). **2026-09-01, LATER THE SAME DAY — THE PROBE ABOVE LOOKED FOR THE WRONG NAME AND THIS ROW'S SECOND CLAUSE IS NOW HALF WRONG.** It searched `.snapshots`; the vendor documents `/.zfs/snapshot`. Re-probed at the documented path on BOTH boxes, with controls, the two halves separate cleanly: **(a) the LIVE repository is deletable by the box — unchanged, R-95 stands;** **(b) the daily SNAPSHOTS of it are not writable by anything — PROVEN, not cited:** a write into `/.zfs/snapshot` is refused (`dest open …: Failure`) while the identical write to the account home succeeds. Seven daily snapshots confirmed in the panel (R-429). **So this row's "the restic repo is NOT protected the same way" is true of the repository and FALSE of its snapshots** — a deletion costs at most the day since the last snapshot, recoverable per-file (vendor). **Two limits kept honest:** a sub-account sees the snapshot directory EMPTY, so per-file recovery is operator-only today (R-432); and a panel-driven restore rolls back the WHOLE box. **The status is NOT moved** — evidence (b) is measured, but the recovery ROUTE has never been walked, which is what PARTIAL means. **Detection shipped hub v0.111.0 (R-431):** an unexplained fall in the snapshot count is noticed within a day — **but only a fall of MORE than half (R-435).** **2026-09-01, THE DRILL — CLAUSE (b) OF THE RE-SCOPE ABOVE IS WITHDRAWN; THE STATUS IS STILL NOT MOVED.** The re-scope said a deletion is *recoverable per-file*. **It is not, from the box: no snapshot is reachable by ANY name.** MEASURED on demo-hp, read-only, no delete verb issued: **777,600** exact names in the vendor form `YYYY-MM-DDTHH-MM-SS` (nine full days, second granularity) plus 126 alternative shapes — **zero hits**, against a control where the identical 600-name batch returns a path that exists (6/6). **Structural cause:** `/home` (`u629488-sub3`) is **st_dev 0,82**, `/.zfs/snapshot` is **st_dev 0,276**, and `/home/.zfs` does not exist — a snapshot under `/.zfs/snapshot` belongs to a different dataset than the one holding the repository. **Clause (a) — the box cannot WRITE into the snapshot area — is unchanged and re-confirmed.** So this row's *"recoverable per-file (vendor)"* and its limit *"per-file recovery is operator-only today (R-432)"* both overstate what exists: the remaining routes are a panel rollback of the WHOLE Storage Box and the provider API (fenced, and the hub's client has no snapshot method at all). **R-432 ANSWERED negatively; R-433 opened. RTO still blank — nothing was recoveredLine truncated
|
||||||
| 11 | **Hub lost** | every box's data plane, every tier, every Lane-1 route | none needed for recovery **of a box**; the hub itself restores from its Longhorn volume backup | operator (`kubectl`) | | 24 h + weekly (Longhorn `RETAIN 1` each) | **UNPROVEN** **`[BETA-DEFERRED]`** | a hub restore has never been performed. The backup target is `nfs://192.168.0.180` — **DooPlex itself** — and exactly **2** restore points exist (LIVE, INV Part D2.2) |
|
| 11 | **Hub lost** | every box's data plane, every tier, every Lane-1 route | none needed for recovery **of a box**; the hub itself restores from its Longhorn volume backup | operator (`kubectl`) | | 24 h + weekly (Longhorn `RETAIN 1` each) | **UNPROVEN** **`[BETA-DEFERRED]`** | a hub restore has never been performed. The backup target is `nfs://192.168.0.180` — **DooPlex itself** — and exactly **2** restore points exist (LIVE, INV Part D2.2) |
|
||||||
| 11b | *consequences while the hub is gone* | — | — | — | | | **[FACT]** **`[BETA-DEFERRED]`** | **day:** nothing customer-visible breaks; events queue (`settings.go:1466-1485`). **week:** the operator alarm plane is dark, no claim/reset codes, no config or floor convergence, no PBS-secret re-issue. **permanently:** escrow custody and break-glass credentials are gone (INV Part D2.4) |
|
| 11b | *consequences while the hub is gone* | — | — | — | | | **[FACT]** **`[BETA-DEFERRED]`** | **day:** nothing customer-visible breaks; events queue (`settings.go:1466-1485`). **week:** the operator alarm plane is dark, no claim/reset codes, no config or floor convergence, no PBS-secret re-issue. **permanently:** escrow custody and break-glass credentials are gone (INV Part D2.4) |
|
||||||
| 12 | **Offsite provider lost (Hetzner)** | everything on-premises: both drives, both whole-guest tiers | none needed — on-premises recovery is unaffected. Re-provision a new offsite target. | operator | | | **[FACT]** **`[BETA-DEFERRED]`** | **restic and PBS share the provider** (INV Part E.1). Whether they share an account and payment method is **UNKNOWN** → §11-D **`[FACT]` 2026-10-03:** the PBS (whole-guest) off-site copy now also exists OFF Hetzner — DooPlex pulls ep0's datastore nightly (decision 70). The restic tier is still Hetzner-only. |
|
| 12 | **Offsite provider lost (Hetzner)** | everything on-premises: both drives, both whole-guest tiers | none needed — on-premises recovery is unaffected. Re-provision a new offsite target. | operator | | | **[FACT]** **`[BETA-DEFERRED]`** | **restic and PBS share the provider** (INV Part E.1). Whether they share an account and payment method is **UNKNOWN** → §11-D **`[FACT]` 2026-10-03:** the PBS (whole-guest) off-site copy now also exists OFF Hetzner — DooPlex pulls ep0's datastore nightly (decision 70). The restic tier is still Hetzner-only. |
|
||||||
|
|||||||
@@ -724,7 +724,11 @@ Evidence: `audits/os-host-lane-2026-10-04/` (parts A–G).
|
|||||||
skips the host step.
|
skips the host step.
|
||||||
- **The host health rule** (`HostHealthVerdict`, pinned in `internal/osupdate`): `felhom-agent`, `pveproxy`,
|
- **The host health rule** (`HostHealthVerdict`, pinned in `internal/osupdate`): `felhom-agent`, `pveproxy`,
|
||||||
`pvedaemon`, `pvestatd` and `pve-cluster` are `active`; the customer guest runs; the guest health rule (§8.1)
|
`pvedaemon`, `pvestatd` and `pve-cluster` are `active`; the customer guest runs; the guest health rule (§8.1)
|
||||||
passes; the tunnel is `running` — an `unknown` tunnel does not fail it, `not_running` does. Same 5-minute wait.
|
passes; the tunnel is `running` — `not_running` and `unknown` fail it (CORRECTED 2026-10-19: this line said
|
||||||
|
„an `unknown` tunnel does not fail it"; the code always failed it, which is what Tester4 hit, R-1070). **Since agent
|
||||||
|
Unreleased `56aabc7` (R-1070):** a guest with NO cloudflared container reads `absent` (Docker 29's lower-case
|
||||||
|
„no such object" used to fall through to `unknown`), and `absent` passes when the reading from before the step also
|
||||||
|
listed no cloudflared container (Docker answered) — a box with no tunnel set up. Same 5-minute wait.
|
||||||
- **Separate approved sets:** host and guest releases are separate (`os-host-…`, `os-guest-…`), each by the same rule
|
- **Separate approved sets:** host and guest releases are separate (`os-host-…`, `os-guest-…`), each by the same rule
|
||||||
(24 h, 1 night of THAT layer, every ring-0 box). Ring 1 receives `host_release` beside `release`.
|
(24 h, 1 night of THAT layer, every ring-0 box). Ring 1 receives `host_release` beside `release`.
|
||||||
- **Reboot needed:** reported when PID 1 or `lxc-start` maps a replaced file, with the date of the first scanned
|
- **Reboot needed:** reported when PID 1 or `lxc-start` maps a replaced file, with the date of the first scanned
|
||||||
|
|||||||
@@ -87,8 +87,8 @@ Title: **"Where a household's data is copied"**
|
|||||||
| **Tier 1 — recovery unit** | each app: its settings, database dump, volumes | data drive 1, on the box | nightly at **02:30** (W) | 1 point per app | not encrypted (it never leaves the home) |
|
| **Tier 1 — recovery unit** | each app: its settings, database dump, volumes | data drive 1, on the box | nightly at **02:30** (W) | 1 point per app | not encrypted (it never leaves the home) |
|
||||||
| **Tier 2 — cross-drive** | a mirror of Tier 1, plus the app's files | data drive 2, on the box (counted only on a different physical disk; in a VM made, not counted) | nightly **03:30** | mirror | not encrypted |
|
| **Tier 2 — cross-drive** | a mirror of Tier 1, plus the app's files | data drive 2, on the box (counted only on a different physical disk; in a VM made, not counted) | nightly **03:30** | mirror | not encrypted |
|
||||||
| **Tier 3 — off-site, per app** | Tier 1 + the app's must-keep files | **Hetzner Storage Box** (Germany), one folder per household | nightly **04:15** | 7 daily, 4 weekly, 6 monthly | the household's own restic password |
|
| **Tier 3 — off-site, per app** | Tier 1 + the app's must-keep files | **Hetzner Storage Box** (Germany), one folder per household | nightly **04:15** | 7 daily, 4 weekly, 6 monthly | the household's own restic password |
|
||||||
| **Whole guest — local** | the guest's system (not the data drives) | the box's own system disk | daily, in the window **04:30–08:30** | up to 3 copies (fewer on a small disk) | not encrypted |
|
| **Whole guest — local** | the guest's system (not the data drives) | the box's own system disk | daily, in the window **04:30–08:30** | one copy per day, up to 3 days (fewer on a small disk) | not encrypted |
|
||||||
| **Whole guest — off-site** | the same | **ep0** (Hetzner, Germany), one namespace per household | weekly, same window | last 2 | the household's own key, escrowed with a recovery code |
|
| **Whole guest — off-site** | the same | **ep0** (Hetzner, Germany), one namespace per household | weekly, same window | one per week, 2 weeks | the household's own key, escrowed with a recovery code |
|
||||||
|
|
||||||
Footnote: "W = the household's backup window start, 02:30 by default. Tiers 1–3 follow at fixed offsets so they cannot run out of order. A missed night is caught up 15 minutes after the box starts."
|
Footnote: "W = the household's backup window start, 02:30 by default. Tiers 1–3 follow at fixed offsets so they cannot run out of order. A missed night is caught up 15 minutes after the box starts."
|
||||||
|
|
||||||
@@ -99,7 +99,7 @@ Footnote: "W = the household's backup window start, 02:30 by default. Tiers 1–
|
|||||||
- one namespace **per household** (whole-guest copies, each household's own key),
|
- one namespace **per household** (whole-guest copies, each household's own key),
|
||||||
with a NEUTRAL note under it (not a red gap box): "By design: each household holds the key to its own copies; the operator cannot open them without their recovery code." — *fix (c), 2026-10-09: this is a property of the design, not a gap, and a red box said the opposite of what it means.*
|
with a NEUTRAL note under it (not a red gap box): "By design: each household holds the key to its own copies; the operator cannot open them without their recovery code." — *fix (c), 2026-10-09: this is a property of the design, not a gap, and a red box said the opposite of what it means.*
|
||||||
- the **`operator`** namespace: two groups — **"hub database"** and **"DooPlex copy"** (code, password manager, secrets, signing keys).
|
- the **`operator`** namespace: two groups — **"hub database"** and **"DooPlex copy"** (code, password manager, secrets, signing keys).
|
||||||
Prune jobs on ep0: households keep the last 2; `operator` keeps 14 daily + 8 weekly.
|
Prune jobs on ep0: households keep one copy per week for 2 weeks (`keep-weekly 2`, R-1162, 2026-10-19; every new household's job is made with its namespace, R-1165); `operator` keeps 14 daily + 8 weekly.
|
||||||
Mark ep0 with a small shield: "**Protected** — holds the only off-premises copy of real household data."
|
Mark ep0 with a small shield: "**Protected** — holds the only off-premises copy of real household data."
|
||||||
- **Hetzner Storage Box** (grey): the shared box for the households' per-app restic copies (Tier 3), one sub-account per household, add-only keys.
|
- **Hetzner Storage Box** (grey): the shared box for the households' per-app restic copies (Tier 3), one sub-account per household, add-only keys.
|
||||||
- **Mail service** (grey, "Resend"): every mail the hub sends — to the operator and to households.
|
- **Mail service** (grey, "Resend"): every mail the hub sends — to the operator and to households.
|
||||||
|
|||||||
@@ -385,7 +385,7 @@ document.addEventListener('DOMContentLoaded', async function() {
|
|||||||
</script>
|
</script>
|
||||||
|
|
||||||
<script type="__bundler/template">
|
<script type="__bundler/template">
|
||||||
"<!DOCTYPE html>\n<html lang=\"en\"><head>\n<meta charset=\"utf-8\">\n<title>Felhom system poster<\u002Ftitle>\n<script src=\"7425d086-bbee-40a4-a354-fbfba43c8a3d\"><\u002Fscript>\n<meta name=\"hz:slide-selector\" content=\"[data-canvas-width]\"><meta name=\"hz:canvas-width\" content=\"3200\"><meta name=\"hz:canvas-height\" content=\"2262\"><\u002Fhead>\n<body>\n<x-dc>\n<helmet>\n<link rel=\"preconnect\" href=\"https://fonts.googleapis.com\">\n<style>/* cyrillic-ext */\n@font-face {\n font-family: 'Plus Jakarta Sans';\n font-style: normal;\n font-weight: 400;\n font-display: swap;\n src: url(https://fonts.gstatic.com/s/plusjakartasans/v12/LDIoaomQNQcsA88c7O9yZ4KMCoOg4Ko70yyygA.woff2) format('woff2');\n unicode-range: U+0460-052F, U+1C80-1C8A, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F;\n}\n/* vietnamese */\n@font-face {\n font-family: 'Plus Jakarta Sans';\n font-style: normal;\n font-weight: 400;\n font-display: swap;\n src: url(https://fonts.gstatic.com/s/plusjakartasans/v12/LDIoaomQNQcsA88c7O9yZ4KMCoOg4Ko50yyygA.woff2) format('woff2');\n unicode-range: U+0102-0103, U+0110-0111, U+0128-0129, U+0168-0169, U+01A0-01A1, U+01AF-01B0, U+0300-0301, U+0303-0304, U+0308-0309, U+0323, U+0329, U+1EA0-1EF9, U+20AB;\n}\n/* latin-ext */\n@font-face {\n font-family: 'Plus Jakarta Sans';\n font-style: normal;\n font-weight: 400;\n font-display: swap;\n src: url(https://fonts.gstatic.com/s/plusjakartasans/v12/LDIoaomQNQcsA88c7O9yZ4KMCoOg4Ko40yyygA.woff2) format('woff2');\n unicode-range: U+0100-02BA, U+02BD-02C5, U+02C7-02CC, U+02CE-02D7, U+02DD-02FF, U+0304, U+0308, U+0329, U+1D00-1DBF, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C4, U+2113, U+2C60-2C7F, U+A720-A7FF;\n}\n/* latin */\n@font-face {\n font-family: 'Plus Jakarta Sans';\n font-style: normal;\n font-weight: 400;\n font-display: swap;\n src: url(\"e228819f-c09e-4d10-9162-36e030c24619\") format('woff2');\n unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;\n}\n/* cyrillic-ext */\n@font-face {\n font-family: 'Plus Jakarta Sans';\n font-style: normal;\n font-weight: 500;\n font-display: swap;\n src: url(https://fonts.gstatic.com/s/plusjakartasans/v12/LDIoaomQNQcsA88c7O9yZ4KMCoOg4Ko70yyygA.woff2) format('woff2');\n unicode-range: U+0460-052F, U+1C80-1C8A, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F;\n}\n/* vietnamese */\n@font-face {\n font-family: 'Plus Jakarta Sans';\n font-style: normal;\n font-weight: 500;\n font-display: swap;\n src: url(https://fonts.gstatic.com/s/plusjakartasans/v12/LDIoaomQNQcsA88c7O9yZ4KMCoOg4Ko50yyygA.woff2) format('woff2');\n unicode-range: U+0102-0103, U+0110-0111, U+0128-0129, U+0168-0169, U+01A0-01A1, U+01AF-01B0, U+0300-0301, U+0303-0304, U+0308-0309, U+0323, U+0329, U+1EA0-1EF9, U+20AB;\n}\n/* latin-ext */\n@font-face {\n font-family: 'Plus Jakarta Sans';\n font-style: normal;\n font-weight: 500;\n font-display: swap;\n src: url(https://fonts.gstatic.com/s/plusjakartasans/v12/LDIoaomQNQcsA88c7O9yZ4KMCoOg4Ko40yyygA.woff2) format('woff2');\n unicode-range: U+0100-02BA, U+02BD-02C5, U+02C7-02CC, U+02CE-02D7, U+02DD-02FF, U+0304, U+0308, U+0329, U+1D00-1DBF, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C4, U+2113, U+2C60-2C7F, U+A720-A7FF;\n}\n/* latin */\n@font-face {\n font-family: 'Plus Jakarta Sans';\n font-style: normal;\n font-weight: 500;\n font-display: swap;\n src: url(\"e228819f-c09e-4d10-9162-36e030c24619\") format('woff2');\n unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;\n}\n/* cyrillic-ext */\n@font-face {\n font-family: 'Plus Jakarta Sans';\n font-style: normal;\n font-weight: 600;\n font-display: swap;\n src: url(https://fonts.gstatic.com/s/plusjakartasans/v12/LDIoaomQNQcsA88c7O9yZ4KMCoOg4Ko70yyygA.woff2) format('woff2');\n unicode-range: U+0460-052F, U+1C80-1C8A, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F;\n}\n/* vietnamese */\n@font-face {\n font-family: 'Plus Jakarta Sans';\n font-style: normal;\n font-weight: 600;\n font-display: swap;\n src: url(https://fonts.gstatic.com/s/plusjakartasans/v12/LDIoaomQNQcsA88c7O9yZ4KMCoOg4Ko50yyygA.woff2) format('woff2');\n unicode-range: U+0102-0103, U+0110-0111, U+0128-0129, U+0168-0169, U+01A0-01A1, U+01AF-01B0, U+0300-0301, U+0303-0304, U+0308-0309, U+0323, U+0329, U+1EA0-1EF9, U+20AB;\n}\n/* latin-ext */\n@font-face {\n font-family: 'Plus Jakarta Sans';\n font-style: normal;\n font-weight: 600;\n font-display: swap;\n src: url(https://fonts.gstatic.com/s/plusjakartasans/v12/LDIoaomQNQcsA88c7O9yZ4KMCoOg4Ko40yyygA.woff2) format('woff2');\n unicode-range: U+0100-02BA, U+02BD-02C5, U+02C7-02CC, U+02CE-02D7, U+02DD-02FF, U+0304, U+0308, U+0329, U+1D00-1DBF, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C4, Line truncated
|
"<!DOCTYPE html>\n<html lang=\"en\"><head>\n<meta charset=\"utf-8\">\n<title>Felhom system poster<\u002Ftitle>\n<script src=\"7425d086-bbee-40a4-a354-fbfba43c8a3d\"><\u002Fscript>\n<meta name=\"hz:slide-selector\" content=\"[data-canvas-width]\"><meta name=\"hz:canvas-width\" content=\"3200\"><meta name=\"hz:canvas-height\" content=\"2262\"><\u002Fhead>\n<body>\n<x-dc>\n<helmet>\n<link rel=\"preconnect\" href=\"https://fonts.googleapis.com\">\n<style>/* cyrillic-ext */\n@font-face {\n font-family: 'Plus Jakarta Sans';\n font-style: normal;\n font-weight: 400;\n font-display: swap;\n src: url(https://fonts.gstatic.com/s/plusjakartasans/v12/LDIoaomQNQcsA88c7O9yZ4KMCoOg4Ko70yyygA.woff2) format('woff2');\n unicode-range: U+0460-052F, U+1C80-1C8A, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F;\n}\n/* vietnamese */\n@font-face {\n font-family: 'Plus Jakarta Sans';\n font-style: normal;\n font-weight: 400;\n font-display: swap;\n src: url(https://fonts.gstatic.com/s/plusjakartasans/v12/LDIoaomQNQcsA88c7O9yZ4KMCoOg4Ko50yyygA.woff2) format('woff2');\n unicode-range: U+0102-0103, U+0110-0111, U+0128-0129, U+0168-0169, U+01A0-01A1, U+01AF-01B0, U+0300-0301, U+0303-0304, U+0308-0309, U+0323, U+0329, U+1EA0-1EF9, U+20AB;\n}\n/* latin-ext */\n@font-face {\n font-family: 'Plus Jakarta Sans';\n font-style: normal;\n font-weight: 400;\n font-display: swap;\n src: url(https://fonts.gstatic.com/s/plusjakartasans/v12/LDIoaomQNQcsA88c7O9yZ4KMCoOg4Ko40yyygA.woff2) format('woff2');\n unicode-range: U+0100-02BA, U+02BD-02C5, U+02C7-02CC, U+02CE-02D7, U+02DD-02FF, U+0304, U+0308, U+0329, U+1D00-1DBF, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C4, U+2113, U+2C60-2C7F, U+A720-A7FF;\n}\n/* latin */\n@font-face {\n font-family: 'Plus Jakarta Sans';\n font-style: normal;\n font-weight: 400;\n font-display: swap;\n src: url(\"e228819f-c09e-4d10-9162-36e030c24619\") format('woff2');\n unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;\n}\n/* cyrillic-ext */\n@font-face {\n font-family: 'Plus Jakarta Sans';\n font-style: normal;\n font-weight: 500;\n font-display: swap;\n src: url(https://fonts.gstatic.com/s/plusjakartasans/v12/LDIoaomQNQcsA88c7O9yZ4KMCoOg4Ko70yyygA.woff2) format('woff2');\n unicode-range: U+0460-052F, U+1C80-1C8A, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F;\n}\n/* vietnamese */\n@font-face {\n font-family: 'Plus Jakarta Sans';\n font-style: normal;\n font-weight: 500;\n font-display: swap;\n src: url(https://fonts.gstatic.com/s/plusjakartasans/v12/LDIoaomQNQcsA88c7O9yZ4KMCoOg4Ko50yyygA.woff2) format('woff2');\n unicode-range: U+0102-0103, U+0110-0111, U+0128-0129, U+0168-0169, U+01A0-01A1, U+01AF-01B0, U+0300-0301, U+0303-0304, U+0308-0309, U+0323, U+0329, U+1EA0-1EF9, U+20AB;\n}\n/* latin-ext */\n@font-face {\n font-family: 'Plus Jakarta Sans';\n font-style: normal;\n font-weight: 500;\n font-display: swap;\n src: url(https://fonts.gstatic.com/s/plusjakartasans/v12/LDIoaomQNQcsA88c7O9yZ4KMCoOg4Ko40yyygA.woff2) format('woff2');\n unicode-range: U+0100-02BA, U+02BD-02C5, U+02C7-02CC, U+02CE-02D7, U+02DD-02FF, U+0304, U+0308, U+0329, U+1D00-1DBF, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C4, U+2113, U+2C60-2C7F, U+A720-A7FF;\n}\n/* latin */\n@font-face {\n font-family: 'Plus Jakarta Sans';\n font-style: normal;\n font-weight: 500;\n font-display: swap;\n src: url(\"e228819f-c09e-4d10-9162-36e030c24619\") format('woff2');\n unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;\n}\n/* cyrillic-ext */\n@font-face {\n font-family: 'Plus Jakarta Sans';\n font-style: normal;\n font-weight: 600;\n font-display: swap;\n src: url(https://fonts.gstatic.com/s/plusjakartasans/v12/LDIoaomQNQcsA88c7O9yZ4KMCoOg4Ko70yyygA.woff2) format('woff2');\n unicode-range: U+0460-052F, U+1C80-1C8A, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F;\n}\n/* vietnamese */\n@font-face {\n font-family: 'Plus Jakarta Sans';\n font-style: normal;\n font-weight: 600;\n font-display: swap;\n src: url(https://fonts.gstatic.com/s/plusjakartasans/v12/LDIoaomQNQcsA88c7O9yZ4KMCoOg4Ko50yyygA.woff2) format('woff2');\n unicode-range: U+0102-0103, U+0110-0111, U+0128-0129, U+0168-0169, U+01A0-01A1, U+01AF-01B0, U+0300-0301, U+0303-0304, U+0308-0309, U+0323, U+0329, U+1EA0-1EF9, U+20AB;\n}\n/* latin-ext */\n@font-face {\n font-family: 'Plus Jakarta Sans';\n font-style: normal;\n font-weight: 600;\n font-display: swap;\n src: url(https://fonts.gstatic.com/s/plusjakartasans/v12/LDIoaomQNQcsA88c7O9yZ4KMCoOg4Ko40yyygA.woff2) format('woff2');\n unicode-range: U+0100-02BA, U+02BD-02C5, U+02C7-02CC, U+02CE-02D7, U+02DD-02FF, U+0304, U+0308, U+0329, U+1D00-1DBF, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C4, Line truncated
|
||||||
</script>
|
</script>
|
||||||
</body>
|
</body>
|
||||||
</html>
|
</html>
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
loop=/dev/loop2
|
||||||
|
--- lsblk p1 (the bug input)
|
||||||
|
{"blockdevices":[{"name":"loop2p1","fstype":null,"pttype":"gpt","mountpoint":null}]}
|
||||||
|
--- blkid -p p1 (empty part)
|
||||||
|
DEVNAME=/dev/loop2p1
|
||||||
|
PART_ENTRY_SCHEME=gpt
|
||||||
|
PART_ENTRY_UUID=2b13a20b-f5a4-4906-9e30-985c170a50ad
|
||||||
|
PART_ENTRY_TYPE=0fc63daf-8483-4772-8e79-3d69d8477de4
|
||||||
|
PART_ENTRY_NUMBER=1
|
||||||
|
PART_ENTRY_OFFSET=2048
|
||||||
|
PART_ENTRY_SIZE=1048576
|
||||||
|
PART_ENTRY_DISK=7:2
|
||||||
|
rc=0
|
||||||
|
--- blkid -p p2 (ext4)
|
||||||
|
TYPE=ext4
|
||||||
|
PART_ENTRY_SCHEME=gpt
|
||||||
|
PART_ENTRY_NUMBER=2
|
||||||
|
rc=0
|
||||||
|
Re-reading the partition table failed.: Invalid argument
|
||||||
|
--- blkid -p p1 (a dos table inside)
|
||||||
|
PTTYPE=dos
|
||||||
|
PART_ENTRY_SCHEME=gpt
|
||||||
|
rc=0
|
||||||
|
ls: cannot access '/root/r1093-scratch.img': No such file or directory
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
tester-1-d70be4 agent_update:
|
||||||
|
uploaded signed op to the hub jobs queue
|
||||||
|
tester-1-d70be4 agent_config_update:
|
||||||
|
uploaded signed op to the hub jobs queue
|
||||||
|
demo-felhom-8363b5 agent_update:
|
||||||
|
uploaded signed op to the hub jobs queue
|
||||||
|
demo-felhom-8363b5 agent_config_update:
|
||||||
|
uploaded signed op to the hub jobs queue
|
||||||
@@ -0,0 +1,11 @@
|
|||||||
|
tester-1: felhom-agent 0.159.2 | demo-felhom: felhom-agent 0.159.2 | 12:25:17
|
||||||
|
Oct 18 14:23:46 felhom felhom-agent[3912599]: time=2026-10-18T14:23:46.774+02:00 level=WARN msg="signedjobs: signed op COMPLETED" job=c9b02a094671155e op=agent_update
|
||||||
|
Oct 18 14:23:48 felhom felhom-agent[3912599]: time=2026-10-18T14:23:48.727+02:00 level=WARN msg="osupdate: capability probe after the config bundle" ok=75 total=75 degraded=""
|
||||||
|
Oct 18 14:23:48 felhom felhom-agent[3912599]: time=2026-10-18T14:23:48.728+02:00 level=WARN msg="signedjobs: signed op COMPLETED" job=fa2a707792f561ed op=agent_config_update
|
||||||
|
Oct 18 14:24:52 felhom felhom-agent[1371657]: time=2026-10-18T14:24:52.767+02:00 level=WARN msg="selfupdate: update committed" version=0.159.2 wrapper=""
|
||||||
|
Oct 18 14:25:00 demo-felhom felhom-agent[2419165]: time=2026-10-18T14:25:00.552+02:00 level=WARN msg="signedjobs: signed op COMPLETED" job=9d4397117eeed3b9 op=agent_update
|
||||||
|
Oct 18 14:25:01 demo-felhom felhom-agent[2419165]: time=2026-10-18T14:25:01.930+02:00 level=WARN msg="osupdate: capability probe after the config bundle" ok=75 total=75 degraded=""
|
||||||
|
Oct 18 14:25:01 demo-felhom felhom-agent[2419165]: time=2026-10-18T14:25:01.930+02:00 level=WARN msg="signedjobs: signed op COMPLETED" job=63f8321055a65cb8 op=agent_config_update
|
||||||
|
|
||||||
|
[exited with code 0]
|
||||||
|
Oct 18 14:26:05 demo-felhom felhom-agent[3800522]: time=2026-10-18T14:26:05.216+02:00 level=WARN msg="selfupdate: update committed" version=0.159.2 wrapper=""
|
||||||
@@ -0,0 +1,23 @@
|
|||||||
|
Oct 18 13:12:26 demo-hp felhom-agent[3566597]: time=2026-10-18T13:12:26.461+02:00 level=INFO msg="backup: space preflight passed" vmid=9201 target=local last_archive_bytes=6916890749 need_bytes=9719855260 avail_bytes=12123615232
|
||||||
|
Oct 18 13:12:27 demo-hp felhom-agent[3566597]: time=2026-10-18T13:12:27.526+02:00 level=INFO msg="local-api: backup reached snapshotted (app may resume)" vmid=9201 target=local job=backup-9201-1792321945880394998
|
||||||
|
Oct 18 13:18:22 demo-hp felhom-agent[3566597]: time=2026-10-18T13:18:22.853+02:00 level=INFO msg="janitor: stale-lock sweep deferred — a heavy operation is in flight" busy=backup:local
|
||||||
|
Oct 18 13:19:59 demo-hp felhom-agent[3566597]: time=2026-10-18T13:19:59.260+02:00 level=INFO msg="backup: completed" vmid=9201 target=local archive=local:backup/vzdump-lxc-9201-2026_10_18-13_12_26.tar.zst size_bytes=6938307162 uncovered_volumes=2
|
||||||
|
Oct 18 13:19:59 demo-hp felhom-agent[3566597]: time=2026-10-18T13:19:59.260+02:00 level=INFO msg="local-api: backup job complete" vmid=9201 target=local job=backup-9201-1792321945880394998 archive=local:backup/vzdump-lxc-9201-2026_10_18-13_12_26.tar.zst
|
||||||
|
Oct 18 13:19:59 demo-hp felhom-agent[3566597]: time=2026-10-18T13:19:59.266+02:00 level=INFO msg="local-api: no OS leg after a manual backup — it follows the night's own backup (R-899)" vmid=9201 job=backup-9201-1792321945880394998
|
||||||
|
== sudo felhom-thin-check lines since the trigger:
|
||||||
|
0
|
||||||
|
== sudo lines since the trigger (any helper):
|
||||||
|
26 COMMAND=/usr/bin/chown
|
||||||
|
36 COMMAND=/usr/bin/lsblk
|
||||||
|
98 COMMAND=/usr/bin/lxc-info
|
||||||
|
78 COMMAND=/usr/bin/mkdir
|
||||||
|
8 COMMAND=/usr/bin/wg
|
||||||
|
36 COMMAND=/usr/sbin/blkid
|
||||||
|
75 COMMAND=/usr/sbin/lvs
|
||||||
|
16 COMMAND=/usr/sbin/nft
|
||||||
|
57 COMMAND=/usr/sbin/pct
|
||||||
|
150 COMMAND=/usr/sbin/smartctl
|
||||||
|
|
||||||
|
[exited with code 0]
|
||||||
|
== control: the night's backup on agent 0.159.1 (2026-10-18 04:33 local) ran the check:
|
||||||
|
Oct 18 04:33:15 demo-hp sudo[1664888]: felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/local/sbin/felhom-thin-check pve data
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
== trigger 2026-10-18T11:11:36Z (local 2026-10-18 13:11:36)
|
||||||
|
200
|
||||||
|
{"data": {"started": true}, "ok": true}
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
felhom-agent 0.159.1
|
||||||
|
Oct 18 04:43:52 demo-hp felhom-agent[3320002]: time=2026-10-18T04:43:52.498+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: DONE rc=0 seconds=0 upgraded=0 (nothing to do)"
|
||||||
|
Oct 18 04:43:53 demo-hp felhom-agent[3320002]: time=2026-10-18T04:43:53.613+02:00 level=INFO msg="osupdate: DONE" run=20261018T024228Z layer=pve vmid=9201 ring=0 trigger=night outcome=nothing healthy=true reason="" upgraded=0 pending=1 not_covered=0 restart_needed=11 reboot_needed=false wrapper_seconds=16.3
|
||||||
|
Oct 18 05:23:06 demo-hp felhom-agent[3320002]: time=2026-10-18T05:23:06.521+02:00 level=INFO msg="backup: restore-test evaluated — nothing due" verdicts="local: no settled archive yet — nothing to prove (newborn or still settling); felhom-pbs: newest settled archive (landed 2026-10-15T20:38:47Z) is already proven"
|
||||||
|
Oct 18 11:23:06 demo-hp felhom-agent[3320002]: time=2026-10-18T11:23:06.557+02:00 level=INFO msg="backup: restore-test evaluated — nothing due" verdicts="local: no settled archive yet — nothing to prove (newborn or still settling); felhom-pbs: newest settled archive (landed 2026-10-15T20:38:47Z) is already proven"
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
wrote envelope to /dev/null
|
||||||
|
uploaded signed op to the hub jobs queue
|
||||||
|
wrote envelope to /dev/null
|
||||||
|
uploaded signed op to the hub jobs queue
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
felhom-agent 0.159.2 11:08:30
|
||||||
|
Oct 18 13:08:17 demo-hp felhom-agent[3320002]: time=2026-10-18T13:08:17.765+02:00 level=WARN msg="signedjobs: signed op COMPLETED" job=763e0b8971dff83c op=agent_update
|
||||||
|
Oct 18 13:08:18 demo-hp felhom-os-apply[3566414]: os-apply: BUNDLE DONE agent=0.159.2 written=0 same=30 self-check=ok signers-created=False
|
||||||
|
Oct 18 13:08:18 demo-hp felhom-agent[3320002]: time=2026-10-18T13:08:18.862+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: BUNDLE DONE agent=0.159.2 written=0 same=30 self-check=ok signers-created=False"
|
||||||
|
Oct 18 13:08:19 demo-hp felhom-agent[3320002]: time=2026-10-18T13:08:19.883+02:00 level=WARN msg="osupdate: capability probe after the config bundle" ok=75 total=75 degraded=""
|
||||||
|
Oct 18 13:08:19 demo-hp felhom-agent[3320002]: time=2026-10-18T13:08:19.884+02:00 level=WARN msg="signedjobs: signed op COMPLETED" job=268dfe26d949094f op=agent_config_update
|
||||||
|
|
||||||
|
[exited with code 0]
|
||||||
@@ -0,0 +1,7 @@
|
|||||||
|
demo-hp-bb76ea last report 2026-10-18 12:38:33 agent 0.159.2 health None
|
||||||
|
tester-1-d70be4 last report 2026-10-18 12:39:04 agent 0.159.2 health None
|
||||||
|
demo-felhom-8363b5 last report 2026-10-18 12:25:17 agent 0.159.2 health None
|
||||||
|
report keys: ['addresses', 'agent_sha256', 'agent_version', 'audit_tail', 'backups', 'capabilities', 'cloudflared', 'controller_supervisor', 'dr_recipe', 'foreign_key_archives', 'guest_disk_trim', 'guest_net', 'guests', 'host', 'host_id', 'leaf_fingerprint', 'mgmt_plane', 'oob', 'pbs_dr', 'pbs_snapshots', 'reported_at', 'restore_tests', 'storage_targets', 'system', 'wireguard']
|
||||||
|
demo-hp-bb76ea 2026-10-18 12:38:33 agent_version 0.159.2 degraded None cap_ok/total None problems None
|
||||||
|
tester-1-d70be4 2026-10-18 12:39:04 agent_version 0.159.2 degraded None cap_ok/total None problems None
|
||||||
|
demo-felhom-8363b5 2026-10-18 12:25:17 agent_version 0.159.2 degraded None cap_ok/total None problems None
|
||||||
@@ -0,0 +1,2 @@
|
|||||||
|
felhom-agent sha256 4d98868e922d6c10665a6c0da9323e8627ea9c589094e81d27d741e625d9d8f9
|
||||||
|
felhom-config-bundle.json sha256 477704ea2980948433ffc34bb59f008cc11410fd9683d386bfd3e2f085d55bc9
|
||||||
@@ -0,0 +1,102 @@
|
|||||||
|
[release-agent] building 0.159.2 …
|
||||||
|
[release-agent] built ok: sha256 4d98868e922d6c10665a6c0da9323e8627ea9c589094e81d27d741e625d9d8f9
|
||||||
|
[release-agent] config bundle built: sha256 477704ea2980948433ffc34bb59f008cc11410fd9683d386bfd3e2f085d55bc9
|
||||||
|
[release-agent] tagging v0.159.2 at 65c1994 …
|
||||||
|
[release-agent] publishing …
|
||||||
|
[publish-agent] publishing /tmp/felhom-agent-bMXfqp (16007752 bytes, sha256 4d98868e922d6c10…) → https://gitea.dooplex.hu/api/packages/admin/generic/felhom-agent/0.159.2/felhom-agent
|
||||||
|
[publish-agent] pre-delete existing artifact: HTTP 404 (404/204 expected)
|
||||||
|
[publish-agent] upload OK (HTTP 201)
|
||||||
|
[publish-agent] round-trip GET verified (sha256 matches)
|
||||||
|
[publish-agent] DONE. Record in the hub operator UI (Configs → Day-0 artifacts): agent 0.159.2 / 4d98868e922d6c10665a6c0da9323e862AGENT_VERSION=0.159.2
|
||||||
|
AGENT_SHA256=4d98868e922d6c10665a6c0da9323e8627ea9c589094e81d27d741e625d9d8f9
|
||||||
|
pre-push [felhom-agent]: running scripts/agent_gates.py --fast ...
|
||||||
|
agent_gates — 4 gate(s) [--fast]
|
||||||
|
--fast SKIPPED (deliberate periodic runs, never in a hook): published, gate-decoys
|
||||||
|
|
||||||
|
==============================================================================
|
||||||
|
== gate: reuse-refs (reuse_refs_check.py /mnt/5_hdd/felhom.eu/git/felhom-agent)
|
||||||
|
==============================================================================
|
||||||
|
note [felhom-agent] line 189: localapi/debuglogs_test.go (resolved by suffix → internal/localapi/debuglogs_test.go)
|
||||||
|
note [felhom-agent] line 205: hub/internal/store/dr_recipe.go (cross-repo → felhom.eu/hub/internal/store/dr_recipe.go)
|
||||||
|
OK [felhom-agent]: 117 cited paths — exact 115, suffix 1, ambiguous 0, cross-repo 1, FAILED 0 (siblings searched: app-catalog-felhom.eu, cc-secrets, felhom-controller, felhom.eu, felhom.eu-media, felhom.eu-sizing-push, felhom.eu-wt-sizing)
|
||||||
|
|
||||||
|
==============================================================================
|
||||||
|
== gate: instructions (instructions_gate.py /mnt/5_hdd/felhom.eu/git/felhom-agent)
|
||||||
|
==============================================================================
|
||||||
|
instructions_gate: /mnt/5_hdd/felhom.eu/git/felhom-agent
|
||||||
|
CLAUDE.md effective lines : 108 (ceiling 200)
|
||||||
|
version literals : 0
|
||||||
|
TEMPORARY blocks : 0
|
||||||
|
rule files : 6 (5 path-scoped)
|
||||||
|
workspace file : SYMLINK -> felhom.eu/documentation/runbooks/workspace-CLAUDE.md (resolves to the versioned copy)
|
||||||
|
memory index : 200 lines (ceiling 200), 24953 bytes (ceiling 25600)
|
||||||
|
memory index content : 28 version literal(s), 3 host address(es), 0 expired statement(s) [WARN only]
|
||||||
|
memory topic files : 181 indexed, 4 orphaned, 41 archived
|
||||||
|
register citations : 6 cited, 1109 register items known
|
||||||
|
|
||||||
|
WARNING: /mnt/5_hdd/felhom.eu/git/.claude-memory/MEMORY.md:31: version literal '0.14.0' — the fleet is not uniform, so it is stale within a day. Not a failure: Claude writes this file between sessions, so this warning is aimed at the model that will next edit it, not at whoever is pushing.
|
||||||
|
- **[restic 0.14.0 `--verify` is size+mtime, NOT content](restic-0140-verify-and-lock-sema
|
||||||
|
WARNING: /mnt/5_hdd/felhom.eu/git/.claude-memory/MEMORY.md:89: version literal '0.142.0' — the fleet is not uniform, so it is stale within a day. Not a failure: Claude writes this file between sessions, so this warning is aimed at the model that will next edit it, not at whoever is pushing.
|
||||||
|
- [Offsite continuity](offsite-continuity-2026-07-17.md) — ctrl 0.142.0 orphan guard + hub
|
||||||
|
WARNING: /mnt/5_hdd/felhom.eu/git/.claude-memory/MEMORY.md:89: version literal '0.60.0' — the fleet is not uniform, so it is stale within a day. Not a failure: Claude writes this file between sessions, so this warning is aimed at the model that will next edit it, not at whoever is pushing.
|
||||||
|
- [Offsite continuity](offsite-continuity-2026-07-17.md) — ctrl 0.142.0 orphan guard + hub
|
||||||
|
WARNING: /mnt/5_hdd/felhom.eu/git/.claude-memory/MEMORY.md:96: version literal '0.152.0' — the fleet is not uniform, so it is stale within a day. Not a failure: Claude writes this file between sessions, so this warning is aimed at the model that will next edit it, not at whoever is pushing.
|
||||||
|
- **[Hub operator UI: CC uses its token HUB_TOKEN, not the password](hub-operator-ui-basic
|
||||||
|
WARNING: /mnt/5_hdd/felhom.eu/git/.claude-memory/MEMORY.md:100: version literal '0.90.1' — the fleet is not uniform, so it is stale within a day. Not a failure: Claude writes this file between sessions, so this warning is aimed at the model that will next edit it, not at whoever is pushing.
|
||||||
|
- [Agent 0.90.1 is wrapper-only](agent-0901-is-wrapper-only.md) — do NOT publish the binar
|
||||||
|
WARNING: /mnt/5_hdd/felhom.eu/git/.claude-memory/MEMORY.md:102: version literal '0.144.0' — the fleet is not uniform, so it is stale within a day. Not a failure: Claude writes this file between sessions, so this warning is aimed at the model that will next edit it, not at whoever is pushing.
|
||||||
|
- [SMB „Megosztás" R-7 slice 1](smb-sharing-r7-slice1-2026-07-18.md) — ctrl 0.144.0 + felh
|
||||||
|
WARNING: /mnt/5_hdd/felhom.eu/git/.claude-memory/MEMORY.md: ... and 22 more version literal(s) — full list from the tally counts above.
|
||||||
|
WARNING: /mnt/5_hdd/felhom.eu/git/.claude-memory/MEMORY.md:58: host address '192.168.0.192' — operations/nodes.md is the single home for these. Not a failure: Claude writes this file between sessions, so this warning is aimed at the model that will next edit it, not at whoever is pushing.
|
||||||
|
- [hub.felhom.eu resolves to the LAN here](hub-resolves-to-lan-on-dooplex-network.md) — 19
|
||||||
|
WARNING: /mnt/5_hdd/felhom.eu/git/.claude-memory/MEMORY.md:77: host address '167.233.158.164' — operations/nodes.md is the single home for these. Not a failure: Claude writes this file between sessions, so this warning is aimed at the model that will next edit it, not at whoever is pushing.
|
||||||
|
- [Offsite PBS box RAM ceiling](offsite-pbs-box-ram-ceiling-2026-07-27.md) — **root SSH =
|
||||||
|
WARNING: /mnt/5_hdd/felhom.eu/git/.claude-memory/MEMORY.md:185: host address '192.168.0.180' — operations/nodes.md is the single home for these. Not a failure: Claude writes this file between sessions, so this warning is aimed at the model that will next edit it, not at whoever is pushing.
|
||||||
|
- **CC runs ON DooPlex** (192.168.0.180) — repos `/mnt/5_hdd/felhom.eu/git/felhom-*`, buil
|
||||||
|
WARNING: /mnt/5_hdd/felhom.eu/git/.claude-memory: 4 top-level topic file(s) are not referenced by MEMORY.md, so nothing will ever read them: f10-demo-hp-offsite-resolved-2026-07-23.md, launcher-v0163-2026-07-24.md, peti-return-p1-stop-2026-07-13.md, stopped-not-fault-v0164-2026-07-24.md
|
||||||
|
|
||||||
|
instructions_gate: OK (11 warning(s))
|
||||||
|
|
||||||
|
==============================================================================
|
||||||
|
== gate: release-complete (check-release-complete.py)
|
||||||
|
==============================================================================
|
||||||
|
check-release-complete — the newest CHANGELOG version is a complete release
|
||||||
|
newest CHANGELOG version: v0.159.1
|
||||||
|
ok tag v0.159.1 -> 5c3b218822, an ancestor of HEAD
|
||||||
|
ok package 0.159.1 is downloadable
|
||||||
|
|
||||||
|
check-release-complete: v0.159.1 is tagged, placed and published.
|
||||||
|
|
||||||
|
==============================================================================
|
||||||
|
== gate: observations (observations_gate.py /mnt/5_hdd/felhom.eu/git/felhom-agent)
|
||||||
|
==============================================================================
|
||||||
|
observations gate OK — REPORT.md has no observations section (nothing to check)
|
||||||
|
|
||||||
|
==============================================================================
|
||||||
|
== summary
|
||||||
|
==============================================================================
|
||||||
|
reuse-refs OK (exit 0)
|
||||||
|
instructions OK (exit 0)
|
||||||
|
release-complete OK (exit 7ea9c589094e81d27d741e625d9d8f9
|
||||||
|
[release-agent] config bundle published (HTTP 201)
|
||||||
|
[release-agent] pushing v0.159.2 …
|
||||||
|
remote: . Processing 1 references
|
||||||
|
remote: Processed 1 references in total
|
||||||
|
To https://gitea.dooplex.hu0)
|
||||||
|
observations OK (exit 0)
|
||||||
|
|
||||||
|
all agent gates OK
|
||||||
|
pre-push [felhom-agent]: gates OK - push proceeding.
|
||||||
|
|
||||||
|
RELEASED — and installable, verified by download, not by this script's own say-so.
|
||||||
|
|
||||||
|
version : 0.159.2
|
||||||
|
tag : v0.159.2
|
||||||
|
sha256 : 4d98868e922d6c10665a6c0da9323e8627ea9c589094e81d27d741e625d9d8f9
|
||||||
|
bundle : 477704ea2980948433ffc34bb59f008cc11410fd9683d386bfd3e2f085d55bc9 (felhom-config-bundle.json — vouch it with the agent)
|
||||||
|
|
||||||
|
NOT VOUCHED. Vouching is what points machines at this version and stays you/admin/felhom-agent.git
|
||||||
|
* [new tag] v0.159.2 -> v0.159.2
|
||||||
|
[release-agent] verifying by independent download …
|
||||||
|
r deliberate act:
|
||||||
|
hub operator UI → Configs → Day-0 artifacts. Until then boxes keep installing the previous one.
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
== published installer read back 2026-10-18T12:32:10Z
|
||||||
|
SCRIPT_VERSION="1.34.0" # the SINGLE version source (F-1): -h and the run banners follow it.
|
||||||
|
served sha256 df722ee49f38da595e7c7017aa932b45853696c5282bc408b67a5fb8f141b67a
|
||||||
|
tag sha256 df722ee49f38da595e7c7017aa932b45853696c5282bc408b67a5fb8f141b67a
|
||||||
@@ -0,0 +1,28 @@
|
|||||||
|
# Website calculator — Immich before/after (2026-10-18)
|
||||||
|
|
||||||
|
Immich peak: **3,290 MB → 3,983 MB** (the kernel in-use figure from the 4,000-photo import, all four containers; the heap alone peaked at 2,777 MB). Immich typical: **1,557 MB, unchanged** (rest after the import: 1,230 MB heap — the raise-only rule keeps the higher). Headroom ×1.5, guest rule and platform unchanged.
|
||||||
|
|
||||||
|
## Immich alone — peak with the platform: 3,760 → 4,453 MB
|
||||||
|
|
||||||
|
| Machine | guest memory | before | after |
|
||||||
|
|---|---|---|---|
|
||||||
|
| 8 GB | 4,096 MB | fits, but tight | does not fit **(changes)** |
|
||||||
|
| 12 GB | 7,580 MB | fits well | fits well |
|
||||||
|
| 16 GB | 11,676 MB | fits well | fits well |
|
||||||
|
| 24 GB | 19,868 MB | fits well | fits well |
|
||||||
|
| 32 GB | 28,060 MB | fits well | fits well |
|
||||||
|
| 48 GB | 44,444 MB | fits well | fits well |
|
||||||
|
| 64 GB | 60,828 MB | fits well | fits well |
|
||||||
|
|
||||||
|
## Family starter set — peak with the platform: 6,021 → 6,714 MB
|
||||||
|
|
||||||
|
| Machine | guest memory | before | after |
|
||||||
|
|---|---|---|---|
|
||||||
|
| 8 GB | 4,096 MB | does not fit | does not fit |
|
||||||
|
| 12 GB | 7,580 MB | fits, but tight | fits, but tight |
|
||||||
|
| 16 GB | 11,676 MB | fits well | fits well |
|
||||||
|
| 24 GB | 19,868 MB | fits well | fits well |
|
||||||
|
| 32 GB | 28,060 MB | fits well | fits well |
|
||||||
|
| 48 GB | 44,444 MB | fits well | fits well |
|
||||||
|
| 64 GB | 60,828 MB | fits well | fits well |
|
||||||
|
|
||||||
@@ -0,0 +1,27 @@
|
|||||||
|
=== demo-hp (enkisfelhom.hu)
|
||||||
|
containers: 24 running; not healthy/running: traefik Up 2 days;
|
||||||
|
unhealthy: 0
|
||||||
|
2026/10/18 11:09:36 updater.go:855: [INFO] [selfupdate] Post-update startup: update successful (0.329.0 → 0.330.0)
|
||||||
|
allow-list refusals since start: 0
|
||||||
|
2026/10/18 11:09:36 sync.go:142: [INFO] [sync] Initial sync: Sablonok naprakészek — nincs változás
|
||||||
|
installed apps (stacks with app.yaml): 12
|
||||||
|
public https://felhom.enkisfelhom.hu/login: 200
|
||||||
|
public https://files.enkisfelhom.hu/: 200
|
||||||
|
=== demo-felhom (demo-felhom.eu)
|
||||||
|
containers: 9 running; not healthy/running: traefik Up 2 days;
|
||||||
|
unhealthy: 0
|
||||||
|
2026/10/18 11:09:25 [INFO] [selfupdate] Post-update startup: update successful (0.329.0 → 0.330.0)
|
||||||
|
allow-list refusals since start: 0
|
||||||
|
2026/10/18 11:09:26 [INFO] [sync] Initial sync: Sablonok naprakészek — nincs változás
|
||||||
|
installed apps (stacks with app.yaml): 5
|
||||||
|
public https://felhom.demo-felhom.eu/login: 200
|
||||||
|
public https://files.demo-felhom.eu/: 200
|
||||||
|
=== tester-1 (enkicsifelhom.hu)
|
||||||
|
containers: 10 running; not healthy/running: traefik Up 2 days;
|
||||||
|
unhealthy: 0
|
||||||
|
2026/10/18 11:09:35 [INFO] [selfupdate] Post-update startup: update successful (0.329.0 → 0.330.0)
|
||||||
|
allow-list refusals since start: 0
|
||||||
|
2026/10/18 11:09:36 [INFO] [sync] Initial sync: Sablonok naprakészek — nincs változás
|
||||||
|
installed apps (stacks with app.yaml): 3
|
||||||
|
public https://felhom.enkicsifelhom.hu/login: 200
|
||||||
|
public https://files.enkicsifelhom.hu/: 200
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
== global floor 2026-10-18T11:10:59Z
|
||||||
|
HTTP/1.1 303 See Other
|
||||||
|
Location: /configuration?flash=floor_set
|
||||||
|
clear tester-1:
|
||||||
|
HTTP/1.1 303 See Other
|
||||||
|
Location: /customers/tester-1?flash=floor_set
|
||||||
|
clear demo-hp:
|
||||||
|
HTTP/1.1 303 See Other
|
||||||
|
Location: /customers/demo-hp?flash=floor_set
|
||||||
|
clear demo-felhom:
|
||||||
|
HTTP/1.1 303 See Other
|
||||||
|
Location: /customers/demo-felhom?flash=floor_set
|
||||||
@@ -0,0 +1,9 @@
|
|||||||
|
set tester-1:
|
||||||
|
HTTP/1.1 303 See Other
|
||||||
|
Location: /customers/tester-1?flash=floor_set
|
||||||
|
set demo-hp:
|
||||||
|
HTTP/1.1 303 See Other
|
||||||
|
Location: /customers/demo-hp?flash=floor_set
|
||||||
|
set demo-felhom:
|
||||||
|
HTTP/1.1 303 See Other
|
||||||
|
Location: /customers/demo-felhom?flash=floor_set
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
('min_controller_version', '0.330.0')
|
||||||
|
('min_controller_version_declared_min_agent', '0.330.0=0.158.1')
|
||||||
|
per-box floors set: 0
|
||||||
@@ -0,0 +1,2 @@
|
|||||||
|
OK http://torrent.demo-felhom.eu/r1033-check?q=1 -> 301 https://torrent.demo-felhom.eu/r1033-check?q=1
|
||||||
|
== every public name redirects http -> https
|
||||||
@@ -0,0 +1,20 @@
|
|||||||
|
# P8 — old-restic check: NOT RUN (the clean-up window has not opened). Read 2026-10-18T14:33:43Z.
|
||||||
|
|
||||||
|
## demo-hp's clean-up windows (hub DB with its WAL, one tar)
|
||||||
|
id|customer|opened_at|closes_by|closed_at|box_result
|
||||||
|
9|demo-hp|2026-10-12 02:16:53|2026-10-12 02:36:53|2026-10-12 02:17:06|pruned
|
||||||
|
6|demo-hp|2026-10-05 05:25:53|2026-10-05 05:45:53|2026-10-05 05:26:04|pruned
|
||||||
|
2|demo-hp|2026-10-04 05:36:14|2026-10-04 05:56:14|2026-10-04 05:36:17|nothing
|
||||||
|
|
||||||
|
## The rule (hub/internal/offsitekeys/service.go:255)
|
||||||
|
windowCadence = 6*24*time.Hour + 12*time.Hour // "weekly", with slack for the night chain's drift
|
||||||
|
Last window 2026-10-12 02:16:53 UTC + 6 d 12 h = 2026-10-18 14:16:53 UTC (the STATUS time).
|
||||||
|
The box asks for a window only AFTER its nightly off-site run (controller backup/offbox_window.go header).
|
||||||
|
|
||||||
|
## demo-hp guest 9201 controller (docker logs since 06:00 UTC, [offbox] lines + scheduler)
|
||||||
|
felhom-controller gitea.dooplex.hu/admin/felhom-controller:0.330.0 Up 3 hours (healthy)
|
||||||
|
2026/10/18 11:09:36 scheduler.go:150: [INFO] [scheduler] Daily job offbox-backup scheduled for 2026-10-19 04:15 CEST
|
||||||
|
2026/10/18 11:09:36 scheduler.go:85: [DEBUG] [scheduler] daily job registered: name="offbox-backup" schedule="04:15" nextRun=2026-10-19T04:15:00+02:00 totalJobs=15
|
||||||
|
2026/10/18 11:09:36 scheduler.go:85: [DEBUG] [scheduler] daily job offbox-backup: next run at 2026-10-19 04:15:00 CEST (waiting 15h5m23s)
|
||||||
|
|
||||||
|
Next run: 2026-10-19 04:15 CEST = 02:15 UTC; the window opens right after it. No window was asked for or granted by CC.
|
||||||
@@ -0,0 +1,23 @@
|
|||||||
|
Mon Oct 19 01:24:39 UTC 2026
|
||||||
|
200
|
||||||
|
{"data": {"started": true}, "ok": true}
|
||||||
|
200
|
||||||
|
{
|
||||||
|
"data": {
|
||||||
|
"backup": {
|
||||||
|
"target_id": "local",
|
||||||
|
"vmid": 9201,
|
||||||
|
"archive": "local:backup/vzdump-lxc-9201-2026_10_19-03_25_27.tar.zst",
|
||||||
|
"mode": "snapshot",
|
||||||
|
"crash_consistent": true,
|
||||||
|
"size_bytes": 6864786948,
|
||||||
|
"success": true,
|
||||||
|
"started_at": "2026-10-19T01:25:26Z",
|
||||||
|
"duration_seconds": 443.050964897
|
||||||
|
},
|
||||||
|
"error": "",
|
||||||
|
"job_id": "backup-9201-1792373126635435331",
|
||||||
|
"phase": "done"
|
||||||
|
},
|
||||||
|
"ok": true
|
||||||
|
}
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
Mon Oct 19 01:52:00 UTC 2026
|
||||||
|
tester-1-d70be4 agent_update:
|
||||||
|
signed: op=agent_update host=tester-1-d70be4 guest="" key_id=felhom-op-1 nonce=3516434bfa0c5c47295e3de0a8bd20c7 expires=2026-10-19T02:37:00Z
|
||||||
|
uploaded signed op to the hub jobs queue
|
||||||
|
tester-1-d70be4 agent_config_update:
|
||||||
|
signed: op=agent_config_update host=tester-1-d70be4 guest="" key_id=felhom-op-1 nonce=2dfc5b3e216030c82add18ba69899b41 expires=2026-10-19T02:37:00Z
|
||||||
|
uploaded signed op to the hub jobs queue
|
||||||
|
demo-felhom-8363b5 agent_update:
|
||||||
|
signed: op=agent_update host=demo-felhom-8363b5 guest="" key_id=felhom-op-1 nonce=8f3a6c280cb124c6f85a4039c5b112dc expires=2026-10-19T02:37:00Z
|
||||||
|
uploaded signed op to the hub jobs queue
|
||||||
|
demo-felhom-8363b5 agent_config_update:
|
||||||
|
signed: op=agent_config_update host=demo-felhom-8363b5 guest="" key_id=felhom-op-1 nonce=dc2ead6937a38d0a2d8acd4abf57bef9 expires=2026-10-19T02:37:00Z
|
||||||
|
uploaded signed op to the hub jobs queue
|
||||||
@@ -0,0 +1,17 @@
|
|||||||
|
2026-10-19T01:55:42Z tester-1: felhom-agent 0.160.0 | demo-felhom: felhom-agent 0.160.0
|
||||||
|
Oct 19 03:54:05 felhom felhom-agent[1371657]: time=2026-10-19T03:54:05.642+02:00 level=WARN msg="signedjobs: signed op COMPLETED" job=a254c4ec38459c7c op=agent_update
|
||||||
|
Oct 19 03:54:06 felhom felhom-agent[1371657]: time=2026-10-19T03:54:06.786+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: BUNDLE DONE agent=0.160.0 written=0 same=30 self-check=ok signers-created=False"
|
||||||
|
Oct 19 03:54:07 felhom felhom-agent[1371657]: time=2026-10-19T03:54:07.652+02:00 level=WARN msg="osupdate: capability probe after the config bundle" ok=75 total=75 degraded=""
|
||||||
|
Oct 19 03:54:07 felhom felhom-agent[1371657]: time=2026-10-19T03:54:07.653+02:00 level=WARN msg="signedjobs: signed op COMPLETED" job=a66f7fc0b9fd8fb6 op=agent_config_update
|
||||||
|
Oct 19 03:54:08 felhom felhom-agent[2443463]: time=2026-10-19T03:54:08.649+02:00 level=INFO msg="backup tier armed" target=local cadence=24h0m0s keep_last=3 prune="keep-daily=3" wait_timeout=30m0s prune_pbs_allowed=false primary=true
|
||||||
|
Oct 19 03:54:08 felhom felhom-agent[2443463]: time=2026-10-19T03:54:08.649+02:00 level=INFO msg="backup tier armed" target=felhom-pbs cadence=168h0m0s keep_last=0 prune="" wait_timeout=12h0m0s prune_pbs_allowed=false primary=false
|
||||||
|
Oct 19 03:55:10 felhom felhom-agent[2443463]: time=2026-10-19T03:55:10.315+02:00 level=WARN msg="selfupdate: update committed" version=0.160.0 wrapper=""
|
||||||
|
Oct 19 03:55:15 demo-felhom felhom-os-apply[498026]: os-apply: BUNDLE DONE agent=0.160.0 written=0 same=30 self-check=ok signers-created=False
|
||||||
|
Oct 19 03:55:15 demo-felhom felhom-agent[3800522]: time=2026-10-19T03:55:15.698+02:00 level=INFO msg="osupdate: wrapper" line="os-apply: BUNDLE DONE agent=0.160.0 written=0 same=30 self-check=ok signers-created=False"
|
||||||
|
Oct 19 03:55:16 demo-felhom felhom-agent[3800522]: time=2026-10-19T03:55:16.340+02:00 level=WARN msg="osupdate: capability probe after the config bundle" ok=75 total=75 degraded=""
|
||||||
|
Oct 19 03:55:16 demo-felhom felhom-agent[3800522]: time=2026-10-19T03:55:16.340+02:00 level=WARN msg="signedjobs: signed op COMPLETED" job=7dde89f299a0684c op=agent_config_update
|
||||||
|
Oct 19 03:55:17 demo-felhom felhom-agent[498196]: time=2026-10-19T03:55:17.698+02:00 level=INFO msg="backup tier armed" target=felhom-backup cadence=24h0m0s keep_last=3 prune="keep-daily=3" wait_timeout=30m0s prune_pbs_allowed=false primary=true
|
||||||
|
Oct 19 03:55:17 demo-felhom felhom-agent[498196]: time=2026-10-19T03:55:17.698+02:00 level=INFO msg="backup tier armed" target=felhom-pbs cadence=168h0m0s keep_last=0 prune="" wait_timeout=12h0m0s prune_pbs_allowed=false primary=false
|
||||||
|
Oct 19 03:56:19 demo-felhom felhom-agent[498196]: time=2026-10-19T03:56:19.667+02:00 level=WARN msg="selfupdate: update committed" version=0.160.0 wrapper=""
|
||||||
|
|
||||||
|
[exited with code 0]
|
||||||
@@ -0,0 +1,29 @@
|
|||||||
|
Volid Format Type Size VMID
|
||||||
|
local:backup/vzdump-lxc-9100-2026_08_21-17_59_15.tar.zst tar.zst backup 656970239 9100
|
||||||
|
local:backup/vzdump-lxc-9201-2026_10_19-03_25_27.tar.zst tar.zst backup 6864786948 9201
|
||||||
|
local:backup/felhom-golden-0.236.0.tar.zst tar.zst backup 654115664
|
||||||
|
Oct 19 03:23:37 demo-hp felhom-agent[2622137]: time=2026-10-19T03:23:37.628+02:00 level=INFO msg="backup tier armed" target=local cadence=24h0m0s keep_last=1 prune="keep-daily=1" wait_timeout=30m0s prune_pbs_allowed=false primary=true
|
||||||
|
Oct 19 03:23:37 demo-hp felhom-agent[2622137]: time=2026-10-19T03:23:37.628+02:00 level=INFO msg="backup tier armed" target=felhom-pbs cadence=168h0m0s keep_last=0 prune="" wait_timeout=12h0m0s prune_pbs_allowed=false primary=false
|
||||||
|
Oct 19 03:24:39 demo-hp felhom-agent[2622137]: time=2026-10-19T03:24:39.730+02:00 level=WARN msg="selfupdate: update committed" version=0.160.0 wrapper=""
|
||||||
|
Oct 19 03:32:49 demo-hp felhom-agent[2622137]: time=2026-10-19T03:32:49.686+02:00 level=INFO msg="backup: completed" vmid=9201 target=local archive=local:backup/vzdump-lxc-9201-2026_10_19-03_25_27.tar.zst size_bytes=6864786948 uncovered_volumes=2
|
||||||
|
Oct 19 03:32:49 demo-hp felhom-agent[2622137]: time=2026-10-19T03:32:49.686+02:00 level=INFO msg="local-api: backup job complete" vmid=9201 target=local job=backup-9201-1792373126635435331 archive=local:backup/vzdump-lxc-9201-2026_10_19-03_25_27.tar.zst
|
||||||
|
-rw------- 1 felhom-agent felhom-agent 323 Oct 19 03:32 /var/lib/felhom-agent/backup-success-state.json
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"target": "felhom-pbs",
|
||||||
|
"vmid": 9201,
|
||||||
|
"started_at": "2026-10-15T20:38:46Z",
|
||||||
|
"recent": [
|
||||||
|
"2026-10-15T20:38:46Z"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"target": "local",
|
||||||
|
"vmid": 9201,
|
||||||
|
"started_at": "2026-10-19T01:25:26Z",
|
||||||
|
"recent": [
|
||||||
|
"2026-10-18T11:12:25Z",
|
||||||
|
"2026-10-19T01:25:26Z"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
]
|
||||||
@@ -0,0 +1,33 @@
|
|||||||
|
Mon Oct 19 01:22:11 UTC 2026
|
||||||
|
felhom-agent 0.159.2
|
||||||
|
demo-hp-bb76ea
|
||||||
|
Volid Format Type Size VMID
|
||||||
|
local:backup/vzdump-lxc-9100-2026_08_21-17_59_15.tar.zst tar.zst backup 656970239 9100
|
||||||
|
local:backup/vzdump-lxc-9201-2026_10_18-13_12_26.tar.zst tar.zst backup 6938307162 9201
|
||||||
|
local:backup/felhom-golden-0.236.0.tar.zst tar.zst backup 654115664
|
||||||
|
"local_backup_retention": 1
|
||||||
|
VMID NAME STATUS MEM(MB) BOOTDISK(GB) PID
|
||||||
|
341 night1004-tester1 running 8192 200.00 1851
|
||||||
|
perl: warning: Setting locale failed.
|
||||||
|
perl: warning: Please check that your locale settings:
|
||||||
|
LANGUAGE = (unset),
|
||||||
|
LC_ALL = (unset),
|
||||||
|
LC_CTYPE = "UTF-8",
|
||||||
|
LC_NUMERIC = (unset),
|
||||||
|
LC_COLLATE = (unset),
|
||||||
|
LC_TIME = (unset),
|
||||||
|
LC_MESSAGES = (unset),
|
||||||
|
LC_MONETARY = (unset),
|
||||||
|
LC_ADDRESS = (unset),
|
||||||
|
LC_IDENTIFICATION = (unset),
|
||||||
|
LC_MEASUREMENT = (unset),
|
||||||
|
LC_PAPER = (unset),
|
||||||
|
LC_TELEPHONE = (unset),
|
||||||
|
LC_NAME = (unset),
|
||||||
|
LANG = "en_US.UTF-8"
|
||||||
|
are supported and installed on your system.
|
||||||
|
perl: warning: Falling back to a fallback locale ("en_US.UTF-8").
|
||||||
|
VMID Status Lock Name
|
||||||
|
9201 running demo-hp
|
||||||
|
9202 running demo-hp-scratch
|
||||||
|
9401 stopped upgrade-harness
|
||||||
@@ -0,0 +1,9 @@
|
|||||||
|
Mon Oct 19 01:22:47 UTC 2026
|
||||||
|
demo-hp-bb76ea agent_update:
|
||||||
|
signed: op=agent_update host=demo-hp-bb76ea guest="" key_id=felhom-op-1 nonce=71396ac328c9ea3192d027b830ec0990 expires=2026-10-19T02:07:47Z
|
||||||
|
wrote envelope to /dev/null
|
||||||
|
uploaded signed op to the hub jobs queue
|
||||||
|
demo-hp-bb76ea agent_config_update:
|
||||||
|
signed: op=agent_config_update host=demo-hp-bb76ea guest="" key_id=felhom-op-1 nonce=603f1d8ed0937492cf01a05146fc9e48 expires=2026-10-19T02:07:47Z
|
||||||
|
wrote envelope to /dev/null
|
||||||
|
uploaded signed op to the hub jobs queue
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
Mon Oct 19 01:47:33 UTC 2026
|
||||||
|
code 0 (the stored demo-hp code, before):
|
||||||
|
HTTP 200 opens; restic_pw_sha256 8a9e33aa
|
||||||
|
control, a wrong code:
|
||||||
|
HTTP 400 the recovery code did not open the sealed bundle — nothing was written
|
||||||
@@ -0,0 +1,11 @@
|
|||||||
|
01:47:43Z start 200 {"data":{"job_id":"escrow-1792374463410689900","phase":"running"},"error":"","ok":true}
|
||||||
|
|
||||||
|
status 200 {"phase": "done", "claimable": true, "uploaded": false, "restic_pw_sealed": true, "detail": ""}
|
||||||
|
claim 200 code words: 10 {"confirm_required": true, "verify_positions": [3, 10]}
|
||||||
|
status after claim {"phase": "done", "claimed": true, "uploaded": false}
|
||||||
|
confirm with one wrong word 422 A megadott szavak nem egyeznek — ellenőrizd a felírt kódot.
|
||||||
|
01:47:47Z confirm with the right words 200 {"uploaded": true}
|
||||||
|
code 1 after the confirm:
|
||||||
|
HTTP 200 opens; restic_pw_sha256 8a9e33aa
|
||||||
|
code 0 after the confirm:
|
||||||
|
HTTP 422 the recovery code is correct, but it belongs to an EARLIER sealed package (superseded 2026-10-19T01:47:47Z), not the one
|
||||||
@@ -0,0 +1,9 @@
|
|||||||
|
01:48:09Z start 200 {"data":{"job_id":"escrow-1792374489199785658","phase":"running"},"error":"","ok":true}
|
||||||
|
|
||||||
|
status 200 {"phase": "done", "claimable": true, "uploaded": false, "restic_pw_sealed": true, "detail": ""}
|
||||||
|
claim 200 code words: 10 {"confirm_required": true, "verify_positions": [5, 10]}
|
||||||
|
status after claim {"phase": "done", "claimed": true, "uploaded": false}
|
||||||
|
after the abandon — code 1 (stored):
|
||||||
|
HTTP 200 opens; restic_pw_sha256 8a9e33aa
|
||||||
|
code 2 (shown, never typed back):
|
||||||
|
HTTP 400 the recovery code did not open the sealed bundle — nothing was written
|
||||||
@@ -0,0 +1,23 @@
|
|||||||
|
time=2026-10-19T03:47:33.706+02:00 level=INFO msg="local-api: recovering the offsite repository password from the sealed escrow (R via body, never logged/persisted)" vmid=9201
|
||||||
|
time=2026-10-19T03:47:34.786+02:00 level=INFO msg="local-api: offsite repository password RECOVERED from the sealed escrow — returning that field ONLY (the tunnel token, the PBS token and the WG key stay inside the agent and are not returned)" vmid=9201 rest
|
||||||
|
time=2026-10-19T03:47:35.044+02:00 level=INFO msg="local-api: recovering the offsite repository password from the sealed escrow (R via body, never logged/persisted)" vmid=9201
|
||||||
|
time=2026-10-19T03:47:36.053+02:00 level=WARN msg="local-api: offsite key recovery: the fetched bundle did not open with the supplied recovery code" vmid=9201 err="escrow: the recovery code did not unwrap the identity escrow (wrong recovery code, or a corrupt
|
||||||
|
time=2026-10-19T03:47:43.409+02:00 level=INFO msg="local-api: staged offsite restic repo password for escrow (field name only)" vmid=9201
|
||||||
|
time=2026-10-19T03:47:43.410+02:00 level=INFO msg="local-api: escrow ceremony started (controller-driven)" vmid=9201 job_id=escrow-1792374463410689900
|
||||||
|
felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/felhom-agent --config /etc/felhom-agent/agent.json --selftest=escrow-create --upload --output=json
|
||||||
|
time=2026-10-19T03:47:45.992+02:00 level=INFO msg="local-api: escrow ceremony complete — R claimable (in-memory, one-shot); package uploads at the claim" job_id=escrow-1792374463410689900 restic_pw_sealed=true uploaded=false claim_ttl_s=600 duration_ms=2581
|
||||||
|
time=2026-10-19T03:47:47.507+02:00 level=INFO msg="local-api: escrow ceremony R claimed — the package waits for the typed-back words (R-1151)" vmid=9201 job_id=escrow-1792374463410689900 confirm_ttl_s=1800
|
||||||
|
time=2026-10-19T03:47:47.550+02:00 level=INFO msg="local-api: escrow confirm — the typed words do not match; still waiting (R-1151)" vmid=9201
|
||||||
|
time=2026-10-19T03:47:47.619+02:00 level=INFO msg="local-api: escrow confirm — words typed back; the new package is on the hub (R-1151)" vmid=9201 job_id=escrow-1792374463410689900 duration_ms=48
|
||||||
|
time=2026-10-19T03:47:48.247+02:00 level=INFO msg="local-api: recovering the offsite repository password from the sealed escrow (R via body, never logged/persisted)" vmid=9201
|
||||||
|
time=2026-10-19T03:47:49.287+02:00 level=INFO msg="local-api: offsite repository password RECOVERED from the sealed escrow — returning that field ONLY (the tunnel token, the PBS token and the WG key stay inside the agent and are not returned)" vmid=9201 rest
|
||||||
|
time=2026-10-19T03:47:49.635+02:00 level=INFO msg="local-api: recovering the offsite repository password from the sealed escrow (R via body, never logged/persisted)" vmid=9201
|
||||||
|
time=2026-10-19T03:48:09.199+02:00 level=INFO msg="local-api: staged offsite restic repo password for escrow (field name only)" vmid=9201
|
||||||
|
time=2026-10-19T03:48:09.199+02:00 level=INFO msg="local-api: escrow ceremony started (controller-driven)" vmid=9201 job_id=escrow-1792374489199785658
|
||||||
|
felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/local/bin/felhom-agent --config /etc/felhom-agent/agent.json --selftest=escrow-create --upload --output=json
|
||||||
|
time=2026-10-19T03:48:11.881+02:00 level=INFO msg="local-api: escrow ceremony complete — R claimable (in-memory, one-shot); package uploads at the claim" job_id=escrow-1792374489199785658 restic_pw_sealed=true uploaded=false claim_ttl_s=600 duration_ms=2681
|
||||||
|
time=2026-10-19T03:48:13.294+02:00 level=INFO msg="local-api: escrow ceremony R claimed — the package waits for the typed-back words (R-1151)" vmid=9201 job_id=escrow-1792374489199785658 confirm_ttl_s=1800
|
||||||
|
time=2026-10-19T03:48:14.059+02:00 level=INFO msg="local-api: recovering the offsite repository password from the sealed escrow (R via body, never logged/persisted)" vmid=9201
|
||||||
|
time=2026-10-19T03:48:15.141+02:00 level=INFO msg="local-api: offsite repository password RECOVERED from the sealed escrow — returning that field ONLY (the tunnel token, the PBS token and the WG key stay inside the agent and are not returned)" vmid=9201 rest
|
||||||
|
time=2026-10-19T03:48:15.389+02:00 level=INFO msg="local-api: recovering the offsite repository password from the sealed escrow (R via body, never logged/persisted)" vmid=9201
|
||||||
|
time=2026-10-19T03:48:17.435+02:00 level=WARN msg="local-api: offsite key recovery: the fetched bundle did not open with the supplied recovery code" vmid=9201 err="escrow: the recovery code did not unwrap the identity escrow (wrong recovery code, or a corrupt
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
Mon Oct 19 02:08:15 UTC 2026
|
||||||
|
demo-felhom-8363b5|0.160.0|2026-10-19 01:55:31
|
||||||
|
demo-hp-bb76ea|0.160.0|2026-10-19 01:53:50
|
||||||
|
tester-1-d70be4|0.160.0|2026-10-19 01:54:15
|
||||||
@@ -0,0 +1,36 @@
|
|||||||
|
== root@192.168.0.154
|
||||||
|
Mon Oct 19 01:51:20 UTC 2026
|
||||||
|
felhom-agent 0.159.2
|
||||||
|
"host_id": "tester-1-d70be4"
|
||||||
|
"local_backup_retention": 3
|
||||||
|
Volid Format Type Size VMID
|
||||||
|
local:backup/vzdump-lxc-9100-2026_10_04-21_40_45.tar.zst tar.zst backup 648135998 9100
|
||||||
|
local:backup/vzdump-lxc-9201-2026_10_15-22_43_40.tar.zst tar.zst backup 3364084456 9201
|
||||||
|
local:backup/vzdump-lxc-9201-2026_10_17-04_32_43.tar.zst tar.zst backup 3479976346 9201
|
||||||
|
local:backup/vzdump-lxc-9201-2026_10_18-04_37_41.tar.zst tar.zst backup 1867626596 9201
|
||||||
|
== felhom-pve
|
||||||
|
Mon Oct 19 01:51:13 UTC 2026
|
||||||
|
felhom-agent 0.159.2
|
||||||
|
"host_id": "demo-felhom-8363b5"
|
||||||
|
"local_backup_retention": 3
|
||||||
|
Volid Format Type Size VMID
|
||||||
|
local:backup/vzdump-lxc-9100-2026_07_18-18_30_52.tar.zst tar.zst backup 612397450 9100
|
||||||
|
local:backup/vzdump-lxc-9100-2026_07_20-17_50_57.tar.zst tar.zst backup 648549366 9100
|
||||||
|
local:backup/vzdump-lxc-9100-2026_08_03-06_58_00.tar.zst tar.zst backup 649547835 9100
|
||||||
|
local:backup/vzdump-lxc-9100-2026_08_03-09_15_34.tar.zst tar.zst backup 649547835 9100
|
||||||
|
local:backup/vzdump-lxc-9201-2026_07_28-01_27_56.tar.zst tar.zst backup 5820882781 9201
|
||||||
|
local:backup/vzdump-lxc-9201-2026_07_28-06_44_22.tar.zst tar.zst backup 5840583923 9201
|
||||||
|
local:backup/vzdump-lxc-9201-2026_07_28-17_53_26.tar.zst tar.zst backup 5929975285 9201
|
||||||
|
{'local_backup_target': 'felhom-backup', 'local_backup_retention': 3, 'restore_test_cadence_seconds': 0, 'backup_cadence_seconds': 0, 'backup_targets': [{'target_id': 'felhom-pbs', 'cadence_seconds': 604800, 'keep_last': 0}]}
|
||||||
|
Name Type Status Total (KiB) Used (KiB) Available (KiB) %
|
||||||
|
felhom-backup dir active 960303848 7518604 903930732 0.78%
|
||||||
|
felhom-pbs pbs active 0 0 0 0.00%
|
||||||
|
local dir active 98497780 33631712 59816520 34.14%
|
||||||
|
local-lvm lvmthin active 365760512 11082543 354677968 3.03%
|
||||||
|
time=2026-10-16T05:03:53.319+02:00 level=INFO msg="backup: completed" vmid=9201 target=felhom-backup archive=felhom-backup:backup/vzdump-lxc-9201-2026_10_16-05_03_12.tar.zst size_bytes=1300947312 uncovered_volumes=2
|
||||||
|
time=2026-10-17T05:07:43.382+02:00 level=INFO msg="backup: completed" vmid=9201 target=felhom-backup archive=felhom-backup:backup/vzdump-lxc-9201-2026_10_17-05_07_02.tar.zst size_bytes=1334976334 uncovered_volumes=2
|
||||||
|
time=2026-10-18T05:08:13.307+02:00 level=INFO msg="backup: completed" vmid=9201 target=felhom-backup archive=felhom-backup:backup/vzdump-lxc-9201-2026_10_18-05_07_32.tar.zst size_bytes=1296570428 uncovered_volumes=2
|
||||||
|
Volid Format Type Size VMID
|
||||||
|
felhom-backup:backup/vzdump-lxc-9201-2026_10_16-05_03_12.tar.zst tar.zst backup 1300947312 9201
|
||||||
|
felhom-backup:backup/vzdump-lxc-9201-2026_10_17-05_07_02.tar.zst tar.zst backup 1334976334 9201
|
||||||
|
felhom-backup:backup/vzdump-lxc-9201-2026_10_18-05_07_32.tar.zst tar.zst backup 1296570428 9201
|
||||||
@@ -0,0 +1,32 @@
|
|||||||
|
time=2026-10-19T03:49:56.342+02:00 level=INFO msg="audit: gate decision" class=storage_wipe host=demo-hp-bb76ea guest="" source=one_shot_job disposition=customer_confirmable allowed=false reason=pending_confirmation key_id="" nonce="" durab
|
||||||
|
time=2026-10-19T03:49:56.342+02:00 level=INFO msg="gate decision (storage wipe)" role=user-data disposition=customer_confirmable allowed=false reason=pending_confirmation durable_id=byid:wwn-0x3333333000000fa0-part1+fsuuid:ef8fc766-1e12-491
|
||||||
|
time=2026-10-19T03:49:56.342+02:00 level=WARN msg="local-api: user-data data-bearing format refused" vmid=9201 device=/dev/sdb1 durable_id=byid:wwn-0x3333333000000fa0-part1+fsuuid:ef8fc766-1e12-491e-b6f2-31433e1989c8 reason=pending_confirma
|
||||||
|
felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/lvs --reportformat json --units b -o lv_name,data_percent,metadata_percent -- pve/data
|
||||||
|
felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/lvs --reportformat json --units b -o lv_name,data_percent,metadata_percent -- pve/data
|
||||||
|
felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/lvs --reportformat json --units b -o lv_name,data_percent,metadata_percent -- pve/data
|
||||||
|
felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/lvs --reportformat json --units b -o lv_name,data_percent,metadata_percent -- pve/data
|
||||||
|
felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/lvs --reportformat json --units b -o lv_name,data_percent,metadata_percent -- pve/data
|
||||||
|
felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/lvs --reportformat json --units b -o lv_name,data_percent,metadata_percent -- pve/data
|
||||||
|
felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/lvs --reportformat json --units b -o lv_name,data_percent,metadata_percent -- pve/data
|
||||||
|
felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/lvs --reportformat json --units b -o lv_name,data_percent,metadata_percent -- pve/data
|
||||||
|
felhom-agent : PWD=/ ; USER=root ; COMMAND=/usr/sbin/lvs --reportformat json --units b -o lv_name,data_percent,metadata_percent -- pve/data
|
||||||
|
backup-success-state.json
|
||||||
|
controller-swap-9201.json
|
||||||
|
drive-intents.json
|
||||||
|
escrow-stage
|
||||||
|
felhom-sshd
|
||||||
|
format-job.json
|
||||||
|
guest-binds.json
|
||||||
|
guest-disk-trim.json
|
||||||
|
guests
|
||||||
|
journal.log
|
||||||
|
local-api.crt
|
||||||
|
local-api.key
|
||||||
|
local-tokens.log
|
||||||
|
nonces.log
|
||||||
|
os
|
||||||
|
pbsdr
|
||||||
|
restore-test-state.json
|
||||||
|
selfupdate
|
||||||
|
units
|
||||||
|
wg
|
||||||
@@ -0,0 +1,39 @@
|
|||||||
|
Mon Oct 19 01:49:49 UTC 2026
|
||||||
|
new test disk: /dev/sdb
|
||||||
|
NAME SIZE MODEL
|
||||||
|
sdb 64M scsi_debug
|
||||||
|
--- the empty partition (the R-1093 input):
|
||||||
|
{
|
||||||
|
"blockdevices": [
|
||||||
|
{
|
||||||
|
"name": "sdb1",
|
||||||
|
"fstype": null,
|
||||||
|
"pttype": "gpt"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
DEVNAME=/dev/sdb1
|
||||||
|
PART_ENTRY_SCHEME=gpt
|
||||||
|
PART_ENTRY_UUID=aec56a69-b9a0-46de-9c8e-911ca8558717
|
||||||
|
PART_ENTRY_TYPE=0fc63daf-8483-4772-8e79-3d69d8477de4
|
||||||
|
PART_ENTRY_NUMBER=1
|
||||||
|
PART_ENTRY_OFFSET=2048
|
||||||
|
PART_ENTRY_SIZE=126976
|
||||||
|
PART_ENTRY_DISK=8:16
|
||||||
|
blkid rc=0
|
||||||
|
lrwxrwxrwx 1 root root 9 Oct 19 03:49 scsi-33333333000000fa0 -> ../../sdb
|
||||||
|
lrwxrwxrwx 1 root root 10 Oct 19 03:49 scsi-33333333000000fa0-part1 -> ../../sdb1
|
||||||
|
lrwxrwxrwx 1 root root 9 Oct 19 03:49 wwn-0x3333333000000fa0 -> ../../sdb
|
||||||
|
--- POST /disks/format on the EMPTY partition (no confirmation sent):
|
||||||
|
HTTP 200 {"ok":true,"data":{"vmid":9201,"device":"/dev/sdb1","formatted":true,"data_bearing":false,"reason":"blank device formatted ext4","durable_id":"byid:wwn-0x3333333000000fa0-part1","fs_uuid":"ef8fc766-1e12-491e-b6f2-31433e1989c8"}}
|
||||||
|
|
||||||
|
--- the partition now:
|
||||||
|
TYPE=ext4
|
||||||
|
PART_ENTRY_SCHEME=gpt
|
||||||
|
--- CONTROL: the same call again, on the now-ext4 partition:
|
||||||
|
HTTP 403 {"ok":false,"data":{"vmid":9201,"device":"/dev/sdb1","formatted":false,"data_bearing":true,"reason":"device has a ext4 filesystem","role":"user-data","needs_confirmation":true,"durable_id":"byid:wwn-0x3333333000000fa0-part1+fsuuid:ef8fc766-1e12-491e-b6f2-31433e1989c8"},"error":"device is data-bearing — customer confirmation required"}
|
||||||
|
|
||||||
|
--- teardown
|
||||||
|
not mounted
|
||||||
|
scsi_debug removed
|
||||||
|
0
|
||||||
@@ -0,0 +1,102 @@
|
|||||||
|
[release-agent] building 0.160.0 …
|
||||||
|
[release-agent] built ok: sha256 7ebb547f2b1ab93544ba828d3040351b60c4126327a5b85bbcb33f8f2b3dfca1
|
||||||
|
[release-agent] config bundle built: sha256 6b06090190ff609af1fe9842447ebf385fd52da4be382bc7ca708b558c8e202f
|
||||||
|
[release-agent] tagging v0.160.0 at c11e3e2 …
|
||||||
|
[release-agent] publishing …
|
||||||
|
[publish-agent] publishing /tmp/felhom-agent-YBD0P9 (16071264 bytes, sha256 7ebb547f2b1ab935…) → https://gitea.dooplex.hu/api/packages/admin/generic/felhom-agent/0.160.0/felhom-agent
|
||||||
|
[publish-agent] pre-delete existing artifact: HTTP 404 (404/204 expected)
|
||||||
|
[publish-agent] upload OK (HTTP 201)
|
||||||
|
[publish-agent] round-trip GET verified (sha256 matches)
|
||||||
|
[publish-agent] DONE. Record in the hub operator UI (Configs → Day-0 artifacts): agent 0.160.0 / 7ebb547f2b1ab93544ba828d3040351b6AGENT_VERSION=0.160.0
|
||||||
|
AGENT_SHA256=7ebb547f2b1ab93544ba828d3040351b60c4126327a5b85bbcb33f8f2b3dfca1
|
||||||
|
pre-push [felhom-agent]: running scripts/agent_gates.py --fast ...
|
||||||
|
agent_gates — 4 gate(s) [--fast]
|
||||||
|
--fast SKIPPED (deliberate periodic runs, never in a hook): published, gate-decoys
|
||||||
|
|
||||||
|
==============================================================================
|
||||||
|
== gate: reuse-refs (reuse_refs_check.py /mnt/5_hdd/felhom.eu/git/felhom-agent)
|
||||||
|
==============================================================================
|
||||||
|
note [felhom-agent] line 190: localapi/debuglogs_test.go (resolved by suffix → internal/localapi/debuglogs_test.go)
|
||||||
|
note [felhom-agent] line 206: hub/internal/store/dr_recipe.go (cross-repo → felhom.eu/hub/internal/store/dr_recipe.go)
|
||||||
|
OK [felhom-agent]: 118 cited paths — exact 116, suffix 1, ambiguous 0, cross-repo 1, FAILED 0 (siblings searched: app-catalog-felhom.eu, cc-secrets, felhom-controller, felhom.eu, felhom.eu-media, felhom.eu-sizing-push, felhom.eu-wt-sizing)
|
||||||
|
|
||||||
|
==============================================================================
|
||||||
|
== gate: instructions (instructions_gate.py /mnt/5_hdd/felhom.eu/git/felhom-agent)
|
||||||
|
==============================================================================
|
||||||
|
instructions_gate: /mnt/5_hdd/felhom.eu/git/felhom-agent
|
||||||
|
CLAUDE.md effective lines : 108 (ceiling 200)
|
||||||
|
version literals : 0
|
||||||
|
TEMPORARY blocks : 0
|
||||||
|
rule files : 6 (5 path-scoped)
|
||||||
|
workspace file : SYMLINK -> felhom.eu/documentation/runbooks/workspace-CLAUDE.md (resolves to the versioned copy)
|
||||||
|
memory index : 200 lines (ceiling 200), 25109 bytes (ceiling 25600)
|
||||||
|
memory index content : 28 version literal(s), 3 host address(es), 0 expired statement(s) [WARN only]
|
||||||
|
memory topic files : 183 indexed, 4 orphaned, 41 archived
|
||||||
|
register citations : 6 cited, 1116 register items known
|
||||||
|
|
||||||
|
WARNING: /mnt/5_hdd/felhom.eu/git/.claude-memory/MEMORY.md:31: version literal '0.14.0' — the fleet is not uniform, so it is stale within a day. Not a failure: Claude writes this file between sessions, so this warning is aimed at the model that will next edit it, not at whoever is pushing.
|
||||||
|
- **[restic 0.14.0 `--verify` is size+mtime, NOT content](restic-0140-verify-and-lock-sema
|
||||||
|
WARNING: /mnt/5_hdd/felhom.eu/git/.claude-memory/MEMORY.md:90: version literal '0.142.0' — the fleet is not uniform, so it is stale within a day. Not a failure: Claude writes this file between sessions, so this warning is aimed at the model that will next edit it, not at whoever is pushing.
|
||||||
|
- [Offsite continuity](offsite-continuity-2026-07-17.md) — ctrl 0.142.0 orphan guard + hub
|
||||||
|
WARNING: /mnt/5_hdd/felhom.eu/git/.claude-memory/MEMORY.md:90: version literal '0.60.0' — the fleet is not uniform, so it is stale within a day. Not a failure: Claude writes this file between sessions, so this warning is aimed at the model that will next edit it, not at whoever is pushing.
|
||||||
|
- [Offsite continuity](offsite-continuity-2026-07-17.md) — ctrl 0.142.0 orphan guard + hub
|
||||||
|
WARNING: /mnt/5_hdd/felhom.eu/git/.claude-memory/MEMORY.md:97: version literal '0.152.0' — the fleet is not uniform, so it is stale within a day. Not a failure: Claude writes this file between sessions, so this warning is aimed at the model that will next edit it, not at whoever is pushing.
|
||||||
|
- **[Hub operator UI: CC uses its token HUB_TOKEN, not the password](hub-operator-ui-basic
|
||||||
|
WARNING: /mnt/5_hdd/felhom.eu/git/.claude-memory/MEMORY.md:101: version literal '0.90.1' — the fleet is not uniform, so it is stale within a day. Not a failure: Claude writes this file between sessions, so this warning is aimed at the model that will next edit it, not at whoever is pushing.
|
||||||
|
- [Agent 0.90.1 is wrapper-only](agent-0901-is-wrapper-only.md) — do NOT publish the binar
|
||||||
|
WARNING: /mnt/5_hdd/felhom.eu/git/.claude-memory/MEMORY.md:103: version literal '0.144.0' — the fleet is not uniform, so it is stale within a day. Not a failure: Claude writes this file between sessions, so this warning is aimed at the model that will next edit it, not at whoever is pushing.
|
||||||
|
- [SMB „Megosztás" R-7 slice 1](smb-sharing-r7-slice1-2026-07-18.md) — ctrl 0.144.0 + felh
|
||||||
|
WARNING: /mnt/5_hdd/felhom.eu/git/.claude-memory/MEMORY.md: ... and 22 more version literal(s) — full list from the tally counts above.
|
||||||
|
WARNING: /mnt/5_hdd/felhom.eu/git/.claude-memory/MEMORY.md:59: host address '192.168.0.192' — operations/nodes.md is the single home for these. Not a failure: Claude writes this file between sessions, so this warning is aimed at the model that will next edit it, not at whoever is pushing.
|
||||||
|
- [hub.felhom.eu resolves to the LAN here](hub-resolves-to-lan-on-dooplex-network.md) — 19
|
||||||
|
WARNING: /mnt/5_hdd/felhom.eu/git/.claude-memory/MEMORY.md:78: host address '167.233.158.164' — operations/nodes.md is the single home for these. Not a failure: Claude writes this file between sessions, so this warning is aimed at the model that will next edit it, not at whoever is pushing.
|
||||||
|
- [Offsite PBS box RAM ceiling](offsite-pbs-box-ram-ceiling-2026-07-27.md) — **root SSH =
|
||||||
|
WARNING: /mnt/5_hdd/felhom.eu/git/.claude-memory/MEMORY.md:186: host address '192.168.0.180' — operations/nodes.md is the single home for these. Not a failure: Claude writes this file between sessions, so this warning is aimed at the model that will next edit it, not at whoever is pushing.
|
||||||
|
- **CC runs ON DooPlex** (192.168.0.180) — repos `/mnt/5_hdd/felhom.eu/git/felhom-*`, buil
|
||||||
|
WARNING: /mnt/5_hdd/felhom.eu/git/.claude-memory: 4 top-level topic file(s) are not referenced by MEMORY.md, so nothing will ever read them: f10-demo-hp-offsite-resolved-2026-07-23.md, launcher-v0163-2026-07-24.md, peti-return-p1-stop-2026-07-13.md, stopped-not-fault-v0164-2026-07-24.md
|
||||||
|
|
||||||
|
instructions_gate: OK (11 warning(s))
|
||||||
|
|
||||||
|
==============================================================================
|
||||||
|
== gate: release-complete (check-release-complete.py)
|
||||||
|
==============================================================================
|
||||||
|
check-release-complete — the newest CHANGELOG version is a complete release
|
||||||
|
newest CHANGELOG version: v0.159.2
|
||||||
|
ok tag v0.159.2 -> 65c1994d3b, an ancestor of HEAD
|
||||||
|
ok package 0.159.2 is downloadable
|
||||||
|
|
||||||
|
check-release-complete: v0.159.2 is tagged, placed and published.
|
||||||
|
|
||||||
|
==============================================================================
|
||||||
|
== gate: observations (observations_gate.py /mnt/5_hdd/felhom.eu/git/felhom-agent)
|
||||||
|
==============================================================================
|
||||||
|
observations gate OK — REPORT.md has no observations section (nothing to check)
|
||||||
|
|
||||||
|
==============================================================================
|
||||||
|
== summary
|
||||||
|
==============================================================================
|
||||||
|
reuse-refs OK (exit 0)
|
||||||
|
instructions OK (exit 0)
|
||||||
|
release-complete OK (exit 0c4126327a5b85bbcb33f8f2b3dfca1
|
||||||
|
[release-agent] config bundle published (HTTP 201)
|
||||||
|
[release-agent] pushing v0.160.0 …
|
||||||
|
remote: . Processing 1 references
|
||||||
|
remote: Processed 1 references in total
|
||||||
|
To https://gitea.dooplex.hu0)
|
||||||
|
observations OK (exit 0)
|
||||||
|
|
||||||
|
all agent gates OK
|
||||||
|
pre-push [felhom-agent]: gates OK - push proceeding.
|
||||||
|
|
||||||
|
RELEASED — and installable, verified by download, not by this script's own say-so.
|
||||||
|
|
||||||
|
version : 0.160.0
|
||||||
|
tag : v0.160.0
|
||||||
|
sha256 : 7ebb547f2b1ab93544ba828d3040351b60c4126327a5b85bbcb33f8f2b3dfca1
|
||||||
|
bundle : 6b06090190ff609af1fe9842447ebf385fd52da4be382bc7ca708b558c8e202f (felhom-config-bundle.json — vouch it with the agent)
|
||||||
|
|
||||||
|
NOT VOUCHED. Vouching is what points machines at this version and stays you/admin/felhom-agent.git
|
||||||
|
* [new tag] v0.160.0 -> v0.160.0
|
||||||
|
[release-agent] verifying by independent download …
|
||||||
|
r deliberate act:
|
||||||
|
hub operator UI → Configs → Day-0 artifacts. Until then boxes keep installing the previous one.
|
||||||
Binary file not shown.
|
After Width: | Height: | Size: 99 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 79 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 36 KiB |
@@ -0,0 +1,17 @@
|
|||||||
|
03:05:34 goto https://felhom.enkisfelhom.hu/login -> https://felhom.enkisfelhom.hu/login
|
||||||
|
03:05:39 eval -> "focused"
|
||||||
|
03:05:43 eval -> "https://felhom.enkisfelhom.hu/launcher"
|
||||||
|
03:05:43 goto https://felhom.enkisfelhom.hu/apps/calibre-web -> https://felhom.enkisfelhom.hu/apps/calibre-web
|
||||||
|
03:05:48 eval -> "https://felhom.enkisfelhom.hu/__files/open?to=… | button: Fájlok ↗ | folder: /files/NVME 1TB/media/books"
|
||||||
|
03:05:48 shot 01-app-page-files-button
|
||||||
|
03:05:49 eval -> "pressed Fájlok"
|
||||||
|
03:05:57 eval -> "files.enkisfelhom.hu/files/NVME 1TB/media/books | login form: no | title: FileBrowser Quantum - Files - books"
|
||||||
|
03:05:57 eval -> "FileBrowser user: felhom-household | admin: false | modify: true | loginMethod: proxy"
|
||||||
|
03:05:59 shot 02-filebrowser-opened-signed-in
|
||||||
|
03:05:59 goto https://felhom.enkisfelhom.hu/launcher -> https://felhom.enkisfelhom.hu/launcher
|
||||||
|
03:06:04 eval -> "signed out on the dashboard"
|
||||||
|
03:06:08 eval -> "https://felhom.enkisfelhom.hu/login"
|
||||||
|
03:06:08 goto https://files.enkisfelhom.hu/files/ -> https://felhom.enkisfelhom.hu/login
|
||||||
|
03:06:15 eval -> "felhom.enkisfelhom.hu/login | login form: YES"
|
||||||
|
03:06:15 shot 03-after-sign-out-asks-again
|
||||||
|
LOG ('netfail', 'net::ERR_ABORTED', 'Fetch')
|
||||||
@@ -0,0 +1,9 @@
|
|||||||
|
== through traefik (https://files.enkisfelhom.hu, no files cookie), GET /api/users?id=self
|
||||||
|
forged header named X-Felhom-User -> HTTP 401 (user None)
|
||||||
|
forged header named RANDOM -> HTTP 401 (user None)
|
||||||
|
forged header named SECRET -> HTTP 401 (user None)
|
||||||
|
== from the bookstack container (on traefik-public), straight to http://filebrowser:80/api/users?id=self — no traefik
|
||||||
|
forged header named X-Felhom-User -> HTTP 401 (user None)
|
||||||
|
forged header named RANDOM -> HTTP 401 (user None)
|
||||||
|
== CONTROL: the same container with the box's REAL secret name (read from the box, never printed)
|
||||||
|
real secret name -> HTTP 200 (user felhom-household)
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
Mon Oct 19 03:08:04 UTC 2026
|
||||||
|
200
|
||||||
|
{
|
||||||
|
"ok": true,
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
Mon Oct 19 03:08:23 UTC 2026
|
||||||
|
05:08:24 [1] deploy -> 202 {'ok': True, 'message': 'Telepítés elindítva – az állapot a kártyán követhető'}
|
||||||
|
05:09:29 [1] deployed, controller state=running, pinned={'onlyoffice': 'onlyoffice/documentserver:9.4.0'}
|
||||||
|
deploy: True
|
||||||
|
Mon Oct 19 03:09:29 UTC 2026
|
||||||
@@ -0,0 +1,16 @@
|
|||||||
|
Mon Oct 19 03:09:40 UTC 2026
|
||||||
|
onlyoffice health: healthy
|
||||||
|
200
|
||||||
|
{"ok": true, "data": {"enabled": false, "status": "disabled"}, "message": "Integráció kikapcsolva"}
|
||||||
|
03:09:42 files.<domain> 200
|
||||||
|
onlyoffice Up 28 seconds (healthy)
|
||||||
|
filebrowser Up 6 hours (healthy)
|
||||||
|
Mon Oct 19 03:10:35 UTC 2026
|
||||||
|
200
|
||||||
|
{"ok": true, "data": {"enabled": true, "enabled_at": "2026-10-19T03:10:36Z", "status": "active"}, "message": "Integráció engedélyezve"}
|
||||||
|
03:10:39 files.<domain> 200 (try 1)
|
||||||
|
filebrowser Up 5 seconds (healthy)
|
||||||
|
onlyoffice Up About a minute (healthy)
|
||||||
|
url: "https://office.enkisfelhom.hu"
|
||||||
|
internalUrl: "http://onlyoffice:80"
|
||||||
|
2
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
login 200 token_len 355
|
||||||
|
resource 200 application/vnd.openxmlformats-officedocument.wordprocessingml.document
|
||||||
|
03:12:19Z preview small HTTP 500 content-type application/json; charset=utf-8 bytes 167 in 0.0s b'{"status":500,"message":"failed to create image for PDF file: failed to open PDF from memory for file \'/srv/scratch_hdd/r1153/teszt.docx\': fitz: cannot open document"}'
|
||||||
|
03:12:19Z preview large HTTP 500 content-type application/json; charset=utf-8 bytes 167 in 0.0s b'{"status":500,"message":"failed to create image for PDF file: failed to open PDF from memory for file \'/srv/scratch_hdd/r1153/teszt.docx\': fitz: cannot open document"}'
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
== /r1153/sample.docx
|
||||||
|
login 200 token_len 355
|
||||||
|
resource 200 application/vnd.openxmlformats-officedocument.wordprocessingml.document
|
||||||
|
03:13:40Z preview small HTTP 200 content-type image/jpeg bytes 8714 in 0.3s
|
||||||
|
03:13:40Z preview large HTTP 200 content-type image/jpeg bytes 22734 in 0.3s
|
||||||
|
== /r1153/teszt.docx
|
||||||
|
login 200 token_len 355
|
||||||
|
resource 200 application/vnd.openxmlformats-officedocument.wordprocessingml.document
|
||||||
|
03:13:40Z preview small HTTP 500 content-type application/json; charset=utf-8 bytes 167 in 0.0s b'{"status":500,"message":"failed to create image for PDF file: failed to open PDF from memory for file \'/srv/scratch_hdd/r1153/teszt.docx\': fitz: cannot open document"}'
|
||||||
|
03:13:40Z preview large HTTP 500 content-type application/json; charset=utf-8 bytes 167 in 0.0s b'{"status":500,"message":"failed to create image for PDF file: failed to open PDF from memory for file \'/srv/scratch_hdd/r1153/teszt.docx\': fitz: cannot open document"}'
|
||||||
@@ -0,0 +1,9 @@
|
|||||||
|
Mon Oct 19 03:13:53 UTC 2026
|
||||||
|
{"ok": true, "data": {"enabled": false, "status": "disabled"}, "message": "Integráció kikapcsolva"}
|
||||||
|
03:13:57 files.<domain> 200 (try 1)
|
||||||
|
== link OFF, /r1153/sample.docx
|
||||||
|
03:13:58Z preview small HTTP 200 content-type image/jpeg bytes 8714 in 0.2s
|
||||||
|
03:13:58Z preview large HTTP 200 content-type image/jpeg bytes 22734 in 0.2s
|
||||||
|
== link OFF, /r1153/teszt.docx
|
||||||
|
03:13:58Z preview small HTTP 500 content-type application/json; charset=utf-8 bytes 167 in 0.0s b'{"status":500,"message":"failed to create image for PDF file: failed to open PDF from memory for file \'/srv/scratch_hdd/r1153/teszt.docx\': fitz: cannot open document"}'
|
||||||
|
03:13:58Z preview large HTTP 500 content-type application/json; charset=utf-8 bytes 167 in 0.0s b'{"status":500,"message":"failed to create image for PDF file: failed to open PDF from memory for file \'/srv/scratch_hdd/r1153/teszt.docx\': fitz: cannot open document"}'
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
Mon Oct 19 03:14:09 UTC 2026
|
||||||
|
test files removed
|
||||||
|
05:14:54 [X] remove (with drive data) -> 200 {'ok': True, 'data': {'removed': 'onlyoffice', 'volumes_removed': ['onlyoffice_onlyoffice_data', 'onlyoffice_onlyoffice_logs'], 'hdd_paths_removed': [], 'hdd_pa
|
||||||
|
05:15:03 [X] after remove: deployed=False leftovers='/opt/docker/stacks/onlyoffice'
|
||||||
|
remove: 200
|
||||||
|
0
|
||||||
|
0
|
||||||
|
ls: cannot access '/opt/docker/stacks/onlyoffice/app.yaml': No such file or directory
|
||||||
Binary file not shown.
|
After Width: | Height: | Size: 8.5 KiB |
@@ -0,0 +1,4 @@
|
|||||||
|
## per-customer floors 0.332.0 / MinAgent 0.158.1 — 03:16:43
|
||||||
|
tester-1: HTTP/1.1 303 See Other Location: /customers/tester-1?flash=floor_set
|
||||||
|
demo-hp: HTTP/1.1 303 See Other Location: /customers/demo-hp?flash=floor_set
|
||||||
|
demo-felhom: HTTP/1.1 303 See Other Location: /customers/demo-felhom?flash=floor_set
|
||||||
@@ -0,0 +1,14 @@
|
|||||||
|
03:16:55 demo-hp=0.332.0-rc.3 tester-1=0.332.0 demo-felhom=0.332.0
|
||||||
|
03:17:28 demo-hp=0.332.0
|
||||||
|
== hp
|
||||||
|
2026/10/19 03:16:57 scheduler.go:85: [DEBUG] [scheduler] periodic job registered: name="selfupdate-check" interval=6h0m0s totalJobs=23
|
||||||
|
2026/10/19 03:17:04 updater.go:104: [DEBUG] [selfupdate] SetFloor: floor "" → "0.332.0"
|
||||||
|
2026/10/19 03:17:04 updater.go:104: [DEBUG] [selfupdate] maybeAutoUpdate: current 0.332.0 >= floor 0.332.0 — no action
|
||||||
|
== root@192.168.0.154
|
||||||
|
2026/10/19 03:16:55 [INFO] [selfupdate] Post-update startup: update successful (0.331.0 → 0.332.0)
|
||||||
|
2026/10/19 03:16:55 [INFO] [scheduler] Registered periodic job: selfupdate-check (every 6h0m0s)
|
||||||
|
2026/10/19 03:17:26 [INFO] [selfupdate] Current version 0.332.0 is up to date
|
||||||
|
== felhom-pve
|
||||||
|
2026/10/19 03:16:45 [INFO] [selfupdate] Post-update startup: update successful (0.331.0 → 0.332.0)
|
||||||
|
2026/10/19 03:16:45 [INFO] [scheduler] Registered periodic job: selfupdate-check (every 6h0m0s)
|
||||||
|
2026/10/19 03:17:17 [INFO] [selfupdate] Current version 0.332.0 is up to date
|
||||||
@@ -0,0 +1,33 @@
|
|||||||
|
=== demo-hp (enkisfelhom.hu) 03:17:58
|
||||||
|
containers: 24 running; not healthy/running: traefik Up 3 days;
|
||||||
|
unhealthy: 0
|
||||||
|
2026/10/19 03:16:57 updater.go:855: [INFO] [selfupdate] Post-update startup: update successful (0.332.0-rc.3 → 0.332.0)
|
||||||
|
allow-list refusals since start: 0
|
||||||
|
2026/10/19 03:16:58 sync.go:142: [INFO] [sync] Initial sync: Sablonok naprakészek — nincs változás
|
||||||
|
installed apps (stacks with app.yaml): 12
|
||||||
|
public https://felhom.enkisfelhom.hu/login: 200
|
||||||
|
public https://files.enkisfelhom.hu/: 200
|
||||||
|
=== demo-felhom (demo-felhom.eu) 03:18:03
|
||||||
|
containers: 9 running; not healthy/running: traefik Up 3 days;
|
||||||
|
unhealthy: 0
|
||||||
|
2026/10/19 03:16:45 [INFO] [selfupdate] Post-update startup: update successful (0.331.0 → 0.332.0)
|
||||||
|
allow-list refusals since start: 0
|
||||||
|
2026/10/19 03:16:46 [INFO] [sync] Initial sync: Sablonok naprakészek — nincs változás
|
||||||
|
installed apps (stacks with app.yaml): 5
|
||||||
|
2026/10/19 03:16:45 [INFO] [infra] wrote filebrowser route → /opt/docker/stacks/traefik/dynamic/filebrowser.yml (Host files.demo-felhom.eu → filebrowser:80 by name, R-1126; one sign-in true, R-1100)
|
||||||
|
2026/10/19 03:16:46 [INFO] [web] FileBrowser mounts synced (recreated) — 1 storage path(s), config updated
|
||||||
|
2026/10/19 03:16:52 [INFO] [infra] filebrowser: default login refused for this run — joined traefik-public, files.<domain> served (R-1126)
|
||||||
|
public https://felhom.demo-felhom.eu/login: 200
|
||||||
|
public https://files.demo-felhom.eu/: 200
|
||||||
|
=== tester-1 (enkicsifelhom.hu) 03:18:06
|
||||||
|
containers: 10 running; not healthy/running: traefik Up 3 days;
|
||||||
|
unhealthy: 0
|
||||||
|
2026/10/19 03:16:55 [INFO] [selfupdate] Post-update startup: update successful (0.331.0 → 0.332.0)
|
||||||
|
allow-list refusals since start: 0
|
||||||
|
2026/10/19 03:16:56 [INFO] [sync] Initial sync: Sablonok naprakészek — nincs változás
|
||||||
|
installed apps (stacks with app.yaml): 3
|
||||||
|
2026/10/19 03:16:55 [INFO] [infra] wrote filebrowser route → /opt/docker/stacks/traefik/dynamic/filebrowser.yml (Host files.enkicsifelhom.hu → filebrowser:80 by name, R-1126; one sign-in true, R-1100)
|
||||||
|
2026/10/19 03:16:56 [INFO] [web] FileBrowser mounts synced (recreated) — 5 storage path(s), config updated
|
||||||
|
2026/10/19 03:17:02 [INFO] [infra] filebrowser: default login refused for this run — joined traefik-public, files.<domain> served (R-1126)
|
||||||
|
public https://felhom.enkicsifelhom.hu/login: 200
|
||||||
|
public https://files.enkicsifelhom.hu/: 200
|
||||||
@@ -0,0 +1,33 @@
|
|||||||
|
control NEGATIVE (a stub answering 200 over plain http): convicted, as it must (200 -)
|
||||||
|
== tester-1 (enkicsifelhom.hu): 6 public name(s)
|
||||||
|
OK control POSITIVE http://felhom.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://felhom.enkicsifelhom.hu/r1033-check?q=1
|
||||||
|
OK http://files.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://files.enkicsifelhom.hu/r1033-check?q=1
|
||||||
|
OK http://no-such-name-r1033.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://no-such-name-r1033.enkicsifelhom.hu/r1033-check?q=1
|
||||||
|
OK http://paperless.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://paperless.enkicsifelhom.hu/r1033-check?q=1
|
||||||
|
OK http://paste.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://paste.enkicsifelhom.hu/r1033-check?q=1
|
||||||
|
OK http://wiki.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://wiki.enkicsifelhom.hu/r1033-check?q=1
|
||||||
|
== demo-hp (enkisfelhom.hu): 14 public name(s)
|
||||||
|
OK http://arcade.enkisfelhom.hu/r1033-check?q=1 -> 301 https://arcade.enkisfelhom.hu/r1033-check?q=1
|
||||||
|
OK http://books.enkisfelhom.hu/r1033-check?q=1 -> 301 https://books.enkisfelhom.hu/r1033-check?q=1
|
||||||
|
OK http://docs.enkisfelhom.hu/r1033-check?q=1 -> 301 https://docs.enkisfelhom.hu/r1033-check?q=1
|
||||||
|
OK http://felhom.enkisfelhom.hu/r1033-check?q=1 -> 301 https://felhom.enkisfelhom.hu/r1033-check?q=1
|
||||||
|
OK http://files.enkisfelhom.hu/r1033-check?q=1 -> 301 https://files.enkisfelhom.hu/r1033-check?q=1
|
||||||
|
OK http://gist.enkisfelhom.hu/r1033-check?q=1 -> 301 https://gist.enkisfelhom.hu/r1033-check?q=1
|
||||||
|
OK http://ha.enkisfelhom.hu/r1033-check?q=1 -> 301 https://ha.enkisfelhom.hu/r1033-check?q=1
|
||||||
|
OK http://no-such-name-r1033.enkisfelhom.hu/r1033-check?q=1 -> 301 https://no-such-name-r1033.enkisfelhom.hu/r1033-check?q=1
|
||||||
|
OK http://papir.enkisfelhom.hu/r1033-check?q=1 -> 301 https://papir.enkisfelhom.hu/r1033-check?q=1
|
||||||
|
OK http://paste.enkisfelhom.hu/r1033-check?q=1 -> 301 https://paste.enkisfelhom.hu/r1033-check?q=1
|
||||||
|
OK http://pdf.enkisfelhom.hu/r1033-check?q=1 -> 301 https://pdf.enkisfelhom.hu/r1033-check?q=1
|
||||||
|
OK http://plex.enkisfelhom.hu/r1033-check?q=1 -> 301 https://plex.enkisfelhom.hu/r1033-check?q=1
|
||||||
|
OK http://travel.enkisfelhom.hu/r1033-check?q=1 -> 301 https://travel.enkisfelhom.hu/r1033-check?q=1
|
||||||
|
OK http://wiki.enkisfelhom.hu/r1033-check?q=1 -> 301 https://wiki.enkisfelhom.hu/r1033-check?q=1
|
||||||
|
== demo-felhom (demo-felhom.eu): 8 public name(s)
|
||||||
|
OK http://felhom.demo-felhom.eu/r1033-check?q=1 -> 301 https://felhom.demo-felhom.eu/r1033-check?q=1
|
||||||
|
OK http://files.demo-felhom.eu/r1033-check?q=1 -> 301 https://files.demo-felhom.eu/r1033-check?q=1
|
||||||
|
OK http://gist.demo-felhom.eu/r1033-check?q=1 -> 301 https://gist.demo-felhom.eu/r1033-check?q=1
|
||||||
|
OK http://no-such-name-r1033.demo-felhom.eu/r1033-check?q=1 -> 301 https://no-such-name-r1033.demo-felhom.eu/r1033-check?q=1
|
||||||
|
OK http://prowlarr.demo-felhom.eu/r1033-check?q=1 -> 301 https://prowlarr.demo-felhom.eu/r1033-check?q=1
|
||||||
|
OK http://radarr.demo-felhom.eu/r1033-check?q=1 -> 301 https://radarr.demo-felhom.eu/r1033-check?q=1
|
||||||
|
OK http://sonarr.demo-felhom.eu/r1033-check?q=1 -> 301 https://sonarr.demo-felhom.eu/r1033-check?q=1
|
||||||
|
OK http://torrent.demo-felhom.eu/r1033-check?q=1 -> 301 https://torrent.demo-felhom.eu/r1033-check?q=1
|
||||||
|
== every public name redirects http -> https
|
||||||
@@ -0,0 +1,16 @@
|
|||||||
|
=== demo-hp (enkisfelhom.hu) 03:23:50
|
||||||
|
dashboard sign-in: 302
|
||||||
|
the Fajlok button goes to: https://files.enkisfelhom.hu/_felhom/sso
|
||||||
|
the pass answer goes to: https://files.enkisfelhom.hu/files/; files cookie set: 1
|
||||||
|
FileBrowser with the files cookie: felhom-household admin: False modify: True
|
||||||
|
FileBrowser without it, a fresh browser: HTTP 401
|
||||||
|
dashboard sign-out: 302
|
||||||
|
after the sign-out, the files cookie sends the browser to: https://felhom.enkisfelhom.hu/__files/open?to=%2Ffiles%2F
|
||||||
|
=== tester-1 (enkicsifelhom.hu) 03:23:51
|
||||||
|
dashboard sign-in: 302
|
||||||
|
the Fajlok button goes to: https://files.enkicsifelhom.hu/_felhom/sso
|
||||||
|
the pass answer goes to: https://files.enkicsifelhom.hu/files/; files cookie set: 1
|
||||||
|
FileBrowser with the files cookie: felhom-household admin: False modify: True
|
||||||
|
FileBrowser without it, a fresh browser: HTTP 401
|
||||||
|
dashboard sign-out: 302
|
||||||
|
after the sign-out, the files cookie sends the browser to: https://felhom.enkicsifelhom.hu/__files/open?to=%2Ffiles%2F
|
||||||
@@ -0,0 +1,7 @@
|
|||||||
|
=== demo-felhom (demo-felhom.eu) 03:24:06 — no dashboard password is held for this box: file and refusal checks only
|
||||||
|
-rw-------
|
||||||
|
valid shape: 1
|
||||||
|
config carries the secret: 1, route carries it: 1
|
||||||
|
forged secret name through traefik, no files cookie: HTTP 401
|
||||||
|
from outside, old public name: HTTP 401
|
||||||
|
from outside, the pass path with a forged pass: HTTP 403
|
||||||
@@ -0,0 +1,16 @@
|
|||||||
|
## global floor 0.332.0 / 0.158.1 — 03:24:18
|
||||||
|
HTTP/1.1 303 See Other
|
||||||
|
Location: /configuration?flash=floor_set
|
||||||
|
## read back (DB with WAL) 03:24:18
|
||||||
|
min_controller_version|0.332.0
|
||||||
|
min_controller_version_declared_min_agent|0.332.0=0.158.1
|
||||||
|
## per-box floors cleared — 03:24:28
|
||||||
|
tester-1: HTTP/1.1 303 See Other Location: /customers/tester-1?flash=floor_set
|
||||||
|
demo-hp: HTTP/1.1 303 See Other Location: /customers/demo-hp?flash=floor_set
|
||||||
|
demo-felhom: HTTP/1.1 303 See Other Location: /customers/demo-felhom?flash=floor_set
|
||||||
|
min_controller_declared_min_agent
|
||||||
|
min_controller_set_at
|
||||||
|
min_controller_version
|
||||||
|
demo-felhom|''|''
|
||||||
|
demo-hp|''|''
|
||||||
|
tester-1|''|''
|
||||||
@@ -0,0 +1,29 @@
|
|||||||
|
Mon Oct 19 02:14:15 UTC 2026
|
||||||
|
application.argoproj.io/felhom patched
|
||||||
|
deployment "hub" successfully rolled out
|
||||||
|
sync=Synced health=Progressing rev=200ab6530bf2bec9a9b341e4bc8e2bfd09f6a609
|
||||||
|
gitea.dooplex.hu/admin/felhom-hub:0.159.0
|
||||||
|
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
|
||||||
|
hub-67bff8bcb6-8pgxf 0/1 Running 0 9s 10.42.0.56 dooplex <none> <none>
|
||||||
|
2026/10/19 04:14:21 [INFO] Offsite pool-box checker initialized: box=611714 fill warn=80% crit=90%, oversub warn=2.00x, unreachable after 3 consecutive failed reads, refresh 15m0s
|
||||||
|
2026/10/19 04:14:21 [INFO] osupdates: approval rule = every ring-0 box healthy for 24h0m0s and 1 night run(s)
|
||||||
|
2026/10/19 04:14:21 [INFO] osupdates: the Docker engine set is approved only by the operator, after 2 healthy ring-0 night(s)
|
||||||
|
2026/10/19 04:14:21 [INFO] osupdates: alarms = no OS leg 168h0m0s, reboot needed 336h0m0s, ring 0 stalled 168h0m0s, not covered 336h0m0s
|
||||||
|
2026/10/19 04:14:21 [INFO] Off-site key registrar enabled; daily key check at 07:10 Budapest
|
||||||
|
2026/10/19 04:14:21 [INFO] access log: /data/access-log, one file per UTC day, kept 90 days
|
||||||
|
2026/10/19 04:14:21 [INFO] Registry version checker started (every 6h)
|
||||||
|
2026/10/19 04:14:21 [INFO] offsite-key-audit: next run at 2026-10-19 07:10 CEST (in 2h55m39s)
|
||||||
|
2026/10/19 04:14:21 [INFO] WG peer-sync enabled (endpoint 167.233.158.164:22, user felhom-peersync)
|
||||||
|
2026/10/19 04:14:21 [INFO] PBS DR tenantsync enabled (endpoint 167.233.158.164:22, user felhom-peersync; WG-registration auto-provision hook armed)
|
||||||
|
2026/10/19 04:14:21 [INFO] PBS-DR box checker initialized: fill warn=80% crit=90%, unreachable after 3 consecutive failed reads, refresh 15m0s
|
||||||
|
2026/10/19 04:14:21 [INFO] agent-plane poke enabled (endpoint 167.233.158.164:22, user felhom-peersync; web + api admin seams armed)
|
||||||
|
2026/10/19 04:14:21 [INFO] PBS-DR self-heal reconciler started (interval 5m)
|
||||||
|
2026/10/19 04:14:21 [INFO] offsite credential self-heal reconciler started (interval 5m, debounce 2 reports)
|
||||||
|
2026/10/19 04:14:21 [DEBUG] Registry version check: latest = 0.331.0
|
||||||
|
Mon Oct 19 02:15:36 UTC 2026
|
||||||
|
sync=Synced health=Healthy
|
||||||
|
hub-67bff8bcb6-8pgxf 1/1 Running 0 79s
|
||||||
|
--- floor (DB with WAL):
|
||||||
|
min_controller_version|0.331.0
|
||||||
|
min_controller_version_declared_min_agent|0.331.0=0.158.1
|
||||||
|
--- errors since start:
|
||||||
@@ -0,0 +1,9 @@
|
|||||||
|
Mon Oct 19 02:32:34 UTC 2026
|
||||||
|
hosts (agent reports) after the 02:14:21 start:
|
||||||
|
demo-felhom-8363b5|0.160.0|2026-10-19 02:25:35
|
||||||
|
demo-hp-bb76ea|0.160.0|2026-10-19 02:23:50
|
||||||
|
tester-1-d70be4|0.160.0|2026-10-19 02:24:21
|
||||||
|
controller reports:
|
||||||
|
demo-felhom|2026-10-19 02:29:12
|
||||||
|
demo-hp|2026-10-19 02:31:13
|
||||||
|
tester-1|2026-10-19 02:29:13
|
||||||
@@ -0,0 +1,11 @@
|
|||||||
|
control (before the window): jobs with run_attempt>1: 17
|
||||||
|
('felhom-controller', 1401, 2, '2026-10-06T00:44:57Z', 'success')
|
||||||
|
('felhom-controller', 1357, 2, '2026-10-05T15:05:37Z', 'success')
|
||||||
|
('felhom-controller', 1549, 2, '2026-10-08T08:26:11Z', 'success')
|
||||||
|
('felhom-controller', 1639, 2, '2026-10-10T13:09:06Z', 'success')
|
||||||
|
('felhom-agent', 1579, 2, '2026-10-09T04:58:15Z', 'success')
|
||||||
|
('app-catalog-felhom.eu', 1368, 2, '2026-10-05T17:15:12Z', 'success')
|
||||||
|
window since 2026-10-12T12:00:00Z : jobs with run_attempt>1: 3
|
||||||
|
('felhom-controller', 1944, 2, '2026-10-16T13:50:35Z', 'success')
|
||||||
|
('felhom-controller', 1959, 2, '2026-10-16T17:07:53Z', 'success')
|
||||||
|
('felhom-controller', 1991, 2, '2026-10-17T09:59:02Z', 'success')
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
felhom.eu 1645 log bytes 38590
|
||||||
|
felhom.eu 1646 log bytes 38503
|
||||||
|
felhom-controller 1639 log bytes 18965
|
||||||
|
felhom.eu 1738 log bytes 46399
|
||||||
@@ -0,0 +1,26 @@
|
|||||||
|
window since 2026-10-12T12:00:00Z: 381 jobs in 4 repos; not-success 25; with no log (lost) 0
|
||||||
|
not-success ('felhom.eu', 1738, 'c2a6cf7a', 'completed', 'failure', '2026-10-13T04:47:12Z', 1) has log
|
||||||
|
not-success ('felhom.eu', 1747, 'ed59bf3c', 'completed', 'failure', '2026-10-13T17:28:57Z', 1) has log
|
||||||
|
not-success ('felhom.eu', 1796, 'a2ca6eb4', 'completed', 'failure', '2026-10-14T16:51:48Z', 1) has log
|
||||||
|
not-success ('felhom.eu', 1889, '6749118c', 'completed', 'failure', '2026-10-15T19:21:34Z', 1) has log
|
||||||
|
not-success ('felhom.eu', 1993, 'b9c4f73b', 'completed', 'failure', '2026-10-17T10:02:08Z', 1) has log
|
||||||
|
not-success ('felhom.eu', 1997, '498b9139', 'completed', 'failure', '2026-10-17T10:42:11Z', 1) has log
|
||||||
|
not-success ('felhom-controller', 1865, 'e66d4669', 'completed', 'failure', '2026-10-15T16:37:58Z', 1) has log
|
||||||
|
not-success ('felhom-controller', 1867, 'fe0228c8', 'completed', 'failure', '2026-10-15T17:05:00Z', 1) has log
|
||||||
|
not-success ('felhom-controller', 1869, 'db52f5a4', 'completed', 'failure', '2026-10-15T17:27:25Z', 1) has log
|
||||||
|
not-success ('felhom-controller', 1870, '83392064', 'completed', 'failure', '2026-10-15T17:32:48Z', 1) has log
|
||||||
|
not-success ('felhom-controller', 1871, '9cc7ffcf', 'completed', 'failure', '2026-10-15T18:08:11Z', 1) has log
|
||||||
|
not-success ('felhom-controller', 1873, '7f3e58ff', 'completed', 'failure', '2026-10-15T18:30:00Z', 1) has log
|
||||||
|
not-success ('felhom-controller', 1875, '48d86748', 'completed', 'failure', '2026-10-15T18:33:27Z', 1) has log
|
||||||
|
not-success ('felhom-controller', 1876, '006ceab5', 'completed', 'failure', '2026-10-15T18:41:12Z', 1) has log
|
||||||
|
not-success ('felhom-controller', 1877, 'f908eee2', 'completed', 'failure', '2026-10-15T18:45:59Z', 1) has log
|
||||||
|
not-success ('felhom-controller', 1878, '02252556', 'completed', 'failure', '2026-10-15T18:46:27Z', 1) has log
|
||||||
|
not-success ('felhom-controller', 1879, '9e68e526', 'completed', 'failure', '2026-10-15T18:51:46Z', 1) has log
|
||||||
|
not-success ('felhom-controller', 1880, '8c7aa10a', 'completed', 'failure', '2026-10-15T18:52:08Z', 1) has log
|
||||||
|
not-success ('felhom-controller', 1882, 'ec438479', 'completed', 'failure', '2026-10-15T18:57:00Z', 1) has log
|
||||||
|
not-success ('felhom-controller', 1883, 'f9151f08', 'completed', 'failure', '2026-10-15T19:04:54Z', 1) has log
|
||||||
|
not-success ('felhom-controller', 1884, '4261a38b', 'completed', 'failure', '2026-10-15T19:05:47Z', 1) has log
|
||||||
|
not-success ('felhom-controller', 1887, 'c6a56712', 'completed', 'failure', '2026-10-15T19:09:40Z', 1) has log
|
||||||
|
not-success ('felhom-controller', 1990, '334df7db', 'completed', 'failure', '2026-10-17T09:53:24Z', 1) has log
|
||||||
|
not-success ('felhom-controller', 2036, 'e30091bf', 'completed', 'failure', '2026-10-17T15:30:09Z', 1) has log
|
||||||
|
not-success ('app-catalog-felhom.eu', 2067, 'c97eb1c9', 'completed', 'failure', '2026-10-18T18:45:53Z', 1) has log
|
||||||
@@ -0,0 +1,16 @@
|
|||||||
|
task fields: ['created_at', 'display_title', 'event', 'head_branch', 'head_sha', 'id', 'name', 'run_number', 'run_started_at', 'status', 'updated_at', 'url', 'workflow_id']
|
||||||
|
{'id': 1991, 'name': 'gates', 'head_sha': 'b69585789a9fa34a7c33faf24da44926f166a9da', 'run_number': 393, 'status': 'success', 'event': 'push', 'created_at': '2026-10-16T21:50:09Z', 'run_started_at': '2026-10-16T21:50:09Z', 'updated_at': '2026-10-16T21:50:58Z'}
|
||||||
|
{'id': 1959, 'name': 'gates', 'head_sha': 'f30a80841e372abf9d7992dc9eb3eec607078e90', 'run_number': 379, 'status': 'success', 'event': 'push', 'created_at': '2026-10-16T13:19:24Z', 'run_started_at': '2026-10-16T13:19:24Z', 'updated_at': '2026-10-16T13:20:14Z'}
|
||||||
|
{'id': 1639, 'name': 'gates', 'head_sha': '1cb43a21a2dc5cd44fbcc0749b460dba1ffa8088', 'run_number': 326, 'status': 'success', 'event': 'push', 'created_at': '2026-10-10T07:55:47Z', 'run_started_at': '2026-10-10T07:55:47Z', 'updated_at': '2026-10-10T07:56:52Z'}
|
||||||
|
job 1944 sha c02c9d83 attempt 2 created 2026-10-16T13:50:35Z started 2026-10-16T14:02:29Z completed 2026-10-16T14:03:21Z
|
||||||
|
job 1959 sha 29a3ce8c attempt 2 created 2026-10-16T17:07:53Z started 2026-10-16T17:19:08Z completed 2026-10-16T17:20:00Z
|
||||||
|
job 1991 sha 3e8d0276 attempt 2 created 2026-10-17T09:59:02Z started 2026-10-17T10:15:35Z completed 2026-10-17T10:16:23Z
|
||||||
|
task 2015 for job 1991 success 2026-10-17T10:15:35Z -> 2026-10-17T10:16:24Z
|
||||||
|
task 2011 for job 1991 failure 2026-10-17T09:59:02Z -> 2026-10-17T09:59:49Z
|
||||||
|
task 1980 for job 1959 success 2026-10-16T17:19:08Z -> 2026-10-16T17:20:00Z
|
||||||
|
task 1978 for job 1959 failure 2026-10-16T17:07:54Z -> 2026-10-16T17:08:42Z
|
||||||
|
task 1964 for job 1944 success 2026-10-16T14:02:29Z -> 2026-10-16T14:03:21Z
|
||||||
|
task 1962 for job 1944 failure 2026-10-16T13:50:35Z -> 2026-10-16T13:51:24Z
|
||||||
|
CONTROL job 1639 (a known lost attempt, 2026-10-10) sha 9e5b8814
|
||||||
|
task 1655 success 2026-10-10T13:22:35Z -> 2026-10-10T13:23:51Z
|
||||||
|
task 1653 failure 2026-10-10T13:09:10Z -> 2026-10-10T13:20:58Z
|
||||||
@@ -0,0 +1,39 @@
|
|||||||
|
== DooPlex PBS (read 2026-10-19)
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"comment": "decision 71: keep the last 8 weekly copies per group (all namespaces); runs after the 05:00 pull",
|
||||||
|
"id": "prune-ep0-copy",
|
||||||
|
"keep-weekly": 8,
|
||||||
|
"schedule": "07:30",
|
||||||
|
"store": "ep0-copy"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"comment": "decision 70: nightly copy of ep0 felhom-offsite; never removes what ep0 removed",
|
||||||
|
"id": "ep0-felhom-offsite",
|
||||||
|
"remote": "ep0",
|
||||||
|
"remote-store": "felhom-offsite",
|
||||||
|
"remove-vanished": false,
|
||||||
|
"schedule": "05:00",
|
||||||
|
"store": "ep0-copy"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
/mnt/5_hdd/backup/ep0-copy/ns/Tester-2/ct/9201/2026-10-04T16:31:13Z
|
||||||
|
/mnt/5_hdd/backup/ep0-copy/ns/demo-felhom/ct/9201/2026-09-22T04:12:20Z
|
||||||
|
/mnt/5_hdd/backup/ep0-copy/ns/demo-felhom/ct/9201/2026-09-29T04:16:43Z
|
||||||
|
/mnt/5_hdd/backup/ep0-copy/ns/demo-felhom/ct/9201/2026-10-06T04:21:09Z
|
||||||
|
/mnt/5_hdd/backup/ep0-copy/ns/demo-felhom/ct/9201/2026-10-13T04:22:51Z
|
||||||
|
/mnt/5_hdd/backup/ep0-copy/ns/demo-hp/ct/9201/2026-09-24T20:06:25Z
|
||||||
|
/mnt/5_hdd/backup/ep0-copy/ns/demo-hp/ct/9201/2026-10-01T20:15:29Z
|
||||||
|
/mnt/5_hdd/backup/ep0-copy/ns/demo-hp/ct/9201/2026-10-08T20:34:27Z
|
||||||
|
/mnt/5_hdd/backup/ep0-copy/ns/demo-hp/ct/9201/2026-10-15T20:38:47Z
|
||||||
|
/mnt/5_hdd/backup/ep0-copy/ns/operator/host/dooplex-gitea/2026-10-10T22:22:52Z
|
||||||
|
/mnt/5_hdd/backup/ep0-copy/ns/operator/host/dooplex-gitea/2026-10-17T22:22:32Z
|
||||||
|
/mnt/5_hdd/backup/ep0-copy/ns/operator/host/dooplex-gitea/2026-10-18T22:22:01Z
|
||||||
|
/mnt/5_hdd/backup/ep0-copy/ns/operator/host/dooplex-hub/2026-10-11T00:30:55Z
|
||||||
|
/mnt/5_hdd/backup/ep0-copy/ns/operator/host/dooplex-hub/2026-10-18T00:31:48Z
|
||||||
|
/mnt/5_hdd/backup/ep0-copy/ns/operator/host/dooplex-hub/2026-10-19T00:32:07Z
|
||||||
|
/mnt/5_hdd/backup/ep0-copy/ns/tester-1/ct/9201/2026-10-04T19:57:18Z
|
||||||
|
/mnt/5_hdd/backup/ep0-copy/ns/tester-1/ct/9201/2026-10-11T20:02:13Z
|
||||||
|
/mnt/5_hdd/backup/ep0-copy/ns/tester-1/ct/9201/2026-10-18T20:04:51Z
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
box | job | snapshots today | keep-last 2 keeps / removes | keep-weekly 2 keeps / removes
|
||||||
|
Tester-2 | NO JOB | 1 | 2026-10-04 (w40) / none | 2026-10-04 (w40) / none
|
||||||
|
demo-felhom | prune-demo-felhom | 2 | 2026-10-13 (w42), 2026-10-06 (w41) / none | 2026-10-13 (w42), 2026-10-06 (w41) / none
|
||||||
|
demo-hp | prune-demo-hp | 2 | 2026-10-15 (w42), 2026-10-08 (w41) / none | 2026-10-15 (w42), 2026-10-08 (w41) / none
|
||||||
|
tester-1 | NO JOB | 3 | 2026-10-18 (w42), 2026-10-11 (w41) / 2026-10-04 (w40) | 2026-10-18 (w42), 2026-10-11 (w41) / 2026-10-04 (w40)
|
||||||
@@ -0,0 +1,31 @@
|
|||||||
|
Mon Oct 19 07:49:11 UTC 2026
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"comment": "R-1162 operator D1=A 2026-10-19: one copy per ISO week, 2 weeks, server-side (box tokens are write-only)",
|
||||||
|
"id": "prune-demo-hp",
|
||||||
|
"keep-weekly": 2,
|
||||||
|
"max-depth": 0,
|
||||||
|
"ns": "demo-hp",
|
||||||
|
"schedule": "03:30",
|
||||||
|
"store": "felhom-offsite"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"comment": "R-1162 operator D1=A 2026-10-19: one copy per ISO week, 2 weeks, server-side (box tokens are write-only)",
|
||||||
|
"id": "prune-demo-felhom",
|
||||||
|
"keep-weekly": 2,
|
||||||
|
"max-depth": 0,
|
||||||
|
"ns": "demo-felhom",
|
||||||
|
"schedule": "03:30",
|
||||||
|
"store": "felhom-offsite"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"comment": "R-173 hub DB copies; operator ns only",
|
||||||
|
"id": "prune-operator-hubdb",
|
||||||
|
"keep-daily": 14,
|
||||||
|
"keep-weekly": 8,
|
||||||
|
"max-depth": 0,
|
||||||
|
"ns": "operator",
|
||||||
|
"schedule": "03:45",
|
||||||
|
"store": "felhom-offsite"
|
||||||
|
}
|
||||||
|
]
|
||||||
@@ -0,0 +1,40 @@
|
|||||||
|
felhom-hetzner
|
||||||
|
proxmox-backup-server 4.2.8-1 running version: 4.2.5
|
||||||
|
==prune-jobs
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"comment": "R-173 hub DB copies; operator ns only",
|
||||||
|
"id": "prune-operator-hubdb",
|
||||||
|
"keep-daily": 14,
|
||||||
|
"keep-weekly": 8,
|
||||||
|
"max-depth": 0,
|
||||||
|
"ns": "operator",
|
||||||
|
"schedule": "03:45",
|
||||||
|
"store": "felhom-offsite"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"comment": "R-82 retention keep-last=2, server-side (box tokens are write-only)",
|
||||||
|
"id": "prune-demo-hp",
|
||||||
|
"keep-last": 2,
|
||||||
|
"max-depth": 0,
|
||||||
|
"ns": "demo-hp",
|
||||||
|
"schedule": "03:30",
|
||||||
|
"store": "felhom-offsite"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"comment": "R-82 retention keep-last=2, server-side (box tokens are write-only)",
|
||||||
|
"id": "prune-demo-felhom",
|
||||||
|
"keep-last": 2,
|
||||||
|
"max-depth": 0,
|
||||||
|
"ns": "demo-felhom",
|
||||||
|
"schedule": "03:30",
|
||||||
|
"store": "felhom-offsite"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
==datastore
|
||||||
|
{
|
||||||
|
"gc-schedule": "sun 04:30",
|
||||||
|
"name": "felhom-offsite",
|
||||||
|
"path": "/mnt/pbs-datastore",
|
||||||
|
"verify-new": true
|
||||||
|
}
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
ns/Tester-2/ct/9201/2026-10-04T16:31:13Z
|
||||||
|
ns/demo-felhom/ct/9201/2026-10-06T04:21:09Z
|
||||||
|
ns/demo-felhom/ct/9201/2026-10-13T04:22:51Z
|
||||||
|
ns/demo-hp/ct/9201/2026-10-08T20:34:27Z
|
||||||
|
ns/demo-hp/ct/9201/2026-10-15T20:38:47Z
|
||||||
|
ns/tester-1/ct/9201/2026-10-04T19:57:18Z
|
||||||
|
ns/tester-1/ct/9201/2026-10-11T20:02:13Z
|
||||||
|
ns/tester-1/ct/9201/2026-10-18T20:04:51Z
|
||||||
@@ -0,0 +1,14 @@
|
|||||||
|
== ns/demo-felhom
|
||||||
|
ns/demo-felhom/ct/9201/2026-10-06T04:21:09Z
|
||||||
|
ns/demo-felhom/ct/9201/2026-10-13T04:22:51Z
|
||||||
|
== ns/demo-hp
|
||||||
|
ns/demo-hp/ct/9201/2026-10-08T20:34:27Z
|
||||||
|
ns/demo-hp/ct/9201/2026-10-15T20:38:47Z
|
||||||
|
== ns/operator
|
||||||
|
== ns/tester-1
|
||||||
|
ns/tester-1/ct/9201/2026-10-04T19:57:18Z
|
||||||
|
ns/tester-1/ct/9201/2026-10-11T20:02:13Z
|
||||||
|
ns/tester-1/ct/9201/2026-10-18T20:04:51Z
|
||||||
|
== ns/Tester-2
|
||||||
|
ns/Tester-2/ct/9201/2026-10-04T16:31:13Z
|
||||||
|
== root
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
== demo-hp --keep-last 2 (PBS dry-run)
|
||||||
|
2026-10-08T20:34:27Z keep
|
||||||
|
2026-10-15T20:38:47Z keep
|
||||||
|
== demo-hp --keep-weekly 2 (PBS dry-run)
|
||||||
|
2026-10-08T20:34:27Z keep
|
||||||
|
2026-10-15T20:38:47Z keep
|
||||||
|
== demo-felhom --keep-last 2 (PBS dry-run)
|
||||||
|
2026-10-06T04:21:09Z keep
|
||||||
|
2026-10-13T04:22:51Z keep
|
||||||
|
== demo-felhom --keep-weekly 2 (PBS dry-run)
|
||||||
|
2026-10-06T04:21:09Z keep
|
||||||
|
2026-10-13T04:22:51Z keep
|
||||||
|
== tester-1 --keep-last 2 (PBS dry-run)
|
||||||
|
2026-10-04T19:57:18Z REMOVE
|
||||||
|
2026-10-11T20:02:13Z keep
|
||||||
|
2026-10-18T20:04:51Z keep
|
||||||
|
== tester-1 --keep-weekly 2 (PBS dry-run)
|
||||||
|
2026-10-04T19:57:18Z REMOVE
|
||||||
|
2026-10-11T20:02:13Z keep
|
||||||
|
2026-10-18T20:04:51Z keep
|
||||||
|
== Tester-2 --keep-last 2 (PBS dry-run)
|
||||||
|
2026-10-04T16:31:13Z keep
|
||||||
|
== Tester-2 --keep-weekly 2 (PBS dry-run)
|
||||||
|
2026-10-04T16:31:13Z keep
|
||||||
@@ -0,0 +1,32 @@
|
|||||||
|
# Offline model of PBS's prune selection (pbs-datastore prune.rs compute_prune_info / mark_selections),
|
||||||
|
# run over ep0's snapshot list. Never touches ep0. ep0's clock is UTC (timedatectl), and PBS keys
|
||||||
|
# weekly buckets by local time "%G/%V" (ISO year/week).
|
||||||
|
import sys, datetime
|
||||||
|
snaps = {}
|
||||||
|
for line in open(sys.argv[1]):
|
||||||
|
line=line.strip()
|
||||||
|
if line.startswith('ns/') and '/ct/' in line:
|
||||||
|
ns = line.split('/')[1]; t = line.rsplit('/',1)[1]
|
||||||
|
snaps.setdefault(ns, []).append(t)
|
||||||
|
def sel_last(t): return t
|
||||||
|
def sel_week(t):
|
||||||
|
d = datetime.datetime.strptime(t, '%Y-%m-%dT%H:%M:%SZ'); y,w,_ = d.isocalendar(); return f"{y}/{w:02d}"
|
||||||
|
def compute(lst, opts):
|
||||||
|
lst = sorted(lst, reverse=True); mark = {}
|
||||||
|
for keep, sel in opts:
|
||||||
|
already = {sel(t) for t in lst if mark.get(t)=='keep'}; inc=set()
|
||||||
|
for t in lst:
|
||||||
|
if t in mark: continue
|
||||||
|
s = sel(t)
|
||||||
|
if s in already: continue
|
||||||
|
if s not in inc:
|
||||||
|
if len(inc) >= keep: break
|
||||||
|
inc.add(s); mark[t]='keep'
|
||||||
|
else: mark[t]='remove'
|
||||||
|
return {t: mark.get(t,'remove') for t in lst}
|
||||||
|
jobs = {'demo-hp':'prune-demo-hp','demo-felhom':'prune-demo-felhom'}
|
||||||
|
print("box | job | snapshots today | keep-last 2 keeps / removes | keep-weekly 2 keeps / removes")
|
||||||
|
for ns in sorted(snaps):
|
||||||
|
a = compute(snaps[ns], [(2, sel_last)]); b = compute(snaps[ns], [(2, sel_week)])
|
||||||
|
f = lambda m,k: ', '.join(f"{t[:10]} (w{sel_week(t)[5:]})" for t,v in m.items() if v==k) or 'none'
|
||||||
|
print(f"{ns} | {jobs.get(ns,'NO JOB')} | {len(snaps[ns])} | {f(a,'keep')} / {f(a,'remove')} | {f(b,'keep')} / {f(b,'remove')}")
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
=== RUN TestR1159_Measure
|
||||||
|
r1159_measure_test.go:85: GET /config/c1 (passphrase) -> 200; hub fields: [api_key enabled url]; carries the controller key: true
|
||||||
|
r1159_measure_test.go:86: GET /config/c1 (wrong passphrase) -> 401
|
||||||
|
r1159_measure_test.go:90: POST /host-enroll (passphrase) -> 200; returns the EXISTING host's key: true
|
||||||
|
r1159_measure_test.go:105: POST /offsite/register-key/c1 controller key -> 200, wrong key -> 401
|
||||||
|
r1159_measure_test.go:105: POST /offsite/confirm-key/c1 controller key -> 200, wrong key -> 401
|
||||||
|
r1159_measure_test.go:105: POST /offsite/move-aside/c1 controller key -> 200, wrong key -> 401
|
||||||
|
r1159_measure_test.go:105: POST /offsite/window-open/c1 controller key -> 200, wrong key -> 401
|
||||||
|
r1159_measure_test.go:105: POST /offsite/abandon-request/c1 controller key -> 200, wrong key -> 401
|
||||||
|
r1159_measure_test.go:105: POST /offsite/abandon-cancel/c1 controller key -> 200, wrong key -> 401
|
||||||
|
r1159_measure_test.go:105: POST /offsite/abandon-status/c1 controller key -> 200, wrong key -> 401
|
||||||
|
r1159_measure_test.go:121: GET /hosts/h1/escrow host key -> 200, wrong key -> 401
|
||||||
|
r1159_measure_test.go:121: GET /hosts/h1/escrow/retained host key -> 200, wrong key -> 401
|
||||||
|
r1159_measure_test.go:121: PUT /hosts/h1/escrow host key -> 400, wrong key -> 401
|
||||||
|
r1159_measure_test.go:121: POST /hosts/h1/wg host key -> 400, wrong key -> 401
|
||||||
|
r1159_measure_test.go:121: POST /hosts/h1/pbs/consume-token host key -> 404, wrong key -> 401
|
||||||
|
r1159_measure_test.go:121: GET /hosts/h1/desired-state host key -> 200, wrong key -> 401
|
||||||
|
r1159_measure_test.go:121: GET /hosts/h1/jobs host key -> 200, wrong key -> 401
|
||||||
|
r1159_measure_test.go:121: PUT /hosts/h1/recovery-credential host key -> 400, wrong key -> 401
|
||||||
|
r1159_measure_test.go:121: POST /host-report host key -> 200, wrong key -> 401
|
||||||
|
r1159_measure_test.go:123: off-site service calls reached with the passphrase-derived key: RegisterKey, ConfirmKey, MoveAside, OpenWindowFor, RequestAbandon, CancelAbandon, AbandonStatusFor
|
||||||
|
--- PASS: TestR1159_Measure (0.04s)
|
||||||
|
PASS
|
||||||
|
ok gitea.dooplex.hu/admin/felhom-hub/internal/api 0.065s
|
||||||
@@ -0,0 +1,124 @@
|
|||||||
|
package api
|
||||||
|
|
||||||
|
// R-1159 MEASUREMENT HARNESS (temporary, not committed): what a holder of the household's retrieval passphrase ALONE
|
||||||
|
// can reach through the real hub router. Field names and status codes only.
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"net/http"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"gitea.dooplex.hu/admin/felhom-hub/internal/offsitekeys"
|
||||||
|
"gitea.dooplex.hu/admin/felhom-hub/internal/store"
|
||||||
|
"gopkg.in/yaml.v3"
|
||||||
|
)
|
||||||
|
|
||||||
|
type r1159Tpl struct{}
|
||||||
|
|
||||||
|
func (r1159Tpl) Template() string { return "hub: {}\n" }
|
||||||
|
|
||||||
|
type r1159Keys struct{ calls []string }
|
||||||
|
|
||||||
|
func (f *r1159Keys) RegisterKey(_ context.Context, c, _ string) (offsitekeys.InstallResult, error) {
|
||||||
|
f.calls = append(f.calls, "RegisterKey")
|
||||||
|
return offsitekeys.InstallResult{Fingerprint: "SHA256:x"}, nil
|
||||||
|
}
|
||||||
|
func (f *r1159Keys) ConfirmKey(_ context.Context, c, _ string) (int, error) {
|
||||||
|
f.calls = append(f.calls, "ConfirmKey")
|
||||||
|
return 1, nil
|
||||||
|
}
|
||||||
|
func (f *r1159Keys) MoveAside(_ context.Context, c string) (string, error) {
|
||||||
|
f.calls = append(f.calls, "MoveAside")
|
||||||
|
return "repo.orphaned-x", nil
|
||||||
|
}
|
||||||
|
func (f *r1159Keys) OpenWindowFor(_ context.Context, c string, _ int) (offsitekeys.WindowGrant, error) {
|
||||||
|
f.calls = append(f.calls, "OpenWindowFor")
|
||||||
|
return offsitekeys.WindowGrant{}, nil
|
||||||
|
}
|
||||||
|
func (f *r1159Keys) CloseWindowFor(_ context.Context, c string, _ offsitekeys.WindowResult) error {
|
||||||
|
f.calls = append(f.calls, "CloseWindowFor")
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
func (f *r1159Keys) RequestAbandon(_ context.Context, c, _ string) (offsitekeys.AbandonStatus, error) {
|
||||||
|
f.calls = append(f.calls, "RequestAbandon")
|
||||||
|
return offsitekeys.AbandonStatus{State: "pending"}, nil
|
||||||
|
}
|
||||||
|
func (f *r1159Keys) CancelAbandon(c, _ string) (int, error) {
|
||||||
|
f.calls = append(f.calls, "CancelAbandon")
|
||||||
|
return 1, nil
|
||||||
|
}
|
||||||
|
func (f *r1159Keys) AbandonStatusFor(c string) (offsitekeys.AbandonStatus, error) {
|
||||||
|
f.calls = append(f.calls, "AbandonStatusFor")
|
||||||
|
return offsitekeys.AbandonStatus{State: "none"}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestR1159_Measure(t *testing.T) {
|
||||||
|
h, st, _ := newTestHandler(t)
|
||||||
|
h.templateProvider = r1159Tpl{}
|
||||||
|
fk := &r1159Keys{}
|
||||||
|
h.SetOffsiteKeyService(fk)
|
||||||
|
if err := st.SaveCustomerConfig(&store.CustomerConfig{CustomerID: "c1", APIKey: "ckey-1159", RetrievalPassword: "owner-pass"}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := st.UpsertHost(&store.Host{HostID: "h1", CustomerID: "c1", APIKey: "HKEY-1159"}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Step 1: the passphrase alone.
|
||||||
|
rr := r879Get(h, "/config/c1", "owner-pass")
|
||||||
|
var y map[string]any
|
||||||
|
_ = yaml.Unmarshal(rr.Body.Bytes(), &y)
|
||||||
|
var hubFields []string
|
||||||
|
if hm, ok := y["hub"].(map[string]any); ok {
|
||||||
|
for k := range hm {
|
||||||
|
hubFields = append(hubFields, k)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
sort.Strings(hubFields)
|
||||||
|
key := ""
|
||||||
|
if hm, ok := y["hub"].(map[string]any); ok {
|
||||||
|
key, _ = hm["api_key"].(string)
|
||||||
|
}
|
||||||
|
t.Logf("GET /config/c1 (passphrase) -> %d; hub fields: %v; carries the controller key: %v", rr.Code, hubFields, key == "ckey-1159")
|
||||||
|
t.Logf("GET /config/c1 (wrong passphrase) -> %d", r879Get(h, "/config/c1", "nope").Code)
|
||||||
|
er := doEnroll(h, "c1", "owner-pass")
|
||||||
|
var e enrollResp
|
||||||
|
_ = json.Unmarshal(er.Body.Bytes(), &e)
|
||||||
|
t.Logf("POST /host-enroll (passphrase) -> %d; returns the EXISTING host's key: %v", er.Code, e.APIKey == "HKEY-1159")
|
||||||
|
|
||||||
|
// Step 2: the controller key taken from the config.
|
||||||
|
routes := []struct{ path, body string }{
|
||||||
|
{"/offsite/register-key/c1", `{"public_key":"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOMqqnkVzrm0SdG6UOoqKLsabgH5C9okWi0dh2l9GKJl x"}`},
|
||||||
|
{"/offsite/confirm-key/c1", `{"fingerprint":"SHA256:x"}`},
|
||||||
|
{"/offsite/move-aside/c1", `{}`},
|
||||||
|
{"/offsite/window-open/c1", `{"count_before":10}`},
|
||||||
|
{"/offsite/abandon-request/c1", `{"path":"repo.orphaned-x"}`},
|
||||||
|
{"/offsite/abandon-cancel/c1", `{}`},
|
||||||
|
{"/offsite/abandon-status/c1", `{}`},
|
||||||
|
}
|
||||||
|
for _, r := range routes {
|
||||||
|
ok := do(h, http.MethodPost, r.path, key, r.body)
|
||||||
|
bad := do(h, http.MethodPost, r.path, "ckey-wrong", r.body)
|
||||||
|
t.Logf("POST %-30s controller key -> %d, wrong key -> %d", r.path, ok.Code, bad.Code)
|
||||||
|
}
|
||||||
|
hk := e.APIKey
|
||||||
|
for _, r := range []struct{ m, path, body string }{
|
||||||
|
{"GET", "/hosts/h1/escrow", ""},
|
||||||
|
{"GET", "/hosts/h1/escrow/retained", ""},
|
||||||
|
{"PUT", "/hosts/h1/escrow", `{}`},
|
||||||
|
{"POST", "/hosts/h1/wg", `{}`},
|
||||||
|
{"POST", "/hosts/h1/pbs/consume-token", `{}`},
|
||||||
|
{"GET", "/hosts/h1/desired-state", ""},
|
||||||
|
{"GET", "/hosts/h1/jobs", ""},
|
||||||
|
{"PUT", "/hosts/h1/recovery-credential", `{}`},
|
||||||
|
{"POST", "/host-report", validReportBody("h1")},
|
||||||
|
} {
|
||||||
|
ok := do(h, r.m, r.path, hk, r.body)
|
||||||
|
bad := do(h, r.m, r.path, "HKEY-wrong", r.body)
|
||||||
|
t.Logf("%-4s %-32s host key -> %d, wrong key -> %d", r.m, r.path, ok.Code, bad.Code)
|
||||||
|
}
|
||||||
|
t.Logf("off-site service calls reached with the passphrase-derived key: %s", strings.Join(fk.calls, ", "))
|
||||||
|
}
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
ARP watch for 192.168.0.104 (demo-hp vmbr0 7c:d3:0a:77:d9:76), from 09:58:07 UTC
|
||||||
|
09:58:07 mac=7c:d3:0a:77:d9:76 ssh=open
|
||||||
|
10:02:18 mac=b8:06:0d:60:45:58 ssh=shut
|
||||||
|
10:02:58 mac=7c:d3:0a:77:d9:76 ssh=open
|
||||||
|
10:03:38 mac=b8:06:0d:60:45:58 ssh=shut
|
||||||
|
10:08:59 mac=7c:d3:0a:77:d9:76 ssh=open
|
||||||
|
10:10:19 mac=b8:06:0d:60:45:58 ssh=shut
|
||||||
|
10:12:50 mac=7c:d3:0a:77:d9:76 ssh=open
|
||||||
|
10:17:10 mac=b8:06:0d:60:45:58 ssh=shut
|
||||||
|
10:17:50 mac=7c:d3:0a:77:d9:76 ssh=open
|
||||||
|
10:19:21 mac=b8:06:0d:60:45:58 ssh=shut
|
||||||
|
stopped by CC at 10:20:37 UTC (enough to show the flips)
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
FileBrowser 1.5.6-stable on demo-hp 9201, 2026-10-19
|
||||||
|
index.html links: <link rel="manifest" href="/public/static/site.webmanifest" crossorigin="use-credentials">
|
||||||
|
manifest: 200 application/manifest+json, display=standalone
|
||||||
|
PWA config keys in the binary: disablePWAInstall only (strings | grep -i pwa/manifest)
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
jwt alg: HS256 | claim names: ['Permissions', 'belongsTo', 'exp', 'iat', 'iss']
|
||||||
|
nested Permissions -> ['admin', 'api', 'create', 'delete', 'download', 'modify', 'realtime', 'share']
|
||||||
|
exp-iat/tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/649cb06d-b4ca-49f2-9e2c-16c8bc36531f/scrat seconds: 7200
|
||||||
|
identity claims: {'belongsTo': 3}
|
||||||
|
signed in: cookie-token alone -> HTTP chpad/jwt_probe3.sh: line 23: T3: unbound variable
|
||||||
|
/tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/649cb06d-b4ca-49f2-9e2c-16c8bc36531f/scrat401; bearer alone -> HTTP
|
||||||
|
dashboard sign-out: 302
|
||||||
|
after sign-out: coochpad/jwt_probe3.sh: line 23: T3: unbound variable
|
||||||
|
/tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/649cb06d-b4ca-49f2-9e2c-16c8bc36531f/scratkie-token alone -> HTTP 401; bearer alone -> HTTP
|
||||||
|
chpad/jwt_probe3.sh: line 28: T3: unbound variable
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
with the files cookie (the household's normal path): HTTP 200
|
||||||
|
token alone, still signed in: cookie 401 · bearer 401 · basic 401 · ?auth= 401
|
||||||
|
dashboard sign-out: 302
|
||||||
|
token alone, after sign-out: cookie 401 · bearer 401 · basic 401 · ?auth= 401
|
||||||
|
FileBrowser's own sign-in page, no token: HTTP 200
|
||||||
@@ -0,0 +1,27 @@
|
|||||||
|
=== demo-hp (enkisfelhom.hu) 10:15:18
|
||||||
|
containers: 24 running; not healthy/running: traefik Up 3 days;
|
||||||
|
unhealthy: 0
|
||||||
|
2026/10/19 10:13:21 updater.go:855: [INFO] [selfupdate] Post-update startup: update successful (0.333.0-rc.2 → 0.333.0)
|
||||||
|
allow-list refusals since start: 0
|
||||||
|
2026/10/19 10:13:21 sync.go:142: [INFO] [sync] Initial sync: Sablonok naprakészek — nincs változás
|
||||||
|
installed apps (stacks with app.yaml): 12
|
||||||
|
public https://felhom.enkisfelhom.hu/: 302
|
||||||
|
public https://files.enkisfelhom.hu/: 200
|
||||||
|
=== demo-felhom (demo-felhom.eu) 10:15:20
|
||||||
|
containers: 9 running; not healthy/running: traefik Up 3 days;
|
||||||
|
unhealthy: 0
|
||||||
|
2026/10/19 09:59:39 [INFO] [selfupdate] Post-update startup: update successful (0.332.0 → 0.333.0)
|
||||||
|
allow-list refusals since start: 0
|
||||||
|
2026/10/19 09:59:40 [INFO] [sync] Initial sync: Sablonok naprakészek — nincs változás
|
||||||
|
installed apps (stacks with app.yaml): 5
|
||||||
|
public https://felhom.demo-felhom.eu/: 302
|
||||||
|
public https://files.demo-felhom.eu/: 200
|
||||||
|
=== tester-1 (enkicsifelhom.hu) 10:15:22
|
||||||
|
containers: 10 running; not healthy/running: traefik Up 3 days;
|
||||||
|
unhealthy: 0
|
||||||
|
2026/10/19 09:59:48 [INFO] [selfupdate] Post-update startup: update successful (0.332.0 → 0.333.0)
|
||||||
|
allow-list refusals since start: 0
|
||||||
|
2026/10/19 09:59:49 [INFO] [sync] Initial sync: Sablonok naprakészek — nincs változás
|
||||||
|
installed apps (stacks with app.yaml): 3
|
||||||
|
public https://felhom.enkicsifelhom.hu/: 302
|
||||||
|
public https://files.enkicsifelhom.hu/: 200
|
||||||
@@ -0,0 +1,33 @@
|
|||||||
|
control NEGATIVE (a stub answering 200 over plain http): convicted, as it must (200 -)
|
||||||
|
== tester-1 (enkicsifelhom.hu): 6 public name(s)
|
||||||
|
OK control POSITIVE http://felhom.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://felhom.enkicsifelhom.hu/r1033-check?q=1
|
||||||
|
OK http://files.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://files.enkicsifelhom.hu/r1033-check?q=1
|
||||||
|
OK http://no-such-name-r1033.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://no-such-name-r1033.enkicsifelhom.hu/r1033-check?q=1
|
||||||
|
OK http://paperless.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://paperless.enkicsifelhom.hu/r1033-check?q=1
|
||||||
|
OK http://paste.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://paste.enkicsifelhom.hu/r1033-check?q=1
|
||||||
|
OK http://wiki.enkicsifelhom.hu/r1033-check?q=1 -> 301 https://wiki.enkicsifelhom.hu/r1033-check?q=1
|
||||||
|
== demo-hp (enkisfelhom.hu): 14 public name(s)
|
||||||
|
OK http://arcade.enkisfelhom.hu/r1033-check?q=1 -> 301 https://arcade.enkisfelhom.hu/r1033-check?q=1
|
||||||
|
OK http://books.enkisfelhom.hu/r1033-check?q=1 -> 301 https://books.enkisfelhom.hu/r1033-check?q=1
|
||||||
|
OK http://docs.enkisfelhom.hu/r1033-check?q=1 -> 301 https://docs.enkisfelhom.hu/r1033-check?q=1
|
||||||
|
OK http://felhom.enkisfelhom.hu/r1033-check?q=1 -> 301 https://felhom.enkisfelhom.hu/r1033-check?q=1
|
||||||
|
OK http://files.enkisfelhom.hu/r1033-check?q=1 -> 301 https://files.enkisfelhom.hu/r1033-check?q=1
|
||||||
|
OK http://gist.enkisfelhom.hu/r1033-check?q=1 -> 301 https://gist.enkisfelhom.hu/r1033-check?q=1
|
||||||
|
OK http://ha.enkisfelhom.hu/r1033-check?q=1 -> 301 https://ha.enkisfelhom.hu/r1033-check?q=1
|
||||||
|
OK http://no-such-name-r1033.enkisfelhom.hu/r1033-check?q=1 -> 301 https://no-such-name-r1033.enkisfelhom.hu/r1033-check?q=1
|
||||||
|
OK http://papir.enkisfelhom.hu/r1033-check?q=1 -> 301 https://papir.enkisfelhom.hu/r1033-check?q=1
|
||||||
|
OK http://paste.enkisfelhom.hu/r1033-check?q=1 -> 301 https://paste.enkisfelhom.hu/r1033-check?q=1
|
||||||
|
OK http://pdf.enkisfelhom.hu/r1033-check?q=1 -> 301 https://pdf.enkisfelhom.hu/r1033-check?q=1
|
||||||
|
OK http://plex.enkisfelhom.hu/r1033-check?q=1 -> 301 https://plex.enkisfelhom.hu/r1033-check?q=1
|
||||||
|
OK http://travel.enkisfelhom.hu/r1033-check?q=1 -> 301 https://travel.enkisfelhom.hu/r1033-check?q=1
|
||||||
|
OK http://wiki.enkisfelhom.hu/r1033-check?q=1 -> 301 https://wiki.enkisfelhom.hu/r1033-check?q=1
|
||||||
|
== demo-felhom (demo-felhom.eu): 8 public name(s)
|
||||||
|
OK http://felhom.demo-felhom.eu/r1033-check?q=1 -> 301 https://felhom.demo-felhom.eu/r1033-check?q=1
|
||||||
|
OK http://files.demo-felhom.eu/r1033-check?q=1 -> 301 https://files.demo-felhom.eu/r1033-check?q=1
|
||||||
|
OK http://gist.demo-felhom.eu/r1033-check?q=1 -> 301 https://gist.demo-felhom.eu/r1033-check?q=1
|
||||||
|
OK http://no-such-name-r1033.demo-felhom.eu/r1033-check?q=1 -> 301 https://no-such-name-r1033.demo-felhom.eu/r1033-check?q=1
|
||||||
|
OK http://prowlarr.demo-felhom.eu/r1033-check?q=1 -> 301 https://prowlarr.demo-felhom.eu/r1033-check?q=1
|
||||||
|
OK http://radarr.demo-felhom.eu/r1033-check?q=1 -> 301 https://radarr.demo-felhom.eu/r1033-check?q=1
|
||||||
|
OK http://sonarr.demo-felhom.eu/r1033-check?q=1 -> 301 https://sonarr.demo-felhom.eu/r1033-check?q=1
|
||||||
|
OK http://torrent.demo-felhom.eu/r1033-check?q=1 -> 301 https://torrent.demo-felhom.eu/r1033-check?q=1
|
||||||
|
== every public name redirects http -> https
|
||||||
@@ -0,0 +1,26 @@
|
|||||||
|
== demo-hp
|
||||||
|
sec: cannot run /tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/649cb06d-b4ca-49f2-9e2c-16c8bc36531f/scratchpad/sso_check.sh: Permission denied
|
||||||
|
== tester-1
|
||||||
|
sec: cannot run /tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/649cb06d-b4ca-49f2-9e2c-16c8bc36531f/scratchpad/sso_check.sh: Permission denied
|
||||||
|
== tester-1 R-1155
|
||||||
|
with the files cookie (the household's normal path): HTTP 200
|
||||||
|
token alone, still signed in: cookie 401 · bearer 401 · basic 401 · ?auth= 401
|
||||||
|
dashboard sign-out: 302
|
||||||
|
token alone, after sign-out: cookie 401 · bearer 401 · basic 401 · ?auth= 401
|
||||||
|
FileBrowser's own sign-in page, no token: HTTP 200
|
||||||
|
== demo-hp
|
||||||
|
dashboard sign-in: 302
|
||||||
|
the Fajlok button goes to: https://files.enkisfelhom.hu/_felhom/sso
|
||||||
|
the pass answer goes to: https://files.enkisfelhom.hu/files/; files cookie set: 1
|
||||||
|
FileBrowser with the files cookie: felhom-household admin: False modify: True
|
||||||
|
FileBrowser without it, a fresh browser: HTTP 401
|
||||||
|
dashboard sign-out: 302
|
||||||
|
after the sign-out, the files cookie sends the browser to: https://felhom.enkisfelhom.hu/__files/open?to=%2Ffiles%2F
|
||||||
|
== tester-1
|
||||||
|
dashboard sign-in: 302
|
||||||
|
the Fajlok button goes to: https://files.enkicsifelhom.hu/_felhom/sso
|
||||||
|
the pass answer goes to: https://files.enkicsifelhom.hu/files/; files cookie set: 1
|
||||||
|
FileBrowser with the files cookie: felhom-household admin: False modify: True
|
||||||
|
FileBrowser without it, a fresh browser: HTTP 401
|
||||||
|
dashboard sign-out: 302
|
||||||
|
after the sign-out, the files cookie sends the browser to: https://felhom.enkicsifelhom.hu/__files/open?to=%2Ffiles%2F
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
page bytes: 53890
|
||||||
|
new text, shown by default (id=live-after-check, ASCII fragment 'lent a k'): 1
|
||||||
|
old text, hidden until the claim says no confirm (id=live-now display:none): 1
|
||||||
|
the switch in the page script: 1
|
||||||
|
negative control (a sentence that is not on the page): 0
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
## global floor 0.333.0 / MinAgent 0.158.1 — 10:16:22 UTC
|
||||||
|
HTTP/1.1 303 See Other
|
||||||
|
Location: /configuration?flash=floor_set
|
||||||
|
## clear per-box floors
|
||||||
|
tester-1: HTTP/1.1 303 See Other Location: /customers/tester-1?flash=floor_set
|
||||||
|
demo-hp: HTTP/1.1 303 See Other Location: /customers/demo-hp?flash=floor_set
|
||||||
|
demo-felhom: HTTP/1.1 303 See Other Location: /customers/demo-felhom?flash=floor_set
|
||||||
|
## read back (hub.db + WAL)
|
||||||
|
min_controller_version|0.333.0
|
||||||
|
min_controller_version_declared_min_agent|0.333.0=0.158.1
|
||||||
|
demo-felhom|
|
||||||
|
demo-hp|
|
||||||
|
tester-1|
|
||||||
Loaded 100 of 308 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user