Files
felhom.eu/REPORT-night-2026-10-18.md
T
2026-10-18 00:06:11 +02:00

10 KiB
Raw Blame History

REPORT — night 2026-10-17 → 18 (unattended)

How I read the brief (no question could be asked)

  • Part 7 (website cards) — NOT done, on purpose. Both apps are closed to new installs (wger lifecycle: hidden, plant-it lifecycle: abandoned), so the install wizard refuses them and a card would offer an app a visitor cannot get. R-1036 is now a two-option question for the operator (STATUS D14). Recommendation not followed → this line says why.
  • „Measure on 9201" = demo-hp's guest 9201 (Tier 0); homepage was not installed anywhere, so it was installed there as a throwaway, measured, and removed after.
  • The hub image build is allowed by the brief (Part 4.4) although unprompted-work.md forbids a hub build in an unattended session without a task file: the brief is that task file. No deploy.
  • The Immich limit was raised on 9202 with docker update for the measurement only (after two OOM kills), then put back to the template's new values through the product (sync + restart).
  • R-1063 was not pressed live: demo-hp's only user-data drive carries its apps and is not a backup target, so the agent would say yes and really retire it. Fake-agent tests over TLS instead.

The morning list (top of STATUS)

  1. Vouch golden 0.329.0 and agent 0.159.1 together (Configuration → Day-0 artifacts): golden sha256 58dc40df555676d87200e5cb24cdc65f39cbacb2417e15a1955013526de4f68c, agent sha256 de8f6ad96860ec399757a88d8bbb0191c089c9bb99db8212482c12daab41aff9 (read from the registry tonight — it matches the brief), bundle 2af3d27d686fe611f3f44e89ba3d5123a5572f51eaf87e0ac63df00a41e67cd6, Min agent 0.158.1.
  2. May I deploy hub 0.158.0 (agent-only vouch, off-site snapshot alarm, 90-day access log)? Image sha256:e420397895b5e0857605cde72e649be032be31656bf569657e9864b8552a24aa, built from felhom.eu 7d64038f.
  3. Decide D14 (the two website cards) and D15 (may the public estimator's Immich figure move up).

Each part

Part Done? What
1.1 R-981 homepage socket done The socket served nothing: homepage's rendered docker.yaml is empty, no container carries homepage.* labels, no controller code names homepage. Fix: the socket is removed (template + allow-list); the allow-list now has no exception at all. Proven on 9201 (rc.2, then 0.329.0): homepage opens (container 200, traefik 200, public 200), docker inspect shows no socket mount, 0 refusals on all three test boxes.
1.2 R-1142 own appdata folder done Root = ${HDD_PATH}/appdata/<folder>/. The boring choice: a map compiled into the controller (allowlist.json appdata_folders), not a .felhom.yml field — the catalog is the threat, so it must not name the folder. Map: paperless-ngx → paperless (the only slug ≠ folder in all 65 templates, the drill catalog and the three test boxes). Catalog gate mirrored; the lists stay byte-identical.
1.3 R-1063 drive retirement order done (unit-proven) Agent asked first; 403/409 → nothing changes, plain 409 (HU + EN). New gap R-1144.
1.4 R-1069 virtio refusal done (unit-proven) The agent's refusal → disk.err.no_durable_id (HU + EN). Agent unchanged. No page changed → no new fixture.
1.5 R-1143 :latest done Release tags only; proven live (rc.1/rc.2 pushed no :latest).
2 Controller release done 0.329.0: per-box floors → all three moved in < 8 min → box walk green (0 refusals, all containers healthy, login + files 200, homepage public 200, https-only check OK) → global floor 0.329.0 / MinAgent 0.158.1 (303 floor_set, read back from the hub DB with its WAL) → per-box floors cleared.
3 Golden done 0.329.0 baked at the floor's version, registry round trip = bake sha, teardown clean. Not vouched. Sunday runbook: no duplicate bake (wording added). No controller push after the bake.
4 Hub built, not deployed v0.158.0: agent-only vouch, R-895 alarms, R-1140 access log — all red-proofed; 05/09 sentences.
5 Agent committed, not released felhom-agent 6545ccc („Unreleased"): the orphan check runs only after a failed snapshot step. R-1141: --rotate-recovery-only in felhom.eu (publishes with the next installer tag). No password rotated.
6 Immich done Peaks below; limits raised (catalog 8cf03e0). Synthetic library removed.
7 Website not done See above (D14).

Immich peaks (9202, 4,000 synthetic photos + all jobs; 30-s samples)

Container Limit (before) Anon peak Peak / limit memory.current peak at the limit with room Typical (median anon) New limit
immich-server 2048M 1,596 MB 77 % — OOM-killed twice 2,047 2,848 828 3072M
immich-machine-learning 1536M 1,175 MB 76 % 1,498 1,159 989 2048M
immich-postgres 768M 76 MB 9 % 359 — 61 768M
immich-redis 128M 10 MB 7 % 43 — 10 128M

mem_request stays 2048M (typical total 1,888 MB): the install check judges the request, so an 8 GB box still installs Immich alone (same verdict as yesterday); only the soft overcommit warning can appear on a guest below 6 GB. Result: 4,030 photos, 75 people. Docker reported OOMKilled=false for both kills; the kernel log shows them (audits/night-2026-10-18/P6/oom-kill-dmesg.txt) — the known LXC blind spot.

Versions and commits

  • felhom-controller 0.329.0 — 2fe6341, 03ab121 (review fixes); images rc.1, rc.2, 0.329.0 (aee51f3a…).
  • app-catalog-felhom.eu — d928932 (homepage, gate), fc93846 (gate review fix), 8cf03e0 (Immich).
  • felhom.eu — 7d64038f (hub v0.158.0 code, installer flag, docs); this commit (register, STATUS, evidence, runbook).
  • felhom-agent — 6545ccc (Unreleased).
  • Hub image 0.158.0 built (not deployed). Golden 0.329.0 published (not vouched).
  • CI, matched by head_sha on the jobs endpoint: controller 03ab121 job 2044 success · catalog 8cf03e0 job 2046 success · felhom.eu 070cea5b job 2048 success · agent 65c1994 job 2047 success.
  • Fixtures: none changed — the three new texts travel in JSON answers; every page fixture byte-identical.

Red-proofs (each: the old shape put back, the test seen failing, the fix restored)

audits/night-2026-10-18/P1/: redproof-privpolicy.txt (R-1142: six binds admitted; R-981: „1 entries"), redproof-userdata-env.txt (review fixes), redproof-catalog-gate-r1142.txt (four decoys), redproof-r1063.txt, redproof-r1069.txt, redproof-r1143.txt. Hub (helper): agent-only vouch, two R-895 verdicts, the access log's secret test and its wiring. Agent (helper): the check on a successful backup („ran 2 times, want 1"), four installer mutations.

The security review of my own pushed commit

A background review flagged „authorization bypass" in privpolicy.go; its text did not reach me, so I looked myself and found two real holes, both fixed and red-proofed before any release: an app NAMED like a folder the map gives another app (paperless) got that folder; and with no drive, policyEnv took the app's own USERDATA_PATH (a deploy field a catalog push can add) as a root. If the review meant something else, it is still open.

Register — 175 before, 173 after; opened 4, closed 6

Closed: R-981, R-1142, R-1063, R-1069, R-1143, R-994. Opened: R-1144, R-1145, R-1146, R-1147. Narrowed: R-1140, R-895, R-1141. Waiting on the operator: R-1036.

Observations

  1. „Retire anyway" stops the drive's apps before the host is asked; on a refusal they stay stopped. FILED: R-1144.
  2. The installer sets the console password before it vaults it. FILED: R-1145.
  3. The installer harness is red on main and no gate runs it. FILED: R-1146.
  4. The public estimator's Immich figure is from a 200-photo test. FILED: R-1147.
  5. R-1140's „≈ 500 lines a day" was 3.7× low (≈ 1,870). NOT-A-FINDING: corrected in the row; the log is still small.
  6. The controller's test_build_push_once.sh „local build" case failed when run in place (its workspace sat inside the source). NOT-A-FINDING: fixed in the session (the test runs a copy, as the build dir does).
  7. The agent's new orphan-check shape is not proven live. NOT-A-FINDING: it is unreleased; the next release's proof shows one green backup with no check and one failed backup carrying the finding (agent CHANGELOG).
  8. The Immich teardown also deleted 20 older test photos from an earlier session on the scratch box (the device search matched them). NOT-A-FINDING: 9202 holds throwaways only; said here.
  9. The scratch Immich's admin password was replaced to run the bench; the new one was shredded. NOT-A-FINDING: a later bench resets it the same way (bcrypt into the user table — immich-admin needs a terminal).

Teardown — three layers

  • Machines: 9201 — homepage removed through the product (no container, no volume, no app.yaml); controller 0.329.0. 9202 — synthetic library and bench folder removed, Immich back on the template's limits (3072M/2048M), healthy; 9202 stays on its hand-set controller 0.327.1 (self-update off, by design). Drill VM — CT 9100 purged, token shredded, VM off, disk back to virgin. DooPlex — the 11 GB photo set deleted from the scratchpad; no Docker cleanup run.
  • Hosts: nothing changed on demo-hp, felhom-pve or Tester 1's host.
  • Hub: global floor 0.329.0 / 0.158.1; no per-box floor; no deploy; no vouch.

Instruction files and the poster

  • runbooks/nightly-rotation.md (a runbook, not an instruction file): the Sunday golden step does not bake a duplicate of an unvouched golden at the floor's version.
  • Instruction files: none edited.
  • System poster: none of the three — no fact in felhom-system-poster.facts.md changed tonight (the hub's 90-day access log becomes one when hub 0.158.0 is deployed; that session updates the facts file).
  • unproven.py --summary: not walked 35 of 55 — unchanged.

Architecture documents read: 02 (the catalog allow-list), 03 (the R-1012 check), 05 (Day-0 manifest, R-120), 07 (the off-site tier), 09 §3 decisions 162 and 250.