Files
felhom.eu/REPORT-night-2026-10-18.md
T
2026-10-18 00:06:11 +02:00

122 lines
10 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# REPORT — night 2026-10-17 → 18 (unattended)
## How I read the brief (no question could be asked)
- **Part 7 (website cards) — NOT done, on purpose.** Both apps are closed to new installs (wger `lifecycle: hidden`,
plant-it `lifecycle: abandoned`), so the install wizard refuses them and a card would offer an app a visitor cannot get.
R-1036 is now a two-option question for the operator (STATUS D14). Recommendation not followed → this line says why.
- **„Measure on 9201"** = demo-hp's guest 9201 (Tier 0); homepage was not installed anywhere, so it was installed there
as a throwaway, measured, and removed after.
- **The hub image build** is allowed by the brief (Part 4.4) although `unprompted-work.md` forbids a hub build in an
unattended session without a task file: the brief is that task file. No deploy.
- **The Immich limit was raised on 9202 with `docker update`** for the measurement only (after two OOM kills), then put
back to the template's new values through the product (sync + restart).
- **R-1063 was not pressed live**: demo-hp's only user-data drive carries its apps and is not a backup target, so the
agent would say yes and really retire it. Fake-agent tests over TLS instead.
## The morning list (top of STATUS)
1. **Vouch golden 0.329.0 and agent 0.159.1 together** (Configuration → Day-0 artifacts): golden sha256
`58dc40df555676d87200e5cb24cdc65f39cbacb2417e15a1955013526de4f68c`, agent sha256
`de8f6ad96860ec399757a88d8bbb0191c089c9bb99db8212482c12daab41aff9` (read from the registry tonight — it matches the
brief), bundle `2af3d27d686fe611f3f44e89ba3d5123a5572f51eaf87e0ac63df00a41e67cd6`, Min agent **0.158.1**.
2. **May I deploy hub 0.158.0** (agent-only vouch, off-site snapshot alarm, 90-day access log)? Image
`sha256:e420397895b5e0857605cde72e649be032be31656bf569657e9864b8552a24aa`, built from felhom.eu `7d64038f`.
3. Decide D14 (the two website cards) and D15 (may the public estimator's Immich figure move up).
## Each part
| Part | Done? | What |
|---|---|---|
| 1.1 R-981 homepage socket | **done** | The socket served nothing: homepage's rendered `docker.yaml` is empty, no container carries `homepage.*` labels, no controller code names homepage. **Fix: the socket is removed** (template + allow-list); the allow-list now has **no exception at all**. Proven on 9201 (rc.2, then 0.329.0): homepage opens (container 200, traefik 200, public 200), `docker inspect` shows no socket mount, 0 refusals on all three test boxes. |
| 1.2 R-1142 own appdata folder | **done** | Root = `${HDD_PATH}/appdata/<folder>/`. **The boring choice: a map compiled into the controller** (`allowlist.json` `appdata_folders`), not a `.felhom.yml` field — the catalog is the threat, so it must not name the folder. Map: **`paperless-ngx → paperless`** (the only slug ≠ folder in all 65 templates, the drill catalog and the three test boxes). Catalog gate mirrored; the lists stay byte-identical. |
| 1.3 R-1063 drive retirement order | **done** (unit-proven) | Agent asked first; 403/409 → nothing changes, plain 409 (HU + EN). New gap R-1144. |
| 1.4 R-1069 virtio refusal | **done** (unit-proven) | The agent's refusal → `disk.err.no_durable_id` (HU + EN). Agent unchanged. No page changed → no new fixture. |
| 1.5 R-1143 `:latest` | **done** | Release tags only; proven live (rc.1/rc.2 pushed no `:latest`). |
| 2 Controller release | **done** | 0.329.0: per-box floors → all three moved in < 8 min → box walk green (0 refusals, all containers healthy, login + files 200, homepage public 200, https-only check OK) → **global floor 0.329.0 / MinAgent 0.158.1** (303 `floor_set`, read back from the hub DB with its WAL) → per-box floors cleared. |
| 3 Golden | **done** | 0.329.0 baked at the floor's version, registry round trip = bake sha, teardown clean. **Not vouched.** Sunday runbook: no duplicate bake (wording added). No controller push after the bake. |
| 4 Hub | **built, not deployed** | v0.158.0: agent-only vouch, R-895 alarms, R-1140 access log — all red-proofed; `05`/`09` sentences. |
| 5 Agent | **committed, not released** | felhom-agent `6545ccc` („Unreleased"): the orphan check runs only after a failed snapshot step. R-1141: `--rotate-recovery-only` in felhom.eu (publishes with the next installer tag). No password rotated. |
| 6 Immich | **done** | Peaks below; limits raised (catalog `8cf03e0`). Synthetic library removed. |
| 7 Website | **not done** | See above (D14). |
## Immich peaks (9202, 4,000 synthetic photos + all jobs; 30-s samples)
| Container | Limit (before) | Anon peak | Peak / limit | memory.current peak at the limit | with room | Typical (median anon) | New limit |
|---|---|---|---|---|---|---|---|
| immich-server | 2048M | 1,596 MB | **77 %** — OOM-killed twice | 2,047 | 2,848 | 828 | **3072M** |
| immich-machine-learning | 1536M | 1,175 MB | **76 %** | 1,498 | 1,159 | 989 | **2048M** |
| immich-postgres | 768M | 76 MB | 9 % | 359 | — | 61 | 768M |
| immich-redis | 128M | 10 MB | 7 % | 43 | — | 10 | 128M |
`mem_request` stays 2048M (typical total 1,888 MB): the install check judges the request, so an 8 GB box still installs
Immich alone (same verdict as yesterday); only the soft overcommit warning can appear on a guest below 6 GB. Result: 4,030
photos, 75 people. Docker reported `OOMKilled=false` for both kills; the kernel log shows them
(`audits/night-2026-10-18/P6/oom-kill-dmesg.txt`) — the known LXC blind spot.
## Versions and commits
- felhom-controller **0.329.0** — `2fe6341`, `03ab121` (review fixes); images rc.1, rc.2, 0.329.0 (`aee51f3a…`).
- app-catalog-felhom.eu — `d928932` (homepage, gate), `fc93846` (gate review fix), `8cf03e0` (Immich).
- felhom.eu — `7d64038f` (hub v0.158.0 code, installer flag, docs); this commit (register, STATUS, evidence, runbook).
- felhom-agent — `6545ccc` (Unreleased).
- Hub image 0.158.0 built (not deployed). Golden 0.329.0 published (not vouched).
- **CI, matched by `head_sha` on the jobs endpoint:** controller `03ab121` job 2044 success · catalog `8cf03e0` job 2046 success · felhom.eu `070cea5b` job 2048 success · agent `65c1994` job 2047 success.
- Fixtures: none changed — the three new texts travel in JSON answers; every page fixture byte-identical.
## Red-proofs (each: the old shape put back, the test seen failing, the fix restored)
`audits/night-2026-10-18/P1/`: `redproof-privpolicy.txt` (R-1142: six binds admitted; R-981: „1 entries"),
`redproof-userdata-env.txt` (review fixes), `redproof-catalog-gate-r1142.txt` (four decoys), `redproof-r1063.txt`,
`redproof-r1069.txt`, `redproof-r1143.txt`. Hub (helper): agent-only vouch, two R-895 verdicts, the access log's secret
test and its wiring. Agent (helper): the check on a successful backup („ran 2 times, want 1"), four installer mutations.
## The security review of my own pushed commit
A background review flagged „authorization bypass" in `privpolicy.go`; its text did not reach me, so I looked myself and
found two real holes, both fixed and red-proofed before any release: an app NAMED like a folder the map gives another
app (`paperless`) got that folder; and with no drive, `policyEnv` took the app's own `USERDATA_PATH` (a deploy field a
catalog push can add) as a root. If the review meant something else, it is still open.
## Register — 175 before, 173 after; opened 4, closed 6
Closed: R-981, R-1142, R-1063, R-1069, R-1143, R-994. Opened: R-1144, R-1145, R-1146, R-1147. Narrowed: R-1140, R-895,
R-1141. Waiting on the operator: R-1036.
## Observations
1. „Retire anyway" stops the drive's apps before the host is asked; on a refusal they stay stopped. FILED: R-1144.
2. The installer sets the console password before it vaults it. FILED: R-1145.
3. The installer harness is red on main and no gate runs it. FILED: R-1146.
4. The public estimator's Immich figure is from a 200-photo test. FILED: R-1147.
5. R-1140's „≈ 500 lines a day" was 3.7× low (≈ 1,870). NOT-A-FINDING: corrected in the row; the log is still small.
6. The controller's `test_build_push_once.sh` „local build" case failed when run in place (its workspace sat inside the
source). NOT-A-FINDING: fixed in the session (the test runs a copy, as the build dir does).
7. The agent's new orphan-check shape is not proven live. NOT-A-FINDING: it is unreleased; the next release's proof
shows one green backup with no check and one failed backup carrying the finding (agent CHANGELOG).
8. The Immich teardown also deleted 20 older test photos from an earlier session on the scratch box (the device search
matched them). NOT-A-FINDING: 9202 holds throwaways only; said here.
9. The scratch Immich's admin password was replaced to run the bench; the new one was shredded. NOT-A-FINDING: a later
bench resets it the same way (bcrypt into the `user` table — `immich-admin` needs a terminal).
## Teardown — three layers
- **Machines:** 9201 — homepage removed through the product (no container, no volume, no `app.yaml`); controller
0.329.0. 9202 — synthetic library and bench folder removed, Immich back on the template's limits (3072M/2048M), healthy;
9202 stays on its hand-set controller 0.327.1 (self-update off, by design). Drill VM — CT 9100 purged, token shredded,
VM off, disk back to `virgin`. DooPlex — the 11 GB photo set deleted from the scratchpad; no Docker cleanup run.
- **Hosts:** nothing changed on demo-hp, felhom-pve or Tester 1's host.
- **Hub:** global floor 0.329.0 / 0.158.1; no per-box floor; no deploy; no vouch.
## Instruction files and the poster
- `runbooks/nightly-rotation.md` (a runbook, not an instruction file): the Sunday golden step does not bake a duplicate
of an unvouched golden at the floor's version.
- Instruction files: none edited.
- System poster: **none of the three** — no fact in `felhom-system-poster.facts.md` changed tonight (the hub's 90-day
access log becomes one when hub 0.158.0 is deployed; that session updates the facts file).
- `unproven.py --summary`: not walked 35 of 55 — unchanged.
Architecture documents read: `02` (the catalog allow-list), `03` (the R-1012 check), `05` (Day-0 manifest, R-120),
`07` (the off-site tier), `09` §3 decisions 162 and 250.