122 lines
10 KiB
Markdown
122 lines
10 KiB
Markdown
# REPORT — night 2026-10-17 → 18 (unattended)
|
||
|
||
## How I read the brief (no question could be asked)
|
||
|
||
- **Part 7 (website cards) — NOT done, on purpose.** Both apps are closed to new installs (wger `lifecycle: hidden`,
|
||
plant-it `lifecycle: abandoned`), so the install wizard refuses them and a card would offer an app a visitor cannot get.
|
||
R-1036 is now a two-option question for the operator (STATUS D14). Recommendation not followed → this line says why.
|
||
- **„Measure on 9201"** = demo-hp's guest 9201 (Tier 0); homepage was not installed anywhere, so it was installed there
|
||
as a throwaway, measured, and removed after.
|
||
- **The hub image build** is allowed by the brief (Part 4.4) although `unprompted-work.md` forbids a hub build in an
|
||
unattended session without a task file: the brief is that task file. No deploy.
|
||
- **The Immich limit was raised on 9202 with `docker update`** for the measurement only (after two OOM kills), then put
|
||
back to the template's new values through the product (sync + restart).
|
||
- **R-1063 was not pressed live**: demo-hp's only user-data drive carries its apps and is not a backup target, so the
|
||
agent would say yes and really retire it. Fake-agent tests over TLS instead.
|
||
|
||
## The morning list (top of STATUS)
|
||
|
||
1. **Vouch golden 0.329.0 and agent 0.159.1 together** (Configuration → Day-0 artifacts): golden sha256
|
||
`58dc40df555676d87200e5cb24cdc65f39cbacb2417e15a1955013526de4f68c`, agent sha256
|
||
`de8f6ad96860ec399757a88d8bbb0191c089c9bb99db8212482c12daab41aff9` (read from the registry tonight — it matches the
|
||
brief), bundle `2af3d27d686fe611f3f44e89ba3d5123a5572f51eaf87e0ac63df00a41e67cd6`, Min agent **0.158.1**.
|
||
2. **May I deploy hub 0.158.0** (agent-only vouch, off-site snapshot alarm, 90-day access log)? Image
|
||
`sha256:e420397895b5e0857605cde72e649be032be31656bf569657e9864b8552a24aa`, built from felhom.eu `7d64038f`.
|
||
3. Decide D14 (the two website cards) and D15 (may the public estimator's Immich figure move up).
|
||
|
||
## Each part
|
||
|
||
| Part | Done? | What |
|
||
|---|---|---|
|
||
| 1.1 R-981 homepage socket | **done** | The socket served nothing: homepage's rendered `docker.yaml` is empty, no container carries `homepage.*` labels, no controller code names homepage. **Fix: the socket is removed** (template + allow-list); the allow-list now has **no exception at all**. Proven on 9201 (rc.2, then 0.329.0): homepage opens (container 200, traefik 200, public 200), `docker inspect` shows no socket mount, 0 refusals on all three test boxes. |
|
||
| 1.2 R-1142 own appdata folder | **done** | Root = `${HDD_PATH}/appdata/<folder>/`. **The boring choice: a map compiled into the controller** (`allowlist.json` `appdata_folders`), not a `.felhom.yml` field — the catalog is the threat, so it must not name the folder. Map: **`paperless-ngx → paperless`** (the only slug ≠ folder in all 65 templates, the drill catalog and the three test boxes). Catalog gate mirrored; the lists stay byte-identical. |
|
||
| 1.3 R-1063 drive retirement order | **done** (unit-proven) | Agent asked first; 403/409 → nothing changes, plain 409 (HU + EN). New gap R-1144. |
|
||
| 1.4 R-1069 virtio refusal | **done** (unit-proven) | The agent's refusal → `disk.err.no_durable_id` (HU + EN). Agent unchanged. No page changed → no new fixture. |
|
||
| 1.5 R-1143 `:latest` | **done** | Release tags only; proven live (rc.1/rc.2 pushed no `:latest`). |
|
||
| 2 Controller release | **done** | 0.329.0: per-box floors → all three moved in < 8 min → box walk green (0 refusals, all containers healthy, login + files 200, homepage public 200, https-only check OK) → **global floor 0.329.0 / MinAgent 0.158.1** (303 `floor_set`, read back from the hub DB with its WAL) → per-box floors cleared. |
|
||
| 3 Golden | **done** | 0.329.0 baked at the floor's version, registry round trip = bake sha, teardown clean. **Not vouched.** Sunday runbook: no duplicate bake (wording added). No controller push after the bake. |
|
||
| 4 Hub | **built, not deployed** | v0.158.0: agent-only vouch, R-895 alarms, R-1140 access log — all red-proofed; `05`/`09` sentences. |
|
||
| 5 Agent | **committed, not released** | felhom-agent `6545ccc` („Unreleased"): the orphan check runs only after a failed snapshot step. R-1141: `--rotate-recovery-only` in felhom.eu (publishes with the next installer tag). No password rotated. |
|
||
| 6 Immich | **done** | Peaks below; limits raised (catalog `8cf03e0`). Synthetic library removed. |
|
||
| 7 Website | **not done** | See above (D14). |
|
||
|
||
## Immich peaks (9202, 4,000 synthetic photos + all jobs; 30-s samples)
|
||
|
||
| Container | Limit (before) | Anon peak | Peak / limit | memory.current peak at the limit | with room | Typical (median anon) | New limit |
|
||
|---|---|---|---|---|---|---|---|
|
||
| immich-server | 2048M | 1,596 MB | **77 %** — OOM-killed twice | 2,047 | 2,848 | 828 | **3072M** |
|
||
| immich-machine-learning | 1536M | 1,175 MB | **76 %** | 1,498 | 1,159 | 989 | **2048M** |
|
||
| immich-postgres | 768M | 76 MB | 9 % | 359 | — | 61 | 768M |
|
||
| immich-redis | 128M | 10 MB | 7 % | 43 | — | 10 | 128M |
|
||
|
||
`mem_request` stays 2048M (typical total 1,888 MB): the install check judges the request, so an 8 GB box still installs
|
||
Immich alone (same verdict as yesterday); only the soft overcommit warning can appear on a guest below 6 GB. Result: 4,030
|
||
photos, 75 people. Docker reported `OOMKilled=false` for both kills; the kernel log shows them
|
||
(`audits/night-2026-10-18/P6/oom-kill-dmesg.txt`) — the known LXC blind spot.
|
||
|
||
## Versions and commits
|
||
|
||
- felhom-controller **0.329.0** — `2fe6341`, `03ab121` (review fixes); images rc.1, rc.2, 0.329.0 (`aee51f3a…`).
|
||
- app-catalog-felhom.eu — `d928932` (homepage, gate), `fc93846` (gate review fix), `8cf03e0` (Immich).
|
||
- felhom.eu — `7d64038f` (hub v0.158.0 code, installer flag, docs); this commit (register, STATUS, evidence, runbook).
|
||
- felhom-agent — `6545ccc` (Unreleased).
|
||
- Hub image 0.158.0 built (not deployed). Golden 0.329.0 published (not vouched).
|
||
- **CI, matched by `head_sha` on the jobs endpoint:** controller `03ab121` job 2044 success · catalog `8cf03e0` job 2046 success · felhom.eu `070cea5b` job 2048 success · agent `65c1994` job 2047 success.
|
||
- Fixtures: none changed — the three new texts travel in JSON answers; every page fixture byte-identical.
|
||
|
||
## Red-proofs (each: the old shape put back, the test seen failing, the fix restored)
|
||
|
||
`audits/night-2026-10-18/P1/`: `redproof-privpolicy.txt` (R-1142: six binds admitted; R-981: „1 entries"),
|
||
`redproof-userdata-env.txt` (review fixes), `redproof-catalog-gate-r1142.txt` (four decoys), `redproof-r1063.txt`,
|
||
`redproof-r1069.txt`, `redproof-r1143.txt`. Hub (helper): agent-only vouch, two R-895 verdicts, the access log's secret
|
||
test and its wiring. Agent (helper): the check on a successful backup („ran 2 times, want 1"), four installer mutations.
|
||
|
||
## The security review of my own pushed commit
|
||
|
||
A background review flagged „authorization bypass" in `privpolicy.go`; its text did not reach me, so I looked myself and
|
||
found two real holes, both fixed and red-proofed before any release: an app NAMED like a folder the map gives another
|
||
app (`paperless`) got that folder; and with no drive, `policyEnv` took the app's own `USERDATA_PATH` (a deploy field a
|
||
catalog push can add) as a root. If the review meant something else, it is still open.
|
||
|
||
## Register — 175 before, 173 after; opened 4, closed 6
|
||
|
||
Closed: R-981, R-1142, R-1063, R-1069, R-1143, R-994. Opened: R-1144, R-1145, R-1146, R-1147. Narrowed: R-1140, R-895,
|
||
R-1141. Waiting on the operator: R-1036.
|
||
|
||
## Observations
|
||
|
||
1. „Retire anyway" stops the drive's apps before the host is asked; on a refusal they stay stopped. FILED: R-1144.
|
||
2. The installer sets the console password before it vaults it. FILED: R-1145.
|
||
3. The installer harness is red on main and no gate runs it. FILED: R-1146.
|
||
4. The public estimator's Immich figure is from a 200-photo test. FILED: R-1147.
|
||
5. R-1140's „≈ 500 lines a day" was 3.7× low (≈ 1,870). NOT-A-FINDING: corrected in the row; the log is still small.
|
||
6. The controller's `test_build_push_once.sh` „local build" case failed when run in place (its workspace sat inside the
|
||
source). NOT-A-FINDING: fixed in the session (the test runs a copy, as the build dir does).
|
||
7. The agent's new orphan-check shape is not proven live. NOT-A-FINDING: it is unreleased; the next release's proof
|
||
shows one green backup with no check and one failed backup carrying the finding (agent CHANGELOG).
|
||
8. The Immich teardown also deleted 20 older test photos from an earlier session on the scratch box (the device search
|
||
matched them). NOT-A-FINDING: 9202 holds throwaways only; said here.
|
||
9. The scratch Immich's admin password was replaced to run the bench; the new one was shredded. NOT-A-FINDING: a later
|
||
bench resets it the same way (bcrypt into the `user` table — `immich-admin` needs a terminal).
|
||
|
||
## Teardown — three layers
|
||
|
||
- **Machines:** 9201 — homepage removed through the product (no container, no volume, no `app.yaml`); controller
|
||
0.329.0. 9202 — synthetic library and bench folder removed, Immich back on the template's limits (3072M/2048M), healthy;
|
||
9202 stays on its hand-set controller 0.327.1 (self-update off, by design). Drill VM — CT 9100 purged, token shredded,
|
||
VM off, disk back to `virgin`. DooPlex — the 11 GB photo set deleted from the scratchpad; no Docker cleanup run.
|
||
- **Hosts:** nothing changed on demo-hp, felhom-pve or Tester 1's host.
|
||
- **Hub:** global floor 0.329.0 / 0.158.1; no per-box floor; no deploy; no vouch.
|
||
|
||
## Instruction files and the poster
|
||
|
||
- `runbooks/nightly-rotation.md` (a runbook, not an instruction file): the Sunday golden step does not bake a duplicate
|
||
of an unvouched golden at the floor's version.
|
||
- Instruction files: none edited.
|
||
- System poster: **none of the three** — no fact in `felhom-system-poster.facts.md` changed tonight (the hub's 90-day
|
||
access log becomes one when hub 0.158.0 is deployed; that session updates the facts file).
|
||
- `unproven.py --summary`: not walked 35 of 55 — unchanged.
|
||
|
||
Architecture documents read: `02` (the catalog allow-list), `03` (the R-1012 check), `05` (Day-0 manifest, R-120),
|
||
`07` (the off-site tier), `09` §3 decisions 162 and 250.
|