Read-only measurement on demo-hp, demo-felhom and Tester 1, two readings. Audit folder documentation/audits/space-list-2026-11-10/ (README, TABLE, MATRIX, evidence, tools). 360 answers: A 294, D 0, K 19, N 47. Register 225 -> 230: R-1368 (host places that only grow: kernels, agent binaries), R-1369 (no written way out of a full disk), R-1370 (no space budget for the host disk), R-1371 (no space alarm; host-disk mail goes to the household), R-1372 (small guest places). A dated line on nine existing rows. Section headings set to the real counts; R-1268 and R-1312 moved to their category's section. STATUS: new Updated paragraph, decisions D67-D70. N100 diary line. Two stale facts corrected in 05-hub-architecture.md and 11-os-updates.md. Nothing was changed on any machine. Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
The Space list — 2026-11-10 (pilot of the gap review)
What this is. Every place that holds data on a Felhom box, with eight questions for each place. Measured
read-only on demo-hp, demo-felhom (N100) and Tester 1. Nothing was changed on any machine.
Facts: TABLE.md · marks: MATRIX.md · proof: evidence/ · scripts: tools/.
Baselines: felhom.eu a6705ceb04 · controller da060d7e7f · agent 41d5eb8051 · catalog f4c5c797d9.
Wrong claims in the prompt (named first). The prompt calls alpine:3.20, curl and hello-world the
product's helper images. The product's helper is alpine with no tag. alpine:3.20 and curl came from test
tools. hello-world comes from the template bake. One closed register row is wrong too: R-521 says the guest
root disk has a fill watcher. It has none since the two guest volumes became one.
1. Five lines
- Places: 45 customer places (18 host, 17 guest, 10 data drive) · 7 test-only · 6 off-box, from documents only.
- No document named: 8 of the 45 (H4, H5, H12, H15, G9, G11, G15, G17). 16 more are named only in passing.
- Marks, 360 answers: A 294 · D 0 · K 19 · N 47. The 47 N fall into 5 classes = 5 new rows.
- Worst N: nobody removes old kernels (H9). A ring-0 box holds 8 kernels = 8.15 GB and gets about one more each week. The update wrapper refuses to remove a package, and nothing else does.
- You decide: four things — section 5.
2. The N answers, by class (one register row each; members in MATRIX.md)
- C1 = R-1368 (10) — host places that only grow: old kernels, the agent's self-update folder (49 binaries, 740 MB), template archives, old agent binary copies, logs of failed backup runs.
- C2 = R-1369 (24) — a disk is full and there is no written way out. One missing runbook, seen from 21 places.
- C3 = R-1370 (1) — the host system disk has no space budget.
- C4 = R-1371 (7) — no space alarm: the guest root disk has no watcher (three places lie on it); four small places. The row also holds two findings that are not N: the host-disk critical mail goes to the household, and one filesystem is watched twice.
- C5 = R-1372 (5) — small guest places nobody described: helper images, off-site restore copies with no age limit (1.65 GB of two removed apps on the N100), log caps never re-checked, guest log in memory.
3. Things that can grow without a limit, by ruling (not defects)
Undo images: a count per app, no byte limit (decision 53). Kept data of a removed app (decision 40). The household's
own files stay on every removal (decision 67). A set-aside unit, until the household removes it (decision 256). A
restore's undo copy under a support hold (decision 281). The second-drive copy of a removed app (D25). The backup
unit of a removed app that the household kept (07-backup-architecture.md:380). Off-box, from documents only: one image per
version for ever in the registry, pruned by hand (decision 62).
4. The four questions
- Helper images. The product pulls
alpinewith no tag at seven call sites (first:felhom-controller/controller/internal/backup/backup.go:988). The bake pullshello-world(felhom-agent/configs/build-golden.sh:232). None of the three image rules (decisions 53, 56, 285) can delete a helper image (internal/stacks/image_retention.go:244-250). They are small: 8.4 MB, 4 months old on two boxes. - Container log caps. 10 MB × 3 files on all three boxes, for every container; no container escapes the cap. Worst case: demo-hp 780 MB · N100 270 MB · Tester 1 300 MB. The cap is written once at the bake; nothing checks it later.
- The host's own disk. Yes, the hub warns: host system disk at 90 % and 95 %; thin pool at 85 % and 90 % (the worse of data and metadata). The mail goes to you; the critical one also to the household. Nothing refuses or repairs. No architecture document names the host-disk alarm.
- Kernels and boot. Nothing removes old kernels.
/bootis a folder on the system disk: 0.82 / 0.94 / 0.33 GB. The EFI partition is 1 % full on each box. Kernels: 8 / 8 / 3 = 8.15 / 8.15 / 3.05 GB.
System logs, two readings (07:39 and 09:45 UTC, 2 h 6 min apart; evidence/reading-1/, evidence/reading-2/).
demo-hp host journal: 873.7 → 898.9 MB = about 12 MB an hour. The journal grows in 8 MiB steps, so two hours give
only a rough rate (190–380 MB a day). It was 98 % of all growth on that system disk in the window (+25.7 MB).
The N100 and Tester 1 sit at the journal's cap (4.30 GB each); their oldest entries did not move. Lines in the last
hour: 13,521 / 13,506 / 11,177, and sudo wrote 95 / 94 / 93 % of them. Guest system log: 8 MB in memory, both
readings. Container logs on demo-hp: +5.7 MB (2.7 MB an hour, under the 780 MB worst case). Unnamed volumes: no
change in two hours; they come at night (about 29 a day on demo-hp).
5. Decisions for you (also in STATUS.md)
| # | Question | Options and cost | CC's pick | If you do nothing |
|---|---|---|---|---|
| D67 | May the nightly update remove old kernels by itself? | Yes: keep the running one, the one before and the newest; it widens rule R4; one agent release. No: you remove them by hand on each box. | Yes, ring 0 first | a ring-0 box gains about 1 GB a week |
| D68 | Who gets the "host disk critical" mail? | Operator only: one hub change. Keep: the household gets a mail it cannot act on. | Operator only | the household is mailed about a disk only you can free |
| D69 | Write a short "disk space" section in an architecture document? | Yes: about one session; each place gets a limit and a cleaner. No: this table stays the only reference and goes stale. | Yes | 8 places stay unnamed; new ones appear unseen |
| D70 | Run the next list ("Endings") with this method? | Yes, with the three changes below. No: stop the gap review here. | Yes | the plan stops after its pilot |
6. The method's verdict
It found real things nobody had asked about. New: kernels and agent binaries are never removed; the guest root
disk has no watcher and a closed row said it had; a household gets the host-disk mail; off-site restore copies have
no age limit. Four of the five new rows hold a measured fact, not an opinion.
Cost: about 1.5 hours of work and a 1.5-hour wait for the second reading (the plan said 4–5 hours). Noise: the 47 N answers hold about 14 separate gaps. Question 7 gave the same
answer 21 times: that is one gap, not 21. Some places are too small to matter (temp folders, the EFI partition).
No D answer was found among 45 places: what was decided here was also built.
Change three things for "Endings": (1) ask question 7 once for each disk, not for each place; (2) drop
places under 100 MB that cannot grow, in the first hour; (3) a helper's answer does not go into the table until
the main session has read its pointer — the off-box places here are still candidates for that reason.
Weak spot: the off-box places (ep0, DooPlex, the off-site box, the hub database) were not measured, and the
Prometheus rules in homelab-manifests were not read. That is the second pass.
For a later session (small fixes, not done today — no code change in this run)
Keep only the newest N files in the agent's self-update folder · watch the guest root disk and de-duplicate
fill-watch targets by filesystem (cmd/controller/main.go:2694-2707 has a stale comment) · pin the alpine
helper · a stale document, on a protected machine's procedure, so not edited here: runbooks/ep0-datastore-copy.md:85
says "NOT INSTALLED" (closed row R-901: installed 2026-10-09).