docs: record the rulings of 2026-11-10 15:19 (D71-D75, decisions 291-296); nothing built
gates / gates (push) Successful in 1m59s
gates / gates (push) Successful in 1m59s
D71: the hub keeps the record of an off-site account after the tier is unticked (R-1375). D72: off-site copies of a removed app end 30 days after 'delete backup data' (R-1374). D73: a household or box delete removes its rows; the delete journal stays (R-1376). D74: the runbook 'a household or a tester leaves' is a later documents-only session. D75: the smallest system disk is 200 GB; the installer's act below it is open (R-1370). Accepted: the heading correction of runbooks/ep0-datastore-copy.md (closed R-901). Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
This commit is contained in:
1 parent
025a1bcd40
commit
e83a78ce61
6 files changed
+45
-16
No files matched your search
@@ -2,7 +2,7 @@
|
||||
|
||||
**Ready for Tester 2? Yes — tell the friend go (`documentation/pilot/TESTER2-handover.md`).**
|
||||
|
||||
**Updated 2026-11-10 (afternoon).** CC listed every way a thing ends on a box or in the hub (74 endings, eight questions each): 22 answers were „nobody asked”, now in six new rows (R-1373…R-1378) and four questions for you (D71–D74). CC removed six old kernels from the HP box by hand with the keep rule you ruled (8 → 2, 6.4 GB back, one restart, all apps healthy after 5 minutes); nothing is built for it yet. Three first space fixes wait in source for a release, the disk-space design (`documentation/architecture/16-storage-physical-identity.md` §14) and the runbook `documentation/runbooks/disk-full.md` (R-1369) are written, and D75 is one more question for you; the report is `REPORT-endings-kernels-fixes-2026-11-10.md`.
|
||||
**Updated 2026-11-10 (late afternoon).** You ruled D71–D75 at 15:19 (`documentation/architecture/09-update-architecture.md` §3, decisions 291–296); none is built. The smallest system disk is 200 GB, and what the installer does below 200 GB (refuse, or warn) is open (R-1370). The report of this run is `REPORT-release-space-fixes-2026-11-10.md`.
|
||||
|
||||
## What works
|
||||
- **The Plex attack did not reach Felhom's secrets** (R-1241).
|
||||
@@ -32,11 +32,6 @@ Newest first. CC's pick is in the „Pick" column. Each row also has its full te
|
||||
|
||||
| # | Row | Question | Pick (why) | The other option costs | If you do nothing | Blocked |
|
||||
|---|---|---|---|---|---|---|
|
||||
| D75 | R-1370 | What is the smallest system disk Felhom accepts? Measured: the host's own files need about 14 GB when the whole-box copies go to a backup drive, and 28–54 GB when they stay on the system disk. | **128 GB only with a backup drive; 200 GB without one** (a 128 GB disk gives the host 41 GB: enough for 14, not for three local copies of a box with many apps) | 128 GB always, as today: a box with many apps and no backup drive stops its local whole-box copies | the installer accepts any disk; a small box can lose its local copies | R-1370 |
|
||||
| D74 | R-1375 | Shall CC write the runbook „a household or a tester leaves” now? | **Yes** (five code doors exist; documents only, one evening) | wait for the first leaver: the first run is by memory | no runbook; a leaver is handled by hand | — |
|
||||
| D73 | R-1376 | When you delete a household or a box in the hub, shall the hub delete every record of it? Today 13 tables keep rows. | **Yes** (add the tables to the purge; one hub change) | keep them as history: say so in the privacy text | rows of deleted households and boxes stay in the hub for ever | R-1376 |
|
||||
| D72 | R-1374 | Shall the off-site snapshots of a removed app end after a fixed time, with a notice first? Today they stay for ever. | **Yes — give the number of days** (paid space stops growing) | keep for ever: the household can restore a removed app at any time; paid space only grows | they stay for ever | R-1374 |
|
||||
| D71 | R-1375 | Shall the hub keep the record of an off-site account after you untick the tier, and take the account down on RESET and delete? | **Yes** (an account can not be forgotten; one hub change) | a hand step in a runbook: it depends on memory | an unticked household that is reset or deleted leaves a paid account with no record | R-1375 |
|
||||
| D66 | R-1365 | Should a security-watch source that stays down mail again every 7 days, as an open notice does? Today it mails once, then only the Monday summary lists it. | **Yes** (a real outage then cannot be forgotten; the seven pins that can never be read no longer count, so it stays quiet when all is well) | no: one line in a long Monday mail stays the only reminder | a source that stays down mails once; the Monday list still shows it | — |
|
||||
| D38 | R-1235 | Put the newest golden (637 MB) into the nightly release copy on ep0 too? | **Yes** (a new install after a DooPlex loss needs it; it fits under the 3 GB cap) | leave it out: after a loss, no new install until a golden is rebuilt and vouched | the same | — |
|
||||
| D37 | R-1236 | Raise Gitea's memory limit from 1 GiB to 2 GiB? It was killed for memory 4 times in 17 days, once during today's release. | **Yes** (DooPlex has the RAM; a DooPlex change, so yours) | keep 1 GiB: a restart in release hours fails box updates and CI for a minute | the same | — |
|
||||
|
||||
@@ -1265,6 +1265,34 @@ Recorded before the work (`audits/space-list-2026-11-10/README.md` §5 held the
|
||||
Hetzner and the mail service are read from source and documents only: no hub database read, no account call.
|
||||
*Operator, 2026-11-10 12:22.*
|
||||
|
||||
### 2026-11-10 (15:19, operator, planning chat) — five rulings on the Endings list and the system disk
|
||||
|
||||
Recorded before any build. **None of these is built.** The questions were D71–D75 in `STATUS.md`
|
||||
(`audits/endings-list-2026-11-10/README.md`; `16-storage-physical-identity.md` §14.3).
|
||||
|
||||
291. **D71 = yes: the hub keeps the record of an off-site account after the tier is unticked.** RESET and delete take
|
||||
the account down, ticked or not. **Cost of the other option:** a hand step in a runbook; a forgotten one leaves a
|
||||
paid account with no record. R-1375. *Operator, 2026-11-10 15:19.*
|
||||
292. **D72 = yes, and it follows the household's own choice.** When the household pressed „delete backup data” for an
|
||||
app, that app's off-site copies end 30 days later, at the next clean-up after that day — the same number as a
|
||||
restore copy (decision 281, D62). When the household kept the app's backups, the off-site copies stay. **This
|
||||
replaces the status row's „a fixed time for every removed app”.** The page must show the end date before anything
|
||||
is deleted; that is written into the design (`07-backup-architecture.md`) before a build. R-1374.
|
||||
*Operator, 2026-11-10 15:19.*
|
||||
293. **D73 = yes, with one exception: deleting a household or a box deletes its rows in the tables that have no delete
|
||||
today.** The small delete journal that the design keeps on purpose (`host_deletions`, `customer_resets`) stays,
|
||||
and the privacy text must say so. R-1376. *Operator, 2026-11-10 15:19.*
|
||||
294. **D74 = yes: the runbook „a household or a tester leaves” is written**, in a later documents-only session.
|
||||
R-1375, and R-810 in `ROADMAP.md`. *Operator, 2026-11-10 15:19.*
|
||||
295. **D75 = one number: the smallest system disk is 200 GB, with or without a backup drive.** This replaces the status
|
||||
row's two-step pick (128 GB with a backup drive, 200 GB without). **Why:** one rule is simpler to promise, and a
|
||||
128 GB disk's pool (54 GiB) is smaller than the guest's promised space (32 + 50 GiB) even with a backup drive
|
||||
(`16` §14.3). **Open, not ruled:** what the installer does below 200 GB — refuse, or warn. That is a design point
|
||||
for a later session (R-1370). demo-hp (128 GB) stays as a test box. *Operator, 2026-11-10 15:19.*
|
||||
296. **Accepted: the heading correction in `runbooks/ep0-datastore-copy.md`** („written 2026-10-08, installed
|
||||
2026-10-09”), made on 2026-11-10 from the closed row R-901. It was not re-checked on DooPlex, and it stays so.
|
||||
*Operator, 2026-11-10 15:19.*
|
||||
|
||||
### 2026-10-28 (day, attended) — RomM kept; the runtime cache fills itself in CI
|
||||
|
||||
277. **D61 = keep RomM offered** (R-1330) while no RomM release carries Node ≥ 24.22.1; the first one that does is taken
|
||||
|
||||
@@ -347,7 +347,7 @@ Do not file these as defects (R-370). A change to one is a decision for the oper
|
||||
7. The backup unit of a removed app that the household kept (`07-backup-architecture.md` §6).
|
||||
8. Images in the registry (decision 62). This is not on a box.
|
||||
|
||||
### 14.3 The host system disk — a budget from measured numbers [FACT for the numbers; the answer is OPEN — D75, R-1370]
|
||||
### 14.3 The host system disk — a budget from measured numbers [FACT for the numbers; the smallest disk is RULED, not built — `09` §3 decision 295]
|
||||
|
||||
Measured 2026-11-10 (`audits/space-list-2026-11-10/evidence/reading-1/`, GB = 10⁹ bytes):
|
||||
|
||||
@@ -384,7 +384,7 @@ the space check). Measured C: 9.1 GB (demo-hp, 26 containers), 3.15 GB (Tester 1
|
||||
| 3 | 9.1 GB | 39.7 GB | 53.7 GB |
|
||||
| 1 | 9.1 GB | 21.5 GB | 35.5 GB |
|
||||
|
||||
**What follows from the numbers. This is not a decision.**
|
||||
**What follows from the numbers.**
|
||||
|
||||
- **With a backup drive** the root needs about 14 GB. The 41 GB root of a 128 GB disk holds that. The limit is then the
|
||||
pool: a 128 GB disk has a 54 GiB pool, and the guest is promised 32 + 50 GiB of it (§7). That is over-promised by
|
||||
@@ -396,8 +396,10 @@ the space check). Measured C: 9.1 GB (demo-hp, 26 containers), 3.15 GB (Tester 1
|
||||
54 GB case with 9 GB to spare.
|
||||
- The installer refuses no disk today, and nothing checks this sum at install time.
|
||||
|
||||
The question for the operator is D75 in `STATUS.md`: the smallest system disk Felhom accepts, with and without a backup
|
||||
drive.
|
||||
**Ruled 2026-11-10 (D75, `09` §3 decision 295): the smallest system disk is 200 GB, with or without a backup drive.**
|
||||
One number, because one rule is simpler to promise, and because the pool of a 128 GB disk is smaller than the guest's
|
||||
promised space even with a backup drive (the first point above). **Not built, and not ruled:** what the installer does
|
||||
below 200 GB — refuse, or warn (R-1370). demo-hp (128 GB) stays as a test box.
|
||||
|
||||
### 14.4 Open design points [each has a row]
|
||||
|
||||
@@ -410,7 +412,7 @@ drive.
|
||||
| Helper images carry no version; the guest's journal has no stated cap; off-site restores have no limit | R-1372 |
|
||||
| Old kernels: the keep rule is ruled and proven by hand, not built | `11-os-updates.md` §5.11.1 |
|
||||
| The parts of the host disk with no limit and no cleaner (H4, H5, H6, H13) | R-1087, R-1368 |
|
||||
| No budget is enforced; no smallest disk is ruled | R-1370, D75 |
|
||||
| The smallest disk is ruled (200 GB, `09` §3 decision 295). Nothing enforces it: what the installer does below 200 GB (refuse, or warn) is open | R-1370 |
|
||||
| What to do by hand when a disk is full | `runbooks/disk-full.md` (R-1369) |
|
||||
|
||||
## Decisions in code no section records [DESIGN — confirmed by the operator 2026-10-25 (D44 = all picks, R-1265)]
|
||||
|
||||
@@ -768,7 +768,7 @@ The full text of every row below: `git show b9073e8fb6:documentation/backlog/OPE
|
||||
| **R-35** | **Config-apply should not end the customer's session.** (P3) | CLOSED 2026-10-09 — DELIVERED and proven live on scratch 9202 (controller 0.304.0): signed in, restarted the controller, the same cookie read 200 (no cookie: 401); after a password change both old cookies read 401 at once and after the next restart; a new sign-in worked; the password was set back. | `audits/release-2026-10-09/proofs/`D4/ |
|
||||
| **R-79** | **`report.Issues` / `report.Warnings` are English on customer-facing surfaces** (P3) | CLOSED 2026-10-09 — DELIVERED and proven live (hub 0.144.0 + controller 0.304.0): a `health_critical` on the Tester 1 box (a protected container stopped) → the household mail in the catch-all has no `{` and ends „A részleteket a vezérlőpultodon látod."; the operator mail still carries the details. Dashboard half on 9202: the health banners read English for `lang=en` and Hungarian for `lang=hu`. | `audits/release-2026-10-09/proofs/`D3/ |
|
||||
| **R-30** | **[P2-HIGH] Liveness presence should come from the wait channel, not the report clock.** (P3) | CLOSED 2026-10-09 — DELIVERED and measured live (hub 0.144.0): the whole Tester 1 machine shut down 05:53:27Z; the host page read „last connected 05:51 UTC" at +114 s; the „box is off" tick opened at 05:57:42Z (6 min after the last call, as designed); control from the ingress log: Tester 1's guest (192.168.0.101) made its last `/api/v1/wait` at 05:51:41Z while the other boxes kept calling. A stopped GUEST on a live host is started again by the agent in 36 s (guest-power watchdog), so only a switched-off box opens the tick. demo-hp was never tick-deleted. | `audits/release-2026-10-09/proofs/`D2/ |
|
||||
| **R-901** | **Two kinds of a household's data outlive the deletion of the customer, and no document says when they go.** (P2) | CLOSED 2026-10-09 — both halves LIVE: the hub's 1-year deletion of a deleted customer's audit rows ships in hub 0.144.0 (daily prune; nothing is a year old yet); the DooPlex `ep0-copy` job installed with the operator present — key root-only, dry run, then the 08:00 timer; the first dry run found a parser defect (ep0 answers `{"data": [...]}`) that the mass-absence guard stopped (nothing recorded, nothing deleted), fixed `69fa9cf7`, red-proved; the first real run read 5 of 5 namespaces, deleted 0. Both times are for the privacy-notice draft (R-813). | `audits/release-2026-10-09/ep0-copy-gc/` |
|
||||
| **R-901** | **Two kinds of a household's data outlive the deletion of the customer, and no document says when they go.** (P2) | CLOSED 2026-10-09 — both halves LIVE: the hub's 1-year deletion of a deleted customer's audit rows ships in hub 0.144.0 (daily prune; nothing is a year old yet); the DooPlex `ep0-copy` job installed with the operator present — key root-only, dry run, then the 08:00 timer; the first dry run found a parser defect (ep0 answers `{"data": [...]}`) that the mass-absence guard stopped (nothing recorded, nothing deleted), fixed `69fa9cf7`, red-proved; the first real run read 5 of 5 namespaces, deleted 0. Both times are for the privacy-notice draft (R-813). **Noted 2026-11-10:** the heading of `runbooks/ep0-datastore-copy.md` was corrected from this row („installed 2026-10-09”) and the operator accepted it (`09` §3 decision 296); the installed job was not re-checked on DooPlex. | `audits/release-2026-10-09/ep0-copy-gc/` |
|
||||
| **R-138** | **A shared-zone `cf_api_token` is a zone-wide DNS-write capability on a customer's box** (P3) | CLOSED 2026-10-09 — DELIVERED (hub 0.144.0) and every stored token checked once, operator present: the hub's own question (`GET /zones`, exactly one zone and it IS the domain) run read-only against the 4 stored tokens — Tester-2, demo-felhom, demo-hp, tester-1 each see 1 zone, their own domain: PASS; control: a made-up token is refused (HTTP 403). A re-save with the SAME token would not have checked it (the edit path checks only a new token or a new domain, by design), so the check ran outside the save. Full row text: `git show 53ec983849:documentation/backlog/OPEN-ITEMS.md`. R-913 keeps the read-scope limit. | `audits/release-2026-10-09/proofs/D6/` |
|
||||
|
||||
## 2026-10-08 (evening) — the operator's decision sheet D1–D10
|
||||
|
||||
@@ -219,7 +219,7 @@ stopping line that lies.
|
||||
| **R-1263** | Backup & restore | P4 | **Small items from review round 2:** a LOCAL restore's put-back mails the operator „its off-site restore FAILED” (`felhom-controller/controller/cmd/controller/main.go:1514-1518`, shared via `restore_unit.go:517-518`); Remove — even „keep my data” — deletes undeclared volumes labelled with the app's project, which no tier holds (`internal/stacks/delete.go:763-764`, R-1117); the capped copy's `--delete` reads the source in the daemon's mount view, the emptiness check in the controller's (`tier2.go:834` vs `:464`); stale comments — `felhom-agent/internal/localapi/backup_cap.go:15` („keep-last”) and `restore_crash_putback.go:32` (names a test file that does not exist); the qBittorrent password goes to whatever answers the DNS name `qbittorrent` (`upwatch/explain.go:94,108-113`); kept-copy Remove has a check-to-remove gap (`tier2_kept.go:92-111`); the release-copy restore check trusts hashes stored on ep0 (`runbooks/total-loss-of-dooplex.md`); rrsync without `-munge`/`-wo` (`audits/day-2026-10-23/08-release-copy-D33.txt:6`). | OPEN | — | fix in each repo's next attended pass | CC |
|
||||
| **R-1294** | Backup & restore | P3 | **demo-hp's weekly off-site clean-up was refused by its own guard on 2026-10-26 (window 16): the plan would remove 209 of 359 snapshots, above the week's cap (179) — nothing was deleted, and each refused week adds to the next plan.** Seen in the operator mailbox during the 2026-10-26 release run (03:17Z, `offsite_prune_guard_refused`, error; details `count_before 359, count_after 359, outcome guard-refused`). The cap refusal is kept on purpose (R-824, R-867 closed the false ones); the open question is whether a store that once exceeds the cap can ever prune again by itself, or refuses every week while it grows. demo-hp is the Tier-0 box with many hand-made backups (sweeps, probes). Earlier cap refusals: demo-hp 2026-10-03 (window 1), demo-felhom 2026-10-05 (window 3). | **OPEN — the grant is SET (D55 = 260, `09` §3 decision 268)**: 2026-10-26 19:10:31Z through the operator's one-purpose token `cc-d55-window`, read back `max:260`; it is used at demo-hp's next window request (after its nightly off-site run, ~03:15Z). Close when that window prunes; then the operator revokes the token. **Earlier:** WAITING-ON-OPERATOR — 2026-10-26 evening: CC could NOT set the D54 grant — `POST /offsite/window-grant/demo-hp` (`max_remove=220`) with CC's hub token answered **403 „missing scope offsite:write"** (the hub log shows the refusal; nothing granted). The plan also grows: restic's dry-run 2026-10-26 18:56Z still 359 copies / plan 209 / default cap 179 (before tonight's run); a simulation of the keep rule adds ~14 per night (14 of 18 groups back up nightly), so ~223 at the night of 10-27 and ~237 at 10-28 — a cap of 220 would be refused again. STATUS D55: the operator grants it, with the cap. The grant opens the box's NEXT window request, which comes after its next nightly off-site run. Ruled D54 = prune once (`09` §3 decision 266). **Earlier:** WAITING-ON-OPERATOR — 2026-10-26 (read only, `audits/r1294-offsite-cap-2026-10-26/`): the plan is HONEST (restic's own dry-run: keep 150, remove 209; every removal 7–30 days old; every app keeps ≥ 9 or its last copies; the bulk is one test day, 2026-10-16, 152 snapshots). It can NOT recover by itself: the cap is half the store and grows ~49/week while the plan grows ~84/week. Only demo-hp has this shape (demo-felhom and Tester 1 pruned normally 10-26). The pick is STATUS D54. **Earlier:** OPEN **2026-11-10 (space list, `documentation/audits/space-list-2026-11-10/TABLE.md`):** place O1, listed from documents only (the off-site box was not read in this run — second pass). **2026-11-10:** the endings list leans on this row (`documentation/audits/endings-list-2026-11-10/TABLE.md` EC23-7). | — | read the guard's cap rule (controller off-site prune) against demo-hp's snapshot list: is the plan honest (old, superseded) or not; if honest, say how such a store gets back under the cap (an attended one-time prune, or a cap that follows the store) — a design question for the operator if it changes what a box may delete | CC |
|
||||
| **R-1362** | Backup & restore | P3 | **With the internet down, the Backups page takes over 2 minutes (135 s and 145 s measured; 5 s with only the hub down).** Walked 2026-11-03 on Tester 1: every other page answers at once on the home network. The page waits on something off-site inside its handler; a household that lost its line and opens exactly this page sees a hang. `audits/security-2026-11-02/part5/outage-wan.log` | OPEN — filed 2026-11-03 | — | name the wait with a timing, then give it a short deadline and show „the off-site state could not be read" | CC |
|
||||
| **R-1374** | Backup & restore | P3 | **Four kinds of copy are kept with no end named, and nobody is reminded of them.** Found 2026-11-10 (the endings list, `documentation/audits/endings-list-2026-11-10/TABLE.md` endings EC02, EC23, EC18, EC27; read in source; 9202 has no off-site store, so not measured). (1) **Off-site snapshots of a removed app**: the removal keeps them by ruling (closed R-474), and the weekly clean-up keeps the last snapshots of each app for ever (`internal/backup/offbox_window.go:80-82`), also for an app that is gone. No rule says when they end. (2) **A whole restore from the second drive** moves the old files beside the restored ones (`internal/backup/tier2_whole.go:52`, `:132`, `:345`); nothing removes them later and no page lists them. (3) **Exported app files** stay in the export folder of the drive; the pages can start, list and read an export, none deletes one, and nothing ages it (`internal/web/handler_export.go:30-87`). **Members (answers marked N):** EC02-4, EC23-4, EC18-4, EC27-4. | OPEN — filed 2026-11-10 (endings list) | — | Operator decision first (STATUS D72): shall a removed app's off-site snapshots end after a fixed time. Then controller: a delete on the export page; a line on the kept-data page for the files a whole restore set aside. | CC + operator |
|
||||
| **R-1374** | Backup & restore | P3 | **Four kinds of copy are kept with no end named, and nobody is reminded of them.** Found 2026-11-10 (the endings list, `documentation/audits/endings-list-2026-11-10/TABLE.md` endings EC02, EC23, EC18, EC27; read in source; 9202 has no off-site store, so not measured). (1) **Off-site snapshots of a removed app**: the removal keeps them by ruling (closed R-474), and the weekly clean-up keeps the last snapshots of each app for ever (`internal/backup/offbox_window.go:80-82`), also for an app that is gone. No rule says when they end. (2) **A whole restore from the second drive** moves the old files beside the restored ones (`internal/backup/tier2_whole.go:52`, `:132`, `:345`); nothing removes them later and no page lists them. (3) **Exported app files** stay in the export folder of the drive; the pages can start, list and read an export, none deletes one, and nothing ages it (`internal/web/handler_export.go:30-87`). **Members (answers marked N):** EC02-4, EC23-4, EC18-4, EC27-4. | OPEN — filed 2026-11-10 (endings list) **2026-11-10 15:19 — RULED (D72 = yes, by the household's own choice; `09` §3 decision 292).** Not built. | — | Controller (D72): when the household pressed „delete backup data” for an app, its off-site copies end 30 days later, at the next clean-up after that day (the number of a restore copy, D62); when the household kept the backups, the off-site copies stay. The page shows the end date before anything is deleted — write that into `07-backup-architecture.md` before the build. Parts (2) and (3): a delete on the export page; a line on the kept-data page for the files a whole restore set aside. | CC |
|
||||
|
||||
## Storage & devices — 15 rows (P3 6, P4 9)
|
||||
|
||||
@@ -283,7 +283,7 @@ stopping line that lies.
|
||||
| **R-1367** | Box system & updates | P3 | **The host journal fills a small system disk: 61% of its lines are `pam_unix(sudo:session)` open/close pairs from the agent's sudo calls.** MEASURED 2026-11-09, one hour, same on demo-hp and the N100: ~14,000 journal lines, ~8,500 of them sudo session lines; the top commands are `smartctl -a` (1,074 per hour — one every ~3 s), `lxc-info -n` (662), `mkdir -p` (540), `lvs` (537), `pct exec` (~520). The journal grows by journald's default cap (10% of the filesystem, max 4 GB): the N100 sits at 4 GB, demo-hp was 3.8 GB on 2026-11-07 and reached 0.7 GB again two days after a vacuum to 500 MB. On a 39 GB root that space is what the whole-guest local copy needs (R-548). | OPEN — filed 2026-11-09 **2026-11-10 (space list, `documentation/audits/space-list-2026-11-10/TABLE.md`):** place H7: the N100 (4.28 GB, 74 files) and Tester 1 (4.29 GB, 81 files) sit at the journal's own cap; Tester 1 writes about 150 MB a day. Lines written by `sudo` are 92 / 92 / 90 % of all lines in one hour — this counts three lines for each call (the command and the session pair), while this row's 61 % counts the session pair only: the same fact, counted two ways. No rank change is proposed: the journal stops at its own cap (about 4.3 GB), so it costs space and log history, not safety. | — | Agent, through the signed config bundle: `Defaults:felhom-agent !pam_session` (and `!syslog` only if the command log has no reader — check the audit trail's needs first) in the sudoers file; a journald cap drop-in (e.g. `SystemMaxUse=500M`) on the host; and read why `smartctl -a` runs every ~3 s (the SMART poll's interval or a loop). Red-proof each in the agent's tests; measure the line rate after on demo-hp. Evidence: `/mnt/5_hdd/felhom.eu/evidence/demo-hp-images-2026-11-09/journal-noise.txt` (off-git) | CC |
|
||||
| **R-1368** | Box system & updates | P3 | **Five places on the host's system disk only grow: nothing deletes them and no rule says how many to keep.** MEASURED 2026-11-10 on demo-hp, demo-felhom and Tester 1 (the space list, `documentation/audits/space-list-2026-11-10/TABLE.md` places H5, H6, H9, H12, H13). (1) **Old kernels** (`/lib/modules`, `/boot`): 8 / 8 / 3 kernels = 8.15 / 8.15 / 3.05 GB. The ring-0 boxes got 7 kernels in 37 days and no kernel was ever removed: the update wrapper refuses a plan that removes a package (`11-os-updates.md` rule R4), and nothing runs `apt autoremove` (searched: architecture folder, both registers, runbooks, three source trees — 0 hits). The N100's system disk grew 4.6 GiB in 26 days, about a kernel a week. (2) **The agent's self-update folder** `/var/lib/felhom-agent/selfupdate`: 49 / 48 / 33 binaries = 740 / 724 / 509 MB, one for each agent version the box ever got; no document names the folder. (3) **Guest template archives** `vzdump-lxc-9100-*` in `/var/lib/vz/dump`: 1 / 4 / 1 = 0.65 / 2.56 / 0.65 GB. (4) **Old agent binary copies** `/usr/local/bin/felhom-agent.bak-*`: 1 / 11 / 0 files, up to 155 MB. (5) **Logs of failed whole-guest backup runs** in the dump folder (small; 9 on demo-hp). **Members (answers marked N):** H5-3, H5-4, H6-3, H9-3, H9-4, H9-8, H12-3, H12-4, H13-3, H13-4. | OPEN — filed 2026-11-10 (space list) **2026-11-10 12:22 — RULED (D67 = yes, as a separate step; `09` §3 decision 286):** the box may remove old kernels by itself; rule R4 stays as it is for update plans; the clean-up is its own step with its own narrow rule (old kernel packages only, only after a good boot on the newest kernel). **2026-11-10:** the endings list leans on this row (`documentation/audits/endings-list-2026-11-10/TABLE.md` EA01-4, EA23-4, EA24-4, EA26-4). **2026-11-10 — the kernel part's fix shape is PROVEN by hand on demo-hp** (`documentation/audits/kernel-cleanup-spike-2026-11-10/`; design `11-os-updates.md` §5.11.1, not built): the rule removed 6 of 8 kernels and nothing else, 6.4 GB gained, boot settings unchanged, one orderly reboot back on the same kernel, a second run removed nothing. Not proven: a box whose running kernel is not its newest; a boot of the kept old kernel. **2026-11-10 — part (2) FIXED IN SOURCE, NOT RELEASED** (felhom-agent `776025c8`): after an update that ended well the agent removes every staged binary but the running version's; nothing is removed while an update is pending. `TestPrune_*` (6 tests, one runs the wrapper's real commit and rollback verbs), red seen three times. A box cleans its folder at the first good update after the release that carries it. Part (4), the `.bak-*` copies, is not reached (root-owned, made by a person). Parts (1), (3), (5) stay open. | the kernel part: the hand proof on demo-hp (`documentation/audits/kernel-cleanup-spike-2026-11-10/`), then an agent release | Agent: after a good update keep the running kernel, the one before it and the newest, and remove the rest — ring 0 first; keep the newest N self-update binaries and `.bak` copies; keep one template archive. Each with a test that a kept item is never removed. | CC |
|
||||
| **R-1369** | Box system & updates | P3 | **When a disk is full there is no written way out: no runbook and no self-repair, for the host system disk, the thin pool, the guest disks or a data drive.** Found 2026-11-10 (the space list, `documentation/audits/space-list-2026-11-10/TABLE.md`, question 7). One missing runbook, seen from 21 places; the search of the runbooks folder for a disk-full procedure gave 0 hits (`evidence/searches.txt` S6). The alarms exist (hub: 90 / 95 % host disk, 85 / 90 % thin pool; controller: 85 / 95 % on the data volume and each drive) but they end in a mail, not in a step. For three small places nobody wrote what fails when they are full (EFI partition, host and guest temp folders). **Members (answers marked N):** H1-7, H2-7, H5-7, H9-7, H10-5, H10-7, H12-7, H13-7, H15-5, G1-7, G2-7, G7-7, G12-7, G13-7, G14-7, G17-5, D1-7, D2-7, D3-7, D6-7, D7-7, D8-7, D9-7, D10-7. | OPEN — filed 2026-11-10 (space list) **2026-11-10 (afternoon) — the runbook is written:** `documentation/runbooks/disk-full.md`, five parts, each hand step marked PROVEN (date and evidence; all on demo-hp) or NOT PROVEN. Still open: one rehearsal of the whole runbook on demo-hp; the NOT PROVEN steps (copies on a former target, template archives, the `.bak-*` binaries, the guest root disk); what fails when the EFI partition or a temp folder is full. | — | Docs: one runbook „a disk is full” with one section per disk (host system disk, thin pool, guest root, data volume, data drive): what to look at first, what is safe to delete (the places of R-1368 and R-1367), what is never deleted by hand (household data, the kept-data rulings). Rehearse it once on demo-hp. | CC |
|
||||
| **R-1370** | Box system & updates | P3 | **The host's system disk has no space budget: no document says how much the host's own files may take, and the installer's sizing rule covers the guest only.** Found 2026-11-10 (the space list, `documentation/audits/space-list-2026-11-10/TABLE.md` place H1, answer H1-3; search S5 — 0 hits). Today: demo-hp 58 % of 41 GB, N100 38 % of 101 GB, Tester 1 34 % of 63 GB — and the parts that grow are the journal (R-1367), old kernels and agent binaries (R-1368), old backup copies (R-1087) and the local whole-guest copy (R-548). Second half, known but with no row until now: on a small disk the guest's own guards read the guest's 70 GiB while the thin pool behind it holds 54 GiB (demo-hp: 102 GiB promised on a 53.9 GiB pool, 70.67 % used), so only the hub's thin-pool alarm stands before a full pool. **Member:** H1-3. | OPEN — filed 2026-11-10 (space list) **2026-11-10 12:22 — RULED (D69 = yes; `09` §3 decision 288):** the section goes inside `16-storage-physical-identity.md`, no new document. **2026-11-10 (afternoon) — the section is written:** `16-storage-physical-identity.md` §14 (45 places, the list „no limit, by ruling”, the budget, the open design points); `PROMPT-TEMPLATE.md` §N.5 keeps it true. The budget from measured numbers: the root needs about 14 GB when the whole-box copies go to a backup drive, and 28–54 GB at keep 3 when they do not. **D75 (for the operator):** the smallest system disk Felhom accepts. CC's pick: 128 GB only with a backup drive, 200 GB without one. The other option, 128 GB always, lets a box with many apps and no backup drive stop its local copies. With no ruling the installer accepts any disk. Open after the ruling: the installer's refusal. | — | Docs: a short „disk space” section in an architecture document: each part of the system disk with its limit and its cleaner, pointing at the audit table. Then the installer can refuse a disk that cannot hold the sum. | CC |
|
||||
| **R-1370** | Box system & updates | P3 | **The host's system disk has no space budget: no document says how much the host's own files may take, and the installer's sizing rule covers the guest only.** Found 2026-11-10 (the space list, `documentation/audits/space-list-2026-11-10/TABLE.md` place H1, answer H1-3; search S5 — 0 hits). Today: demo-hp 58 % of 41 GB, N100 38 % of 101 GB, Tester 1 34 % of 63 GB — and the parts that grow are the journal (R-1367), old kernels and agent binaries (R-1368), old backup copies (R-1087) and the local whole-guest copy (R-548). Second half, known but with no row until now: on a small disk the guest's own guards read the guest's 70 GiB while the thin pool behind it holds 54 GiB (demo-hp: 102 GiB promised on a 53.9 GiB pool, 70.67 % used), so only the hub's thin-pool alarm stands before a full pool. **Member:** H1-3. | OPEN — filed 2026-11-10 (space list) **2026-11-10 12:22 — RULED (D69 = yes; `09` §3 decision 288):** the section goes inside `16-storage-physical-identity.md`, no new document. **2026-11-10 (afternoon) — the section is written:** `16-storage-physical-identity.md` §14 (45 places, the list „no limit, by ruling”, the budget, the open design points); `PROMPT-TEMPLATE.md` §N.5 keeps it true. The budget from measured numbers: the root needs about 14 GB when the whole-box copies go to a backup drive, and 28–54 GB at keep 3 when they do not. **D75 (for the operator):** the smallest system disk Felhom accepts. CC's pick: 128 GB only with a backup drive, 200 GB without one. The other option, 128 GB always, lets a box with many apps and no backup drive stop its local copies. With no ruling the installer accepts any disk. Open after the ruling: the installer's refusal. **2026-11-10 15:19 — RULED (D75 = one number, 200 GB, with or without a backup drive; `09` §3 decision 295).** This replaces CC's two-step pick. **Open, not ruled:** what the installer does below 200 GB — refuse, or warn. demo-hp (128 GB) stays as a test box. `16` §14.3 carries the ruling. | — | Design first: what the installer does below 200 GB (refuse, or warn) — a later session. Then the installer. | CC |
|
||||
| **R-984** | Box system & updates | P4 | **Host and agent hardening, low risk each: the backup-target route has no role check; the unsigned guest spec has no size bounds; signed ops have no maximum validity; the guarded mkfs does not refuse LVs, zvols or RAID devices; the local API is reachable from the LAN by a static route; a sudoers inspect pattern is wider than needed; install passes secrets on the command line; guest settings files are world-readable; the host offers rpcbind and DNS on the LAN.** Fix each in its own small change. Security review 2026-10-12 (`audits/security-review-2026-10-12/`); detail held off-git for the operator (`/mnt/5_hdd/felhom.eu/evidence/security-review-2026-10-12-PRIVATE/`). | OPEN | — | Batch into hardening work | CC |
|
||||
| **R-979** | Box system & updates | P3 | **Controller updates are trusted by registry tag alone (no digest, no signature), and the Update button offers any newer tag, even an unvouched one; the built-in traefik, cloudflared and FileBrowser images are also pinned by tag only.** Whoever can push to the registry decides what runs. Fix: pin by digest from the hub's vouched list; offer only vouched versions. Security review 2026-10-12 (`audits/security-review-2026-10-12/`); detail held off-git for the operator (`/mnt/5_hdd/felhom.eu/evidence/security-review-2026-10-12-PRIVATE/`). | OPEN — design-proposal | — | Design note, then build | CC |
|
||||
| **R-974** | Box system & updates | P3 | **A compromised host agent can likely become root on the host through the root wrapper that applies PBS settings: the wrapper accepts a folder path inside the agent's own writable tree and then acts on it as root.** Static (the symlink behaviour was reasoned, not run). Same class as R-861. Fix: a fixed root-owned folder, and refuse symlinks before any root action. Security review 2026-10-12 (`audits/security-review-2026-10-12/`); detail held off-git for the operator (`/mnt/5_hdd/felhom.eu/evidence/security-review-2026-10-12-PRIVATE/`). | OPEN | — | Fix the wrapper with a test on a throwaway box | CC |
|
||||
@@ -347,8 +347,8 @@ stopping line that lies.
|
||||
| **R-1293** | Hub & operator | P4 | **Once a year the hub's 02:00 database copy never reaches ep0, and the 20:00 copy is pushed twice instead.** Measured 2026-10-26 during R-1183's read (`audits/night-2026-10-25/r1183-measurement-2026-10-26.txt`): on 2026-10-25 (the clocks go back) 02:00–03:00 happens twice. The hub's `nextDailyRun` uses `time.Date`, which picks the SECOND 02:00 (CET, 01:00Z — measured with Go). DooPlex's `felhom-hub-db-backup.timer` fired at the FIRST 02:30 (CEST, 00:31Z) and pushed `hub-20261024T180000Z.db`, 391 min old — under the 7 h limit, so no refusal and no alarm; the 08:30 push took the 08:00 copy. Effect: for one morning a year ep0's newest hub copy is 12 h old, not 6 h; nothing is lost on the hub. The spring night (2027-03-28, 02:00–03:00 does not exist) is not measured. | OPEN | — | pick one: the push refuses (and says so) a newest snapshot older than the slot it serves, or the hub and the timer agree on one 02:00; then read the 2027-03-28 night (DUE-CHECKS) | CC |
|
||||
| **R-1345** | Hub & operator | P4 | **An access token's scopes cannot be changed after it is made — widening or narrowing one means a new token (and a new secret).** The operator, 2026-10-29: to give CC `hosts:read` + `hosts:write` for the live proofs he had to make `CC_HOSTS` (now in `sec`) beside `cc-releases-2`. Wanted: edit a token's scopes in the session (an audit line per change), so a scope can also be taken away after a session without retiring the secret. | OPEN | — | design + build the scope edit (session only), red-first test that an unticked scope is refused at once | CC |
|
||||
| **R-1353** | Hub & operator | P4 | **Statements still over 1 s, named by the new slow-statement line (hub 0.178.0) in its first hour:** a background deadline/restore-test check reading days of full host-report JSON (`GetHostReportsSince`, `store.go`; 1.5–9.2 s cold, from `monitor/deadline.go` and `monitor/restoretest.go` — no page waits on it); the start-up prune of `events` and `app_log_issues` (1.1 s and 1.8 s; no index on `created_at` / `last_seen`); the customer page's newest-report read (`GetCustomer`, 1.3 s cold in the pod probe). With the 1 GiB limit most of this should come from the cache once warm. Evidence: the hub log 2026-10-30 09:22–10:05 UTC; `audits/release-2026-10-30/p2-r1346/`. | OPEN **2026-11-02 read (`audits/security-2026-11-02/part4/hub-slow-lines-2026-11-01-02.txt`; the log covers only the 31 h since DooPlex restarted on 2026-11-01 16:11 CET, the lines from 10-30 are gone with the old pod):** 35 lines, 15 statements. What repeats: `VACUUM INTO` 5× (avg 110 s, the 6-hourly snapshot, expected); `GetHostReportsSince` 9× (avg 7.9 s, max 26.6 s); `GetCustomer`'s newest report 3× (1.1 s). Once, at the cold start: the prune of `events` (10.4 s) and `app_log_issues` (21 s, again 04:30), the OS-release and host lists (2–22 s). No fix before the full week (11-06). | — | read a week of slow-statement lines; index or narrow what still repeats (the history read needs only a few fields, not the whole JSON) | CC |
|
||||
| **R-1375** | Hub & operator | P2 | **The outside part of three endings is done by hand or not at all — the worst: an off-site account can stay at the provider with no record in the hub.** Found 2026-11-10 (the endings list, `documentation/audits/endings-list-2026-11-10/TABLE.md` endings EH06, EH01, EA21 and the ending "a tester ends"; read in source and documents; no account was called). (1) **Unticking the off-site tier** of a household takes nothing down and leaves no reminder (`hub/internal/web/configs.go:1737-1743`). A later RESET takes the account down only when the tier is ticked (`hub/internal/web/customer_reset.go:203-245`), so the sub-account and its data stay, are paid for, and the hub holds no record that they exist. (2) **Switching the second off-site tier off** takes nothing down: storage entry, token and copies stay; the source calls the teardown 'a deliberate future op' (agent `internal/pbsdr/manager.go:274`). (3) **A tester ends**: no handler and no runbook; done by hand once (2026-09-25); no list says what to collect. **Members (answers marked N):** EH06-4, EH01-4, EA21-4, and 'a tester ends' in the table's head. Related: R-810 (a household leaves), R-32. | OPEN — filed 2026-11-10 (endings list) | — | Operator decisions first (STATUS D71, D74). Then hub: keep the account record after an untick and take it down on RESET and delete. Docs: one runbook 'a household or a tester leaves' from the five code doors. | CC + operator |
|
||||
| **R-1376** | Hub & operator | P3 | **Records of a thing that is gone stay in six places.** Found 2026-11-10 (the endings list, `documentation/audits/endings-list-2026-11-10/TABLE.md` endings EH00, EH02, EH21, EA00, EA11, EA27; read in source, `documentation/audits/endings-list-2026-11-10/evidence/hub-delete-statements.txt`; no hub database was read). (1) **Hub:** thirteen tables have no delete statement anywhere. Deleting a box leaves its update settings, its update reports and its kernel notices; deleting a household leaves its off-site key rows, off-site window rows and abandon requests. Most are small; the update reports of each box grow with time. (2) **Host uninstall** leaves the mount units and host mounts of household drives (said when R-179 closed; no open row carried it). (3) **A drive wiped by a signed job** keeps a record and a mount unit that name the old filesystem (agent `internal/signedjobs/wipe.go:61-110`). (4) **The token of a destroyed guest** stays valid until a new guest takes its number: the token store has no remove (agent `internal/localapi/tokenstore.go:127-210`). **Members (answers marked N):** EH00-4, EH02-4, EH21-4, EA00-4, EA11-4, EA27-4. | OPEN — filed 2026-11-10 (endings list) | — | Operator decision first (STATUS D73): delete with the household, or keep as history and say so. Then hub: add the tables to the two purges and to the daily clean-up. Agent: remove a guest's token when the guest is destroyed. | CC + operator |
|
||||
| **R-1375** | Hub & operator | P2 | **The outside part of three endings is done by hand or not at all — the worst: an off-site account can stay at the provider with no record in the hub.** Found 2026-11-10 (the endings list, `documentation/audits/endings-list-2026-11-10/TABLE.md` endings EH06, EH01, EA21 and the ending "a tester ends"; read in source and documents; no account was called). (1) **Unticking the off-site tier** of a household takes nothing down and leaves no reminder (`hub/internal/web/configs.go:1737-1743`). A later RESET takes the account down only when the tier is ticked (`hub/internal/web/customer_reset.go:203-245`), so the sub-account and its data stay, are paid for, and the hub holds no record that they exist. (2) **Switching the second off-site tier off** takes nothing down: storage entry, token and copies stay; the source calls the teardown 'a deliberate future op' (agent `internal/pbsdr/manager.go:274`). (3) **A tester ends**: no handler and no runbook; done by hand once (2026-09-25); no list says what to collect. **Members (answers marked N):** EH06-4, EH01-4, EA21-4, and 'a tester ends' in the table's head. Related: R-810 (a household leaves), R-32. | OPEN — filed 2026-11-10 (endings list) **2026-11-10 15:19 — RULED (D71 = yes, D74 = yes; `09` §3 decisions 291, 294).** Not built. | — | Hub (D71): keep the account record after an untick; RESET and delete take the account down, ticked or not. Docs (D74): the runbook 'a household or a tester leaves', from the five code doors, in a later documents-only session. Parts (2) and (3) of this row have no ruling of their own. | CC |
|
||||
| **R-1376** | Hub & operator | P3 | **Records of a thing that is gone stay in six places.** Found 2026-11-10 (the endings list, `documentation/audits/endings-list-2026-11-10/TABLE.md` endings EH00, EH02, EH21, EA00, EA11, EA27; read in source, `documentation/audits/endings-list-2026-11-10/evidence/hub-delete-statements.txt`; no hub database was read). (1) **Hub:** thirteen tables have no delete statement anywhere. Deleting a box leaves its update settings, its update reports and its kernel notices; deleting a household leaves its off-site key rows, off-site window rows and abandon requests. Most are small; the update reports of each box grow with time. (2) **Host uninstall** leaves the mount units and host mounts of household drives (said when R-179 closed; no open row carried it). (3) **A drive wiped by a signed job** keeps a record and a mount unit that name the old filesystem (agent `internal/signedjobs/wipe.go:61-110`). (4) **The token of a destroyed guest** stays valid until a new guest takes its number: the token store has no remove (agent `internal/localapi/tokenstore.go:127-210`). **Members (answers marked N):** EH00-4, EH02-4, EH21-4, EA00-4, EA11-4, EA27-4. | OPEN — filed 2026-11-10 (endings list) **2026-11-10 15:19 — RULED (D73 = yes, with one exception; `09` §3 decision 293).** Not built. | — | Hub (D73): add the tables with no delete to the two purges and to the daily clean-up; the delete journal (`host_deletions`, `customer_resets`) stays on purpose, and the privacy text says so. Agent: remove a guest's token when the guest is destroyed. Parts (2) and (3) have no ruling of their own. | CC |
|
||||
|
||||
## Business & legal — 14 rows (P2 4, P3 2, P4 8)
|
||||
|
||||
|
||||
@@ -5,6 +5,10 @@ Moved here byte for byte on 2026-10-14 (night session), newest first, so that `S
|
||||
|
||||
**Updated 2026-10-13 (night): demo-hp, demo-felhom and Tester 1 run agent 0.155.0; demo-felhom runs controller 0.310.0, the other two 0.309.0 until the floor is raised. The open-items list is at 127. Reports: `REPORT-poweroff.md`, `REPORT-plugmeter.md`, `REPORT-usenet-2026-10-12.md`, `REPORT-newapps-2026-10-12.md`, `REPORT-disk-evening-2026-10-12.md`, `REPORT-answers-2026-10-12.md`, `REPORT-m3-2026-10-11.md`, `REPORT-redesign-2026-10-10.md`, `REPORT-testers-2026-10-10.md`, `REPORT-register-shrink-2026-10-10.md`, `REPORT-new-apps-2026-10-10.md`, `REPORT-release-2026-10-10.md`, `REPORT-break-the-circle-2026-10-09.md`, `REPORT-dooplex-survival-2026-10-09.md`, `REPORT-day4-2026-10-09.md`.**
|
||||
|
||||
## Moved from STATUS on 2026-11-10 (late afternoon)
|
||||
|
||||
**Updated 2026-11-10 (afternoon).** CC listed every way a thing ends on a box or in the hub (74 endings, eight questions each): 22 answers were „nobody asked”, now in six new rows (R-1373…R-1378) and four questions for you (D71–D74). CC removed six old kernels from the HP box by hand with the keep rule you ruled (8 → 2, 6.4 GB back, one restart, all apps healthy after 5 minutes); nothing is built for it yet. Three first space fixes wait in source for a release, the disk-space design (`documentation/architecture/16-storage-physical-identity.md` §14) and the runbook `documentation/runbooks/disk-full.md` (R-1369) are written, and D75 is one more question for you; the report is `REPORT-endings-kernels-fixes-2026-11-10.md`.
|
||||
|
||||
## Moved from STATUS on 2026-11-10 (afternoon)
|
||||
|
||||
**Updated 2026-11-10.** CC listed every place that holds data on a box — 45 places, read-only on the three test boxes — and asked eight questions of each: 47 answers were „nobody asked”, now in five new rows (R-1368…R-1372). The worst one: nothing removes old kernels from the host; a ring-0 box holds 8 of them (8 GB) and gets about one more each week. You ruled on its four questions at 12:22 (D67–D70: yes, operator only, yes, yes — `09` §3 decisions 286–290); the list is in `documentation/audits/space-list-2026-11-10/README.md`.
|
||||
|
||||
Reference in new issue
Block a user