431 lines
39 KiB
Markdown
431 lines
39 KiB
Markdown
# 16 — Storage that knows its physical disks
|
||
|
||
> **What this is:** the design that lets Felhom know, for every storage place, which physical disk(s) it sits on — and
|
||
> use that to make backup tiers honest, to give one-disk boxes a place for household data, and to allow partitions.
|
||
> Operator ruling 2026-10-15: „Felhom must support drives split into parts; the agent and the controller must know
|
||
> whether two storage places sit on the same physical disk." Triggered by the OMV test (`audits/omv-vm-2026-10-15/`,
|
||
> R-1068). Spike: `audits/storage-identity-2026-10-15/` (Part B there). Marks as in `01`: **[FACT]** read in a named
|
||
> file or measured; **[DESIGN]** a proposal. **Status: ACCEPTED 2026-10-15** — the operator ruled D1–D6 as recommended (§12). **Build A shipped 2026-10-15**
|
||
> (§3, §4, §5's D4 sentence, §8, §10); **build B shipped 2026-10-16** (§5's internal place, §6, §7, §9) — what is built,
|
||
> and where the build differs from this text, is §13.
|
||
> Paths: A = `felhom-agent/`, C = `felhom-controller/controller/`, E = `felhom.eu/`.
|
||
|
||
## 1. Today, in one table (Part A)
|
||
|
||
| Place | Found by | Identified by | Mounted | What lives there | The backup page says |
|
||
|---|---|---|---|---|---|
|
||
| Host root / PVE `local` | PVE storage `local` [FACT] A/internal/storage/observe.go:424; never a candidate [FACT] A/internal/storage/role.go:53-79 | `path:` / `store:local` [FACT] A/internal/storage/durableid.go:54-67; role system | host `/`, `/var/lib/vz` | the **local whole-guest copies** (vzdump, retention 3) [FACT] E/scripts/felhom-host-install.sh:730-734; `07` §6.1 | `backup.target.degraded` „…ugyanazon a lemezen van, mint a rendszer…" [FACT] C/internal/web/backup_target_offer.go:78-100 |
|
||
| Guest `mp0` `/var/lib/felhom` with folders `docker/` and `sys_drive/` (bound to `/var/lib/docker`, `/mnt/sys_drive`) | config default `system_data_path=/mnt/sys_drive` [FACT] C/internal/config/config.go:166 | path only; „same disk" = `st_dev` [FACT] C/internal/system/mounts_linux.go:225-245 | thin LV in `local-lvm`, `backup=1` [FACT] A/configs/build-golden.sh:105, :221-225; measured on all 3 boxes | app volumes; Tier-1 of apps with no drive [FACT] C/internal/backup/backup.go:429-438; Tier-2 state-only fallback [FACT] C/internal/backup/tier2.go:157-178; **inside every whole-guest copy** | „Csak egy másolat készül (nincs második meghajtó)…" — *corrected 2026-10-24:* the i18n keys `note.backup.single_copy` / `note.backup.single_copy_unproven`, picked by the agent's verdict [FACT] C/internal/backup/backup.go:1485-1494 |
|
||
| Enrolled drive (`/mnt/<name>` on the host → `/mnt/felhom-drives/<name>`) | `GET /disks/candidates`: whole disks, each carrying its nested partition candidates (empty / filesystem / in use; corrected 2026-10-24) [FACT] A/internal/storage/candidates.go:84-90 | mount: `uuid:<fs-uuid>`; wipe: `byid:` wwn > nvme-eui > … [FACT] A/internal/storage/durable_device.go:32-51 | systemd unit by fs-UUID through `FELHOM_MOUNT` [FACT] A/internal/storage/mountunit.go:50-65; host bind into `mp8` | `HDD_PATH` apps, media binds [FACT] C/internal/stacks/media_library.go:24-33; Tier-1/Tier-2; optionally the whole-guest target `felhom-backup`; **not in vzdump** (`mp8` is a host bind) [FACT] `07`:477-480 | Tier-2 badges; „Nincs 2. (off-drive) másolat" [FACT] C/internal/i18n/locales/hu.json:303 |
|
||
| Network share | customer input [FACT] A/internal/localapi/netstorage.go:36-40 | none („no durable-id, no SMART") [FACT] netstorage.go:15-19 | `.automount` under `/mnt/felhom-drives/<name>` [FACT] A/internal/storage/netmount.go:18-31 | browse only; never media, never Tier-2 [FACT] media_library.go:83; tier2.go:141-143 | `err.backup.halozati_tarolo_nem_lehet_a_2` [FACT] tier2.go:37-40 |
|
||
| USB disk | same as an enrolled drive; transport never decides [FACT] C/internal/settings/settings.go:609-614 | same; the bridge may pass the real identity (N100: yes) [FACT] spike case 4a | same | same | same; SMART may be missing → „Nincs adat" |
|
||
|
||
**[FACT] What compares physical disks today:** the controller compares only paths and `st_dev` (its own comment:
|
||
„two partitions on one physical disk look "different" here", mounts_linux.go:240-242). The agent has a whole-disk
|
||
compare (`SameWholeDisk`, role.go:207-216) but uses it only for eject/decommission roles. **[FACT] What knows about
|
||
partitions:** the agent accepts `/dev/sdb1` for inspect and mkfs (A/internal/storage/validate.go:28-32) but judges the
|
||
whole disk (claim.go:144-163), lists only whole disks, and calls a disk with a partition table or any filesystem
|
||
„data-bearing" (candidates.go:66-85). Measured: an empty partition next to another is never offered (spike).
|
||
|
||
## 2. The spike in five lines (Part B)
|
||
|
||
**[FACT]** A read-only walk (partition → parent; dm/LVM → slaves; loop → backing file → its disk) named the right
|
||
physical disk in every case on real devices: one-disk N100 and demo-hp (every system place → the one SSD), two disks
|
||
(system vs enrolled drive), two partitions of one disk (both → the same disk), a USB bridge that passes the identity.
|
||
**Inside a VM (Tester 1) it can only say „unknown"**: virtual disks on the same host disk and on a different host disk
|
||
look identical (`QEMU HARDDISK`, serial = the slot name `drive-scsiN`). Not measured: a USB box that hides the serial
|
||
(R-1077). Seen: a cheap SSD with a vendor-less WWN (OUI 000000, R-1076). Full table: `audits/storage-identity-2026-10-15/README.md`.
|
||
|
||
## 3. The model
|
||
|
||
**[DESIGN]** A **storage place** is anything Felhom stores on: the system data (`mp0`), the new internal data place
|
||
(§5), an enrolled whole disk, an enrolled partition, a network share. Each place carries:
|
||
|
||
| Field | Meaning |
|
||
|---|---|
|
||
| `kind` | `system` · `internal` · `disk` · `partition` · `network` |
|
||
| `physical_disks` | the ids of the whole physical disks under it, e.g. `["serial:ATA:SanDisk_X600:182195804614"]` |
|
||
| `identity` | `proven` — every disk under it has a stable id the agent trusts; `unknown` — otherwise, with a reason code: `vm`, `no_serial`, `weak_wwn`, `usb_bridge`, `network` |
|
||
|
||
**Disk id rule [DESIGN]:** `serial:<bus>:<model>:<serial>` from udev (`ID_BUS`, `ID_MODEL`, `ID_SERIAL_SHORT`); NVMe
|
||
`eui`/`nguid` when present. A WWN is used only when its NAA OUI is not `000000` (spike case 4c). Model + serial, not WWN,
|
||
is the primary key: two disks are „the same" only when the key is equal.
|
||
|
||
**Unknown rule [DESIGN]:** `systemd-detect-virt` ≠ `none` → every disk is `unknown/vm`. A disk with no serial → `no_serial`.
|
||
A USB disk whose serial is the bridge's (all the same digit, or equal to a known bridge list) → `usb_bridge`. A network
|
||
share → `network`, `physical_disks: []`.
|
||
|
||
**How it is computed [DESIGN]:** by the **agent**, as root, from `/sys` and udev only (no controller input): the walk
|
||
the spike used, built on what exists — `walkSlaves`/`physicalDisksOf` (role.go:81-131), `parentDisk`
|
||
(A/internal/storage/hostread.go:183-201) — plus the loop step and the mapping guest path → host mount (`/mnt/felhom-drives/<name>` →
|
||
`/mnt/<name>` → device; `mp0` → its LV).
|
||
|
||
**When [DESIGN]:** at agent start, on every disk hot-plug event the agent already reacts to, on enrol/attach, and
|
||
**again right before each Tier-2 run** (a disk swapped in the same bay must not inherit the old verdict).
|
||
|
||
**Reported [DESIGN]:** `GET /disks` gains `physical_disks` + `identity` per place; the hub report's `storage_targets`
|
||
gains the same two fields (A/internal/hub/report.go:379-438), so the hub sees the agent's verdict directly. The controller
|
||
stores nothing of it; it asks.
|
||
|
||
## 4. Backup tiers
|
||
|
||
**[DESIGN] The rule:** a Tier-2 copy **counts as a second copy** only when the source place and the target place share
|
||
**no** physical disk **and** both identities are `proven`. The decision is the agent's (`POST /disks/same-disk` with the
|
||
two guest paths → `{different: bool, proven: bool, reason}`), replacing `sameDevice`/`st_dev` in `tier2.go:83-179` and
|
||
`hasOffDriveTarget` (backup.go:1390-1402).
|
||
|
||
| Situation | Tier-2 | The one-copy notice | Household sentence (HU / EN) |
|
||
|---|---|---|---|
|
||
| Different disks, both proven | made, counted | hidden | — |
|
||
| Same disk (two partitions, internal place + system) | per decision D1 | shown | „Csak egy helyi másolat készül: nincs második lemez, vagy ugyanazon a lemezen van, mint az adatok. Ez a véletlen törlés ellen véd, lemezhiba ellen nem. Lemezhiba ellen egy másik lemez vagy a távoli mentés véd." / „Only one local copy is made: there is no second disk, or it is on the same disk as the data. This protects against mistakes, not against a disk failure. A different disk or the off-site copy protects against a disk failure." |
|
||
| Unknown (VM, hidden serial) | per decision D2 | shown | „A Felhom nem tudja ellenőrizni, hogy a második meghajtó valóban másik lemez-e (virtuális gépben és egyes USB-házakban ez nem látszik), ezért nem számolja második másolatnak." / „Felhom cannot check that the second drive is really a different disk (inside a virtual machine and behind some USB enclosures this is not visible), so it does not count it as a second copy." |
|
||
| Network share | never (unchanged) | shown | unchanged |
|
||
|
||
The notice moves from a Go constant to i18n, in the informal voice (closes R-1072's half). Off-site (Tier-3, PBS) and
|
||
the local whole-guest copy are **unchanged**. **[FACT]** Today Tester 1 counts Tier-2 copies between virtual disks that
|
||
all sit on one host NVMe (R-1074).
|
||
|
||
## 5. One-disk boxes — the internal data place
|
||
|
||
**[FACT]** The install button for a data-folder app finds no drive on a one-disk box and refuses with the raw field
|
||
name (R-1068). **[FACT]** The only place on that disk today, `sys_drive`, is a folder of `mp0`, which is `backup=1`:
|
||
**anything put there rides in every local and off-site whole-guest copy** (`07` §6.1, §7.5.1). Films there would
|
||
overflow the local copy (R-548) and travel to ep0.
|
||
|
||
**[DESIGN]** The installer (new installs) creates a second thin LV on `local-lvm`, the **internal data place**
|
||
(„Belső tárhely"), formats it, and the agent mounts it **on the host** by fs-UUID at `/mnt/belso`, bound to
|
||
`/mnt/felhom-drives/belso` — exactly like an enrolled drive. So:
|
||
- it is **not** in vzdump (a host mount, like every drive) and not in PBS;
|
||
- the controller registers it as the default drive (`kind: internal`), so media and `HDD_PATH` apps install with no
|
||
extra step — **no sentence at all**; the media binds cover it like any drive;
|
||
- its `physical_disks` = the system disk, so a Tier-2 copy between it and `sys_drive` never counts (§4);
|
||
- the agent's format/wipe guard keeps refusing it as a **system** disk (role system, claim.go:72) — only the installer
|
||
makes or removes it.
|
||
|
||
On a one-disk box **without** an internal place (old boxes, §9), the install of a data-folder app answers in words:
|
||
„Ehhez az alkalmazáshoz egy adattároló kell. A Meghajtók oldalon adhatsz hozzá egyet." — never the field name.
|
||
Registering `sys_drive` as a drive (the „way through" R-1068 guessed) is **rejected** (§10).
|
||
|
||
## 6. Partitions
|
||
|
||
**[DESIGN]** A place may be one partition. The agent lists, under each whole disk, its partitions as separate
|
||
candidates with their own state:
|
||
|
||
| Partition state | Offered as | Who may |
|
||
|---|---|---|
|
||
| empty (no signature, no children) | „format this part" | the household, self-serve (it holds nothing) |
|
||
| a filesystem with no files (agent mounts read-only, counts entries) | „use this part" or „format this part" | the household |
|
||
| a filesystem with files | „use this part" (attach) only; format = the customer-confirmed data-bearing path, bound to the partition's durable id | the household with confirmation (today's user-data rule, A/internal/localapi/disks.go:912-946) |
|
||
| LVM / RAID / LUKS / swap / mounted elsewhere / on the system disk | not offered | nobody (claim.go:72-103, unchanged) |
|
||
|
||
The classifier judges **the partition** (its own signature, children, mounts) and still refuses on the whole disk's
|
||
system role and on any sibling that is foreign-mounted or a PV — those siblings make the **disk** unsafe, not the
|
||
partition. A partition's durable id = `<disk id>:part<N>` + its fs-UUID; the wipe re-resolve (wipe_reresolve.go:53-93)
|
||
binds both. **Creating or deleting partitions: never from the dashboard in this round — the installer only** (D5).
|
||
|
||
## 7. The installer
|
||
|
||
**[FACT]** Today `step_grows` gives the guest's one data volume (`mp0`) 70 / 130 / 250 GiB by the thin pool's free space
|
||
(<150 / ≥150 / ≥300 GiB), and nothing above 250 (E/scripts/felhom-host-install.sh `step_grows`, now at :2475 — line corrected 2026-10-24). Measured pools: 128 GB
|
||
disk → 54 GiB, 200 GB → 113 GiB, 512 GB → 349 GiB (PVE default layout; root capped at 96 GiB).
|
||
|
||
**[DESIGN] The rule:** `mp0` = 70 GiB (pool < 300), 130 (300–800), 250 (≥ 800). Reserve = **max(16 GiB, 10 % of the
|
||
pool)**, kept free for thin snapshots during a whole-guest copy, image pulls during updates, and the restore test (which
|
||
sizes itself, A19). Internal place = pool − 32 (rootfs) − `mp0` − reserve, made only when ≥ 20 GiB. The local
|
||
whole-guest copies stay on host `local` (root), which the data place does not touch — R-548 is not made worse: `mp0`
|
||
does not grow.
|
||
|
||
| Disk | Thin pool (GiB) | Today: data volume / unused | Proposed `mp0` | Reserve | **Internal place** |
|
||
|---|---|---|---|---|---|
|
||
| 256 GB | ≈ 140 (estimate) | 70 / ≈ 38 | 70 | 16 | **≈ 22** (settings and small apps, not a film library) |
|
||
| 512 GB | 349 (N100, measured) | 250 / ≈ 67 | 130 | 35 | **≈ 152** |
|
||
| 1 TB | ≈ 800 (estimate) | 250 / ≈ 518 | 250 | 80 | **≈ 438** |
|
||
| 2 TB | ≈ 1,720 (estimate) | 250 / ≈ 1,438 | 250 | 172 | **≈ 1,266** |
|
||
|
||
**RULED 2026-10-16 (operator, after the build B space check `audits/storage-identity-buildB-2026-10-15/SPACE-CHECK.md`):
|
||
these numbers replace the table above.** `mp0` = **50 GiB** (pool < 300), **100** (300–800), **130** (≥ 800); reserve =
|
||
**max(16 GiB, 10 %), at most 48 GiB**; internal place = pool − 32 − `mp0` − reserve, only when ≥ 20 GiB; nothing promised
|
||
beyond pool − reserve. Result: ≈ 42 / 182 / 590 / 1,510 GiB for 256 GB / 512 GB / 1 TB / 2 TB. The host root is left as
|
||
the Proxmox installer makes it (SPACE-CHECK §3).
|
||
|
||
## 8. Virtual machines
|
||
|
||
**[FACT]** Inside a VM the agent cannot tell virtual disks apart by host disk (spike cases 5–6). **[DESIGN]** Every place
|
||
on a VM box is `identity: unknown/vm`; Tier-2 there never counts (D2), and the page says why. The OMV guide tells the
|
||
household: a second disk given to the Felhom VM must be **a different physical disk on the host**, passed whole, on bus
|
||
`sata` (R-1069); otherwise it adds space, not safety. Disk health is OMV's job there (R-1073).
|
||
|
||
## 9. Existing boxes
|
||
|
||
**[DESIGN]** The agent computes identities on its next update — no migration, no change to what is stored. A box with two
|
||
real disks keeps every Tier-2 copy and every sentence (both proven, different). A box where two places share a disk
|
||
(none known in the field; Tester 1 in its VM) shows the honest notice. One-disk boxes get no internal place by
|
||
themselves; adding one later is a signed operator job (an LV in free thin space), not a household button. Hub: the two
|
||
new fields appear in reports; nothing is backfilled.
|
||
|
||
## 10. Threat note
|
||
|
||
**[DESIGN]** „Different disks" is agent-computed evidence, like the data-bearing verdict (`03` §6): the controller can
|
||
only ask, never assert. A compromised controller could show a false sentence on its own page, but cannot make the agent
|
||
call two places different, cannot change the hub's copy of the verdict, and cannot format a partition the agent judges
|
||
data-bearing. The agent never trusts a guest path it cannot map to a host mount it made.
|
||
|
||
## 11. Rejected options (one line each)
|
||
|
||
- **Register `sys_drive` as the media drive** — it is inside `mp0`, so films enter every whole-guest copy, local and off-site.
|
||
- **Grow `mp0` without a cap** — the same problem, at full size.
|
||
- **Keep `st_dev` and add a partition check in the controller** — the controller cannot see LVM, loop or the host; the agent can.
|
||
- **WWN as the primary disk key** — cheap disks ship vendor-less WWNs (N100).
|
||
- **Ask the hypervisor (OMV/libvirt) which host disk backs a virtual disk** — Felhom has no access there, and it would trust a party it does not control.
|
||
- **Let the dashboard create partitions** — a new destructive power for the household with no recovery; not needed for the cases in hand.
|
||
|
||
## 12. Decisions for the operator (D1–D6) — RULED 2026-10-15: all six as recommended
|
||
|
||
| # | Question | Options | Recommendation | If you do nothing |
|
||
|---|---|---|---|---|
|
||
| D1 | A second copy on the **same** disk (two partitions, internal place → `sys_drive`)? | (a) do not make it; say „only one copy" honestly · (b) make it, label it „same disk: against mistakes, not against a disk failure" | **(a)** — the first copy already protects against mistakes; (b) spends space and invites a false sense of safety | today's `st_dev` rule stays: two partitions would count as two disks |
|
||
| D2 | **Unknown** identity (VM, hidden serial)? | (a) never count it; the copy is still made (it may be a real second disk) and the notice stays · (b) the household confirms „these are different disks" once, recorded with the disk ids | **(a)** for now; revisit (b) when a VM install is a supported shape | VM boxes keep counting same-disk copies as second copies (Tester 1 does today) |
|
||
| D3 | The installer's internal place on a one-disk box, and its reserve | (a) §7: `mp0` 70/130/250, reserve max(16 GiB, 10 %), the rest internal · (b) a larger reserve, 20 % · (c) no internal place; household adds a drive | **(a)** | new one-disk boxes keep refusing media apps (R-1068) and leave most of a big disk unused |
|
||
| D4 | Media and data-folder apps on a one-disk **old** box | (a) a plain sentence pointing to Meghajtók; internal place only by a signed operator job · (b) register `sys_drive` | **(a)** — (b) puts films into the whole-guest copies | the raw field error stays (R-1068) |
|
||
| D5 | Who creates or deletes partitions | (a) the installer only, this round · (b) also a dashboard wizard | **(a)** | — (nothing is built either way) |
|
||
| D6 | R-1071 — films from a NAS share, read-only, for Jellyfin | (a) a separate decision; this design only labels a NAS place `network`, never a copy target · (b) fold a read-only media bind kind into this build | **(a)** — the model makes it easier (a NAS becomes one more place kind with clear rules), but it is a media decision, not an identity one | stays open as R-1071 |
|
||
|
||
**After the rulings, the build task changes:** agent (place identity in `/disks` and the report, the same-disk call,
|
||
partition candidates, the loop step) · controller (Tier-2 and the notice ask the agent; i18n sentences; the install
|
||
refusal in words; register the internal place) · installer (`step_grows` + the internal LV) · hub (store and show the two
|
||
fields) · website and guides (the OMV guide's second-disk line; the backup explanation page).
|
||
|
||
## 13. Build status
|
||
|
||
**Build A — honest backup copies (D1, D2, D4), shipped 2026-10-15** (agent v0.157.0, controller v0.317.0 with MinAgent
|
||
0.157.0, hub v0.154.0; evidence `audits/storage-identity-buildA-2026-10-15/`):
|
||
|
||
| Section | Built | Proven |
|
||
|---|---|---|
|
||
| §3 model | the agent computes `kind`, `physical_disks`, `identity` + reason for every PVE storage and every own mount; `GET /disks` and the hub report carry them | live: N100 and demo-hp `proven`, Tester 1 `unknown/vm`, the PBS store `network`; the hub stores and shows them |
|
||
| §4 tiers | `POST /disks/same-disk`; Tier-2 choice, the pre-run guard and the one-copy notice ask it (D1: same disk → no copy; D2: unknown → copy made, not counted); a NAS never counts and never hides the notice (R-1075) | live: Tester 1 shows the „cannot check" notice and its copies are made, not counted; demo-hp and N100 count theirs, no notice. **Unit only:** the same-disk skip (no box has two places on one disk), the NAS notice (no box with a NAS can reach the agent, R-1083) |
|
||
| §5 D4 | the install of a data-folder app on a box with no drive answers „Ehhez az alkalmazáshoz egy adattároló kell. A Meghajtók oldalon adhatsz hozzá egyet." with a link to the Meghajtók page; never the field name | **unit only** — no one-disk box was at hand |
|
||
| §8 VMs | every place on a VM box is `unknown/vm` | live, Tester 1 |
|
||
| §10 | the agent maps only paths to mounts it made; an absent drive (its folder on the root disk) is `unmapped`, never „the system disk" | unit, and live (`unmapped` for an absent drive on demo-hp) |
|
||
|
||
**Where the build differs from the text above:**
|
||
- §3 „When": the identity is computed **on every read** (each `GET /disks`, each report, each same-disk call), not cached
|
||
at start and on hot-plug. The rule's purpose — a swapped disk never inherits an old verdict — holds with no cache.
|
||
- `usb_bridge`: built from the spike's rule, **never seen on a real device** (the N100's USB enclosure passes the real serial).
|
||
- §4 said the notice closes „R-1072's half": it closed all of R-1072 — the other sentence was already informal.
|
||
- An older agent (no same-disk call): the controller keeps the pre-build `st_dev` rule and logs it once.
|
||
- §11 rejects `sys_drive` for household data, but the drives page still offers `/mnt/sys_drive` to attach (R-280) — R-1082.
|
||
|
||
**Build B — the internal place, partitions, the installer (2026-10-16)** (agent v0.158.1 — v0.158.0 superseded; controller
|
||
v0.318.0; installer 1.33.0; evidence `audits/storage-identity-buildB-2026-10-15/`; the D3 space check: its `SPACE-CHECK.md`,
|
||
sizes ruled above in §7):
|
||
|
||
| Section | Built | Where it differs from the text above |
|
||
|---|---|---|
|
||
| §5 internal place | the root helper `felhom-internal-place` (bundle) makes `pve/felhom-belso` by the ruled rule; the installer's step 5b (appliance only) and the signed `add_internal_place` (an old box) run it; the agent mounts it at `/mnt/belso` like a drive (kind `internal`); the controller binds and registers it as the default drive | the place is made only in **appliance** mode (a byo host's pool is the owner's); every failing read in the helper refuses (security review of 0.158.0) |
|
||
| §6 partitions | each part of a free disk is listed (empty / filesystem / in_use); „format this part", „use this part"; a data-bearing part's wipe id binds disk id + part number + fs-UUID | the „filesystem with no files" state is folded into `filesystem` (no root read-only mount; the typed confirmation covers both). **A part can be formatted only while nothing on its disk is mounted**: the root mkfs wrapper refuses any mounted node and deliberately trusts no „it is Felhom's own drive" claim (the agent could fake it — security review of 0.158.0). So a household formats its free parts before using any part; a free part next to an in-use drive can still be USED, not formatted |
|
||
| §7 installer | `step_grows` by the ruled table; step 5b | the host root is left as Proxmox makes it (SPACE-CHECK §3) |
|
||
| §9 old boxes | the signed `add_internal_place` (no parameters; the size is the rule's) | — |
|
||
|
||
**Proven live 2026-10-16** (`audits/storage-identity-buildB-2026-10-15/README.md`): a fresh one-disk install (512 GB VM,
|
||
the public installer 1.33.0, agent 0.158.1 and golden 0.318.0 vouched) → data volume 100 GiB, internal place 213 GiB,
|
||
registered as the default drive with no press; Jellyfin installed with no extra step onto it; a whole-box copy holds nothing
|
||
of it; partitions formatted and used through the wizard; the honest backup offer on a VM. **Not as written:** an empty part
|
||
asks for the typed confirmation (the probe reads the parent's partition table — R-1093) *(fixed since in agent v0.160.0:
|
||
the parent's PTTYPE is set aside when `blkid -p` proves the node is a partition of that scheme — `git tag --contains
|
||
f7b7308`; noted 2026-10-24)*; a 200 GB disk gets no internal
|
||
place under the ruled sizes (15 GiB < 20).
|
||
|
||
Also in build B: R-1084 (the whole-box backup offer asks the same-disk question) and R-1082 (`/mnt/sys_drive` is never
|
||
offered to attach, by device). A bundle that adds a path needs a step bundle on boxes with an older wrapper (R16, R-880):
|
||
0.158.1 was delivered through `0.158.1-step1`.
|
||
|
||
## 14. Disk space — every place, its limit, its cleaner, its alarm (D69, decision 288)
|
||
|
||
**What this is.** The design of disk space on a box. One line for each place that holds data: what makes it grow, its
|
||
limit, what cleans it, and which alarm fires. Written 2026-11-10 from the space list. **The evidence is not here** — it
|
||
is `audits/space-list-2026-11-10/` (`TABLE.md` holds each place with its `file:line` and its measurement; the place ids
|
||
below are the pointers into it). Counts on that day: 45 places; 21 have a name in this folder, 16 a mention, 8 none.
|
||
|
||
**[DESIGN] The rule that keeps this true.** A change that adds a place that holds data, or changes the limit, the
|
||
cleaner or the alarm of one, changes this section in the same session (`PROMPT-TEMPLATE.md` §N.5). A place with no
|
||
limit is on the list in §14.2 with its ruling, or it has a register row.
|
||
|
||
### 14.1 The places [FACT — read and measured 2026-11-10, `audits/space-list-2026-11-10/TABLE.md`]
|
||
|
||
The alarms, by short name:
|
||
|
||
- **host disk** — the hub's `host_disk_warning` at 90 % and `host_disk_critical` at 95 %. To the operator only (D68,
|
||
decision 287, `08-alarm-ladder.md`). That is in the hub's source and not in the hub that runs: until the next hub
|
||
release the household can still get the critical one.
|
||
- **pool** — the hub's `storage_fill_warning` at 85 % and `storage_fill_critical` at 90 %, on the worse of the pool's
|
||
data and metadata. To the operator.
|
||
- **data volume** — the controller, every 10 minutes: warning at 85 % or under 5 GiB free, critical at 95 % or under
|
||
2 GiB. To the household.
|
||
- **drive** — the same watch of the controller, on each registered drive. To the household.
|
||
|
||
**Host system disk** (`/`, the volume `pve/root`)
|
||
|
||
| Id | Place | Grows with | Limit | Cleaner | Alarm |
|
||
|---|---|---|---|---|---|
|
||
| H1 | the system disk as a whole | its parts below (72–82 % of the used bytes) | the size Proxmox gave it; no budget per part (§14.3, R-1370) | each part's own; the whole disk only by the operator's hand (`runbooks/disk-full.md`) | host disk |
|
||
| H3 | whole-box copies on the current local target | the guest's size (R-1366) | the newest N (default 3) | Proxmox after each good run; the agent removes half-written copies | a skipped run is reported to the hub and shown on the backup page; host disk |
|
||
| H4 | whole-box copies on a former target | a move of the target | none (R-1087) | nobody | host disk |
|
||
| H5 | guest template archives | each install and re-install | none (R-1368) | nobody; only an uninstall with `--remove-golden` | host disk |
|
||
| H6 | backup logs and half-written copies | each run | a failed run's log: none (R-1368) | Proxmox; the agent | host disk |
|
||
| H7 | the host journal | the agent's `sudo` calls (R-1367) | the journald default: 10 % of the disk, at most 4 GB | journald | host disk |
|
||
| H8 | other host logs, task logs, graphs | each task | rotation | `logrotate`, Proxmox | host disk |
|
||
| H9 | old kernels | each approved kernel | the running one, the one before it and the newest: 3 at most, about 3.4 GB (decision 286, `11-os-updates.md` §5.11.1) | the box itself at night, on a ring-0 box (agent v0.174.0); a ring-1 box: nobody yet (needs a ruling) | host disk; a refused or failed clean-up is an operator event |
|
||
| H10 | the EFI partition | a boot-loader update | 1.07 GB, 1 % used | none needed | none (R-1371) |
|
||
| H11 | the host package cache | each OS update | emptied in each run | the update wrapper | host disk |
|
||
| H12 | the agent's self-update folder | each agent update | the running version's binary, and the file of a failed update until the next good one (about 16 MB each; measured 2026-11-10 on three boxes, R-1368) | the agent, at the end of each good update | host disk |
|
||
| H13 | old agent binaries | `.prev`: one file. `.bak-*`: each re-run of the installer | none for `.bak-*` (R-1368) | nobody | host disk |
|
||
| H14 | wrapper state and previous config bundles | each bundle | the last 3 | the wrapper | host disk |
|
||
| H15 | host `/tmp` and `/var/tmp` | nothing that lasts | `/tmp` is in memory: half of memory | systemd, after 10 and 30 days | none for `/tmp` (R-1371) |
|
||
| H16 | a bare folder under a drive that is not mounted | a write while the drive is away | the write is refused | nothing to clean | `storage_disconnected`, to the household |
|
||
| H18 | swap | fixed | its volume | the kernel | `host_swap_short`, to the operator |
|
||
|
||
**Thin pool** (`pve/data`)
|
||
|
||
| Id | Place | Grows with | Limit | Cleaner | Alarm |
|
||
|---|---|---|---|---|---|
|
||
| H2 | the pool | each block a guest writes | the pool's size; auto-extend is off | the agent's weekly trim (decision 139); a left-over scratch guest: a retry every 10 minutes | pool |
|
||
| H17 | the restore test's scratch guest | each restore test | it needs restored size × 1.2 + 5 GiB free, off the tested guest's pool when it can | the agent at the end; a retry every 10 minutes | pool |
|
||
|
||
**Guest root disk** (32 GiB; 4–5 % used on three boxes)
|
||
|
||
| Id | Place | Grows with | Limit | Cleaner | Alarm |
|
||
|---|---|---|---|---|---|
|
||
| G1 | the root disk as a whole | guest OS updates | 32 GiB; the agent can grow it, never shrink it | the update wrapper, `logrotate` | none (R-1371) |
|
||
| G13 | app definitions, `/opt/docker/stacks` | the catalog's size | under 2 MB measured | an app removal deletes its `app.yaml` | none (R-1371) |
|
||
| G15 | the guest's system log | each line | the journald default; it is in memory (R-1372) | journald; a restart | none |
|
||
| G16 | the guest package cache | each guest OS update | emptied in each run | the wrapper | none (R-1371) |
|
||
| G17 | guest `/tmp` | nothing that lasts | in memory | systemd; a restart | none |
|
||
|
||
**Guest data volume** (`mp0`, `/var/lib/felhom`)
|
||
|
||
| Id | Place | Grows with | Limit | Cleaner | Alarm |
|
||
|---|---|---|---|---|---|
|
||
| G2 | the data volume as a whole | G3–G12 and G14 | 50 / 100 / 130 GiB (§7; older boxes 70 or 250). A new app is refused under max(5 GB, 10 %) free; a backup is refused at 97 % or under 1 GiB | the three image rules below; the weekly trim | data volume. A box with one volume counts it twice until the next controller release: two warnings for one fill (R-1371) |
|
||
| G3 | app images, and one undo image for each app | each install and update | a count: the current and the one before, for each app. No limit in bytes, by ruling (§14.2) | the controller | data volume |
|
||
| G4 | old controller images | each controller update | the running one and the one before | the controller | data volume |
|
||
| G5 | infrastructure images | a change of the pinned version | the pinned one and the one before | the controller | data volume |
|
||
| G6 | helper images (`alpine`, `hello-world`) | pulled once | no version is pinned (R-1372) | nobody | data volume |
|
||
| G7 | named app volumes | the app's use | none for each app; the volume's | only an app removal | data volume |
|
||
| G8 | volumes with no name | each re-create of a container | none (R-995, R-1267) | nobody | data volume (bytes only) |
|
||
| G9 | containers' writable layers; the off-site cache | temporary files; the off-site store's size | none of its own | a re-create drops it | data volume |
|
||
| G10 | container logs | each line | 10 MB × 3 for each container | Docker | data volume |
|
||
| G11 | the build cache | nothing | empty | — | data volume |
|
||
| G12 | the controller's state volume | graph rows; the catalog | rows older than 30 days are deleted | the controller | data volume |
|
||
| G14 | backups of apps with no drive (`sys_drive`) | dumps | one point for each app; refused at 97 % or under 1 GiB | the nightly replace; moved units are pruned | data volume |
|
||
|
||
**A data drive**
|
||
|
||
| Id | Place | Grows with | Limit | Cleaner | Alarm |
|
||
|---|---|---|---|---|---|
|
||
| D1 | `backups/primary` | dumps | one point for each app. A removed app's kept unit: none, by ruling (§14.2) | the nightly replace; a removal with "delete the backups too" | drive |
|
||
| D2 | `backups/secondary` | the apps' data | it needs max(2 GB, 20 %) free. A removed app's copy: none, by ruling (§14.2) | the mirror; the household, from the backup page | drive |
|
||
| D3 | `backups/offsite-restore` | each off-site restore or check | none (R-1372) | the household's delete button | drive |
|
||
| D4 | `backups/offsite-proof` | one unit a night, for minutes | a headroom check | the proof itself | drive |
|
||
| D5 | `backups/restore-undo` | each restore | 30 days unused; none under a support hold, by ruling (§14.2) | the controller's sweep | drive; the household is told on a delete |
|
||
| D6 | `kept` | each "remove, keep my data" | none, by ruling (§14.2) | the household | drive |
|
||
| D7 | `appdata` | the app's use | the drive | a removal "with data" | drive |
|
||
| D8 | `userdata` | the household | none, by ruling (§14.2) | the household | drive |
|
||
| D9 | `shares` | the household's copies | the drive | the household | drive |
|
||
| D10 | `backups/set-aside` | each re-install that finds a unit | none, by ruling (§14.2) | the household; a removal with "delete its backups too" | drive |
|
||
|
||
### 14.2 What can grow without a limit, by ruling [FACT — each line is a ruling, not a defect]
|
||
|
||
Do not file these as defects (R-370). A change to one is a decision for the operator.
|
||
|
||
1. Undo images: one for each app, no limit in bytes (decision 53).
|
||
2. The kept data of a removed app: only the household deletes it, with the app's name typed (decision 40).
|
||
3. The household's own files (decision 67).
|
||
4. A set-aside backup unit (decision 256).
|
||
5. A restore's undo copy while a support hold is on it (decision 281).
|
||
6. The second-drive copy of a removed app (D25).
|
||
7. The backup unit of a removed app that the household kept (`07-backup-architecture.md` §6).
|
||
8. Images in the registry (decision 62). This is not on a box.
|
||
|
||
### 14.3 The host system disk — a budget from measured numbers [FACT for the numbers; the smallest disk is RULED, not built — `09` §3 decision 295]
|
||
|
||
Measured 2026-11-10 (`audits/space-list-2026-11-10/evidence/reading-1/`, GB = 10⁹ bytes):
|
||
|
||
| | 128 GB disk (demo-hp) | 200 GB disk (Tester 1) | 512 GB disk (N100) |
|
||
|---|---|---|---|
|
||
| root volume, as Proxmox made it | 41.4 | 62.9 | 100.9 (the 96 GiB cap) |
|
||
| used | 24.0 | 21.6 | 38.1 |
|
||
| Proxmox, Debian and all that has no line of its own | 3.3 | 3.2 | 3.4 |
|
||
|
||
No rule is written here for a size between these three: the Proxmox installer's formula was not read.
|
||
|
||
**What the root needs when the whole-box copies go to a backup drive:**
|
||
|
||
| Part | GB | From |
|
||
|---|---|---|
|
||
| Proxmox, Debian, small parts | 4 | the last line above |
|
||
| kernels: the three the ruling keeps, and one in install | 4.4 | one kernel = 1.02 GB of modules + 0.10 GB in `/boot` |
|
||
| the host journal at its cap | 4.3 | measured on two boxes (H7) |
|
||
| one guest template archive | 0.65 | H5 |
|
||
| the agent's files, after the self-update fix | 0.1 | H12 (0.75 before the fix) |
|
||
| free space an OS update asks for | 0.5 | `felhom-os-apply`, the free-space check |
|
||
| **Sum** | **≈ 14** | |
|
||
|
||
With the parts nobody cleans today (8 kernels, 49 agent binaries, 4 templates) the same sum is about 21 GB, and it grows
|
||
by 1.1 GB with each kernel.
|
||
|
||
**What local whole-box copies add.** Let C be one packed copy. The copies need keep × C. A new copy also needs
|
||
1.25 × C + 1.07 GB free before it starts, because Proxmox prunes only after a good run (A `internal/backup/runner.go`,
|
||
the space check). Measured C: 9.1 GB (demo-hp, 26 containers), 3.15 GB (Tester 1), 1.45 GB (N100).
|
||
|
||
| Keep | C | Copies + the free space the next one needs | With the 14 GB above |
|
||
|---|---|---|---|
|
||
| 3 | 3.15 GB | 14.5 GB | 28.5 GB |
|
||
| 3 | 9.1 GB | 39.7 GB | 53.7 GB |
|
||
| 1 | 9.1 GB | 21.5 GB | 35.5 GB |
|
||
|
||
**What follows from the numbers.**
|
||
|
||
- **With a backup drive** the root needs about 14 GB. The 41 GB root of a 128 GB disk holds that. The limit is then the
|
||
pool: a 128 GB disk has a 54 GiB pool, and the guest is promised 32 + 50 GiB of it (§7). That is over-promised by
|
||
design and it is watched (the pool alarm). The first measured disk whose pool holds 32 + 50 + the 16 GiB reserve is
|
||
the 200 GB one (113 GiB).
|
||
- **With no backup drive** the root needs 28 to 54 GB at keep 3, by the number of apps. A 128 GB disk holds a small
|
||
household. It does not hold demo-hp's app set at keep 3. demo-hp keeps 1, and 35.5 GB in a 41.4 GB root leaves
|
||
little room: a copy that cannot fit is skipped before it starts (R-548). The 63 GB root of a 200 GB disk holds the
|
||
54 GB case with 9 GB to spare.
|
||
- The installer refuses no disk today, and nothing checks this sum at install time.
|
||
|
||
**Ruled 2026-11-10 (D75, `09` §3 decision 295): the smallest system disk is 200 GB, with or without a backup drive.**
|
||
One number, because one rule is simpler to promise, and because the pool of a 128 GB disk is smaller than the guest's
|
||
promised space even with a backup drive (the first point above). **Not built, and not ruled:** what the installer does
|
||
below 200 GB — refuse, or warn (R-1370). demo-hp (128 GB) stays as a test box.
|
||
|
||
### 14.4 Open design points [each has a row]
|
||
|
||
| Point | Row |
|
||
|---|---|
|
||
| Nobody watches the guest's root disk (G1, G13, G16). It is 4–5 % used today; app definitions lie on it | R-1371 |
|
||
| No alarm for the EFI partition (H10) or for the places in memory (H15, G15, G17) | R-1371 |
|
||
| A drive in the backup role reports as `usb` or `local-dir`. So the household gets the `storage_fill` mail for a drive it must not clean by hand | R-1371 |
|
||
| The hub's first list for a household still holds `storage_fill_critical`. It is replaced when the box sends its own list. **Not measured on the running hub:** the release run of 2026-11-10 deployed the hub that blocks the host-disk alarm at send time, but the pages that show a household's list refuse a token („session only"), and no test alarm was made. The proof is the tests and the deployed commit (`audits/release-space-fixes-2026-11-10/hub/`) | R-1371 |
|
||
| Helper images carry no version; the guest's journal has no stated cap; off-site restores have no limit | R-1372 |
|
||
| Old kernels: the clean-up is built for ring-0 boxes (agent v0.174.0); a ring-1 box still keeps every kernel | `11-os-updates.md` §5.11.1 |
|
||
| The parts of the host disk with no limit and no cleaner (H4, H5, H6, H13) | R-1087, R-1368 |
|
||
| The smallest disk is ruled (200 GB, `09` §3 decision 295). Nothing enforces it: what the installer does below 200 GB (refuse, or warn) is open | R-1370 |
|
||
| What to do by hand when a disk is full | `runbooks/disk-full.md` (R-1369) |
|
||
|
||
## Decisions in code no section records [DESIGN — confirmed by the operator 2026-10-25 (D44 = all picks, R-1265)]
|
||
|
||
Read at `file:line` on `main` by the main session:
|
||
|
||
- **`systemd-detect-virt --vm` is read once per agent process**; empty output becomes `unknown`, which is not `none`,
|
||
so every disk then reads unknown/vm — the fail-closed direction (`felhom-agent/internal/storage/identity.go:195-205`).
|
||
- **A path the controller cannot read counts as the SAME disk** (R-668, v0.269.0): no second copy is ever claimed on
|
||
it (`felhom-controller/controller/internal/backup/backup.go:1436-1450`).
|
||
- **The internal place („Belső tárhely") refuses eject, decommission and wipe** — the controller (`refuseInternalLifecycle`)
|
||
and the agent both — and allows moving its data to another drive (`internal/web/internal_place.go:20-26`).
|
||
- **The internal place's size subtracts EVERY thin volume already promised in the pool**, not only the 32 GiB rootfs and
|
||
the data volume (`felhom-agent/configs/felhom-internal-place:11-14`).
|
||
|