Files
felhom.eu/documentation/architecture/16-storage-physical-identity.md
T
admin 3038b23ec7
gates / gates (push) Successful in 6m43s
Storage that knows its physical disks: design draft 16 + spike; R-1074..R-1079; STATUS D10
Spike (read-only probe, tools/physdisk_probe.py): the agent-side walk (partition -> disk, dm/LVM
slaves, loop -> backing file) names the right physical disk for one-disk boxes, two disks, two
partitions of one disk and a pass-through USB bridge; inside a VM (Tester 1) every disk is "unknown".
Today's agent lists whole disks only and calls an empty partition table "data-bearing"; the
controller's Tier-2 "different disk" is st_dev. sys_drive is a folder of mp0 (backup=1), so the
design gives one-disk boxes a separate internal data place outside the whole-guest copy instead.
Draft only; waits on the operator's rulings D1-D6 (16 section 12). 07 section 6.1's stale mp1 cell
corrected. Register 163 -> 169. Scratch loop disk and Tester 1 scratch disk removed.

Claude-Session: https://claude.ai/code/session_012qRErfCoiTkvDK9N5XHbzb
2026-10-15 20:08:08 +02:00

18 KiB
Raw Blame History

16 — Storage that knows its physical disks

What this is: the design that lets Felhom know, for every storage place, which physical disk(s) it sits on — and use that to make backup tiers honest, to give one-disk boxes a place for household data, and to allow partitions. Operator ruling 2026-10-15: „Felhom must support drives split into parts; the agent and the controller must know whether two storage places sit on the same physical disk." Triggered by the OMV test (audits/omv-vm-2026-10-15/, R-1068). Spike: audits/storage-identity-2026-10-15/ (Part B there). Marks as in 01: [FACT] read in a named file or measured; [DESIGN] a proposal. Status: DRAFT — waits on the operator's rulings in §11. Nothing is built. Paths: A = felhom-agent/, C = felhom-controller/controller/, E = felhom.eu/.

1. Today, in one table (Part A)

Place Found by Identified by Mounted What lives there The backup page says
Host root / PVE local PVE storage local [FACT] A/internal/storage/observe.go:424; never a candidate [FACT] A/internal/storage/role.go:53-79 path: / store:local [FACT] A/internal/storage/durableid.go:54-67; role system host /, /var/lib/vz the local whole-guest copies (vzdump, retention 3) [FACT] E/scripts/felhom-host-install.sh:730-734; 07 §6.1 backup.target.degraded „…ugyanazon a lemezen van, mint a rendszer…" [FACT] C/internal/web/backup_target_offer.go:78-100
Guest mp0 /var/lib/felhom with folders docker/ and sys_drive/ (bound to /var/lib/docker, /mnt/sys_drive) config default system_data_path=/mnt/sys_drive [FACT] C/internal/config/config.go:166 path only; „same disk" = st_dev [FACT] C/internal/system/mounts_linux.go:225-245 thin LV in local-lvm, backup=1 [FACT] A/configs/build-golden.sh:105, :221-225; measured on all 3 boxes app volumes; Tier-1 of apps with no drive [FACT] C/internal/backup/backup.go:429-438; Tier-2 state-only fallback [FACT] C/internal/backup/tier2.go:157-178; inside every whole-guest copy „Csak egy másolat készül (nincs második meghajtó)…" — a Go constant, not i18n [FACT] backup.go:359-361, :1387-1410
Enrolled drive (/mnt/<name> on the host → /mnt/felhom-drives/<name>) GET /disks/candidates: whole disks only [FACT] A/internal/storage/candidates.go:33-53 mount: uuid:<fs-uuid>; wipe: byid: wwn > nvme-eui > … [FACT] A/internal/storage/durable_device.go:32-51 systemd unit by fs-UUID through FELHOM_MOUNT [FACT] A/internal/storage/mountunit.go:50-65; host bind into mp8 HDD_PATH apps, media binds [FACT] C/internal/stacks/media_library.go:24-33; Tier-1/Tier-2; optionally the whole-guest target felhom-backup; not in vzdump (mp8 is a host bind) [FACT] 07:477-480 Tier-2 badges; „Nincs 2. (off-drive) másolat" [FACT] C/internal/i18n/locales/hu.json:303
Network share customer input [FACT] A/internal/localapi/netstorage.go:36-40 none („no durable-id, no SMART") [FACT] netstorage.go:15-19 .automount under /mnt/felhom-drives/<name> [FACT] A/internal/storage/netmount.go:18-31 browse only; never media, never Tier-2 [FACT] media_library.go:83; tier2.go:141-143 err.backup.halozati_tarolo_nem_lehet_a_2 [FACT] tier2.go:37-40
USB disk same as an enrolled drive; transport never decides [FACT] C/internal/settings/settings.go:609-614 same; the bridge may pass the real identity (N100: yes) [FACT] spike case 4a same same same; SMART may be missing → „Nincs adat"

[FACT] What compares physical disks today: the controller compares only paths and st_dev (its own comment: „two partitions on one physical disk look "different" here", mounts_linux.go:240-242). The agent has a whole-disk compare (SameWholeDisk, role.go:207-216) but uses it only for eject/decommission roles. [FACT] What knows about partitions: the agent accepts /dev/sdb1 for inspect and mkfs (A/internal/storage/validate.go:28-32) but judges the whole disk (claim.go:144-163), lists only whole disks, and calls a disk with a partition table or any filesystem „data-bearing" (candidates.go:66-85). Measured: an empty partition next to another is never offered (spike).

2. The spike in five lines (Part B)

[FACT] A read-only walk (partition → parent; dm/LVM → slaves; loop → backing file → its disk) named the right physical disk in every case on real devices: one-disk N100 and demo-hp (every system place → the one SSD), two disks (system vs enrolled drive), two partitions of one disk (both → the same disk), a USB bridge that passes the identity. Inside a VM (Tester 1) it can only say „unknown": virtual disks on the same host disk and on a different host disk look identical (QEMU HARDDISK, serial = the slot name drive-scsiN). Not measured: a USB box that hides the serial (R-1077). Seen: a cheap SSD with a vendor-less WWN (OUI 000000, R-1076). Full table: audits/storage-identity-2026-10-15/README.md.

3. The model

[DESIGN] A storage place is anything Felhom stores on: the system data (mp0), the new internal data place (§5), an enrolled whole disk, an enrolled partition, a network share. Each place carries:

Field Meaning
kind system · internal · disk · partition · network
physical_disks the ids of the whole physical disks under it, e.g. ["serial:ATA:SanDisk_X600:182195804614"]
identity proven — every disk under it has a stable id the agent trusts; unknown — otherwise, with a reason code: vm, no_serial, weak_wwn, usb_bridge, network

Disk id rule [DESIGN]: serial:<bus>:<model>:<serial> from udev (ID_BUS, ID_MODEL, ID_SERIAL_SHORT); NVMe eui/nguid when present. A WWN is used only when its NAA OUI is not 000000 (spike case 4c). Model + serial, not WWN, is the primary key: two disks are „the same" only when the key is equal.

Unknown rule [DESIGN]: systemd-detect-virt ≠ none → every disk is unknown/vm. A disk with no serial → no_serial. A USB disk whose serial is the bridge's (all the same digit, or equal to a known bridge list) → usb_bridge. A network share → network, physical_disks: [].

How it is computed [DESIGN]: by the agent, as root, from /sys and udev only (no controller input): the walk the spike used, built on what exists — walkSlaves/physicalDisksOf (role.go:81-131), parentDisk (A/internal/storage/hostread.go:183-201) — plus the loop step and the mapping guest path → host mount (/mnt/felhom-drives/<name> → /mnt/<name> → device; mp0 → its LV).

When [DESIGN]: at agent start, on every disk hot-plug event the agent already reacts to, on enrol/attach, and again right before each Tier-2 run (a disk swapped in the same bay must not inherit the old verdict).

Reported [DESIGN]: GET /disks gains physical_disks + identity per place; the hub report's storage_targets gains the same two fields (A/internal/hub/report.go:379-438), so the hub sees the agent's verdict directly. The controller stores nothing of it; it asks.

4. Backup tiers

[DESIGN] The rule: a Tier-2 copy counts as a second copy only when the source place and the target place share no physical disk and both identities are proven. The decision is the agent's (POST /disks/same-disk with the two guest paths → {different: bool, proven: bool, reason}), replacing sameDevice/st_dev in tier2.go:83-179 and hasOffDriveTarget (backup.go:1390-1402).

Situation Tier-2 The one-copy notice Household sentence (HU / EN)
Different disks, both proven made, counted hidden —
Same disk (two partitions, internal place + system) per decision D1 shown „Csak egy helyi másolat készül: nincs második lemez, vagy ugyanazon a lemezen van, mint az adatok. Ez a véletlen törlés ellen véd, lemezhiba ellen nem. Lemezhiba ellen egy másik lemez vagy a távoli mentés véd." / „Only one local copy is made: there is no second disk, or it is on the same disk as the data. This protects against mistakes, not against a disk failure. A different disk or the off-site copy protects against a disk failure."
Unknown (VM, hidden serial) per decision D2 shown „A Felhom nem tudja ellenőrizni, hogy a második meghajtó valóban másik lemez-e (virtuális gépben és egyes USB-házakban ez nem látszik), ezért nem számolja második másolatnak." / „Felhom cannot check that the second drive is really a different disk (inside a virtual machine and behind some USB enclosures this is not visible), so it does not count it as a second copy."
Network share never (unchanged) shown unchanged

The notice moves from a Go constant to i18n, in the informal voice (closes R-1072's half). Off-site (Tier-3, PBS) and the local whole-guest copy are unchanged. [FACT] Today Tester 1 counts Tier-2 copies between virtual disks that all sit on one host NVMe (R-1074).

5. One-disk boxes — the internal data place

[FACT] The install button for a data-folder app finds no drive on a one-disk box and refuses with the raw field name (R-1068). [FACT] The only place on that disk today, sys_drive, is a folder of mp0, which is backup=1: anything put there rides in every local and off-site whole-guest copy (07 §6.1, §7.5.1). Films there would overflow the local copy (R-548) and travel to ep0.

[DESIGN] The installer (new installs) creates a second thin LV on local-lvm, the internal data place („Belső tárhely"), formats it, and the agent mounts it on the host by fs-UUID at /mnt/belso, bound to /mnt/felhom-drives/belso — exactly like an enrolled drive. So:

  • it is not in vzdump (a host mount, like every drive) and not in PBS;
  • the controller registers it as the default drive (kind: internal), so media and HDD_PATH apps install with no extra step — no sentence at all; the media binds cover it like any drive;
  • its physical_disks = the system disk, so a Tier-2 copy between it and sys_drive never counts (§4);
  • the agent's format/wipe guard keeps refusing it as a system disk (role system, claim.go:72) — only the installer makes or removes it.

On a one-disk box without an internal place (old boxes, §9), the install of a data-folder app answers in words: „Ehhez az alkalmazáshoz egy adattároló kell. A Meghajtók oldalon adhatsz hozzá egyet." — never the field name. Registering sys_drive as a drive (the „way through" R-1068 guessed) is rejected (§10).

6. Partitions

[DESIGN] A place may be one partition. The agent lists, under each whole disk, its partitions as separate candidates with their own state:

Partition state Offered as Who may
empty (no signature, no children) „format this part" the household, self-serve (it holds nothing)
a filesystem with no files (agent mounts read-only, counts entries) „use this part" or „format this part" the household
a filesystem with files „use this part" (attach) only; format = the customer-confirmed data-bearing path, bound to the partition's durable id the household with confirmation (today's user-data rule, A/internal/localapi/disks.go:912-946)
LVM / RAID / LUKS / swap / mounted elsewhere / on the system disk not offered nobody (claim.go:72-103, unchanged)

The classifier judges the partition (its own signature, children, mounts) and still refuses on the whole disk's system role and on any sibling that is foreign-mounted or a PV — those siblings make the disk unsafe, not the partition. A partition's durable id = <disk id>:part<N> + its fs-UUID; the wipe re-resolve (wipe_reresolve.go:53-93) binds both. Creating or deleting partitions: never from the dashboard in this round — the installer only (D5).

7. The installer

[FACT] Today step_grows gives the guest's one data volume (mp0) 70 / 130 / 250 GiB by the thin pool's free space (<150 / ≥150 / ≥300 GiB), and nothing above 250 (E/scripts/felhom-host-install.sh:2261-2275). Measured pools: 128 GB disk → 54 GiB, 200 GB → 113 GiB, 512 GB → 349 GiB (PVE default layout; root capped at 96 GiB).

[DESIGN] The rule: mp0 = 70 GiB (pool < 300), 130 (300–800), 250 (≥ 800). Reserve = max(16 GiB, 10 % of the pool), kept free for thin snapshots during a whole-guest copy, image pulls during updates, and the restore test (which sizes itself, A19). Internal place = pool − 32 (rootfs) − mp0 − reserve, made only when ≥ 20 GiB. The local whole-guest copies stay on host local (root), which the data place does not touch — R-548 is not made worse: mp0 does not grow.

Disk Thin pool (GiB) Today: data volume / unused Proposed mp0 Reserve Internal place
256 GB ≈ 140 (estimate) 70 / ≈ 38 70 16 ≈ 22 (settings and small apps, not a film library)
512 GB 349 (N100, measured) 250 / ≈ 67 130 35 ≈ 152
1 TB ≈ 800 (estimate) 250 / ≈ 518 250 80 ≈ 438
2 TB ≈ 1,720 (estimate) 250 / ≈ 1,438 250 172 ≈ 1,266

8. Virtual machines

[FACT] Inside a VM the agent cannot tell virtual disks apart by host disk (spike cases 5–6). [DESIGN] Every place on a VM box is identity: unknown/vm; Tier-2 there never counts (D2), and the page says why. The OMV guide tells the household: a second disk given to the Felhom VM must be a different physical disk on the host, passed whole, on bus sata (R-1069); otherwise it adds space, not safety. Disk health is OMV's job there (R-1073).

9. Existing boxes

[DESIGN] The agent computes identities on its next update — no migration, no change to what is stored. A box with two real disks keeps every Tier-2 copy and every sentence (both proven, different). A box where two places share a disk (none known in the field; Tester 1 in its VM) shows the honest notice. One-disk boxes get no internal place by themselves; adding one later is a signed operator job (an LV in free thin space), not a household button. Hub: the two new fields appear in reports; nothing is backfilled.

10. Threat note

[DESIGN] „Different disks" is agent-computed evidence, like the data-bearing verdict (03 §6): the controller can only ask, never assert. A compromised controller could show a false sentence on its own page, but cannot make the agent call two places different, cannot change the hub's copy of the verdict, and cannot format a partition the agent judges data-bearing. The agent never trusts a guest path it cannot map to a host mount it made.

11. Rejected options (one line each)

  • Register sys_drive as the media drive — it is inside mp0, so films enter every whole-guest copy, local and off-site.
  • Grow mp0 without a cap — the same problem, at full size.
  • Keep st_dev and add a partition check in the controller — the controller cannot see LVM, loop or the host; the agent can.
  • WWN as the primary disk key — cheap disks ship vendor-less WWNs (N100).
  • Ask the hypervisor (OMV/libvirt) which host disk backs a virtual disk — Felhom has no access there, and it would trust a party it does not control.
  • Let the dashboard create partitions — a new destructive power for the household with no recovery; not needed for the cases in hand.

12. Decisions for the operator (D1–D6) — STOP here

# Question Options Recommendation If you do nothing
D1 A second copy on the same disk (two partitions, internal place → sys_drive)? (a) do not make it; say „only one copy" honestly · (b) make it, label it „same disk: against mistakes, not against a disk failure" (a) — the first copy already protects against mistakes; (b) spends space and invites a false sense of safety today's st_dev rule stays: two partitions would count as two disks
D2 Unknown identity (VM, hidden serial)? (a) never count it; the copy is still made (it may be a real second disk) and the notice stays · (b) the household confirms „these are different disks" once, recorded with the disk ids (a) for now; revisit (b) when a VM install is a supported shape VM boxes keep counting same-disk copies as second copies (Tester 1 does today)
D3 The installer's internal place on a one-disk box, and its reserve (a) §7: mp0 70/130/250, reserve max(16 GiB, 10 %), the rest internal · (b) a larger reserve, 20 % · (c) no internal place; household adds a drive (a) new one-disk boxes keep refusing media apps (R-1068) and leave most of a big disk unused
D4 Media and data-folder apps on a one-disk old box (a) a plain sentence pointing to Meghajtók; internal place only by a signed operator job · (b) register sys_drive (a) — (b) puts films into the whole-guest copies the raw field error stays (R-1068)
D5 Who creates or deletes partitions (a) the installer only, this round · (b) also a dashboard wizard (a) — (nothing is built either way)
D6 R-1071 — films from a NAS share, read-only, for Jellyfin (a) a separate decision; this design only labels a NAS place network, never a copy target · (b) fold a read-only media bind kind into this build (a) — the model makes it easier (a NAS becomes one more place kind with clear rules), but it is a media decision, not an identity one stays open as R-1071

After the rulings, the build task changes: agent (place identity in /disks and the report, the same-disk call, partition candidates, the loop step) · controller (Tier-2 and the notice ask the agent; i18n sentences; the install refusal in words; register the internal place) · installer (step_grows + the internal LV) · hub (store and show the two fields) · website and guides (the OMV guide's second-disk line; the backup explanation page).