admin
67356c9e5c
R-351 - the restore never read back where the backup said the data lived, and a second press
started a second restore. Both shipped in controller v0.217.0.
R-352 - four measured untruths about where an app's data goes:
1. 40 of 53 catalogue templates declare no data path (13 declare env_var: HDD_PATH)
2. GetDefaultStoragePath() has three non-test callers and NONE of them places data; its
field comment "new apps use this by default" has never been true
3. the first-tier backup follows the data onto the same disk (GetAppDrivePath ->
systemDataPath) - the posture Tier 2 refuses outright at tier2.go:329
4. "1 alkalmazas hasznalja" counts only Env["HDD_PATH"] == path, so it can never include
the 40-class; it means "1 of the apps that CAN use a drive does"
Visibility shipped tonight; PLACEMENT IS OPEN and is the operator's ruling. An earlier
recommendation to refuse deployment until a drive is registered was WITHDRAWN - it assumed
the customer had failed to choose, and they had no choice to make.
R-353 - a restore reported success having returned configuration and no data. OpenGist's unit
holds manifest.json + compose/ and nothing else (volume_dumps: None, db_dumps: None); the
off-site snapshot was 182.3 KB; the outcome said only "completed in 8.666896042s". Ranked as
the NEXT SESSION'S FIRST ITEM. Compounding and recorded as UNKNOWN rather than fine: whether
the 40-class reaches the off-site tier at all has not been observed - runVolumeDumps covers
them on paper, but no nightly dump run had happened on a one-hour-old box.
New: documentation/backlog/SPEC-app-data-placement-2026-08-21.md - specification only, nothing
implemented, listing the five points a placement ruling must settle. Records that the OpenGist
instance meant to be left as evidence was removed by someone between 16:57 and 17:02 UTC (not
by this session); its unit and manifest survive, and privatebin is now a live specimen.
Ceiling moved R-350 -> R-353.
163 lines
8.7 KiB
Markdown
163 lines
8.7 KiB
Markdown
# SPEC — where an app's data is placed, and who decides
|
|
|
|
**Filed 2026-08-21. Status: SPECIFICATION ONLY — nothing here is implemented, and nothing here may be
|
|
implemented without the operator's ruling. It deserves its own session.**
|
|
|
|
Measured on `demo-hp` (HP t740, controller 0.216.0, agent 0.130.0) on 2026-08-21, during and after the
|
|
box was reinstalled. Every claim below carries the `file:line` or the live observation it came from.
|
|
|
|
---
|
|
|
|
## 1. What was found
|
|
|
|
An operator walking the deploy screens saw that OpenGist's deploy page offered **only Domain and
|
|
Subdomain** — no storage field of any kind — while the Drives page showed the NVMe at
|
|
`/mnt/felhom-drives/hdd_1` marked **Alapértelmezett** and **Aktív**, with **"1 alkalmazás használja"**.
|
|
OpenGist's data and its first-tier backup were both under `/mnt/sys_drive/`.
|
|
|
|
The first hypothesis — that a customer had typed a bad path, or had failed to choose a drive — is
|
|
**wrong**. There is nothing to type and nothing to choose. The finding is narrower and worse:
|
|
|
|
> **The configured default data store is not consulted on the deploy route at all.**
|
|
|
|
## 2. The four measured facts
|
|
|
|
### 2.1 The affected class is 40 of 53 catalogue templates
|
|
|
|
```
|
|
find . -name '.felhom.yml' | wc -l -> 53
|
|
grep -rl 'env_var: HDD_PATH' --include='.felhom.yml' . | wc -> 13
|
|
do not -> 40
|
|
```
|
|
|
|
The 13 that declare a data path: `audiobookshelf calibre-web emby immich jellyfin komga navidrome
|
|
nextcloud paperless-ngx plex radarr romm sonarr`. All media libraries. The same 13 carry a `backup:`
|
|
block.
|
|
|
|
Negative control: `grep -c HDD_PATH templates/opengist/{.felhom.yml,docker-compose.yml}` returns **0**
|
|
for both — such an app cannot receive an `HDD_PATH` even if one were supplied.
|
|
|
|
### 2.2 The default store is a preference with no effect on what it names
|
|
|
|
`settings.GetDefaultStoragePath()` (`hub`-side equivalent none; controller
|
|
`internal/settings/settings.go:1196`) has exactly **three** non-test callers:
|
|
|
|
| Caller | What it actually decides |
|
|
|---|---|
|
|
| `controller/cmd/controller/main.go:410` | which drive the **metrics collector** measures |
|
|
| `controller/internal/web/server.go:733` (`primaryHDDPath`) | the dashboard **SystemInfo** panel (`handlers.go:176, 732, 876`) |
|
|
| `controller/internal/web/handler_export_upload.go:154` | where an uploaded `.fab` **import** lands |
|
|
|
|
`grep -nE 'GetDefaultStoragePath|primaryHDDPath|IsDefault' controller/internal/stacks/deploy.go
|
|
controller/internal/stacks/manager.go` returns **nothing**.
|
|
|
|
The field's own comment at `internal/settings/settings.go:453` reads `// new apps use this by default`.
|
|
**No new app has ever used it.** That comment is an invariant with no test pinning it — the class
|
|
CLAUDE.md names.
|
|
|
|
Where the data actually goes: a **named Docker volume** on the guest root filesystem. Verified live:
|
|
|
|
```
|
|
privatebin volume src=/var/lib/docker/volumes/privatebin_privatebin_data/_data
|
|
calibre-web bind src=/mnt/felhom-drives/hdd_1/userdata/media/books
|
|
```
|
|
|
|
`withPathVars` (`internal/stacks/deploy.go:600-607`) injects `USERDATA_PATH` **only if `hdd != ""`**.
|
|
|
|
### 2.3 The first-tier backup follows the data onto the same disk
|
|
|
|
`GetAppDrivePath` (`internal/backup/backup.go:324-334`): no `HDD_PATH` → returns `m.systemDataPath`.
|
|
Live:
|
|
|
|
```
|
|
/mnt/sys_drive/felhom-data/backups/primary/opengist/manifest.json
|
|
drive='/mnt/sys_drive' namespace_root='/mnt/sys_drive/felhom-data'
|
|
```
|
|
|
|
So for these 40 apps **the data and its nearest copy sit on the same physical device**, reached by a
|
|
customer doing nothing wrong.
|
|
|
|
The project already treats that posture as unacceptable — for the *other* tier. Tier 2 refuses it
|
|
outright at `internal/backup/tier2.go:329`, recording
|
|
`a kiválasztott cél ugyanazon a fizikai lemezen van`. Tier 1 has no such notion, and for a
|
|
drive-resident app it is correct that it has none: the unit is meant to live beside the data so a
|
|
restore needs the drive and nothing else. The defect is not Tier 1's rule; it is that these apps are
|
|
on the system drive in the first place.
|
|
|
|
### 2.4 The Drives page count cannot include most apps
|
|
|
|
`countAppsUsingPath` (`internal/web/handlers.go:2118-2131`) counts only
|
|
`appCfg.Env["HDD_PATH"] == storagePath`. An app with no `HDD_PATH` can never match any drive.
|
|
|
|
So **"1 alkalmazás használja" truthfully means "1 of the apps that CAN use a drive does"**, and the
|
|
40-of-53 class is invisible on that page. The code already names the class deliberately at
|
|
`handlers.go:2140`: `// An app with no HDD_PATH (SSD-resident) is never "missing".` This is an
|
|
unstated design, not an accident.
|
|
|
|
## 3. Is that class protected?
|
|
|
|
**Whole-machine tier: yes.** Verified by `df`, not assumed — `/mnt/sys_drive`, `/var/lib/docker` and
|
|
`/var/lib/felhom` are all on `pve-vm-9201-disk-1`, which is `mp0` in `/etc/pve/lxc/9201.conf` with
|
|
`backup=1`. Named volumes and sys-drive units land in the guest backup.
|
|
|
|
**Off-site tier: covered by code, NOT demonstrated.** `runVolumeDumps`
|
|
(`internal/backup/backup.go:607+`) iterates every deployed unprotected stack with named volumes; its
|
|
drive-state gates use `GetAppDrivePath`, which returns the system path — neither disconnected nor
|
|
decommissioned, so the gates pass. On paper these apps are dumped.
|
|
|
|
**It has not been seen happen.** All three units on the box reported `volume_dumps: None,
|
|
db_dumps: None` — including `calibre-web`, which is on the data drive. No nightly dump run had
|
|
occurred on a box one hour old. **This is recorded as unknown rather than fine.**
|
|
|
|
## 4. What is ruled, and what is not
|
|
|
|
**Ruled and shipped 2026-08-21 (R-351, controller):** the deploy page now **states where the app's
|
|
data will live before the button is pressed** — naming the system drive for the 40-class, and the
|
|
selected drive for the 13. Visibility only. **No placement changed. Nothing was migrated.**
|
|
|
|
**Explicitly NOT done, and not to be done without a ruling:**
|
|
|
|
- **No storage selector was added for apps that do not need one.** The field is not the point; the
|
|
placement is. Adding a drive dropdown to 40 apps whose compose never references a path would be a
|
|
control that changes nothing.
|
|
- **Deployment is NOT refused when no drive is registered.** An earlier draft of this session
|
|
recommended that and it was **withdrawn**: it was built on the belief that the customer had failed
|
|
to choose. They had no choice to make. Refusing 40 of 53 apps for a drive they cannot use would
|
|
break the ordinary path to fix a hazard the customer never touched.
|
|
- **No placement change, no migration.** Moving where apps write has consequences for every existing
|
|
deployment on every box in the fleet.
|
|
|
|
## 5. The open question this document exists to hand over
|
|
|
|
**Should a named-volume app's data live on the default data drive rather than the system drive?**
|
|
|
|
Points the next session must settle, each of which is a reason this was not decided tonight:
|
|
|
|
1. **It is a compose-template question, not only a controller question.** A named volume
|
|
(`opengist_data:/opengist`) has no path to redirect. Either the templates gain binds under
|
|
`${USERDATA_PATH}` — a 40-template catalogue change — or Docker's data-root moves, which relocates
|
|
*every* container's storage including the controller's own.
|
|
2. **Existing deployments.** Any change must answer what happens to the apps already running on the
|
|
system drive. Leaving them and changing only new installs creates two classes with no visible
|
|
difference — which is how this defect became invisible in the first place.
|
|
3. **The system drive is not always wrong.** A small config-only app on the SSD is a reasonable
|
|
placement; a media library is not. A rule that says "always the data drive" would move things that
|
|
were fine.
|
|
4. **`IsDefault` must either be consulted or removed.** A setting that names a behaviour it does not
|
|
have is worse than no setting. Whichever way the placement question goes, that field's comment at
|
|
`settings.go:453` has to become true or go away — **with a test pinning it**, since it has been a
|
|
wish since it was written.
|
|
5. **The Drives page count needs the same decision.** Whatever the rule becomes, the page must stop
|
|
implying that the apps it does not count are not using storage.
|
|
|
|
## 6. Evidence deliberately left in place
|
|
|
|
The OpenGist instance on `demo-hp` was to be left exactly where it was, as a real example of the
|
|
defect on real hardware. **It was removed by someone between 16:57 and 17:02 UTC on 2026-08-21** —
|
|
`ScanStacks: found stack "opengist" deployed=false` from 17:02:58 onward — not by this session.
|
|
|
|
**Its evidence survives:** the recovery unit and manifest at
|
|
`/mnt/sys_drive/felhom-data/backups/primary/opengist/` are intact and carry
|
|
`drive='/mnt/sys_drive'`. **`privatebin` is now a live specimen of the same class** on the same box,
|
|
with its data at `/var/lib/docker/volumes/privatebin_privatebin_data/_data`.
|