docs: README migrate/decommission section + REPORT for B1+B2 (v0.65.0)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -1,94 +1,69 @@
|
||||
# REPORT — TASK A storage-lifecycle cleanups → controller v0.64.0 (2026-06-14)
|
||||
# REPORT — TASK B1+B2 data migration + self-serve decommission → controller v0.65.0 (2026-06-14)
|
||||
|
||||
Two settings-layer cleanups (A1, A2) + two doc-only changes (A3, A4) from the F9 storage-registration
|
||||
diagnosis. Trunk-based, all on `main`, no branches. Each code change has a table-driven test that fails on
|
||||
the pre-fix code. Built, pushed, and deployed to demo guest 9201; verified `:0.64.0 Up (healthy)` with a
|
||||
clean startup (`controller_started (0.64.0)`, no errors).
|
||||
Customer-self-serve storage **migration** (move app data between drives) and **decommission** (retire a
|
||||
drive), implemented trunk-based on `main` against the locked spike design
|
||||
(`felhom.eu/documentation/audits/SPIKE-decommission-migration-2026-06-14.md`). Pairs with **agent
|
||||
v0.32.0**. Each change carries non-hollow tests; the load-bearing companions were mutation-proven to
|
||||
fail on the pre-fix code. Built + deployed both images to demo guest 9201 / felhom-pve.
|
||||
|
||||
> Verify against pushed source, not this report.
|
||||
> Validate against pushed source, not this report. **Live decommission/migration of real data is NOT
|
||||
> yet validated — that is the supervised B3 session (see the end).**
|
||||
|
||||
## Pre-flight
|
||||
## Confirmed baselines
|
||||
- Controller `main` was `c37ee6d` = **v0.64.0** → shipped **v0.65.0**.
|
||||
- Agent `main` was `8e6d00a` = **v0.31.0** → shipped **v0.32.0**.
|
||||
|
||||
`main` is the only branch in felhom-controller; HEAD was `688ba0d v0.63.0` (CHANGELOG top + live container
|
||||
both `:0.63.0`). Confirmed before starting.
|
||||
## Commits (controller repo unless noted)
|
||||
| # | Commit | Scope |
|
||||
|---|---|---|
|
||||
| 1 | `b57150e` | B1 migration engine (`internal/stacks/migrate.go`, MigrateAll+MigrateApp) + tests + main wiring + backup mutual-exclusion (Change 3) |
|
||||
| 2 | `16a4c3e` | B1 UI — `/api/storage/migrate{,-app,/status}` + settings.html & app_info.html + progress panel |
|
||||
| 3 | `f43697c` (felhom-agent) | v0.32.0 — `POST /disks/decommission` + intent-aware `ReassertGuestBinds` + `GuestBindStore.Remove` + tests |
|
||||
| 4 | `f2596ea` | B2b decommission orchestration + "Hiányzó tárhely" indicator + Change 4 re-enroll fix + tests |
|
||||
|
||||
## Per-item results
|
||||
## What shipped
|
||||
- **B1 engine** — crash-safe, resumable, journaled (`<dataDir>/migration.json`). Pipeline validate → stop
|
||||
→ copy (`rsync -a --checksum`, additive, **no `--delete`**; conflict-merge walk for non-app content) →
|
||||
verify (`rsync -ani --checksum`, zero pending) → flip+redeploy (`RedeployFromEnv`) → **cleanup gated on
|
||||
all-verified AND all-redeployed**. Two entry points (whole-namespace + per-app incl. SSD→drive) over one
|
||||
pipeline. Single-flight; mutually exclusive with the backup orchestrator (migration refuses while a
|
||||
backup runs; scheduled DB-dump/Tier-2 skip while a migration runs).
|
||||
- **B1 UI** — migrate-all (settings page) + per-app "Áthelyezés másik tárhelyre" (app-info page) + shared
|
||||
status poll/progress panel.
|
||||
- **B2a (agent)** — self-serve `POST /disks/decommission` mirroring eject (user-data role gate, no
|
||||
signature, never formats): intent=decommissioned + bind-prune + unmount. **`ReassertGuestBinds` is now
|
||||
intent-aware** (skips non-enrolled) — the F9-reconnect correctness fix. Operator-signed path untouched.
|
||||
- **B2b** — `handleStorageDecommission` (migrate-all-or-none): migrate-then-decommission via the migration
|
||||
done-hook, or decommission-anyway (stop apps, keep HDD_PATH). `agentapi.Decommission` added. Soft-marker
|
||||
retained. **Change 4:** re-enroll clears the marker (`ClearDecommissioned` had zero callers before).
|
||||
**"Hiányzó tárhely"** badge for apps on a missing drive.
|
||||
|
||||
### A1 — `AutoDiscoverStoragePaths` made ADDITIVE
|
||||
- **File:** `controller/internal/settings/settings.go` (the function previously at L584).
|
||||
- **Change:** removed the `if len(s.StoragePaths) > 0 { return }` early-return. It now registers only
|
||||
discovered paths NOT already present. Invariants enforced:
|
||||
- Never removes/modifies a manually-added path.
|
||||
- SKIPS any path already in the registry IN ANY STATE — including a `Decommissioned` soft-marked entry —
|
||||
so it is never re-added or reactivated (`if existing[path] { continue }`, keyed on `filepath.Clean`).
|
||||
- Never flips `IsDefault`: a newly-discovered path becomes default ONLY if the registry currently has no
|
||||
default at all (and then only the first such new path) — `IsDefault: !hasDefault`, with `hasDefault`
|
||||
seeded from existing entries and set true after the first new default.
|
||||
- **Tests:** new `controller/internal/settings/storage_discovery_test.go`
|
||||
(`TestAutoDiscoverStoragePaths_Additive`, table-driven, + `..._DecommissionedNotReactivated`):
|
||||
- (a) non-empty registry + a deployed-app path missing from registry → exactly that path registered;
|
||||
pre-existing entry byte-identical via `reflect.DeepEqual` (IsDefault asserted unchanged); new path not
|
||||
promoted to default.
|
||||
- (b) Decommissioned path present + a deployed app still pointing at it → NOT reactivated; entry
|
||||
byte-identical; path count unchanged.
|
||||
- extra cases: empty registry (first becomes default), registry-with-no-default (first new becomes
|
||||
default), fallback path pickup.
|
||||
- **Non-hollow proof:** temporarily removing the `if existing[path] { continue }` guard makes
|
||||
`..._DecommissionedNotReactivated` FAIL (`path count changed: got 4 want 2`); restored → green.
|
||||
- **Commit:** `2d4d432` (shared with A2). **Test result:** `ok internal/settings`.
|
||||
|
||||
### A2 — internal-SSD label disambiguation
|
||||
- **File:** `controller/internal/settings/settings.go` (`InferStorageLabel`).
|
||||
- **Change:** when `filepath.Base(path) == appbackup.FelhomDataDir` (the `"felhom-data"` namespace dir), the
|
||||
label is now `Belső SSD (rendszer)` instead of the colliding `Tárhely (felhom-data)`. Imported
|
||||
`internal/appbackup` for the constant (no import cycle — appbackup only depends on `internal/util`).
|
||||
Still overridable via `SetStorageLabel`.
|
||||
- **User-drive safety confirmation:** Model-A user drives register their MOUNT ROOT (e.g. `/mnt/felhom-usb`)
|
||||
— confirmed live on 9201, where the user drive is registered as `/mnt/felhom-usb`, never
|
||||
`.../felhom-data` — so the `base == felhom-data` match cannot mislabel a user drive.
|
||||
- **Untouched:** the separate host-metrics label in `web/agent_host_metrics_handler.go:91` was left as-is
|
||||
(display-only host-metrics path; A2 is the storage-registry label only).
|
||||
- **Tests:** `TestInferStorageLabel` covers `/mnt/sys_drive/felhom-data` → `Belső SSD (rendszer)`, a
|
||||
second felhom-data path, plus the existing hdd/ssd/usb/`Tárhely` branches (note: `felhom-usb` does NOT
|
||||
start with `usb`, so it correctly labels `Tárhely (felhom-usb)` — pre-existing behavior).
|
||||
- **Demo seeded-label update (before/after):** the seeded `settings.json` on 9201 had
|
||||
`"/mnt/sys_drive/felhom-data" → "Tárhely (felhom-data)"`. Updated **through the real UI endpoint**
|
||||
(`POST /settings/storage/label` → `SetStorageLabel`, the same handler the dashboard form posts to) to
|
||||
`"Belső SSD (rendszer)"`; verified persisted in `settings.json`. (Note: the public URL
|
||||
`https://felhom.demo-felhom.eu` returned 200 without applying the change — the Cloudflare/Traefik path
|
||||
mangled the POST body; posting to the container's own `127.0.0.1:8080` returned 302 success and
|
||||
persisted. Same handler either way.) Before: `Tárhely (felhom-data)`. After: `Belső SSD (rendszer)`.
|
||||
- **Commit:** `2d4d432` (shared with A1). **Test result:** `ok internal/settings`.
|
||||
|
||||
### A3 — doc: reject auto-register-on-attach (no deploy)
|
||||
- **File:** `felhom.eu/documentation/backlog/DIAGNOSIS-f9-storage-registration-gap-2026-06-14.md` (the doc
|
||||
lives in `backlog/`, not `audits/` as the task said).
|
||||
- **Change:** recommendation #1 annotated **REJECTED (2026-06-14)** — contradicts the new→enrolled
|
||||
manual-enrollment model; manual enroll is by design. Noted that the "make discovery additive" sub-point
|
||||
WAS adopted (v0.64.0 A1) but only for paths deployed apps reference, not auto-register of agent-attached
|
||||
drives.
|
||||
- **Commit:** `b5a9614` (felhom.eu repo).
|
||||
|
||||
### A4 — doc: live-validation rule (no deploy)
|
||||
- **Files:** `felhom-controller/CLAUDE.md` and `felhom-agent/CLAUDE.md` — added the verbatim "Live
|
||||
validation" paragraph to both.
|
||||
- **Commits:** `e850402` (felhom-controller), `8e6d00a` (felhom-agent).
|
||||
## Tests (per-commit green: `go build && go vet && go test ./...`)
|
||||
- `internal/stacks/migrate_test.go` — siblingName, conflict-merge (differ→`(1)`, identical→dedup, re-run
|
||||
idempotent no `(1)(1)`, 3rd→`(2)`), assert-only verify, skip-dirs, cleanup-only-after-redeploy
|
||||
(+companion: redeploy-fail → source intact), verify-catches-corruption, resume (no re-copy),
|
||||
collision-refused, backup-exclusion, single-flight, SSD→drive resolution, cleanup gate.
|
||||
- `internal/backup/migration_exclusion_test.go` — scheduled DB-dump **skipped** while migrating.
|
||||
- `internal/web/decommission_test.go` — finalize soft-marks + calls agent (entry retained), re-enroll
|
||||
clears marker (Change 4), missing-storage label, unflagged-migration no-op.
|
||||
- **Mutation-proven companions** (fail on pre-fix, pass after revert): migration collision guard, cleanup
|
||||
gate, Change-4 re-enroll clearing, and (agent) the intent-aware reassert (`disks.go` guard removed →
|
||||
decommissioned drive re-bound → test FAILS).
|
||||
|
||||
## Build / deploy / verify
|
||||
- Built+pushed `gitea.dooplex.hu/admin/felhom-controller:0.65.0` (192.168.0.180); deployed to guest 9201
|
||||
via golden/bootstrap → `docker ps`: `:0.65.0 Up (healthy)`, clean startup (`controller_started (0.65.0)`,
|
||||
no errors), `GET /api/storage/migrate/status` → `{"job":null}` (idle, wired).
|
||||
- Built agent v0.32.0 on 180; installed on felhom-pve (`/usr/local/bin/felhom-agent`, backup
|
||||
`.bak-0.31.0`), `systemctl restart felhom-agent` → active, `--version` 0.32.0.
|
||||
- **Live-proven (incidental, safe):** on agent restart the new intent-aware re-assert logged
|
||||
`F9 re-assert: skipping non-enrolled drive (intent-gated) vmid=9201 durable_id=uuid:ce9d… intent=ejected`
|
||||
— the exact drive that lingered in `guest-binds.json` (B0 spike finding) is now correctly NOT re-bound.
|
||||
|
||||
- `go build ./... && go vet ./internal/settings/ && go test ./...` → all green locally.
|
||||
- Built + pushed `gitea.dooplex.hu/admin/felhom-controller:0.64.0` on 192.168.0.180.
|
||||
- Deployed to guest 9201 via the golden/bootstrap mechanism (`docker pull` → write
|
||||
`/etc/felhom-controller-image` → restart `felhom-controller-bootstrap.service`).
|
||||
- **Verify:** `docker ps` → `gitea.dooplex.hu/admin/felhom-controller:0.64.0 Up (healthy)`; logs show
|
||||
`Event pushed: controller_started (info) — Controller elindult (0.64.0)` and no startup errors. No
|
||||
rollback needed.
|
||||
|
||||
## Commit hashes (per item)
|
||||
|
||||
| Item | Repo | Commit |
|
||||
|---|---|---|
|
||||
| A1 + A2 (code + tests + CHANGELOG + README) | felhom-controller | `2d4d432` |
|
||||
| A4 (CLAUDE.md) | felhom-controller | `e850402` |
|
||||
| A4 (CLAUDE.md) | felhom-agent | `8e6d00a` |
|
||||
| A3 (diagnosis annotation) | felhom.eu | `b5a9614` |
|
||||
## NOT yet live-validated — awaiting supervised B3
|
||||
The real end-to-end UI flow with actual data was deliberately NOT run (per the task STOP):
|
||||
- put data on a drive → decommission (migrate-to-another OR decommission-anyway) → verify data integrity
|
||||
end-to-end → drive un-enrolled and does NOT auto-mount/re-bind on reconnect → orphaned apps show
|
||||
"Hiányzó tárhely" → re-enroll clears it. Operator-present, real UI (no agent/CLI shortcuts, per the A4
|
||||
rule). The engine/endpoint logic is unit-proven and the intent-aware reassert is already proven live by
|
||||
the restart log above; the data-bearing round-trip is what B3 covers.
|
||||
|
||||
+20
-1
@@ -677,7 +677,26 @@ not just those with HDD data. Non-HDD apps can configure destination, method, an
|
||||
> **"Belső SSD (rendszer)"** instead of the colliding "Tárhely (felhom-data)". Model-A user drives
|
||||
> register their MOUNT ROOT (e.g. `/mnt/felhom-usb`), never `.../felhom-data`, so this can't mislabel a
|
||||
> user drive. Still overridable via `SetStorageLabel`.
|
||||
> - **Migration** (drive + per-stack) is **deferred** to its own slice (buttons disabled "Hamarosan").
|
||||
> - **(v0.65.0) Data migration** (`internal/stacks/migrate.go`) — move app data between drives, in-process
|
||||
> over the controller's `/mnt:/mnt:rslave` RW mount; crash-safe + resumable via a journal
|
||||
> (`<dataDir>/migration.json`). `MigrateAll(source,target)` moves the whole felhom-data namespace (every
|
||||
> app + a conflict-merge walk for non-app/customer content); `MigrateApp(app,target)` moves one app's
|
||||
> subtree (drive→drive AND SSD→drive). Pipeline: validate → stop → copy (`rsync -a --checksum`, additive,
|
||||
> **no `--delete`**) → verify (`rsync -ani --checksum`, zero pending) → flip+redeploy (`RedeployFromEnv`)
|
||||
> → cleanup. **CLEANUP — the only destructive step — is gated on every unit verified AND every app
|
||||
> redeployed.** Conflict-merge: skip-identical (checksum vs target + its `(N)` siblings), rename-on-differ
|
||||
> to lowest-free `<base>(N)<ext>`, never overwrite; idempotent. Single-flight + mutually exclusive with the
|
||||
> backup orchestrator. UI: `POST /api/storage/migrate{,-app}` + `GET /api/storage/migrate/status` (poll);
|
||||
> migrate-all on the settings page, per-app on the app-info page, shared Hungarian progress panel.
|
||||
> - **(v0.65.0) Self-serve decommission** (`handleStorageDecommission`, `POST /api/storage/decommission`) —
|
||||
> retire a drive, non-destructive (never formats). Two choices (no partial): migrate-all-then-decommission
|
||||
> (runs `MigrateAll`; the migration done-hook soft-marks the source + calls the agent on success), or
|
||||
> decommission-anyway (type-to-confirm; stops the apps, keeps their `HDD_PATH` so they show "Hiányzó
|
||||
> tárhely"). Both end at `settings.SetDecommissioned` (soft marker retained — blocks A1 resurrection) +
|
||||
> `agentapi.Decommission` (agent v0.32.0: `POST /disks/decommission`, role-gated user-data, intent +
|
||||
> bind-prune + unmount). Re-enrolling a decommissioned drive (`registerStoragePath`) clears the marker +
|
||||
> restores `Schedulable`. A deployed app whose drive is decommissioned/disconnected/absent shows the
|
||||
> **"Hiányzó tárhely"** warning badge on the dashboard/stacks/app card.
|
||||
>
|
||||
> The privileged controller-side disk subsections **below are historical** (the `internal/storage/*` scan/
|
||||
> format code was removed in 8C — execution is the agent's now).
|
||||
|
||||
Reference in New Issue
Block a user