controller v0.73.0: DR recipe — emit secret-free customer+apps half in hub report
DR recipe slice (controller half), grounded in SPIKE-dr-recipe-2026-06-16. The
controller emitter is the BOUNDARY enforcement point: v1 ships an explicit
allowlist {catalog_ref, enabled, storage_bindings} and reads NOTHING from
AppConfig.Env, so no ENC:/token/password can leak. storage_bindings parsed from
the compose (${HDD_PATH}/${USERDATA_PATH} volume binds -> {container_path,
drive, subpath}).
Load-bearing tests: TestBuildAppRecipe_NoSecrets (synthetic-secret app -> none
leak) + TestBuildAppRecipe_AllowlistIsLoadBearing (red-proof companion) +
TestAppStorageBindings + TestBuildDRRecipeAppHalf. Red-proofed live: forcing the
emitter to dump Env makes the boundary test fail. recipe_version=1.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -1,5 +1,29 @@
|
|||||||
## Changelog
|
## Changelog
|
||||||
|
|
||||||
|
### v0.73.0 — DR recipe: emit the secret-free customer+apps half in the hub report (2026-06-16)
|
||||||
|
|
||||||
|
**DR recipe slice (controller half).** Additive `dr_recipe` section on the controller's hub report — the
|
||||||
|
customer + apps half of the secret-free reconstruction recipe (`SPIKE-dr-recipe-2026-06-16.md`). The hub
|
||||||
|
assembles it with the agent's storage/guest/PBS half into one customer recipe.
|
||||||
|
|
||||||
|
- `internal/report/dr_recipe.go` — `DRRecipeAppHalf{recipe_version, customer{id,display,domain}, apps[]}`
|
||||||
|
built by the pure `BuildDRRecipeAppHalf(...)` over the DEPLOYED, non-protected stacks. Per app:
|
||||||
|
`AppRecipe{catalog_ref (Meta.Slug, falls back to name), enabled, storage_bindings[]}`. Storage bindings
|
||||||
|
are parsed from the compose (`appStorageBindings`) — each `${HDD_PATH}`/`${USERDATA_PATH}` volume bind
|
||||||
|
becomes `{container_path, drive (basename of HDD_PATH), subpath}` (e.g. romm → felhom-flash:userdata/roms);
|
||||||
|
named volumes are excluded. Wired into `BuildReport`.
|
||||||
|
- **THE BOUNDARY (the emitter is the enforcement point).** v1 ships an explicit ALLOWLIST — only the three
|
||||||
|
fields above — and the emitter reads **NOTHING** from `AppConfig.Env`, so no `ENC:` value / token /
|
||||||
|
password can ride along. Allowlist, not denylist → a new field is excluded by default.
|
||||||
|
- Tests (the load-bearing no-secrets boundary test + companion): `TestBuildAppRecipe_NoSecrets` feeds an
|
||||||
|
app whose `Env` carries synthetic secrets (an `ENC:` value + a token-shaped value) and asserts the
|
||||||
|
emitted recipe contains NONE of those values and NO credential-shaped key;
|
||||||
|
`TestBuildAppRecipe_AllowlistIsLoadBearing` is the red-proof (a guard-removed shape leaks the token, the
|
||||||
|
production emitter does not); `TestAppStorageBindings` (+ `_NoHDD`) pins the compose parse; and
|
||||||
|
`TestBuildDRRecipeAppHalf` checks assemble-correctness (deployed/non-protected only) with a whole-half
|
||||||
|
secret sweep. Red-proofed live: forcing the emitter to dump `Env` makes the boundary test fail.
|
||||||
|
`recipe_version=1`, ignore-unknown on read.
|
||||||
|
|
||||||
### v0.72.0 — FileBrowser converges on boot-recreate (2026-06-16)
|
### v0.72.0 — FileBrowser converges on boot-recreate (2026-06-16)
|
||||||
|
|
||||||
Follow-up to v0.71.0: a host-reboot test found `processGuestBootChange` recreated the drive-backed app
|
Follow-up to v0.71.0: a host-reboot test found `processGuestBootChange` recreated the drive-backed app
|
||||||
|
|||||||
@@ -1,58 +1,58 @@
|
|||||||
# REPORT — controller v0.72.0: FileBrowser converges on boot-recreate (2026-06-16)
|
# REPORT — controller v0.73.0: DR recipe (customer + apps half)
|
||||||
|
|
||||||
**Deployed:** controller **v0.72.0** on guest 9201 / felhom-pve (bootstrap-managed; healthy).
|
**TASK — DR recipe slice (controller half).** Emit the secret-free customer + apps half of the
|
||||||
**Commit (trunk):** `6ea2538` (Task B) + `c8d…`-era REPORT update.
|
reconstruction recipe as an additive hub-report section. Grounded in `SPIKE-dr-recipe-2026-06-16.md`.
|
||||||
**Live-accepted:** two REAL host reboots of felhom-pve — FileBrowser re-synced after the drive-backed
|
**The controller emitter is the boundary enforcement point** — it is the component that distinguishes
|
||||||
apps were recreated, both times; all drive-backed apps recovered with zero manual intervention.
|
secret from non-secret deploy fields, so v1 ships only an allowlist and the no-secrets boundary test
|
||||||
|
lives here.
|
||||||
|
|
||||||
## Task B — the gap
|
## Implementation
|
||||||
|
|
||||||
Follow-up to v0.71.0's guest-reboot recovery. A host-reboot test found `processGuestBootChange`
|
- `internal/report/dr_recipe.go`:
|
||||||
recreated the drive-backed app stacks (so their `${HDD_PATH}` binds re-resolved against the now-live
|
- `DRRecipeAppHalf{recipe_version, customer{id,display,domain}, apps[]}`.
|
||||||
drives) but **never re-synced FileBrowser**. FileBrowser is base-infra: it binds each drive's
|
- `BuildDRRecipeAppHalf(custID, custName, domain, stacks, composeReader)` — pure given the reader;
|
||||||
`userdata` directory but has no `HDD_PATH`, so it is **not** in the boot-recreate set — its mounts went
|
one `AppRecipe` per DEPLOYED, non-protected stack.
|
||||||
stale after a reboot (the early first-boot bring-up bound them before the drives were live).
|
- `AppRecipe{catalog_ref, enabled, storage_bindings}` — the **entire v1 surface**. `buildAppRecipe`
|
||||||
|
reads NOTHING from `AppConfig.Env`.
|
||||||
|
- `appStorageBindings(composeYAML, hddPath)` — pure compose parser; each `${HDD_PATH}`/`${USERDATA_PATH}`
|
||||||
|
volume bind → `{container_path, drive=basename(HDD_PATH), subpath}`; named volumes excluded.
|
||||||
|
- `readComposeFile` — the production reader (best-effort; unreadable → no bindings, never a failure).
|
||||||
|
- `Report.DRRecipe *DRRecipeAppHalf` wired into `BuildReport` (customer fields + `GetStacks()`).
|
||||||
|
|
||||||
## Fix
|
## The boundary (the Phase-1 lesson)
|
||||||
|
|
||||||
In `processGuestBootChange` (`internal/web/intermediary.go`), **after** `pollLiveBinds` confirms the
|
The recipe carries ONLY `{catalog_ref, enabled, storage_bindings}` — identifiers/paths. It NEVER touches
|
||||||
live binds and the drive-backed apps are recreated, the boot-recreate path now triggers
|
`AppConfig.Env`, where the controller keeps `ENC:` secrets. This is an **allowlist** (a new field is
|
||||||
`go s.SyncFileBrowserMounts()` so FileBrowser converges against the now-live drives. The recreate loop
|
excluded by default), the inverse of the retired infra-backup that shipped `restic_password` /
|
||||||
was refactored into a pure, seam-testable helper:
|
`cf_api_token`. Secrets stay in the PBS whole-CT snapshot + escrow, recovered with R, never here.
|
||||||
|
|
||||||
```go
|
## Tests (the load-bearing boundary test + companion)
|
||||||
func recreateDriveBackedApps(stacks []bootStack, presentStable map[string]bool,
|
|
||||||
recreate func(bootStack), syncFB func()) (recreated, skipped int)
|
|
||||||
```
|
|
||||||
|
|
||||||
`syncFB` is invoked exactly once, AFTER all recreates — the FileBrowser sync can never run before the
|
- `TestBuildAppRecipe_NoSecrets` — emit a recipe for an app whose `Env` carries an `ENC:` value + a
|
||||||
drive-backed apps are back.
|
token-shaped value; assert NONE of the values and NO credential-shaped key survive; assert the
|
||||||
|
allowlisted facts DID emit (non-vacuous).
|
||||||
|
- `TestBuildAppRecipe_AllowlistIsLoadBearing` — the companion/red-proof: a guard-removed shape leaks the
|
||||||
|
token; the production emitter does not.
|
||||||
|
- `TestAppStorageBindings` (+ `_NoHDD`) — pins the compose parse (roms + resources bindings; named volume
|
||||||
|
excluded; rootfs app → 0 bindings).
|
||||||
|
- `TestBuildDRRecipeAppHalf` — assemble-correctness (deployed + non-protected only) + whole-half secret
|
||||||
|
sweep.
|
||||||
|
- **Live red-proof:** forcing `buildAppRecipe` to dump `Env` made `TestBuildAppRecipe_NoSecrets` and the
|
||||||
|
companion FAIL (caught the token value + the `DB_PASSWORD`/`SECRET_KEY`/`IGDB_CLIENT_SECRET` keys);
|
||||||
|
reverted → green.
|
||||||
|
|
||||||
## Tests (non-hollow, seam = the FB sync)
|
## Versioning
|
||||||
|
|
||||||
- `TestRecreateDriveBackedApps_SyncsFileBrowserAfterRecreate` — records the call sequence; asserts
|
`recipe_version=1`; read is ignore-unknown for forward-compat. The hub assembles this half with the
|
||||||
`syncFB` runs exactly once and strictly AFTER every `recreate`. Red-proofed (stubbing out the
|
agent's storage/guest/PBS half (agent v0.38.0) into one customer recipe.
|
||||||
`syncFB()` call makes it fail).
|
|
||||||
- `TestRecreateDriveBackedApps_SyncsEvenWithNoRecreate` — FileBrowser still converges when nothing
|
|
||||||
needed recreating (e.g. binds already present).
|
|
||||||
|
|
||||||
`go build`, `go vet`, `go test ./...` all green on the build server (192.168.0.180).
|
## Gate / deploy
|
||||||
|
|
||||||
## Live acceptance — real host reboot ×2 on felhom-pve
|
`go build`, `go vet`, `go test ./...` all green (local + build server). Built + pushed image v0.73.0;
|
||||||
|
deployed to guest 9201 (bootstrap-managed).
|
||||||
|
|
||||||
Both reboots: `processGuestBootChange` fired on the new boot-id, confirmed the live binds, recreated
|
## Deferred (NOT in this slice)
|
||||||
every drive-backed app, **then** ran the FileBrowser sync. Captured controller logs:
|
|
||||||
|
|
||||||
```
|
Free-form non-secret deploy fields (the SPIKE's `non_secret_deploy_fields`) — v1 is the three-field
|
||||||
[gate] boot 1781625729-1612: live bind confirmed — recreating drive-backed app … onto /mnt/felhom-drives/felhom-flash (×8 apps)
|
allowlist only; additional fields land incrementally behind the same allowlist test. No re-provisioning
|
||||||
[gate] boot 1781625729-1612: re-syncing FileBrowser mounts against the live binds
|
automation; no recovery-mode consumption.
|
||||||
[web] FileBrowser mounts synced — 3 storage path(s), config updated
|
|
||||||
```
|
|
||||||
|
|
||||||
(reboot #2 identical on boot-id `1781625955-1516`.) Post-reboot FileBrowser binds all three drives
|
|
||||||
non-stale — `felhom-usb`, `felhom-flash`, `felhom-data` → `/srv/felhom-*` — and the underlying agent
|
|
||||||
tolerated a `/dev/sdb`↔`/dev/sdc` reshuffle by mounting each drive by UUID (agent v0.37.0, Task A).
|
|
||||||
The earlier first-boot `Failed to recreate FileBrowser` line (drives not yet live) is the exact pre-fix
|
|
||||||
symptom; the boot-recreate path now recovers it.
|
|
||||||
|
|
||||||
Demo dashboard has no password set → controller API is open on the in-guest path; no secrets committed.
|
|
||||||
|
|||||||
@@ -1248,6 +1248,7 @@ Periodic JSON push (default every 15 min) to the central felhom-hub service:
|
|||||||
- **Geo-restriction (always present, v0.70.0):** `geo_restriction` is always populated — `Enabled=false` with an empty country list when never configured — so the Hub always renders the geo section ("Inaktív" when off) instead of hiding it. `buildGeoRestrictionReport` in `internal/report/builder.go`.
|
- **Geo-restriction (always present, v0.70.0):** `geo_restriction` is always populated — `Enabled=false` with an empty country list when never configured — so the Hub always renders the geo section ("Inaktív" when off) instead of hiding it. `buildGeoRestrictionReport` in `internal/report/builder.go`.
|
||||||
- **App telemetry** (v0.28.0+): Per-stack memory (current/avg/peak) and CPU averages from the last 15 minutes of metrics data, plus log scan results (error/warning counts with deduplicated issues). Only non-protected, deployed stacks are included. Backward-compatible: old Hub versions silently ignore this field.
|
- **App telemetry** (v0.28.0+): Per-stack memory (current/avg/peak) and CPU averages from the last 15 minutes of metrics data, plus log scan results (error/warning counts with deduplicated issues). Only non-protected, deployed stacks are included. Backward-compatible: old Hub versions silently ignore this field.
|
||||||
- **Controller telemetry** (v0.32.4+): The controller's own container (`felhom-controller`) is included as a special entry in the `app_telemetry` array. Its memory/CPU metrics come from the same metrics collector, and its log warnings/errors are scanned via `docker logs` using the same pipeline as app containers. This reuses all existing Hub telemetry infrastructure (memory trend charts, known issues, fleet aggregation) with zero Hub-side changes.
|
- **Controller telemetry** (v0.32.4+): The controller's own container (`felhom-controller`) is included as a special entry in the `app_telemetry` array. Its memory/CPU metrics come from the same metrics collector, and its log warnings/errors are scanned via `docker logs` using the same pipeline as app containers. This reuses all existing Hub telemetry infrastructure (memory trend charts, known issues, fleet aggregation) with zero Hub-side changes.
|
||||||
|
- **DR recipe — customer + apps half (v0.73.0):** `dr_recipe` is the controller half of the secret-free reconstruction recipe (`SPIKE-dr-recipe-2026-06-16.md`) that complements escrow (keys) + PBS/restic (bytes). `BuildDRRecipeAppHalf` (`internal/report/dr_recipe.go`) emits `{recipe_version, customer{id,display,domain}, apps[]}`; each deployed, non-protected app contributes `AppRecipe{catalog_ref, enabled, storage_bindings}` where bindings are parsed from the compose (`${HDD_PATH}`/`${USERDATA_PATH}` volume binds → `{container_path, drive, subpath}`, e.g. romm → `felhom-flash:userdata/roms`). **THE BOUNDARY:** the emitter is the enforcement point — it ships an explicit allowlist of those three fields and reads NOTHING from `AppConfig.Env`, so no `ENC:`/token/password can leak (allowlist, not denylist → new fields excluded by default). The load-bearing `TestBuildAppRecipe_NoSecrets` + its red-proof companion live here. The hub assembles this half with the agent's storage/guest/PBS half into one customer recipe. `recipe_version=1`, ignore-unknown on read.
|
||||||
|
|
||||||
Bearer token authentication, 3-attempt retry with 5-second backoff. Push status tracked via `PushStatus` struct (LastAttempt, LastSuccess, LastError, consecutive failures) — used by the monitoring page and alert system to show Hub connection health.
|
Bearer token authentication, 3-attempt retry with 5-second backoff. Push status tracked via `PushStatus` struct (LastAttempt, LastSuccess, LastError, consecutive failures) — used by the monitoring page and alert system to show Hub connection health.
|
||||||
|
|
||||||
|
|||||||
@@ -161,6 +161,11 @@ func BuildReport(
|
|||||||
// "Inaktív" hub-side.
|
// "Inaktív" hub-side.
|
||||||
r.GeoRestriction = buildGeoRestrictionReport(geoRestriction)
|
r.GeoRestriction = buildGeoRestrictionReport(geoRestriction)
|
||||||
|
|
||||||
|
// DR recipe app-half — customer identity + per-app {catalog_ref, enabled, storage_bindings}.
|
||||||
|
// Allowlist-only (the boundary): NO env/secret fields. The hub assembles it with the agent half.
|
||||||
|
r.DRRecipe = BuildDRRecipeAppHalf(cfg.Customer.ID, cfg.Customer.Name, cfg.Customer.Domain,
|
||||||
|
stackMgr.GetStacks(), readComposeFile)
|
||||||
|
|
||||||
if debug && logger != nil {
|
if debug && logger != nil {
|
||||||
logger.Printf("[DEBUG] [report] BuildReport: complete — containers=%d, health=%s, deployed=%d, available=%d, app_telemetry=%d",
|
logger.Printf("[DEBUG] [report] BuildReport: complete — containers=%d, health=%s, deployed=%d, available=%d, app_telemetry=%d",
|
||||||
r.Containers.Total, r.Health.Status, len(r.Stacks.Deployed), len(r.Stacks.Available), len(r.AppTelemetry))
|
r.Containers.Total, r.Health.Status, len(r.Stacks.Deployed), len(r.Stacks.Available), len(r.AppTelemetry))
|
||||||
|
|||||||
@@ -0,0 +1,161 @@
|
|||||||
|
package report
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bufio"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"gitea.dooplex.hu/admin/felhom-controller/internal/stacks"
|
||||||
|
)
|
||||||
|
|
||||||
|
// readComposeFile is the production composeReader: best-effort read of a stack's docker-compose.yml
|
||||||
|
// (an unreadable file yields "" → no storage bindings for that app, never a failure).
|
||||||
|
func readComposeFile(path string) string {
|
||||||
|
data, err := os.ReadFile(path)
|
||||||
|
if err != nil {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
return string(data)
|
||||||
|
}
|
||||||
|
|
||||||
|
// DR recipe — the controller (customer + apps) HALF of the secret-free reconstruction recipe
|
||||||
|
// (SPIKE-dr-recipe-2026-06-16). The recipe is the non-secret re-provision plan that complements escrow
|
||||||
|
// (keys) + PBS/restic (bytes). This emitter is THE BOUNDARY ENFORCEMENT POINT: the controller is the
|
||||||
|
// component that distinguishes secret from non-secret deploy fields (it encrypts ENC: secrets in
|
||||||
|
// app.yaml), so v1 ships ONLY an explicit allowlist — {catalog_ref, enabled, storage_bindings} — and
|
||||||
|
// NEVER touches AppConfig.Env. A new field cannot leak a secret because it is excluded by default
|
||||||
|
// (allowlist, not denylist). TestBuildAppRecipe_NoSecrets is the load-bearing proof.
|
||||||
|
//
|
||||||
|
// recipe_version=1; read is ignore-unknown (forward-compat). The hub assembles this half with the
|
||||||
|
// agent's storage/guest/PBS half into one customer recipe keyed by customer + recipe_version.
|
||||||
|
const DRRecipeVersion = 1
|
||||||
|
|
||||||
|
// DRRecipeAppHalf is the controller-emitted half.
|
||||||
|
type DRRecipeAppHalf struct {
|
||||||
|
RecipeVersion int `json:"recipe_version"`
|
||||||
|
Customer DRCustomer `json:"customer"`
|
||||||
|
Apps []AppRecipe `json:"apps"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// DRCustomer is the customer identity — public identifiers only.
|
||||||
|
type DRCustomer struct {
|
||||||
|
ID string `json:"id"`
|
||||||
|
Display string `json:"display"`
|
||||||
|
Domain string `json:"domain"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// AppRecipe is the v1 per-app allowlist. NOTHING from AppConfig.Env is emitted — the three fields below
|
||||||
|
// are the entire surface, so no env secret (ENC: value, token, password) can ride along.
|
||||||
|
type AppRecipe struct {
|
||||||
|
CatalogRef string `json:"catalog_ref"`
|
||||||
|
Enabled bool `json:"enabled"`
|
||||||
|
StorageBindings []StorageBinding `json:"storage_bindings"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// StorageBinding names WHERE an app's data lives on a user-data drive: the container mount target, the
|
||||||
|
// drive name, and the path under the drive. Identifiers/paths only — e.g. romm.library → felhom-flash
|
||||||
|
// : userdata/roms. The hub correlates `drive` to the agent half's durable-id.
|
||||||
|
type StorageBinding struct {
|
||||||
|
ContainerPath string `json:"container_path"`
|
||||||
|
Drive string `json:"drive"`
|
||||||
|
Subpath string `json:"subpath"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// BuildDRRecipeAppHalf assembles the controller half: customer identity + an AppRecipe per DEPLOYED,
|
||||||
|
// non-protected stack. composeReader returns a stack's docker-compose.yml content (seam: tests inject a
|
||||||
|
// fake; production passes os.ReadFile-backed). Pure given the reader → unit-tested directly.
|
||||||
|
func BuildDRRecipeAppHalf(custID, custName, domain string, all []stacks.Stack, composeReader func(path string) string) *DRRecipeAppHalf {
|
||||||
|
half := &DRRecipeAppHalf{
|
||||||
|
RecipeVersion: DRRecipeVersion,
|
||||||
|
Customer: DRCustomer{ID: custID, Display: custName, Domain: domain},
|
||||||
|
Apps: []AppRecipe{},
|
||||||
|
}
|
||||||
|
for _, s := range all {
|
||||||
|
if s.Protected || !s.Deployed {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
var composeYAML string
|
||||||
|
if composeReader != nil && s.ComposePath != "" {
|
||||||
|
composeYAML = composeReader(s.ComposePath)
|
||||||
|
}
|
||||||
|
half.Apps = append(half.Apps, buildAppRecipe(s, composeYAML))
|
||||||
|
}
|
||||||
|
return half
|
||||||
|
}
|
||||||
|
|
||||||
|
// buildAppRecipe constructs ONE app's recipe from the allowlist ONLY. This function deliberately reads
|
||||||
|
// NOTHING from s.AppConfig.Env — that is the boundary. catalog_ref = the catalog slug (falls back to the
|
||||||
|
// stack name for an orphaned app); storage_bindings parsed from the compose against the app's HDD_PATH.
|
||||||
|
func buildAppRecipe(s stacks.Stack, composeYAML string) AppRecipe {
|
||||||
|
catalogRef := s.Meta.Slug
|
||||||
|
if catalogRef == "" {
|
||||||
|
catalogRef = s.Name
|
||||||
|
}
|
||||||
|
var hddPath string
|
||||||
|
if s.AppConfig != nil {
|
||||||
|
hddPath = s.AppConfig.Env["HDD_PATH"] // a path identifier, NOT a secret (secrets are ENC: in Env)
|
||||||
|
}
|
||||||
|
return AppRecipe{
|
||||||
|
CatalogRef: catalogRef,
|
||||||
|
Enabled: s.Deployed,
|
||||||
|
StorageBindings: appStorageBindings(composeYAML, hddPath),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// appStorageBindings parses a docker-compose.yml for volume binds that land on the app's user-data drive
|
||||||
|
// (under HDD_PATH, or its USERDATA_PATH=<HDD_PATH>/userdata sibling) and returns {container_path, drive,
|
||||||
|
// subpath}. Pure (operates on the compose text), so it is unit-tested without files. The drive name is
|
||||||
|
// the basename of HDD_PATH (e.g. /mnt/felhom-drives/felhom-flash → "felhom-flash").
|
||||||
|
func appStorageBindings(composeYAML, hddPath string) []StorageBinding {
|
||||||
|
if hddPath == "" || composeYAML == "" {
|
||||||
|
return []StorageBinding{}
|
||||||
|
}
|
||||||
|
cleanHDD := filepath.ToSlash(filepath.Clean(hddPath))
|
||||||
|
userdata := cleanHDD + "/userdata"
|
||||||
|
drive := filepath.Base(cleanHDD)
|
||||||
|
|
||||||
|
bindings := []StorageBinding{}
|
||||||
|
seen := map[string]bool{}
|
||||||
|
scanner := bufio.NewScanner(strings.NewReader(composeYAML))
|
||||||
|
inVolumes := false
|
||||||
|
for scanner.Scan() {
|
||||||
|
line := strings.TrimSpace(scanner.Text())
|
||||||
|
if strings.HasPrefix(line, "volumes:") {
|
||||||
|
inVolumes = true
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if inVolumes && !strings.HasPrefix(line, "-") && !strings.HasPrefix(line, "#") && line != "" {
|
||||||
|
inVolumes = false
|
||||||
|
}
|
||||||
|
if !inVolumes || !strings.HasPrefix(line, "- ") {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
mountStr := strings.Trim(strings.TrimPrefix(line, "- "), "\"'")
|
||||||
|
parts := strings.SplitN(mountStr, ":", 3)
|
||||||
|
if len(parts) < 2 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
host := parts[0]
|
||||||
|
host = strings.ReplaceAll(host, "${USERDATA_PATH}", userdata)
|
||||||
|
host = strings.ReplaceAll(host, "${HDD_PATH}", cleanHDD)
|
||||||
|
// Also tolerate the un-braced $VAR form.
|
||||||
|
host = strings.ReplaceAll(host, "$USERDATA_PATH", userdata)
|
||||||
|
host = strings.ReplaceAll(host, "$HDD_PATH", cleanHDD)
|
||||||
|
host = filepath.ToSlash(filepath.Clean(host))
|
||||||
|
|
||||||
|
if host != cleanHDD && !strings.HasPrefix(host, cleanHDD+"/") {
|
||||||
|
continue // not on this drive
|
||||||
|
}
|
||||||
|
subpath := strings.TrimPrefix(host, cleanHDD)
|
||||||
|
subpath = strings.TrimPrefix(subpath, "/")
|
||||||
|
container := parts[1]
|
||||||
|
key := container + "\x00" + subpath
|
||||||
|
if seen[key] {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
seen[key] = true
|
||||||
|
bindings = append(bindings, StorageBinding{ContainerPath: container, Drive: drive, Subpath: subpath})
|
||||||
|
}
|
||||||
|
return bindings
|
||||||
|
}
|
||||||
@@ -0,0 +1,201 @@
|
|||||||
|
package report
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"regexp"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"gitea.dooplex.hu/admin/felhom-controller/internal/stacks"
|
||||||
|
)
|
||||||
|
|
||||||
|
// secretNameRe matches any JSON key that smells like a credential — mirrors the agent half.
|
||||||
|
var secretNameRe = regexp.MustCompile(`(?i)(password|secret|token|hash|passphrase|api[_-]?key|\bkey\b|enc:)`)
|
||||||
|
|
||||||
|
// rommCompose is a realistic catalog compose: user-data binds via ${USERDATA_PATH}/${HDD_PATH}, plus a
|
||||||
|
// secret-laden environment section (which must NEVER reach the recipe — bindings come from volumes only).
|
||||||
|
const rommCompose = `services:
|
||||||
|
romm:
|
||||||
|
image: romm:latest
|
||||||
|
environment:
|
||||||
|
- DB_PASSWORD=${DB_PASSWORD}
|
||||||
|
- IGDB_CLIENT_SECRET=${IGDB_CLIENT_SECRET}
|
||||||
|
volumes:
|
||||||
|
- ${USERDATA_PATH}/roms:/roms
|
||||||
|
- ${HDD_PATH}/appdata/romm/resources:/romm/resources
|
||||||
|
- romm_redis_data:/data
|
||||||
|
volumes:
|
||||||
|
romm_redis_data:
|
||||||
|
`
|
||||||
|
|
||||||
|
// secretLadenStack builds a romm Stack whose persisted Env carries synthetic secrets (an ENC: value and
|
||||||
|
// a token-shaped value) — exactly what the recipe must keep out.
|
||||||
|
func secretLadenStack() stacks.Stack {
|
||||||
|
return stacks.Stack{
|
||||||
|
Name: "romm",
|
||||||
|
Meta: stacks.Metadata{Slug: "romm", DisplayName: "RomM"},
|
||||||
|
ComposePath: "/stacks/romm/docker-compose.yml",
|
||||||
|
Deployed: true,
|
||||||
|
AppConfig: &stacks.AppConfig{
|
||||||
|
Deployed: true,
|
||||||
|
Env: map[string]string{
|
||||||
|
"HDD_PATH": "/mnt/felhom-drives/felhom-flash",
|
||||||
|
"DB_PASSWORD": "ENC:U2FsdGVkX1+DEADBEEFsecret==",
|
||||||
|
"IGDB_CLIENT_SECRET": "tok_live_SUPERSECRET_must_not_leak",
|
||||||
|
"SECRET_KEY": "ENC:another_encrypted_blob",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestBuildAppRecipe_NoSecrets is THE load-bearing boundary test: the emitted AppRecipe for an app whose
|
||||||
|
// deploy config contains secrets (ENC: + token-shaped) carries NONE of those values and NO
|
||||||
|
// credential-shaped key — only the {catalog_ref, enabled, storage_bindings} allowlist.
|
||||||
|
func TestBuildAppRecipe_NoSecrets(t *testing.T) {
|
||||||
|
s := secretLadenStack()
|
||||||
|
rec := buildAppRecipe(s, rommCompose)
|
||||||
|
|
||||||
|
b, err := json.Marshal(rec)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
out := string(b)
|
||||||
|
|
||||||
|
// (1) none of the secret VALUES survived.
|
||||||
|
for _, leak := range []string{"ENC:", "U2FsdGVkX1", "DEADBEEF", "tok_live_SUPERSECRET_must_not_leak", "another_encrypted_blob"} {
|
||||||
|
if strings.Contains(out, leak) {
|
||||||
|
t.Errorf("SECRET LEAK: emitted recipe contains %q\n recipe: %s", leak, out)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// (2) no credential-shaped KEY survived (DB_PASSWORD / SECRET_KEY / IGDB_CLIENT_SECRET as keys).
|
||||||
|
assertNoSecretKeys(t, b)
|
||||||
|
|
||||||
|
// (3) positive: we DID emit the allowlisted facts (not a vacuous pass).
|
||||||
|
if rec.CatalogRef != "romm" || !rec.Enabled {
|
||||||
|
t.Errorf("expected catalog_ref=romm enabled=true, got %+v", rec)
|
||||||
|
}
|
||||||
|
if len(rec.StorageBindings) != 2 {
|
||||||
|
t.Fatalf("expected 2 storage bindings (roms + resources), got %+v", rec.StorageBindings)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestBuildAppRecipe_AllowlistIsLoadBearing is the companion (red-proof of the boundary test): a NAIVE
|
||||||
|
// emitter that dumps AppConfig.Env (i.e. the allowlist guard removed) WOULD leak the token — proving the
|
||||||
|
// no-secrets assertion above is real, not vacuous. The production emitter must NOT leak it.
|
||||||
|
func TestBuildAppRecipe_AllowlistIsLoadBearing(t *testing.T) {
|
||||||
|
s := secretLadenStack()
|
||||||
|
const token = "tok_live_SUPERSECRET_must_not_leak"
|
||||||
|
|
||||||
|
// The "guard removed" shape — dumping Env alongside the app. This is what the boundary forbids.
|
||||||
|
unsafe, _ := json.Marshal(map[string]any{"catalog_ref": s.Meta.Slug, "env": s.AppConfig.Env})
|
||||||
|
if !strings.Contains(string(unsafe), token) {
|
||||||
|
t.Fatal("companion is broken: the unsafe (guard-removed) shape should contain the secret token")
|
||||||
|
}
|
||||||
|
|
||||||
|
// The REAL emitter must keep it out — same fixture, allowlist intact.
|
||||||
|
real, _ := json.Marshal(buildAppRecipe(s, rommCompose))
|
||||||
|
if strings.Contains(string(real), token) {
|
||||||
|
t.Fatalf("BOUNDARY VIOLATION: production emitter leaked the token: %s", real)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAppStorageBindings(t *testing.T) {
|
||||||
|
got := appStorageBindings(rommCompose, "/mnt/felhom-drives/felhom-flash")
|
||||||
|
want := map[string]StorageBinding{
|
||||||
|
"/roms": {ContainerPath: "/roms", Drive: "felhom-flash", Subpath: "userdata/roms"},
|
||||||
|
"/romm/resources": {ContainerPath: "/romm/resources", Drive: "felhom-flash", Subpath: "appdata/romm/resources"},
|
||||||
|
}
|
||||||
|
if len(got) != len(want) {
|
||||||
|
t.Fatalf("got %d bindings, want %d: %+v", len(got), len(want), got)
|
||||||
|
}
|
||||||
|
for _, b := range got {
|
||||||
|
w, ok := want[b.ContainerPath]
|
||||||
|
if !ok || b != w {
|
||||||
|
t.Errorf("binding %+v unexpected (want %+v)", b, w)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// The named volume (romm_redis_data) is NOT a drive bind → excluded.
|
||||||
|
for _, b := range got {
|
||||||
|
if strings.Contains(b.Subpath, "redis") {
|
||||||
|
t.Errorf("named volume leaked into bindings: %+v", b)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestAppStorageBindings_NoHDD: an app with no HDD_PATH (rootfs-only) yields no bindings, non-nil slice.
|
||||||
|
func TestAppStorageBindings_NoHDD(t *testing.T) {
|
||||||
|
got := appStorageBindings(rommCompose, "")
|
||||||
|
if got == nil || len(got) != 0 {
|
||||||
|
t.Errorf("no HDD_PATH should yield empty (non-nil) bindings, got %+v", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestBuildDRRecipeAppHalf(t *testing.T) {
|
||||||
|
reader := func(path string) string {
|
||||||
|
if path == "/stacks/romm/docker-compose.yml" {
|
||||||
|
return rommCompose
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
all := []stacks.Stack{
|
||||||
|
secretLadenStack(),
|
||||||
|
{Name: "traefik", Protected: true, Deployed: true}, // protected → excluded
|
||||||
|
{Name: "vikunja", Meta: stacks.Metadata{Slug: "vikunja"}, Deployed: false}, // not deployed → excluded
|
||||||
|
{Name: "actualbudget", Meta: stacks.Metadata{Slug: "actualbudget"}, Deployed: true}, // rootfs app, no compose path
|
||||||
|
}
|
||||||
|
half := BuildDRRecipeAppHalf("cust-demo", "Demo Customer", "demo-felhom.eu", all, reader)
|
||||||
|
|
||||||
|
if half.RecipeVersion != 1 {
|
||||||
|
t.Errorf("recipe_version=%d want 1", half.RecipeVersion)
|
||||||
|
}
|
||||||
|
if half.Customer.ID != "cust-demo" || half.Customer.Display != "Demo Customer" || half.Customer.Domain != "demo-felhom.eu" {
|
||||||
|
t.Errorf("customer = %+v", half.Customer)
|
||||||
|
}
|
||||||
|
// Only romm + actualbudget (deployed, non-protected). traefik (protected) + vikunja (not deployed) out.
|
||||||
|
if len(half.Apps) != 2 {
|
||||||
|
t.Fatalf("expected 2 apps, got %d: %+v", len(half.Apps), half.Apps)
|
||||||
|
}
|
||||||
|
byRef := map[string]AppRecipe{}
|
||||||
|
for _, a := range half.Apps {
|
||||||
|
byRef[a.CatalogRef] = a
|
||||||
|
}
|
||||||
|
if r, ok := byRef["romm"]; !ok || len(r.StorageBindings) != 2 {
|
||||||
|
t.Errorf("romm recipe wrong: %+v", r)
|
||||||
|
}
|
||||||
|
if r, ok := byRef["actualbudget"]; !ok || len(r.StorageBindings) != 0 {
|
||||||
|
t.Errorf("actualbudget (rootfs) should have 0 bindings: %+v", r)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Whole-half no-secrets sweep.
|
||||||
|
b, _ := json.Marshal(half)
|
||||||
|
assertNoSecretKeys(t, b)
|
||||||
|
if strings.Contains(string(b), "tok_live_SUPERSECRET_must_not_leak") {
|
||||||
|
t.Errorf("SECRET LEAK in assembled app-half: %s", b)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// assertNoSecretKeys walks decoded JSON and fails on any object key matching secretNameRe.
|
||||||
|
func assertNoSecretKeys(t *testing.T, jsonBytes []byte) {
|
||||||
|
t.Helper()
|
||||||
|
var v any
|
||||||
|
if err := json.Unmarshal(jsonBytes, &v); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
var walk func(prefix string, node any)
|
||||||
|
walk = func(prefix string, node any) {
|
||||||
|
switch n := node.(type) {
|
||||||
|
case map[string]any:
|
||||||
|
for k, child := range n {
|
||||||
|
if secretNameRe.MatchString(k) {
|
||||||
|
t.Errorf("secret-shaped key %q at %s — the recipe must carry no credential field", k, prefix)
|
||||||
|
}
|
||||||
|
walk(prefix+"."+k, child)
|
||||||
|
}
|
||||||
|
case []any:
|
||||||
|
for _, child := range n {
|
||||||
|
walk(prefix, child)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
walk("<root>", v)
|
||||||
|
}
|
||||||
@@ -24,6 +24,10 @@ type Report struct {
|
|||||||
Stacks StacksReport `json:"stacks"`
|
Stacks StacksReport `json:"stacks"`
|
||||||
AppTelemetry []AppTelemetry `json:"app_telemetry,omitempty"`
|
AppTelemetry []AppTelemetry `json:"app_telemetry,omitempty"`
|
||||||
GeoRestriction *GeoRestrictionReport `json:"geo_restriction,omitempty"`
|
GeoRestriction *GeoRestrictionReport `json:"geo_restriction,omitempty"`
|
||||||
|
|
||||||
|
// DR recipe — the controller (customer + apps) half of the secret-free reconstruction recipe
|
||||||
|
// (SPIKE-dr-recipe-2026-06-16). The hub assembles it with the agent's storage/guest/PBS half.
|
||||||
|
DRRecipe *DRRecipeAppHalf `json:"dr_recipe,omitempty"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// SystemReport holds host-level system info.
|
// SystemReport holds host-level system info.
|
||||||
|
|||||||
Reference in New Issue
Block a user