Files
admin f94543ee5c
gates / gates (push) Successful in 10s
v0.217.0: prefill from the app's own backup, where-the-data-goes on deploy, bounded inventory fan-out
Completes R-351 and ships R-352's visibility half. Gates 11/11 OK, suite 28 packages ok,
go vet clean, -race clean on the changed package - all run and read BEFORE this commit.

PART 2 SCENARIO A - the deploy page prefills the address and data folder from the app's OWN
backup. backup.RecordedUnitForStack scans every readable namespace root (the app is NOT
installed in this case, so there is no own drive to ask) and reads manifest.json plus the
captured compose/app.yaml. Local file reads only: no network, no restic, no restore.
RecordedAddress.Known() requires BOTH halves on purpose - an absent SUBDOMAIN makes the live
deploy path substitute the CATALOG default (stacks/deploy.go:88-90), and offering that back as
"what your backup says" would be a fabricated fact. The prefill is labelled as coming from the
backup and stays editable: a memory, not a lock.

PART 1 VISIBILITY (R-352) - the deploy page now states where the app's data will live before
the button is pressed. Measured 2026-08-21: 13 of 53 catalogue templates declare a storage
field; the other 40 have none and their data goes to the system drive, which no screen said.
Metadata.HasDeployField answers "does this app have somewhere to PUT a recorded value?" - for
the 40-class a recorded placement is a fact to state, never a value written into a field that
does not exist. NO PLACEMENT CHANGED. NOTHING MIGRATED. The rest is a filed specification.

PART 4 - measured before theorising, on the live off-site target:
  snapshots --json 2605 ms once; stats 2697 ms PER APP, sequential, 5 app tags
  => 2605 + 5*2697 = ~16.1 s, matching the reported ten-to-fifteen seconds.
The cause is the shape already on file, so the per-app size calls now run concurrently,
BOUNDED TO 4. The bound is the safety property, not the speed one: the repository is a Hetzner
Storage Box with a session cap, and a refused size call returns SizeBytes 0 - a silent
UNDER-REPORT of the customer's data rather than a visible failure. Peak-in-flight is asserted.
OffsiteInventoryList had no test at all before this.

TEMPLATE SAFETY - every Restore* key is set UNCONDITIONALLY in the deploy handler, because a
template doing index/eq against an undefined key errors at RENDER time: green build, green vet,
green suite, 500 on the page. Four render tests, one per branch, because the existing deploy
render test only renders AutoFields and never reaches these blocks.

RED-PROOFS, mutation asserted applied then reverted to 0:
  A   three template guards dropped (count asserted 3) -> the blank form returned
  P4  inventorySizeConcurrency = 1 -> "peak in flight was 1", elapsed 282ms = sequential

DOCS: CHANGELOG v0.217.0 (MinAgent 0.129.0 unchanged), CONTEXT (the restore's own memory +
what is next), controller/README.md (Backup System), REUSE.md (4 new rows), REPORT.md
overwritten - the previous REPORT preserved to audits/REPORT-v0.216.0-2026-08-14.md first.

NOT fixed here, filed as R-353 and named the next session's first item: a restore whose unit
carries no db_dumps and no volume_dumps still reports a bare completion.
2026-08-21 21:29:01 +02:00

260 lines
12 KiB
Go

package backup
import (
"fmt"
"io/fs"
"os"
"path/filepath"
"strings"
"gopkg.in/yaml.v3"
)
// R-351 — THE RESTORE ALREADY KNOWS WHERE THE APP LIVED. IT JUST NEVER LOOKED.
//
// Every recovery unit's manifest.json carries `drive` and `namespace_root` (recovery_unit.go:48-49),
// written at capture time from the app's own live placement. Measured on demo-hp 2026-08-21:
//
// opengist drive=/mnt/sys_drive nsroot=/mnt/sys_drive/felhom-data
// calibre-web drive=/mnt/felhom-drives/hdd_1 nsroot=/mnt/felhom-drives/hdd_1
//
// Before this file, NO non-test code in the repository read either field back. `grep -rE
// '\.Drive\b|\.NamespaceRoot\b' --include=*.go` returned only the appbackup.NamespaceRoot FUNCTION
// and Tier2Target's unrelated field. The reconstitution opened the manifest
// (offbox_reconstitute.go:235) and took only the coherence stamp from it, then resolved its
// destination from the LIVE app instead. One side wrote the fact; the other never received it —
// the same shape as several defects closed this month.
//
// THE CONSEQUENCE THAT MADE THIS WORTH FIXING: a restore into a destination that differs from the
// one the backup recorded succeeded SILENTLY, under a green message. Nothing compared them.
//
// WHAT THIS IS NOT. It is not a lock. A domain can legitimately change and hardware can move, so a
// difference is NAMED and the customer decides — it is their own previous answer being shown back to
// them, not a rule imposed on them. What must never happen is the difference passing unremarked.
// RecordedPlacement is what a backup says about where an app's data lived when it was captured.
// Empty fields mean the manifest did not record them — an older unit, or one written before the
// field existed. That is an UNKNOWN and is never rendered as a value.
type RecordedPlacement struct {
Drive string // manifest.Drive — the in-guest mount (HDD_PATH), or the system data path
NamespaceRoot string // manifest.NamespaceRoot — the resolved felhom-data namespace root
}
// Known reports whether the backup recorded a destination at all. A unit whose manifest predates the
// field, or could not be read, is NOT known — and "we cannot tell" is a different answer from "they
// match", which is why this is a method rather than a `!= ""` scattered over the callers.
func (p RecordedPlacement) Known() bool { return strings.TrimSpace(p.Drive) != "" }
// PlacementCheck is the verdict of comparing what the backup recorded against where the restore is
// actually about to write.
type PlacementCheck struct {
// Recorded is what the backup said. Zero value when the manifest carried nothing.
Recorded RecordedPlacement
// LiveDrive / LiveNamespaceRoot are where this restore will write, resolved the way the
// reconstitution resolves it today.
LiveDrive string
LiveNamespaceRoot string
// Known mirrors Recorded.Known(), carried on the verdict so a template never has to re-derive it.
Known bool
// Mismatch is true ONLY when the backup recorded a destination AND it differs from the live one.
// An unknown recording is never a mismatch: refusing on an absence would block every pre-field
// unit, and asserting a match we cannot see would be worse.
Mismatch bool
}
// CheckPlacement compares a manifest's recorded placement against the live destination.
//
// Deliberately PURE and total: a nil manifest, an empty manifest and a manifest whose drive is blank
// all produce the same honest "not known, no mismatch" verdict. Paths are compared Cleaned, because
// `/mnt/x` and `/mnt/x/` are the same destination and a trailing slash must not manufacture a
// mismatch the customer then has to dismiss. Comparison is on the DRIVE, not the namespace root: the
// root is derived from the drive (namespaceRoot appends felhom-data only on the system-data
// fallback), so comparing both would report one difference twice.
func CheckPlacement(man *RecoveryManifest, liveDrive, liveNamespaceRoot string) PlacementCheck {
c := PlacementCheck{
LiveDrive: strings.TrimSpace(liveDrive),
LiveNamespaceRoot: strings.TrimSpace(liveNamespaceRoot),
}
if man != nil {
c.Recorded = RecordedPlacement{
Drive: strings.TrimSpace(man.Drive),
NamespaceRoot: strings.TrimSpace(man.NamespaceRoot),
}
}
c.Known = c.Recorded.Known()
if !c.Known || c.LiveDrive == "" {
return c
}
c.Mismatch = filepath.Clean(c.Recorded.Drive) != filepath.Clean(c.LiveDrive)
return c
}
// RecordedAddress is the web address the backup recorded for an app, read from the app.yaml the
// recovery unit captured beside its manifest.
//
// RECORDED, NEVER INFERRED. Both halves must come from the captured file. When the captured app.yaml
// carries no SUBDOMAIN, the LIVE deploy path falls back to the catalog's default
// (stacks/deploy.go:88-90) — that default is a catalog guess, not the customer's answer, and
// presenting it as "what your backup says" would be a fabricated fact. This project has ruled twice
// that a guess dressed as a fact is how these bugs are built, so an absent SUBDOMAIN is UNKNOWN here.
type RecordedAddress struct {
Subdomain string
Domain string
}
// Known reports whether BOTH halves were recorded. A half-known address is not an address: showing
// „gist." or „.enkisfelhom.hu" as a prefill is worse than showing nothing.
func (a RecordedAddress) Known() bool {
return strings.TrimSpace(a.Subdomain) != "" && strings.TrimSpace(a.Domain) != ""
}
// FQDN is the address as the customer knows it, or "" when it is not fully known.
func (a RecordedAddress) FQDN() string {
if !a.Known() {
return ""
}
return strings.TrimSpace(a.Subdomain) + "." + strings.TrimSpace(a.Domain)
}
// unitAppConfig is the slice of the captured app.yaml this package needs. Deliberately a LOCAL
// minimal struct rather than stacks.AppConfig: internal/stacks imports nothing from here today and
// reaching across for one map would couple the backup layer to the deploy layer's schema for no
// gain. Unknown keys are ignored by yaml.v3, so a richer app.yaml still parses.
type unitAppConfig struct {
Env map[string]string `yaml:"env"`
}
// RecordedUnitForStack reads what an app's most readable recovery unit says about where it lived and
// what address it answered on. Returns ok=false when no unit can be read at all.
//
// SEARCH ORDER, and why it is not just "the app's own drive": the case this exists for is an app
// that is NOT INSTALLED on a rebuilt box, so GetStackHDDPath returns "" and there is no own drive to
// consult. It therefore checks every namespace root the box can currently see — the registered
// storage paths and the system data path — and takes the first unit it can read. That is a handful
// of stat calls on local disk: no network, no restic, no restore.
//
// A drive that is NOT attached contributes nothing, which is the honest outcome: we cannot read a
// unit that is not here, and the reconstitution's own refusal owns that case with a route.
func (m *Manager) RecordedUnitForStack(stack string) (RecordedPlacement, RecordedAddress, bool) {
if !isSafeStackName(stack) {
return RecordedPlacement{}, RecordedAddress{}, false
}
seen := map[string]bool{}
var roots []string
addRoot := func(drive string) {
drive = strings.TrimSpace(drive)
if drive == "" {
return
}
r := m.namespaceRoot(drive)
if r != "" && !seen[r] {
seen[r] = true
roots = append(roots, r)
}
}
// The app's own drive first when it HAS one — that unit is the authoritative one for an
// installed app, and checking it first keeps the common case to a single stat.
if m.stackProvider != nil {
addRoot(m.stackProvider.GetStackHDDPath(stack))
}
if m.settings != nil {
for _, sp := range m.settings.GetStoragePaths() {
addRoot(sp.Path)
}
}
addRoot(m.systemDataPath)
for _, root := range roots {
unit := RecoveryUnitPath(root, stack)
man := readManifest(filepath.Join(unit, "manifest.json"))
if man == nil {
continue
}
place := RecordedPlacement{
Drive: strings.TrimSpace(man.Drive),
NamespaceRoot: strings.TrimSpace(man.NamespaceRoot),
}
addr := readRecordedAddress(filepath.Join(unit, "compose", "app.yaml"))
if !place.Known() && !addr.Known() {
continue // a unit that tells us nothing is not an answer
}
return place, addr, true
}
return RecordedPlacement{}, RecordedAddress{}, false
}
// readRecordedAddress parses SUBDOMAIN/DOMAIN out of a captured app.yaml. Returns the zero value on
// any failure — absent file, unreadable, malformed, or either half missing.
func readRecordedAddress(appYAMLPath string) RecordedAddress {
raw, err := os.ReadFile(appYAMLPath)
if err != nil {
return RecordedAddress{}
}
var cfg unitAppConfig
if yaml.Unmarshal(raw, &cfg) != nil || cfg.Env == nil {
return RecordedAddress{}
}
return RecordedAddress{
Subdomain: strings.TrimSpace(cfg.Env["SUBDOMAIN"]),
Domain: strings.TrimSpace(cfg.Env["DOMAIN"]),
}
}
// scratchManifestMaxDepth bounds the walk below. The unit sits a handful of levels under the scratch
// root (the restore mirrors the snapshot's absolute path), and an unbounded walk over a scratch that
// also holds a full userdata tree would stat a customer's entire library to find one small file.
const scratchManifestMaxDepth = 8
// recordedPlacementFromScratch reads the placement out of the unit manifest inside a PREPARED
// restore scratch, without restoring anything.
//
// It exists for the not-installed refusal (scenario B), which fires BEFORE the live namespace root
// can be resolved — so it cannot use the manifest the reconstitution opens later. The scratch is
// already on local disk by then; this is a bounded walk and a file read, never a network call.
//
// Returns the zero value on ANY failure — unreadable, absent, malformed. A refusal that cannot name
// the recorded place must fall back to the plain sentence rather than print an empty path, which
// would read as "the backup says it lived nowhere".
func (m *Manager) recordedPlacementFromScratch(scratch string) RecordedPlacement {
var found RecordedPlacement
root := filepath.Clean(scratch)
rootDepth := strings.Count(root, string(os.PathSeparator))
_ = filepath.WalkDir(root, func(path string, d fs.DirEntry, err error) error {
if err != nil {
return nil // an unreadable subtree is not fatal: keep looking elsewhere
}
if d.IsDir() {
if strings.Count(filepath.Clean(path), string(os.PathSeparator))-rootDepth >= scratchManifestMaxDepth {
return fs.SkipDir
}
return nil
}
if d.Name() != "manifest.json" {
return nil
}
if man := readManifest(path); man != nil && strings.TrimSpace(man.Drive) != "" {
found = RecordedPlacement{
Drive: strings.TrimSpace(man.Drive),
NamespaceRoot: strings.TrimSpace(man.NamespaceRoot),
}
return fs.SkipAll
}
return nil
})
return found
}
// PlacementMismatchMessage is the Hungarian refusal shown when the destination differs from the one
// the backup recorded. It NAMES BOTH VALUES — which is the whole point: "a destination differs" that
// does not say from what leaves the customer with a decision they cannot make.
//
// It is a refusal with a route, not a dead end: the caller re-offers the action with the
// acknowledgement field set, so the customer can proceed deliberately (scenario C).
func PlacementMismatchMessage(stack string, c PlacementCheck) string {
return fmt.Sprintf(
"A(z) %s mentése szerint az adatok korábban itt voltak: %s. Most viszont ide állna vissza: %s. "+
"Ez lehet szándékos — például ha meghajtót cseréltél —, de magától nem folytatjuk. "+
"Ha így jó, erősítsd meg alább, és a visszaállítás az új helyre fut.",
stack, c.Recorded.Drive, c.LiveDrive)
}