v0.235.0: freeze the version, keep the fixes flowing (operator ruling 2026-09-06)

Slice 3. R-447 was BLOCKED because R-438 established that RestartStack's use of up -d to pick up template changes was CHOSEN and written down in its own comment. The operator ruled Option 1, and this implements it. The rule: while the catalog offers the same version you run, its fixes flow to you; the moment it moves to a newer version you are frozen until you update. NOTHING was added to any of the thirteen compose up -d call sites. Most of them are repairs - the boot reconciler, the drive-return gate, the app-stop guard - and a repair path that refuses to repair leaves a customer's app down, which is worse than the problem. They are made safe by removing the reason. app.yaml gains pinned_images: what the app is SUPPOSED to run. It is NOT installed_images, which is an observation; letting a reading become a deployment is the R-166 category error one field over. Four writers, each also storing the exact definition as applied-compose.yml. UpdateStack advances the pin and re-renders BEFORE the pull, because pull and up -d act on the file on disk, and a pin set afterwards would pull the frozen version and report success. The syncer renders instead of copying, through one nil-safe seam. Catalog images equal the pin -> verbatim, so fixes and self-healing both survive; they differ -> the WHOLE stored definition, never a substitution of refs into a newer template (wger 2.6 needs a DB config the older template cannot supply). This is deliberately not 'skip deployed apps', which was option B and was rejected. AdoptPins runs once at boot after the backfill, files only, and skips loudly rather than inventing a pin. syncer.Start() moved to after it: the initial sync would otherwise run while every app was unpinned and overwrite a deployed app's version once per boot. THE BADGE HAD TO CHANGE OR SLICE 2 WOULD HAVE INVERTED SILENTLY. TemplateImages reads the LIVE compose file, which is now the frozen one, so the comparison would have answered Naprakesz on exactly the apps that are behind - with every test green, because the new field has the same type. It now reads CatalogImages. +16 tests (1729 -> 1745), 28 packages green. Three red-proofs run and reverted. A test also caught the syncer writing an empty compose file over a live app.
2026-09-06 09:45:34 +02:00
parent 998aa31958
commit 8a0e0a59ad
13 changed files with 1437 additions and 17 deletions
@@ -0,0 +1,358 @@
+package stacks
+
+import (
+	"fmt"
+	"os"
+	"path/filepath"
+	"sort"
+	"strings"
+)
+
+// AppliedComposeFile is the stored copy of the docker-compose.yml an app was last brought up FROM.
+//
+// WHY IT LIVES IN THE STACK DIRECTORY, and why this exact name: `Syncer.copyTemplates` copies
+// EXACTLY `docker-compose.yml` and `.felhom.yml` and nothing else, so any other name in that
+// directory is safe from the catalog. Keeping it beside the app means it travels through every path
+// that already moves a stack dir, and anyone debugging the box can see it without a tool.
+//
+// It is the FROZEN definition: when the catalog has moved past an app's pin, this whole file is
+// written to docker-compose.yml. Never a substitution of pinned refs into a newer template — a
+// template's env and volumes can belong to its version (`wger 2.6` needs a full DB config the older
+// template cannot supply), and an old image under a new template is a third broken state nobody
+// chose.
+const AppliedComposeFile = "applied-compose.yml"
+
+// AppliedComposePath is the single definition of where the stored definition lives.
+func AppliedComposePath(stackDir string) string {
+	return filepath.Join(stackDir, AppliedComposeFile)
+}
+
+// StoreAppliedDefinition writes the compose definition this app is pinned to, atomically.
+//
+// Atomic tmp+rename for the same reason SaveAppConfig is: the syncer may read this file at any
+// moment, and a half-written compose file rendered over a live app is a broken app.
+func StoreAppliedDefinition(stackDir string, data []byte) error {
+	if len(data) == 0 {
+		// NEVER store an empty definition. An empty applied file would later be rendered over the
+		// live compose file and take the app down — see the render table's "unreadable or empty"
+		// row, which treats it as absent precisely so this cannot happen.
+		return fmt.Errorf("refusing to store an empty applied definition for %s", filepath.Base(stackDir))
+	}
+	path := AppliedComposePath(stackDir)
+	tmp := path + ".tmp"
+	if err := os.WriteFile(tmp, data, 0o644); err != nil {
+		return fmt.Errorf("writing %s: %w", tmp, err)
+	}
+	if err := os.Rename(tmp, path); err != nil {
+		_ = os.Remove(tmp)
+		return fmt.Errorf("renaming %s to %s: %w", tmp, path, err)
+	}
+	return nil
+}
+
+// LoadAppliedDefinition returns the stored definition, or an error when there is none or it is
+// unusable. An empty file is an ERROR, not empty content — see the render table.
+func LoadAppliedDefinition(stackDir string) ([]byte, error) {
+	data, err := os.ReadFile(AppliedComposePath(stackDir))
+	if err != nil {
+		return nil, err
+	}
+	if len(strings.TrimSpace(string(data))) == 0 {
+		return nil, fmt.Errorf("stored applied definition for %s is empty", filepath.Base(stackDir))
+	}
+	return data, nil
+}
+
+// SetPin records what an app is SUPPOSED to run and stores the definition that pin came from.
+//
+// PIN FIRST, THEN STORE, and the degradation is deliberate: if the pin lands and the store fails,
+// the app is pinned with no stored definition, and the render table's own row for that case copies
+// the catalog verbatim and WARNs. That is today's behaviour plus a loud line — the same outcome as
+// being unpinned, never a silent freeze onto a definition we do not have.
+//
+// `composeSrc` MUST be the exact bytes the pin was derived from. Passing a different file is how a
+// stack ends up frozen onto something it never ran.
+func (m *Manager) SetPin(name, stackDir string, pin map[string]string, composeSrc []byte) error {
+	if len(pin) == 0 {
+		return fmt.Errorf("refusing to pin %s to an empty image set", name)
+	}
+	cfg := LoadAppConfig(stackDir)
+	if cfg == nil {
+		// No app.yaml means nothing is deployed here. Not an error — the same no-op
+		// SetDesiredState makes for the same reason.
+		m.logger.Printf("[DEBUG] [stacks] pin %s: no app.yaml — nothing deployed here, nothing to pin", name)
+		return nil
+	}
+
+	if !samePin(cfg.PinnedImages, pin) {
+		cfg.PinnedImages = pin
+		meta := LoadMetadata(stackDir)
+		if err := SaveAppConfig(stackDir, cfg, m.encKey, SensitiveEnvVars(&meta)); err != nil {
+			return fmt.Errorf("recording pin for %s: %w", name, err)
+		}
+		m.logger.Printf("[INFO] [stacks] pin %s: %s", name, summarisePin(pin))
+
+		m.mu.Lock()
+		if st, ok := m.stacks[name]; ok && st.AppConfig != nil {
+			st.AppConfig.PinnedImages = pin
+		}
+		m.mu.Unlock()
+	}
+
+	if err := StoreAppliedDefinition(stackDir, composeSrc); err != nil {
+		// Loud, and NOT fatal to the pin — see the comment above.
+		m.logger.Printf("[ERROR] [stacks] pin %s: the pin is recorded but its definition could not be stored (the app is unaffected; the catalog will be copied verbatim until this is fixed): %v", name, err)
+	}
+	return nil
+}
+
+// samePin compares two pins by content so an unchanged pin does not rewrite app.yaml — that file
+// holds encrypted secrets and rewriting it for no new information is pure risk (the SetDesiredState
+// rule).
+func samePin(a, b map[string]string) bool {
+	if len(a) != len(b) {
+		return false
+	}
+	for k, va := range a {
+		if vb, ok := b[k]; !ok || va != vb {
+			return false
+		}
+	}
+	return true
+}
+
+// summarisePin renders a pin for ONE log line. Image refs only — this file never logs anything out
+// of app.yaml's env map, which holds encrypted secrets.
+func summarisePin(pin map[string]string) string {
+	svcs := make([]string, 0, len(pin))
+	for svc := range pin {
+		svcs = append(svcs, svc)
+	}
+	sort.Strings(svcs)
+	parts := make([]string, 0, len(svcs))
+	for _, svc := range svcs {
+		parts = append(parts, svc+"="+pin[svc])
+	}
+	return strings.Join(parts, ", ")
+}
+
+// ComposePathIn resolves a stack directory's compose file, honouring the .yml/.yaml pair exactly as
+// ScanStacks does. One rule, so a caller cannot pin from a file the scanner would not have read.
+func ComposePathIn(stackDir string) string {
+	p := filepath.Join(stackDir, "docker-compose.yml")
+	if _, err := os.Stat(p); err == nil {
+		return p
+	}
+	alt := filepath.Join(stackDir, "docker-compose.yaml")
+	if _, err := os.Stat(alt); err == nil {
+		return alt
+	}
+	return p // the canonical name; the caller's read will report the real error
+}
+
+// PinFromCompose reads a compose file and returns both the pin it implies and its exact bytes, so a
+// caller cannot accidentally pin from one file and store another.
+func PinFromCompose(composePath string) (map[string]string, []byte, error) {
+	images, err := ParseComposeImages(composePath)
+	if err != nil {
+		return nil, nil, err
+	}
+	if len(images) == 0 {
+		return nil, nil, fmt.Errorf("%s declares no images", composePath)
+	}
+	data, err := os.ReadFile(composePath)
+	if err != nil {
+		return nil, nil, err
+	}
+	return images, data, nil
+}
+
+// --- what the syncer is told ---
+
+// RenderPlan is the per-app answer the stack manager gives the catalog syncer.
+//
+// It carries FACTS, not a decision: the render table lives in the syncer, which is the thing doing
+// the writing. The syncer must never read app.yaml itself — that file is the manager's and carries
+// encrypted values — so everything it needs to apply the table comes through here.
+type RenderPlan struct {
+	Deployed    bool
+	Deploying   bool
+	Protected   bool
+	Pinned      map[string]string // service -> image ref; nil/empty means UNPINNED
+	AppliedPath string            // the stored definition; "" when none is stored
+}
+
+// RenderPlanFor answers for one app by name. Unknown apps come back as an empty plan, which the
+// syncer reads as "not deployed" — i.e. copy verbatim, exactly the pre-v0.235.0 behaviour.
+func (m *Manager) RenderPlanFor(name string) RenderPlan {
+	s, ok := m.GetStack(name)
+	if !ok {
+		return RenderPlan{}
+	}
+	plan := RenderPlan{
+		Deployed:  s.Deployed,
+		Deploying: s.Deploying,
+		Protected: s.Protected,
+	}
+	if s.AppConfig != nil && len(s.AppConfig.PinnedImages) > 0 {
+		plan.Pinned = s.AppConfig.PinnedImages
+	}
+	stackDir := filepath.Dir(s.ComposePath)
+	if _, err := LoadAppliedDefinition(stackDir); err == nil {
+		plan.AppliedPath = AppliedComposePath(stackDir)
+	}
+	return plan
+}
+
+// --- adoption ---
+
+// AdoptPins gives a pin to every deployed app that has none, and stores the definition it is
+// running. Call ONCE at startup, immediately AFTER BackfillInstalledImages so an app the backfill
+// has just observed can be pinned in the same boot.
+//
+// ── IT NEVER GUESSES, AND THAT IS MOST OF THE CODE ───────────────────────────────────────────
+//
+// Two skips, each with a reason that is not interchangeable:
+//
+//  1. The observation is INCOMPLETE (stopped, crash-looping, mid-anything). We do not know what the
+//     app runs, so we cannot say what it should run. Reuses observationCoversTemplate — the SAME
+//     completeness rule the backfill uses, deliberately not a second one.
+//  2. The observation is complete but DIFFERS from the current template. The app is already running
+//     something the catalog no longer offers, and we have no stored definition for it. Pinning here
+//     would be right, but the FREEZE would then render a definition we do not possess — and the only
+//     way to manufacture one is to substitute the running refs into the newer template, which is
+//     exactly the third-broken-state this design refuses (see AppliedComposeFile).
+//
+// In both cases the app keeps behaving exactly as it did before v0.235.0, loudly. Absent means
+// unknown; unknown is never resolved by inventing an answer.
+//
+// It reads and writes FILES only. It starts, stops and touches no container.
+func (m *Manager) AdoptPins() int {
+	pinned, alreadyPinned, skippedIncomplete, skippedMismatch := 0, 0, 0, 0
+
+	for _, s := range m.GetStacks() {
+		if !s.Deployed || s.Protected || s.Deploying {
+			continue
+		}
+		if s.AppConfig != nil && len(s.AppConfig.PinnedImages) > 0 {
+			alreadyPinned++
+			continue
+		}
+		stackDir := filepath.Dir(s.ComposePath)
+
+		tpl, err := ParseComposeImages(s.ComposePath)
+		if err != nil || len(tpl) == 0 {
+			m.logger.Printf("[WARN] [stacks] pin adoption: %s left UNPINNED — its compose file declares no readable images (%v). It keeps pre-v0.235.0 behaviour.", s.Name, err)
+			skippedIncomplete++
+			continue
+		}
+
+		var observed map[string]InstalledImage
+		if s.AppConfig != nil {
+			observed = s.AppConfig.InstalledImages
+		}
+		if !observationCoversTemplate(observed, tpl) {
+			m.logger.Printf("[WARN] [stacks] pin adoption: %s left UNPINNED — no complete record of what it is running (%d of %d service(s) observed). It keeps pre-v0.235.0 behaviour.",
+				s.Name, len(observed), len(tpl))
+			skippedIncomplete++
+			continue
+		}
+
+		if diff := pinMismatch(observed, tpl); diff != "" {
+			m.logger.Printf("[WARN] [stacks] pin adoption: %s left UNPINNED — it is running something the current template no longer offers, and there is no stored definition for it: %s. It keeps pre-v0.235.0 behaviour.",
+				s.Name, diff)
+			skippedMismatch++
+			continue
+		}
+
+		_, data, err := PinFromCompose(s.ComposePath)
+		if err != nil {
+			m.logger.Printf("[WARN] [stacks] pin adoption: %s: %v", s.Name, err)
+			skippedIncomplete++
+			continue
+		}
+		if err := m.SetPin(s.Name, stackDir, tpl, data); err != nil {
+			m.logger.Printf("[ERROR] [stacks] pin adoption: %s: %v", s.Name, err)
+			continue
+		}
+		pinned++
+	}
+
+	// A POSITIVE OBSERVABLE EITHER WAY (standing rule 3): "0 pinned" and "the pass never ran" must
+	// not look the same in a log.
+	m.logger.Printf("[INFO] [stacks] pin adoption: %d pinned, %d already pinned, %d left unpinned (%d not completely observed, %d running something the template no longer offers)",
+		pinned, alreadyPinned, skippedIncomplete+skippedMismatch, skippedIncomplete, skippedMismatch)
+	return pinned
+}
+
+// pinMismatch returns a human description of the first service whose RUNNING reference differs from
+// what the template pins, or "" when they agree everywhere. Deterministic order so two runs produce
+// the same line.
+func pinMismatch(observed map[string]InstalledImage, tpl map[string]string) string {
+	svcs := make([]string, 0, len(tpl))
+	for svc := range tpl {
+		svcs = append(svcs, svc)
+	}
+	sort.Strings(svcs)
+	var diffs []string
+	for _, svc := range svcs {
+		got, ok := observed[svc]
+		if !ok {
+			continue // completeness was already checked
+		}
+		if got.Ref != tpl[svc] {
+			diffs = append(diffs, fmt.Sprintf("%s runs %s, template offers %s", svc, got.Ref, tpl[svc]))
+		}
+	}
+	return strings.Join(diffs, "; ")
+}
+
+// CatalogTemplatePath is where the syncer's git clone keeps one app's template. It is the ONLY
+// definition of that layout outside the syncer, and the badge and the update path both use it.
+func (m *Manager) CatalogTemplatePath(appName, filename string) string {
+	return filepath.Join(m.cfg.Paths.DataDir, "catalog-cache", "templates", appName, filename)
+}
+
+// advancePinToCatalog moves a PINNED app onto the catalog's current definition: it writes the
+// catalog template over the live compose file, records the new pin, and stores that definition as
+// the applied one. Called by UpdateStack BEFORE the pull — see the comment at that call site.
+//
+// AN UNPINNED APP IS LEFT ALONE AND THIS RETURNS NIL. It behaves exactly as it did before v0.235.0:
+// the syncer has already copied the catalog verbatim into its stack dir, so `pull` + `up -d` do
+// today's job with no help from here.
+func (m *Manager) advancePinToCatalog(name, stackDir string) error {
+	cfg := LoadAppConfig(stackDir)
+	if cfg == nil || len(cfg.PinnedImages) == 0 {
+		return nil // unpinned — today's behaviour, unchanged
+	}
+
+	src := m.CatalogTemplatePath(name, "docker-compose.yml")
+	pin, data, err := PinFromCompose(src)
+	if err != nil {
+		// REFUSE rather than silently update to the frozen definition (which would be a no-op
+		// reported as success). Names the cause so the operator is not left guessing.
+		return fmt.Errorf("cannot read the catalog's current definition for %s (%s): %w", name, src, err)
+	}
+
+	live := ComposePathIn(stackDir)
+	if err := StoreAppliedDefinition(stackDir, data); err != nil {
+		return fmt.Errorf("storing the new applied definition for %s: %w", name, err)
+	}
+	if err := os.WriteFile(live, data, 0o644); err != nil {
+		return fmt.Errorf("rendering the catalog definition for %s: %w", name, err)
+	}
+
+	cfg.PinnedImages = pin
+	meta := LoadMetadata(stackDir)
+	if err := SaveAppConfig(stackDir, cfg, m.encKey, SensitiveEnvVars(&meta)); err != nil {
+		return fmt.Errorf("recording the advanced pin for %s: %w", name, err)
+	}
+	m.mu.Lock()
+	if st, ok := m.stacks[name]; ok && st.AppConfig != nil {
+		st.AppConfig.PinnedImages = pin
+	}
+	m.mu.Unlock()
+
+	m.logger.Printf("[INFO] [stacks] update %s: pin advanced to the catalog's current definition (%s)", name, summarisePin(pin))
+	return nil
+}