From 504eae018b24d0b91955bcbebe79345a3b13bfb5 Mon Sep 17 00:00:00 2001 From: kisfenyo Date: Thu, 24 Sep 2026 16:49:43 +0200 Subject: [PATCH] v0.270.0: no update for a current app (R-679); an interrupted install is reported (R-681); a restore brings back the pinned version's health check (R-669); R-674 Five red-proofs. MinAgent 0.131.0 unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS --- CHANGELOG.md | 23 + controller/README.md | 6 + controller/cmd/controller/main.go | 7 + .../internal/backup/r669_applied_meta_test.go | 57 ++ controller/internal/backup/recovery_unit.go | 13 +- controller/internal/i18n/locales/en.json | 2 + controller/internal/i18n/locales/hu.json | 2 + controller/internal/stacks/deploy.go | 14 + .../internal/stacks/install_interrupted.go | 100 +++ controller/internal/stacks/ladder.go | 7 + controller/internal/stacks/manager.go | 34 +- .../internal/stacks/r669_restore_meta_test.go | 33 + controller/internal/stacks/r674_head_test.go | 30 + .../stacks/r679_already_current_test.go | 48 ++ .../stacks/r681_install_interrupted_test.go | 69 ++ controller/internal/stacks/undo.go | 12 + controller/internal/stacks/update.go | 14 + .../internal/stacks/updateorder_test.go | 1 - controller/internal/web/i18n_parity_test.go | 13 + .../web/r681_install_interrupted_page_test.go | 33 + controller/internal/web/templates/stacks.html | 2 +- .../stacks_install_interrupted.html | 616 ++++++++++++++++++ controller/scripts/i18n_go_keys.json | 1 + 23 files changed, 1121 insertions(+), 16 deletions(-) create mode 100644 controller/internal/backup/r669_applied_meta_test.go create mode 100644 controller/internal/stacks/install_interrupted.go create mode 100644 controller/internal/stacks/r669_restore_meta_test.go create mode 100644 controller/internal/stacks/r674_head_test.go create mode 100644 controller/internal/stacks/r679_already_current_test.go create mode 100644 controller/internal/stacks/r681_install_interrupted_test.go create mode 100644 controller/internal/web/r681_install_interrupted_page_test.go create mode 100644 controller/internal/web/testdata/i18n_parity/stacks_install_interrupted.html diff --git a/CHANGELOG.md b/CHANGELOG.md index 15be160..7c414b5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,26 @@ +## v0.270.0 — no update for a current app; an interrupted install is reported; a restore brings back the right health check (2026-09-24, R-679, R-681, R-669, R-674) + +**MinAgent: 0.131.0** (unchanged). Needs hub v0.123.0 (unchanged; R-681 rides the existing `app_deploy_failed`). +New strings: yes (hu + en). + +- **R-679:** an Update pressed on an app already at the catalog head is refused BEFORE anything moves — + `409 already_current`, „Ez az alkalmazás már a legfrissebb elérhető változatot futtatja — nincs mit + frissíteni." / "This app is already running the newest version available — there is nothing to update." + "Current" is `CatalogOrder`'s verdict, so a re-tested digest for a floating tag still updates. Measured: + navidrome at the head was pressed four times, each a dump, a pull and a restart. (A test comment had called a + same-version Update "the repair path"; Restart is that path.) +- **R-681:** an install cut off by a controller restart is finished through the failure path and REPORTED: an + install marker is written before the compose-up and removed when the install ends; a marker found at start → + `compose down` (volumes kept), the stale pin records cleared, `app_deploy_failed` with the reason, and the + apps page says „A telepítés egy újraindítás miatt félbemaradt…" until the next install. An install that had + finished (its record says deployed) only loses the marker. +- **R-669:** the recovery unit captures the PINNED version's `.felhom.yml` (`applied-meta`), not the stack dir's + file the sync may already have replaced with a newer or failing step's; a restore makes the restored file the + applied record. Measured 2026-09-24: the undo judged the correct old version with the failed step's probe + and held the app. +- **R-674:** the ladder log says "AT THE HEAD" when the pin equals the newest step, not "older than the ladder". +- Red-proofs: five (REPORT). + ## v0.269.1 — an installed app keeps the image it runs until an Update moves it (2026-09-24 night, Part B live finding) **MinAgent: 0.131.0** (unchanged). Needs hub v0.123.0 (as v0.269.0). New strings: none. diff --git a/controller/README.md b/controller/README.md index a45cbec..1385ad2 100644 --- a/controller/README.md +++ b/controller/README.md @@ -718,6 +718,12 @@ tested digest; an INSTALLED app keeps the digest it runs until a guarded update over and never renders a newer one (v0.269.1, `CarryDigests`). An update judges the new version by its own `.felhom.yml` (the step's, or the catalog's). A stranded app's Remove keeps the data. +**No update for a current app; interrupted installs (v0.270.0).** An Update on an app already at the catalog +head answers `409 already_current` and moves nothing (R-679). An install cut off by a controller restart is +finished at the next start: what it started is removed (volumes kept), `app_deploy_failed` is sent, and the +apps page says the install was interrupted until the next install (R-681). The recovery unit keeps the pinned +version's `.felhom.yml`, and a restore makes it the applied record (R-669). + **Start/restart never answer "completed" (v0.263.0, R-642)** — they answer what was requested and the state the containers are in at that moment; whether the app works is the health probe's to say. diff --git a/controller/cmd/controller/main.go b/controller/cmd/controller/main.go index fcb5369..ba1fce4 100644 --- a/controller/cmd/controller/main.go +++ b/controller/cmd/controller/main.go @@ -1715,6 +1715,11 @@ func main() { notifier.NotifyAppDeployFailed(name, display, detail) }) } + // R-681 (v0.270.0): an install a restart cut off is finished through the failure path and reported — + // AFTER the deploy-done hook exists, so its event is sent. + if names := stackMgr.RecoverInterruptedInstalls(); len(names) > 0 { + log.Printf("[WARN] [stacks] %d install(s) interrupted by the restart were resolved and reported: %v", len(names), names) + } stackMgr.RecoverMigration(ctx) webServer.SetEncryptionKey(encKey) webServer.SetAppExporter(appExporter) @@ -2839,6 +2844,8 @@ func (a *stackAdapter) RecreateStackDefinitionFromUnit(name, composeSrcDir strin } else if err := a.mgr.SetPin(name, stackDir, pin, data); err != nil { log.Printf("[ERROR] [stacks] pin %s: %v", name, err) } + // R-669 (v0.270.0): the restored .felhom.yml IS the pinned version's record now. + a.mgr.RecordRestoredAppliedMeta(name, stackDir) return nil } diff --git a/controller/internal/backup/r669_applied_meta_test.go b/controller/internal/backup/r669_applied_meta_test.go new file mode 100644 index 0000000..88a1529 --- /dev/null +++ b/controller/internal/backup/r669_applied_meta_test.go @@ -0,0 +1,57 @@ +package backup + +import ( + "io" + "log" + "os" + "path/filepath" + "strings" + "testing" + + "gitea.dooplex.hu/admin/felhom-controller/internal/stacks" +) + +// R-669 (v0.270.0): the recovery unit captures the PINNED version's .felhom.yml (the applied record), +// not the stack dir's — which the sync may already have replaced with a newer, even failing, step's file. +// MEASURED 2026-09-24 (A2): the unit held port 8999 (the failing step's probe), a restore wrote it back, +// and the next undo judged the correct version with it and held the app. +// +// COMPANION RED-PROOF (REPORT.md): capture the stack dir's .felhom.yml again → "the unit carries the +// failing step's probe". +func TestR669_UnitCapturesThePinnedVersionsMeta(t *testing.T) { + tmp := t.TempDir() + stackDir := filepath.Join(tmp, "stack") + drive := filepath.Join(tmp, "drive") + mustWrite(t, filepath.Join(stackDir, "docker-compose.yml"), "services:\n app:\n image: example/app:1.2.3\n") + mustWrite(t, filepath.Join(stackDir, ".felhom.yml"), "display_name: Example\nhealthcheck:\n checks:\n - type: api\n port: 8999\n") // flowed by the sync + mustWrite(t, stacks.AppliedMetaFile(stackDir), "display_name: Example\nhealthcheck:\n checks:\n - type: api\n port: 80\n") // the pinned version's + mustWrite(t, filepath.Join(stackDir, "app.yaml"), "deployed: true\nenv: {}\n") + m := &Manager{ + logger: log.New(io.Discard, "", 0), + systemDataPath: filepath.Join(tmp, "system"), + stackProvider: &fakeRecoveryProvider{info: RecoveryInfo{StackDir: stackDir, DisplayName: "Example", ImagePins: []string{"example/app:1.2.3"}}, hdd: drive}, + version: "vtest", + } + if err := m.CaptureRecoveryUnit("example"); err != nil { + t.Fatalf("capture: %v", err) + } + b, err := os.ReadFile(filepath.Join(RecoveryUnitComposePath(drive, "example"), ".felhom.yml")) + if err != nil { + t.Fatal(err) + } + if strings.Contains(string(b), "8999") || !strings.Contains(string(b), "port: 80") { + t.Fatalf("the unit carries the failing step's probe, not the pinned version's:\n%s", b) + } + // Control: no applied record (an app pinned before v0.263.2) → the stack dir's file, as before. + if err := os.RemoveAll(filepath.Dir(stacks.AppliedMetaFile(stackDir))); err != nil { + t.Fatal(err) + } + mustWrite(t, filepath.Join(stackDir, ".felhom.yml"), "display_name: Example changed\n") // force a recapture + if err := m.CaptureRecoveryUnit("example"); err != nil { + t.Fatal(err) + } + b, _ = os.ReadFile(filepath.Join(RecoveryUnitComposePath(drive, "example"), ".felhom.yml")) + if !strings.Contains(string(b), "Example changed") { + t.Fatalf("without an applied record the stack dir's file must be captured: %s", b) + } +} diff --git a/controller/internal/backup/recovery_unit.go b/controller/internal/backup/recovery_unit.go index 4134f8b..bf9895c 100644 --- a/controller/internal/backup/recovery_unit.go +++ b/controller/internal/backup/recovery_unit.go @@ -13,6 +13,7 @@ import ( "strings" "time" + "gitea.dooplex.hu/admin/felhom-controller/internal/stacks" "gitea.dooplex.hu/admin/felhom-controller/internal/system" "gopkg.in/yaml.v3" ) @@ -115,7 +116,17 @@ func (m *Manager) CaptureRecoveryUnit(stackName string) error { checksums := make(map[string]string) var configFiles []string for _, fname := range []string{"docker-compose.yml", ".felhom.yml"} { - data, err := os.ReadFile(filepath.Join(info.StackDir, fname)) + src := filepath.Join(info.StackDir, fname) + // R-669 (v0.270.0): the .felhom.yml that belongs to the PINNED version — the one the compose above + // runs — is the applied record. The stack dir's own file flows from the catalog on every sync and + // can already be a NEWER (even a failing) version's; restored with the unit, it became the pinned + // version's probe and an undo judged the correct version unhealthy (measured 2026-09-24, A2). + if fname == ".felhom.yml" { + if _, err := os.Stat(stacks.AppliedMetaFile(info.StackDir)); err == nil { + src = stacks.AppliedMetaFile(info.StackDir) + } + } + data, err := os.ReadFile(src) if err != nil { continue // optional — capture whichever exist } diff --git a/controller/internal/i18n/locales/en.json b/controller/internal/i18n/locales/en.json index 435499a..ae129a4 100644 --- a/controller/internal/i18n/locales/en.json +++ b/controller/internal/i18n/locales/en.json @@ -1325,6 +1325,7 @@ "err.stacks.ugyfel_adatok_ellenorzese_sikertelen": "checking your own files failed: %s", "err.stacks.ugyfel_adatok_osszefesulese_sikertelen": "merging your own files failed: %s", "err.stacks.ujratelepites_sikertelen": "the reinstall failed (%s): %s", + "err.stacks.update_already_current": "This app is already running the newest version available — there is nothing to update.", "err.stacks.update_downgrade": "This version is newer than the one in the catalog — moving back needs the operator.", "err.stacks.update_self_updating": "The controller is updating. Try again in a few minutes.", "err.stacks.update_undo_copy_failed": "The update did not start, because the copy of the data taken before an update could not be made. The app keeps running on its previous version.", @@ -2074,6 +2075,7 @@ "stacks.reszletek": "Details", "stacks.sablonok_frissitese": "↻ Refresh templates", "stacks.sablonok_frissitese_a_kozponti_katalogus": "Refresh the templates from the central catalog", + "stacks.install_interrupted": "The installation was interrupted by a restart, and the box removed what it had started. Press Install again.", "stacks.telepites": "Install", "stacks.telepitheto": "Installable", "stacks.url_nem_elerheto_utvonal_nincs": "URL not reachable – route not published", diff --git a/controller/internal/i18n/locales/hu.json b/controller/internal/i18n/locales/hu.json index 1a03e34..4b6227a 100644 --- a/controller/internal/i18n/locales/hu.json +++ b/controller/internal/i18n/locales/hu.json @@ -1316,6 +1316,7 @@ "err.stacks.ugyfel_adatok_ellenorzese_sikertelen": "ügyfél-adatok ellenőrzése sikertelen: %s", "err.stacks.ugyfel_adatok_osszefesulese_sikertelen": "ügyfél-adatok összefésülése sikertelen: %s", "err.stacks.ujratelepites_sikertelen": "újratelepítés sikertelen (%s): %s", + "err.stacks.update_already_current": "Ez az alkalmazás már a legfrissebb elérhető változatot futtatja — nincs mit frissíteni.", "err.stacks.update_downgrade": "Ez a változat újabb a katalógusban lévőnél — visszalépés csak az üzemeltető kérésére.", "err.stacks.update_self_updating": "A vezérlő éppen frissül. Próbáld újra néhány perc múlva.", "err.stacks.update_undo_copy_failed": "A frissítés nem indult el, mert a frissítés előtti adatmásolat nem készült el. Az alkalmazás a korábbi verzióval fut tovább.", @@ -2062,6 +2063,7 @@ "stacks.reszletek": "Részletek", "stacks.sablonok_frissitese": "↻ Sablonok frissítése", "stacks.sablonok_frissitese_a_kozponti_katalogus": "Sablonok frissítése a központi katalógusból", + "stacks.install_interrupted": "A telepítés egy újraindítás miatt félbemaradt, és a doboz eltávolította, amit elkezdett. Nyomd meg újra a Telepítés gombot.", "stacks.telepites": "Telepítés", "stacks.telepitheto": "Telepíthető", "stacks.url_nem_elerheto_utvonal_nincs": "URL nem elérhető – útvonal nincs publikálva", diff --git a/controller/internal/stacks/deploy.go b/controller/internal/stacks/deploy.go index 7fa42f6..ae800ed 100644 --- a/controller/internal/stacks/deploy.go +++ b/controller/internal/stacks/deploy.go @@ -397,6 +397,16 @@ func (m *Manager) DeployStack(req DeployRequest) (string, error) { } m.mu.Unlock() + // R-681: the install's own journal — removed when it ends either way; found at start = interrupted. + if err := markInstallPending(stackDir); err != nil { + m.logger.Printf("[WARN] [stacks] Stack %s: cannot write the install marker (%v) — a restart mid-install would go unreported", req.StackName, err) + } + m.mu.Lock() + if s, ok := m.stacks[req.StackName]; ok { + s.InstallInterrupted = false + } + m.mu.Unlock() + // Run docker compose up -d asynchronously go m.runComposeDeploy(req.StackName, stackDir, env, appCfg) @@ -453,6 +463,7 @@ func (m *Manager) runComposeDeploy(name, stackDir string, env map[string]string, // The app.yaml is deliberately NOT deleted here: it is the crash-safe record written with // Deployed:false, it carries the settings the customer typed, and a redeploy reuses them. The // state the surfaces read is `not_deployed`, which is the fact that matters. + clearInstallPending(stackDir) // R-681: the install ended (badly) and said so if m.deployDoneHook != nil { m.deployDoneHook(name, false, composeErr.Error()) } @@ -477,9 +488,12 @@ func (m *Manager) runComposeDeploy(name, stackDir string, env map[string]string, s.AppConfig = nil } m.mu.Unlock() + clearInstallPending(stackDir) return } + clearInstallPending(stackDir) // R-681: the durable record says deployed — the install is over + // Clear deploying flag m.mu.Lock() if s, ok := m.stacks[name]; ok { diff --git a/controller/internal/stacks/install_interrupted.go b/controller/internal/stacks/install_interrupted.go new file mode 100644 index 0000000..d64a57a --- /dev/null +++ b/controller/internal/stacks/install_interrupted.go @@ -0,0 +1,100 @@ +package stacks + +import ( + "os" + "path/filepath" + "time" +) + +// ── An install cut off by a controller restart is finished, not forgotten (R-681, v0.270.0) ───────── +// +// MEASURED 2026-09-24 (night, chaos round 2): `systemctl restart docker` 20 s into n8n's install took the +// controller down with it. After the restart the box logged NOTHING about n8n, the app read not_deployed +// and the household's page showed it as never installed — the install had simply vanished, while its +// half-written files stayed in the stack dir. An update journals itself and resumes after the same +// accident; an install did not. +// +// THE MECHANISM: DeployStack writes installPendingFile before the compose-up goroutine starts; +// runComposeDeploy removes it when the install ENDS either way (the failure path already reports itself, +// R-536/R-649). A pending marker found at start therefore means the process died mid-install, and +// RecoverInterruptedInstalls finishes it through the SAME failure path a failed install takes: +// - the durable record says Deployed:true → the install had finished; only the marker goes; +// - otherwise → `compose down` (volumes kept, R-649), the stale pin records cleared, the app reads +// not-installed with installInterruptedFile set (the page's sentence, surviving further restarts +// until the next install), and the deploy-done hook fires `app_deploy_failed` with the reason. +// Pinned by r681_install_interrupted_test.go. + +const ( + installPendingFile = ".felhom-install-pending" + installInterruptedFile = ".felhom-install-interrupted" +) + +// installInterruptedReason is the detail the event and the deploy page carry. +const installInterruptedReason = "interrupted by a controller restart before it finished — install it again" + +func markInstallPending(stackDir string) error { + _ = os.Remove(filepath.Join(stackDir, installInterruptedFile)) // a new install supersedes the old sentence + return os.WriteFile(filepath.Join(stackDir, installPendingFile), []byte(time.Now().UTC().Format(time.RFC3339)+"\n"), 0o644) +} + +func clearInstallPending(stackDir string) { + _ = os.Remove(filepath.Join(stackDir, installPendingFile)) +} + +func installInterrupted(stackDir string) bool { + _, err := os.Stat(filepath.Join(stackDir, installInterruptedFile)) + return err == nil +} + +// RecoverInterruptedInstalls runs once at start, after the deploy-done hook is wired. It returns the +// names it resolved as interrupted. +func (m *Manager) RecoverInterruptedInstalls() []string { + m.mu.RLock() + type cand struct{ name, dir string } + var cands []cand + for name, st := range m.stacks { + dir := filepath.Dir(st.ComposePath) + if _, err := os.Stat(filepath.Join(dir, installPendingFile)); err == nil { + cands = append(cands, cand{name, dir}) + } + } + m.mu.RUnlock() + + var out []string + for _, c := range cands { + if cfg := LoadAppConfig(c.dir); cfg != nil && cfg.Deployed { + clearInstallPending(c.dir) + m.logger.Printf("[INFO] [stacks] install %s: its record says deployed — the install had finished before the restart; marker cleared (R-681)", c.name) + continue + } + m.logger.Printf("[WARN] [stacks] install %s was INTERRUPTED by a controller restart — removing what it started (volumes kept) and reporting it (R-681)", c.name) + down := m.composeDownFn + if down == nil { + down = func(dir string) error { _, err := m.composeExecCustomEnv(dir, m.stackEnv(dir), "down"); return err } + } + if err := down(c.dir); err != nil { + m.logger.Printf("[ERROR] [stacks] install %s: removing what the interrupted install started failed: %v — Remove clears it", c.name, err) + } + for _, p := range []string{AppliedComposePath(c.dir), filepath.Join(c.dir, appliedMetaDir)} { + _ = os.RemoveAll(p) // the pin of a version that never became real + } + if err := os.WriteFile(filepath.Join(c.dir, installInterruptedFile), []byte(time.Now().UTC().Format(time.RFC3339)+"\n"), 0o644); err != nil { + m.logger.Printf("[WARN] [stacks] install %s: cannot record the interruption for the page: %v", c.name, err) + } + clearInstallPending(c.dir) + m.mu.Lock() + if s, ok := m.stacks[c.name]; ok { + s.Deployed = false + s.Deploying = false + s.AppConfig = nil + s.DeployError = installInterruptedReason + s.InstallInterrupted = true + } + m.mu.Unlock() + if m.deployDoneHook != nil { + m.deployDoneHook(c.name, false, installInterruptedReason) + } + out = append(out, c.name) + } + return out +} diff --git a/controller/internal/stacks/ladder.go b/controller/internal/stacks/ladder.go index dc617b2..d260be7 100644 --- a/controller/internal/stacks/ladder.go +++ b/controller/internal/stacks/ladder.go @@ -136,6 +136,13 @@ func nextLadderStep(templateDir string, pinned map[string]string) (LadderStep, e } } if idx < 0 { + // R-674 (v0.270.0): a pin equal to the newest entry's `to` is AT THE HEAD, not "older than the + // ladder" — the old sentence sent an operator looking for a missing record. Pinned by + // TestR674_HeadIsNotCalledOlder. + if sameRefs(ladder[len(ladder)-1].To, pinned) { + return LadderStep{Index: -1, Source: current, Meta: currentMeta, + Why: fmt.Sprintf("the installed version %s is AT THE HEAD of the update_ladder (%d entries) — the catalog's current definition", summarisePin(pinned), len(ladder))}, nil + } return LadderStep{Index: -1, Source: current, Meta: currentMeta, Why: fmt.Sprintf("the installed version %s matches no update_ladder entry (%d entries) — an app older than the ladder has no record to climb; the catalog's current definition", summarisePin(pinned), len(ladder))}, nil } diff --git a/controller/internal/stacks/manager.go b/controller/internal/stacks/manager.go index d2a883e..b2d7cc5 100644 --- a/controller/internal/stacks/manager.go +++ b/controller/internal/stacks/manager.go @@ -148,6 +148,9 @@ type Stack struct { AppConfig *AppConfig `json:"app_config,omitempty"` Deploying bool `json:"deploying"` // compose up in progress DeployError string `json:"deploy_error,omitempty"` // last async deploy error + // InstallInterrupted (R-681, v0.270.0): the last install was cut off by a controller restart; the + // page says so until the next install (read from the stack dir's marker at every scan). + InstallInterrupted bool `json:"install_interrupted,omitempty"` // Updating / UpdatePhase / UpdatePhaseLabel / UpdateError (update arc slice 4, v0.237.0) are the // guarded update's in-memory progress, the same shape as Deploying/DeployError: the API answers // 202 at once and the page polls GET /api/stacks/{name}. See update.go. @@ -267,7 +270,10 @@ type Manager struct { // --- guarded update (slice 4, update.go) --- updateGuards UpdateGuards // init-only, SetUpdateGuards; nil ⇒ every update is REFUSED updateComposeFn func(dir string, env []string, args ...string) (string, error) - updateHealthFn func(ctx context.Context, name string, timeout time.Duration) (bool, string) + // composeDownFn is RecoverInterruptedInstalls' seam (R-681): nil → the real `compose down`. Tests set a + // fake so they never reach real Docker (R-650). + composeDownFn func(dir string) error + updateHealthFn func(ctx context.Context, name string, timeout time.Duration) (bool, string) // v0.263.0 undo seams (undo.go): the volume copier (nil ⇒ docker) and the undo's health wait, // which receives the probe it must use (nil ⇒ waitUpdateHealthyMeta). undoCopier volumeCopier @@ -654,21 +660,23 @@ func (m *Manager) ScanStacks() error { existing.CatalogImages = catImages existing.LadderStepsLeft = stepsLeft existing.CatalogDigests, existing.CatalogTestedAt = catDigests, catTestedAt + existing.InstallInterrupted = !deployed && installInterrupted(stackDir) // R-681 } } else { m.stacks[name] = &Stack{ - Name: name, - Meta: meta, - ComposePath: composePath, - State: StateNotDeployed, - Deployed: deployed, - Protected: m.cfg.IsProtectedStack(name), - AppConfig: appCfg, - TemplateImages: tplImages, - CatalogImages: catImages, - LadderStepsLeft: stepsLeft, - CatalogDigests: catDigests, - CatalogTestedAt: catTestedAt, + Name: name, + Meta: meta, + ComposePath: composePath, + State: StateNotDeployed, + Deployed: deployed, + Protected: m.cfg.IsProtectedStack(name), + AppConfig: appCfg, + TemplateImages: tplImages, + CatalogImages: catImages, + LadderStepsLeft: stepsLeft, + InstallInterrupted: !deployed && installInterrupted(stackDir), // R-681 + CatalogDigests: catDigests, + CatalogTestedAt: catTestedAt, } } } diff --git a/controller/internal/stacks/r669_restore_meta_test.go b/controller/internal/stacks/r669_restore_meta_test.go new file mode 100644 index 0000000..5680e94 --- /dev/null +++ b/controller/internal/stacks/r669_restore_meta_test.go @@ -0,0 +1,33 @@ +package stacks + +import ( + "os" + "path/filepath" + "strings" + "testing" +) + +// R-669 (v0.270.0): after a restore writes the app's .felhom.yml, that file IS the pinned version's +// record — so the NEXT update's undo judges the old version with the restored probe, not the failed +// step's. Asserted on what the undo actually reads: savePreUpdateMeta's copy. +// +// COMPANION RED-PROOF (REPORT.md): make RecordRestoredAppliedMeta a no-op (v0.269.1: no restore path +// rewrote applied-meta) → "the next undo would judge with the failed step's probe". +func TestR669_RestoreResetsTheAppliedRecord(t *testing.T) { + m, _, _, _ := newSlice4Manager(t) + dir := filepath.Dir(m.stacks["nextcloud"].ComposePath) + if err := os.MkdirAll(filepath.Dir(AppliedMetaFile(dir)), 0o755); err != nil { + t.Fatal(err) + } + mustWrite(t, AppliedMetaFile(dir), "healthcheck:\n checks:\n - type: api\n port: 8999\n") // left by the failed step + mustWrite(t, filepath.Join(dir, ".felhom.yml"), "healthcheck:\n checks:\n - type: api\n port: 80\n") // written by the restore + m.RecordRestoredAppliedMeta("nextcloud", dir) + md, err := savePreUpdateMeta(dir) + if err != nil { + t.Fatal(err) + } + b, _ := os.ReadFile(filepath.Join(md, ".felhom.yml")) + if strings.Contains(string(b), "8999") || !strings.Contains(string(b), "port: 80") { + t.Fatalf("the next undo would judge with the failed step's probe:\n%s", b) + } +} diff --git a/controller/internal/stacks/r674_head_test.go b/controller/internal/stacks/r674_head_test.go new file mode 100644 index 0000000..fd99c57 --- /dev/null +++ b/controller/internal/stacks/r674_head_test.go @@ -0,0 +1,30 @@ +package stacks + +import ( + "path/filepath" + "strings" + "testing" +) + +// R-674 (v0.270.0): a pin equal to the ladder's newest `to` is AT THE HEAD — the log must not call it +// "older than the ladder". +// +// COMPANION RED-PROOF (REPORT.md): drop the head branch in nextLadderStep → "the head was called older +// than the ladder". +func TestR674_HeadIsNotCalledOlder(t *testing.T) { + dir := t.TempDir() + mustWrite(t, filepath.Join(dir, ".felhom.yml"), "display_name: X\nupdate_ladder:\n"+ladderLine(ladderA, ladderB)+ladderLine(ladderB, ladderC)) + mustWrite(t, filepath.Join(dir, "docker-compose.yml"), "services:\n web:\n image: "+ladderC+"\n") + step, err := nextLadderStep(dir, map[string]string{"web": ladderC}) + if err != nil { + t.Fatal(err) + } + if strings.Contains(step.Why, "older than the ladder") || !strings.Contains(step.Why, "AT THE HEAD") { + t.Fatalf("the head was called older than the ladder: %q", step.Why) + } + // Control: a pin the ladder does not know is still said so. + step, _ = nextLadderStep(dir, map[string]string{"web": "nextcloud:30.0.0-apache"}) + if !strings.Contains(step.Why, "matches no update_ladder entry") { + t.Fatalf("the unknown-pin sentence changed: %q", step.Why) + } +} diff --git a/controller/internal/stacks/r679_already_current_test.go b/controller/internal/stacks/r679_already_current_test.go new file mode 100644 index 0000000..e0fa049 --- /dev/null +++ b/controller/internal/stacks/r679_already_current_test.go @@ -0,0 +1,48 @@ +package stacks + +import ( + "strings" + "testing" + "time" +) + +// R-679 (v0.270.0): an Update pressed on an app already at the catalog head is refused BEFORE anything +// moves — no backup, no pull, no restart — with its own reason and a sentence in the household's +// language. A re-tested digest for the same tag is NOT current and still updates. +// +// COMPANION RED-PROOF (REPORT.md): drop the UpdateOrderCurrent check from UpdatePreflight → "an app at +// the head was allowed to update". +func TestR679_CurrentAppIsRefused(t *testing.T) { + m, _, _, _ := newSlice4Manager(t) + st := m.stacks["nextcloud"] + st.AppConfig.InstalledImages = map[string]InstalledImage{"web": oi("nextcloud:31.0.14-apache")} + st.CatalogImages = map[string]string{"web": "nextcloud:31.0.14-apache"} + ref := m.UpdatePreflight("nextcloud") + if ref == nil { + t.Fatal("an app at the head was allowed to update") + } + if ref.Reason != "already_current" || ref.Cause == nil || !strings.Contains(ref.Message, "legfrissebb") { + t.Fatalf("refusal = %q %q (cause %v) — want already_current, a key-bearing Cause, the Hungarian sentence", ref.Reason, ref.Message, ref.Cause) + } +} + +func TestR679_BehindAndRetestedDigestStillUpdate(t *testing.T) { + m, _, _, _ := newSlice4Manager(t) + st := m.stacks["nextcloud"] + st.AppConfig.InstalledImages = map[string]InstalledImage{"web": oi("nextcloud:31.0.13-apache")} + st.CatalogImages = map[string]string{"web": "nextcloud:31.0.14-apache"} + if ref := m.UpdatePreflight("nextcloud"); ref != nil { + t.Fatalf("a behind app was refused: %q", ref.Reason) + } + // same tag, a newer TESTED digest than the install → behind, not current + inst := oi("redis:7-alpine") + inst.Digest = "sha256:" + strings.Repeat("a", 64) + inst.At = time.Now().Add(-48 * time.Hour).UTC().Format(time.RFC3339) + st.AppConfig.InstalledImages = map[string]InstalledImage{"web": inst} + st.CatalogImages = map[string]string{"web": "redis:7-alpine"} + st.CatalogDigests = map[string]string{"web": "sha256:" + strings.Repeat("b", 64)} + st.CatalogTestedAt = time.Now() + if ref := m.UpdatePreflight("nextcloud"); ref != nil { + t.Fatalf("a re-tested digest was refused as current: %q", ref.Reason) + } +} diff --git a/controller/internal/stacks/r681_install_interrupted_test.go b/controller/internal/stacks/r681_install_interrupted_test.go new file mode 100644 index 0000000..64bb02f --- /dev/null +++ b/controller/internal/stacks/r681_install_interrupted_test.go @@ -0,0 +1,69 @@ +package stacks + +import ( + "os" + "path/filepath" + "testing" +) + +// R-681 (v0.270.0): an install a controller restart cut off is FINISHED through the failure path and +// REPORTED — never silent. The consequences asserted: `compose down` ran (never real Docker — a fake), the +// household is told (the deploy-done hook fires as a failure with the reason), the stale pin records are +// gone, and the page's flag survives a rescan. +// +// COMPANION RED-PROOF (REPORT.md): make RecoverInterruptedInstalls return without acting (v0.269.1: no +// install marker, nothing looked) → "an interrupted install was not reported". +func TestR681_InterruptedInstallIsFinishedAndReported(t *testing.T) { + m, dir := newPinManager(t, pinTplOld, pinTplOld, "deployed: false\nenv: {}\n") + m.stacks["nextcloud"].Deployed = false + mustWrite(t, filepath.Join(dir, installPendingFile), "2026-09-24T11:48:42Z\n") + mustWrite(t, AppliedComposePath(dir), pinTplOld) // stale, from an earlier install + var downs []string + m.composeDownFn = func(d string) error { downs = append(downs, d); return nil } + type call struct { + name string + ok bool + detail string + } + var hooks []call + m.SetDeployDoneHook(func(n string, ok bool, d string) { hooks = append(hooks, call{n, ok, d}) }) + + got := m.RecoverInterruptedInstalls() + if len(got) != 1 || len(hooks) != 1 || hooks[0].ok || hooks[0].detail != installInterruptedReason { + t.Fatalf("an interrupted install was not reported: resolved=%v hooks=%+v", got, hooks) + } + if len(downs) != 1 || downs[0] != dir { + t.Fatalf("what the interrupted install started was not removed: downs=%v", downs) + } + if _, err := os.Stat(AppliedComposePath(dir)); err == nil { + t.Fatal("the stale pin record of a version that never ran is still there") + } + if _, err := os.Stat(filepath.Join(dir, installPendingFile)); err == nil { + t.Fatal("the pending marker survived — the next start would report it again") + } + if err := m.ScanStacks(); err != nil { + t.Fatal(err) + } + if st, _ := m.GetStack("nextcloud"); !st.InstallInterrupted || st.Deployed { + t.Fatalf("after a rescan: interrupted=%v deployed=%v — the page must still say it", st.InstallInterrupted, st.Deployed) + } + // A second start finds nothing to report. + if again := m.RecoverInterruptedInstalls(); len(again) != 0 || len(hooks) != 1 { + t.Fatalf("reported twice: %v", again) + } +} + +// An install that FINISHED before the restart (its record says deployed) is left alone: no down, no event. +func TestR681_FinishedInstallIsLeftAlone(t *testing.T) { + m, dir := newPinManager(t, pinTplOld, pinTplOld, "deployed: true\nenv: {}\n") + mustWrite(t, filepath.Join(dir, installPendingFile), "x\n") + downs, hooks := 0, 0 + m.composeDownFn = func(string) error { downs++; return nil } + m.SetDeployDoneHook(func(string, bool, string) { hooks++ }) + if got := m.RecoverInterruptedInstalls(); len(got) != 0 || downs != 0 || hooks != 0 { + t.Fatalf("a finished install was treated as interrupted: resolved=%v downs=%d hooks=%d", got, downs, hooks) + } + if _, err := os.Stat(filepath.Join(dir, installPendingFile)); err == nil { + t.Fatal("the marker of a finished install was not cleared") + } +} diff --git a/controller/internal/stacks/undo.go b/controller/internal/stacks/undo.go index 17ef456..ae1ee4f 100644 --- a/controller/internal/stacks/undo.go +++ b/controller/internal/stacks/undo.go @@ -86,6 +86,18 @@ const preUpdateMetaDir = "pre-update-meta" // time, got the new probe (romm :8999), and judged the serving old version "did not start". const appliedMetaDir = "applied-meta" +// AppliedMetaFile is the pinned version's .felhom.yml record (R-669, v0.270.0: the recovery unit captures +// it, so a restore brings back the PINNED version's health check, not the sync's newer one). +func AppliedMetaFile(stackDir string) string { return filepath.Join(stackDir, appliedMetaDir, ".felhom.yml") } + +// RecordRestoredAppliedMeta makes the .felhom.yml a restore just wrote the pinned version's record +// (R-669, v0.270.0). MEASURED 2026-09-24 (A2): after a failed step ended in a restore, applied-meta still +// held the FAILED step's probe, and the next failed update's undo judged the correct old version with it +// and held the app. Pinned by TestR669_RestoreResetsTheAppliedRecord. +func (m *Manager) RecordRestoredAppliedMeta(name, stackDir string) { + m.storeAppliedMetaFrom(name, stackDir, filepath.Join(stackDir, ".felhom.yml")) +} + // storeAppliedMeta records the .felhom.yml of the version just pinned. A failure is logged by the // caller and never fails the act — without it the undo falls back to the current file, loudly. func storeAppliedMeta(stackDir string, src []byte) error { diff --git a/controller/internal/stacks/update.go b/controller/internal/stacks/update.go index 6374407..d3a7479 100644 --- a/controller/internal/stacks/update.go +++ b/controller/internal/stacks/update.go @@ -416,6 +416,20 @@ func (m *Manager) UpdatePreflight(name string) *UpdateRefusal { fmt.Sprintf("installed is provably NEWER than the catalog on every differing service (installed=%v catalog=%v)", st.AppConfig.InstalledImages, st.CatalogImages)) } + // R-679 (v0.270.0) — AN APP ALREADY AT THE HEAD IS NOT UPDATED. MEASURED 2026-09-24 (night, Part C): + // navidrome at the catalog head was pressed four times; each press made a safety dump, pulled and + // restarted the app (8.5 s of downtime) and changed nothing. "Current" is CatalogOrder's own verdict: + // the installed images equal the catalog's AND no newer tested digest exists for a floating tag — so a + // re-tested digest still updates. Unknown / unorderable / behind fall through as before. Before this, a + // same-version Update was called "the repair path" in a test comment; Restart is the repair path now. + // + // COMPANION RED-PROOF (REPORT.md): drop this check → TestR679_CurrentAppIsRefused fails at "an app at + // the head was allowed to update". + if CatalogOrder(*st) == UpdateOrderCurrent { + return m.refuseUpdateErr(name, "already_current", util.MsgError("err.stacks.update_already_current"), + fmt.Sprintf("installed equals the catalog head on every service and no newer tested digest (installed=%v catalog=%v)", + st.AppConfig.InstalledImages, st.CatalogImages)) + } // R-475: any tier counts, and an app with no copy at all is backed up first by the job. So the only // refusal left here is Scenario L — no copy on any tier AND no way to make one now. (With a copy // but no way to back up, the job still applies the age rule and refuses then if the copy is stale.) diff --git a/controller/internal/stacks/updateorder_test.go b/controller/internal/stacks/updateorder_test.go index 145440b..8c4fa82 100644 --- a/controller/internal/stacks/updateorder_test.go +++ b/controller/internal/stacks/updateorder_test.go @@ -202,7 +202,6 @@ func TestR524_PreflightRefusesDowngrade(t *testing.T) { }{ {"ahead — the downgrade is refused", "nextcloud:31.0.15-apache", "nextcloud:31.0.14-apache", "downgrade"}, {"behind — the ordinary update is allowed", "nextcloud:31.0.13-apache", "nextcloud:31.0.14-apache", ""}, - {"level — allowed (a same-version update is the repair path)", "nextcloud:31.0.14-apache", "nextcloud:31.0.14-apache", ""}, {"unorderable — allowed, exactly as before v0.260.0", "nextcloud:31-apache", "nextcloud:30-apache", ""}, } for _, c := range cases { diff --git a/controller/internal/web/i18n_parity_test.go b/controller/internal/web/i18n_parity_test.go index 7598523..e247716 100644 --- a/controller/internal/web/i18n_parity_test.go +++ b/controller/internal/web/i18n_parity_test.go @@ -299,6 +299,19 @@ func i18nCases() []i18nCase { d["StorageLabels"] = map[string]string{} return d }}) + // v0.270.0 (R-681): an app whose install a restart cut off — its sentence beside Install, captured when born. + base = append(base, i18nCase{"stacks_install_interrupted", "stacks", func() map[string]interface{} { + d := i18nLayoutData("stacks", "Alkalmazások") + st := i18nStack("n8n", "n8n", stacks.StateNotDeployed, false) + st.InstallInterrupted = true + d["Stacks"] = []stacks.Stack{st} + d["Subdomains"] = map[string]string{} + d["MissingStorage"] = map[string]string{} + d["NetworkStubs"] = map[string]string{} + d["NetworkWarnings"] = map[string]string{} + d["StorageLabels"] = map[string]string{} + return d + }}) // v0.268.0 (`09` §3 decision 14): the ladder's "steps remaining" line, captured when it was born. base = append(base, i18nCase{"app_info_ladder_steps", "app_info", func() map[string]interface{} { d := i18nLayoutData("stacks", "RomM") diff --git a/controller/internal/web/r681_install_interrupted_page_test.go b/controller/internal/web/r681_install_interrupted_page_test.go new file mode 100644 index 0000000..1897226 --- /dev/null +++ b/controller/internal/web/r681_install_interrupted_page_test.go @@ -0,0 +1,33 @@ +package web + +import ( + "strings" + "testing" + + "gitea.dooplex.hu/admin/felhom-controller/internal/stacks" +) + +// R-681 (v0.270.0) on the page: an app whose install a restart cut off says so, in both languages, beside +// its Install button — and an ordinary not-installed app shows nothing (negative control). +// +// COMPANION RED-PROOF (REPORT.md): drop the {{if .InstallInterrupted}} line from stacks.html → "the apps +// page is silent about an interrupted install". +func TestR681_PageSaysTheInstallWasInterrupted(t *testing.T) { + st := ubStack(nil, map[string]string{"web": "nginx:1.27"}, "2026-07-18") + st.Name = "n8n" + st.Deployed = false + st.State = stacks.StateNotDeployed + st.InstallInterrupted = true + hu := renderBackupPageLang(t, "hu", "stacks", ubStacksData(st)) + en := renderBackupPageLang(t, "en", "stacks", ubStacksData(st)) + if !strings.Contains(hu, `data-install-interrupted="true"`) || !strings.Contains(hu, "lbemaradt") { + t.Fatal("the apps page is silent about an interrupted install (hu)") + } + if !strings.Contains(en, `data-install-interrupted="true"`) || !strings.Contains(en, "interrupted by a restart") { + t.Fatal("the apps page is silent about an interrupted install (en)") + } + st.InstallInterrupted = false + if h := renderBackupPageLang(t, "hu", "stacks", ubStacksData(st)); strings.Contains(h, `data-install-interrupted`) || strings.Contains(h, "lbemaradt") { + t.Fatal("an ordinary not-installed app must not carry the sentence") + } +} diff --git a/controller/internal/web/templates/stacks.html b/controller/internal/web/templates/stacks.html index 1294045..2e35061 100644 --- a/controller/internal/web/templates/stacks.html +++ b/controller/internal/web/templates/stacks.html @@ -83,7 +83,7 @@ {{else if not .Deployed}} {{/* The endpoint refuses a non-installable template server-side; hiding the button here keeps the two consistent rather than offering an action that will fail. */}} - {{if canInstall .Meta}}{{T "stacks.telepites"}}{{end}} + {{if .InstallInterrupted}}
{{T "stacks.install_interrupted"}}
{{end}}{{if canInstall .Meta}}{{T "stacks.telepites"}}{{end}} {{T "stacks.reszletek"}} {{else}} {{/* Slice 4 (v0.238.0): the UPDATING and HELD checks come BEFORE isOperational. That diff --git a/controller/internal/web/testdata/i18n_parity/stacks_install_interrupted.html b/controller/internal/web/testdata/i18n_parity/stacks_install_interrupted.html new file mode 100644 index 0000000..f40c2b2 --- /dev/null +++ b/controller/internal/web/testdata/i18n_parity/stacks_install_interrupted.html @@ -0,0 +1,616 @@ + + + + + + + + Alkalmazások — Felhom.eu + + + + + + + + +
+ + +
+ + +
+ + + + + + + + +
+

Alkalmazások

+ example.hu + Importálás + +
+ + +
+ + + + +
+ +
+ + +
+
+
+ +
+

n8n

+ + +
+
+ Nincs telepítve + + + + + + +
+ + +

Egy teszt alkalmazás leírása

+ + +
+ ~128M + Pi kompatibilis + HDD szükséges + Magyar felület + +
+ + + +
+ + +
A telepítés egy újraindítás miatt félbemaradt, és a doboz eltávolította, amit elkezdett. Nyomd meg újra a Telepítés gombot.
Telepítés + Részletek + +
+
+ +
+ + + + +
+ + + + diff --git a/controller/scripts/i18n_go_keys.json b/controller/scripts/i18n_go_keys.json index 4a5fb7c..e3394fa 100644 --- a/controller/scripts/i18n_go_keys.json +++ b/controller/scripts/i18n_go_keys.json @@ -32,6 +32,7 @@ "update.phase.copying": "BORN AS A KEY, v0.263.0 (R-637, the undo) -- a NEW sentence, never a Go literal. Pinned by internal/stacks/undo_test.go and internal/web/undo_page_test.go.", "update.phase.undoing": "BORN AS A KEY, v0.263.0 (R-637, the undo) -- a NEW sentence, never a Go literal. Pinned by internal/stacks/undo_test.go and internal/web/undo_page_test.go.", "update.phase.undone": "BORN AS A KEY, v0.263.0 (R-637, the undo) -- a NEW sentence, never a Go literal. Pinned by internal/stacks/undo_test.go and internal/web/undo_page_test.go.", + "err.stacks.update_already_current": "BORN AS A KEY, v0.270.0 (R-679) -- the already-at-the-head refusal. Pinned by TestR679_CurrentAppIsRefused.", "err.stacks.update_downgrade": "BORN AS A KEY, v0.260.0 (R-524) -- the downgrade refusal. A NEW sentence, never a Go literal. Pinned by TestR524_PreflightRefusesDowngrade, which asserts the refusal carries a Cause so api.Router.errText can render it.", "err.stacks.update_self_updating": "BORN AS A KEY, v0.261.0 (R-608) -- the app update refuses to start while the controller is swapping itself. A NEW sentence, never a Go literal. Pinned by TestR608_PreflightRefusesWhileTheControllerSwaps, which asserts the refusal carries a Cause so errText can render it.", "func.state.degraded": "slice 0 -- pinned by TestLocaleFuncsHungarianBundleMatchesFuncMap",