controller v0.271.0: automatic app updates — the update leg after the off-site copy, the backup gate waits, the switch (09 6.4 part 7; R-680, R-678, R-643)
gates / gates (push) Successful in 24s

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
This commit is contained in:
2026-09-24 22:07:24 +02:00
parent 1495ca43fb
commit 9cf13a3add
33 changed files with 1672 additions and 14 deletions
+27
View File
@@ -1,3 +1,30 @@
## v0.271.0 — automatic app updates: the update leg, the failed step remembered, fresh badges (2026-09-25 night, `09` §6.4 part 7, R-680, R-678, R-643)
**MinAgent: 0.131.0** (unchanged). Needs hub v0.123.0 (unchanged; the report's new `update_leg` is additive —
the hub stores the raw report). New strings: yes (hu + en).
- **The update leg** (`09` §3 decisions 11, 12, 14, 15, 20; §6.4.2 point 6 (a)–(d)): chained to the
`offbox-backup` job on every path (`chainUpdateLeg` — nil, error, panic; a box without an off-site target runs it
at W+105m). One app at a time, one tested step per app per night, through the public guarded Update only.
Skipped with a named reason: held, current (not counted), ahead, order unknown, unpinned, no test record,
older than the ladder (logged by name), not `proven`, `needs_person`, `files_may_change` without a fresh whole
copy, the step that failed before on the same ladder, W+5h reached, switched off. Transient refusals retried once.
- **Decision 20:** the full-system backup's gate defers while the leg runs, until W+5h; after it only for a step in
flight, never past W+5h30m. One shared constant (`backupwindow.UpdateLegStopOffsetMin`). The controller's own
self-update also waits for the whole leg.
- **Decision 12 — the switch:** `app_update.unattended` in `settings.json`, absent = ON; a card on the settings page
in both languages (`POST /settings/app-update`). `stacks.update_window` removed (never read).
- **R-680:** an undone or held step is recorded in `app.yaml` (`failed_update_step`, tied to the ladder's print); the
leg never presses it again until the catalog's ladder for that app changes. A person can.
- **R-678:** after `done` (and `undone`) the app's steps-left, badge inputs and pin are re-read before the update
says it finished.
- The household hears nothing new: no mail for a successful automatic step; the app page says „Automatikus
frissítés %s-kor — sikeres." / "Automatic update at %s — done.". The operator: one summary line per night in the
log and the hub report (`update_leg`).
- Decision 28: pinned that the crash-loop stop never fires during an automatic step (verify, undo). **Found:** a
DEPLOY's first start is NOT covered — `Deploying` clears when `compose up -d` returns (R-676 updated).
- Red-proofs: eleven (`felhom.eu/documentation/audits/night-2026-09-25/B/redproofs/`).
## v0.270.0 — no update for a current app; an interrupted install is reported; a restore brings back the right health check (2026-09-24, R-679, R-681, R-669, R-674) ## v0.270.0 — no update for a current app; an interrupted install is reported; a restore brings back the right health check (2026-09-24, R-679, R-681, R-669, R-674)
**MinAgent: 0.131.0** (unchanged). Needs hub v0.123.0 (unchanged; R-681 rides the existing `app_deploy_failed`). **MinAgent: 0.131.0** (unchanged). Needs hub v0.123.0 (unchanged; R-681 rides the existing `app_deploy_failed`).
+16
View File
@@ -724,6 +724,22 @@ finished at the next start: what it started is removed (volumes kept), `app_depl
apps page says the install was interrupted until the next install (R-681). The recovery unit keeps the pinned apps page says the install was interrupted until the next install (R-681). The recovery unit keeps the pinned
version's `.felhom.yml`, and a restore makes it the applied record (R-669). version's `.felhom.yml`, and a restore makes it the applied record (R-669).
**Automatic updates (v0.271.0, `09` §6.4 part 7, decisions 11–15, 20).** Each night, after the off-site leg
(chained in the `offbox-backup` job on every path — configured or not, ok, failed, panicked; a box without an
off-site target runs it at W+105m), the box presses the SAME guarded Update a person presses: one app at a time,
ONE tested step per app per night, only a `proven` ladder entry; never a step marked `needs_person`; a
`files_may_change` step only when a fresh WHOLE copy exists (`backup.FreshWholeCopy`, the hold's truth table);
never an app older than the ladder, held, current, ahead or unorderable; never the step that was undone or held
before while the catalog's ladder is unchanged (`app.yaml` `failed_update_step`, R-680 — a person can still press
it). A passing refusal (`busy`…) is retried once. No step starts at or after W+5h, and the full-system backup's
gate waits for the leg until W+5h (then only for a step in flight, at most to W+5h30m). The controller's own
self-update waits for the whole leg. Switch: settings page „Alkalmazások automatikus frissítése" /
"Automatic app updates", `settings.json` `app_update.unattended`, **absent = ON**. A successful automatic step
sends no mail; the app page says „Automatikus frissítés %s-kor — sikeres." (`app.yaml` `last_auto_update`).
One summary line per night (`[update-leg] update leg (after-offsite): done=… undone=… held=… failed=…
skipped=… [skipped: app=reason, …]`) in the log and in the hub report's `update_leg`. After `done`/`undone` the
app's steps-left and badge are fresh at once (R-678). `stacks.update_window` is removed (it was never read).
**Start/restart never answer "completed" (v0.263.0, R-642)** — they answer what was requested and the **Start/restart never answer "completed" (v0.263.0, R-642)** — they answer what was requested and the
state the containers are in at that moment; whether the app works is the health probe's to say. state the containers are in at that moment; whether the app works is the health probe's to say.
+65 -1
View File
@@ -19,6 +19,7 @@ import (
"crypto/subtle" "crypto/subtle"
"strings" "strings"
"sync"
"gitea.dooplex.hu/admin/felhom-controller/internal/agentapi" "gitea.dooplex.hu/admin/felhom-controller/internal/agentapi"
"gitea.dooplex.hu/admin/felhom-controller/internal/api" "gitea.dooplex.hu/admin/felhom-controller/internal/api"
@@ -567,6 +568,23 @@ func main() {
// An UNWIRED manager also refuses (fail closed); TestSlice4_UpdateGuardsAreWiredAtStartup walks // An UNWIRED manager also refuses (fail closed); TestSlice4_UpdateGuardsAreWiredAtStartup walks
// this file for the call, because a seam built and never wired has shipped here seven times. // this file for the call, because a seam built and never wired has shipped here seven times.
stackMgr.SetUpdateGuards(&updateGuardsAdapter{b: backupMgr, q: quiesceLoop}) stackMgr.SetUpdateGuards(&updateGuardsAdapter{b: backupMgr, q: quiesceLoop})
// v0.271.0 (`09` §6.4 part 7): the automatic update leg. The switch is read at the leg's start and
// before every press; W is the SAME effective window every nightly leg reads. The files-may-change
// mark asks the backup side's truth table (decisions 25/26), never a second definition of "whole".
// Pinned by TestUpdateLegIsWiredAtStartup (an AST walk of this file).
stackMgr.SetUpdateLeg(stacks.UpdateLegOptions{
Enabled: sett.GetAppUpdateUnattended,
WindowStart: func() string {
return backupwindow.EffectiveWindow(sett.GetBackupWindowStart(), cfg.Backup.DBDumpSchedule)
},
FreshWholeCopy: func(ctx context.Context, name string) (bool, string) {
if backupMgr == nil {
return false, "backup is disabled on this box"
}
return backupMgr.FreshWholeCopy(ctx, name, cfg.Update.BackupMaxAgeDuration(), time.Now())
},
Location: budapestLoc(),
})
// v0.238.1: the nightly legs (capture, Tier 2, volume dump) leave an app alone WHILE it is being // v0.238.1: the nightly legs (capture, Tier 2, volume dump) leave an app alone WHILE it is being
// updated, not only once it is held — found live in Scenario F, see backup.Manager.isHeld. // updated, not only once it is held — found live in Scenario F, see backup.Manager.isHeld.
if backupMgr != nil { if backupMgr != nil {
@@ -718,7 +736,13 @@ func main() {
// `pulling`, `starting` and `verifying` do not — and the last two are where the new version // `pulling`, `starting` and `verifying` do not — and the last two are where the new version
// may already have touched the customer's data. // may already have touched the customer's data.
updater.SetAppUpdatingCheck(func() bool { updater.SetAppUpdatingCheck(func() bool {
return stackMgr != nil && stackMgr.AnyUpdating() if stackMgr == nil {
return false
}
// v0.271.0: the whole automatic update leg counts, not only a step in flight — a controller
// swap between two steps would cancel the rest of the night's leg.
legActive, _ := stackMgr.UpdateLegState()
return stackMgr.AnyUpdating() || legActive
}) })
if stackMgr != nil { if stackMgr != nil {
stackMgr.SetSelfUpdatingCheck(updater.IsUpdateRunning) stackMgr.SetSelfUpdatingCheck(updater.IsUpdateRunning)
@@ -1264,12 +1288,19 @@ func main() {
"A korábbi távoli mentések a türelmi idő lejártával törlésre kerültek, az ügyfél döntése alapján. A hozzájuk tartozó lezárt helyreállítási csomag eltávolítását is kértük.", map[string]string{"deleted_path": renamedTo}) "A korábbi távoli mentések a türelmi idő lejártával törlésre kerültek, az ügyfél döntése alapján. A hozzájuk tartozó lezárt helyreállítási csomag eltávolítását is kértük.", map[string]string{"deleted_path": renamedTo})
} }
}) })
// v0.271.0 (`09` §3 decision 11, §6.4.2 point 2): the automatic update leg is CHAINED to the
// off-site job — it runs when this function's off-site half has ended, on every path: configured
// or not, ok, failed, any of RunOffboxBackup's early returns, even a panic. A box with no off-site
// target runs it at W+105m. One call site, no signal to go stale.
sched.Daily("offbox-backup", offboxLeg, func(ctx context.Context) error { sched.Daily("offbox-backup", offboxLeg, func(ctx context.Context) error {
return chainUpdateLeg(ctx, func(ctx context.Context) error {
t := sett.GetOffboxTarget() t := sett.GetOffboxTarget()
if t == nil || !t.Enabled || t.Schedule != "daily" || !backupMgr.OffboxConfigured() { if t == nil || !t.Enabled || t.Schedule != "daily" || !backupMgr.OffboxConfigured() {
logger.Printf("[INFO] [offbox] no scheduled off-site target on this box — the off-site leg does nothing; the update leg runs now")
return nil // not configured / not scheduled return nil // not configured / not scheduled
} }
return backupMgr.RunOffboxBackup(ctx) return backupMgr.RunOffboxBackup(ctx)
}, func(ctx context.Context) { stackMgr.RunUpdateLeg(ctx, "after-offsite") })
}) })
// R-241 — the abandonment terminal step. DAILY and not on the backup leg, deliberately: it must // R-241 — the abandonment terminal step. DAILY and not on the backup leg, deliberately: it must
// run on a box whose off-site tier is NOT configured for runs (an abandoning box may be sitting // run on a box whose off-site tier is NOT configured for runs (an abandoning box may be sitting
@@ -3237,6 +3268,8 @@ func startQuiesceLoop(ctx context.Context, cfg *config.Config, sett *settings.Se
WindowStartFn: func() string { WindowStartFn: func() string {
return backupwindow.EffectiveWindow(sett.GetBackupWindowStart(), cfg.Backup.DBDumpSchedule) return backupwindow.EffectiveWindow(sett.GetBackupWindowStart(), cfg.Backup.DBDumpSchedule)
}, },
// v0.271.0 (`09` decision 20): the full-system backup waits for the automatic update leg.
UpdateLegFn: stackMgr.UpdateLegState,
}) })
loop.Recover() // crash-safety: restart any stacks stranded-down by a mid-quiesce crash loop.Recover() // crash-safety: restart any stacks stranded-down by a mid-quiesce crash
go loop.Run(ctx) go loop.Run(ctx)
@@ -3689,3 +3722,34 @@ func stopUnhealthyApps(logger *log.Logger, d unhealthyDeps, ooms []stacks.OOMCon
} }
return stopped return stopped
} }
// chainUpdateLeg runs the off-site half, then the update leg — ALWAYS, whatever the off-site half did:
// returned nil (ran, or had nothing to do), returned an error, or panicked. The off-site half's error is
// returned (the scheduler logs it); a panic becomes an error. `09` §6.4.2 point 2; pinned by
// TestChainUpdateLeg_EveryPath.
func chainUpdateLeg(ctx context.Context, offsite func(context.Context) error, leg func(context.Context)) (err error) {
defer func() {
if r := recover(); r != nil {
err = fmt.Errorf("the off-site leg panicked: %v", r)
}
leg(ctx)
}()
return offsite(ctx)
}
var (
budapestLocOnce sync.Once
budapestLocVal *time.Location
)
// budapestLoc is the wall clock the backup window is read in (the scheduler's and the quiesce gate's).
func budapestLoc() *time.Location {
budapestLocOnce.Do(func() {
loc, err := time.LoadLocation("Europe/Budapest")
if err != nil {
loc = time.Local
}
budapestLocVal = loc
})
return budapestLocVal
}
@@ -0,0 +1,73 @@
package main
import (
"context"
"errors"
"os"
"strings"
"testing"
)
// v0.271.0 — the automatic update leg (`09` §6.4 part 7).
// TestChainUpdateLeg_EveryPath — the leg runs after the off-site half on EVERY path it can take:
// nothing to do (no target), ran ok, returned an error (incl. every RunOffboxBackup early return, which
// returns to here), and panicked. The off-site error reaches the scheduler unchanged; a panic becomes one.
//
// COMPANION RED-PROOF (REPORT.md): call the leg after `return offsite(ctx)` instead of in the defer —
// the error and panic cases fail at "the leg did not run after the off-site half".
func TestChainUpdateLeg_EveryPath(t *testing.T) {
boom := errors.New("sftp: connection refused")
cases := []struct {
name string
offsite func(context.Context) error
wantErr string
}{
{"no off-site target (nothing to do)", func(context.Context) error { return nil }, ""},
{"off-site ran ok", func(context.Context) error { return nil }, ""},
{"off-site failed / an early return", func(context.Context) error { return boom }, "connection refused"},
{"off-site panicked", func(context.Context) error { panic("nil map") }, "panicked"},
}
for _, c := range cases {
ran := 0
err := chainUpdateLeg(context.Background(), c.offsite, func(context.Context) { ran++ })
if ran != 1 {
t.Errorf("%s: the leg did not run after the off-site half (ran %d times)", c.name, ran)
}
if c.wantErr == "" && err != nil {
t.Errorf("%s: unexpected error %v", c.name, err)
}
if c.wantErr != "" && (err == nil || !strings.Contains(err.Error(), c.wantErr)) {
t.Errorf("%s: error %v, want one containing %q", c.name, err, c.wantErr)
}
}
}
// TestUpdateLegIsWiredAtStartup — the leg, its chain and the gate's interlock are CALLED from main.go (an
// AST walk; a comment naming them would not count). A seam built and never wired is this project's
// commonest defect.
//
// COMPANION RED-PROOF (REPORT.md): comment out the SetUpdateLeg call — this fails.
func TestUpdateLegIsWiredAtStartup(t *testing.T) {
lines, _, _ := slice4CallLines(t)
for _, callee := range []string{"SetUpdateLeg", "chainUpdateLeg", "RunUpdateLeg", "FreshWholeCopy"} {
if len(lines[callee]) == 0 {
t.Errorf("main.go never calls %s — the automatic update leg is not wired", callee)
}
}
src, err := os.ReadFile("main.go")
if err != nil {
t.Fatal(err)
}
if !strings.Contains(string(src), "Enabled: sett.GetAppUpdateUnattended,") {
t.Error("the leg's switch must be the household's app_update.unattended (sett.GetAppUpdateUnattended)")
}
if !strings.Contains(string(src), "UpdateLegFn: stackMgr.UpdateLegState") {
t.Error("the quiesce loop must be given stackMgr.UpdateLegState (decision 20's interlock)")
}
// the chain must sit INSIDE the offbox-backup job, not beside it
i := strings.Index(string(src), `sched.Daily("offbox-backup"`)
if i < 0 || !strings.Contains(string(src)[i:i+600], "chainUpdateLeg(") {
t.Error("the offbox-backup job must run through chainUpdateLeg")
}
}
@@ -57,7 +57,6 @@ stacks:
- "cloudflared" - "cloudflared"
- "felhom-controller" - "felhom-controller"
- "filebrowser" - "filebrowser"
update_window: "03:00-05:00"
compose_command: "" compose_command: ""
# --- Backup --- # --- Backup ---
@@ -726,6 +726,21 @@ func (m *Manager) HoldAfterFailedUpdateWhole(ctx context.Context, stackName stri
return false, nil return false, nil
} }
// FreshWholeCopy answers decision 13's `files_may_change` mark for the automatic update leg (v0.271.0):
// is there a copy on this box, younger than maxAge, that brings the app back WHOLE — the SAME truth
// table the hold uses (WholeOnTier, decisions 25 and 26), so the leg and the hold cannot disagree about
// what "whole" means. The string says why, for the leg's log.
func (m *Manager) FreshWholeCopy(ctx context.Context, stackName string, maxAge time.Duration, now time.Time) (bool, string) {
best, found, seen := m.HoldCopies(ctx, stackName)
if !found {
return false, fmt.Sprintf("no copy on this box brings it back whole (%d copies seen; drive files declared: %v)", len(seen), m.HasDriveFileLegs(stackName))
}
if age := now.Sub(best.At); age > maxAge {
return false, fmt.Sprintf("the newest whole copy (tier %d, %s) is %s old, limit %s", best.Tier, best.At.UTC().Format(time.RFC3339), age.Round(time.Minute), maxAge)
}
return true, fmt.Sprintf("tier %d copy from %s", best.Tier, best.At.UTC().Format(time.RFC3339))
}
// HoldNoWholeCopy reports whether the app's hold names no copy (R-659) — the page then offers no // HoldNoWholeCopy reports whether the app's hold names no copy (R-659) — the page then offers no
// restore button for it. // restore button for it.
func (m *Manager) HoldNoWholeCopy(stackName string) bool { func (m *Manager) HoldNoWholeCopy(stackName string) bool {
@@ -20,6 +20,12 @@ const (
gateEndMin = 360 // whole-guest gate closes (exclusive) at W+6h gateEndMin = 360 // whole-guest gate closes (exclusive) at W+6h
) )
// UpdateLegStopOffsetMin is W+5h: the automatic update leg starts no step at or after it, and the
// full-system backup's gate stops waiting for the leg there, so that backup keeps at least one hour of
// its [W+2h, W+6h) window (`09` §3 decision 20, v0.271.0). Read by stacks.LegDeadline AND the quiesce
// gate — one constant, so the two can never disagree (TestLegDeadlineAndGateShareW5h).
const UpdateLegStopOffsetMin = 300
// ParseHHMM parses "HH:MM" (24h) into minutes-since-midnight. It rejects anything but a valid // ParseHHMM parses "HH:MM" (24h) into minutes-since-midnight. It rejects anything but a valid
// hour:minute — the same contract as the scheduler's parseDailyTime, kept here so this package is // hour:minute — the same contract as the scheduler's parseDailyTime, kept here so this package is
// dependency-free and reusable by the quiesce gate. // dependency-free and reusable by the quiesce gate.
-2
View File
@@ -192,7 +192,6 @@ type GitConfig struct {
type StacksConfig struct { type StacksConfig struct {
Protected []string `yaml:"protected"` Protected []string `yaml:"protected"`
UpdateWindow string `yaml:"update_window"`
ComposeCommand string `yaml:"compose_command"` ComposeCommand string `yaml:"compose_command"`
} }
@@ -399,7 +398,6 @@ func applyDefaults(cfg *Config) {
d(&cfg.Web.SetupListen, ":8081") d(&cfg.Web.SetupListen, ":8081")
d(&cfg.Git.Branch, "main") d(&cfg.Git.Branch, "main")
d(&cfg.Git.SyncInterval, "15m") d(&cfg.Git.SyncInterval, "15m")
d(&cfg.Stacks.UpdateWindow, "03:00-05:00")
d(&cfg.Backup.DBDumpSchedule, "02:30") d(&cfg.Backup.DBDumpSchedule, "02:30")
d(&cfg.Backup.ResticSchedule, "03:00") d(&cfg.Backup.ResticSchedule, "03:00")
d(&cfg.Backup.PruneSchedule, "weekly") d(&cfg.Backup.PruneSchedule, "weekly")
+9 -1
View File
@@ -2404,5 +2404,13 @@
"update.refusal.disk": "Not enough free space for the update: %.1f GB free, and at least %.0f GB is needed to download the new version.", "update.refusal.disk": "Not enough free space for the update: %.1f GB free, and at least %.0f GB is needed to download the new version.",
"update.refusal.migrating": "The update cannot start right now: data is being moved.", "update.refusal.migrating": "The update cannot start right now: data is being moved.",
"update.refusal.no_backup": "%s cannot be updated, because it has no backup it could be restored from, and no new backup can be made of it now. Check on the Backups page that the app's drive is available — after that the update can start.", "update.refusal.no_backup": "%s cannot be updated, because it has no backup it could be restored from, and no new backup can be made of it now. Check on the Backups page that the app's drive is available — after that the update can start.",
"update.refusal.not_deployed": "The app is not installed, so it cannot be updated." "update.refusal.not_deployed": "The app is not installed, so it cannot be updated.",
"settings_system.app_update_title": "Automatic app updates",
"settings_system.app_update_desc": "When this is on, the box updates your apps by itself each night, after the off-site backup: one app at a time, only with a tested step, and if something fails it puts the previous version back.",
"settings_system.app_update_toggle": "Automatic updates on",
"settings_system.app_update_save": "Save",
"settings.app_update_on": "Automatic app updates are on. This applies from the next night.",
"settings.app_update_off": "Automatic app updates are off. From the next night no app updates by itself; you can still update them by hand.",
"settings.app_update_save_error": "The setting could not be saved. Try again.",
"app_info.auto_update_done": "Automatic update at %s — done."
} }
+9 -1
View File
@@ -2392,5 +2392,13 @@
"update.refusal.disk": "Nincs elég szabad hely a frissítéshez: %.1f GB szabad, az új verzió letöltéséhez legalább %.0f GB szükséges.", "update.refusal.disk": "Nincs elég szabad hely a frissítéshez: %.1f GB szabad, az új verzió letöltéséhez legalább %.0f GB szükséges.",
"update.refusal.migrating": "A frissítés most nem indítható: adatáthelyezés folyamatban.", "update.refusal.migrating": "A frissítés most nem indítható: adatáthelyezés folyamatban.",
"update.refusal.no_backup": "A(z) %s nem frissíthető, mert nincs olyan biztonsági mentése, amelyből vissza lehetne állítani, és most új mentés sem készíthető róla. Ellenőrizd a Mentések oldalon, hogy az alkalmazás meghajtója elérhető-e — utána a frissítés elindítható.", "update.refusal.no_backup": "A(z) %s nem frissíthető, mert nincs olyan biztonsági mentése, amelyből vissza lehetne állítani, és most új mentés sem készíthető róla. Ellenőrizd a Mentések oldalon, hogy az alkalmazás meghajtója elérhető-e — utána a frissítés elindítható.",
"update.refusal.not_deployed": "Az alkalmazás nincs telepítve, ezért nem frissíthető." "update.refusal.not_deployed": "Az alkalmazás nincs telepítve, ezért nem frissíthető.",
"settings_system.app_update_title": "Alkalmazások automatikus frissítése",
"settings_system.app_update_desc": "Ha be van kapcsolva, a doboz minden éjjel, a távoli mentés után magától frissíti az alkalmazásaidat: egyszerre egyet, mindig csak kipróbált lépéssel, és ha valami nem sikerül, visszaállítja az előző változatot.",
"settings_system.app_update_toggle": "Automatikus frissítés bekapcsolva",
"settings_system.app_update_save": "Mentés",
"settings.app_update_on": "Az alkalmazások automatikus frissítése bekapcsolva. A következő éjjel már érvényes.",
"settings.app_update_off": "Az alkalmazások automatikus frissítése kikapcsolva. A következő éjjel már egyetlen alkalmazás sem frissül magától; kézzel továbbra is frissítheted őket.",
"settings.app_update_save_error": "A beállítást nem sikerült elmenteni. Próbáld újra.",
"app_info.auto_update_done": "Automatikus frissítés %s-kor — sikeres."
} }
@@ -0,0 +1,85 @@
package quiesce
import (
"context"
"testing"
"gitea.dooplex.hu/admin/felhom-controller/internal/backupwindow"
)
// v0.271.0 — `09` §3 decision 20: on an update night the full-system backup WAITS for the automatic
// update leg, inside its own window, until W+5h; after W+5h only for a step already in flight, and never
// at or past W+5h30m. Window 02:30 → gate [04:30, 08:30), leg stop 07:30, step grace until 08:00.
// TestD20_GateWaitsForTheLeg — the CONSEQUENCE, through runOnce: while the leg runs at 04:45 the due
// backup is not started and no app is stopped; the same poll at 07:30 (W+5h) runs it.
//
// COMPANION RED-PROOF (REPORT.md): drop the updateLegDefers block from runOnce — this test fails at
// "the backup started while the update leg was running (04:45, before W+5h)".
func TestD20_GateWaitsForTheLeg(t *testing.T) {
legRunning := func() (bool, bool) { return true, false }
be := &fakeBackend{due: true, dueAge: i64(20 * 3600), phases: []string{"done"}}
st := &fakeStacks{running: []string{"nextcloud"}}
l := windowLoop(t, be, st, "02:30", atBudapest(4, 45))
l.updateLegFn = legRunning
if err := l.runOnce(context.Background()); err != nil {
t.Fatalf("runOnce: %v", err)
}
if be.startCalls != 0 || len(st.stoppedNames()) != 0 {
t.Fatalf("the backup started while the update leg was running (04:45, before W+5h): start=%d stopped=%v", be.startCalls, st.stoppedNames())
}
be2 := &fakeBackend{due: true, dueAge: i64(20 * 3600), phases: []string{"done"}}
st2 := &fakeStacks{running: []string{"nextcloud"}}
l2 := windowLoop(t, be2, st2, "02:30", atBudapest(7, 30))
l2.updateLegFn = legRunning
if err := l2.runOnce(context.Background()); err != nil {
t.Fatalf("runOnce: %v", err)
}
if be2.startCalls != 1 {
t.Fatalf("at W+5h (07:30) the backup must run even though the leg still says active; start=%d", be2.startCalls)
}
}
// TestD20_UpdateLegDefers — the truth table of the pure predicate.
func TestD20_UpdateLegDefers(t *testing.T) {
cases := []struct {
name string
active, step bool
h, m int
window string
want bool
}{
{"no leg", false, false, 5, 0, "02:30", false},
{"leg running inside the gate", true, false, 4, 45, "02:30", true},
{"leg running one minute before W+5h", true, false, 7, 29, "02:30", true},
{"leg running AT W+5h", true, false, 7, 30, "02:30", false},
{"step in flight at W+5h+10m", true, true, 7, 40, "02:30", true},
{"step in flight at W+5h+30m (cap)", true, true, 8, 0, "02:30", false},
{"leg running across midnight (W 23:00, now 03:00)", true, false, 3, 0, "23:00", true},
{"leg running across midnight at W+5h (04:00)", true, false, 4, 0, "23:00", false},
{"unreadable window never blocks", true, true, 5, 0, "nonsense", false},
}
for _, c := range cases {
fn := func() (bool, bool) { return c.active, c.step }
if got, _ := updateLegDefers(fn, atBudapest(c.h, c.m), c.window); got != c.want {
t.Errorf("%s: updateLegDefers = %v, want %v", c.name, got, c.want)
}
}
if got, _ := updateLegDefers(nil, atBudapest(5, 0), "02:30"); got {
t.Error("an unwired leg must never defer the backup")
}
}
// TestLegDeadlineAndGateShareW5h — the gate and the leg read the SAME offset (stacks.LegDeadline uses
// backupwindow.UpdateLegStopOffsetMin too). If either moves alone, the backup would wait for a leg that
// has stopped starting steps, or the leg would start steps the backup no longer waits for.
func TestLegDeadlineAndGateShareW5h(t *testing.T) {
if backupwindow.UpdateLegStopOffsetMin != 300 {
t.Fatalf("decision 20 says W+5h; the shared constant is %d min", backupwindow.UpdateLegStopOffsetMin)
}
if backupwindow.UpdateLegStopOffsetMin >= 360 || backupwindow.UpdateLegStopOffsetMin+legStepGraceMin >= 360 {
t.Fatal("the leg's stop (plus the in-flight grace) must leave the full-system backup part of its [W+2h, W+6h) window")
}
}
+46 -1
View File
@@ -79,6 +79,11 @@ type Options struct {
// Cadence is the agent's backup cadence, used only by the gate's safety valve (run regardless of // Cadence is the agent's backup cadence, used only by the gate's safety valve (run regardless of
// the window once the last successful backup is older than Cadence+24h). Defaults to 24h. // the window once the last successful backup is older than Cadence+24h). Defaults to 24h.
Cadence time.Duration Cadence time.Duration
// UpdateLegFn (v0.271.0, `09` §3 decision 20) reports whether the automatic update leg is running
// and whether one of its steps is in flight. A SCHEDULED cycle defers while the leg runs, until
// W+5h; after W+5h it waits only for a step already in flight, and never past W+5h30m. nil = no
// interlock (pre-v0.271.0 behaviour). Read with WindowStartFn; without a window it is not consulted.
UpdateLegFn func() (active, stepRunning bool)
} }
// Loop is the quiesce background loop. // Loop is the quiesce background loop.
@@ -94,6 +99,7 @@ type Loop struct {
// windowStartFn (nil = gate disabled) + cadence drive the scheduled-cycle window gate (Part 3). // windowStartFn (nil = gate disabled) + cadence drive the scheduled-cycle window gate (Part 3).
windowStartFn func() string windowStartFn func() string
cadence time.Duration cadence time.Duration
updateLegFn func() (active, stepRunning bool)
// mu single-flights the quiesce cycle across the scheduled loop AND the manual trigger, so the // mu single-flights the quiesce cycle across the scheduled loop AND the manual trigger, so the
// two can never stop the same stacks concurrently (the persisted marker covers crash-safety across // two can never stop the same stacks concurrently (the persisted marker covers crash-safety across
// restarts; this covers concurrency within the process — which a manual trigger introduces). // restarts; this covers concurrency within the process — which a manual trigger introduces).
@@ -152,7 +158,7 @@ func New(o Options) *Loop {
backend: o.Backend, stacks: o.Stacks, markerPath: o.MarkerPath, backend: o.Backend, stacks: o.Stacks, markerPath: o.MarkerPath,
poll: o.Poll, statusPoll: o.StatusPoll, maxQuiesce: o.MaxQuiesce, poll: o.Poll, statusPoll: o.StatusPoll, maxQuiesce: o.MaxQuiesce,
logger: o.Logger, now: time.Now, logger: o.Logger, now: time.Now,
windowStartFn: o.WindowStartFn, cadence: o.Cadence, windowStartFn: o.WindowStartFn, cadence: o.Cadence, updateLegFn: o.UpdateLegFn,
breaker: newFailureBreaker(), breaker: newFailureBreaker(),
contention: newContentionTracker(), contention: newContentionTracker(),
} }
@@ -242,6 +248,13 @@ func (l *Loop) runOnce(ctx context.Context) error {
l.logger.Printf("[DEBUG] [quiesce] scheduled backup due but outside the backup window [%s–%s) — deferring to the next poll inside it", from, to) l.logger.Printf("[DEBUG] [quiesce] scheduled backup due but outside the backup window [%s–%s) — deferring to the next poll inside it", from, to)
return nil return nil
} }
// Decision 20 (v0.271.0): on an update night the full-system backup WAITS for the update leg,
// inside its own window — the leg starts no step at W+5h, so this backup keeps an hour.
// COMPANION RED-PROOF (REPORT.md): drop this block — TestD20_GateWaitsForTheLeg fails.
if wait, why := updateLegDefers(l.updateLegFn, l.now().In(budapestLocation()), window); wait {
l.logger.Printf("[INFO] [quiesce] full-system backup due and inside its window, but %s — deferring to the next poll (`09` decision 20)", why)
return nil
}
} }
return l.quiesceAndPollTiers(ctx, dueTiers) return l.quiesceAndPollTiers(ctx, dueTiers)
@@ -785,3 +798,35 @@ func (l *Loop) clearMarker() error {
} }
return err return err
} }
// updateLegDefers is decision 20's interlock, pure: the full-system backup waits while the automatic
// update leg runs and it is before W+5h; from W+5h it waits only for a step already in flight, and never
// at or past W+5h30m (a step is bounded by its own health timeouts; the cap keeps a stuck flag from
// eating the backup's hour). The offsets come from backupwindow, the constant the leg's own deadline
// reads, so the two stop at the same minute (TestLegDeadlineAndGateShareW5h).
func updateLegDefers(fn func() (bool, bool), now time.Time, windowStart string) (bool, string) {
if fn == nil {
return false, ""
}
active, stepRunning := fn()
if !active && !stepRunning {
return false, ""
}
startMin, err := backupwindow.ParseHHMM(windowStart)
if err != nil {
return false, "" // an unreadable window never blocks the backup
}
nowMin := now.Hour()*60 + now.Minute()
stopMin := backupwindow.UpdateLegStopOffsetMin
if active && within(nowMin, startMin, stopMin) {
return true, fmt.Sprintf("the automatic update leg is running (it starts no step after %s)", backupwindow.FmtHHMM(mod1440(startMin+stopMin)))
}
if stepRunning && within(nowMin, startMin, stopMin+legStepGraceMin) {
return true, fmt.Sprintf("an automatic update step started before %s is still running (waiting at most until %s)",
backupwindow.FmtHHMM(mod1440(startMin+stopMin)), backupwindow.FmtHHMM(mod1440(startMin+stopMin+legStepGraceMin)))
}
return false, ""
}
// legStepGraceMin bounds how long past W+5h the gate waits for a step already in flight.
const legStepGraceMin = 30
+4
View File
@@ -181,6 +181,10 @@ func BuildReport(
// DR recipe app-half — customer identity + per-app {catalog_ref, enabled, storage_bindings}. // DR recipe app-half — customer identity + per-app {catalog_ref, enabled, storage_bindings}.
// Allowlist-only (the boundary): NO env/secret fields. The hub assembles it with the agent half. // Allowlist-only (the boundary): NO env/secret fields. The hub assembles it with the agent half.
if stackMgr != nil {
r.UpdateLeg = stackMgr.LastUpdateLegSummary()
}
r.DRRecipe = BuildDRRecipeAppHalf(cfg.Customer.ID, cfg.Customer.Name, cfg.Customer.Domain, r.DRRecipe = BuildDRRecipeAppHalf(cfg.Customer.ID, cfg.Customer.Name, cfg.Customer.Domain,
stackMgr.GetStacks(), readComposeFile) stackMgr.GetStacks(), readComposeFile)
// fork-4: attach the non-secret offsite restic repo coordinates so DR knows where to recover from. // fork-4: attach the non-secret offsite restic repo coordinates so DR knows where to recover from.
+6
View File
@@ -5,6 +5,7 @@ import (
"gitea.dooplex.hu/admin/felhom-controller/internal/backup" "gitea.dooplex.hu/admin/felhom-controller/internal/backup"
"gitea.dooplex.hu/admin/felhom-controller/internal/metrics" "gitea.dooplex.hu/admin/felhom-controller/internal/metrics"
"gitea.dooplex.hu/admin/felhom-controller/internal/stacks"
) )
// Report is the JSON payload pushed to the central hub. // Report is the JSON payload pushed to the central hub.
@@ -53,6 +54,11 @@ type Report struct {
// customer e-mails can follow it (plan slice 3). Additive: the hub stores the raw report and no hub // customer e-mails can follow it (plan slice 3). Additive: the hub stores the raw report and no hub
// release reads it yet. Always present, never omitted: a box that never chose reports "hu". // release reads it yet. Always present, never omitted: a box that never chose reports "hu".
Language string `json:"language"` Language string `json:"language"`
// UpdateLeg (v0.271.0, `09` §6.4 part 7) — the operator's one summary of the last automatic update
// leg on this box: steps done / undone / held / failed / skipped, each skip with its reason. Absent
// until a leg has run since the controller started. Additive: the hub stores the raw report.
UpdateLeg *stacks.UpdateLegSummary `json:"update_leg,omitempty"`
} }
// SystemReport holds host-level system info. // SystemReport holds host-level system info.
+36
View File
@@ -188,6 +188,12 @@ type Settings struct {
// offsets; overrides yaml when a valid value is present (mirrors PasswordHash precedence). // offsets; overrides yaml when a valid value is present (mirrors PasswordHash precedence).
BackupWindowStart string `json:"backup_window_start,omitempty"` BackupWindowStart string `json:"backup_window_start,omitempty"`
// AppUpdate (v0.271.0, `09` §3 decision 12) — the per-box switch for AUTOMATIC app updates. The key
// is `app_update.unattended`; ABSENT MEANS ON (the operator's ruling: on by default), so a box that
// never saved the switch, and every box upgraded from before v0.271.0, updates its apps by itself.
// Only an explicit false turns it off. NOT `self_update.auto_update` — that is the controller's own.
AppUpdate *AppUpdateSettings `json:"app_update,omitempty"`
// Storage paths registry // Storage paths registry
StoragePaths []StoragePath `json:"storage_paths,omitempty"` StoragePaths []StoragePath `json:"storage_paths,omitempty"`
@@ -886,6 +892,36 @@ func (s *Settings) SetBackupWindowStart(start string) error {
return s.save() return s.save()
} }
// AppUpdateSettings is settings.json's `app_update` object (v0.271.0).
type AppUpdateSettings struct {
// Unattended nil = the default, ON (decision 12). Pinned by TestAppUpdateUnattended_DefaultOn.
Unattended *bool `json:"unattended,omitempty"`
}
// GetAppUpdateUnattended reports whether this box may update its apps by itself (decision 12).
// Absent = ON.
func (s *Settings) GetAppUpdateUnattended() bool {
s.mu.RLock()
defer s.mu.RUnlock()
if s.AppUpdate == nil || s.AppUpdate.Unattended == nil {
return true
}
return *s.AppUpdate.Unattended
}
// SetAppUpdateUnattended stores the household's choice and saves. It takes effect at the leg's next
// start (and before its next step, if a leg is running) — the leg reads it each time.
func (s *Settings) SetAppUpdateUnattended(on bool) error {
s.mu.Lock()
defer s.mu.Unlock()
if s.AppUpdate == nil {
s.AppUpdate = &AppUpdateSettings{}
}
v := on
s.AppUpdate.Unattended = &v
return s.save()
}
// SetConfigLanguage records the language controller.yaml says this box should start in. // SetConfigLanguage records the language controller.yaml says this box should start in.
// //
// Called once at startup and again after each config pull. It never touches s.Language: a // Called once at startup and again after each config pull. It never touches s.Language: a
-1
View File
@@ -479,7 +479,6 @@ func (s *Server) generateManualConfig() string {
b.WriteString(" - \"cloudflared\"\n") b.WriteString(" - \"cloudflared\"\n")
b.WriteString(" - \"felhom-controller\"\n") b.WriteString(" - \"felhom-controller\"\n")
b.WriteString(" - \"filebrowser\"\n") b.WriteString(" - \"filebrowser\"\n")
b.WriteString(" update_window: \"03:00-05:00\"\n")
b.WriteString("\nbackup:\n") b.WriteString("\nbackup:\n")
b.WriteString(" enabled: true\n") b.WriteString(" enabled: true\n")
b.WriteString(" restic_password_file: \"/opt/docker/felhom-controller/data/restic-password\"\n") b.WriteString(" restic_password_file: \"/opt/docker/felhom-controller/data/restic-password\"\n")
+8
View File
@@ -158,6 +158,14 @@ type AppConfig struct {
// written only by a successful undo, cleared by the next successful update. The page shows one // written only by a successful undo, cleared by the next successful update. The page shows one
// line from it; the future automatic caller reads it so it never re-presses the same step. // line from it; the future automatic caller reads it so it never re-presses the same step.
LastUpdateUndone *UpdateUndone `yaml:"last_update_undone,omitempty" json:"last_update_undone,omitempty"` LastUpdateUndone *UpdateUndone `yaml:"last_update_undone,omitempty" json:"last_update_undone,omitempty"`
// FailedStep (v0.271.0, R-680) is the ladder step whose update was UNDONE or HELD — written by the
// undo and by the hold, cleared by the next successful update. The automatic leg never presses it
// again while the catalog's ladder for this app is the one it failed on (Ladder = LadderPrint); a
// person still can.
FailedStep *FailedStep `yaml:"failed_update_step,omitempty" json:"failed_update_step,omitempty"`
// LastAutoUpdate (v0.271.0, `09` §6.4 part 7) is the automatic leg's last step on this app — the
// line on the app page („Automatikus frissítés %s-kor — sikeres").
LastAutoUpdate *AutoUpdateRecord `yaml:"last_auto_update,omitempty" json:"last_auto_update,omitempty"`
} }
// InstalledImage is one compose service's observed image. See AppConfig.InstalledImages. // InstalledImage is one compose service's observed image. See AppConfig.InstalledImages.
+24
View File
@@ -44,6 +44,30 @@ type LadderEntry struct {
Verdict string `yaml:"verdict" json:"verdict"` Verdict string `yaml:"verdict" json:"verdict"`
// TestedAt is when the step was proven (RFC3339) — the badge compares it with the install (v0.269.0). // TestedAt is when the step was proven (RFC3339) — the badge compares it with the install (v0.269.0).
TestedAt string `yaml:"tested_at" json:"tested_at"` TestedAt string `yaml:"tested_at" json:"tested_at"`
// Marks are decision 13's two exceptions the test sets on a step (v0.271.0 reads them): the
// automatic leg never takes a step that needs a person, and takes a files-may-change step only when
// a fresh WHOLE copy exists. A person's press ignores both — the marks bind the leg only.
Marks LadderMarks `yaml:"marks" json:"marks"`
}
// LadderMarks is the `marks` object of a ladder entry. NeedsPerson is JSON null (nil) or the tester's
// reason; an entry with no `marks` key reads as no marks.
type LadderMarks struct {
FilesMayChange bool `yaml:"files_may_change" json:"files_may_change"`
NeedsPerson *string `yaml:"needs_person" json:"needs_person"`
MemoryTight bool `yaml:"memory_tight" json:"memory_tight"`
}
// LadderPrint is a short fingerprint of an app's whole ladder as the box parsed it — what R-680's
// failed-step record is tied to: when the catalog changes the ladder (a new step, a re-test, a mark),
// the print changes and the automatic leg may try again. "" = no ladder.
func LadderPrint(ladder []LadderEntry) string {
if len(ladder) == 0 {
return ""
}
b, _ := json.Marshal(ladder)
sum := sha256.Sum256(b)
return hex.EncodeToString(sum[:])[:16]
} }
type ladderDoc struct { type ladderDoc struct {
+2
View File
@@ -227,6 +227,8 @@ type Manager struct {
updateHealthMetaFn func(ctx context.Context, name string, timeout time.Duration, meta *Metadata) (bool, string) updateHealthMetaFn func(ctx context.Context, name string, timeout time.Duration, meta *Metadata) (bool, string)
// unhealthy (v0.269.0, decision 28): RestartCount / OOM-kill samples per app for the crash-loop stop. // unhealthy (v0.269.0, decision 28): RestartCount / OOM-kill samples per app for the crash-loop stop.
unhealthy unhealthyWatch unhealthy unhealthyWatch
// leg (v0.271.0, `09` §6.4 part 7): the automatic update leg's state (unattended.go).
leg updateLegState
// selfUpdating (v0.261.0) reports whether the CONTROLLER is swapping itself. Set by // selfUpdating (v0.261.0) reports whether the CONTROLLER is swapping itself. Set by
// SetSelfUpdatingCheck; nil means no gate. See update.go. // SetSelfUpdatingCheck; nil means no gate. See update.go.
selfUpdating func() bool selfUpdating func() bool
+510
View File
@@ -0,0 +1,510 @@
package stacks
import (
"context"
"errors"
"fmt"
"path/filepath"
"sort"
"strings"
"sync"
"time"
"gitea.dooplex.hu/admin/felhom-controller/internal/backupwindow"
)
// ── The automatic update leg (`09` §3 decisions 11, 12, 14, 15, 20; §6.4 part 7; controller v0.271.0) ──
//
// WHAT IT IS. One more leg of the nightly chain: after the off-site copy, before the full-system backup.
// It presses the SAME public guarded Update a person presses (StartGuardedUpdate) — no second update
// path — one app at a time, ONE tested step per app per night, and never:
// - an app whose installed version has no ladder entry (older than the ladder, or no test record);
// - a step whose verdict is not `proven`, or that carries the `needs_person` mark;
// - a `files_may_change` step unless a fresh WHOLE copy of the app exists on the box (decision 25/26);
// - the step that was undone or held last time, while the catalog's ladder is unchanged (R-680);
// - a held app, an app that is current, ahead, or unorderable.
// It starts no step at or after W+5h (decision 20); a step already running finishes.
//
// WHERE IT IS CALLED FROM. The `offbox-backup` scheduler job, after the off-site copy, on EVERY path —
// configured, not configured, failed, skipped, panicked (cmd/controller chainUpdateLeg, pinned by
// TestChainUpdateLeg_*). The full-system backup's gate asks UpdateLegState and waits while the leg runs,
// until W+5h (quiesce.Options.UpdateLegFn).
//
// WHAT THE HOUSEHOLD HEARS. Nothing new: an undone or held step already sends app_update_undone /
// app_update_held from the update itself. A successful automatic step sends NO mail; the app page shows
// one line from app.yaml's last_auto_update. The operator gets one summary line per night in the log and
// in the hub report (LastUpdateLegSummary).
// Leg skip reasons — stable keys for the log, the summary and tests.
const (
LegSkipHeld = "held"
LegSkipUnpinned = "unpinned"
LegSkipUnknownOrder = "order_unknown"
LegSkipAhead = "ahead"
LegSkipNoTestRecord = "no_test_record"
LegSkipOlderThanLadder = "older_than_ladder"
LegSkipNotProven = "not_proven"
LegSkipNeedsPerson = "needs_person"
LegSkipFilesNoCopy = "files_may_change_no_whole_copy"
LegSkipFailedBefore = "failed_before"
LegSkipWindowEnd = "window_end"
LegSkipSwitchedOff = "switched_off"
LegSkipCancelled = "cancelled"
legCurrent = "current" // not a skip: nothing to do
LegOutcomeDone = "done"
LegOutcomeUndone = "undone"
LegOutcomeHeld = "held"
LegOutcomeFailed = "failed"
LegOutcomeSkipped = "skipped"
)
// legTransient are the refusals a later press may not meet (R-609's split): retried ONCE later in the leg.
var legTransient = map[string]bool{"busy": true, "updating": true, "deploying": true, "migrating": true, "self_updating": true}
// UpdateLegOptions wires the leg (main.go). INIT-ONLY via SetUpdateLeg.
type UpdateLegOptions struct {
// Enabled is the per-box switch `app_update.unattended` (decision 12, default ON). Read at the start
// and before every press, so switching it off stops the leg before its next step.
Enabled func() bool
// WindowStart is the effective backup window start W "HH:MM" — the leg starts no step at or after W+5h.
WindowStart func() string
// FreshWholeCopy answers decision 13's `files_may_change` mark: is there a fresh copy on this box that
// brings the app back WHOLE (decision 25/26's truth table)? nil = no → such a step is always skipped.
FreshWholeCopy func(ctx context.Context, name string) (bool, string)
// Location is the wall clock W is read in (Europe/Budapest); nil = time.Local.
Location *time.Location
// Poll is how often the leg looks whether its step has ended (default 5 s).
Poll time.Duration
// RetryWait is how long the leg waits before its one retry of transient refusals (default 2 min).
RetryWait time.Duration
// Now is the clock (tests); nil = time.Now.
Now func() time.Time
}
// LegStep is one app's line in the night's summary.
type LegStep struct {
App string `json:"app"`
Outcome string `json:"outcome"` // done | undone | held | failed | skipped
Reason string `json:"reason,omitempty"`
From map[string]string `json:"from,omitempty"`
To map[string]string `json:"to,omitempty"`
Seconds float64 `json:"seconds,omitempty"`
}
// UpdateLegSummary is one night's leg — the operator's summary line and the hub report's `update_leg`.
type UpdateLegSummary struct {
Trigger string `json:"trigger"`
StartedAt time.Time `json:"started_at"`
EndedAt time.Time `json:"ended_at"`
Deadline time.Time `json:"deadline"`
Enabled bool `json:"enabled"`
Done int `json:"done"`
Undone int `json:"undone"`
Held int `json:"held"`
Failed int `json:"failed"`
Skipped int `json:"skipped"`
// Stopped says why the leg ended early: window_end | switched_off | cancelled | "" (it ran through).
Stopped string `json:"stopped,omitempty"`
Steps []LegStep `json:"steps"`
}
// Line is the one operator-English summary line.
func (s *UpdateLegSummary) Line() string {
var skips []string
for _, st := range s.Steps {
if st.Outcome == LegOutcomeSkipped {
skips = append(skips, st.App+"="+st.Reason)
}
}
stop := ""
if s.Stopped != "" {
stop = " stopped=" + s.Stopped
}
return fmt.Sprintf("update leg (%s): done=%d undone=%d held=%d failed=%d skipped=%d%s in %s [skipped: %s]",
s.Trigger, s.Done, s.Undone, s.Held, s.Failed, s.Skipped, stop,
s.EndedAt.Sub(s.StartedAt).Round(time.Second), strings.Join(skips, ", "))
}
func (s *UpdateLegSummary) add(st LegStep) {
s.Steps = append(s.Steps, st)
switch st.Outcome {
case LegOutcomeDone:
s.Done++
case LegOutcomeUndone:
s.Undone++
case LegOutcomeHeld:
s.Held++
case LegOutcomeFailed:
s.Failed++
default:
s.Skipped++
}
}
type updateLegState struct {
run sync.Mutex // single-flight: one leg at a time
mu sync.Mutex // guards the fields below
opts *UpdateLegOptions
active bool
stepRunning bool
last *UpdateLegSummary
}
// SetUpdateLeg wires the leg. INIT-ONLY (main.go). Without it RunUpdateLeg does nothing and says so.
func (m *Manager) SetUpdateLeg(o UpdateLegOptions) {
if o.Poll <= 0 {
o.Poll = 5 * time.Second
}
if o.RetryWait <= 0 {
o.RetryWait = 2 * time.Minute
}
if o.Location == nil {
o.Location = time.Local
}
if o.Now == nil {
o.Now = time.Now
}
m.leg.mu.Lock()
m.leg.opts = &o
m.leg.mu.Unlock()
}
// UpdateLegState is what the full-system backup's gate asks (decision 20): is the leg running, and is one
// of its steps in flight right now.
func (m *Manager) UpdateLegState() (active, stepRunning bool) {
m.leg.mu.Lock()
defer m.leg.mu.Unlock()
return m.leg.active, m.leg.stepRunning
}
// LastUpdateLegSummary is the last leg's summary, or nil when none ran since the controller started.
func (m *Manager) LastUpdateLegSummary() *UpdateLegSummary {
m.leg.mu.Lock()
defer m.leg.mu.Unlock()
if m.leg.last == nil {
return nil
}
c := *m.leg.last
c.Steps = append([]LegStep(nil), m.leg.last.Steps...)
return &c
}
func (m *Manager) setLegFlags(active, step bool) {
m.leg.mu.Lock()
m.leg.active, m.leg.stepRunning = active, step
m.leg.mu.Unlock()
}
// LegDeadline is the instant the leg starts no more steps: W+5h of the night that contains `now` (the
// most recent W at or before now). An unparseable W falls back to the default window.
func LegDeadline(now time.Time, window string, loc *time.Location) time.Time {
if loc == nil {
loc = time.Local
}
startMin, err := backupwindow.ParseHHMM(window)
if err != nil {
startMin, _ = backupwindow.ParseHHMM(backupwindow.DefaultWindow)
}
n := now.In(loc)
w := time.Date(n.Year(), n.Month(), n.Day(), startMin/60, startMin%60, 0, 0, loc)
if w.After(n) {
w = w.AddDate(0, 0, -1)
}
return w.Add(time.Duration(backupwindow.UpdateLegStopOffsetMin) * time.Minute)
}
// RunUpdateLeg runs one night's leg and returns its summary (nil when another leg is running or the leg
// is not wired). trigger names the caller for the log ("after-offsite").
func (m *Manager) RunUpdateLeg(ctx context.Context, trigger string) *UpdateLegSummary {
if !m.leg.run.TryLock() {
m.logger.Printf("[WARN] [update-leg] a leg is already running — this call (%s) does nothing", trigger)
return nil
}
defer m.leg.run.Unlock()
m.leg.mu.Lock()
o := m.leg.opts
m.leg.mu.Unlock()
if o == nil {
m.logger.Printf("[WARN] [update-leg] not wired (SetUpdateLeg) — no automatic app updates")
return nil
}
sum := &UpdateLegSummary{Trigger: trigger, StartedAt: o.Now(), Steps: []LegStep{}}
finish := func() *UpdateLegSummary {
sum.EndedAt = o.Now()
m.logger.Printf("[INFO] [update-leg] %s", sum.Line())
m.leg.mu.Lock()
m.leg.last = sum
m.leg.mu.Unlock()
return sum
}
sum.Enabled = o.Enabled == nil || o.Enabled()
if !sum.Enabled {
sum.Stopped = LegSkipSwitchedOff
m.logger.Printf("[INFO] [update-leg] automatic app updates are switched OFF on this box (app_update.unattended=false) — no app is pressed tonight")
return finish()
}
window := ""
if o.WindowStart != nil {
window = o.WindowStart()
}
sum.Deadline = LegDeadline(sum.StartedAt, window, o.Location)
m.setLegFlags(true, false)
defer m.setLegFlags(false, false)
m.logger.Printf("[INFO] [update-leg] started (%s): window %s, no step starts at or after %s", trigger, window, sum.Deadline.In(o.Location).Format("15:04"))
// The freshest view of the catalog and the pins — a sync may have landed since the last scan.
if err := m.ScanStacks(); err != nil {
m.logger.Printf("[WARN] [update-leg] rescan before the leg failed: %v — using the last scan", err)
}
var names []string
for _, st := range m.GetStacks() {
if st.Deployed && !st.Protected && !m.cfg.IsProtectedStack(st.Name) {
names = append(names, st.Name)
}
}
sort.Strings(names)
var retry []string
for pass := 0; pass < 2 && sum.Stopped == ""; pass++ {
list := names
if pass == 1 {
list, retry = retry, nil
if len(list) == 0 {
break
}
m.logger.Printf("[INFO] [update-leg] %d app(s) were refused for a passing reason — one retry in %s: %v", len(list), o.RetryWait, list)
select {
case <-ctx.Done():
case <-time.After(o.RetryWait):
}
}
for i, name := range list {
if stop := m.legStop(ctx, o, sum); stop != "" {
sum.Stopped = stop
for _, rest := range list[i:] {
sum.add(LegStep{App: rest, Outcome: LegOutcomeSkipped, Reason: stop})
}
if pass == 0 {
for _, rest := range retry {
sum.add(LegStep{App: rest, Outcome: LegOutcomeSkipped, Reason: stop})
}
}
break
}
entry, reason := m.legCandidate(ctx, name, o)
if reason == legCurrent {
continue
}
if reason != "" {
m.logger.Printf("[INFO] [update-leg] %s: skipped — %s", name, reason)
sum.add(LegStep{App: name, Outcome: LegOutcomeSkipped, Reason: reason})
continue
}
t0 := o.Now()
if err := m.StartGuardedUpdate(name); err != nil {
why := "refused"
var ref *UpdateRefusal
if errors.As(err, &ref) {
why = ref.Reason
}
if legTransient[why] && pass == 0 {
m.logger.Printf("[INFO] [update-leg] %s: refused (%s) — a passing reason, retried once later tonight", name, why)
retry = append(retry, name)
continue
}
m.logger.Printf("[INFO] [update-leg] %s: skipped — the update refused (%s)", name, why)
sum.add(LegStep{App: name, Outcome: LegOutcomeSkipped, Reason: "refused:" + why})
continue
}
m.logger.Printf("[INFO] [update-leg] %s: step pressed %s → %s", name, summarisePin(entry.From), summarisePin(entry.To))
m.setLegFlags(true, true)
outcome := m.legWait(ctx, o, name)
m.setLegFlags(true, false)
step := LegStep{App: name, Outcome: outcome, From: entry.From, To: entry.To, Seconds: o.Now().Sub(t0).Round(100 * time.Millisecond).Seconds()}
sum.add(step)
m.recordAutoUpdate(name, &AutoUpdateRecord{At: t0.UTC().Format(time.RFC3339), Outcome: outcome, From: entry.From, To: entry.To})
m.logger.Printf("[INFO] [update-leg] %s: step ended %s after %.1f s", name, outcome, step.Seconds)
}
}
return finish()
}
// legStop is the leg's own stop check before every press: cancelled, switched off, or W+5h reached.
func (m *Manager) legStop(ctx context.Context, o *UpdateLegOptions, sum *UpdateLegSummary) string {
switch {
case ctx.Err() != nil:
m.logger.Printf("[WARN] [update-leg] cancelled (the controller is stopping) — the remaining apps wait for the next night")
return LegSkipCancelled
case o.Enabled != nil && !o.Enabled():
m.logger.Printf("[INFO] [update-leg] the switch was turned OFF during the leg — no further step tonight")
return LegSkipSwitchedOff
case !o.Now().Before(sum.Deadline):
m.logger.Printf("[INFO] [update-leg] W+5h reached (%s) — no new step starts; the full-system backup keeps its hour, the rest waits for the next night", sum.Deadline.In(o.Location).Format("15:04"))
return LegSkipWindowEnd
}
return ""
}
// legWait waits until the pressed step has ended and returns its outcome. A cancelled context stops the
// WAIT only — the update job itself runs on, and its journal finishes it after a restart.
func (m *Manager) legWait(ctx context.Context, o *UpdateLegOptions, name string) string {
for m.IsUpdating(name) {
select {
case <-ctx.Done():
return LegOutcomeFailed
case <-time.After(o.Poll):
}
}
st, ok := m.GetStack(name)
if !ok {
return LegOutcomeFailed
}
switch {
case st.UpdatePhase == UpdatePhaseDone:
return LegOutcomeDone
case st.UpdatePhase == UpdatePhaseUndone:
return LegOutcomeUndone
case st.updateHeld || st.UpdateErrorKey == UpdateErrorKeyHeld || st.HoldReason != "":
return LegOutcomeHeld
}
return LegOutcomeFailed
}
// legCandidate decides whether the leg may press this app tonight, and which step it would take. It
// returns ("", entry) to press, legCurrent when there is nothing to do, or a skip reason.
func (m *Manager) legCandidate(ctx context.Context, name string, o *UpdateLegOptions) (LadderEntry, string) {
st, ok := m.GetStack(name)
if !ok || !st.Deployed {
return LadderEntry{}, legCurrent
}
if st.HoldReason != "" || st.updateHeld {
return LadderEntry{}, LegSkipHeld
}
if g := m.guards(); g != nil {
if held, _ := g.HoldFor(name); held {
return LadderEntry{}, LegSkipHeld
}
}
switch CatalogOrder(*st) {
case UpdateOrderCurrent:
return LadderEntry{}, legCurrent
case UpdateOrderAhead:
return LadderEntry{}, LegSkipAhead
case UpdateOrderBehind:
default:
return LadderEntry{}, LegSkipUnknownOrder
}
if st.AppConfig == nil || len(st.AppConfig.PinnedImages) == 0 {
return LadderEntry{}, LegSkipUnpinned
}
pinned := st.AppConfig.PinnedImages
tplDir := filepath.Dir(m.CatalogTemplatePath(name, "docker-compose.yml"))
ladder, err := LoadLadder(filepath.Join(tplDir, ".felhom.yml"))
if err != nil || len(ladder) == 0 {
return LadderEntry{}, LegSkipNoTestRecord
}
idx := -1
for i := len(ladder) - 1; i >= 0; i-- { // the SAME choice nextLadderStep makes
if sameRefs(ladder[i].From, pinned) {
idx = i
break
}
}
if idx < 0 {
if sameRefs(ladder[len(ladder)-1].To, pinned) {
return LadderEntry{}, LegSkipNoTestRecord // at the head, behind only by something no step records
}
m.logger.Printf("[INFO] [update-leg] %s: the installed version %s matches no update_ladder entry — an app older than the ladder is never pressed by the leg (a person can)", name, summarisePin(pinned))
return LadderEntry{}, LegSkipOlderThanLadder
}
e := ladder[idx]
if e.Verdict != "proven" {
return e, LegSkipNotProven
}
if e.Marks.NeedsPerson != nil && strings.TrimSpace(*e.Marks.NeedsPerson) != "" {
return e, LegSkipNeedsPerson
}
if fs := st.AppConfig.FailedStep; fs != nil && sameRefs(fs.To, e.To) && fs.Ladder == LadderPrint(ladder) {
return e, LegSkipFailedBefore
}
if e.Marks.FilesMayChange {
whole, why := false, "no whole-copy check wired"
if o.FreshWholeCopy != nil {
whole, why = o.FreshWholeCopy(ctx, name)
}
if !whole {
m.logger.Printf("[INFO] [update-leg] %s: the step may change the app's files and no fresh whole copy exists (%s)", name, why)
return e, LegSkipFilesNoCopy
}
}
return e, ""
}
// ── The records the leg and the update write into app.yaml ─────────────────────────────────────────
// FailedStep is R-680's record: the step whose update was undone or held.
type FailedStep struct {
To map[string]string `yaml:"to" json:"to"`
Ladder string `yaml:"ladder" json:"ladder"` // LadderPrint of the ladder it failed on
At string `yaml:"at" json:"at"`
Outcome string `yaml:"outcome" json:"outcome"` // undone | held
}
// AutoUpdateRecord is the leg's last step on an app — the page's line.
type AutoUpdateRecord struct {
At string `yaml:"at" json:"at"`
Outcome string `yaml:"outcome" json:"outcome"`
From map[string]string `yaml:"from,omitempty" json:"from,omitempty"`
To map[string]string `yaml:"to,omitempty" json:"to,omitempty"`
}
// mutateAppConfig loads app.yaml, applies fn (false = nothing to write), saves it and mirrors the
// in-memory copy. A failed write is logged and never fails the caller — these are records.
func (m *Manager) mutateAppConfig(name, dir, what string, fn func(cfg *AppConfig) bool) {
cfg := LoadAppConfig(dir)
if cfg == nil || !fn(cfg) {
return
}
meta := LoadMetadata(dir)
if err := SaveAppConfig(dir, cfg, m.encKey, SensitiveEnvVars(&meta)); err != nil {
m.logger.Printf("[ERROR] [stacks] %s: recording %s failed: %v", name, what, err)
return
}
m.mu.Lock()
if st, ok := m.stacks[name]; ok && st.AppConfig != nil {
fn(st.AppConfig)
}
m.mu.Unlock()
}
// recordFailedStep writes R-680's record for the step `to` (undone or held), tied to the ladder the
// catalog carries now.
func (m *Manager) recordFailedStep(name, dir string, to map[string]string, outcome string) {
if len(to) == 0 {
return
}
ladder, _ := LoadLadder(m.CatalogTemplatePath(name, ".felhom.yml"))
rec := &FailedStep{To: to, Ladder: LadderPrint(ladder), At: m.now().UTC().Format(time.RFC3339), Outcome: outcome}
m.mutateAppConfig(name, dir, "failed_update_step", func(cfg *AppConfig) bool { cfg.FailedStep = rec; return true })
m.logger.Printf("[INFO] [stacks] update %s: step %s recorded as %s — the automatic leg will not press it again until the catalog's ladder changes (ladder %s)", name, summarisePin(to), outcome, rec.Ladder)
}
// clearFailedStep drops R-680's record after a successful update.
func (m *Manager) clearFailedStep(name, dir string) {
m.mutateAppConfig(name, dir, "failed_update_step", func(cfg *AppConfig) bool {
if cfg.FailedStep == nil {
return false
}
cfg.FailedStep = nil
return true
})
}
func (m *Manager) recordAutoUpdate(name string, rec *AutoUpdateRecord) {
st, ok := m.GetStack(name)
if !ok {
return
}
m.mutateAppConfig(name, filepath.Dir(st.ComposePath), "last_auto_update", func(cfg *AppConfig) bool { cfg.LastAutoUpdate = rec; return true })
}
@@ -0,0 +1,454 @@
package stacks
import (
"bytes"
"context"
"path/filepath"
"strconv"
"strings"
"sync"
"testing"
"time"
)
// v0.271.0 — the automatic update leg (`09` §3 decisions 11–15, 20; §6.4 part 7). Every test runs the
// REAL leg over the REAL guarded update job with the process boundaries faked (ladderManager: nextcloud
// pinned at A, the catalog at C, a two-step ladder A→B→C, B's own definition in steps/), and reads the
// EFFECT back: the pin in app.yaml, the definitions `up` ran on, app.yaml's records, the summary.
// legNow is inside the night of window 02:30 (the leg starts at W+105m = 04:15).
var legNow = time.Date(2026, 9, 25, 4, 15, 0, 0, time.UTC)
func legOpts(m *Manager, mut func(o *UpdateLegOptions)) {
o := UpdateLegOptions{
Enabled: func() bool { return true },
WindowStart: func() string { return "02:30" },
Location: time.UTC,
Poll: 2 * time.Millisecond,
RetryWait: time.Millisecond,
Now: func() time.Time { return legNow },
}
if mut != nil {
mut(&o)
}
m.SetUpdateLeg(o)
}
// writeLadder replaces the catalog's .felhom.yml with the given ladder lines.
func writeLadder(t *testing.T, m *Manager, lines ...string) {
t.Helper()
catDir := filepath.Dir(m.CatalogTemplatePath("nextcloud", "docker-compose.yml"))
mustWrite(t, filepath.Join(catDir, ".felhom.yml"), "display_name: Nextcloud\nupdate_ladder:\n"+strings.Join(lines, ""))
}
// ladderLineMarks is ladderLine with a verdict and a marks object.
func ladderLineMarks(from, to, verdict, marks string) string {
return ` - {"from": {"web": "` + from + `"}, "to": {"web": "` + to + `"}, "digest": {"web": "sha256:` +
strings.Repeat("a", 64) + `"}, "verdict": "` + verdict + `", "marks": ` + marks + `}` + "\n"
}
func legStepFor(s *UpdateLegSummary, app string) LegStep {
for _, st := range s.Steps {
if st.App == app {
return st
}
}
return LegStep{}
}
// legManager is ladderManager plus what a real box has and the fakes do not produce: an installed-image
// record (CatalogOrder answers Unknown without one, and the leg never presses Unknown). Every `up` records
// the image it brought up, as recordInstalledImages does on a box from the running container.
func legManager(t *testing.T) (*Manager, string, *fakeGuards, *[]string, *bytes.Buffer) {
t.Helper()
m, dir, g, ups, logBuf := ladderManager(t, true)
setInstalled := func(img string) {
cfg := LoadAppConfig(dir)
cfg.InstalledImages = map[string]InstalledImage{"web": {Ref: img}}
if err := SaveAppConfig(dir, cfg, m.encKey, nil); err != nil {
t.Fatal(err)
}
}
setInstalled(ladderA)
inner := m.updateComposeFn
m.updateComposeFn = func(d string, env []string, args ...string) (string, error) {
out, err := inner(d, env, args...)
if err == nil && args[0] == "up" {
if imgs, perr := ParseComposeImages(ComposePathIn(d)); perr == nil {
setInstalled(strings.SplitN(imgs["web"], "@", 2)[0])
}
}
return out, err
}
return m, dir, g, ups, logBuf
}
func mustLeg(t *testing.T, m *Manager) *UpdateLegSummary {
t.Helper()
if err := m.ScanStacks(); err != nil {
t.Fatal(err)
}
s := m.RunUpdateLeg(context.Background(), "test")
if s == nil {
t.Fatal("the leg did not run")
}
return s
}
// TestLeg_OneStepPerAppPerNight — a box two steps behind takes ONE step a night (decision 14 + the
// brief: "one tested step per app"), with the step's own definition, and the summary says done=1.
//
// COMPANION RED-PROOF (REPORT.md): make the leg loop while the app stays behind — this test fails at
// "the leg took 2 steps in one night".
func TestLeg_OneStepPerAppPerNight(t *testing.T) {
m, dir, _, ups, _ := legManager(t)
legOpts(m, nil)
s := mustLeg(t, m)
if len(*ups) != 1 {
t.Fatalf("the leg took %d steps in one night (ups=%v), want exactly one", len(*ups), *ups)
}
if pinOf(t, dir) != ladderB {
t.Fatalf("pinned %s, want the first tested step B", pinOf(t, dir))
}
if s.Done != 1 || legStepFor(s, "nextcloud").Outcome != LegOutcomeDone {
t.Fatalf("summary %+v, want done=1 for nextcloud", s)
}
cfg := LoadAppConfig(dir)
if cfg.LastAutoUpdate == nil || cfg.LastAutoUpdate.Outcome != LegOutcomeDone || cfg.LastAutoUpdate.To["web"] != ladderB {
t.Fatalf("app.yaml must carry the page's record of the automatic step, got %+v", cfg.LastAutoUpdate)
}
// the next night climbs the next step
s2 := mustLeg(t, m)
if pinOf(t, dir) != ladderC || s2.Done != 1 {
t.Fatalf("night 2 ended on %s (done=%d), want C", pinOf(t, dir), s2.Done)
}
// and the third night finds nothing to do — no press, not even counted as a skip
s3 := mustLeg(t, m)
if len(*ups) != 2 || s3.Done+s3.Skipped != 0 {
t.Fatalf("night 3 at the head pressed or counted something: ups=%v summary=%+v", *ups, s3)
}
}
// TestR678_StepsLeftFreshAtDone — the page's steps-left count and the pin are current the moment the
// update says `done`, with no scan in between. MEASURED 2026-09-24: ~50 s stale, six re-presses.
//
// COMPANION RED-PROOF (REPORT.md): drop the ScanStacks call in verifyAndConclude — this test fails at
// "steps left read 2 right after the step ended done".
func TestR678_StepsLeftFreshAtDone(t *testing.T) {
m, _, _, _, _ := ladderManager(t, true)
if err := m.ScanStacks(); err != nil {
t.Fatal(err)
}
if st, _ := m.GetStack("nextcloud"); st.LadderStepsLeft != 2 {
t.Fatalf("before: steps left %d, want 2", st.LadderStepsLeft)
}
if err := m.StartGuardedUpdate("nextcloud"); err != nil {
t.Fatal(err)
}
st := waitUpdateDone(t, m, "nextcloud")
if st.UpdatePhase != UpdatePhaseDone {
t.Fatalf("ended %q", st.UpdatePhase)
}
if st.LadderStepsLeft != 1 {
t.Fatalf("steps left read %d right after the step ended done, want 1", st.LadderStepsLeft)
}
if st.AppConfig == nil || st.AppConfig.PinnedImages["web"] != ladderB {
t.Fatalf("the in-memory pin is stale after done: %+v", st.AppConfig)
}
}
// TestR680_FailedStepIsNotPressedAgain — B fails and is undone: the box records the failed step, the
// next night's leg skips it (failed_before) and nothing is brought up; a PERSON can still press it; and
// when the catalog's ladder changes, the leg may try again.
//
// COMPANION RED-PROOF (REPORT.md): drop the FailedStep check in legCandidate — this test fails at
// "night 2 pressed the step that was undone on night 1".
func TestR680_FailedStepIsNotPressedAgain(t *testing.T) {
m, dir, _, ups, _ := legManager(t)
m.undoCopier = newFakeCopier(map[string]string{undoVol: "OLD"})
m.updateHealthFn = func(context.Context, string, time.Duration) (bool, string) { return false, "B unhealthy" }
m.updateUndoHealthFn = func(context.Context, string, time.Duration, *Metadata) (bool, string) { return true, "A answers" }
legOpts(m, nil)
s1 := mustLeg(t, m)
if s1.Undone != 1 || pinOf(t, dir) != ladderA {
t.Fatalf("night 1: %+v on %s, want undone back on A", s1, pinOf(t, dir))
}
cfg := LoadAppConfig(dir)
if cfg.FailedStep == nil || cfg.FailedStep.To["web"] != ladderB || cfg.FailedStep.Outcome != "undone" || cfg.FailedStep.Ladder == "" {
t.Fatalf("the failed step must be recorded on the box, got %+v", cfg.FailedStep)
}
upsAfter1 := len(*ups)
s2 := mustLeg(t, m)
if len(*ups) != upsAfter1 {
t.Fatalf("night 2 pressed the step that was undone on night 1 (ups=%v)", *ups)
}
if st := legStepFor(s2, "nextcloud"); st.Outcome != LegOutcomeSkipped || st.Reason != LegSkipFailedBefore {
t.Fatalf("night 2 must skip with %q, got %+v", LegSkipFailedBefore, st)
}
// a person still can — the record binds the leg only
if err := m.StartGuardedUpdate("nextcloud"); err != nil {
t.Fatalf("a person's press must not be refused by the failed-step record: %v", err)
}
waitUpdateDone(t, m, "nextcloud")
// the catalog re-tests the step (a new tested_at): the ladder's print changes → the leg tries again
writeLadder(t, m,
` - {"from": {"web": "`+ladderA+`"}, "to": {"web": "`+ladderB+`"}, "digest": {"web": "sha256:`+strings.Repeat("a", 64)+`"}, "verdict": "proven", "tested_at": "2026-09-26T01:00:00Z"}`+"\n",
ladderLine(ladderB, ladderC))
before := len(*ups)
mustLeg(t, m)
if len(*ups) <= before || (*ups)[before] != ladderB {
t.Fatalf("after the catalog changed the ladder the leg must try the step again (ups=%v)", *ups)
}
}
// TestLeg_NeedsPersonIsNeverTaken — decision 13's mark: the leg never takes a needs_person step.
//
// COMPANION RED-PROOF (REPORT.md): drop the NeedsPerson check — this test fails at "a needs_person step
// was pressed by the leg".
func TestLeg_NeedsPersonIsNeverTaken(t *testing.T) {
m, dir, _, ups, _ := legManager(t)
writeLadder(t, m,
ladderLineMarks(ladderA, ladderB, "proven", `{"files_may_change": false, "needs_person": "the admin must re-login after this step", "memory_tight": false}`),
ladderLine(ladderB, ladderC))
legOpts(m, nil)
s := mustLeg(t, m)
if len(*ups) != 0 || pinOf(t, dir) != ladderA {
t.Fatalf("a needs_person step was pressed by the leg (ups=%v)", *ups)
}
if st := legStepFor(s, "nextcloud"); st.Reason != LegSkipNeedsPerson {
t.Fatalf("skip reason %+v, want %q", st, LegSkipNeedsPerson)
}
}
// TestLeg_FilesMayChangeNeedsAWholeCopy — decision 13's other mark: taken only when a fresh WHOLE copy
// exists (the backup side's truth table, asked through FreshWholeCopy).
func TestLeg_FilesMayChangeNeedsAWholeCopy(t *testing.T) {
m, dir, _, ups, _ := legManager(t)
writeLadder(t, m,
ladderLineMarks(ladderA, ladderB, "proven", `{"files_may_change": true, "needs_person": null, "memory_tight": false}`),
ladderLine(ladderB, ladderC))
whole := false
legOpts(m, func(o *UpdateLegOptions) {
o.FreshWholeCopy = func(context.Context, string) (bool, string) { return whole, "fake" }
})
s := mustLeg(t, m)
if len(*ups) != 0 || legStepFor(s, "nextcloud").Reason != LegSkipFilesNoCopy {
t.Fatalf("without a whole copy the step must be skipped: ups=%v summary=%+v", *ups, s)
}
whole = true
s = mustLeg(t, m)
if len(*ups) != 1 || pinOf(t, dir) != ladderB || s.Done != 1 {
t.Fatalf("with a fresh whole copy the step must be taken: ups=%v pin=%s", *ups, pinOf(t, dir))
}
// unwired check = no whole copy (fail closed)
m2, _, _, ups2, _ := legManager(t)
writeLadder(t, m2,
ladderLineMarks(ladderA, ladderB, "proven", `{"files_may_change": true, "needs_person": null, "memory_tight": false}`),
ladderLine(ladderB, ladderC))
legOpts(m2, nil)
mustLeg(t, m2)
if len(*ups2) != 0 {
t.Fatal("with no whole-copy check wired a files_may_change step must never be taken")
}
}
// TestLeg_OlderThanLadderIsNotTouched — an installed version matching no ladder entry: the leg leaves it
// alone and says so by name (a person's press still jumps, TestLadder_UnknownInstalledJumpsAndSaysSo).
func TestLeg_OlderThanLadderIsNotTouched(t *testing.T) {
m, dir, _, ups, logBuf := legManager(t)
old := "services:\n web:\n image: nextcloud:30.0.0-apache\nvolumes:\n db:\n"
mustWrite(t, ComposePathIn(dir), old)
mustWrite(t, AppliedComposePath(dir), old)
mustWrite(t, filepath.Join(dir, "app.yaml"), "deployed: true\nenv: {}\npinned_images:\n web: nextcloud:30.0.0-apache\ninstalled_images:\n web:\n ref: nextcloud:30.0.0-apache\n")
legOpts(m, nil)
s := mustLeg(t, m)
if len(*ups) != 0 {
t.Fatalf("an app older than the ladder was pressed: %v", *ups)
}
if legStepFor(s, "nextcloud").Reason != LegSkipOlderThanLadder || !strings.Contains(logBuf.String(), "nextcloud:30.0.0-apache") {
t.Fatalf("skip must be older_than_ladder and logged by name; summary %+v", s)
}
}
// TestLeg_OnlyProvenStepsAreTaken — an `unrecorded` (backfilled, never tested) entry is not a test.
func TestLeg_OnlyProvenStepsAreTaken(t *testing.T) {
m, _, _, ups, _ := legManager(t)
writeLadder(t, m,
ladderLineMarks(ladderA, ladderB, "unrecorded", `{"files_may_change": false, "needs_person": null, "memory_tight": false}`),
ladderLine(ladderB, ladderC))
legOpts(m, nil)
s := mustLeg(t, m)
if len(*ups) != 0 || legStepFor(s, "nextcloud").Reason != LegSkipNotProven {
t.Fatalf("an unproven step was taken or mis-reasoned: ups=%v %+v", *ups, s)
}
}
// TestLeg_NoStepAtOrAfterW5h — decision 20: the leg starts nothing at W+5h.
//
// COMPANION RED-PROOF (REPORT.md): drop the deadline arm of legStop — this test fails at "a step was
// started at W+5h".
func TestLeg_NoStepAtOrAfterW5h(t *testing.T) {
m, _, _, ups, _ := legManager(t)
legOpts(m, func(o *UpdateLegOptions) {
o.Now = func() time.Time { return time.Date(2026, 9, 25, 7, 30, 0, 0, time.UTC) } // W 02:30 + 5h
})
s := mustLeg(t, m)
if len(*ups) != 0 {
t.Fatalf("a step was started at W+5h: %v", *ups)
}
if s.Stopped != LegSkipWindowEnd || legStepFor(s, "nextcloud").Reason != LegSkipWindowEnd {
t.Fatalf("summary must say window_end: %+v", s)
}
}
// TestLeg_SwitchOffPressesNothing — decision 12: the switch off means no press at all.
func TestLeg_SwitchOffPressesNothing(t *testing.T) {
m, _, _, ups, _ := legManager(t)
legOpts(m, func(o *UpdateLegOptions) { o.Enabled = func() bool { return false } })
s := mustLeg(t, m)
if len(*ups) != 0 || s.Enabled || s.Stopped != LegSkipSwitchedOff {
t.Fatalf("switch off: ups=%v summary=%+v", *ups, s)
}
}
// TestLeg_TransientRefusalIsRetriedOnce — `busy` is a passing reason: one retry, then a named skip.
func TestLeg_TransientRefusalIsRetriedOnce(t *testing.T) {
m, _, g, ups, logBuf := legManager(t)
g.busy = true
legOpts(m, nil)
s := mustLeg(t, m)
if len(*ups) != 0 || legStepFor(s, "nextcloud").Reason != "refused:busy" {
t.Fatalf("busy: ups=%v summary=%+v", *ups, s)
}
if n := strings.Count(logBuf.String(), "REFUSED (busy)"); n != 2 {
t.Fatalf("a busy refusal must be pressed exactly twice (once + one retry), saw %d", n)
}
}
// TestLeg_HeldAppIsNotPressed — a held app is terminal for the leg.
func TestLeg_HeldAppIsNotPressed(t *testing.T) {
m, _, g, ups, _ := legManager(t)
g.held, g.holdWhy = true, "HELD"
legOpts(m, nil)
s := mustLeg(t, m)
if len(*ups) != 0 || legStepFor(s, "nextcloud").Reason != LegSkipHeld {
t.Fatalf("held: ups=%v summary=%+v", *ups, s)
}
}
// TestLeg_GateSeesTheLegAndItsStep — UpdateLegState answers active while the leg runs and stepRunning
// while its step is in flight (the full-system backup's gate reads exactly this), and both go false after.
func TestLeg_GateSeesTheLegAndItsStep(t *testing.T) {
m, _, _, _, _ := legManager(t)
release := make(chan struct{})
var mu sync.Mutex
var seen [][2]bool
m.updateHealthFn = func(context.Context, string, time.Duration) (bool, string) {
a, s := m.UpdateLegState()
mu.Lock()
seen = append(seen, [2]bool{a, s})
mu.Unlock()
<-release
return true, "ok"
}
legOpts(m, nil)
done := make(chan struct{})
go func() { mustLeg(t, m); close(done) }()
deadline := time.Now().Add(3 * time.Second)
for {
mu.Lock()
n := len(seen)
mu.Unlock()
if n > 0 || time.Now().After(deadline) {
break
}
time.Sleep(2 * time.Millisecond)
}
close(release)
<-done
if len(seen) == 0 || !seen[0][0] || !seen[0][1] {
t.Fatalf("during the step's verify the gate must see (active, stepRunning) = (true, true), saw %v", seen)
}
if a, s := m.UpdateLegState(); a || s {
t.Fatalf("after the leg the gate must see (false, false), got (%v, %v)", a, s)
}
}
// TestD28_NoCrashLoopStopDuringAnAutomaticStep — decision 28's stop must not fire while the leg's step
// restarts the app (pull, up, verify, undo): an app that is Updating is not sampled and its history is
// dropped, so restarts climbing during the step never make a verdict.
//
// COMPANION RED-PROOF (REPORT.md): drop `st.Updating` from ObserveUnhealthy's skip — this test fails at
// "a crash-loop verdict fired during an automatic step".
func TestD28_NoCrashLoopStopDuringAnAutomaticStep(t *testing.T) {
m, _, _, _, _ := legManager(t)
if err := m.ScanStacks(); err != nil {
t.Fatal(err)
}
var cmu sync.Mutex
restarts := 0
m.execFn = func(name string, args ...string) (string, error) {
if name == "docker" && len(args) > 0 && args[0] == "inspect" {
cmu.Lock()
defer cmu.Unlock()
return "/nextcloud-web-1|" + strconv.Itoa(restarts) + "\n", nil
}
return "", nil
}
release := make(chan struct{})
inVerify := make(chan struct{}, 1)
m.updateHealthFn = func(context.Context, string, time.Duration) (bool, string) {
inVerify <- struct{}{}
<-release
return true, "ok"
}
legOpts(m, nil)
done := make(chan struct{})
go func() { m.RunUpdateLeg(context.Background(), "test"); close(done) }()
select {
case <-inVerify:
case <-time.After(3 * time.Second):
t.Fatal("the step never reached its verify")
}
// the container the samples read (set now: the leg's rescan rebuilt the stack before the step)
m.mu.Lock()
m.stacks["nextcloud"].Containers = []ContainerInfo{{Name: "nextcloud-web-1"}}
m.mu.Unlock()
t0 := time.Date(2026, 9, 25, 4, 20, 0, 0, time.UTC)
for i := 0; i < 10; i++ { // ten samples a minute apart, +3 restarts each: 27 restarts in 9 minutes
cmu.Lock()
restarts += 3
cmu.Unlock()
if v := m.ObserveUnhealthy(t0.Add(time.Duration(i)*time.Minute), nil); len(v) > 0 {
close(release)
<-done
t.Fatalf("a crash-loop verdict fired during an automatic step: %+v", v)
}
}
close(release)
<-done
}
// TestLegDeadline — W+5h of the night that contains now, across midnight too.
func TestLegDeadline(t *testing.T) {
loc := time.UTC
cases := []struct {
now time.Time
window string
want time.Time
}{
{time.Date(2026, 9, 25, 4, 15, 0, 0, loc), "02:30", time.Date(2026, 9, 25, 7, 30, 0, 0, loc)},
{time.Date(2026, 9, 25, 1, 0, 0, 0, loc), "23:00", time.Date(2026, 9, 25, 4, 0, 0, 0, loc)},
{time.Date(2026, 9, 25, 2, 0, 0, 0, loc), "02:30", time.Date(2026, 9, 24, 7, 30, 0, 0, loc)}, // before W: last night's
{time.Date(2026, 9, 25, 4, 15, 0, 0, loc), "junk", time.Date(2026, 9, 25, 7, 30, 0, 0, loc)}, // default 02:30
}
for _, c := range cases {
if got := LegDeadline(c.now, c.window, loc); !got.Equal(c.want) {
t.Errorf("LegDeadline(%s, %q) = %s, want %s", c.now.Format("15:04"), c.window, got, c.want)
}
}
}
+4
View File
@@ -553,9 +553,13 @@ func (m *Manager) tryUndo(ctx context.Context, name, dir, why string, entry *upd
m.recordInstalledImages(name, dir, env) m.recordInstalledImages(name, dir, env)
m.removeUndoCopies(name, entry.UndoCopies) m.removeUndoCopies(name, entry.UndoCopies)
m.recordUpdateUndone(name, dir, &UpdateUndone{To: entry.NewPin, At: start.UTC().Format(time.RFC3339), Why: why}) m.recordUpdateUndone(name, dir, &UpdateUndone{To: entry.NewPin, At: start.UTC().Format(time.RFC3339), Why: why})
m.recordFailedStep(name, dir, entry.NewPin, "undone") // R-680
m.removePreUpdateCopies(dir) m.removePreUpdateCopies(dir)
_ = m.RefreshStatus() _ = m.RefreshStatus()
m.clearJournal(name) m.clearJournal(name)
if err := m.ScanStacks(); err != nil { // R-678: the page reads the pin the undo put back
m.logger.Printf("[WARN] [stacks] update %s: rescan after the undo failed: %v", name, err)
}
m.finishUpdate(name, UpdatePhaseUndone, "") m.finishUpdate(name, UpdatePhaseUndone, "")
m.emitUpdateEvent(UpdateEventUndone, name, entry, UpdateRestorePoint{}, true) m.emitUpdateEvent(UpdateEventUndone, name, entry, UpdateRestorePoint{}, true)
m.logger.Printf("[INFO] [stacks] update %s: UNDONE in %s — the previous version is running on the data from before the update (%s)", name, m.now().Sub(start).Round(time.Second), detail) m.logger.Printf("[INFO] [stacks] update %s: UNDONE in %s — the previous version is running on the data from before the update (%s)", name, m.now().Sub(start).Round(time.Second), detail)
+11
View File
@@ -944,8 +944,16 @@ func (m *Manager) verifyAndConclude(ctx context.Context, name, dir string, env [
_ = m.RefreshStatus() _ = m.RefreshStatus()
m.removeUndoCopies(name, entry.UndoCopies) m.removeUndoCopies(name, entry.UndoCopies)
m.recordUpdateUndone(name, dir, nil) // a successful update ends the "undone" note m.recordUpdateUndone(name, dir, nil) // a successful update ends the "undone" note
m.clearFailedStep(name, dir) // R-680: and the failed-step record
m.clearJournal(name) m.clearJournal(name)
m.removePreUpdateCopies(dir) m.removePreUpdateCopies(dir)
// R-678 (v0.271.0): the app's catalog fields — ladder_steps_left, the badge's inputs, the pin — are
// re-read NOW, before Updating goes false, so neither a person nor the automatic leg ever reads the
// pre-update values after `done`. MEASURED 2026-09-24: ~50 s stale, six re-presses by the caller.
// COMPANION RED-PROOF (REPORT.md): drop this scan — TestR678_StepsLeftFreshAtDone fails.
if err := m.ScanStacks(); err != nil {
m.logger.Printf("[WARN] [stacks] update %s: rescan after the update failed: %v — the page catches up at the next scan", name, err)
}
m.finishUpdate(name, UpdatePhaseDone, "") m.finishUpdate(name, UpdatePhaseDone, "")
m.logger.Printf("[INFO] [stacks] update %s: DONE in %s", name, m.now().Sub(start).Round(time.Second)) m.logger.Printf("[INFO] [stacks] update %s: DONE in %s", name, m.now().Sub(start).Round(time.Second))
} }
@@ -1008,6 +1016,9 @@ func (m *Manager) failAndHold(ctx context.Context, name, dir string, env []strin
holdWhy = w holdWhy = w
m.markUpdateHeld(name) m.markUpdateHeld(name)
} }
if entry != nil { // R-680: the automatic leg never presses this step again on this ladder
m.recordFailedStep(name, dir, entry.NewPin, "held")
}
_ = m.RefreshStatus() _ = m.RefreshStatus()
m.clearJournal(name) m.clearJournal(name)
m.removePreUpdateCopies(dir) m.removePreUpdateCopies(dir)
@@ -0,0 +1,93 @@
package web
import (
"html"
"net/http/httptest"
"net/url"
"os"
"path/filepath"
"strings"
"testing"
)
// v0.271.0 — the per-box switch for AUTOMATIC app updates (`09` §3 decision 12: ON by default) on the
// settings page, in both languages, through the REAL router, handler and template; and the saved choice
// read back from settings.json's accessor the leg reads.
//
// COMPANION RED-PROOF (REPORT.md): make GetAppUpdateUnattended return false for an absent key — the
// first assertion fails at "a box that never saved the switch must show it ON".
func TestAppUpdateSwitch_DefaultOnAndSaved(t *testing.T) {
s := testPageServer(t)
hu := html.UnescapeString(getPage(t, s, "/settings").Body.String())
if !strings.Contains(hu, `data-app-update-unattended="true"`) || !strings.Contains(hu, "checked data-app-update-unattended") {
t.Fatal("a box that never saved the switch must show it ON (decision 12)")
}
if !strings.Contains(hu, "Alkalmazások automatikus frissítése") || !strings.Contains(hu, "kipróbált lépéssel") {
t.Error("the Hungarian page must carry the card's title and its one sentence")
}
en := html.UnescapeString(getPage(t, s, "/settings?lang=en").Body.String())
if !strings.Contains(en, "Automatic app updates") || !strings.Contains(en, "only with a tested step") {
t.Error("the English page must carry the card's title and its one sentence")
}
if strings.Contains(en, "kipróbált lépéssel") {
t.Error("the Hungarian sentence must be GONE from the English page")
}
// OFF: an unchecked box posts no field
rec := postForm(t, s, "/settings/app-update", url.Values{})
if rec.Code != 200 {
t.Fatalf("POST off = %d", rec.Code)
}
if s.settings.GetAppUpdateUnattended() {
t.Fatal("saving with the box unchecked must turn the switch OFF")
}
body := html.UnescapeString(rec.Body.String())
if !strings.Contains(body, "kikapcsolva") || !strings.Contains(body, `data-app-update-unattended="false"`) {
t.Error("the page after saving OFF must say so and show the box unchecked")
}
// ON again
postForm(t, s, "/settings/app-update", url.Values{"app_update_unattended": {"on"}})
if !s.settings.GetAppUpdateUnattended() {
t.Fatal("saving with the box checked must turn the switch back ON")
}
}
// TestAutoUpdateLine_PageSaysItOnce — a successful automatic step is told on the app page only, in the
// request's language, and only while the app still runs what that step installed.
//
// COMPANION RED-PROOF (REPORT.md): drop the AutoUpdateLine block from appDetailHandler — this fails at
// "the English page must carry the automatic-update line".
func TestAutoUpdateLine_PageSaysItOnce(t *testing.T) {
s, _ := credsHarness(t)
sd := filepath.Join(s.cfg.Paths.StacksDir, "crafty")
write := func(pin string) {
app := "deployed: true\npinned_images:\n web: " + pin + "\nlast_auto_update:\n at: \"2026-09-25T02:21:00Z\"\n outcome: done\n to:\n web: crafty:2.0.0\n"
if err := os.WriteFile(filepath.Join(sd, "app.yaml"), []byte(app), 0o644); err != nil {
t.Fatal(err)
}
_ = s.stackMgr.ScanStacks()
}
s.loadTemplates()
render := func(lang string) string {
rr := httptest.NewRecorder()
s.appDetailHandler(rr, httptest.NewRequest("GET", "/apps/crafty?lang="+lang, nil), "crafty")
return html.UnescapeString(rr.Body.String())
}
write("crafty:2.0.0")
en, hu := render("en"), render("hu")
const enLine, huLine = "Automatic update at", "Automatikus frissítés"
if !strings.Contains(en, enLine) || !strings.Contains(en, "— done.") {
t.Fatalf("the English page must carry the automatic-update line")
}
if !strings.Contains(hu, huLine+" ") || !strings.Contains(hu, "— sikeres.") || strings.Contains(hu, enLine) {
t.Fatalf("the Hungarian page must carry its own line and not the English one")
}
if !strings.Contains(hu, `data-auto-update="done"`) {
t.Error("the line must render in its own marked alert")
}
// a later manual update moved the pin: the line is history and goes
write("crafty:2.1.0")
if strings.Contains(render("en"), enLine) {
t.Error("the line must go once the app no longer runs what the automatic step installed")
}
}
+55
View File
@@ -839,6 +839,14 @@ func (s *Server) appDetailHandler(w http.ResponseWriter, r *http.Request, slug s
if found.Deployed && found.AppConfig != nil && found.AppConfig.LastUpdateUndone != nil { if found.Deployed && found.AppConfig != nil && found.AppConfig.LastUpdateUndone != nil {
data["UpdateUndoneLine"] = s.msg(r, "app_info.update_undone", found.Name, undoneWhen(found.AppConfig.LastUpdateUndone.At)) data["UpdateUndoneLine"] = s.msg(r, "app_info.update_undone", found.Name, undoneWhen(found.AppConfig.LastUpdateUndone.At))
} }
// v0.271.0 (`09` §6.4 part 7): a SUCCESSFUL automatic update is told on the page only — no mail.
// An undone or held automatic step keeps the lines above (they already say it), so this line is for
// `done` alone, and only while the app still runs what that step installed (a later manual update or
// a restore makes it history, and the page stops saying it).
if found.Deployed && found.AppConfig != nil && found.AppConfig.LastAutoUpdate != nil &&
found.AppConfig.LastAutoUpdate.Outcome == stacks.LegOutcomeDone && sameImageMap(found.AppConfig.LastAutoUpdate.To, found.AppConfig.PinnedImages) {
data["AutoUpdateLine"] = s.msg(r, "app_info.auto_update_done", undoneWhen(found.AppConfig.LastAutoUpdate.At))
}
s.executeTemplate(w, r, "app_info", data) s.executeTemplate(w, r, "app_info", data)
} }
@@ -2144,6 +2152,9 @@ func (s *Server) systemPageData() map[string]interface{} {
data["HealthchecksBase"] = s.cfg.Monitoring.HealthchecksBase data["HealthchecksBase"] = s.cfg.Monitoring.HealthchecksBase
data["HubEnabled"] = s.cfg.Hub.Enabled data["HubEnabled"] = s.cfg.Hub.Enabled
// v0.271.0: the automatic app-update switch (absent = ON).
data["AppUpdateUnattended"] = s.settings == nil || s.settings.GetAppUpdateUnattended()
// Self-update status // Self-update status
data["SelfUpdateEnabled"] = s.cfg.SelfUpdate.Enabled data["SelfUpdateEnabled"] = s.cfg.SelfUpdate.Enabled
if s.updater != nil { if s.updater != nil {
@@ -2824,6 +2835,37 @@ func (s *Server) settingsAppEmailHandler(w http.ResponseWriter, r *http.Request)
s.executeTemplate(w, r, "settings_notifications", data) s.executeTemplate(w, r, "settings_notifications", data)
} }
// settingsAppUpdateHandler saves the per-box switch for AUTOMATIC app updates (v0.271.0, `09` §3
// decision 12). An unchecked box posts nothing, so absence of the field means OFF here — this is the
// household's explicit save, unlike the stored default (absent = ON). The leg reads the switch at its
// next start and before every press, so the choice takes effect before the next night.
func (s *Server) settingsAppUpdateHandler(w http.ResponseWriter, r *http.Request) {
_ = r.ParseForm()
on := r.FormValue("app_update_unattended") == "on" || r.FormValue("app_update_unattended") == "true"
if s.settings == nil {
data := s.systemPageData()
data["AppUpdateError"] = s.msg(r, "settings.app_update_save_error")
s.executeTemplate(w, r, "settings_system", data)
return
}
if err := s.settings.SetAppUpdateUnattended(on); err != nil {
s.logger.Printf("[ERROR] [web] saving app_update.unattended=%v failed: %v", on, err)
data := s.systemPageData()
data["AppUpdateError"] = s.msg(r, "settings.app_update_save_error")
s.executeTemplate(w, r, "settings_system", data)
return
}
s.logger.Printf("[INFO] [web] automatic app updates switched %s by the household (app_update.unattended=%v)", map[bool]string{true: "ON", false: "OFF"}[on], on)
s.reportTriggerNow()
data := s.systemPageData()
if on {
data["AppUpdateSuccess"] = s.msg(r, "settings.app_update_on")
} else {
data["AppUpdateSuccess"] = s.msg(r, "settings.app_update_off")
}
s.executeTemplate(w, r, "settings_system", data)
}
func (s *Server) settingsNotificationsTestHandler(w http.ResponseWriter, r *http.Request) { func (s *Server) settingsNotificationsTestHandler(w http.ResponseWriter, r *http.Request) {
data := s.notificationsPageData() data := s.notificationsPageData()
@@ -3539,6 +3581,19 @@ func generateFileBrowserConfig(paths []settings.StoragePath, importSource bool)
// undoneWhen renders last_update_undone.at for the page line, in the box's time zone; the raw value // undoneWhen renders last_update_undone.at for the page line, in the box's time zone; the raw value
// when it does not parse (a record is never hidden for its format). // when it does not parse (a record is never hidden for its format).
// sameImageMap reports whether two service→image maps are equal (nil equals empty).
func sameImageMap(a, b map[string]string) bool {
if len(a) != len(b) {
return false
}
for k, v := range a {
if b[k] != v {
return false
}
}
return true
}
func undoneWhen(rfc3339 string) string { func undoneWhen(rfc3339 string) string {
t, err := time.Parse(time.RFC3339, rfc3339) t, err := time.Parse(time.RFC3339, rfc3339)
if err != nil { if err != nil {
+2
View File
@@ -674,6 +674,8 @@ func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
s.settingsNotificationsTestHandler(w, r) s.settingsNotificationsTestHandler(w, r)
case path == "/settings/app-email" && r.Method == http.MethodPost: case path == "/settings/app-email" && r.Method == http.MethodPost:
s.settingsAppEmailHandler(w, r) s.settingsAppEmailHandler(w, r)
case path == "/settings/app-update" && r.Method == http.MethodPost:
s.settingsAppUpdateHandler(w, r)
case path == "/settings/storage/add" && r.Method == http.MethodPost: case path == "/settings/storage/add" && r.Method == http.MethodPost:
s.settingsStorageAddHandler(w, r) s.settingsStorageAddHandler(w, r)
case path == "/settings/storage/remove" && r.Method == http.MethodPost: case path == "/settings/storage/remove" && r.Method == http.MethodPost:
@@ -38,6 +38,8 @@
<div class="alert alert-warning" style="margin-top:1rem" data-update-error="true">{{updateErrorText .Stack}}</div> <div class="alert alert-warning" style="margin-top:1rem" data-update-error="true">{{updateErrorText .Stack}}</div>
{{else if .UpdateUndoneLine}} {{else if .UpdateUndoneLine}}
<div class="alert alert-info" style="margin-top:1rem" data-update-undone="true">{{.UpdateUndoneLine}}</div> <div class="alert alert-info" style="margin-top:1rem" data-update-undone="true">{{.UpdateUndoneLine}}</div>
{{else if .AutoUpdateLine}}
<div class="alert alert-info" style="margin-top:1rem" data-auto-update="done">{{.AutoUpdateLine}}</div>
{{end}} {{end}}
{{if .MissingStorageLabel}} {{if .MissingStorageLabel}}
@@ -143,6 +143,26 @@
</div> </div>
</div> </div>
<!-- Section: Automatic app updates (v0.271.0, `09` §3 decision 12) — the per-box switch, ON by default. -->
<div class="settings-card" id="app-update">
<h3>{{T "settings_system.app_update_title"}}</h3>
<p class="settings-card-desc">{{T "settings_system.app_update_desc"}}</p>
{{if .AppUpdateSuccess}}<div class="alert alert-info">{{.AppUpdateSuccess}}</div>{{end}}
{{if .AppUpdateError}}<div class="alert alert-error">{{.AppUpdateError}}</div>{{end}}
<form method="POST" action="/settings/app-update">
{{.CSRFField}}
<div class="form-group">
<label style="display:flex;align-items:center;gap:.5rem">
<input type="checkbox" name="app_update_unattended" value="on" {{if .AppUpdateUnattended}}checked{{end}} data-app-update-unattended="{{.AppUpdateUnattended}}">
{{T "settings_system.app_update_toggle"}}
</label>
</div>
<div class="form-actions">
<button type="submit" class="btn btn-primary">{{T "settings_system.app_update_save"}}</button>
</div>
</form>
</div>
<!-- Section: Network (R-66) — the box's own address. Every value is live-computed per render and <!-- Section: Network (R-66) — the box's own address. Every value is live-computed per render and
stored NOWHERE (S-5: a DHCP lease persisted anywhere eventually misdirects people); a missing stored NOWHERE (S-5: a DHCP lease persisted anywhere eventually misdirects people); a missing
value renders „—" because an address-less row beats a wrong address. --> value renders „—" because an address-less row beats a wrong address. -->
@@ -266,6 +266,26 @@
</div> </div>
<div class="settings-card" id="app-update">
<h3>Alkalmazások automatikus frissítése</h3>
<p class="settings-card-desc">Ha be van kapcsolva, a doboz minden éjjel, a távoli mentés után magától frissíti az alkalmazásaidat: egyszerre egyet, mindig csak kipróbált lépéssel, és ha valami nem sikerül, visszaállítja az előző változatot.</p>
<form method="POST" action="/settings/app-update">
<div class="form-group">
<label style="display:flex;align-items:center;gap:.5rem">
<input type="checkbox" name="app_update_unattended" value="on" data-app-update-unattended="">
Automatikus frissítés bekapcsolva
</label>
</div>
<div class="form-actions">
<button type="submit" class="btn btn-primary">Mentés</button>
</div>
</form>
</div>
<div class="settings-card"> <div class="settings-card">
<h3>Hálózat</h3> <h3>Hálózat</h3>
<div class="settings-grid"> <div class="settings-grid">
@@ -306,6 +306,26 @@
</div> </div>
<div class="settings-card" id="app-update">
<h3>Alkalmazások automatikus frissítése</h3>
<p class="settings-card-desc">Ha be van kapcsolva, a doboz minden éjjel, a távoli mentés után magától frissíti az alkalmazásaidat: egyszerre egyet, mindig csak kipróbált lépéssel, és ha valami nem sikerül, visszaállítja az előző változatot.</p>
<form method="POST" action="/settings/app-update">
<div class="form-group">
<label style="display:flex;align-items:center;gap:.5rem">
<input type="checkbox" name="app_update_unattended" value="on" data-app-update-unattended="">
Automatikus frissítés bekapcsolva
</label>
</div>
<div class="form-actions">
<button type="submit" class="btn btn-primary">Mentés</button>
</div>
</form>
</div>
<div class="settings-card"> <div class="settings-card">
<h3>Hálózat</h3> <h3>Hálózat</h3>
<div class="settings-grid"> <div class="settings-grid">
@@ -223,6 +223,26 @@
</div> </div>
<div class="settings-card" id="app-update">
<h3>Alkalmazások automatikus frissítése</h3>
<p class="settings-card-desc">Ha be van kapcsolva, a doboz minden éjjel, a távoli mentés után magától frissíti az alkalmazásaidat: egyszerre egyet, mindig csak kipróbált lépéssel, és ha valami nem sikerül, visszaállítja az előző változatot.</p>
<form method="POST" action="/settings/app-update">
<div class="form-group">
<label style="display:flex;align-items:center;gap:.5rem">
<input type="checkbox" name="app_update_unattended" value="on" data-app-update-unattended="">
Automatikus frissítés bekapcsolva
</label>
</div>
<div class="form-actions">
<button type="submit" class="btn btn-primary">Mentés</button>
</div>
</form>
</div>
<div class="settings-card"> <div class="settings-card">
<h3>Hálózat</h3> <h3>Hálózat</h3>
<div class="settings-grid"> <div class="settings-grid">
@@ -256,6 +256,26 @@
</div> </div>
<div class="settings-card" id="app-update">
<h3>Alkalmazások automatikus frissítése</h3>
<p class="settings-card-desc">Ha be van kapcsolva, a doboz minden éjjel, a távoli mentés után magától frissíti az alkalmazásaidat: egyszerre egyet, mindig csak kipróbált lépéssel, és ha valami nem sikerül, visszaállítja az előző változatot.</p>
<form method="POST" action="/settings/app-update">
<div class="form-group">
<label style="display:flex;align-items:center;gap:.5rem">
<input type="checkbox" name="app_update_unattended" value="on" data-app-update-unattended="">
Automatikus frissítés bekapcsolva
</label>
</div>
<div class="form-actions">
<button type="submit" class="btn btn-primary">Mentés</button>
</div>
</form>
</div>
<div class="settings-card"> <div class="settings-card">
<h3>Hálózat</h3> <h3>Hálózat</h3>
<div class="settings-grid"> <div class="settings-grid">
+5 -1
View File
@@ -75,7 +75,11 @@
"settings_notifications.app_update_held": "BORN AS A KEY, v0.264.0 (R-606 / the update events) -- a NEW sentence, never a Go literal.", "settings_notifications.app_update_held": "BORN AS A KEY, v0.264.0 (R-606 / the update events) -- a NEW sentence, never a Go literal.",
"badge.update.held": "BORN AS A KEY, v0.265.0 (R-625) -- a NEW badge for a held app; its Hungarian twin in updatebadge.go is pinned by TestR625_HeldBadgeHungarianMatchesTheBundle.", "badge.update.held": "BORN AS A KEY, v0.265.0 (R-625) -- a NEW badge for a held app; its Hungarian twin in updatebadge.go is pinned by TestR625_HeldBadgeHungarianMatchesTheBundle.",
"err.backup.adatbazis_masolat_csonka_nem_indult": "BORN AS A KEY, v0.267.0 (R-640) -- a NEW refusal for a cut-off database copy; pinned by TestR640_*.", "err.backup.adatbazis_masolat_csonka_nem_indult": "BORN AS A KEY, v0.267.0 (R-640) -- a NEW refusal for a cut-off database copy; pinned by TestR640_*.",
"err.backup.adatbazis_masolat_csonka_nem_toltve": "BORN AS A KEY, v0.267.0 (R-640) -- a NEW refusal for a cut-off database copy; pinned by TestR640_*." "err.backup.adatbazis_masolat_csonka_nem_toltve": "BORN AS A KEY, v0.267.0 (R-640) -- a NEW refusal for a cut-off database copy; pinned by TestR640_*.",
"app_info.auto_update_done": "BORN AS A KEY, v0.271.0 (09 6.4 part 7, the automatic update leg) -- a NEW sentence, never a Go literal. Pinned by internal/web/app_update_switch_test.go.",
"settings.app_update_on": "BORN AS A KEY, v0.271.0 (09 6.4 part 7, the automatic update leg) -- a NEW sentence, never a Go literal. Pinned by internal/web/app_update_switch_test.go.",
"settings.app_update_off": "BORN AS A KEY, v0.271.0 (09 6.4 part 7, the automatic update leg) -- a NEW sentence, never a Go literal. Pinned by internal/web/app_update_switch_test.go.",
"settings.app_update_save_error": "BORN AS A KEY, v0.271.0 (09 6.4 part 7, the automatic update leg) -- a NEW sentence, never a Go literal. Pinned by internal/web/app_update_switch_test.go."
}, },
"flash.share.already_on": "A megosztás már be van kapcsolva.", "flash.share.already_on": "A megosztás már be van kapcsolva.",
"flash.share.enable_failed": "A megosztás bekapcsolása nem sikerült.", "flash.share.enable_failed": "A megosztás bekapcsolása nem sikerült.",