v0.147.0 — feedback slice 1: pressing a button says something

The systemic complaint, twice in one evening: you press a button and nothing
happens. No progress, no ETA, no named result. Three worst offenders, fixed on
the two patterns already here (deploy 3-step panel, storage-init status poll).
No new framework — that is a ROADMAP item; three targeted cards ship tonight.

4a — a verification restore names its result. The flash said the app had been
restored "to a verification folder on the drive"; which folder, on which drive,
was invisible, so the customer could not go and look at what they had just asked
for. Full path now. The restore page gained a listing of existing verification
copies (app, size, date, path) — nothing anywhere showed these, so they piled up
and the only way to find them was SSH — each with a double-confirmed delete.

That delete is the only one this release adds, so it names a STACK, never a
path: the Manager resolves the name inside a backups/offsite-restore root it
computed itself and refuses anything landing outside. Red-proofed — neutralise
the name guard and stack:"" resolves to the offsite-restore ROOT and takes every
copy with it. Refusals are asserted as non-effects.

4b — Megosztás enable shows what it is waiting for. Enabling ran ReconcileSamba
synchronously inside the POST handler; on a golden without felhom-samba baked
that is compose pulling ~100MB, i.e. minutes of an apparently-hung form post
followed by "Beállítás mentve." whether or not anything came up. Detached +
polled now, distinguishing "képfájl letöltése" from "indítás" — decided BEFORE
the work starts, since afterwards the image is always present. Success is
probed, not inferred (compose up -d exits 0 on a crash-loop). The password form
starts the same job: with UserSet false reconcile deploys nothing, so on a fresh
box that is where the pull actually happens.

4c — "Távoli mentés most" streams real progress. restic was already reporting
bytes and percent; the runner seam used CombinedOutput() and discarded them. The
manual run now passes --json and scans stdout line-by-line: total bytes, percent,
current app. Manual only — the nightly stays silent, pinned by a test that fails
if it ever passes --json. The poll now arms unconditionally, closing a race the
manual trigger always ran: the redirect rendered before the goroutine wrote
LastStatus=running, so the poll never armed and the page sat static during the
very run just started. Red-proofed twice.

Also closes the golden/controller infra-image drift at the source: infra.Images()
derives from the existing pins and --print-infra-images exposes it, so the golden
bake can stop carrying its own copy. That copy had already drifted — felhom-samba
was never added, so the golden baked 3 of 4, which is why enabling Megosztás
pulled at runtime in the first place.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nn3VgQk9iwEGgyx6QJ2NvE
This commit is contained in:
2026-07-19 09:30:30 +02:00
parent 7f0b41c3e7
commit b5d78d1e0f
22 changed files with 1527 additions and 21 deletions
+5
View File
@@ -35,6 +35,11 @@ type Manager struct {
// offbox (Part B): the restic-SFTP exec seam (nil → real restic) + the failure→operator-alert hook.
offboxRunner offboxRunner
offboxNotify func(dur time.Duration, snapshots int, err error)
// offboxStreamRunner + offboxProgress: the MANUAL run's live progress (v0.147.0, 4c). The stream
// seam scans restic's `--json` stdout line-by-line; the state is what the page polls. Both are
// inert on the nightly path — the sink is installed only for the duration of a manual run.
offboxStreamRunner offboxStreamRunner
offboxProgress offboxProgressState
// offboxOrphanEvent (v0.142.0), if set, pushes a hub event on offsite-repo continuity transitions
// ("offbox_repo_orphaned" / "offbox_repo_reset"); renamedTo names the move-aside path (reset only).
// Wired in main.go to the notifier. Nil-safe.
+19 -1
View File
@@ -572,6 +572,22 @@ func (m *Manager) ensureOffboxRepo(ctx context.Context, base, env []string) erro
// tars) to the SFTP repo, then prunes per the retention policy. Single-flight + migration-guarded. A
// failure (incl. a fail-fast dead-NAS error) records status + alerts the operator. Returns the first error.
func (m *Manager) RunOffboxBackup(ctx context.Context) error {
return m.runOffboxBackup(ctx, false)
}
// RunOffboxBackupWithProgress is the MANUAL („Távoli mentés most") entry point: identical work, but
// with the live progress sink installed so the page can show total bytes, percent and current app
// (v0.147.0, 4c). The nightly scheduled run keeps calling RunOffboxBackup and stays silent — nobody
// is watching a progress bar at 03:00, and a sink left installed would publish stale percentages
// into a page that never asked for them.
func (m *Manager) RunOffboxBackupWithProgress(ctx context.Context) error {
return m.runOffboxBackup(ctx, true)
}
func (m *Manager) runOffboxBackup(ctx context.Context, withProgress bool) error {
if withProgress {
defer m.beginManualProgress()()
}
if !m.OffboxConfigured() {
return fmt.Errorf("off-box backup not configured")
}
@@ -890,7 +906,9 @@ func (m *Manager) runOffboxInternal(ctx context.Context, apps, base, env []strin
}
args := append([]string{"backup", "--tag", "felhom-offbox", "--tag", stack, src}, extra...)
bctx, cancel := context.WithTimeout(ctx, offboxBackupTimeout)
out, berr := m.resticStep(bctx, env, base, "backup:"+stack, args...)
// Streaming twin (4c): identical to resticStep unless a MANUAL run installed a progress sink,
// in which case it adds --json and feeds the parsed status lines to the page's poll.
out, berr := m.resticBackupStep(bctx, env, base, "backup:"+stack, stack, args...)
cancel()
if berr != nil {
m.logger.Printf("[ERROR] [offbox] backup %s failed: %v: %s", stack, berr, truncate(out))
@@ -0,0 +1,257 @@
package backup
import (
"bufio"
"context"
"encoding/json"
"io"
"os"
"os/exec"
"sync"
"time"
)
// Offsite backup progress (v0.147.0, feedback slice 4c).
//
// THE PROBLEM: „Távoli mentés most" started a background restic run and redirected with „A távoli
// mentés elindult". After that the page polled a status field whose only values were running / ok /
// error. For a first offsite push of tens of gigabytes over SFTP that is 20+ minutes of a spinner
// with no total, no percentage and no indication of WHICH app is being pushed — indistinguishable
// from a hang.
//
// restic already reports all of it: `backup --json` writes newline-delimited status objects to
// stdout. We only had to stop throwing them away — the existing runner seam uses CombinedOutput(),
// which buffers everything until exit.
//
// SCOPE: the MANUAL trigger only. The nightly scheduled run stays silent (nobody is watching a
// progress bar at 03:00, and a sink left installed would keep publishing stale percentages into a
// page that never asked). The sink is installed for the duration of a manual run and cleared after.
// OffboxProgress is a snapshot of an in-flight manual offsite backup.
type OffboxProgress struct {
Active bool `json:"active"`
CurrentApp string `json:"current_app"`
Percent float64 `json:"percent"` // 0..100
BytesDone int64 `json:"bytes_done"`
TotalBytes int64 `json:"total_bytes"`
DoneHuman string `json:"done_human"`
TotalHuman string `json:"total_human"`
}
// resticStatusLine is the subset of restic's `--json` status object we consume. restic emits several
// message_types (status, summary, error, verbose_status); anything that is not "status" is ignored
// here rather than treated as garbage, because restic adds new types between versions and an unknown
// type must never break the run.
type resticStatusLine struct {
MessageType string `json:"message_type"`
PercentDone float64 `json:"percent_done"` // 0..1
TotalBytes int64 `json:"total_bytes"`
BytesDone int64 `json:"bytes_done"`
}
// parseResticStatus parses ONE line of restic --json output into a progress delta.
//
// Kept as a pure function precisely so it can be tested without restic, a network, or a repo — the
// parser is the part that silently rots when restic changes its output, and a progress bar that
// quietly stops moving is worse than no progress bar at all.
func parseResticStatus(line string) (pct float64, done, total int64, ok bool) {
var s resticStatusLine
if err := json.Unmarshal([]byte(line), &s); err != nil {
return 0, 0, 0, false
}
if s.MessageType != "status" {
return 0, 0, 0, false
}
pct = s.PercentDone * 100
if pct < 0 {
pct = 0
}
if pct > 100 {
pct = 100
}
return pct, s.BytesDone, s.TotalBytes, true
}
// offboxProgressState is the published snapshot, guarded independently of the Manager mutex so a
// poll never blocks behind the running backup.
type offboxProgressState struct {
mu sync.Mutex
cur OffboxProgress
live bool
}
func (p *offboxProgressState) begin() {
p.mu.Lock()
p.cur = OffboxProgress{Active: true}
p.live = true
p.mu.Unlock()
}
func (p *offboxProgressState) end() {
p.mu.Lock()
p.cur = OffboxProgress{}
p.live = false
p.mu.Unlock()
}
func (p *offboxProgressState) setApp(app string) {
p.mu.Lock()
if p.live {
// A new app resets the byte counters: restic's percentages are per-invocation, and carrying
// the previous app's 100% into the next app's start would show a bar that jumps backwards.
p.cur.CurrentApp = app
p.cur.Percent, p.cur.BytesDone, p.cur.TotalBytes = 0, 0, 0
p.cur.DoneHuman, p.cur.TotalHuman = "", ""
}
p.mu.Unlock()
}
func (p *offboxProgressState) update(pct float64, done, total int64) {
p.mu.Lock()
if p.live {
p.cur.Percent, p.cur.BytesDone, p.cur.TotalBytes = pct, done, total
p.cur.DoneHuman, p.cur.TotalHuman = humanizeBytes(done), humanizeBytes(total)
}
p.mu.Unlock()
}
func (p *offboxProgressState) snapshot() OffboxProgress {
p.mu.Lock()
defer p.mu.Unlock()
return p.cur
}
// OffboxProgressSnapshot is the poll surface for the „Távoli mentés" page.
func (m *Manager) OffboxProgressSnapshot() OffboxProgress { return m.offboxProgress.snapshot() }
// offboxStreamRunner is the streaming restic-exec seam: like offboxRunner, but calls onLine for each
// stdout line AS IT ARRIVES instead of only returning the buffered output at exit. Tests inject a
// fake that emits canned `--json` status lines, so the whole progress path is exercised without
// restic, a network or a repo.
type offboxStreamRunner func(ctx context.Context, env []string, onLine func(string), args ...string) ([]byte, error)
// SetOffboxStreamRunner installs the streaming seam (nil → the real streaming exec).
func (m *Manager) SetOffboxStreamRunner(r offboxStreamRunner) { m.offboxStreamRunner = r }
func (m *Manager) streamRunner() offboxStreamRunner {
if m.offboxStreamRunner != nil {
return m.offboxStreamRunner
}
return defaultOffboxStreamRunner
}
// defaultOffboxStreamRunner runs restic with stdout scanned line-by-line. stderr is captured whole
// (restic's --json progress goes to stdout; errors go to stderr) and appended to the returned output
// so callers keep the same error-diagnosis material CombinedOutput gave them.
func defaultOffboxStreamRunner(ctx context.Context, env []string, onLine func(string), args ...string) ([]byte, error) {
cmd := exec.CommandContext(ctx, "restic", args...)
cmd.Env = append(os.Environ(), env...)
stdout, err := cmd.StdoutPipe()
if err != nil {
return nil, err
}
var stderr syncBuf
cmd.Stderr = &stderr
if err := cmd.Start(); err != nil {
return nil, err
}
var tail lineTail
scanner := bufio.NewScanner(stdout)
// restic status lines are small, but a --json summary listing many paths can exceed the 64KB
// default; a scanner that dies mid-run would silently freeze the progress bar.
scanner.Buffer(make([]byte, 0, 64*1024), 4*1024*1024)
for scanner.Scan() {
line := scanner.Text()
tail.add(line)
if onLine != nil {
onLine(line)
}
}
_, _ = io.Copy(io.Discard, stdout)
werr := cmd.Wait()
// Keep only the tail of stdout: the full --json stream of a large backup is megabytes of status
// spam, and every caller uses this output for error diagnosis (and lock-pattern matching) only.
out := append(tail.bytes(), stderr.bytes()...)
return out, werr
}
// lineTail keeps the last N lines seen, so error diagnosis has context without buffering the whole
// --json stream.
type lineTail struct {
lines []string
}
func (t *lineTail) add(s string) {
const keep = 40
t.lines = append(t.lines, s)
if len(t.lines) > keep {
t.lines = t.lines[len(t.lines)-keep:]
}
}
func (t *lineTail) bytes() []byte {
var b []byte
for _, l := range t.lines {
b = append(b, l...)
b = append(b, '\n')
}
return b
}
type syncBuf struct {
mu sync.Mutex
b []byte
}
func (s *syncBuf) Write(p []byte) (int, error) {
s.mu.Lock()
s.b = append(s.b, p...)
s.mu.Unlock()
return len(p), nil
}
func (s *syncBuf) bytes() []byte {
s.mu.Lock()
defer s.mu.Unlock()
return append([]byte{}, s.b...)
}
// resticBackupStep is resticStep's streaming twin, used ONLY by the app-backup leg when a manual run
// has a progress sink installed. It keeps resticStep's crash-lock self-heal semantics by delegating
// the retry path to resticStep (a retry after an unlock is rare and does not need progress).
func (m *Manager) resticBackupStep(ctx context.Context, env, base []string, label, app string, args ...string) ([]byte, error) {
if !m.offboxProgress.snapshot().Active {
return m.resticStep(ctx, env, base, label, args...) // nightly / no watcher: unchanged path
}
m.offboxProgress.setApp(app)
full := append(append([]string{}, base...), args...)
// --json turns on the machine-readable progress stream. It is added ONLY here, so the nightly
// run's output format (and everything that greps it) is untouched.
full = append(full, "--json")
out, err := m.streamRunner()(ctx, env, func(line string) {
if pct, done, total, ok := parseResticStatus(line); ok {
m.offboxProgress.update(pct, done, total)
}
}, full...)
if err == nil || !offboxLockRe.Match(out) {
return out, err
}
// Lock collision: fall back to the non-streaming step, which owns the unlock --remove-all
// self-heal. Progress stalls for that one retry; correctness beats a moving bar.
m.logger.Printf("[WARN] [offbox] %s hit a lock during a manual run — retrying via the self-healing step", label)
return m.resticStep(ctx, env, base, label, args...)
}
// beginManualProgress installs the progress sink for a manual run and returns the cleanup func.
func (m *Manager) beginManualProgress() func() {
m.offboxProgress.begin()
started := time.Now()
return func() {
m.logger.Printf("[INFO] [offbox] manual run progress reporting ended after %s", time.Since(started).Round(time.Second))
m.offboxProgress.end()
}
}
@@ -0,0 +1,228 @@
package backup
import (
"context"
"strings"
"testing"
)
// v0.147.0 slice 4c — live progress for a MANUAL offsite run.
//
// These tests exercise the WHOLE path a real run takes: a fake restic emits canned `--json` status
// lines through the streaming seam, and the assertion is on what the poll surface
// (OffboxProgressSnapshot) reports — not on the parser in isolation. A parser that works but is
// never wired to the snapshot would leave the customer looking at the same silent spinner, which is
// the bug being fixed.
//
// RED-PROOF (run manually, both confirmed to fail):
// 1. break the parser — flip `s.MessageType != "status"` to `== "status"` in parseResticStatus:
// TestManualRunReportsParsedProgress fails ("percent = 0, want 42").
// 2. drop the wiring — make resticBackupStep always delegate to resticStep:
// the same test fails (no --json, no stream, no snapshot).
// jsonStatus is one restic --json status line.
func jsonStatus(pct float64, done, total int64) string {
return `{"message_type":"status","percent_done":` + ftoa(pct) + `,"total_bytes":` + itoa(total) + `,"bytes_done":` + itoa(done) + `}`
}
func ftoa(f float64) string {
// small helper — avoids strconv import noise in the canned lines
switch f {
case 0:
return "0"
case 0.42:
return "0.42"
case 1:
return "1"
}
return "0.5"
}
func itoa(i int64) string {
if i == 0 {
return "0"
}
var b []byte
neg := i < 0
if neg {
i = -i
}
for i > 0 {
b = append([]byte{byte('0' + i%10)}, b...)
i /= 10
}
if neg {
return "-" + string(b)
}
return string(b)
}
// TestManualRunReportsParsedProgress is the headline: a manual run driven by a fake restic that emits
// --json status lines must make OffboxProgressSnapshot report the parsed percentage, byte counts and
// the app currently being pushed.
func TestManualRunReportsParsedProgress(t *testing.T) {
e := newSharesOffboxEnv(t, "immich")
if err := e.sett.SetSMBEnabled(false); err != nil { // keep this test to the app leg only
t.Fatal(err)
}
var seen []OffboxProgress
var sawJSONFlag bool
e.m.SetOffboxStreamRunner(func(ctx context.Context, env []string, onLine func(string), args ...string) ([]byte, error) {
for _, a := range args {
if a == "--json" {
sawJSONFlag = true
}
}
// Emit a scan phase (total not yet known), then real progress, sampling the published
// snapshot after each line exactly as the polling page would.
onLine(jsonStatus(0, 0, 0))
seen = append(seen, e.m.OffboxProgressSnapshot())
onLine(`{"message_type":"verbose_status","action":"unchanged"}`) // must be ignored, not fatal
onLine(jsonStatus(0.42, 4200, 10000))
seen = append(seen, e.m.OffboxProgressSnapshot())
return []byte(`{"message_type":"summary","snapshot_id":"abc"}`), nil
})
// The non-streaming seam still serves every other restic call (cat config, forget, snapshots…).
e.m.SetOffboxRunner(func(ctx context.Context, env []string, args ...string) ([]byte, error) {
if contains(args, "snapshots") {
return []byte(`[]`), nil
}
return []byte(""), nil
})
if err := e.m.RunOffboxBackupWithProgress(context.Background()); err != nil {
t.Fatalf("manual run: %v", err)
}
if !sawJSONFlag {
t.Fatal("the manual backup leg did not pass --json to restic — nothing could ever be parsed")
}
if len(seen) != 2 {
t.Fatalf("expected 2 sampled snapshots, got %d", len(seen))
}
// While restic is still scanning, total is unknown: report 0 rather than inventing a percentage.
if seen[0].TotalBytes != 0 || seen[0].Percent != 0 {
t.Errorf("scan phase: got %+v, want zeroed counters", seen[0])
}
if seen[0].CurrentApp != "immich" {
t.Errorf("scan phase: current_app = %q, want %q", seen[0].CurrentApp, "immich")
}
got := seen[1]
if got.Percent != 42 {
t.Errorf("percent = %v, want 42", got.Percent)
}
if got.BytesDone != 4200 || got.TotalBytes != 10000 {
t.Errorf("bytes = %d/%d, want 4200/10000", got.BytesDone, got.TotalBytes)
}
if got.CurrentApp != "immich" {
t.Errorf("current_app = %q, want %q", got.CurrentApp, "immich")
}
if got.TotalHuman == "" || got.DoneHuman == "" {
t.Errorf("humanized byte strings not populated: %+v", got)
}
if !got.Active {
t.Error("progress reported inactive during a manual run")
}
// After the run the sink must be torn down, or the page would keep rendering a stale bar.
if after := e.m.OffboxProgressSnapshot(); after.Active || after.Percent != 0 {
t.Errorf("progress still published after the run: %+v", after)
}
}
// TestNightlyRunStaysSilent pins the scope decision: the scheduled run must neither install the sink
// nor pass --json, so its output format (and everything that greps it) is untouched.
func TestNightlyRunStaysSilent(t *testing.T) {
e := newSharesOffboxEnv(t, "immich")
if err := e.sett.SetSMBEnabled(false); err != nil {
t.Fatal(err)
}
e.m.SetOffboxStreamRunner(func(ctx context.Context, env []string, onLine func(string), args ...string) ([]byte, error) {
t.Fatal("the nightly run used the streaming runner — progress must be manual-only")
return nil, nil
})
var backupArgs []string
e.m.SetOffboxRunner(func(ctx context.Context, env []string, args ...string) ([]byte, error) {
if contains(args, "backup") {
backupArgs = append([]string{}, args...)
}
if contains(args, "snapshots") {
return []byte(`[]`), nil
}
return []byte(""), nil
})
if err := e.m.RunOffboxBackup(context.Background()); err != nil {
t.Fatalf("nightly run: %v", err)
}
if backupArgs == nil {
t.Fatal("no backup call was made")
}
if contains(backupArgs, "--json") {
t.Errorf("the nightly run passed --json: %v", backupArgs)
}
if p := e.m.OffboxProgressSnapshot(); p.Active {
t.Errorf("the nightly run published progress: %+v", p)
}
}
// TestParseResticStatusIgnoresNonStatus covers the lines restic actually interleaves with status
// output. An unknown message_type must be ignored, never mistaken for progress — restic adds new
// types between versions.
func TestParseResticStatusIgnoresNonStatus(t *testing.T) {
for _, line := range []string{
`{"message_type":"summary","total_bytes_processed":123}`,
`{"message_type":"verbose_status","action":"new"}`,
`{"message_type":"error","error":{"message":"boom"}}`,
`not json at all`,
``,
`{}`,
} {
if _, _, _, ok := parseResticStatus(line); ok {
t.Errorf("parsed a non-status line as progress: %q", line)
}
}
pct, done, total, ok := parseResticStatus(jsonStatus(0.42, 4200, 10000))
if !ok {
t.Fatal("a real status line was not parsed")
}
if pct != 42 || done != 4200 || total != 10000 {
t.Errorf("got %v %d %d, want 42 4200 10000", pct, done, total)
}
}
// TestParseResticStatusClampsPercent — restic has been seen to report percent_done slightly above 1
// near completion. A bar wider than its track is a visible bug.
func TestParseResticStatusClampsPercent(t *testing.T) {
pct, _, _, ok := parseResticStatus(`{"message_type":"status","percent_done":1.04}`)
if !ok || pct != 100 {
t.Errorf("percent = %v (ok=%v), want clamped to 100", pct, ok)
}
pct, _, _, ok = parseResticStatus(`{"message_type":"status","percent_done":-0.2}`)
if !ok || pct != 0 {
t.Errorf("percent = %v (ok=%v), want clamped to 0", pct, ok)
}
}
// TestLineTailKeepsOnlyTheTail — the --json stream of a large backup is megabytes of status spam, and
// every caller uses this output for error diagnosis and lock-pattern matching. Buffering all of it
// would be a memory leak proportional to backup size.
func TestLineTailKeepsOnlyTheTail(t *testing.T) {
var tl lineTail
for i := 0; i < 500; i++ {
tl.add("line-" + itoa(int64(i)))
}
out := string(tl.bytes())
if strings.Contains(out, "line-0\n") {
t.Error("the oldest line survived — the tail is unbounded")
}
if !strings.Contains(out, "line-499") {
t.Error("the newest line was dropped")
}
if n := strings.Count(out, "\n"); n > 64 {
t.Errorf("tail kept %d lines, want a small bounded number", n)
}
}
+3 -2
View File
@@ -128,9 +128,10 @@ func (m *Manager) offboxSnapshotSize(ctx context.Context, id string) (int64, err
// probe). NEVER cfg.Paths.DataDir (the rootfs — the F-A1 filler). App's HDD drive first; else the first
// schedulable storage path; else a Hungarian refusal.
func (m *Manager) offboxRestoreScratchDir(stack string) (scratch, nsRoot string, err error) {
// offsiteRestoreRootFor is THE place `backups/offsite-restore` is spelled (offbox_verify_copies.go)
// — the listing/delete surface must resolve byte-identical paths to the ones written here.
scratchFor := func(root string) (string, string) {
nr := m.namespaceRoot(root)
return filepath.Join(nr, "backups", "offsite-restore", stack), nr
return filepath.Join(m.offsiteRestoreRootFor(root), stack), m.namespaceRoot(root)
}
isNet := func(path string) bool { return m.settings != nil && m.settings.IsNetworkStoragePath(path) }
// (1) the app's own drive — preferred, but ONLY if it is not NETWORK storage (F-3afix-1). restic
@@ -0,0 +1,145 @@
package backup
import (
"fmt"
"os"
"path/filepath"
"sort"
"strings"
"time"
)
// Verification copies — the listing/delete surface for `<nsRoot>/backups/offsite-restore/<app>`.
//
// WHY THIS EXISTS (v0.147.0, feedback slice 4a): an offsite verification restore wrote its result to
// a path the customer was never told, and nothing anywhere listed what had accumulated. Pressing
// „Ellenőrző visszaállítás" produced a flash saying it had been restored "to a verification folder
// on the drive" — which folder, on which drive, and how much space it was now using were all
// invisible. So copies piled up and the only way to find them was SSH.
//
// The path segments were already open-coded in three places; offsiteRestoreRootFor() is now the one
// place `backups/offsite-restore` is spelled, and offboxRestoreScratchDir() builds on it.
// OffsiteRestoreCopy is one verification copy on disk.
type OffsiteRestoreCopy struct {
Stack string `json:"stack"` // app slug, or SharesPseudoStack for the shares copy
Path string `json:"path"` // absolute path — the thing the customer could not see
Size int64 `json:"size"` // bytes
SizeHuman string `json:"size_human"` // pre-humanized for the template
Created time.Time `json:"created"` // dir mtime; restic writes the tree once, so this is the restore time
}
// offsiteRestoreRootFor returns `<nsRoot>/backups/offsite-restore` for a drive path. THE single place
// these segments are written.
func (m *Manager) offsiteRestoreRootFor(drivePath string) string {
return filepath.Join(m.namespaceRoot(drivePath), "backups", "offsite-restore")
}
// offsiteRestoreDriveRoots returns every drive path a verification copy could live under, in the same
// preference order offboxRestoreScratchDir uses to CHOOSE one — so listing can never miss a copy the
// restore path was capable of creating. Deduplicated, order preserved.
func (m *Manager) offsiteRestoreDriveRoots() []string {
seen := map[string]bool{}
var roots []string
add := func(p string) {
p = strings.TrimSpace(p)
if p == "" || seen[p] {
return
}
seen[p] = true
roots = append(roots, p)
}
// App HDDs first (offboxRestoreScratchDir's rule 1), then every schedulable path (rules 2 and 3).
if m.stackProvider != nil {
for _, s := range m.stackProvider.ListDeployedStacks() {
add(m.stackProvider.GetStackHDDPath(s.Name))
}
}
if m.settings != nil {
for _, sp := range m.settings.GetSchedulableStoragePaths() {
add(sp.Path)
}
}
return roots
}
// ListOffsiteRestoreCopies enumerates every verification copy across every candidate drive, newest
// first. Missing directories are not an error — "none yet" is the normal state.
func (m *Manager) ListOffsiteRestoreCopies() []OffsiteRestoreCopy {
sizer := m.offboxSize()
var out []OffsiteRestoreCopy
seen := map[string]bool{}
for _, drive := range m.offsiteRestoreDriveRoots() {
root := m.offsiteRestoreRootFor(drive)
entries, err := os.ReadDir(root)
if err != nil {
continue // no copies on this drive (or the drive is not mounted) — not an error
}
for _, e := range entries {
if !e.IsDir() {
continue
}
p := filepath.Join(root, e.Name())
if seen[p] {
continue // two stacks can resolve to the same drive; list each path once
}
seen[p] = true
c := OffsiteRestoreCopy{Stack: e.Name(), Path: p}
if fi, err := e.Info(); err == nil {
c.Created = fi.ModTime()
}
c.Size = sizer(p)
c.SizeHuman = humanizeBytes(c.Size)
out = append(out, c)
}
}
sort.Slice(out, func(i, j int) bool { return out[i].Created.After(out[j].Created) })
return out
}
// DeleteOffsiteRestoreCopy removes ONE verification copy.
//
// This is the only delete path v0.147.0 adds, so it is guarded twice over. The stack name must pass
// isSafeStackName (no separators, no traversal), and the resolved path must sit STRICTLY INSIDE a
// `backups/offsite-restore` root that this Manager itself computed — a path that merely looks right
// is refused. Both checks are on the RESOLVED path, not the input, so a symlinked scratch cannot
// walk the delete out of the sandbox.
func (m *Manager) DeleteOffsiteRestoreCopy(stack string) error {
if !isSafeStackName(stack) {
return fmt.Errorf("érvénytelen alkalmazásnév")
}
for _, drive := range m.offsiteRestoreDriveRoots() {
root := m.offsiteRestoreRootFor(drive)
target := filepath.Join(root, stack)
fi, err := os.Stat(target)
if err != nil || !fi.IsDir() {
continue
}
// Prefix safety: only ever remove strictly inside `backups/offsite-restore/`. Same shape as
// the F5 stale-primary prune (backup.go) — refuse loudly rather than best-effort skip, since
// reaching here with an out-of-sandbox path means a helper above is wrong.
cleanTarget := filepath.Clean(target)
cleanRoot := filepath.Clean(root) + string(filepath.Separator)
if !strings.HasPrefix(cleanTarget+string(filepath.Separator), cleanRoot) {
m.logger.Printf("[WARN] [offbox] refusing to delete verification copy outside %s: %s", root, cleanTarget)
return fmt.Errorf("a törlés útvonala kívül esik az ellenőrző mappán")
}
if err := os.RemoveAll(cleanTarget); err != nil {
return fmt.Errorf("a másolat törlése nem sikerült: %w", err)
}
m.logger.Printf("[INFO] [offbox] deleted verification copy: %s", cleanTarget)
return nil
}
return fmt.Errorf("nincs ilyen ellenőrző másolat")
}
// OffsiteRestoreScratchPath exposes WHERE a verification restore for stack would land, so the UI can
// name the full path in the completion message instead of saying "a verification folder somewhere".
func (m *Manager) OffsiteRestoreScratchPath(stack string) string {
scratch, _, err := m.offboxRestoreScratchDir(stack)
if err != nil {
return ""
}
return scratch
}
@@ -0,0 +1,181 @@
package backup
import (
"os"
"path/filepath"
"testing"
)
// v0.147.0 slice 4a — the verification-copy listing/delete surface.
//
// DeleteOffsiteRestoreCopy is the ONLY delete this slice adds, so these tests are about what it must
// REFUSE as much as what it must do. Every refusal is asserted as a NON-EFFECT: the neighbouring copy
// and the customer's live data must still be on disk afterwards. A guard that returns an error but
// deletes anyway passes a naive test and loses data.
//
// RED-PROOF (run manually, confirmed): neutralise the isSafeStackName check in
// DeleteOffsiteRestoreCopy and TestDeleteVerifyCopyRefusesUnsafeNames fails hard — `stack: ""`
// resolves to the offsite-restore ROOT and os.RemoveAll takes every verification copy with it. That
// is the failure this guard exists to prevent, and it is data loss, not a bad error message.
//
// The HasPrefix containment check inside DeleteOffsiteRestoreCopy could NOT be red-proofed
// independently: with isSafeStackName in front of it, no input this API accepts can reach it with an
// escaping path, so removing it leaves every test green (and the variable unused). It is deliberate
// defence-in-depth against a future caller or a refactor that loosens the name check — kept, but
// honestly labelled here as unproven-by-test rather than pretending to a red-proof it does not have.
// verifyCopyEnv wires a manager with one drive and materialised verification copies.
type verifyCopyEnv struct {
m *Manager
drive string
root string // <nsRoot>/backups/offsite-restore
}
func newVerifyCopyEnv(t *testing.T, copies ...string) *verifyCopyEnv {
t.Helper()
drive := t.TempDir()
m, _, _ := classifiedOffboxManager(t, drive)
root := m.offsiteRestoreRootFor(drive)
for _, c := range copies {
p := filepath.Join(root, c)
if err := os.MkdirAll(p, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(p, "payload.bin"), []byte("restored bytes"), 0o644); err != nil {
t.Fatal(err)
}
}
return &verifyCopyEnv{m: m, drive: drive, root: root}
}
func TestListOffsiteRestoreCopiesReportsPathAndSize(t *testing.T) {
e := newVerifyCopyEnv(t, "immich", "nextcloud")
got := e.m.ListOffsiteRestoreCopies()
if len(got) != 2 {
t.Fatalf("listed %d copies, want 2: %+v", len(got), got)
}
byStack := map[string]OffsiteRestoreCopy{}
for _, c := range got {
byStack[c.Stack] = c
}
for _, name := range []string{"immich", "nextcloud"} {
c, ok := byStack[name]
if !ok {
t.Fatalf("%s missing from the listing", name)
}
// THE POINT of the listing: the customer could not previously see WHERE the copy was.
want := filepath.Join(e.root, name)
if c.Path != want {
t.Errorf("%s path = %q, want %q", name, c.Path, want)
}
if c.SizeHuman == "" {
t.Errorf("%s has no humanized size", name)
}
if c.Created.IsZero() {
t.Errorf("%s has no creation time", name)
}
}
}
func TestListOffsiteRestoreCopiesEmptyIsNotAnError(t *testing.T) {
// No offsite-restore directory at all — the normal state on a box that never ran a verification
// restore. Must be an empty list, not a crash and not a phantom entry.
e := newVerifyCopyEnv(t)
if got := e.m.ListOffsiteRestoreCopies(); len(got) != 0 {
t.Errorf("listed %d copies on a clean box, want 0: %+v", len(got), got)
}
}
func TestDeleteVerifyCopyRemovesOnlyTheNamedOne(t *testing.T) {
e := newVerifyCopyEnv(t, "immich", "nextcloud")
// Live customer data next to the sandbox — must be untouched by any delete.
live := filepath.Join(e.drive, "immich-live")
if err := os.MkdirAll(live, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(live, "photo.jpg"), []byte("irreplaceable"), 0o644); err != nil {
t.Fatal(err)
}
if err := e.m.DeleteOffsiteRestoreCopy("immich"); err != nil {
t.Fatalf("delete: %v", err)
}
if _, err := os.Stat(filepath.Join(e.root, "immich")); !os.IsNotExist(err) {
t.Error("the named copy survived the delete")
}
if _, err := os.Stat(filepath.Join(e.root, "nextcloud", "payload.bin")); err != nil {
t.Errorf("a NEIGHBOURING copy was destroyed: %v", err)
}
if _, err := os.Stat(filepath.Join(live, "photo.jpg")); err != nil {
t.Errorf("LIVE CUSTOMER DATA was destroyed: %v", err)
}
}
func TestDeleteVerifyCopyRefusesUnsafeNames(t *testing.T) {
e := newVerifyCopyEnv(t, "immich")
// Something outside the sandbox that a traversal would reach.
outside := filepath.Join(e.drive, "backups", "primary")
if err := os.MkdirAll(outside, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(outside, "unit.tar"), []byte("recovery unit"), 0o644); err != nil {
t.Fatal(err)
}
for _, bad := range []string{
"../primary",
"../../..",
"..",
"immich/../../primary",
"/etc",
"",
} {
if err := e.m.DeleteOffsiteRestoreCopy(bad); err == nil {
t.Errorf("delete(%q) was ACCEPTED — it must be refused", bad)
}
// The refusal must also be a NON-EFFECT.
if _, err := os.Stat(filepath.Join(outside, "unit.tar")); err != nil {
t.Fatalf("delete(%q) destroyed data outside the sandbox: %v", bad, err)
}
if _, err := os.Stat(filepath.Join(e.root, "immich", "payload.bin")); err != nil {
t.Fatalf("delete(%q) destroyed an unrelated copy: %v", bad, err)
}
}
}
func TestDeleteVerifyCopyStaysInsideTheSandbox(t *testing.T) {
e := newVerifyCopyEnv(t, "immich")
// A well-formed name that simply does not exist must be an error, never a silent success that
// could mask a path-resolution bug.
if err := e.m.DeleteOffsiteRestoreCopy("no-such-app"); err == nil {
t.Error("deleting a non-existent copy reported success")
}
// Everything under the sandbox root must resolve strictly inside it.
for _, c := range e.m.ListOffsiteRestoreCopies() {
rel, err := filepath.Rel(e.root, c.Path)
if err != nil || rel == ".." || filepath.IsAbs(rel) || len(rel) > 2 && rel[:2] == ".." {
t.Errorf("listed copy %q resolves outside %q", c.Path, e.root)
}
}
}
// TestOffsiteRestoreScratchPathMatchesTheListing pins the two halves together: the path the UI names
// in the completion flash must be the same path the listing (and therefore the delete button) uses.
// If these ever diverge, the customer is told about a directory the page cannot show or remove.
func TestOffsiteRestoreScratchPathMatchesTheListing(t *testing.T) {
e := newVerifyCopyEnv(t, "immich")
named := e.m.OffsiteRestoreScratchPath("immich")
if named == "" {
t.Fatal("OffsiteRestoreScratchPath returned empty — the flash would fall back to the vague wording")
}
var listed string
for _, c := range e.m.ListOffsiteRestoreCopies() {
if c.Stack == "immich" {
listed = c.Path
}
}
if named != listed {
t.Errorf("flash names %q but the listing shows %q", named, listed)
}
}
+86
View File
@@ -0,0 +1,86 @@
package infra
import (
"go/ast"
"go/parser"
"go/token"
"strconv"
"strings"
"testing"
)
// TestImagesCoversEveryPin is the anti-drift gate for the golden bake.
//
// Images() feeds `felhom-controller --print-infra-images`, which build-golden.sh uses to decide what
// to pre-pull into the appliance image. If someone adds a fifth infra stack — a new `FooImage` const
// — and forgets to add it to Images(), the golden silently bakes 4 of 5 and enabling that stack on a
// fresh box goes back to being a multi-minute silent registry pull. That is exactly how felhom-samba
// was missed, so this test reads the CONST BLOCK OUT OF THE SOURCE rather than restating the list:
// a hand-written expected list would need the same edit and would rot the same way.
func TestImagesCoversEveryPin(t *testing.T) {
fset := token.NewFileSet()
f, err := parser.ParseFile(fset, "infra.go", nil, 0)
if err != nil {
t.Fatalf("parsing infra.go: %v", err)
}
pins := map[string]string{} // const name -> image ref
for _, decl := range f.Decls {
gd, ok := decl.(*ast.GenDecl)
if !ok || gd.Tok != token.CONST {
continue
}
for _, spec := range gd.Specs {
vs, ok := spec.(*ast.ValueSpec)
if !ok || len(vs.Names) != 1 || len(vs.Values) != 1 {
continue
}
name := vs.Names[0].Name
if !strings.HasSuffix(name, "Image") {
continue
}
lit, ok := vs.Values[0].(*ast.BasicLit)
if !ok || lit.Kind != token.STRING {
continue
}
val, err := strconv.Unquote(lit.Value)
if err != nil {
t.Fatalf("unquoting %s: %v", name, err)
}
pins[name] = val
}
}
if len(pins) == 0 {
t.Fatal("found no *Image consts in infra.go — the parser walk is broken, not the pins")
}
got := map[string]bool{}
for _, img := range Images() {
got[img] = true
}
for name, img := range pins {
if !got[img] {
t.Errorf("const %s = %q is not in Images() — the golden bake would not pre-pull it, so "+
"enabling that stack on a fresh box would block on a silent registry pull", name, img)
}
}
if len(Images()) != len(pins) {
t.Errorf("Images() has %d entries but infra.go declares %d *Image consts (%v) — they must "+
"correspond one-to-one", len(Images()), len(pins), pins)
}
}
// TestImagesArePinned guards the other half of the contract: a floating tag would make the golden
// bake unreproducible (the bake and a later deploy could resolve the same name to different digests).
func TestImagesArePinned(t *testing.T) {
for _, img := range Images() {
if !strings.Contains(img, ":") {
t.Errorf("infra image %q has no tag — implicitly :latest", img)
continue
}
if strings.HasSuffix(img, ":latest") {
t.Errorf("infra image %q is :latest — a floating tag breaks reproducible golden bakes", img)
}
}
}
+17
View File
@@ -30,6 +30,23 @@ const (
SambaImage = "gitea.dooplex.hu/admin/felhom-samba:1.0.0"
)
// Images returns every controller-managed infra image, derived from the pins above so there is
// exactly one place a tag is written.
//
// WHY THIS IS EXPORTED: the golden bake (felhom-agent configs/build-golden.sh) pre-pulls these into
// the appliance image so enabling an infra stack on a fresh box is near-instant instead of a silent
// multi-minute registry pull. It used to carry its OWN hand-maintained bash array of tags — which
// drifted the moment felhom-samba was added: the golden baked three of the four, so turning on
// Megosztás pulled from the registry with zero feedback. The bake now asks the controller BINARY it
// is about to bake (`felhom-controller --print-infra-images`), so the golden and the controller it
// ships cannot disagree by construction.
//
// A new infra stack is therefore two edits in this file (the const, and this slice) and zero edits
// anywhere else. If you add a const and forget the slice, TestImagesCoversEveryPin fails.
func Images() []string {
return []string{TraefikImage, CloudflaredImage, FileBrowserImage, SambaImage}
}
//go:embed templates/*.tmpl
var templateFS embed.FS
+1
View File
@@ -118,6 +118,7 @@ type Manager struct {
sambaUpFn func(dir string) error
sambaPasswdFn func(password string) error
sambaRunFn func() bool // replaces the docker-inspect liveness probe
sambaImgFn func() bool // replaces the `docker image inspect` local-presence probe (4b card)
}
// NewManager creates a new stack manager.
+14
View File
@@ -85,6 +85,20 @@ func (m *Manager) sambaIsRunning() bool {
return containerRunning(sambaContainer)
}
// SambaImagePresent reports whether the pinned samba image is already in local Docker storage.
//
// This is what makes the progress card HONEST rather than decorative: on a golden that baked the
// image (see felhom-agent build-golden.sh) the bring-up is seconds and the card should say
// „elindítás"; on a box that must fetch ~100MB from the registry it is minutes and the card must say
// „képfájl letöltése" so the wait is explained instead of silent. Asked BEFORE compose runs, because
// afterwards the answer is always yes.
func (m *Manager) SambaImagePresent() bool {
if m.sambaImgFn != nil {
return m.sambaImgFn()
}
return exec.Command("docker", "image", "inspect", infra.SambaImage).Run() == nil
}
// shareAvailable reports whether a share's folder can be exported right now: its owning registered
// storage path must be neither disconnected nor decommissioned, AND the folder must exist. A dead
// mount is NEVER exported — publishing a missing mountpoint would show an empty share and let a
+5
View File
@@ -823,6 +823,11 @@ func (s *Server) backupsRestoreHandler(w http.ResponseWriter, r *http.Request) {
data["SharesScratchReady"] = s.backupMgr.SharesScratchReady()
data["SharesDisplayName"] = backup.SharesDisplayName
}
// v0.147.0 (4a): what verification restores have actually left on disk. Previously nothing on any
// page listed these, so they accumulated invisibly and the only way to find them was SSH.
if s.backupMgr != nil {
data["OffsiteRestoreCopies"] = s.backupMgr.ListOffsiteRestoreCopies()
}
s.executeTemplate(w, r, "backups_restore", data)
}
+56 -3
View File
@@ -219,11 +219,13 @@ func (s *Server) offboxRunHandler(w http.ResponseWriter, r *http.Request) {
go func() {
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Hour)
defer cancel()
if err := s.backupMgr.RunOffboxBackup(ctx); err != nil {
// ...WithProgress: this is the MANUAL trigger, so the page gets live bytes/percent/current app
// (4c). The nightly scheduler keeps calling RunOffboxBackup and stays silent.
if err := s.backupMgr.RunOffboxBackupWithProgress(ctx); err != nil {
s.logger.Printf("[WARN] [web] manual off-box backup failed: %v", err)
}
}()
offboxRedirect(w, r, "A távoli mentés elindult (a futás után az állapot frissül).", false)
offboxRedirect(w, r, "A távoli mentés elindult az állapot itt frissül.", false)
}
// offboxResetHandler is the CLAIMED confirmed orphaned-repo reset (Scenario C): move the old (recovery-
@@ -266,6 +268,11 @@ func (s *Server) offboxStatusHandler(w http.ResponseWriter, r *http.Request) {
resp["last_error"] = t.LastError
resp["orphaned"] = t.RepoState == "orphaned"
}
// v0.147.0 (4c): live progress for a MANUAL run. Absent/inactive on the nightly path, so the
// page simply keeps its previous „Fut…" behavior there.
if s.backupMgr != nil {
resp["progress"] = s.backupMgr.OffboxProgressSnapshot()
}
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode(resp)
}
@@ -319,11 +326,57 @@ func (s *Server) offboxRestoreHandler(w http.ResponseWriter, r *http.Request) {
return
}
s.logger.Printf("[INFO] [web] off-box restore %s completed (full=%v, async)", app, full)
s.backupMgr.EndRestoreOp(true, "A(z) "+app+" visszaállítva ellenőrző mappába a meghajtón (a meglévő adatok változatlanok).")
// NAME THE RESULT (v0.147.0, 4a). The old message said the app had been restored "to a
// verification folder on the drive" — which folder, on which drive, was invisible, so the
// customer had no way to look at what they had just asked for. Resolve the real path and say
// it. Fall back to the vague wording only if the path can no longer be resolved.
where := s.backupMgr.OffsiteRestoreScratchPath(app)
msg := "A(z) " + app + " visszaállítva ellenőrző mappába a meghajtón (a meglévő adatok változatlanok)."
if where != "" {
msg = "A(z) " + app + " visszaállítva ellenőrző mappába: " + where + " (a meglévő adatok változatlanok)."
}
s.backupMgr.EndRestoreOp(true, msg)
}()
offboxRedirectTo(w, r, "/backups/restore", "A távoli visszaállítás elindult — az állapot itt frissül.", false)
}
// offboxVerifyCopyDeleteHandler removes ONE verification copy (v0.147.0, 4a).
//
// The only delete this slice adds, so it is deliberately narrow: it names a STACK, never a path — the
// customer cannot hand us a path to remove. The Manager resolves that name inside a
// `backups/offsite-restore` root it computed itself and refuses anything that lands outside (see
// DeleteOffsiteRestoreCopy). The template double-confirms before POSTing.
//
// It refuses while a backup/restore op is running: the copy being deleted could be the one currently
// being written.
func (s *Server) offboxVerifyCopyDeleteHandler(w http.ResponseWriter, r *http.Request) {
if s.backupMgr == nil {
offboxRedirectTo(w, r, "/backups/restore", "A mentéskezelő nem érhető el.", true)
return
}
_ = r.ParseForm()
stack := strings.TrimSpace(r.FormValue("stack"))
if stack == "" {
offboxRedirectTo(w, r, "/backups/restore", "Hiányzó ellenőrző másolat.", true)
return
}
if r.FormValue("confirm") != "1" {
offboxRedirectTo(w, r, "/backups/restore", "A törlés megerősítés nélkül nem hajtható végre.", true)
return
}
if s.backupMgr.IsRunning() {
offboxRedirectTo(w, r, "/backups/restore", "Egy mentési/visszaállítási művelet fut — a törlés most nem biztonságos.", true)
return
}
if err := s.backupMgr.DeleteOffsiteRestoreCopy(stack); err != nil {
s.logger.Printf("[WARN] [web] verification-copy delete %s: %v", stack, err)
offboxRedirectTo(w, r, "/backups/restore", "A másolat törlése nem sikerült: "+err.Error(), true)
return
}
s.logger.Printf("[INFO] [web] verification copy deleted: %s", stack)
offboxRedirectTo(w, r, "/backups/restore", "Az ellenőrző másolat törölve. A tényleges adataid változatlanok.", false)
}
// offboxPlaceHandler places a COMPLETED full-restore scratch into the app's live locations via a
// missing-only merge (§7.3). Never overwrites existing files. Async on a background context.
func (s *Server) offboxPlaceHandler(w http.ResponseWriter, r *http.Request) {
+156
View File
@@ -0,0 +1,156 @@
package web
import (
"sync"
"time"
)
// Samba bring-up progress (v0.147.0, feedback slice 4b).
//
// THE PROBLEM: enabling Megosztás on a fresh box ran `ReconcileSamba()` synchronously inside the
// POST handler. On a golden that had not baked felhom-samba, that call is `docker compose up -d`
// pulling ~100MB from a private registry — minutes of an apparently-hung form post, then a redirect
// with a flash reading „Beállítás mentve." whether or not anything had actually come up. Observed
// live, twice. The image is now baked (felhom-agent build-golden.sh, golden >= 0.147.x), but this
// card still covers the pre-0.147 goldens and every future image update.
//
// SHAPE: deliberately the storage-init / netstorage-add one (storage_init_job.go) — detached job,
// single-flight slot, deep-copied snapshot, phase strings the template maps to Hungarian. No new
// framework; a unified async-job feedback layer is a ROADMAP item, not this slice.
//
// State is in-memory and lost on restart, exactly like RestoreOpStatus. That is acceptable here: the
// terminal truth is „is the container running", which the page re-reads from the stack manager on
// every load anyway — the job only explains the WAIT.
type sambaEnsureJob struct {
Phase string `json:"phase"`
Error string `json:"error,omitempty"`
StartedAt time.Time `json:"started_at"`
UpdatedAt time.Time `json:"updated_at"`
}
const (
// sambaPhasePulling — the pinned image is NOT in local Docker storage, so compose will fetch it.
// This is the phase worth naming: it is the multi-minute one, and the only honest explanation for
// why nothing appears to happen.
sambaPhasePulling = "pulling"
// sambaPhaseStarting — image already local (baked golden / previously pulled): seconds.
sambaPhaseStarting = "starting"
// sambaPhaseRunning — terminal success, PROBED (compose up -d exits 0 on a crash-loop, so the
// job's success condition is container liveness, never the compose exit code).
sambaPhaseRunning = "running"
// sambaPhaseNeedsPassword — not a failure: sharing is on but the household password is unset, so
// reconcile deliberately deploys nothing. The card must say so instead of spinning forever.
sambaPhaseNeedsPassword = "needs_password"
sambaPhaseFailed = "failed"
sambaPhaseIdle = "idle"
)
// A pull that has not finished in 15 minutes is not slow, it is broken (the registry is unreachable
// or the disk is full) — end the job so the card can say so rather than spin indefinitely.
const sambaEnsureDeadline = 15 * time.Minute
type sambaEnsureState struct {
mu sync.Mutex
running bool
cur *sambaEnsureJob
}
func (s *sambaEnsureState) acquire(job *sambaEnsureJob) bool {
s.mu.Lock()
defer s.mu.Unlock()
if s.running {
return false
}
s.running = true
cp := *job
s.cur = &cp
return true
}
func (s *sambaEnsureState) release() {
s.mu.Lock()
s.running = false
s.mu.Unlock()
}
func (s *sambaEnsureState) set(job *sambaEnsureJob) {
s.mu.Lock()
cp := *job
s.cur = &cp
s.mu.Unlock()
}
// snapshot returns a copy of the last / in-flight job (nil = never ran this process).
func (s *sambaEnsureState) snapshot() *sambaEnsureJob {
s.mu.Lock()
defer s.mu.Unlock()
if s.cur == nil {
return nil
}
cp := *s.cur
return &cp
}
// startSambaEnsure claims the single-flight slot and launches the detached reconcile. false = one is
// already in flight (a double-submit must not start a second compose up on the same stack dir).
//
// The opening phase is decided BEFORE the work starts, by asking whether the image is already local
// — afterwards the answer is always yes and the card could never truthfully say „letöltés".
func (s *Server) startSambaEnsure() bool {
now := time.Now().UTC()
phase := sambaPhaseStarting
if s.stackMgr != nil && !s.stackMgr.SambaImagePresent() {
phase = sambaPhasePulling
}
job := &sambaEnsureJob{Phase: phase, StartedAt: now, UpdatedAt: now}
if !s.sambaEnsure.acquire(job) {
return false
}
go s.runSambaEnsureJob(job)
return true
}
func (s *Server) runSambaEnsureJob(job *sambaEnsureJob) {
defer s.sambaEnsure.release()
advance := func(phase, errMsg string) {
job.Phase = phase
job.Error = errMsg
job.UpdatedAt = time.Now().UTC()
s.sambaEnsure.set(job)
}
done := make(chan error, 1)
go func() { done <- s.stackMgr.ReconcileSamba() }()
select {
case err := <-done:
if err != nil {
s.logger.Printf("[ERROR] [sharing] samba reconcile failed after %s: %v", time.Since(job.StartedAt).Round(time.Second), err)
advance(sambaPhaseFailed, err.Error())
return
}
case <-time.After(sambaEnsureDeadline):
// The reconcile goroutine is left running — compose owns its own lifecycle and killing it
// mid-pull would leave a partial layer set. We stop REPORTING on it, which is the honest
// thing the customer needs; a later page load re-probes liveness for the real answer.
s.logger.Printf("[ERROR] [sharing] samba reconcile still running after %s — giving up on the progress card", sambaEnsureDeadline)
advance(sambaPhaseFailed, "a megosztási szolgáltatás előkészítése túl sokáig tartott")
return
}
// Reconcile returned nil — but nil ALSO covers "deliberately did nothing". Distinguish the two,
// because a card that says „fut" while nothing is deployed is the same silence in a new costume.
if s.settings != nil && !s.settings.GetSMBSettings().UserSet {
advance(sambaPhaseNeedsPassword, "")
return
}
if !s.stackMgr.SambaRunning() {
s.logger.Printf("[ERROR] [sharing] samba reconcile reported success but the container is not running")
advance(sambaPhaseFailed, "a megosztási szolgáltatás nem indult el")
return
}
s.logger.Printf("[INFO] [sharing] samba ready after %s", time.Since(job.StartedAt).Round(time.Second))
advance(sambaPhaseRunning, "")
}
@@ -0,0 +1,86 @@
package web
import (
"testing"
)
// v0.147.0 slice 4b — the Megosztás bring-up progress card.
//
// The card's whole value is that it distinguishes states the old synchronous handler collapsed into
// one silent form post. So the tests are about the DISTINCTIONS:
// - "pulling" vs "starting" — decided by local image presence, and decided BEFORE the work starts
// (afterwards the image is always present and the card could never truthfully say „letöltés").
// - reconcile-returned-nil is NOT the same as running: it also covers "deliberately deployed
// nothing because there is no household password yet".
// - compose up -d exits 0 on a crash-loop, so success must be PROBED, never inferred.
//
// SCOPE, STATED HONESTLY: these cover the single-flight slot, snapshot isolation and the phase
// vocabulary. They do NOT drive runSambaEnsureJob end-to-end — that needs a real *stacks.Manager
// (the Server field is the concrete type, not an interface), and introducing an interface purely for
// this card was more churn than the slice warranted. The pulling-vs-starting decision and the
// probed-liveness terminal state are therefore covered by the LIVE validation on the demo box, not
// by a unit test. Recorded here rather than left as an implied gap; the ROADMAP's unified
// async-job feedback item is where that seam belongs.
func TestSambaEnsureSingleFlight(t *testing.T) {
var st sambaEnsureState
job := &sambaEnsureJob{Phase: sambaPhaseStarting}
if !st.acquire(job) {
t.Fatal("first acquire refused")
}
// A double-submit must not start a second `compose up -d` against the same stack dir.
if st.acquire(&sambaEnsureJob{Phase: sambaPhaseStarting}) {
t.Error("second acquire succeeded while a job was in flight")
}
st.release()
if !st.acquire(&sambaEnsureJob{Phase: sambaPhaseStarting}) {
t.Error("acquire refused after release")
}
}
func TestSambaEnsureSnapshotIsACopy(t *testing.T) {
var st sambaEnsureState
job := &sambaEnsureJob{Phase: sambaPhasePulling}
st.acquire(job)
snap := st.snapshot()
snap.Phase = "mutated-by-caller"
if again := st.snapshot(); again.Phase != sambaPhasePulling {
t.Errorf("a caller mutated the shared job through its snapshot: %q", again.Phase)
}
// And the live job must not leak into an already-taken snapshot either.
snap2 := st.snapshot()
job.Phase = sambaPhaseRunning
st.set(job)
if snap2.Phase != sambaPhasePulling {
t.Errorf("an earlier snapshot changed under the caller: %q", snap2.Phase)
}
}
func TestSambaEnsureStateStartsNil(t *testing.T) {
var st sambaEnsureState
// nil = never ran this process. The status handler maps that to `idle` and lets live container
// state win, so a page loaded after a restart still tells the truth.
if j := st.snapshot(); j != nil {
t.Errorf("fresh state reported a job: %+v", j)
}
}
// TestSambaPhaseConstantsAreDistinct guards the template contract: sharing.html maps these exact
// strings, and a duplicate would silently render the wrong Hungarian sentence.
func TestSambaPhaseConstantsAreDistinct(t *testing.T) {
all := []string{
sambaPhasePulling, sambaPhaseStarting, sambaPhaseRunning,
sambaPhaseNeedsPassword, sambaPhaseFailed, sambaPhaseIdle,
}
seen := map[string]bool{}
for _, p := range all {
if p == "" {
t.Error("a phase constant is empty — the template would fall through to 'no card'")
}
if seen[p] {
t.Errorf("duplicate phase constant %q", p)
}
seen[p] = true
}
}
+8
View File
@@ -94,6 +94,8 @@ type Server struct {
// netAgentFn nil → the shared agentClient(); netProbeFn nil → runNetProbe (the uid-1000 re-exec).
netAdd netAddState
storageInit storageInitState
// sambaEnsure is the Megosztás bring-up progress slot (v0.147.0, 4b) — same single-flight shape.
sambaEnsure sambaEnsureState
netAgentFn func() (netAgent, error)
// fabUpload is the chunked browser .fab upload single-flight slot (v0.128.0).
fabUpload uploadState
@@ -359,6 +361,8 @@ func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
s.sharingPageHandler(w, r)
case path == "/sharing/enable" && r.Method == http.MethodPost:
s.sharingEnableHandler(w, r)
case path == "/sharing/status" && r.Method == http.MethodGet:
s.sharingStatusHandler(w, r)
case path == "/sharing/password" && r.Method == http.MethodPost:
s.sharingPasswordHandler(w, r)
case path == "/sharing/shares" && r.Method == http.MethodPost:
@@ -418,6 +422,10 @@ func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
s.offboxRestoreHandler(w, r)
case path == "/backup/offbox/place" && r.Method == http.MethodPost:
s.offboxPlaceHandler(w, r)
// v0.147.0 (4a): remove ONE verification copy. The only delete path this slice adds — see
// DeleteOffsiteRestoreCopy for the prefix guard.
case path == "/backup/offbox/verify-copy/delete" && r.Method == http.MethodPost:
s.offboxVerifyCopyDeleteHandler(w, r)
// R-7b: „Megosztások" restore — a sibling of the per-app pair above.
case path == "/backup/shares/restore" && r.Method == http.MethodPost:
s.sharesRestoreHandler(w, r)
+33 -6
View File
@@ -221,10 +221,33 @@ func (s *Server) sharingEnableHandler(w http.ResponseWriter, r *http.Request) {
sharingRedirect(w, r, "A hálózati megosztás kikapcsolva. A mappák és a fájlok megmaradtak.")
return
}
if err := s.stackMgr.ReconcileSamba(); err != nil {
s.logger.Printf("[WARN] [sharing] reconcile failed: %v", err)
// v0.147.0 (4b): the bring-up runs DETACHED and the page polls it. Synchronously it was a form
// post that hung for minutes on a first-enable image pull and then flashed „Beállítás mentve."
// regardless of whether anything actually came up.
if !s.startSambaEnsure() {
sharingRedirect(w, r, "A megosztási szolgáltatás előkészítése már folyamatban van.")
return
}
sharingRedirect(w, r, "Beállítás mentve.")
sharingRedirect(w, r, "Beállítás mentve. A megosztási szolgáltatás előkészítése folyamatban…")
}
// sharingStatusHandler is the 4b poll target (GET /sharing/status). Reports the ensure job's phase
// plus the live container state, so a page loaded AFTER the job finished (or after a restart, when
// the in-memory job is gone) still shows the truth.
func (s *Server) sharingStatusHandler(w http.ResponseWriter, r *http.Request) {
phase := sambaPhaseIdle
errMsg := ""
if job := s.sambaEnsure.snapshot(); job != nil {
phase, errMsg = job.Phase, job.Error
}
running := s.stackMgr != nil && s.stackMgr.SambaRunning()
// A stale `idle`/`running` job must never contradict reality: liveness wins on a fresh page.
if phase == sambaPhaseIdle && running {
phase = sambaPhaseRunning
}
writeDiskJSON(w, http.StatusOK, true, "", map[string]any{
"phase": phase, "error": errMsg, "running": running,
})
}
// sharingPasswordHandler sets the household SMB password (POST /sharing/password).
@@ -247,10 +270,14 @@ func (s *Server) sharingPasswordHandler(w http.ResponseWriter, r *http.Request)
sharingRedirect(w, r, "A jelszó beállítása nem sikerült.")
return
}
if err := s.stackMgr.ReconcileSamba(); err != nil {
s.logger.Printf("[WARN] [sharing] reconcile after password failed: %v", err)
// Setting the password is the moment the stack ACTUALLY first comes up: with UserSet false,
// reconcile deliberately deploys nothing, so on a fresh box this — not the enable toggle — is
// where the image pull happens. Same detached job, same card.
if !s.startSambaEnsure() {
sharingRedirect(w, r, "Megosztási jelszó beállítva. Az előkészítés már folyamatban van.")
return
}
sharingRedirect(w, r, "Megosztási jelszó beállítva.")
sharingRedirect(w, r, "Megosztási jelszó beállítva. A megosztási szolgáltatás előkészítése folyamatban…")
}
// sharingShareCreateHandler creates a share (POST /sharing/shares). Either a NEW folder under
@@ -126,6 +126,15 @@
<button type="submit" class="btn btn-sm btn-primary">Távoli mentés most</button>
</form>
</div>
<!-- v0.147.0 (4c): live progress for a MANUAL run — total bytes, percent and which app is being
pushed, parsed from restic's --json status stream. Hidden until a run reports progress; the
nightly run never populates it. -->
<div id="offbox-progress" class="alert alert-info" style="display:none;margin-top:.75rem">
<span id="offbox-progress-text"></span>
<div class="progress-bar-task" style="margin-top:.5rem">
<div id="offbox-progress-bar" class="progress-fill" style="width:0%"></div>
</div>
</div>
<h4 style="margin-top:1.25rem">Mely alkalmazások mentődnek a távoli tárolóra?</h4>
{{if and .OffboxApps (eq .Offbox.EscrowState "escrowed") (eq .OffboxToggledCount 0)}}
<p class="form-hint">Nincs távoli mentésre jelölt alkalmazás — jelölj ki legalább egyet.</p>
@@ -172,19 +181,55 @@
{{end}}
<script>
/* Part C (v0.142.0): while a remote backup is in flight the status card shows "Fut…" — poll the run
status and reload once it reaches a terminal state, so the customer sees Rendben/Hiba + fresh
numbers WITHOUT a manual reload. Polls only when a run is running; stops (page reload) at terminal. */
/* Part C (v0.142.0) + 4c (v0.147.0): poll the run status, render live progress for a manual run, and
reload once it reaches a terminal state so the customer sees Rendben/Hiba + fresh numbers WITHOUT a
manual reload.
v0.147.0 changed WHEN polling starts. It used to begin only if the page already rendered "Fut…",
which loses a race the manual trigger always runs: the POST redirects and this page renders before
the detached goroutine has written LastStatus=running, so the poll never armed and the customer
watched a static page during the very run they had just started. Now it always arms and gives up
after GRACE quiet ticks if nothing is (or ever was) in flight. */
(function(){
var v=document.getElementById('offbox-status-value');
if(!v || v.textContent.indexOf('Fut')<0) return; // not running nothing to poll
var timer=setInterval(function(){
var v = document.getElementById('offbox-status-value');
var box = document.getElementById('offbox-progress');
var text = document.getElementById('offbox-progress-text');
var bar = document.getElementById('offbox-progress-bar');
var GRACE = 5; // ~15s of quiet before concluding nothing is running
var quiet = 0;
var sawRunning = (v && v.textContent.indexOf('Fut') >= 0);
var timer = setInterval(function(){
fetch('/backup/offbox/status',{headers:{'Accept':'application/json'}})
.then(function(r){return r.json();})
.then(function(d){
if(d && d.status==='running') return; // still running → keep polling
clearInterval(timer);
location.reload(); // terminal → re-render (fresh numbers, warnings, or the orphan card)
if(!d) return;
var p = d.progress || {};
var running = (d.status === 'running') || p.active;
if(running){
sawRunning = true; quiet = 0;
if(p.active && box){
/* Only claim a percentage once restic has told us a total — before the scan finishes,
percent is 0 of 0, and a bar pinned at 0% reads as "stuck" rather than "measuring". */
var label = p.current_app ? ('Mentés: ' + p.current_app) : 'Mentés folyamatban';
if(p.total_bytes > 0){
label += ' — ' + Math.round(p.percent) + '% (' + p.done_human + ' / ' + p.total_human + ')';
bar.style.width = Math.max(0, Math.min(100, p.percent)) + '%';
} else {
label += ' — a mentendő adatok felmérése…';
bar.style.width = '0%';
}
text.textContent = label;
box.style.display = '';
}
return;
}
/* Not running. Terminal only if we ever saw it running — otherwise this is the pre-start
race (or a page nobody triggered anything from). */
if(sawRunning){ clearInterval(timer); location.reload(); return; }
if(++quiet >= GRACE){ clearInterval(timer); }
})
.catch(function(){ /* transient — keep polling */ });
}, 3000);
@@ -121,6 +121,32 @@
{{template "app_list_row_end"}}
</div>
{{end}}
<!-- v0.147.0 (4a): what the verification restores actually LEFT on disk. Until now the result of
an ellenőrző visszaállítás was invisible — the flash said "a verification folder on the
drive" without naming it, and nothing listed what had accumulated. Full path, size and date,
so the copy can be found, opened, and cleaned up. -->
<div class="backup-tier-divider" style="margin-top:1.5rem"></div>
<h3 style="margin-bottom:.5rem">Meglévő ellenőrző másolatok</h3>
{{if .OffsiteRestoreCopies}}
<p class="form-hint" style="margin-bottom:.75rem">Ezek a visszaállított másolatok helyet foglalnak a meghajtón. A tényleges adataidat nem érintik, bármikor törölhetők.</p>
<div class="app-row-list">
{{range .OffsiteRestoreCopies}}
{{template "app_list_row" dict "Slug" .Stack "Name" .Stack "Secondary" (printf "%s · %s · %s" .SizeHuman (.Created.Format "2006. 01. 02. 15:04") .Path)}}
<form method="POST" action="/backup/offbox/verify-copy/delete" style="display:inline">
{{$.CSRFField}}
<input type="hidden" name="stack" value="{{.Stack}}">
<input type="hidden" name="confirm" value="1">
<button type="button" class="btn btn-xs btn-danger-outline"
data-copy-path="{{.Path}}"
onclick="confirmDeleteVerifyCopy(this)">Másolat törlése</button>
</form>
{{template "app_list_row_end"}}
{{end}}
</div>
{{else}}
<p class="form-hint">Nincs ellenőrző másolat a meghajtón.</p>
{{end}}
</div>
{{end}}
@@ -178,6 +204,19 @@ function fabDownload(btn){
})
.catch(function(){ fabSetStatus('Hiba: a becslés nem érhető el.'); });
}
// v0.147.0 (4a): the ONLY delete this page offers. Two inline acknowledgements — the house
// double-confirm idiom (deploy.html's stale-data delete), never native confirm(), which is an
// OS-modal that blocks browser automation (F-11). The first step names the exact path so the
// customer is agreeing to a specific directory, not to the word "delete".
function confirmDeleteVerifyCopy(btn){
var path = btn.getAttribute('data-copy-path') || '';
felhomConfirm(btn, 'Biztosan törlöd ezt az ellenőrző másolatot? ' + path + ' — a tényleges adataid változatlanok maradnak.', function(){
felhomConfirm(btn, 'UTOLSÓ MEGERŐSÍTÉS: a másolat véglegesen törlődik.', function(){
var f = btn.closest('form');
if (f) { if (f.requestSubmit) f.requestSubmit(); else f.submit(); }
});
});
}
function fabStart(stack, next){
fetch('/api/export/download/start', {method:'POST', headers:Object.assign({'Content-Type':'application/json'}, csrfHeaders()), body: JSON.stringify({stack_name: stack, password: fabPassword()})})
.then(function(r){ return r.json(); })
@@ -38,6 +38,13 @@
</div>
</form>
<!-- v0.147.0 (4b): bring-up progress. Hidden until the poll reports a non-idle phase, so a page
with nothing in flight looks exactly as it did before. This is what makes the first-enable
wait survivable on a golden that has not baked felhom-samba: the pull is named, not silent. -->
<div id="samba-progress" class="alert alert-info" style="display:none">
<span id="samba-progress-text"></span>
</div>
<div class="form-group">
<span>Állapot:</span>
{{if .SMBRunning}}
@@ -282,6 +289,54 @@ function shareBrowseLoad(p){
});
});
}
/* v0.147.0 (4b) — Megosztás bring-up poll. Same shape as the storage-init status poll
(storage_init.html): 1.5s tick, phase string mapped to Hungarian, terminal states stop the timer.
The card only ever appears while something is genuinely in flight. */
(function(){
var box = document.getElementById('samba-progress');
var txt = document.getElementById('samba-progress-text');
if (!box || !txt) return;
var PHASES = {
pulling: 'A megosztási szolgáltatás előkészítése… (képfájl letöltése — ez több percig tarthat)',
starting: 'A megosztási szolgáltatás indítása…',
needs_password: 'A megosztás be van kapcsolva, de még nincs megosztási jelszó — add meg alább.'
};
var timer = null;
function stop(){ if (timer) { clearInterval(timer); timer = null; } }
function show(cls, text){
box.className = 'alert ' + cls;
txt.textContent = text;
box.style.display = '';
}
function tick(){
fetch('/sharing/status', {credentials:'same-origin'})
.then(function(r){ return r.json(); })
.then(function(j){
if (!j || !j.ok || !j.data) return; // transient — keep polling
var ph = j.data.phase;
if (PHASES[ph]) { show('alert-info', PHASES[ph]); return; }
if (ph === 'running') {
stop();
show('alert-success', 'A megosztási szolgáltatás fut.');
/* Repaint the „Állapot" badge, which was rendered server-side as „áll". */
setTimeout(function(){ location.reload(); }, 1200);
return;
}
if (ph === 'failed') {
stop();
show('alert-error', 'A megosztási szolgáltatás nem indult el' + (j.data.error ? ': ' + j.data.error : '.'));
return;
}
/* idle and nothing running: nothing to report. */
stop();
box.style.display = 'none';
})
.catch(function(){ /* transient — keep polling */ });
}
tick();
timer = setInterval(tick, 1500);
})();
</script>
{{template "layout_end" .}}