R-1256: four restore edges (review round 2 HUNT-1 H4/H5/H6/H8) — each test seen failing first
gates / gates (push) Successful in 31s
gates / gates (push) Successful in 31s
H4: a failed definition write in the off-site restore writes the live definition back before the start;
a failed write-back holds the app (restore_mixed).
H5: a put-back whose stop fails pours nothing, keeps the copy and holds the app (off-site, local, power-cut);
a failed definition write-back keeps the copy (D20); a failed 'volume rm -f' fails that volume (no merge).
H6: TryBeginRestoreOp/RestoreBusy: check every operation and record the restore in one step under m.mu;
every web handler and the kept loads begin through it; install-time kept load asks about the mirror too.
H8: SweepUnusedRestoreCopies every 6 h: removes a copy only with no hold, no marker naming it, no restore
running, and older than 14 days.
Commit only; rides the next controller release.
This commit is contained in:
1 parent
ed3927f03a
commit
342a394bff
20 files changed
+860
-50
No files matched your search
@@ -62,6 +62,53 @@ type RestoreOpStatus struct {
|
||||
func (m *Manager) BeginRestoreOp(op, stack string) {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
m.beginRestoreOpLocked(op, stack)
|
||||
}
|
||||
|
||||
// Why a new restore may not begin now (R-1256 H6). "" = it may.
|
||||
const (
|
||||
RestoreBusyRestore = "restore" // a restore op is in flight (the record the pages follow)
|
||||
RestoreBusyBackup = "backup" // the single-flight backup/restore run is held (the night run never records an op)
|
||||
RestoreBusyMirror = "mirror" // a second-drive copy runs, or one a previous process lost may still run
|
||||
)
|
||||
|
||||
// TryBeginRestoreOp is BeginRestoreOp that refuses — in the SAME step, under m.mu — while another restore op, a
|
||||
// backup run or a second-drive copy runs. R-1256 H6: the pages asked (RestoreStatus, IsRunning,
|
||||
// Tier2CopyBlocksRestore) and recorded the op later; a mirror that began in between ran beside the restore. The mirror
|
||||
// asks under the same m.mu (beginTier2Run), so one of the two always sees the other. Lock order m.mu → tier2Copies,
|
||||
// as in beginTier2Run. Returns the busy reason ("" = begun) and, for a mirror, its app ("" when unknown).
|
||||
// Pinned by TestR1256_H6_*.
|
||||
func (m *Manager) TryBeginRestoreOp(op, stack string) (busy, mirrorApp string) {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
if busy, app := m.restoreBusyLocked(); busy != "" {
|
||||
return busy, app
|
||||
}
|
||||
m.beginRestoreOpLocked(op, stack)
|
||||
return "", ""
|
||||
}
|
||||
|
||||
// RestoreBusy is TryBeginRestoreOp's question alone — for a refusal asked early, before a request is read further.
|
||||
func (m *Manager) RestoreBusy() (busy, mirrorApp string) {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
return m.restoreBusyLocked()
|
||||
}
|
||||
|
||||
func (m *Manager) restoreBusyLocked() (string, string) {
|
||||
if m.opRunning {
|
||||
return RestoreBusyRestore, m.opStack
|
||||
}
|
||||
if m.running {
|
||||
return RestoreBusyBackup, ""
|
||||
}
|
||||
if app, blocked := m.Tier2CopyBlocksRestore(); blocked {
|
||||
return RestoreBusyMirror, app
|
||||
}
|
||||
return "", ""
|
||||
}
|
||||
|
||||
func (m *Manager) beginRestoreOpLocked(op, stack string) {
|
||||
m.opRunning = true
|
||||
m.opName = op
|
||||
m.opStack = stack
|
||||
|
||||
Reference in new issue
Block a user