felhom-controller/REPORT.md

# REPORT — R-82 Slice B: one quiesce window, two backup tiers (v0.175.0) (2026-07-26)

**Overwritten** per the standing rule. Controller **v0.174.0 → v0.175.0**.
Full cross-repo arc: `felhom.eu/REPORT.md`.

**MinAgent UNCHANGED — deliberately.** This release degrades gracefully against ANY older agent. It
does not require agent v0.97.0.

---

## 1. Why the controller had to change

The agent gained per-target backup tiers (R-82 Slice A): local **daily** + offsite **weekly**. But
the **controller** owns quiescing — it polls `/backup/due`, stops the app stacks, calls `POST
/backup`, and restarts them. So the multi-tier schedule has to be reconciled here, and one night a
week both tiers come due at once.

Two quiesce cycles that night would mean **two app outages for one night's work** — undoing the
entire argument for weekly-over-daily, since the quiesce blip was the only real cost of the offsite
tier.

## 2. The dedup rule — specified, not emergent

| local due | PBS due | result |
|---|---|---|
| yes | no | one quiesce, local backup |
| no | yes | one quiesce, PBS backup |
| **yes** | **yes** | **ONE quiesce window, BOTH backups inside it** |
| no | no | no quiesce |

## 3. What shipped

- **`quiesce.TieredBackend`** (optional extension to `Backend`) + `BackupTier`,
  `ErrTiersUnsupported`. A backend that does not implement it drives the pre-R-82 path unchanged.
- **`agentapi`**: `BackupTiers`, `BackupDueFor`, `StartBackupFor`, `BackupStatusFor`.
  `targetQuery("")` yields an **empty** suffix, so an untargeted call hits the untargeted route
  byte-for-byte.
- **`Loop.resolveDueTiers`** — the dedup rule in one place, tiers in agent order.
- **`quiesceAndPollTiers` + `pollTier`** — one marker, one stop, N sequential backups, one resume.

## 4. Two decisions worth stating plainly

**The app stays quiesced until the LAST tier snapshots.** Resuming after tier 1's snapshot would
leave the following tier capturing a **running** app — losing app-consistency on exactly the DR tier
we most want it on. The consequence is user-visible and documented: on the both-due night downtime is
*(first tier's full backup)* + *(last tier's snapshot)*, not one snapshot. Tiers therefore run
**fast-first** — vzdump holds a guest lock so they are necessarily sequential, and the agent
advertises primary (local) first. The reverse order would make downtime ≈ *(offsite backup)* +
*(local snapshot)*, far worse.

**A manual "Mentés most" covers EVERY tier**, in one window, due-ness ignored. A manual run that
silently skipped the DR tier would be the same applied-and-empty fault in a different costume.

## 5. Capability detection

`GET /backup/tiers` 404 ⇒ pre-R-82 agent. This is the project's documented **route-probe** mechanism
(`internal/agentapi/features.go`). It is deliberately **not** a `featureProbes` row: that table
answers a yes/no at a UI entry point, whereas the loop needs the tier **list**, so a row would be a
second probe of the same route for no gain. The degrade logs **exactly once per process** — once
because it is a steady state during a rollout, never zero times because a silent degrade is
indistinguishable from multi-tier working.

## 6. v0.175.0 — a tier that overruns the quiesce bound defers the rest

Operator ruling 2026-07-26: *"let the first backup run as long as needed; other backups shouldn't
start until finished."*

A first FULL offsite snapshot legitimately runs for **hours**. When `max_quiesce` elapses the app
resumes — correct, unchanged — but the loop then started the **next** tier while the first was still
uploading. That is now a `break`.

Why it matters: vzdump still holds the guest lock, so the second start would be **refused by the
agent (409, v0.99.0)** or fail on the lock — and a failed backup never satisfies a cadence, so the
tier would stay permanently due and retry into the same wall every poll.

## 7. Tests

`go build ./... && go vet ./... && go test ./...` — **rc=0, 27 packages** (vet run unpiped; note
`grep FAIL` returns rc=1 when it finds nothing, which is not a failure signal — that idiom caused a
red commit elsewhere in this arc).

+12 tests in `internal/quiesce/tiers_test.go`. Red-proofs observed and restored:

- **Both-due night** — a per-tier cycle instead of one window fails with
  `want EXACTLY 1 stop and 1 start, got stops=2 starts=2`. **The COUNT is the assertion**; asserting
  only "both backups ran" would pass against a double-quiesce implementation.
- **New controller ↔ old agent** — treating `ErrTiersUnsupported` as "nothing due" fails with
  `OLD AGENT: a backup MUST still be taken via the untargeted path; got started=[]`. The hollow
  version of this test asserts only "no error", which passes while silently skipping the backup.
- **Overrun defer** — dropping the `break` fails with
  `the second tier MUST NOT start while the first is still running; started=[local felhom-pbs]`.

## 8. NOT live-validated — and NOT deployed

The controller change is **unit-proven only**. Both production controllers still run **v0.173.0**;
this release was deliberately not deployed, because the multi-tier path only engages against an agent
that serves `/backup/tiers`, and the agent rollout was still mid-flight (demo-hp's first offsite
backup in progress). Specifically unproven live:

1. A real both-due night with exactly one stop/start pair observed on a box.
2. The degrade path against a genuinely old agent in production.
3. The overrun defer firing on a real long-running offsite backup.

Deploying is the natural next step once demo-hp's first offsite backup lands and the agent rollout
is settled.