v0.189.0 — desired state + the app-stop crash marker (R-166 / D-b)
gates / gates (push) Successful in 8s
gates / gates (push) Successful in 8s
The box stops inferring the customer's intent from a container count and reads
what they actually asked for.
Part 1 — desired state. AppConfig gains a tri-state `desired_state`
(""/running/stopped), written ONLY by the customer's own action: the API action
switch, DeployStack, UpdateOptionalConfig's redeploy branch, and the .fab
import. Intent is written BEFORE the act and a failed write REFUSES the act.
StartStack/StopStack are deliberately not writers — 14 callers, only 2 are the
customer. bootrecon.isBootOrphan now reads intent instead of len(Containers)>0,
which closes R-157 mechanism B (a power cut or interrupted deploy left an app
with zero containers, read as a deliberate stop, and stranded silently).
ABSENT MEANS UNKNOWN, NEVER "running": every pre-v0.189.0 app.yaml reads absent,
so the legacy fallback is byte-identical to the old rule. A running-only startup
backfill converges the unambiguous cases; `stopped` is never inferred.
Part 2 — backup.AppStopGuard, a persisted marker over every stop→work→start
window (volume dump, offbox reconstitute, .fab export). Its own file, never
quiesce's. Written before the stop, cleared only after a restart that succeeded,
kept when one fails. Recover() completes before the boot reconciler is launched
and returns its outcome, which main.go reports on the existing backup_failed
event once the notifier exists. A defer is not the mechanism — a SIGKILL runs
none (Campaign 8 fault 10).
Also: SaveAppConfig rebuilt AppConfig field-by-field (the R-100 shape) and would
have dropped desired_state on every save across nine call sites. Replaced with
copy-and-overlay. Measured: app.yaml does not round-trip unknown YAML keys.
No hub change, no agent coupling, no user-visible string. 27/27 packages green;
7 red-proofs observed FAIL then restored.
This commit is contained in:
@@ -1,5 +1,104 @@
|
||||
## Changelog
|
||||
|
||||
### v0.189.0 — the box stops guessing what the customer wanted (2026-08-02, R-166 / decision D-b)
|
||||
|
||||
**The defect.** When an app was not running, the controller had to work out *why*, and it worked it
|
||||
out by **counting containers**: zero containers meant "the customer stopped it" (leave alone), some
|
||||
containers meant "something broke" (recover). That inference is wrong in two ways, and both were
|
||||
silent:
|
||||
|
||||
- a **power cut mid-compose** or an **interrupted deploy** also leaves an app with zero containers —
|
||||
read as a deliberate stop, so the app simply stayed gone until a human noticed (**R-157 mechanism
|
||||
B**);
|
||||
- a **backup that stops an app** to copy it safely, then dies, leaves it stopped with **nothing on
|
||||
disk** recording that a backup stopped it or that it was owed a restart.
|
||||
|
||||
Neither is a guess the controller should be making, because the one fact that settles it — what the
|
||||
customer actually asked for — **was written down nowhere**. `app.yaml` recorded that an app was
|
||||
*installed*; it never recorded whether it was meant to be *running*.
|
||||
|
||||
**Part 1 — desired state, owned by the customer's action.** `AppConfig` gains `desired_state`, a
|
||||
**tri-state** `""` / `running` / `stopped` (`yaml:"desired_state,omitempty"`), with named constants.
|
||||
`Manager.SetDesiredState` is the only writer, and its callers are the only places a human's decision
|
||||
enters the system: the `/api/stacks/{name}/{action}` switch (`start`/`restart`/`update` → running,
|
||||
`stop` → stopped), `DeployStack`, `UpdateOptionalConfig`'s redeploy branch, and the `.fab` import
|
||||
adapter. Intent is written **BEFORE** the act, and an action whose intent cannot be recorded is
|
||||
**REFUSED** — proceeding would recreate the ambiguity being removed.
|
||||
|
||||
**`StartStack`/`StopStack` are deliberately NOT writers.** A census on 2026-08-02 found **14 call
|
||||
sites, of which exactly 2 are the customer**; the other twelve are machines (quiesce, the backup
|
||||
volume dump, offbox reconstitution, app export/restore, the storage drive-absent gate, the migration
|
||||
engine, the boot reconciler itself). Recording intent in the primitive would make a nightly backup
|
||||
indistinguishable from the customer pressing Stop — the exact confusion this release ends.
|
||||
|
||||
**ABSENT MEANS UNKNOWN, NEVER "running" — the single most important line in the change.** Every
|
||||
`app.yaml` on every existing box predates the field, so absent is what the whole fleet reads on
|
||||
upgrade. Treating it as running would start, on the first boot after the upgrade, every app its owner
|
||||
had deliberately stopped. Where intent is unknown the boot reconciler falls back to the **old
|
||||
container-count rule, byte-for-byte**, rather than inventing an answer.
|
||||
|
||||
**The boot-orphan decision (`bootrecon.isBootOrphan`), replacing the container-count term:**
|
||||
|
||||
| `desired_state` | containers | result |
|
||||
|---|---|---|
|
||||
| `stopped` | any | never an orphan |
|
||||
| `running` | 0 | **ORPHAN** — the R-157 case, invisible before this release |
|
||||
| `running` | >0 + down | ORPHAN (unchanged) |
|
||||
| `running` | >0 + up | not an orphan |
|
||||
| absent | 0 | not an orphan — **exactly** the pre-v0.189.0 behaviour |
|
||||
| absent | >0 + down | ORPHAN — **exactly** the pre-v0.189.0 behaviour |
|
||||
|
||||
`Protected` and `Deploying` guards unchanged. A **running-only** startup backfill converges apps that
|
||||
are deployed AND observed up; `stopped` is **never** backfilled, from any signal — inferring it from
|
||||
zero containers is the defect itself, so an ambiguous app stays ambiguous and keeps legacy behaviour
|
||||
until the customer next presses a button.
|
||||
|
||||
**Part 2 — the app-stop crash marker (`backup.AppStopGuard`).** `<data_dir>/appstop-state.json`,
|
||||
atomic (tmp + **fsync** + rename, 0600), modelled on the quiesce marker and deliberately **its own
|
||||
file** — same shape, different owner, different lifetime; sharing would give one file two writers.
|
||||
Written **before** the stop, cleared only after a restart that **succeeded**; a FAILED restart keeps
|
||||
it so the next startup retries. `Recover()` runs at startup and **completes before** the
|
||||
boot-reconcile goroutine is launched, so an app the marker explains is not also reported as an
|
||||
unexplained boot orphan. A corrupt marker is quarantined loudly, never silently skipped.
|
||||
|
||||
**A `defer` is not the mechanism, and the code says so.** Campaign 8 fault 10 established on live
|
||||
hardware that a SIGKILL runs no deferred function; the marker is what covers the hard crash. Its
|
||||
test simulates a real abort (an unwind that skips the restart statement) rather than a graceful
|
||||
return — an earlier version of that test called `Begin` itself and **survived the red-proof that
|
||||
deleted the production call**, which is exactly the hollowness §10 exists to catch.
|
||||
|
||||
**All three stop-and-restart sites are covered**, with no uncovered sibling to imply the class is
|
||||
handled: `DumpAppVolumesSafe`, `offbox_reconstitute.go` (all four bring-up paths, via one
|
||||
`restartStack` closure so the success path cannot silently skip the clear), and `appexport`'s export
|
||||
— the last through a two-method consumer-side seam so the exporter shares the ONE marker file instead
|
||||
of opening a second. The reason string lives only in `backup`; the adapter in `main.go` supplies it.
|
||||
|
||||
**Also fixed, and it would have silently eaten this feature: `SaveAppConfig` rebuilt `AppConfig`
|
||||
field-by-field.** That is the R-100 shape (v0.181.0 shipped with two live instances of it). The
|
||||
literal named five fields, so the sixth — `desired_state` — would have been **dropped on every save**,
|
||||
and nine call sites share that path: a customer's Stop would have been erased by the next unrelated
|
||||
`app.yaml` write. Replaced with copy-and-overlay (`saveCfg := *cfg`), safe by construction. Measured
|
||||
and documented: `app.yaml` does **not** round-trip keys the struct does not model (the trip goes
|
||||
through the struct), pinned by `TestSaveAppConfig_UnknownYAMLKeysAreDropped`.
|
||||
|
||||
**Operator visibility (§2.4).** An interrupted operation rides the **existing** `backup_failed` event
|
||||
type. A new type would need the hub's `allowedEventTypes` + `customerMessages` pair changed — a wire
|
||||
change, and this release ships **no hub change and no hub version bump**. `Recover()` **returns** its
|
||||
outcome rather than pushing it through a notifier seam, because it must complete before the boot
|
||||
reconciler (`main.go:~236`) while the notifier is not constructed until `~307`; a seam wired after the
|
||||
fact is a seam that never fires.
|
||||
|
||||
**No user-visible string changed** — N/A for UI work. No template, funcmap, notifier-type, event-type,
|
||||
backup-content, retention, tier or restore change. **No agent coupling; MinAgent unchanged.**
|
||||
|
||||
**Tests: +37 across 5 packages (27/27 packages green).** Red-proofs, each observed FAIL then restored:
|
||||
B (restore `len(Containers) > 0`), C (treat absent as running — the fleet-wide upgrade regression),
|
||||
D (drop the up-state guard from the backfill), E (delete the production `Begin` call), H (restore the
|
||||
field-by-field `SaveAppConfig` literal), §8.2 (move the intent write below the action switch), and
|
||||
the seam test I — which fails while the commented-out call **is still present as a substring**, the
|
||||
distinction that made the controller's first version of that test pass its own red-proof in
|
||||
2026-07-21.
|
||||
|
||||
### CI — the gate entry point runs on every push (2026-08-02, R-168) — NO VERSION BUMP
|
||||
|
||||
**No version bump, no build, no deploy** — this adds a workflow file only. Stated explicitly so the
|
||||
|
||||
Reference in New Issue
Block a user