R-225 — an unread store said '0 pillanatkép / 0 / 50 GB' above a card stating
it held backups under another key. An SFTP listing found snapshot f3d9cd67 and
12 535 KB really there; snapshot_count and repo_size_bytes were simply ABSENT
and the zero value spoke for them. StatsKnown is now NAMED, for the same reason
OffsiteInventory.Empty is: zero is what an unread store and an empty one both
look like, and on the wire 'absent' and '0' are the same bytes. The fill bar
renders only when the fill is known — a 0%-wide bar is a picture of emptiness,
and a picture is a claim. A measured zero still says zero.
R-227 — WHICH LAYER ANSWERS: traefik, and this repo generates its config. But
traefik v3 serves no static files, so a branded proxy page needs a new always-up
container for every 502 on the box — out of proportion, and scoped in the report
rather than built. Shipped instead: the unlock posts via fetch and answers a
gateway failure in Hungarian without leaving the page. Progressive enhancement —
with no JS the plain POST is unchanged and still shows the proxy's error, which
the report says plainly rather than implying otherwise.
R-228 — the set-aside history was recorded in orphaned_renamed_to and read by
nobody: a census found zero references in any template or handler, while 12 535
KB sat at that path. It is surfaced as two facts and stops. It does NOT promise
the history can be reopened, because it cannot be by anyone today (R-199's
inventory is unbuilt) — and the set-aside CONFIRMATION copy was corrected for
the same reason: 'a helyreállítási kód nélkül többé nem lesznek megnyithatók'
implied that WITH the code they could be. The field's own comment called it
'recovery-code-recoverable', which was the same over-promise in the code.
Tests: scenarios F, G, H as render tests per branch of each gate. Red-proofs,
each demonstrated failing then restored: remove the StatsKnown guards (F,
'R-225 RETURNED: an unread store reports a snapshot COUNT of zero'), delete the
set-aside block (H). The F assertion on the fill bar is scoped to the bar's own
container — a bare width:0% search matched unrelated elements and would have
passed for the wrong reason.
28 packages ok, vet clean, all controller gates OK (the emoji gate caught a
warning sign in a template comment).
The failure branch was a two-way choice — superseded? M4 : M1 — and BOTH are
statements about the customer's code. rerr was never inspected, so a hub that
refused, an agent that was stopped and a genuinely mistyped code all produced
the same accusation. Measured live 2026-08-05 with a CORRECT current code: hub
firewalled off 0.0556s, agent stopped 0.0299s, against ~1.0s for a real unseal.
Five classes, from the VALUE and never the text:
hub-unreachable 502/503 from the agent — the code was NOT used
agent-unreachable no agent verdict at all (transport) — NOT used
no-bundle 404
bundle-too-old 409
asked-and-refused 400 — the ONLY class that may mention typing
unknown everything else -> NEUTRAL, the safe default
agentapi.RecoveryRefusal carries the status as a value (refusalError flattened
it into a sentence, and a sentence is not something a caller can branch on).
THE OLD-AGENT CASE IS WHY THIS NEEDS A COUPLING. Agent < 0.126.0 answers 400
for both a fetch failure and a wrong code, so a 400 from one cannot be read as
a refusal. FeatureRecoveryFailureClass (MinAgent 0.126.0) withholds that
reading and the 400 degrades to neutral. The gate BLOCKS NOTHING — it only
decides whether the customer may be told to check their typing.
R-226: the superseded message now names BOTH possibilities and restores the
ten-words prompt. The two are indistinguishable at the engine; the honest
message says so. It still does not promise the earlier package can be opened.
Elapsed time is logged (it is what diagnosed this) and is NEVER a classifier.
Tests: scenarios A-E at the HANDLER + the classifier table asserting the same
sentence under two statuses classifies two ways. Red-proofs, each demonstrated
failing then restored: delete the 502 case (A), remove the mistype clause (C),
default to the accusation (D), route an instant transport failure to the typing
message (E).
Two existing tests encoded the defect and were corrected, not deleted: the web
fake returned a BARE error for 'wrong code' (which is the shape of a failure we
cannot classify), and R-222's test forbade any mention of typing on a
superseded box — half of which R-226 deliberately reverses.
28 packages ok, vet clean, all controller gates OK.
R-216: the offsite key recovery is a coupled feature and now says so. featureProbes +
featureMinAgent 0.125.0 + a Supports gate at the unlock entry point, FAILING CLOSED — an
agent that cannot answer is named as such instead of the customer's code being blamed.
Measured live: a 404 from agent 0.120.0 came back as "we did not accept your recovery
code, check that all ten words", in 0.134 s, against a perfect code.
R-218: delete the repo-password short-circuit in needsOffsiteCredential. The declaration
stops when the TIER WORKS, not when a key exists — installing a key is the recovery
screen's whole job, so succeeding at recovery was switching off the mechanism that would
have delivered the coordinates to use it.
R-219: the unlock finishes the job — place the key, bring the tier up, then list. Without
it the promised listing could never render on the shape the screen exists for.
R-217: an unreadable store no longer claims to have opened with unattributable content
(the OffsiteInventory{} zero value). Opened / empty / unreadable are three states.
R-222: a code that is right about a RETAINED earlier package is named, not blamed. States
what the hub knows and promises nothing — no read path exists.
R-215: GET /recovery is gated on the same predicate as the interception.
Five red-proofs, each demonstrated failing and restored.
A customer whose machine was rebuilt had everything needed to get their data
back and no way to find out: the only route was a command line. This is the
screen that closes that.
IT UNLOCKS, AND ONLY UNLOCKS (operator ruling). It explains, takes the recovery
code, opens the repository and shows what is in there — apps, dates, sizes. It
restores nothing: restore is already per-app and lives in the backups area, and
a screen that unlocks and then offers to overwrite is two decisions wearing one
button.
ONE CORE, TWO CALLERS. RecoverInstallCore is split out of RecoverAndInstall; the
CLI wrapper keeps its exit codes and printed lines byte-identical, and the
handler drives the same function. Two implementations of the one operation that
can permanently lose a customer's data would drift, and only one would be
tested. Asserted from source on both sides by AST.
THREE WAYS OUT, none a dismiss button: recover; 'most nem' (the full page stops
interrupting, the backups-area entry point stays PERMANENTLY, bound to the offer
and never to the postpone flag); and 'I do not want the old data' — confirmed
TWICE and reaching the SHIPPED move-aside, which sets aside and never deletes.
THE CODE IS HANDLED NO MORE LOOSELY THAN ON THE COMMAND LINE: POST body only,
never logged, never persisted, never echoed, cleared on every path, no-store,
autocomplete off. No lockout — the code is a ten-word phrase, and locking a
customer out of their own data for a typo is worse than anything it prevents.
TWO DEFECTS THE TESTS CAUGHT, both fixed: an UNCLAIMED (legacy-open) box would
have been shown the page, because RequireAuth passes such a box through; and the
inventory nil-dereferenced when no off-site target was configured, which is
exactly the pristine rebuilt shape.
An absent off-site object has four meanings — never configured, mid-restart, a
transient config read failure, and rebuilt-and-stranded — and the hub cannot tell
them apart. The box can, from two local facts it holds with certainty, so it says
so instead of leaving the hub to deduce it from a silence (operator ruling).
The ACK's identity_blob_present is now recorded on EVERY ACK, before the gates
that used to discard it: on a box with no off-site target the auto-confirm returns
immediately, which is exactly a rebuilt box, so the one fact distinguishing it from
a box that never had off-site backups was thrown away every cycle.
The declaration needs BOTH halves — a fresh data area AND a hub-held recovery
package. Freshness alone is a box that never had off-site backups; dropping that
condition makes the whole fleet ask for credentials, which is what the Scenario B
test exists to catch.
The object carries enabled:false and zero sizes, which is what makes it inert to
the hub's existing fill and staleness checkers and to a pre-upgrade hub. A
configured box's JSON is byte-identical to v0.198.0's.
mode=unit restores the recovery unit — the app's definition, configuration
and database dumps — and NOT the customer's own files: RestoreOffboxScratch
passes --include <unit path> and the userdata in the same snapshot is excluded
by it. The outcome was one sentence for both modes and named neither scope,
so on the last step of a disaster recovery the customer was told the app had
been restored after the thing they were looking for had not been.
restoreScratchOutcomeMsg states what came back, what did not, and the next
step that gets it. The wizard's intent card states its scope before the choice.
The full-restore size gate is untouched and pinned as unchanged; the default
stays unit, since all three wizard forms set mode explicitly.
--print-reset-code runs as a separate process and persists the new code;
the running server's cache was never told, so the code the customer was told
to type was refused until the controller restarted. Nothing said so — during
the 2026-08-04 drill that cost two attempts with an operator present.
effectiveClaimCode now reads through to the persisted state before applying
the settings-vs-config precedence, which is itself unchanged. Read-through,
not a TTL: a TTL would leave a window in which a superseded code still works,
which is worse than the bug. Fails closed on an unreadable state; an absent
file is not an error.
The gap was already detected and warned about, in Hungarian, naming the app and the folders --
that warning is what stopped the R-201 drill. The defect was that the run still reported `ok`
beside it, and a warning standing beside a success is read as a success.
last_status gains "incomplete": minted, because "ok" | "error" | "running" had nothing meaning
"it ran, and this app is not fully protected". NOT "error" -- the rest of the run worked and
what was captured is real, so SnapshotCount and the LastSuccess anchor still record it. Half a
backup is not no backup.
The gaps are now recorded STRUCTURALLY (offboxRunResult.mandatoryGaps), not only as prose, so
the verdict has something to act on. It reaches the operator through the EXISTING per-run digest
(backup_run_failures) rather than a new event type -- a new type is a two-repo change and the
hub drops anything outside allowedEventTypes.
The stat-filter gains the ClassMandatory check Tier 2 already had. It is a NO-OP today
(TierOffsite admits mandatory only), so no customer-visible warning disappears -- demonstrated
by widening the tier filter alone and watching the check hold the line.
ANTICIPATED: calibre-web on demo-hp has exactly this gap, so its off-site status becomes
incomplete the moment this ships. That is correct and is the point.
Red-proofs: my first Scenario-C proof PASSED because the test only reached offboxCaptureSet
while the mutation lives in runOffboxInternal -- a mutation the test cannot observe is not a
red-proof, and the fix was the test. The run-level test now fails under both mutations
(unreachable gap recording; unconditional ok).
ExportDataMounts lives in delete.go, which reads as a destructive path. IT IS NOT: its single
production caller is the .fab export adapter, and nothing deletes based on its result. The
delete path's own guard, ProtectedHDDPaths, is layout-agnostic by construction -- it protects
BOTH <hdd>/... and <hdd>/felhom-data/... -- so deletion was never affected by the
namespace-root defect. That scope note is now in the function's doc comment, because the file
placement will mislead the next reader exactly as it misled the spec for this change.
Separated into its own commit anyway, so a change to a function whose filename says "delete"
is reviewable on its own.
An empty nsRoot falls back to hddPath -- the pre-R-203 shape -- so any caller not yet updated
keeps working on enrolled drives.
Tests cover both drive kinds and assert the NEGATIVE: no emitted path lies outside the app's
own data roots. Red-proof: leaving the site bare fails the system-drive row, emitting
/mnt/sys_drive/userdata where the canonical root is /mnt/sys_drive/felhom-data/userdata.
appbackup's path helpers take a NAMESPACE ROOT. Five call sites passed a bare DRIVE path.
On an enrolled drive the two coincide, so nothing showed; on the system-data fallback they
differ by exactly the felhom-data segment, and the app then bound a directory the off-site
capture set never looked at -- while the run reported ok. Measured live on demo-hp: the app
wrote to /mnt/sys_drive/userdata/media/books, the capture set looked for
/mnt/sys_drive/felhom-data/userdata/media/books.
THE RULE NOW HAS ONE EXPRESSION. appbackup.NamespaceRootFor / IsEnrolledDrive encode the
drive-kind comparison; backup.Manager.namespaceRoot and stacks.Manager.inGuest delegate to
it. There were already TWO copies and they differed -- the backup package's compared without
filepath.Clean, the stacks package's with it, so a trailing slash from config would have
flipped the mode in one and not the other.
Sites routed through it:
- stacks/deploy.go withPathVars -> ${USERDATA_PATH} (the live defect)
- appexport/fabplan.go + export.go (via a new provider method)
- web/handlers.go FileBrowser mounts (latent: the system drive is
deliberately never a registered StoragePath, so this is the identity today)
ComputeFabBuckets now receives the namespace root, which is what ComputeCaptureSet has always
received -- so the export's classified paths and the backup's capture set describe the same
directories by construction instead of by coincidence.
Tests are table-driven over BOTH drive kinds, because this survived by being invisible on the
kind that already worked. Red-proofs observed: restoring the bare-path call fails the
system-drive row with the two paths differing by /felhom-data; inverting the drive-kind
comparison fails every enrolled row.
--recover-offsite-install is the sibling of --recover-offsite-check: same fetch/unseal path
through the agent, same STDIN discipline for R, but it PLACES the recovered repository
password via InjectOffboxPassword so a rebuilt box reopens the history it inherited.
Doing this by hand would put the offsite DATA key through a terminal, a clipboard and shell
history. In-process the value goes agent -> this process -> the 0600 file and is rendered
nowhere.
The confirmation is a SECOND invocation: without --confirm-install it prints both hashes and
writes nothing, so the operator sees the comparison before any write is possible.
Three outcomes, named distinctly: installed (no local password -- the rebuilt-box shape),
unchanged (identical key already present, nothing written), refused (a DIFFERENT key present;
installing would clobber the key the current repository is encrypted under, and no force
option is offered). Exit 2 for the refusal, distinct from 1 for a failed step.
Red-proof: removing the confirmation gate makes the dry run write, failing the test. The
R-persistence test carries a positive control -- a planted copy is found, then removed and not
found -- because an absence check is worth only what its sensitivity is.
--recover-offsite-check is a docker exec diagnostic in the shape of --print-reset-code: it
reads the customer's recovery code from STDIN, asks the agent to fetch this host's sealed
bundle and open it, and reports whether the recovered key matches the one on disk BY SHA256.
Two hashes and a verdict; never a password, never R, never a blob.
R comes from stdin and not a flag because a flag value is visible in ps, in shell history, in
a container's command line and in any transcript of the session that ran it.
IT COMPARES; IT DOES NOT INSTALL. The recovered password is never written to
offbox/repo_password -- installing changes a live box on a path nobody has walked, and that
link is next session's, with the drill around it. A test asserts the data dir is byte-unchanged
after a check; its red-proof (adding the install call) fails it.
Exit codes: 0 match, 2 clean MISMATCH, 1 a step failed -- "it failed" and "it worked and
disagreed" must never share a status. A box with no local password reports distinctly: that is
the rebuilt-box shape, where the next step is to install rather than compare.
Nothing customer-reachable ships here: no card, no form, no preview.
MEASURED, not supposed. On 2026-08-03 nine per-app recovery_unit_capture_failed
events reached the hub and TWO operator emails went out. The hub's operator
cooldown key is customerID:eventType(+tier) and that event carries `app` but no
`tier`, so the key held no app identifier: the first refused app took the hour's
slot and every other app's failure was discarded BEFORE anything was written
down, leaving no row on any channel.
The obvious fix — put `app` in the key — was ruled against: on a full disk it
produces one email per app, the volume problem wearing the correctness problem's
clothes.
internal/backup/runsummary.go: a per-run collector with exactly admissionSet's
lifetime, fed by all three write legs, emitting backup_run_failures ONCE at the
end and only when something failed. A clean run emits nothing.
The per-app event stays and becomes the RECORD — the hub routes it record-only,
stored and logged every time, never competing for an email slot. The record and
the notification are now different things.
Deliberate skips (disconnected, decommissioned) are excluded: they have their
own alert, and a nightly email about an unplugged drive is one the operator
learns to ignore.
A manual run always reports: the digest carries a unique run_id the cooldown
cannot collapse. Someone pressing the button is actively trying to get a backup.
THE PERIODIC SWEEP GETS A DIGEST TOO. With the per-app event now record-only, a
capture failure found between runs would be recorded and never notified — a new
silence introduced while closing one. That path emits a digest with NO run_id,
so the ordinary 1-hour cooldown caps it exactly as before while the mail now
lists every failing app instead of whichever was first.
A refusal is recorded ONCE, where the verdict is taken, not at the three legs
that consult it — R-181's contract is one verdict per app per run. Noting it per
leg listed one refused app three times and produced "2 of 1 apps failed". Found
by the digest's own test, not in review.
Silence is safe because the hub's deadline check raises expected_backup_missed
from report freshness, independently of any mail this box sends
(monitor/deadline.go:396,417). Confirmed, not assumed.
7 new tests, 4 red-proofs. The main.go seam walk did NOT fail on its first
attempt — the AST test walked the backup package and not main.go; the test was
fixed and the mutation re-run rather than the pass recorded.
Found by v0.193.0's own live proof run. The estimate was printed fixed to two
decimal GiB, so every app under ~10 MB rendered as "estimated 0.00 GiB write" —
which reads as "no estimate was available" and is the opposite of what happened.
Observed live on demo-hp 08:59:46: opengist's real 178 KB estimate printed as
0.00 GiB.
Shipped in the same session because it is the same defect class R-181 is about:
a message an operator cannot rely on is worse than no message.
The arithmetic is unchanged and still in GiB — the reserve's own unit, so the
comparison against FloorFreeGiB reads directly. Only the rendering moved to
humanizeBytes. estimatedWriteGiB -> estimatedWriteBytes, with the GiB conversion
done once at the point of comparison.
B2's capture floor (v0.192.0) was consulted in exactly ONE place —
captureAllRecoveryUnits, which writes a few KB. The two legs that write the BULK
into the same backups/primary/<app> tree, the DB dump and the volume dump, ran
FIRST and unguarded. Measured live on demo-hp 2026-08-03 06:40:03: opengist's
volume dump wrote 2.0 GB with no check, free fell to 1.0 GB, and the floor then
refused the cheap write it had already lost the argument to. Its refusal message
claimed "the previous unit is untouched" — measured false: that app's tar had
gone 182,272 B -> 2,147,666,432 B under a stale manifest. Sixth entry in
CLAUDE.md's table of shipped guarantees the code did not provide.
Fix: ONE admission verdict per app per run (internal/backup/admission.go), taken
before that app's FIRST write and covering all three legs — they write under one
per-app root, which is why one verdict can honestly cover them.
- Lazy, at the app's first write, NOT once at run start: app A's dump can put app
B under the reserve, so a run-start verdict reads a disk that no longer exists.
- Remembered for the run, never re-decided between an app's own legs — that is
the split this closes. Reset per run.
- Placed ahead of DumpAppVolumesSafe, which stops the stack as its first act, so
a refused app is never bounced. After the volume-less check, which has no write.
- Exactly one operator alert per refused app per run.
- Leg order unchanged: volume dumps still precede the capture.
The floor is now SIZE-AWARE: it asks whether THIS app's write would cross the
reserve, not only whether the filesystem is already below it — which is how an
app was admitted at 96% and then allowed to write 2 GB. Estimate = the app's
previous .sql + .tar on disk. No history -> headroom-only, deliberately, and the
alert says so.
A container-based du per volume was MEASURED and rejected: 66 timed runs on
demo-hp guest 9201, median ~355 ms/volume (341-404) on volumes holding tens of
KB — container start-up, not the walk. Decisive on top: docker run needs the
writable layer, so it can fail under exactly the pressure the reserve handles.
The message was NOT weakened; the behaviour was moved so the wording became true.
It now also names which term bound. Every claim is checked against a sha256
fingerprint of the tree it describes, never against the log line.
Still refuses and never deletes: nothing here is generational.
11 new tests through the production functions. The DB leg cannot run without
Docker, so its gate is pinned by an AST walk of backup.go asserting admitApp
precedes DumpOne (strings.Contains is insufficient — a commented-out call still
contains the string). 4 red-proofs demonstrated failing then restored.
Ships BEFORE the disk-layout merge it exists for, and is harmless on a box
that never gets it. The mp1 partition was a BULKHEAD as well as a ceiling:
it kept a runaway capture from filling the space the container runtime
needs, because /var/lib/docker was a different filesystem. After the merge
it is the same one, and a full Docker data-root is a stopped box.
The floor sits in captureAllRecoveryUnits, checked BEFORE anything is
written: below the reserve, that ONE app's capture is refused, its previous
unit is left byte-identical, the R-158 alert fires with the space figures,
and the loop continues.
Two terms whichever binds first (97% used / 1 GiB free) in fillwatch's
shape, deliberately BEYOND its critical band (95% / 2 GiB) so the customer
is always warned before a refusal can happen — a floor that fires before
its own warning is a silent failure wearing a threshold.
Headroom, never unit size: a per-unit cap would be R-163 rebuilt inside one
volume. Refuses, never deletes: nothing here is generational, so pruning
could only destroy a different app's only local copy; pruneStalePrimaryDirs
is an orphan sweep, not retention, and must not be repurposed.
Tests 1184 -> 1191. One fixture strengthened mid-red-proof: the "old 20 G
ceiling is gone" test sat at exactly 20 GB and survived a literal
UsedGB > 20 cap — hollow. Now 120 GB, and the mutation fails it.
Earned during v0.191.1's own live validation. After the customer had been
warned, a restart produced ZERO fillwatch lines — equally consistent with
'ran and chose silence' and 'never ran'. Proving the checker was alive
needed a deliberate crossing into the critical band.
For an edge-triggered check the quiet run IS the healthy steady state, so
that ambiguity is permanent rather than rare. Check now logs a per-RUN
summary on every run, counting unreadable separately from healthy so a
drive that has quietly gone unreadable cannot read as 'all fine'.
Found live on 9201: neither sched.Daily nor sched.Every fires on
registration, so a box booting with a filesystem already over the line
would stay silent for up to 24h — the R-100 shape, and the same gap the
hub's own checkers avoid by leaving already-breached keys unseeded at init.
The watcher now runs once 90s after startup as well. Safe because the check
is edge-triggered against persisted state: an already-warned filesystem
stays silent. The delay lets mounts settle so a drive still returning reads
as unreadable and is skipped rather than warned about. Pinned by an AST
assertion — the schedule registration alone no longer satisfies the test.
R-167: new internal/fillwatch warns the CUSTOMER before a filesystem fills.
It emits the PRE-EXISTING disk_warning/disk_critical pair, which was
allowlisted, copy'd, default-enabled and checkbox'd with no producer in any
repo — the sixth "built but never wired" instance here. Two threshold terms
(85% or 5 GiB free; critical 95%/2 GiB) because a percentage alone lies at
both ends of this fleet's size range. Edge-triggered on escalation only,
state persisted, hysteresis dead zone at 75%/7 GiB pinned by a test. A nil
usage read is never a warning and never clears one. Per filesystem, never
per app. Daily 03:30, before the nightly app-data legs.
R-158: new unitNotify seam fires per app when a Tier-1 recovery-unit capture
fails, loop continuing, carrying the target filesystem's used/free bytes.
Operator-tier (recovery_unit_capture_failed) — deliberately NOT backup_failed,
which is customer-enabled and would email the customer about a failure they
cannot act on. D-c overrides R-158's own proposal here.
R-174: the app-stop guard no longer starts apps onto MISSING drives — a
regression in v0.189.0 code, found by review and closed the same session.
SetStarter got the raw stack manager, whose StartStack has no drive gate,
and Recover runs at startup. R-171 one path over. bootDriveGate could not be
reused whole (its holder #2 is the guard's own marker, and holders #1/#2 read
vars assigned after Recover runs), so holder #3 is extracted into a shared
driveStartGate with a test pinning the delegation. ErrStartRefused splits a
refusal from a failure: both keep the marker, only Failed alarms, because
routing a deliberate hold into NotifyBackupFailed is the same false alarm.
Tests 1157 -> 1184. All red-proofs demonstrated failing and restored.
Found by live validation on 9201, not by review. GetStacks() is the Manager's
in-memory map refreshed by the scheduler every 10s; sampling it every 5s without
refreshing means two identical samples can mean the cache did not update rather
than that the fleet settled. A container removed ~5s before the window closed was
still in the sampled fleet and the sweep logged 'no boot-orphaned apps' for an app
that had none. sampleBootFleet now refreshes first; a refresh error degrades
rather than aborting the window.
R-171 (a regression v0.189.0 introduced, CONFIRMED on hardware before any fix
was written). Replacing isBootOrphan's container-count term with recorded intent
made a drive-gate-stopped app read as a boot orphan: the gate stops apps with
`compose down` (zero containers) and never touches desired_state, because it is
not the customer. Observed on 9201 with the drive held unmounted — the sweep
found and started it, burned both attempts, and handed it to the dead-app alarm.
The write hazard did not materialise (the unbound mountpoint is host-root-owned
and the guest is unprivileged) but that protection is accidental and untested.
New consumer-side seam bootrecon.StartGate, fail-safe (cannot determine ⇒ do not
start), wired in main.go. The rule is not new: the API's startGatedByMissingDrive
already refuses this; the sweep bypassed it.
R-157 mechanism A. The sweep looked once at T+5s, deriving candidates from a
fleet docker was still restoring — three of six hard resets. Now a settle-then-
sweep window: sample every 5s, settled after 3 identical samples, sweep ONCE at
the end; ends on settled or a 50s budget, and the log says which. The budget is
50s because settle+budget+one retry must stay under the 90s dead-app grace — a
test rejected 60s at 95s. A window that overruns emits a LATE RECOVERY warn
rather than the grace being widened to hide it.
Widening the window made two more holders reachable, so the one gate covers all
three: an absent drive, a quiesce, and an in-flight app-data operation — reusing
quiesce.SuppressedStacks() and a new read-only AppStopGuard.HeldStacks().
R-170. shouldRecreateOnBoot now reads desired_state with the identical three-way
table; absent keeps the old hasContainers behaviour exactly. Its comment argued
for the container count and was rewritten. presentStable is untouched. The two
gates' agreement is pinned from both sides against one fixture table.
27/27 packages green; 6 red-proofs observed FAIL then restored.
The box stops inferring the customer's intent from a container count and reads
what they actually asked for.
Part 1 — desired state. AppConfig gains a tri-state `desired_state`
(""/running/stopped), written ONLY by the customer's own action: the API action
switch, DeployStack, UpdateOptionalConfig's redeploy branch, and the .fab
import. Intent is written BEFORE the act and a failed write REFUSES the act.
StartStack/StopStack are deliberately not writers — 14 callers, only 2 are the
customer. bootrecon.isBootOrphan now reads intent instead of len(Containers)>0,
which closes R-157 mechanism B (a power cut or interrupted deploy left an app
with zero containers, read as a deliberate stop, and stranded silently).
ABSENT MEANS UNKNOWN, NEVER "running": every pre-v0.189.0 app.yaml reads absent,
so the legacy fallback is byte-identical to the old rule. A running-only startup
backfill converges the unambiguous cases; `stopped` is never inferred.
Part 2 — backup.AppStopGuard, a persisted marker over every stop→work→start
window (volume dump, offbox reconstitute, .fab export). Its own file, never
quiesce's. Written before the stop, cleared only after a restart that succeeded,
kept when one fails. Recover() completes before the boot reconciler is launched
and returns its outcome, which main.go reports on the existing backup_failed
event once the notifier exists. A defer is not the mechanism — a SIGKILL runs
none (Campaign 8 fault 10).
Also: SaveAppConfig rebuilt AppConfig field-by-field (the R-100 shape) and would
have dropped desired_state on every save across nine call sites. Replaced with
copy-and-overlay. Measured: app.yaml does not round-trip unknown YAML keys.
No hub change, no agent coupling, no user-visible string. 27/27 packages green;
7 red-proofs observed FAIL then restored.
Reports, does not refuse — pushes go straight to main with no pull request, so there is no
merge for a status check to stand at. The refusing half is .githooks/pre-push, which is
per-clone and --no-verify-able; this half notices when that was skipped.
CI reproduces the workspace's SIBLING LAYOUT on purpose: this repo's entry point invokes
the shared reuse_refs_check that lives in the felhom.eu clone next door and is never copied
here, and this repo's REUSE.md cites wgsync/reconciler.go, which lives in the hub. Without
the sibling the gate fails closed with 'gate is MISSING' — correctly, but for the wrong
reason. No uses: step anywhere; no version bump; nothing built or deployed.
README's Enforcement block now names controller_gates.py as THE entry point rather than
listing five of the seven gates individually. CHANGELOG records the allowlist entry, the
runner, the hook, the seam test and their red-proofs, and states the no-bump explicitly so
the omission reads as a decision.
A census of all thirteen gate scripts across the four felhom repos on 2026-08-02 found that
every check a CLAUDE.md names was passing and two of the four nobody is told to run were
failing. This repo had seven gates and CLAUDE.md named two; four more were reachable only via
a line in REUSE.md, and docker_run_volume_path_gate.py — RED at census time — through one line
in REUSE.md and nothing else.
controller_gates.py runs all seven plus reuse_refs_check on the repo root, streams each gate's
own output, and exits worst-wins non-zero. The shared reuse checker stays in felhom.eu/scripts/
and is invoked across the workspace — never copied here, which would recreate the drift it
detects; an absent sibling clone FAILS the gate and prints the path tried.
.githooks/pre-push runs it with --fast and refuses the push. Per-clone and --no-verify-able,
both stated in the hook itself; a manual run WARNS when the clone is unarmed.
test_controller_gates.py is a SEAM test — it asserts each member gate's own distinctive stdout,
not the runner's summary line. Red-proofed: an inert run_gate still prints 'all controller
gates OK' and exits 0, and turns the seam test red.
Tooling only: no Go change, no image build, no deploy, no version bump.
realVolumeSize mounts the NAMED VOLUME read-only into a throwaway alpine to du it
from a container view. No host path is involved — docker resolves the volume name
daemon-side — so it is the same safe class as the internal/backup/backup.go entry.
The gate was right to demand review; this diff is that review, on its own, because
burying an allowlist widening inside a feature commit is how an allowlist stops
meaning anything.
Tooling only: no Go change, no build, no deploy, no version bump.
The recovery unit on the customer's drive now carries the PORTABLE secret
class, so Tier-1/Tier-2 restore no longer depends on the whole-guest tier.
A customer needs the drive and nothing else.
Part 0's rulings overturned the brief's recommendation, on evidence:
- the data_key flag is untrustworthy (4+ encryption keys the catalog itself
labels as such are unflagged) -> R-127
- a DB password is not resettable in practice: POSTGRES_PASSWORD is ignored
once PGDATA is non-empty, so a regenerated value leaves the app unable to
authenticate against its own restored rows while the dump replay still
reports success (proven on a throwaway postgres:16-alpine)
Ruling (operator): type:secret travels, type:password never does, minus the
nonPortableSecrets code register. Plaintext -- withholding the internet-
reachable class is what licenses that, and the two are coupled.
Precedence: the UNIT WINS over the guest -- the unit's secrets were captured
in the same run as the dumps beside them, so they match the data being
restored. The fail-closed data-key gate is unchanged.
Secret values are never logged; the manifest records NAMES only.
This is D5's precondition and it is now met.
An app's namespace root IS its backup root: namespaceRoot returns a non-system
drive path as-is, so the recovery unit lands at <HDD_PATH>/backups/primary/<stack>/.
On a NAS that sits inside the share, which FileBrowser binds WHOLE — share root,
:rslave, download:true.
The bind was NOT narrowed, and establishing why inverted the fix. The share-root
:rslave bind is load-bearing (a 2026-07-22 probe proved an in-container access
through it wakes the idle automount trigger), and scoping is undefinable anyway:
apps on a share store at <share>/<app>, there is no userdata/ layer, and creating
one would write Felhom convention onto a customer's own NAS, which R-67 forbids.
So the browsing surface cannot be narrowed and the backup tree must never be
placed under it. Operator ruling: refuse the placement, keep the browse bind.
Tier 2 already refuses network targets for this reason (F-6C-1).
Nothing stranded: zero apps on network storage across all six hub customers
including Peti. R-67's browse capability is byte-identical.
FIVE surfaces, not the four the register named — settings.RefuseAsAppNamespace is
the single predicate. The deploy POST is the real boundary (it accepts any
caller-supplied HDD_PATH; DeployStack validates only os.Stat). Surface 4,
handleStorageDecommission mode=migrate, guarded only its SOURCE, so a whole
namespace could be decommissioned ONTO a NAS — that one is not in the register.
Fails closed: /mnt/felhom-drives holds both kinds, Kind exists only on a
registered path, so an unregistered path under that root refuses.
Supersedes README's "NAS backup locality — decision A" (v0.118.0).
9 tests, all non-effect (nil stackMgr, so a guard that misses panics rather than
passing). 4 red-proofs, each mutation asserted to have landed.
Suite rc=0, 27 packages, 0 FAIL. vet rc=0. Template + emoji gates OK.
admin