R-87 re-scoped by its own spike and built as Option C. MinAgent 0.129.0 unchanged.
THE QUESTION NOTHING ASKED. The weekly check proves the stored bytes are the bytes we
stored; it cannot tell us we stored the WRONG thing. A hollow recovery unit backs up
cleanly, checks cleanly at 100 percent depth, restores cleanly and gives the customer
nothing back - measured on demo-hp 2026-08-31, 120082104 B to 7036 B in one nightly run
recorded as a success (R-403). No tier and no cadence asked it. Now offsite-proof does,
nightly, on one app.
IT DOES NOT prove a restore puts data back into a running app. That stays drill work and
07 section 8 matrix row 4 is NOT moved.
THE ACCEPTANCE RULE HAS TWO PARTS AND THE OBVIOUS ONE IS A TRAP. "Check the unit against
its own packing list" PASSES a hollow unit, because a hollow unit declares nothing. So:
(1) everything declared is present, AND (2) the manifest declares what the app is supposed
to have. Part 2 is the whole value. RED-PROOFED: the naive rule makes the hollow-unit test
read verdict "pass".
THE EXPECTATION COMES FROM INSIDE THE UNIT, never the live box - the snapshot may predate
the app's shape, and GetDockerVolumes describes the running app. Database half is
DBServiceNames, the same discriminator RestoreFromRecoveryUnit uses. Volume half is
ParseComposeNamedVolumes as an EXISTENCE check, not a name match: tars are
<project>_<volume>.tar and ResolveDockerVolumeNames derives the project from the compose
file's parent dir, which inside a unit is the literal string "compose". Measured on all
eight real units on demo-hp the counts match exactly and the naming held every time - but
"held on eight" is not "derivable" (R-355). Half a rule that is true beats a whole rule
that is invented.
THREE OUTCOMES: pass, fail (readable and empty), cannot judge. An app that legitimately
has neither a database nor volumes PASSES. RED-PROOFED: alarming on any empty unit makes
that test read verdict "fail".
IT NEVER WRITES TO THE REPOSITORY and that is asserted on the ARGV as a non-effect:
--no-lock, no unlockStale, and m.runner() rather than resticStep so the unlock --remove-all
escalation is unreachable. RED-PROOFED: routing it the customer path's way makes the test
fail on "unlock" appearing in the argv.
IT TAKES acquireRunning ITSELF and skips rather than waits, because RestoreOffboxScratch
does not take it (R-408) while offbox_integrity.go states that invariant as universal.
DUE-NESS IS PER SNAPSHOT (R-86's model), never per clock. RED-PROOFED: recording a
timestamp fails the stored-value test AND breaks the rotation - night 2 re-picks night 1's
app.
ITS SCRATCH IS A SEPARATE ROOT (backups/offsite-proof) and that is a safety decision, not
tidiness: the job deletes its copy on every path, and sharing backups/offsite-restore/<app>
would mean a nightly background job deleting the verification copy a CUSTOMER is looking
at. It is also invisible to placement, so a proof copy can never be pushed into a live app.
SHARED RATHER THAN FORKED: offboxScratchDirIn parameterises the scratch resolver on its
ROOT builder, and unitOnlyHeadroom extracts the free-space gate, so the customer path and
the proof refuse at the same floor with the same Hungarian sentence. RestoreOffboxScratch's
behaviour is unchanged.
NEW EVENT offsite_proof_empty, severity error, operator-only - deliberately NOT
backup_integrity_failed, whose hub template says the store is DAMAGED. Here the store is
sound and the content is absent: different cause, different action. The hub half shipped
FIRST, in felhom.eu 1aeaa30 (hub v0.110.0, live and verified), because an unallowlisted
type is 400'd and vanishes.
33 new tests, all groups green; full suite 1689 tests, 28 packages, rc=0. All 13 controller
gates OK. Five red-proofs run and recorded in REPORT.md.
A golden carrying 0.231.0 is OWED - the fleet is on 0.230.0. Viktor's call (R-242).
MEASURED FIRST, then fixed. On the shipped v0.229.0, on demo-hp, an app's Tier-2 copy went from
120 082 104 B (4 database dumps + 3 named-volume tars) to 7 036 B (none of either) in ONE nightly
run, and the run recorded itself a success: 'Tier 2 copied docmost -> ... (14.9 KB, 0 leg(s), 0s)'.
Evidence: felhom.eu/documentation/audits/DRILL-r403-tier2-delete-2026-08-31/.
The mechanism was three individually-correct lines: RunTier2 guards the unit leg with os.Stat only
(does the folder exist), rsyncMirror is rsync -a --delete, and nothing between them compared source
to destination. An EMPTY unit is a folder that exists.
THE GUARD. One predicate, unitCarriesData/unitIsHollow (r403_hollow.go), asking the MANIFEST and
never the byte size - a big compose tree with no dumps is dangerous, a tiny unit for a tiny app is
fine. Fail closed on an absent or unparseable manifest. RunTier2 skips the unit leg when the source
is hollow AND the destination is not; the other legs still run, the run is not failed, and the skip
is recorded for the SURFACE (CrossDriveBackup.UnitLegSkipped + UnitPackageDate) as well as logged.
--delete STAYS and shrinking stays legal. 07 section 8 row 5's derived-copy rule is unchanged; the
fence is exactly one shape. TestR403_DataLegShrinkIsUnaffected is the guard on the guard.
THE HONESTY. A preserved package is older than the run that preserved it, so the card carries a
notice and the unit-restore confirm names the PACKAGE's date - read from the mirrored manifest's own
created_at, not from the status record - plus a clause saying why it is older.
THE CAUSE. RestoreTier2Unit now refills a hollow or absent primary unit from the mirror it just
restored from, INSIDE the call before returning. The hollow manifest was written two seconds after
a restore by the 5-minute capture job; any follow-up job races it. The capture itself is NOT guarded:
a capture describing an empty drive as empty is correct, and with the primary refilled there is no
hollow state left to describe. Never over a complete primary, never after a failed restore.
recordTier2Success and tier2UnitConfirmMsg keep their old signatures as thin callers, so no existing
test needed editing. New seam unitRehydrate, separate from tier2Mirror on purpose.
22 new Go tests. Red-proofs run and reverted: A6 (predicate -> size threshold), B1 (guard removed ->
the copy's 3 files are DELETED and the seam is called), B6 (a general never-shrink rule -> the shrink
case fails), C2 (only-when-hollow dropped -> the complete primary is overwritten).
R-399: monitoring.integrity.read_data_subset defaults to 100%. A pack damaged
without changing its size made plain `restic check` report "no errors were found"
on demo-hp 2026-08-30; every read-data form caught it. Cost on that 134 MB store:
35.0s structure vs 39.2s at 100%. "off" (any case) is the off token; empty means
not-configured, therefore the default; a malformed value falls back to the DEFAULT,
never to structure. A completed check over 5 minutes logs a WARN naming the
duration, the depth and R-401 — operator log only, no hub event, no depth change.
The depth is now recorded with the verdict (LastIntegrityDepth; empty = NOT
RECORDED, never "structure").
R-400: 24 debug-page references, 17 dispatched, 7 dead — three of which fetched on
page LOAD, so those panels were permanently blank. backup/crossdrive implemented;
backup/infra, hub/infra-push, dr/infra-status, storage/watchdog-status and both
storage/simulate-* deleted with their panels and JavaScript.
scripts/debug_route_gate.py fails in both directions and is registered after the
seven were resolved. 18 referenced, 18 dispatched, none orphaned.
Corrections: the dead-field warning in report/types.go said the controller runs no
integrity check and the notifiers are called from nowhere — both false since
v0.227.0. controller.yaml.example gains its missing integrity: block.
integrityCheckTimeout's "ships OFF" comment rewritten.
Nothing ever verified that the off-site copies are still readable. The
whole-guest tier has verify jobs; the tier holding the customer's documents and
photos had none -- the complete set of restic verbs this controller used
contained no `check`. We would have found out at restore time, with a customer
waiting. On 2026-08-21 a deliberately damaged pack was caught at once by plain
`restic check`; we had never run it.
R-397: NotifyIntegrityOK/NotifyIntegrityFailed existed with no caller, the hub
allowlists both event types and carries the Hungarian text for both, the
settings checkbox exists, and the debug button posts to /api/debug/backup/
integrity. Everything was built except the part that runs. SIXTH instance of
that shape in this project.
THE HAZARD SHAPES THE WHOLE DESIGN. resticStep self-heals a crash lock by
running `unlock --remove-all` and retrying, and its own comment records why that
is safe: every caller holds the in-process single-flight mutex, so any lock it
meets is stale. A check that did not take that flag could meet a LIVE prune's
lock from this same box, remove it, and retry over the top of it. So the check
TAKES THE FLAG and SKIPS rather than waits -- waiting would pin the nightly
backup behind it, and a skip costs nothing because due-ness makes tomorrow try
again. TestR359_SkipsWhenRunningFlagHeld asserts the NON-EFFECTS: restic never
invoked, `unlock` never in any argv. Its red-proof prints the real thing --
restic running `check` while the flag was held.
DUE-NESS, NOT A WEEKDAY. Daily job, weekly behaviour: "is the last successful
check older than 7 days?" not "is it Sunday?". R-341 is exactly the other shape,
a dated check quietly missed and never caught up. No Weekly primitive added.
THREE OUTCOMES, NOT TWO. Skipped, Unreachable and failed are different facts.
"I could not look" is not "I looked and it is broken" -- R-339 already owns
reachability, and a second alarm for the same fact trains the operator to
discount the one alarm that means the backups are damaged. A timeout is
unreachable, never damage. A failure advances due-ness (a broken store must not
be re-checked nightly); a skip and an unreachable store do not.
Success is severity `info`, which severityNotifies DROPS -- it mails NOBODY, by
design. A weekly success e-mail is how people stop reading their alerts.
The customer gets a SENTENCE; restic's words go to the log, truncated (R-379:
615 bytes of raw database text reached a customer once). read-data-subset ships
OFF and a malformed value is refused at read time rather than handed to restic,
where one typo would fail the whole check.
Published on OffboxReportStatus, NOT on report.BackupReport's IntegrityOK --
those were retired by R-331 YESTERDAY and TestBackupReport_DeadFieldsStayZero
still passes unmodified.
Also: the monitoring page stopped promising a Sunday job that never existed, and
the debug button got its dispatch case.
PART 0 WAS NOT BUILT, AND R-398 WAS MY OWN MISTAKE. The seam it asked for
already exists: offboxRunner/SetOffboxRunner/m.runner() has been injectable
since the off-site tier shipped, and other tests drive restic-backed paths
through it. A resticStepFn seam would have been WORSE here -- it would replace
the `unlock --remove-all` escalation and hide it from the assertions that must
see it. R-358's AST ordering test is converted to a real execution test instead,
which immediately surfaced something the AST walk could not: unlockStale
legitimately runs before the restore.
Four red-proofs, each printing the pre-fix behaviour. Green gate: 28 packages,
rc 0. All 12 controller gates OK.
R-379 and R-380 were one failure. Both ended with a half-restored database; the
only difference was whether it looked broken. Postgres emptied and crash-looped;
MariaDB applied part of the dump and reported health=healthy with a zero-row
schema-version table. Measured live on demo-hp 2026-08-22.
The undo copy was already taken and already good - proven by hand that day on
both engines. Nothing in the product could apply it. Now it does, with the same
ImportDump call, before any restart and inside the DB-only window.
The WHOLE undo set, matched on this run's stamp. writeSafetyDump returned one
path for an app with two databases; a rollback on that would restore one and
leave the other half-written.
When the rollback also fails the app is HELD STOPPED (operator ruling): a running
app on a half-written database lets the customer make the damage permanent. Every
start path refuses it - customer button, appstop Recover, boot sweep - via the
shared driveStartGate, checked ABOVE its driveless early return because these
apps have no drive. The marker is ended so nothing auto-restarts it. The row goes
red. Cleared with --clear-restore-hold, an operator CLI route.
--single-transaction is a belt on Postgres only; MariaDB DDL is not transactional
and that is why the rollback is the fix.
R-381: the engine's stderr stops reaching the customer (615 bytes on MariaDB, its
middle rows out of their own database) and starts reaching the operator log,
which never had it.
R-382: the summary log prints the volume count it already held.
Undo copies resolve to their own app, are marked IsUndo, and are capped at 3 per
app, pruned from the capture side. The reported render-as-an-app symptom did NOT
reproduce - the live page was read first and had zero occurrences.
Tests 1468 -> 1483. Eight red-proofs; ONE PASSED and is reported: the R-381
behavioural test injected below ImportDump. A guard at that layer now convicts.
The retrieval clause rendered unconditionally on every page and is false on a
reachable state - the same screen where the orphan card says we cannot tell.
The condition is a fingerprint PINNED at the decision, not a comparison against
the current key. The obvious proxy asks about the wrong key: the set-aside
copies were written under an older key the box no longer has, so on a
twice-rebuilt box the proxy promises about copies nothing can open. Demonstrated
- under the proxy, the replaced-package and legacy cases both flip back to
promising.
The pin is a recorded assumption and says so: nothing on the box records which
key wrote those copies. Empty is not a match. A countdown started before this
carries no pin and takes the cautious branch, not a backfill.
A sweep of all 36 templates found a fourth instance (backups page, same
condition applied) and a fifth (the confirmation screen, correctly left alone -
true at the moment of the decision).
New retrieval_promise_gate registers each claim with a reason rather than
banning a verb: a string ban failed twice, and the honest replacement copy
contains the stem.
REMINDERS (SEC 2.3). The offer epoch now stamps when it began, and the
undecided reminder escalates in EMPHASIS at 1, 3, 7 and 14 days.
THE READING IS STATED BECAUSE THE SPEC IS AMBIGUOUS, and it is written into
the code where it can be corrected. For an ABANDONING box, 5/3/1 are
unambiguously days REMAINING before a deletion. An undecided box has no
deadline - nothing counts down to anything, because SEC 7.5 deliberately does
NOT auto-abandon - so 14/7/3/1 cannot be "remaining" and are taken as days
ELAPSED, with the wording firming up rather than the bar appearing and
disappearing. If the operator meant something else, one function changes.
The stamp is re-set on every entry into the offered state, so a box that
settles and is later rebuilt starts its ladder again instead of inheriting an
old one.
OPERATOR LEVERS (SEC 7.5). --abandon-status, --abandon-extend=N and
--abandon-stop on the controller CLI, beside the existing operator
subcommands. They exist because the path that ACTUALLY happens is the customer
telephoning, and support needs something to press.
They live on the CLI and not in the customer UI deliberately: extending a
deletion the customer asked for is an operator judgement, and a customer who
wants it stopped already has the self-service route - they recover with their
code, which cancels it.
BOTH REFUSE RATHER THAN NO-OP, in two situations: when no countdown is
running, and when the store has already been deleted. A silent success is the
thing an operator most easily mistakes for "handled" - they would tell the
customer their data was safe when it is gone. Pinned by two tests.
--abandon-extend counts from NOW, not from the old due date, and a test proves
the old date passes without deleting anything.
Green: go build, go vet, go test ./... all pass; controller gates OK.
FULL PAGE ONCE PER ENTRY, NOT ONCE EVER. "Most nem" used to set a flag that
nothing ever cleared, so a box that abandoned its history and was rebuilt
months later - a genuinely NEW situation - would never see the page again. The
offer now carries an EPOCH, advanced on the edge into the offered state, and a
dismissal is recorded against the epoch it was made in. A fresh entry passes
the dismissal by arithmetic, with nothing to clear and nothing that can be
forgotten to clear.
That is NOT the flag the operator's ruling forbids. The forbidden thing
remembers that the customer decided so the screen can be suppressed while the
state stays wrong. This records WHICH SITUATION a dismissal was about.
A REAL BUG, caught by the test and not by review: the first draft returned
early from recoveryInterrupts when the offer was false, so the FALLING edge
was never recorded, RecoveryOfferActive stayed true through a settled period,
and the next entry counted as a continuation. The page never came back - the
exact defect the epoch exists to fix, reintroduced inside the fix. The sync is
now unconditional and the ordering is commented as load-bearing.
THREE LEVERS, THREE SCOPES, and none of them removes the route:
- clicking the bar away -> a browser SESSION cookie, cleared on login, so
the reminder is genuinely back at the next login. Nothing persisted.
- "ne emlekeztessen ujra" -> durable, epoch-scoped, silences the BANNER ONLY.
It starts no countdown, abandons nothing, and a fresh entry reminds again.
- "most nem" -> suppresses the full page only, as before.
The entry point on /backups/remote is bound to the OFFER and to nothing else,
pinned by a test that fires all three dismissals and asserts it survives.
SEC 7.3 / Q7 - THE TRAP DOES NOT SURVIVE THIS SESSION. While a recovery is
outstanding the "Helyrealitasi kod letrehozasa" button is UNAVAILABLE, not
merely captioned: creating a new code seals the current key, demotes the
package that opens the earlier history to retained custody that no shipped
path can read (R-199), and re-enables the recovery screen through the orphan
route while invalidating the code that screen accepts. A warning beside a
button is a warning people click past. The card now explains and points at
/recovery instead.
SEC 2.4 - the abandon confirmation changes with the behaviour. It used to
promise "felretesszuk - nem toroljuk". It now states the grace in days (from
the constant the countdown actually uses, never a literal in prose), that the
sealed package goes with it, that the customer can change their mind, where
the date is visible, and that the question does not come back afterwards.
The countdown is shown on /backups/remote for the WHOLE window - the bar
elsewhere is a nudge, this is the record, and a deletion date must be findable
on a quiet day too.
Tests: once-per-entry across a full settle-and-re-enter cycle; the banner
dismissal proven to be a session cookie (MaxAge 0, no Expires) and to persist
nothing; the opt-out proven to silence the banner while leaving the offer, the
route and the countdown untouched, and to remind again on a fresh entry; the
entry point surviving all three dismissals; a settled box showing nothing; and
the back-redirect refusing "//evil.example".
An existing test (TestRecovery_E) was updated: it asserted the legacy boolean,
which the epoch replaces. It now asserts the dismissal landed on the current
epoch, which is the stronger property.
Green: go build, go vet, go test ./... all pass; controller gates OK.
Until now "set aside" renamed the remote store and touched neither the escrow
nor the key, so the hub went on holding a sealed package for a key the box no
longer used. Shape (c) compares those two, finds them different, and offers
recovery - correctly, and for ever. A customer who had already said "I do not
want the old data" would be asked again at every login.
The operator's ruling is that the answer is NOT a "they decided" flag: fix the
state, do not remember that it is wrong. So the decision starts a countdown,
at the end of which the set-aside store and the sealed package that protects
it are removed TOGETHER. Afterwards shape (c) has nothing to compare and the
offer falls silent on its own - because the state is right, not because
something remembers it once was not.
THE GRACE IS REAL. The recovery offer stays reachable for the whole 14 days;
that is the change-of-mind path, and a grace in which recovery is impossible
would be decorative.
BOTH HALVES OR NEITHER. Removing only the store leaves a package that opens
nothing; removing only the package leaves ciphertext nobody can ever decrypt.
The two cannot be atomic across two machines, so it is a two-phase commit:
delete the store, record a durable marker, and keep DECLARING
offsite.abandon_purge_requested until the hub's ACK stops reporting a
superseded package. A crash between the halves re-declares on the next sweep;
it never leaves the pair half-removed and silent.
HUB HALF - SEC 8.2 ANSWERED: yes, the hub was needed, and only for this.
store.PurgeSupersededEscrowForCustomer is the one place R-198's retention is
ever undone, and it never touches host_escrow (the package covering the key
the box uses now). The handler acts on the DECLARATION, never an inference,
and is placed immediately BEFORE the ACK is built - so
GetEscrowStatusForCustomer reads the effect and the SAME response closes the
box's two-phase commit. No second round-trip and no window where the box
thinks it is still owed. felhom-agent was NOT touched.
The countdown starts in ResetOrphanedRepo, NOT in the shared helper: the
helper is also the unclaimed auto-reset path, where nobody decided anything,
and an as-delivered box tidying a stranger's leftover store must not get a
customer's deletion clock. Pinned by a test.
Cancellation is wired into the recovery unlock, BEFORE the tier-up and the
listing - those can fail, and a countdown surviving a successful unlock
because a later step errored would delete the history the customer just
proved they can open.
The sweep is a Daily job at 05:10, not on the backup leg: it must run on a box
whose tier is not configured for runs. Quiet by construction on every box with
no countdown, and that silence is asserted.
Tests (all clock-injected; SEC 7.4 forbids shortening a live timer):
Scenario E (aside + package kept + countdown + offer still reachable, and
NOTHING deleted), Scenario F (both halves, the declaration repeating, the
close-out), Scenario G (cancel, path still nameable, no later deletion),
plus: not closed out while the package remains, a transport failure leaves the
countdown due and retrying, the no-op sweep issues zero remote commands, and
the unclaimed auto-reset starts no countdown.
RED-PROOFS, each with the mutation confirmed present in the file first:
F1) store deletion skipped -> Scenario F FAILS (no rm issued)
F2) declaration dropped from the report -> Scenario F FAILS (the hub is
never asked; the package would outlive the store for ever)
G) CancelAbandon made a no-op -> Scenario G FAILS (uncancellable countdown)
Green: controller and hub both build, vet and test clean; controller gates OK.
NOTHING WAS DELETED ANYWHERE - the terminal step has only ever run against
in-test fakes.
THE FACT WAS COMPUTED EVERY CYCLE AND KEPT NOWHERE. EscrowAutoConfirmer.Reconcile
has compared the hub's restic_pw_sha256 against the local key on every ACK since
SLICE 3. On the final-walk venue it logged, at 03:28:03Z and thirty-five minutes
before the customer looked, "the hub's escrow blob does not cover the CURRENT repo
password (hub hash 30ef574f != local 9b4a9a9d)" - and dropped it. The recovery
screen, evaluating in the same process, went on asking a question that could not
see it.
Now persisted: settings.HubEscrowKeySHA256 + HubEscrowKeyCheckedAt, recorded
UNCONDITIONALLY in Reconcile beside RecordPresence and RecordSuperseded - same
place, same reason: the box that needs it most is the rebuilt one with no target,
on which every gate below returns early.
OffsiteRecoveryOffer gains SHAPE (c): the hub holds a package for a key OTHER than
the one we are using. (a) and (b) are both proxies for that question and both have
now been wrong in opposite directions - (a) goes false the moment anything mints,
(b) is unreachable while the escrow is pending.
SEC 7.2, decided deliberately and stated in the code:
- a KNOWN DIFFERENCE offers, however old the reading. Age is not gated on. Both
sides are local; only the hub's half can be stale, and what the hub holds does
not change without a ceremony THIS box runs, which refreshes the hash on the
next ACK. Gating on age would make a box offline from the hub silently stop
offering - the exact failure this session removes. CheckedAt is persisted for
diagnosis, not as a gate.
- an ABSENT hash falls back to (a)/(b) and does NOT offer. "" is the hub
positively saying its package seals no repository password (legacy hash-less
escrow). Nothing to compare, and offering would put a permanent screen in
front of every legacy box.
The write damper: CheckedAt refreshes on every ack carrying a hash, but a save is
skipped when both the hash and the UTC day are unchanged, so an idle box does not
rewrite settings.json every fifteen minutes. It records WHEN WE LAST HEARD, not
when it last changed - the R-100 distinction.
Tests: Scenario C (a differing key offers, with both proxies asserted false first),
Scenario D (a matching key offers nothing), fact 1 still required, shape (a) still
works, and both SEC 7.2 halves.
RED-PROOFS, each with the mutation confirmed present in the file first:
D) hubHash != localHash conjunct dropped -> Scenario D FAILS (a healthy box
offered recovery forever); Scenario C still passes
WIRING) RecordEscrowKeyHash removed from the EscrowAutoConfirmer literal in
main.go -> TestMainWiresRecordEscrowKeyHash FAILS. This is the ships-inert
shape: unwired, everything compiles, every test in the package passes, the
auto-confirm still works, and shape (c) reads an empty hash forever.
Green: go build, go vet, go test ./... all pass.
R-218's declaration half shipped in v0.201.0 and works. Its consume half never
existed. Reconcile ran exactly twice per process — at start-up and when the
recovery screen drives it — and BOTH fire before the hub has anything staged,
because the hub stages in RESPONSE to the declaration those runs precede.
Measured on the R-201 re-walk: unlock reconcile 11:43:07, hub staged 11:44:57
saying 'next cycle', a full report cycle ran 11:55:46, still unconsumed at
12:06. A guest command line applied it in 18 seconds — everything correct except
the trigger.
Bridge.RetryIfDeclared re-runs the SAME reconcile on a 5-minute tick, driven from
the box's own published declaration (OffboxReportStatus().State) — the very
statement the hub acts on, so the two cannot disagree.
Poll, not an ACK flag, decided on the promise: the no-target message says 'amint
megvannak' (no deadline) and the card says 'within a day'. Five minutes is inside
both by a wide margin and needs no hub change.
It stops by construction — a healthy box does no work and logs nothing — and the
settle gate is deliberately kept via ReconcileWhenSettled.
The marker was investigated and left alone: applied_marker lives in the guest's
DataDir, which a rebuild destroys, so it cannot suppress a legitimate re-run.
R-220's customer half: the refusal no longer tells the customer to choose from a
list that may be empty. It names the rebuild, points at the Meghajtók page, and
promises no outcome.
Red-proofs: remove the retry -> credential uncollected (the dead end reproduced);
drop the stop condition -> a healthy box hammers the hub; call Reconcile instead
of ReconcileWhenSettled -> settle gate bypassed; restore the old sentence -> the
impossible action returns.
28 packages ok, vet clean, all controller gates OK.
R-225 — an unread store said '0 pillanatkép / 0 / 50 GB' above a card stating
it held backups under another key. An SFTP listing found snapshot f3d9cd67 and
12 535 KB really there; snapshot_count and repo_size_bytes were simply ABSENT
and the zero value spoke for them. StatsKnown is now NAMED, for the same reason
OffsiteInventory.Empty is: zero is what an unread store and an empty one both
look like, and on the wire 'absent' and '0' are the same bytes. The fill bar
renders only when the fill is known — a 0%-wide bar is a picture of emptiness,
and a picture is a claim. A measured zero still says zero.
R-227 — WHICH LAYER ANSWERS: traefik, and this repo generates its config. But
traefik v3 serves no static files, so a branded proxy page needs a new always-up
container for every 502 on the box — out of proportion, and scoped in the report
rather than built. Shipped instead: the unlock posts via fetch and answers a
gateway failure in Hungarian without leaving the page. Progressive enhancement —
with no JS the plain POST is unchanged and still shows the proxy's error, which
the report says plainly rather than implying otherwise.
R-228 — the set-aside history was recorded in orphaned_renamed_to and read by
nobody: a census found zero references in any template or handler, while 12 535
KB sat at that path. It is surfaced as two facts and stops. It does NOT promise
the history can be reopened, because it cannot be by anyone today (R-199's
inventory is unbuilt) — and the set-aside CONFIRMATION copy was corrected for
the same reason: 'a helyreállítási kód nélkül többé nem lesznek megnyithatók'
implied that WITH the code they could be. The field's own comment called it
'recovery-code-recoverable', which was the same over-promise in the code.
Tests: scenarios F, G, H as render tests per branch of each gate. Red-proofs,
each demonstrated failing then restored: remove the StatsKnown guards (F,
'R-225 RETURNED: an unread store reports a snapshot COUNT of zero'), delete the
set-aside block (H). The F assertion on the fill bar is scoped to the bar's own
container — a bare width:0% search matched unrelated elements and would have
passed for the wrong reason.
28 packages ok, vet clean, all controller gates OK (the emoji gate caught a
warning sign in a template comment).
R-216: the offsite key recovery is a coupled feature and now says so. featureProbes +
featureMinAgent 0.125.0 + a Supports gate at the unlock entry point, FAILING CLOSED — an
agent that cannot answer is named as such instead of the customer's code being blamed.
Measured live: a 404 from agent 0.120.0 came back as "we did not accept your recovery
code, check that all ten words", in 0.134 s, against a perfect code.
R-218: delete the repo-password short-circuit in needsOffsiteCredential. The declaration
stops when the TIER WORKS, not when a key exists — installing a key is the recovery
screen's whole job, so succeeding at recovery was switching off the mechanism that would
have delivered the coordinates to use it.
R-219: the unlock finishes the job — place the key, bring the tier up, then list. Without
it the promised listing could never render on the shape the screen exists for.
R-217: an unreadable store no longer claims to have opened with unattributable content
(the OffsiteInventory{} zero value). Opened / empty / unreadable are three states.
R-222: a code that is right about a RETAINED earlier package is named, not blamed. States
what the hub knows and promises nothing — no read path exists.
R-215: GET /recovery is gated on the same predicate as the interception.
Five red-proofs, each demonstrated failing and restored.
A customer whose machine was rebuilt had everything needed to get their data
back and no way to find out: the only route was a command line. This is the
screen that closes that.
IT UNLOCKS, AND ONLY UNLOCKS (operator ruling). It explains, takes the recovery
code, opens the repository and shows what is in there — apps, dates, sizes. It
restores nothing: restore is already per-app and lives in the backups area, and
a screen that unlocks and then offers to overwrite is two decisions wearing one
button.
ONE CORE, TWO CALLERS. RecoverInstallCore is split out of RecoverAndInstall; the
CLI wrapper keeps its exit codes and printed lines byte-identical, and the
handler drives the same function. Two implementations of the one operation that
can permanently lose a customer's data would drift, and only one would be
tested. Asserted from source on both sides by AST.
THREE WAYS OUT, none a dismiss button: recover; 'most nem' (the full page stops
interrupting, the backups-area entry point stays PERMANENTLY, bound to the offer
and never to the postpone flag); and 'I do not want the old data' — confirmed
TWICE and reaching the SHIPPED move-aside, which sets aside and never deletes.
THE CODE IS HANDLED NO MORE LOOSELY THAN ON THE COMMAND LINE: POST body only,
never logged, never persisted, never echoed, cleared on every path, no-store,
autocomplete off. No lockout — the code is a ten-word phrase, and locking a
customer out of their own data for a typo is worse than anything it prevents.
TWO DEFECTS THE TESTS CAUGHT, both fixed: an UNCLAIMED (legacy-open) box would
have been shown the page, because RequireAuth passes such a box through; and the
inventory nil-dereferenced when no off-site target was configured, which is
exactly the pristine rebuilt shape.
An absent off-site object has four meanings — never configured, mid-restart, a
transient config read failure, and rebuilt-and-stranded — and the hub cannot tell
them apart. The box can, from two local facts it holds with certainty, so it says
so instead of leaving the hub to deduce it from a silence (operator ruling).
The ACK's identity_blob_present is now recorded on EVERY ACK, before the gates
that used to discard it: on a box with no off-site target the auto-confirm returns
immediately, which is exactly a rebuilt box, so the one fact distinguishing it from
a box that never had off-site backups was thrown away every cycle.
The declaration needs BOTH halves — a fresh data area AND a hub-held recovery
package. Freshness alone is a box that never had off-site backups; dropping that
condition makes the whole fleet ask for credentials, which is what the Scenario B
test exists to catch.
The object carries enabled:false and zero sizes, which is what makes it inert to
the hub's existing fill and staleness checkers and to a pre-upgrade hub. A
configured box's JSON is byte-identical to v0.198.0's.
--print-reset-code runs as a separate process and persists the new code;
the running server's cache was never told, so the code the customer was told
to type was refused until the controller restarted. Nothing said so — during
the 2026-08-04 drill that cost two attempts with an operator present.
effectiveClaimCode now reads through to the persisted state before applying
the settings-vs-config precedence, which is itself unchanged. Read-through,
not a TTL: a TTL would leave a window in which a superseded code still works,
which is worse than the bug. Fails closed on an unreadable state; an absent
file is not an error.
The gap was already detected and warned about, in Hungarian, naming the app and the folders --
that warning is what stopped the R-201 drill. The defect was that the run still reported `ok`
beside it, and a warning standing beside a success is read as a success.
last_status gains "incomplete": minted, because "ok" | "error" | "running" had nothing meaning
"it ran, and this app is not fully protected". NOT "error" -- the rest of the run worked and
what was captured is real, so SnapshotCount and the LastSuccess anchor still record it. Half a
backup is not no backup.
The gaps are now recorded STRUCTURALLY (offboxRunResult.mandatoryGaps), not only as prose, so
the verdict has something to act on. It reaches the operator through the EXISTING per-run digest
(backup_run_failures) rather than a new event type -- a new type is a two-repo change and the
hub drops anything outside allowedEventTypes.
The stat-filter gains the ClassMandatory check Tier 2 already had. It is a NO-OP today
(TierOffsite admits mandatory only), so no customer-visible warning disappears -- demonstrated
by widening the tier filter alone and watching the check hold the line.
ANTICIPATED: calibre-web on demo-hp has exactly this gap, so its off-site status becomes
incomplete the moment this ships. That is correct and is the point.
Red-proofs: my first Scenario-C proof PASSED because the test only reached offboxCaptureSet
while the mutation lives in runOffboxInternal -- a mutation the test cannot observe is not a
red-proof, and the fix was the test. The run-level test now fails under both mutations
(unreachable gap recording; unconditional ok).
This is D5's precondition and it is now met.
An app's namespace root IS its backup root: namespaceRoot returns a non-system
drive path as-is, so the recovery unit lands at <HDD_PATH>/backups/primary/<stack>/.
On a NAS that sits inside the share, which FileBrowser binds WHOLE — share root,
:rslave, download:true.
The bind was NOT narrowed, and establishing why inverted the fix. The share-root
:rslave bind is load-bearing (a 2026-07-22 probe proved an in-container access
through it wakes the idle automount trigger), and scoping is undefinable anyway:
apps on a share store at <share>/<app>, there is no userdata/ layer, and creating
one would write Felhom convention onto a customer's own NAS, which R-67 forbids.
So the browsing surface cannot be narrowed and the backup tree must never be
placed under it. Operator ruling: refuse the placement, keep the browse bind.
Tier 2 already refuses network targets for this reason (F-6C-1).
Nothing stranded: zero apps on network storage across all six hub customers
including Peti. R-67's browse capability is byte-identical.
FIVE surfaces, not the four the register named — settings.RefuseAsAppNamespace is
the single predicate. The deploy POST is the real boundary (it accepts any
caller-supplied HDD_PATH; DeployStack validates only os.Stat). Surface 4,
handleStorageDecommission mode=migrate, guarded only its SOURCE, so a whole
namespace could be decommissioned ONTO a NAS — that one is not in the register.
Fails closed: /mnt/felhom-drives holds both kinds, Kind exists only on a
registered path, so an unregistered path under that root refuses.
Supersedes README's "NAS backup locality — decision A" (v0.118.0).
9 tests, all non-effect (nil stackMgr, so a guard that misses panics rather than
passing). 4 red-proofs, each mutation asserted to have landed.
Suite rc=0, 27 packages, 0 FAIL. vet rc=0. Template + emoji gates OK.
Foundation only, no version bump -- nothing customer-visible changes yet. The
offer UI (Part 3), the degraded banner (Part 4) and the controller half of the
absent-target signal (Part 5) are NOT here; they are tracked as E-2 in
OPEN-ITEMS.md so this cannot become a sixth "seam built but never wired". The
fifth was found by E-2's own Phase 0: NotifyStorageDisconnected is defined and
called nowhere, so a drive going absent emits no event at all today.
StoragePath gains BackupTarget bool -- the sibling role to Schedulable/
IsDefault/Kind. It is INTENT, not truth: the authority is the agent's
backup.local_backup_target, and truth is read from GET /backup/tiers. This
records what the customer ASSIGNED so the controller can render the state,
notice the drive going absent, and detect drift.
Invariants, each pinned by a test asserting the CONSEQUENCE not the mechanism:
- a drive NEVER acquires the role by appearing (registration does not set it)
- exactly one carrier; assigning moves rather than duplicates
- sticky: a new bigger/faster drive does not steal an assigned target
- an absent target STAYS assigned -- clearing on disconnect would be a silent
retarget by omission ("no target configured" instead of "drive missing")
- a network share is refused (R-108 risk model; the role is about LOCAL disk
failure)
Red-proof C: adding auto-elevation to AddStoragePath fails
TestRegisteringDrivesNeverAssignsTheBackupTarget with
registering drives assigned the backup target "/mnt/hdd_1"
Attributes may suggest and refuse the absurd, never select: demo-felhom's backup
drive is an external USB HDD and BOTH demo boxes report removable=0, so a
transport rule disqualifies the reference drive and a removable rule finds no
candidate at all.
Green gate: build + vet + test rc=0, run separately from this commit.
Tier2LastRun is the attempt clock and was rendered as 'Legutóbbi másolat' in the restore
confirm dialog. New LastSuccess + SuccessTracked anchor; tier2Update makes the three
rebuild sites safe by construction. F-DIAG: six distinct causes, target-aware redaction.
LastRun records an attempt, not a result. New OffboxTarget.LastSuccess, set only on the
success branch via the pure offboxAnchorAfterRun rule, carried to the hub as last_success.
Closes two silent-wipe sites (settings save, hub re-apply).
ONE setting (window start W) drives every nightly leg at fixed, never-stored
offsets: DB dump at W, tier-2 at W+60m, off-box at W+105m (wrap-safe). Precedence
settings > controller.yaml db_dump_schedule > 02:30.
- scheduler.UpdateDaily: retime a daily job at runtime (no restart) via a per-job
buffered resched chan + a select case in runDailyJob.
- new pure package internal/backupwindow (LegTimes/GateWindow/EffectiveWindow).
- quiesce disk-tier window gate: scheduled cycles run only inside [W+2h,W+6h) with a
safety valve (age>cadence+24h runs regardless); manual TriggerNow never gated.
Backend.Due now also returns the backup age (from the agent's own /backup/due).
- backup page: Mentési időablak card (time input + derived leg/gate rows); POST
/backups/window validates -> saves -> UpdateDaily x3 -> flash.
Tests: 5 groups, all red-proofed. Agent/cadence//backup/due untouched.
Mint a 160-bit capability URL (/s/<token>) serving a standalone read-only guest
launcher: same tiles, opens apps in new tabs, no account, no admin session.
Information only, zero control — every privilege stays behind each app's own auth.
- /s/ pre-auth pass-through (after the claim gate) + session-CSRF exemption; guest
password POST carries its own pre-auth HMAC CSRF.
- Constant-time token match; empty stored token = disabled = byte-identical mux 404.
- Optional per-share password: separate bcrypt hash + own attempt map; signed cookie
= HMAC(token|passwordHash) keyed with web.session_secret, so rotate/change invalidates.
- Guest labels ride the v0.164.0 ruling; never expose internal state vocabulary.
- Token redacted in logs (/s/<redacted>); never in CHANGELOG/REPORT/CONTEXT.
- Admin modal: copy-link, QR (go-qrcode), set/clear password, rotate, disable.
- Tests: Groups A-G (14) + 3 red-proofs verified red.
ONE additional restic call tagged [felhom-offbox, _shares] carrying the payload
staging dir + every mandatory share folder. Hooked into runOffboxInternal AFTER the
per-app loop and BEFORE retention, so forget --group-by host,tags covers the _shares
group with no flag change. Reuses resticStep, the caller's repo-ensure and
single-flight, and the SAME enlargement-gate arithmetic.
- quota gate degrades the push to MANIFEST-ONLY, never to nothing
- EnlargedBlocked keeps the RAW _shares key (templates index by it); the display
mapping applies only at the notification + Hungarian-prose boundaries
- OffboxTarget gains SharesLastRun/Status/Count for per-tier page truth
- zero-toggle notice suppressed when the shares leg provided coverage
- reserved-name defense: an app keyed _shares is excluded from the run loudly
RED-PROOFS RUN AND REVERTED (all fired):
1. shares leg appends into the app's argv -> isolation test FAILS
2. mandatory->offsite mapping inverted -> Scenario A + B FAIL
3. manifest-only degradation dropped -> Scenario C FAILS
SMBSettings + SMBShare registry in settings (password never stored — only UserSet);
NetBIOS-safe name validation. Pure infra renderers: RenderSambaConfig (hardened
global block: SMB2 floor, bind interfaces only=lo eth0, disable netbios=no, force
user block) + RenderSambaCompose (network_mode host, pinned image, :ro bind for
read-only shares, passdb volume). Exact smb.conf golden + CRUD/validation tests.
After a completed escrow ceremony the Távoli mentés page showed the yellow
"Helyreállítási kód szükséges" card for ~15 min until the next hub-report ACK
flipped pending→escrowed. Phase-0 diagnosis (read-only) = verdict A (report-cycle
lag), already resolved on the demo box (escrow_state:"escrowed"); hub Hypothesis B
verified false (SaveHostEscrow ON CONFLICT already clears stale_at on upload) → no
hub change.
- settings.OffboxTarget.CeremonyCompletedAt: stamped on the recovery-code claim,
zeroed on the auto-confirmer Flip + the deprecated manual confirm; persisted.
- web/handlers.go: offboxCeremonyWaitState + escrowCeremonyGraceWindow (35m).
- backups_remote.html: info "megerősítésre vár, legfeljebb 15 perc" card → warn
"a megerősítés nem érkezett meg" past the window. Existing branches untouched.
- backups_escrow.html: "Mi történik ezután?" note on the wizard's final step.
- Test web/escrow_wait_state_test.go (truth table + red-proof recorded in REPORT).
No scheduler/agent/hub/endpoint changes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017qDiBqKKQ5vPB5fXBqu7Kp
tier2_capture.go: classified apps get TierSecondary per-bind legs (paperless copy shrinks — export
drops); legacy apps keep the byte-identical resolver set. v2 relpath-mirroring layout
(backups/secondary/<stack>/{marker LAST, recovery-unit/, hdd/<rel>/, userdata/<rel>/}); N>1 native
(errTier2MultiDir/tier2AppDataName deleted). Migration=delete-and-rebuild + reconcile; all RemoveAll
via tier2SafeRemove (refuses outside backups/secondary/). SSD=state-only tier. selectTier2Target
never picks network storage (pinned+auto, F-6C-1). Restore reads v2 behind a marker gate.
Part 0: offbox_enlarge_blocked is a persisted one-time Load seed (opt-out sticks), not a getter
append. Part 0.5: offsite restore scratch prefers a local (non-network) path.
Full v2 test suite + all 10 §10 red-proofs verified. Destructive writes bounded to backups/secondary/.
PlaceOffsiteRestore: live target via raw GetStackHDDPath not AppNamespaceRoot (F-3a-1a: no SSD
merge; undeployed refused), placement headroom gate (F-3a-1b), stat pre-pass over all placements
before any copy (F-3a-4: no partial writes), scratch removed on success/kept on failure (F-3a-2).
mapOffsiteRestorePaths refuses the namespace root itself (F-3a-3).
Delivery chain: DefaultEnabledEvents + GetNotificationPrefs append-if-absent migration + settings
checkbox + handler slice; paired with hub v0.55.0 allowlist (no customerMessages entry — raw
dynamic message survives). +8 tests; all 6 controller §10 red-proofs verified.
The customer sets + owns the dashboard password via a hub-emailed one-time
claim code. An unclaimed box (code hash present, no password) serves ONLY the
claim page — every other route → claim page (302) or 401, so a Day-0 box is
never open on the internet. A set password disables the gate (auth wins).
Reset rides the same code engine (login "Elfelejtett jelszó"). Legacy-open
(no password, no hash) shows a red transition banner until the hub delivers a
hash. Report ACK caches the code state idempotently by generation; report
carries claimed (set-only). --print-reset-code root escape hatch. Requires
hub v0.50.0. Gate-coverage signature test + 4 red-proofs proven.
QuotaGB rides the descriptor into OffboxTarget; RepoSizeBytes persisted
from restic stats. Pre-run gate: >=100% refuses NEW backups (Hungarian
notice + operator alert) but prune STILL runs (red-proofed) and restore is
never gated; >=80% warns. /backups usage bar (quota>0 only). The hub
report gains the non-secret offsite status object for the OffsiteChecker.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
Pairs with agent v0.77.0. StageEscrowSecret pushes the repo password to the
agent (POST /escrow/stage-secret) at offsite-enable → EscrowState="pending".
Atomicity gate: RunOffboxBackup (scheduler + handler) refuses until
EscrowState="escrowed" (operator POST /backup/offbox/confirm-escrow after the
escrow ceremony) — no un-recoverable offsite ciphertext can exist. DR:
POST /backup/offbox/inject-password pre-places a recovered 64-hex password 0600
(honored by WriteOffboxSecrets' IsNotExist guard; refuses clobber without
force). DR recipe gains non-secret offsite_restic coords (DRResticCoord); SFTP
key regenerated at DR, not escrowed. New settings.OffboxTarget.EscrowState.
Tests + atomicity & inject companion red-proofs green; UI gates pass. NOT yet
live-validated (supervised ceremony).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
offbox located each toggled app's recovery unit via AppNamespaceRoot→GetAppDrivePath,
which reads the app's LIVE app.yaml HDD_PATH and silently falls back to systemDataPath
when the app isn't deployed → looked on the wrong drive, backed up nothing, reported
ok/0 (DIAG root cause). Now:
- discoverOffboxUnit/offboxCandidateNSRoots scan the durable storage registry
(schedulable non-decommissioned paths ∪ systemDataPath) for backups/primary/<app>,
independent of deploy state; newest-by-manifest-CreatedAt wins on drive churn.
- RunOffboxBackup: runOffboxInternal returns (backedUp, missing, err); 0-of-N toggled →
hard error + operator alert; partial → ok + new OffboxTarget.LastWarning (shown on
/backups, preserved across config edit).
- AppNamespaceRoot + primary WRITE paths unchanged.
- 6 non-hollow tests (A-E + edge) + both companion red-proofs run (reverted).
- NOT yet live-validated against the Storage Box (spike creds torn down).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
PushResponse.ConfigVersion from the report ACK; ConfigRefresher reconciles vs.
the last-applied version (settings.applied_config_version) and on a change calls
bootstrap.RefreshConfig (re-pull controller.yaml + re-merge local_api) then
GracefulSelfRestart. First-run records baseline (no restart); unchanged = no-op
(no storm); failed pull keeps config + retries. Companion to hub v0.26.0.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
Encrypted restic repo over SFTP for the app-data tier (the off-site 3-2-1 leg). A dead
NAS fails fast via -oConnectTimeout (spike Q8), never hangs the runner; secrets are 0600
files (ride DR via PBS whole-CT); init-if-absent, retention forget --prune, restore,
single-flight, per-app toggle + UI. restic re-added to the image.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
Controller-side of NAS network storage, proxying to agent A1 /netstorage/*. Distinct
'network' storage kind (no drive lifecycle); add/list/remove + per-share health UI;
unreachable NAS is a recoverable warning, never the drive missing/stop cascade; SMB
creds pass through to the agent, never persisted.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HxLA1mZurFq9kt8hneFeCs
Drives are visible in-guest only at the STABLE /mnt/felhom-drives/<name>; the
registered path + HDD_PATH + FileBrowser source repoint there while agent calls
map back to raw /mnt/<name> (agentWhere). Enroll binds-under-parent before
register. Drive-absent GATE (planDriveGates + 30s driveGateLoop) stops/blocks
apps when a drive vanishes and auto-restarts on return; start-gate refuses start
when the drive is absent. H1 endpoints (disconnect/reconnect/restart-apps) routed
onto host-side ops. Non-hollow tests + companions.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A1: AutoDiscoverStoragePaths no longer bails on a non-empty registry;
registers only deployed-app paths missing from the registry. Never
mutates/removes existing entries, never re-adds or reactivates a path
present in ANY state (incl. Decommissioned), never flips IsDefault.
A2: InferStorageLabel maps base==felhom-data namespace dir to
'Belső SSD (rendszer)' to disambiguate the internal system volume.
Table-driven tests incl. a companion that fails without the
skip-by-presence guard.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ListDumpFiles ran ValidateDump (line-by-line scan) for every dump on every ~5-min
RefreshCache cycle — wasted I/O+CPU on large customer dumps. ListDumpFiles now takes
an optional cached(name,size,mod) lookup; on a (size+modtime) match it reuses the
prior result and skips ValidateDump. settings.DBValidationCache gains Size+ModTime;
listAllDumpFiles builds the lookup from the persisted cache and writes back only fresh
validations (cache miss), so an unchanged dump triggers neither a re-validation nor a
settings.json write each cycle. nil cached = legacy validate-always (back-compat).
Tests: cache-hit skips validate (sentinel), cache-miss validates, nil validates.
Part A of the UI-fixes/storage-spike spec.
A1: enrichHostStorageTargets sorts /api/host-metrics storage_targets
server-side and attaches friendly Hungarian labels + purpose, fixing the
#host-storage-bars reorder-on-poll bug. Display labels only — PVE storage
ids are never renamed.
A2: new GET/POST /stacks/{name}/backup Tier-2 config panel; the "2. mentés"
Beállítás button is repointed there from the dead-end deploy page. Customer
can pin a target drive or disable Tier 2; preference is preserved across the
runner's status writes. Always visible (single-SSD + non-HDD apps included).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Deactivated drives (Schedulable=false) now treated like disconnected for
Tier2 backups. New IsStoragePathSchedulable() checks active+connected+not
decommissioned. UI shows yellow "Cél meghajtó inaktív" badge, scheduler
skips silently with WARN log.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Previously, removing a storage drive from the controller only marked it as
disconnected if the StoragePath entry still existed with Disconnected:true.
Drives removed entirely from storage_paths were invisible to the check,
causing Tier2 backup UI to show green "Sikeres" and scheduler to attempt
backups to a no-longer-managed destination.
New IsStoragePathKnown() method covers both cases. UI shows yellow
"Cél meghajtó leválasztva" and scheduler skips silently.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add structured operational logging at INFO, WARN, and ERROR levels to
every controller module. Standardize custom prefixes ([GEO], [SCHED],
[SYNC]) to use [INFO/WARN/ERROR] [module] format. Fix misleveled logs
(WARN->ERROR for data loss scenarios, WARN->INFO for routine operations).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add detailed [DEBUG] logging to every controller module when
logging.level is set to "debug". Each module with stateful debug
uses SetDebug(bool) wired from main.go. Covers stacks, backup,
cloudflare, integrations, system, monitor, settings, scheduler,
web handlers, storage, metrics, API, selfupdate, and assets.
Also includes the app export/import (.fab bundles) feature from
v0.32.0 and its debug page integration.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Generic integration system for connecting deployed apps via toggle UI.
First handlers: OnlyOffice→FileBrowser (config.yaml patch) and
OnlyOffice→Nextcloud (occ CLI). Lifecycle hooks auto-suspend on
stop and re-apply on start.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
admin