7cba0bfca7f9fe6dda8669bcc466b63924b0e40f
91 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
7cba0bfca7 |
v0.278.0: a hold left by an earlier install no longer holds the new one (R-704)
gates / gates (push) Successful in 26s
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS |
||
|
|
6479933e8e |
v0.277.0: kept data loads from the off-site copy too (R-691 (2))
gates / gates (push) Successful in 25s
Use my kept data / Load consider the off-site snapshot when it is newer than every local copy or the only one; the unit is downloaded alone, judged (drive, data, recorded data version) and only then restored. The page names the copy and its date. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS |
||
|
|
b6810f14ff |
v0.275.0: a backup's data and its version travel together (R-696, 07 §6.6, D4 option A); R-695, R-691, R-694
gates / gates (push) Successful in 23s
The unit's data files are stamped with the versions that wrote them; the capture keeps the definition the data belongs to; a restore never starts data under another version's definition (unit restores refuse a mismatch; the off-site restore writes the snapshot's definition); every tier's time is its data's; the conversion-copy release needs a dump on the new engine. File-browser sync single-flight + no empty kept folder (R-695); the kept view joins the folder's owning group, language switch resyncs (R-691); a restore-generated login is not shown as the password (R-694). Red-proofs in felhom.eu/documentation/audits/version-travel-2026-09-26/. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS |
||
|
|
43e99d160c |
kept data: the choice at reinstall, the list, the read-only view, the load (09 decision 36); R-690 fixed
gates / gates (push) Successful in 26s
An install over an app's kept drive folder (appdata/<app> non-empty) asks the household: "use my kept data" (a load from the newest copy of THIS drive's install, own unit or second-drive mirror, then the template's after_load) or "start fresh" (the folder is renamed into <drive>/kept/<app>/<date>/ with the removed app's unit; nothing deleted). The install API answers 409 kept_data_choice until one is chosen; DeployStack refuses too. New page Megorzott adatok / Kept data (/kept-data): Load / Look / Delete (typed confirmation, the only deletion of kept data). FileBrowser gets a read-only source. The drive-full warning names the kept folders. <drive>/kept is protected and outside every backup leg. R-690: the removed-app restore (R-487) never found a unit on a DATA drive — it asked GetStackComposePath (true for every catalog app) and restored nextcloud with no env. Now isStackDeployed; pinned with a production-shaped provider. Red-proofs: audits/night-2026-09-26/E/redproofs/. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS |
||
|
|
7162f489c9 |
README + REUSE: the PostgreSQL conversion (v0.273.0)
gates / gates (push) Successful in 25s
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS |
||
|
|
5b1b191ffc |
v0.269.1: an installed app keeps the image digest it runs until a guarded Update moves it
gates / gates (push) Successful in 26s
Found live on 9202 (night 2026-09-24 Part B): the sync rendered the ladder's newest tested digest into a RUNNING app's compose, so the next restart would pull a new image with no backup and no undo. stacks.CarryDigests keeps the running digest for an installed app; a fresh install still takes the tested digest. Red-proofed. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS |
||
|
|
3c6b49b31c |
controller v0.269.0: whole restore from the second drive; crash loops stopped; exact image digests; steps judged by their own .felhom.yml (decisions 26-28, R-661 R-666 R-667 R-668 R-664 R-665 R-662, 09 6.4 part 6)
gates / gates (push) Successful in 27s
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS |
||
|
|
206b0357d1 |
controller v0.268.0: the undo finds volumes by definition; a held app names only a whole copy; one press = one tested step (R-658, R-659, R-660, R-651; 09 §6.4 part 5)
gates / gates (push) Successful in 27s
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS |
||
|
|
8fc2b4a1a9 |
v0.263.0: a failed update puts the app back by itself (09 decision 15, R-637)
gates / gates (push) Successful in 26s
The guarded update gains a folder copy of the app's named volumes, taken after the pull where the app stops anyway (decision 19, chosen by the 2026-09-23 bake-off). On a failed health check the box undoes: every copy validated by its finished-marker first, volumes refilled, definition and pin from the job's own pre-update copies, the old version checked with the OLD .felhom.yml probe. It holds only if the undo fails, and the hold sentence says so and what state the data is in. Bind-mounted folders are never touched. - R-637 built; R-638/R-640/R-641 do not arise with a folder copy; R-639 (pre-update copies incl. .felhom.yml kept until the undo is over). - journal phases copying/undoing with power-cut recovery. - app.yaml last_update_undone + one line on the app page (hu/en). - R-642: start/restart never answer "completed". - Removal deletes kept undo copies. MinAgent unchanged (0.131.0). Nine red-proofs in REPORT.md. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS |
||
|
|
811f75736e |
v0.261.0 — the controller no longer swaps itself out from under an app update (R-608, R-609)
gates / gates (push) Successful in 23s
The controller self-updates daily at 04:30 by default, and after any hub report once a floor sits above the box. That swap restarts the controller container. The window proposed for automatic app updates is 02:30-05:00. It contains 04:30. R-608 — a two-way lock, wired in main.go (stacks never imports selfupdate): - stacks.Manager.AnyUpdating() -> Updater.SetAppUpdatingCheck, consulted in the same three places as the existing backupRunning gate. - Updater.IsUpdateRunning -> Manager.SetSelfUpdatingCheck; UpdatePreflight refuses `self_updating`. - MEASURED: the gap was narrower than assumed. The update's `backing-up` phase already takes the backup single-flight, so that one phase was covered. The other six were not, and `starting`/`verifying` are where data may have moved. - The lock must NOT latch: a held app does not block the controller's own updates, including the release that might fix the hold. R-609 — the 409 carries `data.reason`, additively. transient (busy, updating, deploying, migrating, self_updating) vs terminal (held, downgrade). Found while writing the test: the router refuses a HELD app on its own line before the preflight, so `held` would have been the one reason missing. Five red-proofs, each seen to fail. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS |
||
|
|
d0d431b42b |
Correct a stale count repeated in four places: 10 floating pins, not 23
gates / gates (push) Successful in 25s
Recounted at catalog 18a6d2d8: 66 unique pins — 48 full X.Y.Z, 6 two-part lines, 4 major lines (10 float), 8 exact versions wearing a variant suffix. The '23' carried since v0.233.0 matches no definition the catalog supports. Definition written down beside the number so it can be rechecked. Also: CONTEXT said the fleet floor was 0.257.0; the hub says 0.259.0. Comment and doc only — no behaviour change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS |
||
|
|
8f8a64cad7 |
v0.260.0 — a box ahead of the catalog reads „Naprakész", and the pin never moves backwards (R-524)
gates / gates (push) Successful in 24s
MEASURED 2026-09-15 (BIGNIGHT Phase 6): privatebin updated 2.0.5 -> 2.0.6, catalog reverted to 2.0.5, and the box read „Frissítés elérhető — ma" over an Update that would have moved the pin BACKWARDS onto a possibly-migrated datadir. - stacks.CatalogOrder: the comparison gains a fourth answer (Ahead) and moves out of web, so the badge and UpdatePreflight cannot drift apart. - The badge: ahead reads „Naprakész"/"Up to date", tag-ok, with a title saying why. - The refusal: UpdatePreflight returns `downgrade` (409), born as a bundle key; the API now renders update refusals through errText so it reaches English households. - Ahead is narrow: every differing service must be orderable AND newer, else Behind. - Ordering is util.Version.Compare behind a tag normaliser — no second comparator. - Three red-proofs, each seen to fail. R-589 was already fixed in v0.258.0; only its register row was stale. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS |
||
|
|
60f0a86bd4 |
v0.257.0: the app catalog can speak English (R-560, slice 5 Part A)
gates / gates (push) Successful in 25s
The READ PATH for a second language in `.felhom.yml`. An `i18n: {en: …}` sibling
block inside the same file; `Metadata.For(lang)` merges it FIELD BY FIELD over the
Hungarian, so a missing or blank English field shows the Hungarian one and a
half-translated app is a legal, shippable state.
`For("hu")` is the parsed struct with `I18n` cleared and nothing else — measured
against all 53 real catalog files, copied into `internal/stacks/testdata/catalog/`.
Lists replace whole; every other list is matched by its own key, never by position.
`For` never writes through the receiver: the metadata is the stack manager's, shared
by concurrent requests, and an in-place merge would leak one household's language
into another household's page.
Pages reach catalog copy only through `LocalizeStacks`/`LocalizeStackPtr`/`MetaFor`,
and `TestNoDirectMetaCopyReadOnPages` keeps a named, reasoned allow-list of every
direct `.Meta.<copy>` read in `internal/web` so the NEXT page to read one fails the
suite instead of quietly rendering Hungarian to an English household.
Eight red-proofs. Two of them convicted a hollow TEST rather than the code: a struct
copy shares its slices' backing arrays, so the obvious DeepEqual mutation check
passed a deliberately broken merge; and a one-entry fixture cannot tell key matching
from position matching. Both rewritten, both then seen to fail.
MinAgent: 0.131.0 (unchanged). Older controllers are unaffected — `LoadMetadata`
uses non-strict `yaml.Unmarshal`, so a pre-0.257.0 box drops the whole block.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
|
||
|
|
c00fed6db3 |
R-553: four decisions stop reading their own Hungarian words (sites 1-4)
Every Hungarian sentence is byte-identical; each decision now reads a signal set where the message is
made. util.KindErrorf builds the same bytes fmt.Errorf did while carrying a sentinel for errors.Is.
- Deploy status (api/router.go): deployStatusFor() by kind — stacks.ErrAlreadyDeployed (409),
ErrRequiredField / ErrPathMissing / ErrNotEnoughMemory (400). The „kötelező" / „memória" /
"does not exist" / "already deployed" text chain is gone.
- Off-site failure class (backup/offbox.go): ErrOffsiteQuota replaces the „tárhelykeretet" match. The
restic/ssh signatures stay text matches on purpose — that output is not ours and is not translated.
- Alert placement (web/alerts.go): monitor.HealthReport carries WarningKinds parallel to Warnings;
the "not on a separate drive" warning is inline by KIND. The hub report is untouched (builder.go
copies Status/Issues/Warnings only) — pinned by a wire test.
- Stale off-site note (web/handlers.go): settings LastWarningKind + backup.OffboxWarnNoAppsSelected.
The text test survives ONLY for kind == "" (a box whose last run predates 0.251.0) and is removed
when R-570 closes; slice 2 must not translate that producer before then.
Tests (all red-proofed by restoring the pre-fix predicate — see the audit's redproofs.txt):
TestR553_Deploy_DecisionSurvivesWordingChange, TestR553_DeployHandlerUsesTheKind,
TestR553_DeployProducersCarryKindAndKeepTheirWords (through the real DeployStack),
TestR553_OffsiteQuota_{Decision,HeadLine}SurvivesWordingChange, TestR553_OffboxRunRecordsTheKind,
TestR553_StorageWarningsCarryKindsAndKeepTheirWords, TestR553_DiskWarningPlacementSurvivesWordingChange,
TestR553_HubReportWarningsAreUnchangedOnTheWire, TestR553_StaleNote*, TestR553_WarningKindIsPersistedAndCopied.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
|
||
|
|
ff68b0b433 |
v0.248.0: i18n slice 1 release A — apps and settings in English, Hungarian byte-identical (R-556)
gates / gates (push) Successful in 20s
Ten pages converted against fixtures captured from unconverted templates (
|
||
|
|
612c417024 |
v0.247.0: i18n spike — the dashboard can speak English, Hungarian byte-identical
gates / gates (push) Successful in 19s
Message bundles (internal/i18n) expanded into templates before parsing, one template set per language. Launcher, /backups, /apps/<slug> and the layout converted; household language setting, POST /settings/language, ?lang= override, report field. Parity test against fixtures captured from unconverted templates; copy gates read templates expanded; new i18n_missing_gate. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS |
||
|
|
3e813307cc |
controller v0.241.0: a bind-data app leans on off-site before its own unit; the hold names what the copy holds (R-479)
gates / gates (push) Successful in 13s
Operator ruling 2026-09-13. An app with classified binds walks second drive -> off-site -> own unit (its unit holds no files); volume apps keep 2 -> 1 -> 3. RestoreHold.CopyHolds records what the chosen copy holds and the sentence ends with it; older holds keep their tier-only sentence. Tests on both halves; red-proof: a layout-blind order fails the bind case. |
||
|
|
bdcbd50b42 |
controller v0.240.0: seven defects from the any-tier proof and the first nightly rotation
gates / gates (push) Successful in 13s
R-486 (P1): removing an app with its backups KEPT keeps its Tier-2 record, so the second-drive restore is no longer refused over an intact mirror. R-484: postgis/pgvector/timescaledb images are Postgres (logical dumps). R-485: the backup card sizes the recovery unit and the mirror(s). R-480: a held update's sentence leaves the card once the hold is lifted. R-477: the update's off-site lookup is one snapshots call, no stats. R-478: a copy older than this install's deploy does not count. R-474: "delete backups" deletes the unit, the mirror(s) and the prefs. Tests and red-proofs per row; evidence in felhom.eu documentation/audits/v0240-2026-09-13/ and nightly-2026-09-13-adventurelog/. |
||
|
|
b93c1543da |
controller v0.239.0: any backup tier lets an app update (R-475)
gates / gates (push) Successful in 14s
Operator ruling 2026-09-13. The update precondition walks Tier 2, Tier 1 (own recovery unit, "helyi") and Tier 3 (off-site, 15 s bound; unreachable counts as absent with a WARN) and leans on the first FRESH copy; the backup_max_age rule applies to whichever tier is chosen. No copy anywhere: back up first. Refused only when nothing exists and no backup can be taken. RunAppBackupNow tolerates a Tier-2 failure (WARN) and marks the captured unit proven current. The hold names the tier (második meghajtó / saját meghajtó / távoli mentés) and the date; pre-v0.239.0 holds keep their text. A successful off-site restore now lifts an update hold. The backups page still uses Tier2UnitRestorePoint unchanged. Scenarios G-M tested; red-proofs M, L, the tail and the off-site clear in felhom.eu documentation/audits/rulings-r472-r475-2026-09-13/. |
||
|
|
cbcca03061 |
v0.238.1: the nightly backup leaves an app alone WHILE it is being updated, not only once it is held (slice 4 follow-up)
gates / gates (push) Successful in 13s
Found live in v0.238.0 Scenario F on demo-hp: during an update's 5-minute health wait the app is not yet held, and the periodic recovery-unit capture at 10:17:09 wrote the never-started definition (alpine:3.20) into its PRIMARY unit, 53 s before the hold landed. The Tier-2 mirror the hold names survived only because Tier 2 runs daily; a nightly Tier 2 inside a verify window would have mirrored the broken definition over the copy the customer is told to restore from. backup.Manager.isHeld — consulted by the capture sweep, the Tier-2 run and the volume dump — is now also true while a guarded update is moving the app, via SetUpdatingCheck wired in main.go to stacks.Manager.IsUpdating. Test with positive control + red-proof; wiring pinned. Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS |
||
|
|
0d402f711d |
v0.237.0: the Update button takes a backup first, and tells the truth (update arc slice 4 — R-448, R-443, R-439)
gates / gates (push) Successful in 13s
POST /api/stacks/{name}/update is now a guarded job answering 202:
cheap refusals (hold — R-439, busy, migration, deploying, memory via the
deploy's own memoryVerdict, a fixed 2 GB disk floor, and no restorable
Tier-2 copy) → backup-first when the proven copy is older than
update.backup_max_age (24h) → safety dump BEFORE the pin moves → pin →
pull (failure puts the pin back) → up → health (.felhom.yml check or 60 s
settle, update.health_timeout 5m). Not healthy → the app is stopped and
HELD (RestoreHold reason update_failed, same store and gate as R-379) and
the page names the backup to restore from; the pin stays. Success is only
ever update_phase=done after health (R-443). UpdateStack is deleted.
The restorable-unit predicate is EXTRACTED to backup.Tier2UnitRestorePoint
and shared with the backups page (row pinned unchanged). The copy is aged
by the last successful Tier-2 copy, not the manifest created_at — measured
on demo-hp that created_at moves only on definition changes.
Crash safety: update-journal.json before each phase; RecoverUpdates before
the boot sweep, ResumeInterruptedUpdates after the guards are wired.
Three unattended start paths ignored a hold and now honour it: the
drive-return gate (restart + boot recreate) and the nightly volume dump.
The nightly capture and Tier-2 run skip held apps so the restore point
survives. No automatic rollback — measured per-app; route back = restore.
Tests A–H across stacks/backup/api/web/cmd; six red-proofs seen to fail.
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
|
||
|
|
42a73e667a |
v0.236.0: "delete my data too" deletes the data, or says that it could not (R-442)
gates / gates (push) Successful in 13s
Removal resolves the drive from the app's own app.yaml HDD_PATH (the 07 ~L437 rule), never the global cfg.Paths.HDDPath which no box sets. A data removal that cannot be resolved, or whose drive is absent, is refused with a typed RemoveRefusedError -> 409 + exact Hungarian sentence, before compose down, and the app is kept. SSD app -> hdd_paths_removed: [] never null; missing folders stated; backup-path refusals reach the response. 15 tests, two red-proofs run (pre-fix fallback -> C fails with err=nil and the handler 200s; "no drive refuses" -> D fails). Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS |
||
|
|
8a0e0a59ad |
v0.235.0: freeze the version, keep the fixes flowing (operator ruling 2026-09-06)
gates / gates (push) Successful in 12s
Slice 3. R-447 was BLOCKED because R-438 established that RestartStack's use of up -d to pick up template changes was CHOSEN and written down in its own comment. The operator ruled Option 1, and this implements it. The rule: while the catalog offers the same version you run, its fixes flow to you; the moment it moves to a newer version you are frozen until you update. NOTHING was added to any of the thirteen compose up -d call sites. Most of them are repairs - the boot reconciler, the drive-return gate, the app-stop guard - and a repair path that refuses to repair leaves a customer's app down, which is worse than the problem. They are made safe by removing the reason. app.yaml gains pinned_images: what the app is SUPPOSED to run. It is NOT installed_images, which is an observation; letting a reading become a deployment is the R-166 category error one field over. Four writers, each also storing the exact definition as applied-compose.yml. UpdateStack advances the pin and re-renders BEFORE the pull, because pull and up -d act on the file on disk, and a pin set afterwards would pull the frozen version and report success. The syncer renders instead of copying, through one nil-safe seam. Catalog images equal the pin -> verbatim, so fixes and self-healing both survive; they differ -> the WHOLE stored definition, never a substitution of refs into a newer template (wger 2.6 needs a DB config the older template cannot supply). This is deliberately not 'skip deployed apps', which was option B and was rejected. AdoptPins runs once at boot after the backfill, files only, and skips loudly rather than inventing a pin. syncer.Start() moved to after it: the initial sync would otherwise run while every app was unpinned and overwrite a deployed app's version once per boot. THE BADGE HAD TO CHANGE OR SLICE 2 WOULD HAVE INVERTED SILENTLY. TemplateImages reads the LIVE compose file, which is now the frozen one, so the comparison would have answered Naprakesz on exactly the apps that are behind - with every test green, because the new field has the same type. It now reads CatalogImages. +16 tests (1729 -> 1745), 28 packages green. Three red-proofs run and reverted. A test also caught the syncer writing an empty compose file over a live app. |
||
|
|
38d28b5b62 |
v0.234.0: seed installed_images at startup, so the label appears on an app nobody touched
gates / gates (push) Successful in 13s
The operator looked at demo-felhom the morning after v0.233.0 and found OpenGist - up 15 hours, running exactly the catalog pin - showing no badge at all. v0.233.0 wrote the record only from the four bring-up paths, so an app nobody restarts carried no record indefinitely. On a quiet box that is every app, which is the box we most want to see. The known limitation WAS the feature not working. BackfillInstalledImages runs once at startup, beside BackfillDesiredState and before the boot reconciler. It READS containers: starts nothing, restarts nothing, writes no compose file. It never overwrites an existing record. And it REFUSES to seed a partial observation, which is why this is not a three-line loop: the badge reads a service-count mismatch as BEHIND, so seeding a degraded app from what is visible would render 'Frissites elerheto' over an app that is perfectly current. The bring-up paths may write a partial because they follow a successful up -d where a gap is real news; a backfill meets any state. Same data, two writers, two admission rules - deliberately. Also fixes a calendar bomb of mine: the render test hardcoded catalog_since and the string '46 napja', but the render path reads time.Now(), so it was green on the day it was written and red the next morning. Now derived. Filed as R-457 with six other candidate files named as unchecked, not accused. +5 tests (1724 -> 1729), 28 packages green. Red-proof of the partial guard run and reverted; the wiring and its ORDER pinned by an AST walk. |
||
|
|
8025304acc |
v0.233.0: record what each compose service actually installed, and badge whether it is current
gates / gates (push) Successful in 12s
Update arc slices 1 and 2. NEITHER CHANGES ANY BEHAVIOUR — no new endpoint, no auto-update, the three lifecycle buttons byte-identical. Slice 1 — app.yaml gains installed_images, keyed by compose SERVICE name, each entry carrying ref + repo digest + first-seen timestamp. Written by Manager.recordInstalledImages after a successful compose up from StartStack, RestartStack, UpdateStack and runComposeDeploy. Read from the CONTAINER, never from docker-compose.yml: the syncer overwrites a deployed app's compose on a 15-minute cycle and the two disagreed for 25 minutes in the spike's own measurement. A failed write NEVER refuses the action - the deliberate opposite of SetDesiredState, because this is an observation and that is an intent. Not called from StartStackServices (the R-47 DB-only window). Its own docker seam with a context and a 30s timeout, which neither existing exec helper has. Slice 2 — .felhom.yml gains optional catalog_since; web.updateBadge compares the recorded ref per service against what the current template pins and returns a *MetaBadge through the EXISTING meta_badge partial. No new markup, no new CSS. NO RECORD RENDERS NOTHING: absent means unknown and never means current. No version number reaches the customer and no registry is queried. Known limitation, filed not hidden: 23 catalog pins float, so those apps can read Naprakesz when the image behind the tag has moved. +17 tests (1707 -> 1724), 28 packages green. Wiring proven through a real RestartStack plus an AST walk of the four call sites. Three companion red-proofs run and reverted. |
||
|
|
a4444088ad |
R-404: the golden NOTICE, in the repo where the debt is created - NOT A RELEASE
gates / gates (push) Successful in 12s
No version heading on purpose. No Go code, no image, no version bump; giving this one would create the exact golden debt the change is about. Until today this repo - where a release actually happens - had NO golden-currency check at all, while felhom.eu ran one on every push including documents-only ones that can neither create the debt nor clear it. The person who could act heard nothing; the person who could not act was blocked, thirteen --no-verify uses' worth. golden_notice.py is ADVISORY IN EVERY CASE, and that is the only correct behaviour rather than timidity: at the moment a release is committed the golden legitimately does not exist yet, so blocking there would refuse the commit that STARTS the process - and blocking later is the mistake being undone. NO SECOND IMPLEMENTATION: it IMPORTS felhom.eu/scripts/golden_currency_gate.py and calls that gate's own released_versions()/newest_baked(), so it is the same comparison read in the other direction. Cross-repo shape copied from instructions_gate.py; never a copy of the script, because a copy recreates the drift these gates exist to detect. An absent sibling clone is INCONCLUSIVE and silent about currency - it never guesses. controller_gates.py GAINED A FIFTH `blocking` FIELD. It could not express a reporting-only gate at all before: every registered gate's non-zero exit failed the run, so the only way to add a notice was to give it the power to refuse a push. The capability was added rather than the notice compromised (R-420). False for exactly one gate, and test_golden_notice.py asserts it stays one. Tests N1-N4 with a positive control that every other gate is still blocking. RED-PROOF RUN: making the debt branch return 1 fails N1 - in production that would refuse the commit that starts a release. |
||
|
|
e43b5ec07d |
v0.231.0 - the box proves its own off-site copy still holds something (R-87)
gates / gates (push) Successful in 11s
R-87 re-scoped by its own spike and built as Option C. MinAgent 0.129.0 unchanged. THE QUESTION NOTHING ASKED. The weekly check proves the stored bytes are the bytes we stored; it cannot tell us we stored the WRONG thing. A hollow recovery unit backs up cleanly, checks cleanly at 100 percent depth, restores cleanly and gives the customer nothing back - measured on demo-hp 2026-08-31, 120082104 B to 7036 B in one nightly run recorded as a success (R-403). No tier and no cadence asked it. Now offsite-proof does, nightly, on one app. IT DOES NOT prove a restore puts data back into a running app. That stays drill work and 07 section 8 matrix row 4 is NOT moved. THE ACCEPTANCE RULE HAS TWO PARTS AND THE OBVIOUS ONE IS A TRAP. "Check the unit against its own packing list" PASSES a hollow unit, because a hollow unit declares nothing. So: (1) everything declared is present, AND (2) the manifest declares what the app is supposed to have. Part 2 is the whole value. RED-PROOFED: the naive rule makes the hollow-unit test read verdict "pass". THE EXPECTATION COMES FROM INSIDE THE UNIT, never the live box - the snapshot may predate the app's shape, and GetDockerVolumes describes the running app. Database half is DBServiceNames, the same discriminator RestoreFromRecoveryUnit uses. Volume half is ParseComposeNamedVolumes as an EXISTENCE check, not a name match: tars are <project>_<volume>.tar and ResolveDockerVolumeNames derives the project from the compose file's parent dir, which inside a unit is the literal string "compose". Measured on all eight real units on demo-hp the counts match exactly and the naming held every time - but "held on eight" is not "derivable" (R-355). Half a rule that is true beats a whole rule that is invented. THREE OUTCOMES: pass, fail (readable and empty), cannot judge. An app that legitimately has neither a database nor volumes PASSES. RED-PROOFED: alarming on any empty unit makes that test read verdict "fail". IT NEVER WRITES TO THE REPOSITORY and that is asserted on the ARGV as a non-effect: --no-lock, no unlockStale, and m.runner() rather than resticStep so the unlock --remove-all escalation is unreachable. RED-PROOFED: routing it the customer path's way makes the test fail on "unlock" appearing in the argv. IT TAKES acquireRunning ITSELF and skips rather than waits, because RestoreOffboxScratch does not take it (R-408) while offbox_integrity.go states that invariant as universal. DUE-NESS IS PER SNAPSHOT (R-86's model), never per clock. RED-PROOFED: recording a timestamp fails the stored-value test AND breaks the rotation - night 2 re-picks night 1's app. ITS SCRATCH IS A SEPARATE ROOT (backups/offsite-proof) and that is a safety decision, not tidiness: the job deletes its copy on every path, and sharing backups/offsite-restore/<app> would mean a nightly background job deleting the verification copy a CUSTOMER is looking at. It is also invisible to placement, so a proof copy can never be pushed into a live app. SHARED RATHER THAN FORKED: offboxScratchDirIn parameterises the scratch resolver on its ROOT builder, and unitOnlyHeadroom extracts the free-space gate, so the customer path and the proof refuse at the same floor with the same Hungarian sentence. RestoreOffboxScratch's behaviour is unchanged. NEW EVENT offsite_proof_empty, severity error, operator-only - deliberately NOT backup_integrity_failed, whose hub template says the store is DAMAGED. Here the store is sound and the content is absent: different cause, different action. The hub half shipped FIRST, in felhom.eu 1aeaa30 (hub v0.110.0, live and verified), because an unallowlisted type is 400'd and vanishes. 33 new tests, all groups green; full suite 1689 tests, 28 packages, rc=0. All 13 controller gates OK. Five red-proofs run and recorded in REPORT.md. A golden carrying 0.231.0 is OWED - the fleet is on 0.230.0. Viktor's call (R-242). |
||
|
|
8aa95b5831 |
docs(v0.229.0): R-102 + R-103 — CHANGELOG, CONTEXT rulings, README, REUSE, REPORT
gates / gates (push) Failing after 13s
CHANGELOG v0.229.0. CONTEXT records three rulings: the source moves and the destination does not; two predicates and not one wider one (R-356's cost restated); and a destructive operation reached from a non-destructive surface carries the difference in the CONFIRM, not the label. README documents the new action and route and corrects the coverage note to the measured count. REUSE maps the four unit-directory-relative primitives and the new manager methods, with the traps. REPORT covers the live drill on demo-hp (docmost, class B, primary unit moved aside — 3 volumes of 3 and 1 database of 1 in 28.65 s, accented filename byte-identical verified as hex, the app reading its own row over TCP; Scenario D with the guest app.yaml also aside, secrets recovered=2/2), the settled count (A=7 B=45 C=1, and why the earlier 9/43/1 was wrong), the five named red-proofs, and seven observations including R-403 and a process error of mine that changed the box and is now in memory. |
||
|
|
3c49dc8ea4 |
v0.228.0 — the off-site check reads the data; the debug page stops lying (R-399 + R-400)
gates / gates (push) Successful in 12s
R-399: monitoring.integrity.read_data_subset defaults to 100%. A pack damaged without changing its size made plain `restic check` report "no errors were found" on demo-hp 2026-08-30; every read-data form caught it. Cost on that 134 MB store: 35.0s structure vs 39.2s at 100%. "off" (any case) is the off token; empty means not-configured, therefore the default; a malformed value falls back to the DEFAULT, never to structure. A completed check over 5 minutes logs a WARN naming the duration, the depth and R-401 — operator log only, no hub event, no depth change. The depth is now recorded with the verdict (LastIntegrityDepth; empty = NOT RECORDED, never "structure"). R-400: 24 debug-page references, 17 dispatched, 7 dead — three of which fetched on page LOAD, so those panels were permanently blank. backup/crossdrive implemented; backup/infra, hub/infra-push, dr/infra-status, storage/watchdog-status and both storage/simulate-* deleted with their panels and JavaScript. scripts/debug_route_gate.py fails in both directions and is registered after the seven were resolved. 18 referenced, 18 dispatched, none orphaned. Corrections: the dead-field warning in report/types.go said the controller runs no integrity check and the notifiers are called from nowhere — both false since v0.227.0. controller.yaml.example gains its missing integrity: block. integrityCheckTimeout's "ships OFF" comment rewritten. |
||
|
|
45770f2282 |
v0.227.1: the damage classifier matched restic's ordinary progress output
gates / gates (push) Successful in 11s
A patch and not a rebuilt 0.227.0: that tag was already running on demo-hp, and re-pushing changed bytes under a live tag is the :latest hazard with extra steps. looksLikeRepositoryDamage matched bare "pack ", "tree ", "snapshot ", "blob ". A HEALTHY restic check prints "check all packs" and "check snapshots, trees and blobs" -- so any check that failed for a NON-damage reason, a connection dropped mid-run for instance, would have been classified as a corrupted repository and told the customer their backups may be damaged. That is the false alarm that teaches an operator to ignore the true one. Caught by the NEGATIVE control, built from the real bytes of a real passing check on demo-hp. The spec made the negative control mandatory and this is what it was for: a control that has only ever seen the failing case proves nothing. Signatures are now phrases from restic's own error wording. Also in this commit: CONTEXT.md records the three rulings (take the flag and skip, due-ness not a weekday, publish on OffboxReportStatus not the R-331 dead fields) plus the measurement a future session would otherwise assume wrongly -- THE STRUCTURE CHECK DOES NOT CATCH SILENT CORRUPTION. README documents the job, the route and the config, and corrects a line that listed four debug backup routes when only two exist. REUSE gains three rows, including one that records R-398 was my own mistake so nobody re-files it. |
||
|
|
b8af72764d |
R-353/R-357/R-358/R-360: the restore tells the truth (v0.226.0)
gates / gates (push) Successful in 11s
Four defects on the restore surface, all proven on demo-hp during the 2026-08-21 backup-truth drill, all still in shipped code. They share one acceptance idea: a restore surface must state what it actually did, and must refuse what it cannot do. VERSION NOTE. The task specifying this targeted v0.224.0 against baseline |
||
|
|
e5eee501b5 |
R-331 (controller half): forward stats_known so the hub can tell empty from unmeasured (v0.225.0)
gates / gates (push) Successful in 12s
The hub's operator Backup card read `Snapshots 0 / Repo Size 0 MB / Integrity Unknown` for EVERY customer, because it rendered the report's `backup` object -- whose snapshot/size/integrity fields have had NO producer since disk-tier restic moved to the host agent (slice 8C). buildBackupReport leaves them zero deliberately and says so. Measured on demo-hp 2026-08-30 while that night's log said `[offbox] backup OK: 8 app(s) backed up, 67 snapshot(s), 2m14s`. The live numbers were always in the report's `offsite` object, which the hub already reads for its Offsite page and its fill/staleness alarms. The hub fix is to render that -- and that made exactly ONE field mandatory that was not being forwarded. snapshot_count:0 means two opposite things: "holds nothing" and "never measured". R-225 measured that confusion inside this repo (a rebuilt box rendered 0 pillanatkep over a store really holding snapshot f3d9cd67), and settings.OffboxTarget.StatsKnown fixed it for the controller's own UI. It was never put on the wire, so the hub was free to make the identical mistake one layer up -- and did. OffboxReportStatus.StatsKnown now carries it, omitempty, so an older controller sends no key and a reader degrades to UNKNOWN, never to EMPTY. Absence is ignorance, not emptiness. The four dead BackupReport fields stay on the wire (historical reports in the hub store must keep parsing) but now carry a warning naming R-331 and pointing at Offsite. TestBackupReport_DeadFieldsStayZero fails the moment a producer appears for one -- the prompt to update the hub card in the SAME change rather than ship a field nothing renders. RED-PROOF: drop `StatsKnown: t.StatsKnown` -> "a MEASURED empty repository reported stats_known=<nil>". Tests assert the JSON the hub sees, not the Go struct: measured-empty and never-measured must differ ON THE WIRE, which is the entire point of the field. Green gate clean: 28 packages, rc 0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LB8FmJaGd2cyjvy6dbEjpM |
||
|
|
92cebb8c95 |
R-330: stop the backup alarming about the apps it is holding down (v0.224.0)
gates / gates (push) Successful in 11s
Measured live on demo-hp 2026-08-30 (controller 0.223.0): the nightly db-dump
and offbox-backup legs stop each stack ~13s to tar its volumes while the
deadapp-check job scans every 30s, so the scan caught whichever stack was
mid-cycle and pushed app_start_failed to the customer. 61 e-mails about apps
that were never broken.
The defect is not a missing mechanism. quiesce/suppress.go solved exactly this
in v0.179.0 and works -- but classifyRunStates read only the quiesce loop's set,
and that loop covers the WHOLE-GUEST backup. The per-app legs stop stacks
through Manager.DumpAppVolumesSafe, which registered with nothing. Two
mechanisms stop apps on purpose; only one told the alarm. Fifth instance of the
"seam built but never wired" class, and the first where the unwired half was a
consumer.
The suppression now rides AppStopGuard, which already brackets every deliberate
stop in the product (Begin before the stop, End after a successful restart) at
all three call sites, and which main.go hands as ONE object to the backup
manager and the exporter. scanDeployedAppRunStates takes the union of both sets.
All three per-app stop paths are covered, not only the reported nightly one.
It cannot latch -- End() runs only on a restart that SUCCEEDED, so unlike the
quiesce loop an open-ended hold is a real hazard here:
1. ReleaseFailed drops the entry IMMEDIATELY on a restart that broke, wired at
every failure path, so the app alarms on the next scan;
2. Begin REPLACES the set (one marker file = one operation);
3. appStopMaxHold (6h) caps a hold nothing released, logged at WARN.
Grace is 180s, deliberately quiesce's own constant and derivation. Suppression
is NOT persisted: after a crash the guard holds nothing and a down app must
alarm. ReleaseFailed keeps the durable crash marker; a test pins that.
Three companion red-proofs, each printing the pre-fix value (REPORT.md section 5):
- drop markStopped from Begin -> "suppressed at stop = map[]"
- drop ReleaseFailed from the dump -> "map[bookstack:true] after a restart that FAILED"
- pass nil instead of appStopGuard -> the AST wiring test fails
The third is load-bearing: the component was never the broken part, so a suite
that only injected it would have been green against the shipped defect.
Green gate clean: go build + go vet + go test ./... -- 28 packages, rc 0.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LB8FmJaGd2cyjvy6dbEjpM
|
||
|
|
f94543ee5c |
v0.217.0: prefill from the app's own backup, where-the-data-goes on deploy, bounded inventory fan-out
gates / gates (push) Successful in 10s
Completes R-351 and ships R-352's visibility half. Gates 11/11 OK, suite 28 packages ok, go vet clean, -race clean on the changed package - all run and read BEFORE this commit. PART 2 SCENARIO A - the deploy page prefills the address and data folder from the app's OWN backup. backup.RecordedUnitForStack scans every readable namespace root (the app is NOT installed in this case, so there is no own drive to ask) and reads manifest.json plus the captured compose/app.yaml. Local file reads only: no network, no restic, no restore. RecordedAddress.Known() requires BOTH halves on purpose - an absent SUBDOMAIN makes the live deploy path substitute the CATALOG default (stacks/deploy.go:88-90), and offering that back as "what your backup says" would be a fabricated fact. The prefill is labelled as coming from the backup and stays editable: a memory, not a lock. PART 1 VISIBILITY (R-352) - the deploy page now states where the app's data will live before the button is pressed. Measured 2026-08-21: 13 of 53 catalogue templates declare a storage field; the other 40 have none and their data goes to the system drive, which no screen said. Metadata.HasDeployField answers "does this app have somewhere to PUT a recorded value?" - for the 40-class a recorded placement is a fact to state, never a value written into a field that does not exist. NO PLACEMENT CHANGED. NOTHING MIGRATED. The rest is a filed specification. PART 4 - measured before theorising, on the live off-site target: snapshots --json 2605 ms once; stats 2697 ms PER APP, sequential, 5 app tags => 2605 + 5*2697 = ~16.1 s, matching the reported ten-to-fifteen seconds. The cause is the shape already on file, so the per-app size calls now run concurrently, BOUNDED TO 4. The bound is the safety property, not the speed one: the repository is a Hetzner Storage Box with a session cap, and a refused size call returns SizeBytes 0 - a silent UNDER-REPORT of the customer's data rather than a visible failure. Peak-in-flight is asserted. OffsiteInventoryList had no test at all before this. TEMPLATE SAFETY - every Restore* key is set UNCONDITIONALLY in the deploy handler, because a template doing index/eq against an undefined key errors at RENDER time: green build, green vet, green suite, 500 on the page. Four render tests, one per branch, because the existing deploy render test only renders AutoFields and never reaches these blocks. RED-PROOFS, mutation asserted applied then reverted to 0: A three template guards dropped (count asserted 3) -> the blank form returned P4 inventorySizeConcurrency = 1 -> "peak in flight was 1", elapsed 282ms = sequential DOCS: CHANGELOG v0.217.0 (MinAgent 0.129.0 unchanged), CONTEXT (the restore's own memory + what is next), controller/README.md (Backup System), REUSE.md (4 new rows), REPORT.md overwritten - the previous REPORT preserved to audits/REPORT-v0.216.0-2026-08-14.md first. NOT fixed here, filed as R-353 and named the next session's first item: a restore whose unit carries no db_dumps and no volume_dumps still reports a bare completion. |
||
|
|
8144a70a72 |
docs(v0.215.0): CHANGELOG, CONTEXT decisions, README feature, REUSE entries
gates / gates (push) Successful in 10s
- CHANGELOG leads with the severity fix and the live warning-vs-warn proof. - CONTEXT records the settled decisions so they are not re-litigated: Hiba is the label for predicted failure (no fourth word); sustain before count and why; the provenance of 64/55/60; phase 2 owns the new SMART attributes because they are a wire change under G-1; phase 1 state is one record per disk, not a series. - README documents the 14-row ladder, the persisted state, the hourly cadence and the five message shapes. - REUSE pins the severity wire contract on PushEvent — the defect's real home, so the next typo'd severity is caught at the table rather than in production — and records priorFor vs cardPriorFor, which differ by one observation and make the chip disagree with the email if mixed up. |
||
|
|
27d1165962 |
v0.208.0 — R-254: the last two secrets leave the page source, plus a gate against a fourth
gates / gates (push) Successful in 17s
Site one. app_info.html rendered {{.InitialCreds.Password}} into a hidden span —
a REAL per-install credential, read live out of the running container, in the
response body of every render. The page now carries the non-secret half plus a
boolean; the value comes from POST /apps/<slug>/initial-credentials/reveal, which
RE-READS the container rather than serving a cached copy (caching it in the
handler would put it back in the body one layer in). no-store, CSRF-covered,
logged as an act. Both buttons go through it. A reveal that cannot read the value
SAYS SO rather than returning an empty string that renders as a blank password.
Site two, established before changing. The hidden input is NOT the defect and was
left alone: it fires only pre-deploy, and README §318 documents why the value must
round-trip — the customer notes the generated secrets down and submitting them
back is what makes the saved value the same one they saw. The defect was the
neighbouring READONLY input, which on an ALREADY-DEPLOYED app rendered the secret
into a page with nothing to submit. Fixed by POST /stacks/<name>/auto-field/reveal,
authorised by requiring a type:secret auto-field of that stack. Both directions
pinned.
The premise that this contradicted a repo rule does not hold: the rule is
CONTEXT.md:2070 'Password fields require explicit input — prevents accidental
empty-password deployments', about EMPTINESS. No line in the repo says 'no silent
auto-fill'.
The gate. scripts/secret_in_markup_gate.py, registered in controller_gates.py,
convicts any template expression that names a secret unless allowlisted with a
reason. Its limits are MEASURED and in its docstring: it catches a launder through
a local variable (the assignment names the secret) but is blind to a secret
arriving under a neutral page-data key — verified both ways. That is the shape of
site two, which this gate would NOT have caught. The runtime body assertion covers
all shapes but only 4 of 27 page templates; the other 23 are R-255, filed rather
than glossed. Two nets, different holes, both named.
Correction to v0.207.0's report: HTML comments do NOT ship in the response body
here — html/template strips them, text/template does not. Measured. A red-proof
planting a secret in a comment therefore correctly does not fail.
|
||
|
|
8dbbc98ff2 |
v0.207.0 — R-249: the retrieval passphrase leaves the page body; R-252/R-253: two refusals learn to say what to do
gates / gates (push) Successful in 18s
R-249. settings_security.html rendered the passphrase into a display:none span behind a Megjelenit button. That toggle stops a browser DRAWING the value and nothing else — the plaintext was in the response body of every render, so a curl of the page returned it. Found by exactly that: it landed in a session transcript while driving the documented rebuild path. The codebase already stated this rule for the recovery code and this page did not follow it (escrow_handlers.go: 'reveal (claim XHR only — R is NEVER templated server-side into HTML)'). The page now carries only HasRetrievalPassword; the value comes from POST /settings/retrieval-password/reveal — CSRF-covered because POST, no-store, and LOGGED as an act, which reading it off the markup never was. The tests assert the RAW RESPONSE BODY. Every test that asked what the customer sees passed while the bytes carried the secret; that is why this survived. Census: the render-then-hide pattern appears twice more — app_info.html (a real per-install app password in a hidden span) and deploy.html. Filed as R-254, NOT fixed here. R-252. A rebuilt box keeps its drives but loses their REGISTRATION. The restore page now states that before the customer presses anything, says the backups and drives are both still there, and links to Tarhely > Meghajtok. Page and resolver ask ONE question — HasRestoreDestination() reads the same GetSchedulableStoragePaths() the scratch resolver reads. R-253. The list promised 'a visszaallitas elobb ujratelepiti' three lines above a refusal that fired BECAUSE the app was not installed. The promise was the wrong half: reconstitution writes to the app's own GetStackHDDPath, which exists only once the CUSTOMER has chosen a drive at deploy time. Auto-reinstalling would mean the product making that choice for them. Copy now says to install first and routes to /stacks/<app>/deploy. Both notices are conditional — a healthy box renders as before, pinned by a test that fails if either becomes unconditional. |
||
|
|
0a9158d53e |
docs for v0.206.0: CHANGELOG, CONTEXT, REUSE, README
gates / gates (push) Successful in 19s
CHANGELOG v0.206.0 with the ruling that reversed the fix, the three changes,
the SEC 7.2 staleness decision, Q7's closed trap, and the two bugs the tests
caught rather than review.
CONTEXT carries the three rules this session established, in the form the next
session needs them:
- a box does not create a repository key while the hub holds a sealed
package for it;
- the fact that answers a question must be kept where the question is asked;
- fix the state, do not remember that it is wrong.
REUSE gains four rows, each carrying the trap rather than just the signature:
the mint guard is a CONJUNCTION and t.Enabled is load-bearing in the derived
predicate; the discriminator ships INERT unless wired in main.go's confirmer
literal; the countdown removes BOTH halves or neither and must be driven by an
injected clock; and the epoch must be synced FIRST and unconditionally or the
falling edge is lost.
README documents the three customer-visible changes and the operator levers.
No version literal was edited: the controller version is ldflags-only.
|
||
|
|
88897a224e |
v0.194.0 — one operator email per backup run, and nothing dropped without a trace (R-182)
gates / gates (push) Successful in 8s
MEASURED, not supposed. On 2026-08-03 nine per-app recovery_unit_capture_failed events reached the hub and TWO operator emails went out. The hub's operator cooldown key is customerID:eventType(+tier) and that event carries `app` but no `tier`, so the key held no app identifier: the first refused app took the hour's slot and every other app's failure was discarded BEFORE anything was written down, leaving no row on any channel. The obvious fix — put `app` in the key — was ruled against: on a full disk it produces one email per app, the volume problem wearing the correctness problem's clothes. internal/backup/runsummary.go: a per-run collector with exactly admissionSet's lifetime, fed by all three write legs, emitting backup_run_failures ONCE at the end and only when something failed. A clean run emits nothing. The per-app event stays and becomes the RECORD — the hub routes it record-only, stored and logged every time, never competing for an email slot. The record and the notification are now different things. Deliberate skips (disconnected, decommissioned) are excluded: they have their own alert, and a nightly email about an unplugged drive is one the operator learns to ignore. A manual run always reports: the digest carries a unique run_id the cooldown cannot collapse. Someone pressing the button is actively trying to get a backup. THE PERIODIC SWEEP GETS A DIGEST TOO. With the per-app event now record-only, a capture failure found between runs would be recorded and never notified — a new silence introduced while closing one. That path emits a digest with NO run_id, so the ordinary 1-hour cooldown caps it exactly as before while the mail now lists every failing app instead of whichever was first. A refusal is recorded ONCE, where the verdict is taken, not at the three legs that consult it — R-181's contract is one verdict per app per run. Noting it per leg listed one refused app three times and produced "2 of 1 apps failed". Found by the digest's own test, not in review. Silence is safe because the hub's deadline check raises expected_backup_missed from report freshness, independently of any mail this box sends (monitor/deadline.go:396,417). Confirmed, not assumed. 7 new tests, 4 red-proofs. The main.go seam walk did NOT fail on its first attempt — the AST test walked the backup package and not main.go; the test was fixed and the mutation re-run rather than the pass recorded. |
||
|
|
6c43bf6156 |
v0.193.1 — the refusal's size estimate is rendered in bytes, not "0.00 GiB" (R-181 follow-on)
gates / gates (push) Successful in 9s
Found by v0.193.0's own live proof run. The estimate was printed fixed to two decimal GiB, so every app under ~10 MB rendered as "estimated 0.00 GiB write" — which reads as "no estimate was available" and is the opposite of what happened. Observed live on demo-hp 08:59:46: opengist's real 178 KB estimate printed as 0.00 GiB. Shipped in the same session because it is the same defect class R-181 is about: a message an operator cannot rely on is worse than no message. The arithmetic is unchanged and still in GiB — the reserve's own unit, so the comparison against FloorFreeGiB reads directly. Only the rendering moved to humanizeBytes. estimatedWriteGiB -> estimatedWriteBytes, with the GiB conversion done once at the point of comparison. |
||
|
|
fef07c3923 |
v0.193.0 — the reserve guards the write that fills the disk, and its promise is true (R-181)
gates / gates (push) Successful in 9s
B2's capture floor (v0.192.0) was consulted in exactly ONE place — captureAllRecoveryUnits, which writes a few KB. The two legs that write the BULK into the same backups/primary/<app> tree, the DB dump and the volume dump, ran FIRST and unguarded. Measured live on demo-hp 2026-08-03 06:40:03: opengist's volume dump wrote 2.0 GB with no check, free fell to 1.0 GB, and the floor then refused the cheap write it had already lost the argument to. Its refusal message claimed "the previous unit is untouched" — measured false: that app's tar had gone 182,272 B -> 2,147,666,432 B under a stale manifest. Sixth entry in CLAUDE.md's table of shipped guarantees the code did not provide. Fix: ONE admission verdict per app per run (internal/backup/admission.go), taken before that app's FIRST write and covering all three legs — they write under one per-app root, which is why one verdict can honestly cover them. - Lazy, at the app's first write, NOT once at run start: app A's dump can put app B under the reserve, so a run-start verdict reads a disk that no longer exists. - Remembered for the run, never re-decided between an app's own legs — that is the split this closes. Reset per run. - Placed ahead of DumpAppVolumesSafe, which stops the stack as its first act, so a refused app is never bounced. After the volume-less check, which has no write. - Exactly one operator alert per refused app per run. - Leg order unchanged: volume dumps still precede the capture. The floor is now SIZE-AWARE: it asks whether THIS app's write would cross the reserve, not only whether the filesystem is already below it — which is how an app was admitted at 96% and then allowed to write 2 GB. Estimate = the app's previous .sql + .tar on disk. No history -> headroom-only, deliberately, and the alert says so. A container-based du per volume was MEASURED and rejected: 66 timed runs on demo-hp guest 9201, median ~355 ms/volume (341-404) on volumes holding tens of KB — container start-up, not the walk. Decisive on top: docker run needs the writable layer, so it can fail under exactly the pressure the reserve handles. The message was NOT weakened; the behaviour was moved so the wording became true. It now also names which term bound. Every claim is checked against a sha256 fingerprint of the tree it describes, never against the log line. Still refuses and never deletes: nothing here is generational. 11 new tests through the production functions. The DB leg cannot run without Docker, so its gate is pinned by an AST walk of backup.go asserting admitApp precedes DumpOne (strings.Contains is insufficient — a commented-out call still contains the string). 4 red-proofs demonstrated failing then restored. |
||
|
|
4be6467b50 |
v0.192.0 — the capture floor replaces the bulkhead (R-165, decision B2)
gates / gates (push) Successful in 8s
Ships BEFORE the disk-layout merge it exists for, and is harmless on a box that never gets it. The mp1 partition was a BULKHEAD as well as a ceiling: it kept a runaway capture from filling the space the container runtime needs, because /var/lib/docker was a different filesystem. After the merge it is the same one, and a full Docker data-root is a stopped box. The floor sits in captureAllRecoveryUnits, checked BEFORE anything is written: below the reserve, that ONE app's capture is refused, its previous unit is left byte-identical, the R-158 alert fires with the space figures, and the loop continues. Two terms whichever binds first (97% used / 1 GiB free) in fillwatch's shape, deliberately BEYOND its critical band (95% / 2 GiB) so the customer is always warned before a refusal can happen — a floor that fires before its own warning is a silent failure wearing a threshold. Headroom, never unit size: a per-unit cap would be R-163 rebuilt inside one volume. Refuses, never deletes: nothing here is generational, so pruning could only destroy a different app's only local copy; pruneStalePrimaryDirs is an orphan sweep, not retention, and must not be repurposed. Tests 1184 -> 1191. One fixture strengthened mid-red-proof: the "old 20 G ceiling is gone" test sat at exactly 20 GB and survived a literal UsedGB > 20 cap — hollow. Now 120 GB, and the mutation fails it. |
||
|
|
cf48214f6c |
v0.191.0 — warn before the wall comes down (R-167, R-158, R-174)
gates / gates (push) Successful in 9s
R-167: new internal/fillwatch warns the CUSTOMER before a filesystem fills. It emits the PRE-EXISTING disk_warning/disk_critical pair, which was allowlisted, copy'd, default-enabled and checkbox'd with no producer in any repo — the sixth "built but never wired" instance here. Two threshold terms (85% or 5 GiB free; critical 95%/2 GiB) because a percentage alone lies at both ends of this fleet's size range. Edge-triggered on escalation only, state persisted, hysteresis dead zone at 75%/7 GiB pinned by a test. A nil usage read is never a warning and never clears one. Per filesystem, never per app. Daily 03:30, before the nightly app-data legs. R-158: new unitNotify seam fires per app when a Tier-1 recovery-unit capture fails, loop continuing, carrying the target filesystem's used/free bytes. Operator-tier (recovery_unit_capture_failed) — deliberately NOT backup_failed, which is customer-enabled and would email the customer about a failure they cannot act on. D-c overrides R-158's own proposal here. R-174: the app-stop guard no longer starts apps onto MISSING drives — a regression in v0.189.0 code, found by review and closed the same session. SetStarter got the raw stack manager, whose StartStack has no drive gate, and Recover runs at startup. R-171 one path over. bootDriveGate could not be reused whole (its holder #2 is the guard's own marker, and holders #1/#2 read vars assigned after Recover runs), so holder #3 is extracted into a shared driveStartGate with a test pinning the delegation. ErrStartRefused splits a refusal from a failure: both keep the marker, only Failed alarms, because routing a deliberate hold into NotifyBackupFailed is the same false alarm. Tests 1157 -> 1184. All red-proofs demonstrated failing and restored. |
||
|
|
582135f861 |
v0.190.0 — the boot settle window, both gates on intent, and R-171
gates / gates (push) Successful in 8s
R-171 (a regression v0.189.0 introduced, CONFIRMED on hardware before any fix was written). Replacing isBootOrphan's container-count term with recorded intent made a drive-gate-stopped app read as a boot orphan: the gate stops apps with `compose down` (zero containers) and never touches desired_state, because it is not the customer. Observed on 9201 with the drive held unmounted — the sweep found and started it, burned both attempts, and handed it to the dead-app alarm. The write hazard did not materialise (the unbound mountpoint is host-root-owned and the guest is unprivileged) but that protection is accidental and untested. New consumer-side seam bootrecon.StartGate, fail-safe (cannot determine ⇒ do not start), wired in main.go. The rule is not new: the API's startGatedByMissingDrive already refuses this; the sweep bypassed it. R-157 mechanism A. The sweep looked once at T+5s, deriving candidates from a fleet docker was still restoring — three of six hard resets. Now a settle-then- sweep window: sample every 5s, settled after 3 identical samples, sweep ONCE at the end; ends on settled or a 50s budget, and the log says which. The budget is 50s because settle+budget+one retry must stay under the 90s dead-app grace — a test rejected 60s at 95s. A window that overruns emits a LATE RECOVERY warn rather than the grace being widened to hide it. Widening the window made two more holders reachable, so the one gate covers all three: an absent drive, a quiesce, and an in-flight app-data operation — reusing quiesce.SuppressedStacks() and a new read-only AppStopGuard.HeldStacks(). R-170. shouldRecreateOnBoot now reads desired_state with the identical three-way table; absent keeps the old hasContainers behaviour exactly. Its comment argued for the container count and was rewritten. presentStable is untouched. The two gates' agreement is pinned from both sides against one fixture table. 27/27 packages green; 6 red-proofs observed FAIL then restored. |
||
|
|
dbcb306fcf |
v0.189.0 — desired state + the app-stop crash marker (R-166 / D-b)
gates / gates (push) Successful in 8s
The box stops inferring the customer's intent from a container count and reads
what they actually asked for.
Part 1 — desired state. AppConfig gains a tri-state `desired_state`
(""/running/stopped), written ONLY by the customer's own action: the API action
switch, DeployStack, UpdateOptionalConfig's redeploy branch, and the .fab
import. Intent is written BEFORE the act and a failed write REFUSES the act.
StartStack/StopStack are deliberately not writers — 14 callers, only 2 are the
customer. bootrecon.isBootOrphan now reads intent instead of len(Containers)>0,
which closes R-157 mechanism B (a power cut or interrupted deploy left an app
with zero containers, read as a deliberate stop, and stranded silently).
ABSENT MEANS UNKNOWN, NEVER "running": every pre-v0.189.0 app.yaml reads absent,
so the legacy fallback is byte-identical to the old rule. A running-only startup
backfill converges the unambiguous cases; `stopped` is never inferred.
Part 2 — backup.AppStopGuard, a persisted marker over every stop→work→start
window (volume dump, offbox reconstitute, .fab export). Its own file, never
quiesce's. Written before the stop, cleared only after a restart that succeeded,
kept when one fails. Recover() completes before the boot reconciler is launched
and returns its outcome, which main.go reports on the existing backup_failed
event once the notifier exists. A defer is not the mechanism — a SIGKILL runs
none (Campaign 8 fault 10).
Also: SaveAppConfig rebuilt AppConfig field-by-field (the R-100 shape) and would
have dropped desired_state on every save across nine call sites. Replaced with
copy-and-overlay. Measured: app.yaml does not round-trip unknown YAML keys.
No hub change, no agent coupling, no user-visible string. 27/27 packages green;
7 red-proofs observed FAIL then restored.
|
||
|
|
4ed938cce4 |
D5: an app restore works from the drive alone (v0.188.0)
The recovery unit on the customer's drive now carries the PORTABLE secret class, so Tier-1/Tier-2 restore no longer depends on the whole-guest tier. A customer needs the drive and nothing else. Part 0's rulings overturned the brief's recommendation, on evidence: - the data_key flag is untrustworthy (4+ encryption keys the catalog itself labels as such are unflagged) -> R-127 - a DB password is not resettable in practice: POSTGRES_PASSWORD is ignored once PGDATA is non-empty, so a regenerated value leaves the app unable to authenticate against its own restored rows while the dump replay still reports success (proven on a throwaway postgres:16-alpine) Ruling (operator): type:secret travels, type:password never does, minus the nonPortableSecrets code register. Plaintext -- withholding the internet- reachable class is what licenses that, and the two are coupled. Precedence: the UNIT WINS over the guest -- the unit's secrets were captured in the same run as the dumps beside them, so they match the data being restored. The fail-closed data-key gate is unchanged. Secret values are never logged; the manifest records NAMES only. |
||
|
|
2f27a363d5 |
R-108: network storage may not host an app's data namespace (v0.187.0)
This is D5's precondition and it is now met. An app's namespace root IS its backup root: namespaceRoot returns a non-system drive path as-is, so the recovery unit lands at <HDD_PATH>/backups/primary/<stack>/. On a NAS that sits inside the share, which FileBrowser binds WHOLE — share root, :rslave, download:true. The bind was NOT narrowed, and establishing why inverted the fix. The share-root :rslave bind is load-bearing (a 2026-07-22 probe proved an in-container access through it wakes the idle automount trigger), and scoping is undefinable anyway: apps on a share store at <share>/<app>, there is no userdata/ layer, and creating one would write Felhom convention onto a customer's own NAS, which R-67 forbids. So the browsing surface cannot be narrowed and the backup tree must never be placed under it. Operator ruling: refuse the placement, keep the browse bind. Tier 2 already refuses network targets for this reason (F-6C-1). Nothing stranded: zero apps on network storage across all six hub customers including Peti. R-67's browse capability is byte-identical. FIVE surfaces, not the four the register named — settings.RefuseAsAppNamespace is the single predicate. The deploy POST is the real boundary (it accepts any caller-supplied HDD_PATH; DeployStack validates only os.Stat). Surface 4, handleStorageDecommission mode=migrate, guarded only its SOURCE, so a whole namespace could be decommissioned ONTO a NAS — that one is not in the register. Fails closed: /mnt/felhom-drives holds both kinds, Kind exists only on a registered path, so an unregistered path under that root refuses. Supersedes README's "NAS backup locality — decision A" (v0.118.0). 9 tests, all non-effect (nil stackMgr, so a guard that misses panics rather than passing). 4 red-proofs, each mutation asserted to have landed. Suite rc=0, 27 packages, 0 FAIL. vet rc=0. Template + emoji gates OK. |
||
|
|
b331f18424 |
v0.186.0 — R-114 + R-112: tell the truth about the backup target, then show it
Two defects E-2d found on a real box, fixed in this order deliberately: the
message is corrected BEFORE it is put on screen, because switching on a banner
that lies is worse than a silent one.
R-114 — the third state. resolveBackupTargetState had two outcomes: a disk
claims the target (healthy), or nothing does (degraded, "the backup is on the
system disk"). The state "configured, and its drive is gone" had no branch, so
it fell into the second and inherited its message AND its offer. Observed live
with the target detached: degraded:true, target:"felhom-backup" plus the
system-disk copy (false -- the backup was on a drive that had vanished) plus
offer_path naming that same vanished drive as the remedy.
New BackupTargetState.TargetAbsent discriminates. Degraded keeps its meaning
("is there a problem") so the wire contract is unchanged for every consumer;
TargetAbsent answers "which problem", because the two have opposite remedies --
attach any second drive, versus reconnect THAT one. Copy routed through
degradedMessageFor so one place still decides what a customer reads. The offer
is suppressed on the branch itself, NOT left to firstOfferableDrive's
Disconnected skip: that flag is set by the agent-side gate in another repo
(R-113), and this state must be correct independently of it.
R-112 — the state finally has a consumer. The endpoint was byte-correct and
nothing in the product ever asked for it: templates fetch 18 distinct
/api/storage/* endpoints and backup-target[/assign] were the only two with zero
references. Server-rendered on /backups now, following the existing
SingleCopyWarning banner pattern -- not a 19th JS fetch, because a banner that
needs JavaScript to appear is one more thing that can silently not happen.
backupTargetView returns nil for healthy and unknown so those render nothing at
all. The offer control POSTs to the existing assign endpoint behind the standard
inline confirm, never auto-submits, and surfaces restart_required honestly
instead of adding a self-restart.
Scenario E (the seam test) drives backupsHandler over httptest and asserts the
RENDERED HTML -- handler -> view -> resolver -> template. It deliberately does
not call the resolver and assert a string, which would prove the resolver that
was never broken. Deleting the one line that sets data["BackupTarget"]
reproduces the R-112 state and fails every render assertion.
Tests 326 -> 338 (+12) in internal/web; suite green (27 packages); both template
gates pass. Three red-proofs run and reverted, files byte-identical after.
MinAgent unchanged at 0.113.0: R-114 reads BackupTarget/MountPath/GuestPath/Role,
none of which R-113 altered (it changed BoundUnderParent, which this code does
not read). demo-hp on agent 0.113.0 is not held.
The absent copy is verbatim the hub's customerMessages["backup_target_absent"]
so the banner and the email tell one story -- filed as a two-repo drift risk,
not solved.
NOT LIVE-VALIDATED. Scenario C cannot occur on a healthy box; Session C proves it.
|
||
|
|
fd50a73e65 |
C9-F1 + C9-F2: a restore that restored nothing, and a crash loop nobody saw (v0.183.0)
Both are the system reporting healthy while the customer is not, and both live in the same status-derivation code. Neither is fixed by making the system quieter. C9-F1 (HIGH) — Tier-2 writes recovery-unit/ on EVERY run and RestoreTier2Files has never read it (tier2_restore.go:101-104 reads hdd/ + userdata/ only). Phase 0 enumerated all 53 catalog templates against both demo boxes: 43 apps have NO readable subtree, so the button stopped the app, restored 0 files, restarted it and said "Nincs hiányzó fájl — minden fájl megvan a helyén." — at the moment the customer pressed it because files were missing, with 156 MB of BookStack's data unread in the same copy. 9 apps have file legs but never their DB or volumes, so the same sentence was also a clean bill of health over data never opened (immich: 1.3 GB Postgres unit). Honesty half shipped: a pre-flight coverage check refuses UP FRONT without stopping the app and NAMES the action that works; a run that proceeds claims only what it EXAMINED and discloses that the database and volumes are not covered. Completeness is filed as C9-F1b — routing to the Tier-1 unit restore puts a destructive operation behind a non-destructive button, so its confirm copy has to carry that difference. C9-F4 filed: nothing reads the Tier-2 recovery-unit/ mirror, so the second local copy that exists for drive loss is unreachable by any customer action. C9-F2 (HIGH) — a crash loop was counted as working. StateRestarting is deliberately NOT added to IsDownState (that alarms on every deploy fleet-wide, the over-correction F-A1 nearly cost us); a sustained run becomes down after crashLoopAfter = 5m, set above the 120s deploy timeout, Mealie's 60s start_period and R-97b's 180s grace. The dashboard counter uses the same predicate, so it no longer contradicts the alarm on the same screen. README's claim that faults "still surface as restarting" was a wish with no test — corrected in place; it is the seventh such instance. Six red-proofs observed, including the one that matters most: adding StateRestarting to IsDownState fails the brief-restart test with "every deploy and update would page the operator". go test ./... rc=0, 27 packages, run and read separately from this commit. |
||
|
|
e9c99566b0 |
R-97: a failing backup is heard, and stops blaming the apps (v0.177.0)
R-97a: internal/quiesce had no route to the hub at all — three failed whole-guest backups on 2026-07-27 produced zero events. TierNotifier is a seam (not an import), wired by an init-only setter because main.go builds the notifier after the loop. Edge-triggered: the failure fires when the R-88 breaker ARMS, not per retry, and recovery rides recordSuccess's existing bool. Uses NEW operator-only event types; reusing backup_failed would have emailed the customer in Hungarian about a backup they cannot act on, since it has a customerMessages entry and is in live enabled_events. Requires hub >= v0.78.0. R-97b: v0.164.0's state filter cannot see an app caught MID-RESTART, which is how BookStack alarmed. The fix is a suppression window keyed to the quiesce CYCLE, consumed at the same single derivation point. 180s grace, derived from the deploy flow's 120s health timeout and Mealie's 60s start_period; it expires, so an app that genuinely fails to come back still alarms. |