Found by the LIVE validation on demo-hp, not by review. Scenario A passed - a
non-image catalog change reached the pinned app on the real 15-minute cycle - and
that is exactly what exposed the gap: the stored applied-compose.yml is written
when the PIN is written, so the fix landed in the live compose file and not in the
store. The first time the catalog then moved a version, the freeze would have
rendered the pre-fix definition and reverted every fix delivered since - silently
undoing the half of the operator's ruling that says fixes keep flowing.
The equal-images branch now refreshes the store as it delivers. The images cannot
move in that branch by construction, so no version moves and no intent is
rewritten. RenderPlan gains StackDir so the syncer can write it.
TestFixRefreshesTheStoredDefinition asserts both halves: the fix reaches the
store, and it survives the freeze that follows.
Slice 3. R-447 was BLOCKED because R-438 established that RestartStack's use of
up -d to pick up template changes was CHOSEN and written down in its own comment.
The operator ruled Option 1, and this implements it.
The rule: while the catalog offers the same version you run, its fixes flow to
you; the moment it moves to a newer version you are frozen until you update.
NOTHING was added to any of the thirteen compose up -d call sites. Most of them
are repairs - the boot reconciler, the drive-return gate, the app-stop guard -
and a repair path that refuses to repair leaves a customer's app down, which is
worse than the problem. They are made safe by removing the reason.
app.yaml gains pinned_images: what the app is SUPPOSED to run. It is NOT
installed_images, which is an observation; letting a reading become a deployment
is the R-166 category error one field over. Four writers, each also storing the
exact definition as applied-compose.yml. UpdateStack advances the pin and
re-renders BEFORE the pull, because pull and up -d act on the file on disk, and a
pin set afterwards would pull the frozen version and report success.
The syncer renders instead of copying, through one nil-safe seam. Catalog images
equal the pin -> verbatim, so fixes and self-healing both survive; they differ ->
the WHOLE stored definition, never a substitution of refs into a newer template
(wger 2.6 needs a DB config the older template cannot supply). This is
deliberately not 'skip deployed apps', which was option B and was rejected.
AdoptPins runs once at boot after the backfill, files only, and skips loudly
rather than inventing a pin. syncer.Start() moved to after it: the initial sync
would otherwise run while every app was unpinned and overwrite a deployed app's
version once per boot.
THE BADGE HAD TO CHANGE OR SLICE 2 WOULD HAVE INVERTED SILENTLY. TemplateImages
reads the LIVE compose file, which is now the frozen one, so the comparison would
have answered Naprakesz on exactly the apps that are behind - with every test
green, because the new field has the same type. It now reads CatalogImages.
+16 tests (1729 -> 1745), 28 packages green. Three red-proofs run and reverted.
A test also caught the syncer writing an empty compose file over a live app.
The operator looked at demo-felhom the morning after v0.233.0 and found OpenGist
- up 15 hours, running exactly the catalog pin - showing no badge at all.
v0.233.0 wrote the record only from the four bring-up paths, so an app nobody
restarts carried no record indefinitely. On a quiet box that is every app, which
is the box we most want to see. The known limitation WAS the feature not working.
BackfillInstalledImages runs once at startup, beside BackfillDesiredState and
before the boot reconciler. It READS containers: starts nothing, restarts
nothing, writes no compose file. It never overwrites an existing record.
And it REFUSES to seed a partial observation, which is why this is not a
three-line loop: the badge reads a service-count mismatch as BEHIND, so seeding a
degraded app from what is visible would render 'Frissites elerheto' over an app
that is perfectly current. The bring-up paths may write a partial because they
follow a successful up -d where a gap is real news; a backfill meets any state.
Same data, two writers, two admission rules - deliberately.
Also fixes a calendar bomb of mine: the render test hardcoded catalog_since and
the string '46 napja', but the render path reads time.Now(), so it was green on
the day it was written and red the next morning. Now derived. Filed as R-457
with six other candidate files named as unchecked, not accused.
+5 tests (1724 -> 1729), 28 packages green. Red-proof of the partial guard run
and reverted; the wiring and its ORDER pinned by an AST walk.
Update arc slices 1 and 2. NEITHER CHANGES ANY BEHAVIOUR — no new endpoint, no
auto-update, the three lifecycle buttons byte-identical.
Slice 1 — app.yaml gains installed_images, keyed by compose SERVICE name, each
entry carrying ref + repo digest + first-seen timestamp. Written by
Manager.recordInstalledImages after a successful compose up from StartStack,
RestartStack, UpdateStack and runComposeDeploy. Read from the CONTAINER, never
from docker-compose.yml: the syncer overwrites a deployed app's compose on a
15-minute cycle and the two disagreed for 25 minutes in the spike's own
measurement. A failed write NEVER refuses the action - the deliberate opposite
of SetDesiredState, because this is an observation and that is an intent. Not
called from StartStackServices (the R-47 DB-only window). Its own docker seam
with a context and a 30s timeout, which neither existing exec helper has.
Slice 2 — .felhom.yml gains optional catalog_since; web.updateBadge compares the
recorded ref per service against what the current template pins and returns a
*MetaBadge through the EXISTING meta_badge partial. No new markup, no new CSS.
NO RECORD RENDERS NOTHING: absent means unknown and never means current. No
version number reaches the customer and no registry is queried.
Known limitation, filed not hidden: 23 catalog pins float, so those apps can read
Naprakesz when the image behind the tag has moved.
+17 tests (1707 -> 1724), 28 packages green. Wiring proven through a real
RestartStack plus an AST walk of the four call sites. Three companion red-proofs
run and reverted.
R-384. aggregateState returned StateUnhealthy the moment unhealthy > 0, and the
R-51 mixed-case block that asks "is a supervised member dead?" sat below it. A
two-container app whose database exits goes unhealthy BECAUSE it cannot reach
that database - so the symptom the dead database causes was what suppressed the
alarm for it. unhealthy is not a down state, so classifyRunStates never marked
the app down and app_start_failed never fired.
Measured live on demo-hp 2026-08-22: bookstack-db stopped at 21:27:01 and the
F-OBS heartbeat printed "0 currently down" throughout. R-51's 18-hour immich
failure, back through a different door.
Two things moved, and either alone leaves the defect standing: the supervised
test is hoisted above the unhealthy/starting/restarting returns, and "some
members are up" now counts ANY member not in the down bucket. The old guard was
running > 0, which made the R-51 block unreachable in exactly the case it was
written for.
IsDownState is byte-identical - unhealthy stays excluded, because an unhealthy
container is running and folding it in reintroduces the flapping that exclusion
exists to stop. No new state was minted. Only the ORDER changed. The priority
comment was rewritten because it asserted an ordering the code no longer has.
Three subtests in TestAggregateState_UnchangedBranches were AMENDED: they
asserted an unhealthy/starting/restarting member beat an exited peer on
unless-stopped, which pinned the defect as settled behaviour. They keep their
intent with the down member given a benign policy.
R-383. The double-failure message said the previous state's backup EXISTS,
built from the returned path without asking the filesystem - and a missing file
is one of the two ways that rollback fails. undoCopyPhrase now describes the
copy from disk: present, partial, missing (still naming where it should be), or
never written. Zero-length counts as missing.
Test count 1494 -> 1504. Four red-proofs planted, four seen failing; the two
halves of R-384 convict independently.
Completes R-351 and ships R-352's visibility half. Gates 11/11 OK, suite 28 packages ok,
go vet clean, -race clean on the changed package - all run and read BEFORE this commit.
PART 2 SCENARIO A - the deploy page prefills the address and data folder from the app's OWN
backup. backup.RecordedUnitForStack scans every readable namespace root (the app is NOT
installed in this case, so there is no own drive to ask) and reads manifest.json plus the
captured compose/app.yaml. Local file reads only: no network, no restic, no restore.
RecordedAddress.Known() requires BOTH halves on purpose - an absent SUBDOMAIN makes the live
deploy path substitute the CATALOG default (stacks/deploy.go:88-90), and offering that back as
"what your backup says" would be a fabricated fact. The prefill is labelled as coming from the
backup and stays editable: a memory, not a lock.
PART 1 VISIBILITY (R-352) - the deploy page now states where the app's data will live before
the button is pressed. Measured 2026-08-21: 13 of 53 catalogue templates declare a storage
field; the other 40 have none and their data goes to the system drive, which no screen said.
Metadata.HasDeployField answers "does this app have somewhere to PUT a recorded value?" - for
the 40-class a recorded placement is a fact to state, never a value written into a field that
does not exist. NO PLACEMENT CHANGED. NOTHING MIGRATED. The rest is a filed specification.
PART 4 - measured before theorising, on the live off-site target:
snapshots --json 2605 ms once; stats 2697 ms PER APP, sequential, 5 app tags
=> 2605 + 5*2697 = ~16.1 s, matching the reported ten-to-fifteen seconds.
The cause is the shape already on file, so the per-app size calls now run concurrently,
BOUNDED TO 4. The bound is the safety property, not the speed one: the repository is a Hetzner
Storage Box with a session cap, and a refused size call returns SizeBytes 0 - a silent
UNDER-REPORT of the customer's data rather than a visible failure. Peak-in-flight is asserted.
OffsiteInventoryList had no test at all before this.
TEMPLATE SAFETY - every Restore* key is set UNCONDITIONALLY in the deploy handler, because a
template doing index/eq against an undefined key errors at RENDER time: green build, green vet,
green suite, 500 on the page. Four render tests, one per branch, because the existing deploy
render test only renders AutoFields and never reaches these blocks.
RED-PROOFS, mutation asserted applied then reverted to 0:
A three template guards dropped (count asserted 3) -> the blank form returned
P4 inventorySizeConcurrency = 1 -> "peak in flight was 1", elapsed 282ms = sequential
DOCS: CHANGELOG v0.217.0 (MinAgent 0.129.0 unchanged), CONTEXT (the restore's own memory +
what is next), controller/README.md (Backup System), REUSE.md (4 new rows), REPORT.md
overwritten - the previous REPORT preserved to audits/REPORT-v0.216.0-2026-08-14.md first.
NOT fixed here, filed as R-353 and named the next session's first item: a restore whose unit
carries no db_dumps and no volume_dumps still reports a bare completion.
ExportDataMounts lives in delete.go, which reads as a destructive path. IT IS NOT: its single
production caller is the .fab export adapter, and nothing deletes based on its result. The
delete path's own guard, ProtectedHDDPaths, is layout-agnostic by construction -- it protects
BOTH <hdd>/... and <hdd>/felhom-data/... -- so deletion was never affected by the
namespace-root defect. That scope note is now in the function's doc comment, because the file
placement will mislead the next reader exactly as it misled the spec for this change.
Separated into its own commit anyway, so a change to a function whose filename says "delete"
is reviewable on its own.
An empty nsRoot falls back to hddPath -- the pre-R-203 shape -- so any caller not yet updated
keeps working on enrolled drives.
Tests cover both drive kinds and assert the NEGATIVE: no emitted path lies outside the app's
own data roots. Red-proof: leaving the site bare fails the system-drive row, emitting
/mnt/sys_drive/userdata where the canonical root is /mnt/sys_drive/felhom-data/userdata.
appbackup's path helpers take a NAMESPACE ROOT. Five call sites passed a bare DRIVE path.
On an enrolled drive the two coincide, so nothing showed; on the system-data fallback they
differ by exactly the felhom-data segment, and the app then bound a directory the off-site
capture set never looked at -- while the run reported ok. Measured live on demo-hp: the app
wrote to /mnt/sys_drive/userdata/media/books, the capture set looked for
/mnt/sys_drive/felhom-data/userdata/media/books.
THE RULE NOW HAS ONE EXPRESSION. appbackup.NamespaceRootFor / IsEnrolledDrive encode the
drive-kind comparison; backup.Manager.namespaceRoot and stacks.Manager.inGuest delegate to
it. There were already TWO copies and they differed -- the backup package's compared without
filepath.Clean, the stacks package's with it, so a trailing slash from config would have
flipped the mode in one and not the other.
Sites routed through it:
- stacks/deploy.go withPathVars -> ${USERDATA_PATH} (the live defect)
- appexport/fabplan.go + export.go (via a new provider method)
- web/handlers.go FileBrowser mounts (latent: the system drive is
deliberately never a registered StoragePath, so this is the identity today)
ComputeFabBuckets now receives the namespace root, which is what ComputeCaptureSet has always
received -- so the export's classified paths and the backup's capture set describe the same
directories by construction instead of by coincidence.
Tests are table-driven over BOTH drive kinds, because this survived by being invisible on the
kind that already worked. Red-proofs observed: restoring the bare-path call fails the
system-drive row with the two paths differing by /felhom-data; inverting the drive-kind
comparison fails every enrolled row.
R-171 (a regression v0.189.0 introduced, CONFIRMED on hardware before any fix
was written). Replacing isBootOrphan's container-count term with recorded intent
made a drive-gate-stopped app read as a boot orphan: the gate stops apps with
`compose down` (zero containers) and never touches desired_state, because it is
not the customer. Observed on 9201 with the drive held unmounted — the sweep
found and started it, burned both attempts, and handed it to the dead-app alarm.
The write hazard did not materialise (the unbound mountpoint is host-root-owned
and the guest is unprivileged) but that protection is accidental and untested.
New consumer-side seam bootrecon.StartGate, fail-safe (cannot determine ⇒ do not
start), wired in main.go. The rule is not new: the API's startGatedByMissingDrive
already refuses this; the sweep bypassed it.
R-157 mechanism A. The sweep looked once at T+5s, deriving candidates from a
fleet docker was still restoring — three of six hard resets. Now a settle-then-
sweep window: sample every 5s, settled after 3 identical samples, sweep ONCE at
the end; ends on settled or a 50s budget, and the log says which. The budget is
50s because settle+budget+one retry must stay under the 90s dead-app grace — a
test rejected 60s at 95s. A window that overruns emits a LATE RECOVERY warn
rather than the grace being widened to hide it.
Widening the window made two more holders reachable, so the one gate covers all
three: an absent drive, a quiesce, and an in-flight app-data operation — reusing
quiesce.SuppressedStacks() and a new read-only AppStopGuard.HeldStacks().
R-170. shouldRecreateOnBoot now reads desired_state with the identical three-way
table; absent keeps the old hasContainers behaviour exactly. Its comment argued
for the container count and was rewritten. presentStable is untouched. The two
gates' agreement is pinned from both sides against one fixture table.
27/27 packages green; 6 red-proofs observed FAIL then restored.
The box stops inferring the customer's intent from a container count and reads
what they actually asked for.
Part 1 — desired state. AppConfig gains a tri-state `desired_state`
(""/running/stopped), written ONLY by the customer's own action: the API action
switch, DeployStack, UpdateOptionalConfig's redeploy branch, and the .fab
import. Intent is written BEFORE the act and a failed write REFUSES the act.
StartStack/StopStack are deliberately not writers — 14 callers, only 2 are the
customer. bootrecon.isBootOrphan now reads intent instead of len(Containers)>0,
which closes R-157 mechanism B (a power cut or interrupted deploy left an app
with zero containers, read as a deliberate stop, and stranded silently).
ABSENT MEANS UNKNOWN, NEVER "running": every pre-v0.189.0 app.yaml reads absent,
so the legacy fallback is byte-identical to the old rule. A running-only startup
backfill converges the unambiguous cases; `stopped` is never inferred.
Part 2 — backup.AppStopGuard, a persisted marker over every stop→work→start
window (volume dump, offbox reconstitute, .fab export). Its own file, never
quiesce's. Written before the stop, cleared only after a restart that succeeded,
kept when one fails. Recover() completes before the boot reconciler is launched
and returns its outcome, which main.go reports on the existing backup_failed
event once the notifier exists. A defer is not the mechanism — a SIGKILL runs
none (Campaign 8 fault 10).
Also: SaveAppConfig rebuilt AppConfig field-by-field (the R-100 shape) and would
have dropped desired_state on every save across nine call sites. Replaced with
copy-and-overlay. Measured: app.yaml does not round-trip unknown YAML keys.
No hub change, no agent coupling, no user-visible string. 27/27 packages green;
7 red-proofs observed FAIL then restored.
The recovery unit on the customer's drive now carries the PORTABLE secret
class, so Tier-1/Tier-2 restore no longer depends on the whole-guest tier.
A customer needs the drive and nothing else.
Part 0's rulings overturned the brief's recommendation, on evidence:
- the data_key flag is untrustworthy (4+ encryption keys the catalog itself
labels as such are unflagged) -> R-127
- a DB password is not resettable in practice: POSTGRES_PASSWORD is ignored
once PGDATA is non-empty, so a regenerated value leaves the app unable to
authenticate against its own restored rows while the dump replay still
reports success (proven on a throwaway postgres:16-alpine)
Ruling (operator): type:secret travels, type:password never does, minus the
nonPortableSecrets code register. Plaintext -- withholding the internet-
reachable class is what licenses that, and the two are coupled.
Precedence: the UNIT WINS over the guest -- the unit's secrets were captured
in the same run as the dumps beside them, so they match the data being
restored. The fail-closed data-key gate is unchanged.
Secret values are never logged; the manifest records NAMES only.
Both are the system reporting healthy while the customer is not, and both live in the same
status-derivation code. Neither is fixed by making the system quieter.
C9-F1 (HIGH) — Tier-2 writes recovery-unit/ on EVERY run and RestoreTier2Files has never read
it (tier2_restore.go:101-104 reads hdd/ + userdata/ only). Phase 0 enumerated all 53 catalog
templates against both demo boxes: 43 apps have NO readable subtree, so the button stopped the
app, restored 0 files, restarted it and said "Nincs hiányzó fájl — minden fájl megvan a helyén."
— at the moment the customer pressed it because files were missing, with 156 MB of BookStack's
data unread in the same copy. 9 apps have file legs but never their DB or volumes, so the same
sentence was also a clean bill of health over data never opened (immich: 1.3 GB Postgres unit).
Honesty half shipped: a pre-flight coverage check refuses UP FRONT without stopping the app and
NAMES the action that works; a run that proceeds claims only what it EXAMINED and discloses that
the database and volumes are not covered. Completeness is filed as C9-F1b — routing to the
Tier-1 unit restore puts a destructive operation behind a non-destructive button, so its confirm
copy has to carry that difference. C9-F4 filed: nothing reads the Tier-2 recovery-unit/ mirror,
so the second local copy that exists for drive loss is unreachable by any customer action.
C9-F2 (HIGH) — a crash loop was counted as working. StateRestarting is deliberately NOT added to
IsDownState (that alarms on every deploy fleet-wide, the over-correction F-A1 nearly cost us); a
sustained run becomes down after crashLoopAfter = 5m, set above the 120s deploy timeout, Mealie's
60s start_period and R-97b's 180s grace. The dashboard counter uses the same predicate, so it no
longer contradicts the alarm on the same screen. README's claim that faults "still surface as
restarting" was a wish with no test — corrected in place; it is the seventh such instance.
Six red-proofs observed, including the one that matters most: adding StateRestarting to
IsDownState fails the brief-restart test with "every deploy and update would page the operator".
go test ./... rc=0, 27 packages, run and read separately from this commit.
Found on the demo-felhom deploy leg: ensuring only <sysNS>/userdata/import left
its parent at 755 root:root, because EnsureUserdataDir MkdirAll's intermediates
at plain 0755 and chmods only the leaf. That made the system drive's userdata
root the one on the box outside the 2775/gid-1000 convention.
${IMPORT_PATH} = <system namespace root>/userdata/import — ONE drop-zone per box,
on the system drive, injected at BOTH compose-env builders with NO per-drive
fallback (unresolvable leaves it unset so compose fails loudly rather than
quietly building a second, dead drop-zone).
Third BindRoot (RootImport) + Import list in BackupSpec, extended through
ValidateBackupSpec/ClassifyBinds. Load-bearing: a stale `userdata: import/<app>`
entry against the moved bind would be a WHOLE-BLOCK reject, taking the app's
mandatory hdd classification with it.
Exhaustive-root audit: resolveAbs/structuralGuard/ComputeCaptureSet/
ComputeFabBuckets now take importRoot explicitly (an import bind resolved
against hddPath would name a directory on the wrong drive); unresolvable is
refused loudly into Skipped. GetImportRoot added to both provider interfaces.
Catalog-derived skeleton: UserdataSkeleton() -> UserdataSkeletonCarry() +
BuildUserdataSkeleton(), SORTED. The carry-list makes zero-removals true by
construction (`documents` is in no catalog app but on both boxes) and is the
fresh-box floor. The sort is not tidiness: the naive map-order derivation
measured 20 distinct outputs from 20 identical runs, which with fbNeedsRecreate
is a fleet-wide FileBrowser restart loop.
One authoritative compose parser: ParseComposeUserdataMounts now delegates to
ParseComposeClassifiableBinds. Import root excluded from per-app migration.
Surfaces: FileBrowser /srv/beolvasas source; app-page "Hova tegyem a fajlokat?"
with PathEscape deep links (never QueryEscape) and class-driven copy;
data_paths: annotation with the Fork-3 asymmetry; system-owned beolvasas SMB
share refused server-side at handler AND store, button omitted in template.
Caught on the way: the sharing template's row struct was function-local, so
adding {{if .System}} would have 500'd every share row. ShareRow is now
package-level and the render test uses the handler's own type.
Tests 915 -> 949, all green. MinAgent unchanged.
gofmt -w across the controller tree (46 files) so gofmt -l is empty — disarms the
formatting landmine where a targeted edit + accidental gofmt -w swept ~46 unrelated
files. Pure formatting: whitespace + gofmt's optional-semicolon removal in reflowed
inline closures. One doc comment reworded ('' -> 'the empty string') to avoid gofmt's
Go-1.19 doc-comment typographic substitition ('' -> curly quote) muddying its meaning.
No build/vet/test behavior change.
New /launcher page: a grid of large tappable tiles, one per openable deployed
app (subdomain presence is the single openability criterion, shared with the
Megnyitás button via the extracted Server.subdomainMap helper). Colored tiles
(deterministic slug color or .felhom.yml brand_color), white glyph/monogram
fallback, target=_blank links for operational apps, greyed unclickable tiles for
stopped ones. First sidebar item; / stays the Vezérlőpult.
Universal app placeholder: new AppPlaceholderSVG served at
/static/app-placeholder.svg, now the default FallbackIcon on app_list_row so a
logo-less app shows a placeholder instead of visibility:hidden. Brand mark is
never an app placeholder.
New Metadata.BrandColor; new funcmap tileColor/initial. 10 new test functions +
4 red-proofs. No agent coupling; MinAgent unchanged.
Leg A: „Hálózat" card on Beállítások → Rendszer — Helyi cím (LAN),
Hálózati név (only while Megosztás is enabled), Átjáró; live per render,
stored nowhere (S-5), „—" on unavailable.
Leg B: network section in the Debug system dump (interfaces/route/DNS/
lan_address), best-effort per item via the samba-netns door.
Leg C: NetBIOS trap named — Szerver field helper text + a purely lexical
hint on unreachable failures for single-label non-IP names.
Design note: all guest-net reads go through docker exec into the
host-networked felhom-samba container (stacks/guestnet.go, one seam) —
the controller's own netns is the docker bridge, so /proc/net/route etc.
would answer 172.x (the S-2 trap). Red-proofs: A2 gate-drop and C2
lexical-invert both failed as required.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UuFPHmHNrCJj1VhY6QdDMU
Shipped in v0.158.0, caught live within the hour. /apps/<slug> returned 500
for EVERY app: html/template cannot call a pointer-receiver method on a
non-addressable value, and appDetailHandler passes Meta as a VALUE inside a
map[string]interface{}.
It compiled and every test passed because nothing rendered app_info - the
catalog tests used the funcmap route, which takes a value and works either
way. A template method call is only checked when the template runs.
Adds TestAppInfoRendersForEveryLifecycle with the handler's exact data shape.
The previous commit landed only the new test/badge files: a 'git stash' used
to compare REUSE.md ref-check output silently dropped the staged index, so
every modification to an existing file was left behind and that commit does
not build. This adds the metadata field, the predicates, the fail-closed
deploy gate, the catalog filter, the funcmap entries, the template edits and
the docs that those tests exercise.
The catalog knew only 'present' or 'gone', and 'gone' orphans every customer
already running the app. lifecycle: in .felhom.yml withdraws an app from new
installs without touching anyone running it.
Deploy gate is server-side and fail-closed, before any mutation, with the
ruled Hungarian refusal - hiding a button is not a gate. Unknown values fail
OPEN (available + one WARN), deliberately opposite, so a typo or a newer
catalog cannot pull a working app out of every customer's list.
Orphan detection never sees the field - a red-proof adds that filter and shows
the abandoned app immediately reading as an orphan.
Badge plumbing is generic (MetaBadge + meta_badge partial) so R-56's
difficulty labels drop in with no new markup.
R-51: aggregateState's mixed branch returned StateRunning ("partial"), so a stack whose
MAIN container was dead behind live helpers alerted on nothing — immich-server sat Exited
for 18 h, 100 % unreachable, no banner and no app_start_failed (audit F4). New
StateDegraded: a DOWN member whose docker restart policy is always/unless-stopped is a
fault (degraded, a down state); no/on-failure is a finished one-shot and stays benign; an
unreadable policy fails CLOSED. The unhealthy/restarting/paused/unknown exclusions are
byte-identical — folding unhealthy into down is the flapping fix-3 avoided.
R-52: new internal/bootrecon — one bounded start-once sweep at startup (2 attempts, 30 s
apart) for apps an interrupted boot left behind, inside the 90 s boot grace so a success
is silent and a failure still alerts. A zero-container stack is NEVER touched: the UI's
Stop is compose down, so a deliberate stop survives a reboot.
Both features carry a production-path wiring test (the v0.154.0 / v0.91.0 inert-seam
class). The main() assertion is an AST walk, not strings.Contains — the substring version
passed its own red-proof, because a commented-out call still contains the string.
Red-proofs run and restored: mix branch reverted -> "running" on the immich fixture;
boot hook commented out -> wiring test fails; zero-container gate dropped -> the
user-stopped app gets started.
NOTE: controller/cmd/controller/ is matched by .gitignore's `controller` entry, so new
files there need `git add -f` (and ripgrep silently skips main.go without --no-ignore).
Closes R-47. No new agent coupling — MinAgent stays 0.90.0.
The replay needs a running DB container, so both restore paths started the
WHOLE stack first, giving the application a window to rebuild the very schema
objects the dump was about to create. Measured live on 2026-07-19 (H4,
DIAG-immich-restore-round2): immich-server rebuilt clip_index two seconds
before the dump's CREATE INDEX, the replay aborted "already exists" under
ON_ERROR_STOP=1, and immich reported schema drift. The data survived only
because pg_dump emits COPY before CREATE INDEX.
Both paths now open a DB-ONLY window: only the stack's database service(s)
come up, the dump is replayed with the app still down, and the full start
runs only after the replay exits 0. Fail-closed: a dump with no identifiable
DB service refuses BEFORE the first mutation. Every exit from the window
still does a best-effort full start, so a failed restore never leaves a box
with a database and no application.
New: appbackup.DBServiceNames (yaml.v3 services-map parse — never a line
scan; immich's top-level volume keys are the decoy) sharing dbTypeForImage
with DiscoverDatabases; stacks.Manager.StartStackServices (refuses an empty
list — argument-less `up -d` is a full start); RedeployFromEnv split into
PersistUnitRedeployConfig + its unchanged tail. StackDataProvider's
RecreateStackFromUnit becomes RecreateStackDefinitionFromUnit — the hidden
`up -d` inside the old name is what carried the defect on the local path.
19 new tests (ordering plus state-at-replay-time, zero-mutation fail-closed
effects, replay-failure bring-up, parser decoys, empty-list refusal); three
companion red-proofs run and reverted. 23/23 packages green.
Not yet live-validated: STOP-1 supervised reconstitute, golden 0.153.0.
S-1: /sharing/status coerced idle->running on the PHASE channel, so the first
poll of every steady-state page load reported a terminal job that never ran and
the client's repaint-reload fired ~1.2s apart, forever. The coercion's real duty
(liveness must never be contradicted) belongs to the 'running' LEVEL field
beside it, and is now pinned by its own regression test.
S-4 core: a terminal 'running' is served exactly once, so a REAL bring-up cannot
re-arm the reload on the page it just caused. failed/needs_password/in-flight
are never consumed. Unified async-job feedback stays the ROADMAP item.
S-2/S-5: new connect card with the Windows form, the Mac form and the direct
smb://<IP>, read from the SAMBA container's netns (the controller is on a docker
bridge and would answer 172.x). Derived per render, cached nowhere - the address
is a DHCP lease. Underivable => the line is omitted.
sharing.html's <script> block is byte-identical to v0.150.0. Red-proofed three
ways. 23/23 packages green.
The systemic complaint, twice in one evening: you press a button and nothing
happens. No progress, no ETA, no named result. Three worst offenders, fixed on
the two patterns already here (deploy 3-step panel, storage-init status poll).
No new framework — that is a ROADMAP item; three targeted cards ship tonight.
4a — a verification restore names its result. The flash said the app had been
restored "to a verification folder on the drive"; which folder, on which drive,
was invisible, so the customer could not go and look at what they had just asked
for. Full path now. The restore page gained a listing of existing verification
copies (app, size, date, path) — nothing anywhere showed these, so they piled up
and the only way to find them was SSH — each with a double-confirmed delete.
That delete is the only one this release adds, so it names a STACK, never a
path: the Manager resolves the name inside a backups/offsite-restore root it
computed itself and refuses anything landing outside. Red-proofed — neutralise
the name guard and stack:"" resolves to the offsite-restore ROOT and takes every
copy with it. Refusals are asserted as non-effects.
4b — Megosztás enable shows what it is waiting for. Enabling ran ReconcileSamba
synchronously inside the POST handler; on a golden without felhom-samba baked
that is compose pulling ~100MB, i.e. minutes of an apparently-hung form post
followed by "Beállítás mentve." whether or not anything came up. Detached +
polled now, distinguishing "képfájl letöltése" from "indítás" — decided BEFORE
the work starts, since afterwards the image is always present. Success is
probed, not inferred (compose up -d exits 0 on a crash-loop). The password form
starts the same job: with UserSet false reconcile deploys nothing, so on a fresh
box that is where the pull actually happens.
4c — "Távoli mentés most" streams real progress. restic was already reporting
bytes and percent; the runner seam used CombinedOutput() and discarded them. The
manual run now passes --json and scans stdout line-by-line: total bytes, percent,
current app. Manual only — the nightly stays silent, pinned by a test that fails
if it ever passes --json. The poll now arms unconditionally, closing a race the
manual trigger always ran: the redirect rendered before the goroutine wrote
LastStatus=running, so the poll never armed and the page sat static during the
very run just started. Red-proofed twice.
Also closes the golden/controller infra-image drift at the source: infra.Images()
derives from the existing pins and --print-infra-images exposes it, so the golden
bake can stop carrying its own copy. That copy had already drifted — felhom-samba
was never added, so the golden baked 3 of 4, which is why enabling Megosztás
pulled at runtime in the first place.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nn3VgQk9iwEGgyx6QJ2NvE
The samba_classify.go SCOPE NOTE and the README KNOWN GAP both described a gap that
R-7b closes; both now describe the sibling-shares-source execution instead.
Sibling shares source for the local cross-drive tier. Reuses the tier2Mirror seam,
selectTier2TargetFrom (narrow source-drive seam extracted from selectTier2Target),
tier2ReconcileRoots (pure extraction), tier2SafeRemove, the marker-LAST discipline
and the recordTier2* helpers. Per-app paths are untouched.
- shares_payload.go: deterministic _shares-manifest.json + best-effort passdb capture
- tier2_shares.go: per-source-drive legs -> cross-drive target, payload, marker LAST
- infra.SambaContainerName/SambaPassdbVolume/Mount: single source of truth for the
container identity (renderer, stacks execs, backup execs, monitor all read it)
- RESERVED-NAME finding: ValidateSMBShareName did NOT exclude a leading underscore,
so "_shares" was an accepted share name. Now refused; RunAllTier2 additionally
skips a "_shares" stack loudly as defense in depth.
- fix: shareSourceDrive returned a slash-normalised path, which made the target
selector's source-drive equality check miss (a group could target its own drive)
New top-nav category with the Halozati megosztas page: enable/server-name card,
household password, shares table (Nev/Mappa/Irasvedett/Felhomentes/Torles), and
a create flow (new folder under <storage>/shares or an existing folder via the
browse modal). Every customer path goes through sharingResolvePath: absolute ->
EvalSymlinks -> containment in a registered live storage root -> deny-listed
system subtree check -> is-a-directory. Refusals are UNIFORM so the picker is
never a filesystem oracle. Deny-list derived from ProtectedHDDPaths (provably a
subset); the drive root is an exact-match denial so user-data folders under it
stay shareable. samba infra metadata + i-share icon. Gates green.
ClassifiedBinds("samba") resolves from the shares registry instead of catalog
metadata (samba has no .felhom.yml; its binds are absolute share paths). [R4]
Offsite ON -> mandatory (offsite + tier-2); OFF -> optional (tier-2 only);
smb.conf/passdb never classified. Verified through the REAL ComputeCaptureSet
tier filter incl. the negative (optional NOT in offsite). Zero engine edits.
Part-4 Step-1 finding: tier-2 (RunTier2) short-circuits on os.Stat(unitDir)
BEFORE GetStackClassifiedBinds, and the offsite runner enumerates
settings.GetOffboxApps() — both are recovery-unit shaped, which a share-only
infra stack has not. Wiring share data into a live run is therefore more than an
enumeration tweak; reported as a design fork per the STOP clause, not improvised.
ensureSamba joins EnsureBaseStack after filebrowser, gated on SMB.Enabled
(cloudflared conditional precedent); reconcile is idempotent (unchanged config +
running container = ZERO compose calls, asserted via seam). Atomic tmp+fsync+
rename config writes. Password applied via smbpasswd on STDIN (never argv/log/
settings). Disable = compose down, volumes + folders KEPT. samba added to
IsProtectedStack in code (controller.yaml is golden-generated and predates it),
which also makes the app-backup loops correctly skip it.
Task 2 of the backup-classification-redesign arc. Ships the referential-
coupling classification as DATA + PARSER + PURE CLASSIFIER, deliberately
inert — no backup tier changes behavior. Task 3 (tier policy engine) and
Task 4 (manual .fab UI) consume it.
- appbackup/classify.go: BackupSpec/BindSpec/ComposeBind/ClassifiedBind;
ClassifyBinds (SQ5 two-level default — explicit beats :ro; unlisted
writable→mandatory, unlisted :ro→excluded; nil spec→legacy/false);
ValidateBackupSpec (whole-block-reject on any defect, first defect named).
- stacks/classify_binds.go: ParseComposeClassifiableBinds — ${VAR}-relative
binds + :ro flag (NOT ParseComposeHDDMounts/ExportDataMounts, the traps).
- Metadata.Backup + LoadMetadata as the single validation choke point (bad
catalog block → nil + one ERROR → legacy, within one sync cycle).
- Manager.ClassifiedBinds + StackDataProvider.GetStackClassifiedBinds seam
(delegated by stackAdapter, nil-stubbed in every fake) — wired + tested
now so Task 3 consumes a tested seam.
INERT: full pre-existing suite green with zero test-logic edits. +14 tests;
red-proofs RP-1..RP-4 confirmed. The 13 catalog backup: blocks ship in the
same app-catalog change (this controller deploys first).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A45Qop8YY8tS94bz63LFne
The controller assumed an app's HDD appdata dir is always appdata/<stackName>.
paperless-ngx writes appdata/paperless (stack paperless-ngx), so every consumer
keying by stack name silently missed it via a stat-and-skip. One canonical
resolver appbackup.AppDataDirNames derives the real dir name(s) from the app's
compose ${HDD_PATH} binds; all consumers use it.
- F-S2 (tier-2): RunTier2 mirrors the resolved appdata/<name> (paperless docs
got NO tier-2 copy before). Tier2Info size + RestoreTier2Files live dir use it.
WARN when a declared appdata dir is absent. New tier2Mirror seam.
- F-S3 (migrate, NEW): all six per-app appdata legs (collision/size/copy/verify/
cleanup/skip-set) now loop resolved names. scope="app" migration of paperless
previously copied nothing and left an empty media dir (scope="all" was saved by
the merge walk). WARN on missing declared dir in the copy leg.
- Multi-dir (N>1) refusal: tier-2 backup/info/restore refuse loudly (Hungarian);
migrate supports N. No catalog app hits it today; lifted by Task 3.
- Display: storage page sums resolved dirs.
- Truth repair: the v0.130.0 "tier-2 copies the namespace wholesale" claim is
false; corrected in CHANGELOG + main.go export-adapter comment.
+9 tests; red-proofs RP-1..RP-5 all confirmed. Controller-only, no agent/hub
coupling. Task 1 of the backup-classification-redesign arc.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A45Qop8YY8tS94bz63LFne
exportAdapter.GetStackHDDMounts now returns stacks.ExportDataMounts — the
${HDD_PATH} binds UNIONed with the ${USERDATA_PATH} ROOT (captured at the
root, not per-bind, so the manifest's basename keying round-trips through the
existing import mapping without touching restore). Containment-aware dedupe
both directions. The backup-side stackAdapter is intentionally unchanged.
Red-proof: pre-fix behavior fails TestExportDataMounts_UserdataConvention/
MixedBindsUnion/LiteralUserdataBindDeduped (run->fail->revert recorded).
The proceed-path for a missing RESETTABLE secret redeployed the app with the
secret blank (compose "Defaulting to a blank string" → exit 1, live-hit in the
2026-07-04 drill Phase 5). Now the restore generates a fresh credential instead:
- stacks.Manager.GenerateSecretForField: replacement value from the field's
catalog generate spec via the deploy flow's generateValue (no logic copied);
refuses data-keys (defense-in-depth), spec-less and non-secret fields.
- backup.Manager.SetSecretGenerator seam (wired in main.go), consulted in
RestoreFromRecoveryUnit AFTER the untouched fail-closed gate, for missing
names NOT in DataKeyEnvVars. The generated value rides fullEnv into
RecreateStackFromUnit → RedeployFromEnv → SaveAppConfig, so it persists
encrypted in the guest app.yaml and round-trips on the next backup/restore
(no second write path). reconcileRestoreSecrets stays pure and untouched.
- WARNs now discriminate: "generated replacement for X (credential was reset)"
vs "X unrecoverable and has no generator — app may fail to start". Values are
never logged (asserted in test).
- Residual case (documented, not pretended away): if a restored volume tar
carries the OLD internal credential hash, the app may still fail auth until a
manual in-DB reset — generation fully fixes only the fresh-init case.
Companion red-proof: pre-fix behaviour (generation skipped) fails
TestRestoreGeneratesMissingResettableSecret on the non-empty DB_PASSWORD
assertion (verified, reverted). Data-key gate proven unreachable by generation
in TestRestoreGenerationNeverReachesDataKeys.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
Gap 1: third shim listener :2526, plaintext, does NOT advertise STARTTLS (TLSConfig
nil) — for opportunistic-STARTTLS clients with no cert-skip (cal.com, nextcloud).
Gap 2: SMTPMapping tls_mode (picks port 2525/2526/2465) + from_domain_var (split
local-part + domain for nextcloud's MAIL_FROM_ADDRESS/MAIL_DOMAIN). Default keeps
existing apps on 2525. Hub untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Belt (ensureUserdataMounts) + FileBrowser sync skip ensure/mount when an external
drive root is not a live mountpoint -> no 'mkdir userdata: permission denied' + no
rootfs-shadow during a drive-absent window. System/local path never gated. Reuses
system.IsMountPoint; matches planDriveGates external-only rule. T1-T4 + red-proofs.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
DeployStack's initial compose-up builds env from deploy values (not stackEnv), so
v0.66.0 missed USERDATA_PATH on first deploy → ${USERDATA_PATH} resolved to '' and
Docker bound a root-owned dir at the container root (found live: radarr /media/movies
was 0:0 755). Shared withUserdataPath injector now used by stackEnv AND
composeExecWithEnv. Regression test included.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
agentapi.Decommission + handleStorageDecommission (migrate-all-or-none, Change 2):
migrate-then-decommission via the migration done-hook, or decommission-anyway (stop
apps, keep HDD_PATH). 'Hiányzó tárhely' badge on dashboard/stacks/app card when an
app's drive is decommissioned/disconnected/absent. Change 4: registerStoragePath
clears the decommissioned marker on re-enroll (ClearDecommissioned had no callers).
Non-hollow tests incl. mutation-proven Change-4 companion.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
internal/stacks/migrate.go: crash-safe, resumable namespace migration over the
controller's /mnt RW mount. Two entry points (whole-namespace + per-app) share one
journaled pipeline: validate -> stop -> copy (rsync -a --checksum, additive; conflict-
merge walk for non-app content) -> verify -> flip+redeploy (RedeployFromEnv) -> cleanup.
CLEANUP (the only destructive step) is gated on all units verified AND all apps
redeployed. Single-flight; mutual exclusion with the backup orchestrator (Change 3).
Non-hollow tests incl. mutation-proven collision + cleanup-gate companions.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The cgroup-only approach was a no-op on the demo: the controller container's OWN
cgroup is unlimited (the 2GB cap is on the LXC ancestor, hidden), and /proc has no
lxcfs, so it kept reporting the host's 16GB. The Docker daemon runs IN the LXC, so
'docker info' MemTotal reports the guest's real cap (2048MB) — now the authoritative
source (cgroup limit preferred when present, e.g. non-nested). The deploy memory
guard now uses the controller's committed-app memory (sum of running mem requests)
for 'used' — accurate and cheap — instead of host /proc RSS (unobservable per guest;
would make the guard never/always fire). /api/system/info reports the guest cap as
total and committed memory as used. Tests: cgroup-limit path, docker-info fallback
(nested case), GuestMemTotalMB fallback (dockerMemTotalFn stub).
deploy.go, one slice (both edit SaveAppConfig / the deploy goroutine):
CTRL-T2-1 (ghost-deployed on crash): DeployStack wrote app.yaml Deployed:true to
disk BEFORE the async 'docker compose up -d'; a crash during the image-pull
window left a ghost-deployed stack (Deployed:true, no containers) that DeployStack
then refused to redeploy. Now the env is persisted with Deployed:false
(transitional), and Deployed:true is written by runComposeDeploy ONLY after up -d
succeeds. In-memory Deployed stays true during the pull to preserve the
no-stale-Telepítés-button UX. On a post-success save failure, revert so the stack
is redeployable.
H10 (plaintext secret on encrypt failure): SaveAppConfig logged a WARN then fell
through to persist the secret in PLAINTEXT. Now fail-closed: return an error on
crypto.Encrypt failure, never write plaintext. Callers already propagate it.
Regression tests: H10 fail-closed (+ good-key encrypts) and the CTRL-T2-1
transitional durable-state contract (transitional reads not-deployed).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Restore recreates an app from its on-drive unit + the guest's own secrets,
regenerating nothing. reconcileRestoreSecrets (pure, unit-tested) merges the unit's
non-secret env with secrets recovered from the live app.yaml and FAILS CLOSED if a
data-encrypting key is unrecoverable (refuse — a PBS whole-guest restore is needed —
rather than regenerate and corrupt). Resettable secrets missing → warn + proceed.
- backup: RestoreFromRecoveryUnit (manifest -> recover secrets -> gate -> restore
volumes -> recreate definition + redeploy w/ re-pull); falls back to volume-only.
- seams: RecoverStackSecrets/RecreateStackFromUnit (adapter +encKey),
stacks.RedeployFromEnv. Wired into /backup/restore.
- tests: gate (refuse/proceed/verbatim) + data_key parsing.
Gate + reconcile + data_key parsing unit-tested; capture live-validated (v0.53.1).
Full readable-data e2e vs AdventureLog needs the auth-gated dashboard restore — pending.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
admin