R-97a: internal/quiesce had no route to the hub at all — three failed whole-guest
backups on 2026-07-27 produced zero events. TierNotifier is a seam (not an import),
wired by an init-only setter because main.go builds the notifier after the loop.
Edge-triggered: the failure fires when the R-88 breaker ARMS, not per retry, and
recovery rides recordSuccess's existing bool. Uses NEW operator-only event types;
reusing backup_failed would have emailed the customer in Hungarian about a backup
they cannot act on, since it has a customerMessages entry and is in live
enabled_events. Requires hub >= v0.78.0.
R-97b: v0.164.0's state filter cannot see an app caught MID-RESTART, which is how
BookStack alarmed. The fix is a suppression window keyed to the quiesce CYCLE,
consumed at the same single derivation point. 180s grace, derived from the deploy
flow's 120s health timeout and Mealie's 60s start_period; it expires, so an app
that genuinely fails to come back still alarms.
The quiesce loop is silent by design when nothing is due, so 'no error lines' is
not evidence. The observable that is: every /backup/due makes the agent read that
tier's storage content, which lands in pveproxy/access.log. Both tiers polled
every 5 min, both HTTP 200, tick phase consistent with the deploy restart.
Includes the Phase 0 findings: the nil is agent-side (case a, a type boundary
that cannot represent 'unknown'), the four restarts were deliberate rather than
self-update, and the loop produced zero backup_failed events because quiesce
never imports notify (filed R-97).
internal/quiesce had no failure counter, no backoff and no breaker, and the driver
is a plain 5-minute ticker — so a tier that was due and kept failing stopped and
restarted every customer app stack every 5 minutes indefinitely. Live on
demo-felhom 2026-07-27: three cycles in eleven minutes against an unreachable PBS
tier; it ended only because PBS recovered.
The breaker gates the QUIESCE, not the backup — the harm was the outage taken to
attempt it, so backed-off tiers are dropped before any stack is stopped. Per
target (a broken offsite tier must not suppress a healthy local one), 15m→30m→
1h→2h→4h capped, reset on success, never permanent, never applied to TriggerNow,
and stillRunning is not a failure. State is in-memory on purpose: forgetting a
backoff costs one attempt; persisting one could outlive the fix.
Part 3 invariant recorded on scheduledRunAllowed — a missing value means UNKNOWN,
and only a positive 'never' may fire the safety valve. Fourth instance of the
class (hub v0.12.0, v0.73.0, R-81, R-88).
Part 2 (unknown != never) is NOT in this commit: the agent returns byte-identical
responses for 'read errored' and 'never backed up', so the controller cannot tell
them apart. That needs an agent wire change and is tracked separately.
The section read 'NOT deployed'. True when written (17:56 CEST), stale 26 min
later when I deployed (18:22 CEST) and did not update it. REPORT.md is defined
as 'overwrite with the most recent state', so an artifact this project validates
against was left asserting the opposite of reality.
Precision matters on the apparent contradiction: a 17:01 quiesce log 'before' a
17:56 commit is a TIMEZONE artifact (controller logs UTC, git commits CEST), not
evidence of a false claim. In one clock: REPORT 17:56 -> deploy 18:22 -> quiesce
19:01. Consistent. That host=CEST/controller=UTC mismatch is a recorded project
trap and caught me once earlier the same day.
Also records the live multi-tier quiesce evidence and what remains unproven.
The multi-tier path only engages against an agent serving /backup/tiers, and the
agent rollout was still mid-flight, so both production controllers remain on
v0.173.0. Names the three things that are therefore unproven live.
Operator ruling 2026-07-26: let the first backup run as long as needed; other
backups shouldn't start until finished.
A first FULL offsite snapshot runs for hours, far past max_quiesce. When that
bound elapses the app resumes (unchanged), but the loop then started the NEXT
tier while the first was still uploading. Now it breaks and defers the rest to
a later poll — vzdump still holds the guest lock, so the second start would be
refused by the agent (409, v0.99.0) or fail on the lock, and a failed backup
never satisfies a cadence, so the tier would retry into the same wall forever.
pollTier returns (phase, stillRunning, err). The app still resumes exactly once.
Red-proof observed and restored; full suite green.
MinAgent UNCHANGED — degrades gracefully against ANY older agent.
The agent gained per-target tiers in v0.97.0. The controller owns quiescing,
so the multi-tier schedule is reconciled here: every due tier is collected up
front and run inside ONE quiesce window (one stop, N sequential backups, one
resume). Two cycles on the weekly night would mean two app outages for one
night's work.
Dedup rule: local-only -> one quiesce; PBS-only -> one quiesce; BOTH due ->
ONE window with both backups inside; neither -> no quiesce.
- quiesce.TieredBackend + BackupTier + ErrTiersUnsupported (optional extension)
- agentapi: BackupTiers/BackupDueFor/StartBackupFor/BackupStatusFor;
targetQuery("") yields an EMPTY suffix so untargeted hits the pre-R-82 route
byte-for-byte
- Loop.resolveDueTiers = the dedup rule in one place, agent order preserved
- quiesceAndPollTiers + pollTier: app stays quiesced until the LAST tier
snapshots (resuming earlier loses app-consistency on the DR tier). Consequence
stated in the docs: both-due-night downtime = first tier's full backup + last
tier's snapshot, which is why tiers run fast-first.
- Manual 'Mentes most' covers EVERY tier, due-ness ignored.
- Window-gate safety valve now uses the OLDEST due tier, so a stale DR tier
cannot be starved by a fresher local one.
Capability detection: /backup/tiers 404 = pre-R-82 agent (the documented
route-probe mechanism). Not a featureProbes row on purpose — the loop needs the
tier LIST, not a yes/no. Degrade logged exactly once per process.
Tests +11, full suite green. Red-proofs #2 and #3 observed and restored.
Source: felhom.eu/documentation/audits/DIAG-agent-channel-2026-07-26.md
bootstrap.DetectEndpointDrift names a controller.yaml vs bootstrap.json
local_api.endpoint divergence -- one ERROR carrying BOTH values and BOTH paths,
its own event type local_api_endpoint_drift, and its own Hungarian banner shown
ABOVE the channel banner because drift is the cause and "agent unreachable" the
symptom. It writes NOTHING: reconciling from bootstrap.json would clobber a
correct controller.yaml on any half-provisioned or hand-repaired guest, so the
authority ruling is deferred to R-78. Fail-safe silent on absent/unparseable/
incomplete bootstrap and on an empty endpoint (ensureLocalAPI's fill-if-missing
path is untouched). Fingerprint compared as a BOOLEAN only; token never
compared, logged or exposed.
EffectiveProtected now gates samba on Enabled && UserSet, mirroring BOTH of
reconcileSambaAt's early returns, and the doc comment is corrected in the same
change -- it claimed "detection and deployment agree in both directions" while
citing only !smb.Enabled, an assertion that went false when !smb.UserSet was
added. Not over-suppressed: sharing on WITH a password and a dead container
still alarms.
Channel log: the debounce placeholder is stateUnconfirmed (rendered "unseeded")
instead of "up", so a born-down channel no longer logs "up->down" and orUnseeded
stops being dead code. Logging only -- the placeholder is still matched in the
re-arm condition, so F2 born-down alerting is byte-for-byte unchanged and all
nine pre-existing channelhealth tests pass.
Tests 951 -> 959, all green. Red-proofs A (both directions), E and F.
MinAgent unchanged; felhom-agent untouched.
Found on the demo-hp live leg: with import, import/paperless and import/calibre
in the carry-list, the derived skeleton RE-CREATES a per-drive drop-zone on every
drive forever — the dead lookalike the canonical root exists to remove, and one
that is never backed up (class: excluded).
Not a zero-removals violation: nothing deletes what an existing box has. Both
demo boxes' old drop-zones were verified to hold zero files before the change.
Found on the demo-felhom deploy leg: ensuring only <sysNS>/userdata/import left
its parent at 755 root:root, because EnsureUserdataDir MkdirAll's intermediates
at plain 0755 and chmods only the leaf. That made the system drive's userdata
root the one on the box outside the 2775/gid-1000 convention.
${IMPORT_PATH} = <system namespace root>/userdata/import — ONE drop-zone per box,
on the system drive, injected at BOTH compose-env builders with NO per-drive
fallback (unresolvable leaves it unset so compose fails loudly rather than
quietly building a second, dead drop-zone).
Third BindRoot (RootImport) + Import list in BackupSpec, extended through
ValidateBackupSpec/ClassifyBinds. Load-bearing: a stale `userdata: import/<app>`
entry against the moved bind would be a WHOLE-BLOCK reject, taking the app's
mandatory hdd classification with it.
Exhaustive-root audit: resolveAbs/structuralGuard/ComputeCaptureSet/
ComputeFabBuckets now take importRoot explicitly (an import bind resolved
against hddPath would name a directory on the wrong drive); unresolvable is
refused loudly into Skipped. GetImportRoot added to both provider interfaces.
Catalog-derived skeleton: UserdataSkeleton() -> UserdataSkeletonCarry() +
BuildUserdataSkeleton(), SORTED. The carry-list makes zero-removals true by
construction (`documents` is in no catalog app but on both boxes) and is the
fresh-box floor. The sort is not tidiness: the naive map-order derivation
measured 20 distinct outputs from 20 identical runs, which with fbNeedsRecreate
is a fleet-wide FileBrowser restart loop.
One authoritative compose parser: ParseComposeUserdataMounts now delegates to
ParseComposeClassifiableBinds. Import root excluded from per-app migration.
Surfaces: FileBrowser /srv/beolvasas source; app-page "Hova tegyem a fajlokat?"
with PathEscape deep links (never QueryEscape) and class-driven copy;
data_paths: annotation with the Fork-3 asymmetry; system-owned beolvasas SMB
share refused server-side at handler AND store, button omitted in template.
Caught on the way: the sharing template's row struct was function-local, so
adding {{if .System}} would have 500'd every share row. ShareRow is now
package-level and the render test uses the handler's own type.
Tests 915 -> 949, all green. MinAgent unchanged.
agentapi.SmartSummary.ModelName mirrors the agent's model_name; the card row label
prefers the device model over the raw name/UUID, falling back to Name(+hint) on an
old agent. Additive. Test + red-proof (drop fallback -> A4 fails).
/ now 302s to /launcher (ONE canonical URL per page — launcher body never served at /);
the Vezérlőpult keeps /dashboard + its nav slot (operator ruling, reversing v0.163.0).
Nav: Indítópult active on /launcher, Vezérlőpult href=/dashboard active there. Post-login
default (/) and the topbar logo (/) flow through the redirect; login target unchanged.
Repointed 2 dashboard-card tests /→/dashboard. Corrected the stale vacation/agent-DOWN
note in CLAUDE.md (agent is up at 192.168.0.162:8443).
Tests: 302 target+status, /dashboard 200, nav hrefs/active; red-proof verified.
gofmt -w across the controller tree (46 files) so gofmt -l is empty — disarms the
formatting landmine where a targeted edit + accidental gofmt -w swept ~46 unrelated
files. Pure formatting: whitespace + gofmt's optional-semicolon removal in reflowed
inline closures. One doc comment reworded ('' -> 'the empty string') to avoid gofmt's
Go-1.19 doc-comment typographic substitition ('' -> curly quote) muddying its meaning.
No build/vet/test behavior change.
The agent defaults SMART to UNKNOWN on non-physical targets, so they showed as
spurious 'Nincs adat' rows. isPhysicalDisk now excludes those types (card + check).
Test strengthened: a PBS/LVM fixture with UNKNOWN SMART must still be excluded.
Consumes the agent v0.94.0 smart payload (MinAgent floor unchanged; feature-detect
by presence). One pure verdict fn agentapi.DiskVerdictFor shared by the dashboard
card and the 6h check. Card via a 60s /disks TTL cache (anti-smartctl-storm);
unreachable agent -> Nincs adat, page never blocks. disk-health-check (6h) emits
disk_health_degraded on a degradation only vs an in-memory baseline (first run
silent, recovery/UNKNOWN never notify, multi-attr -> one event). No global banner
(deliberate). Pairs with the hub allowlist bump.
Tests: verdict table (>=90 red-proof), notifier emit, check first-run-silent
(red-proof), degradation-once, recovery-silent, UNKNOWN-excluded, FAILING-critical,
nil-smart card, TTL cache.
ONE setting (window start W) drives every nightly leg at fixed, never-stored
offsets: DB dump at W, tier-2 at W+60m, off-box at W+105m (wrap-safe). Precedence
settings > controller.yaml db_dump_schedule > 02:30.
- scheduler.UpdateDaily: retime a daily job at runtime (no restart) via a per-job
buffered resched chan + a select case in runDailyJob.
- new pure package internal/backupwindow (LegTimes/GateWindow/EffectiveWindow).
- quiesce disk-tier window gate: scheduled cycles run only inside [W+2h,W+6h) with a
safety valve (age>cadence+24h runs regardless); manual TriggerNow never gated.
Backend.Due now also returns the backup age (from the agent's own /backup/due).
- backup page: Mentési időablak card (time input + derived leg/gate rows); POST
/backups/window validates -> saves -> UpdateDaily x3 -> flash.
Tests: 5 groups, all red-proofed. Agent/cadence//backup/due untouched.
.sidebar-logo was left-aligned; add auto side-margins so the 140px logo centers in
the header (desktop sidebar + mobile drawer share the element). Adds TestSidebarLogo_Centered.
Viktor pushed the text-outlined logo.svg to felhom.eu main (be9edb4) — wordmark is
now 17 real path glyphs. Swapped FelhomLogoSVG to it; stripped Inkscape's leftover
empty <text/> shells + font-* style leftovers on the paths via lxml DOM pass (glyphs
untouched, no text-to-path by CC), dropped editor namedview. Cleaned FelhomFaviconSVG
vestigial <text>. Both constants now free of <text>/font-family; viewBoxes unchanged;
palette + gradients preserved. TestLogoSVG_NoLiveText committed green.
Mobile nav was broken — the <=768px block predated the v0.146.0 accordion and
flattened .nav-links into a horizontal overflow-x strip, clipping the accordion's
nested sub-lists. Replaced with a sticky top bar + off-canvas left drawer that
reuses the vertical sidebar (accordion untouched), plus a no-js static fallback.
Removed the sidebar customer-name span (kept on login). Added ?v={{.Version}}
cache-bust to logo/favicon URLs (Cloudflare 4h edge-cache; 0.126.1 failure mode).
Part 4 (outlined-logo constant swap) gated out per §3a: live felhom.eu main still
serves a logo.svg with live <text>/font-family; constants unchanged.
5 new tests via the real layout/CSS render; nav_accordion invariants unchanged.
Feature-detected navigator.share opens the OS share sheet with title+text+URL only
(no QR files: — narrow support drops the URL when given file+URL). Hidden unless
supported; Link másolása stays the universal fallback and catches the non-cancel
rejection; AbortError is silent. Template JS + tests only. 2 red-proofs verified red.
Mint a 160-bit capability URL (/s/<token>) serving a standalone read-only guest
launcher: same tiles, opens apps in new tabs, no account, no admin session.
Information only, zero control — every privilege stays behind each app's own auth.
- /s/ pre-auth pass-through (after the claim gate) + session-CSRF exemption; guest
password POST carries its own pre-auth HMAC CSRF.
- Constant-time token match; empty stored token = disabled = byte-identical mux 404.
- Optional per-share password: separate bcrypt hash + own attempt map; signed cookie
= HMAC(token|passwordHash) keyed with web.session_secret, so rotate/change invalidates.
- Guest labels ride the v0.164.0 ruling; never expose internal state vocabulary.
- Token redacted in logs (/s/<redacted>); never in CHANGELOG/REPORT/CONTEXT.
- Admin modal: copy-link, QR (go-qrcode), set/clear password, rotate, disable.
- Tests: Groups A-G (14) + 3 red-proofs verified red.
A UI stop (Leallitas -> compose down -> StateStopped) is the user's own
action, not a fault, and must not raise the deadapp banner OR the
app_start_failed event. Filter at the single fix-3 derivation point:
extract scanDeployedAppRunStates's pure core to classifyRunStates and
change the down predicate to IsDownState(st.State) && st.State !=
StateStopped. Suppresses StateStopped from both the banner dead-list and
the notifier Down-set at once.
Rests on two invariants (recorded at the seam, README, CONTEXT):
I1 StopStack = compose down => zero containers => StateStopped
I2 P2 census: all catalog services unless-stopped => faults never rest
at stopped (they surface as exited/degraded).
IsDownState unchanged; out-of-band 'compose stop' (containers remain ->
exited) still alerts. Tests +4 (notify 3->4, main 4->7), both red-proofs
verified. No template/funcmap/notifier/counter/copy change.
(A) The launcher monogram bled through every transparent white logo — it rendered
unconditionally under the img. Now hidden by default (.launch-mono display:none),
revealed only when the img chain fails (onerror adds .launch-tile--noimg).
(B) The /static/app-placeholder.svg default reached only app_list_row. The four
other sanctioned app-logo onerror chains now match the canonical SVG->PNG->placeholder
grammar: backups_apps (aligned row), stacks (infra->infra-logo else placeholder),
app_info hero (screenshots still vanish), deploy (keeps .LogoURL/.LogoPNGURL).
Template/CSS only; no handler/funcmap change. 5 tests + 2 red-proofs.
New /launcher page: a grid of large tappable tiles, one per openable deployed
app (subdomain presence is the single openability criterion, shared with the
Megnyitás button via the extracted Server.subdomainMap helper). Colored tiles
(deterministic slug color or .felhom.yml brand_color), white glyph/monogram
fallback, target=_blank links for operational apps, greyed unclickable tiles for
stopped ones. First sidebar item; / stays the Vezérlőpult.
Universal app placeholder: new AppPlaceholderSVG served at
/static/app-placeholder.svg, now the default FallbackIcon on app_list_row so a
logo-less app shows a placeholder instead of visibility:hidden. Brand mark is
never an app placeholder.
New Metadata.BrandColor; new funcmap tileColor/initial. 10 new test functions +
4 red-proofs. No agent coupling; MinAgent unchanged.
demo-felhom 9201 + demo-hp 9201 (G1 break-glass, secret shredded) both clean +
healthy on 0.162.0 with the settle-gate GO line. Honest B' accounting recorded:
the floor is in-memory (not persisted) so both above-floor boxes logged
"awaiting floor knowledge" then GOed ~10s later once the report ACK landed —
the report-ACK latency the 90s sub-bound is sized to, not a regression. The
zero-wait-when-floor-known invariant stays unit-proven (test E).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N7Drmtm2RzoqbkJZCNSFNQ
The day-0 race (DIAG-f10): a fresh box boots below the operator floor, the
apply-bridge consumes the single-use offsite password, then ~35s later the
managed auto-floor update replaces the container mid-install -> the new process
finds no installed key -> consume -> 404 -> offsite dead until an operator
Re-issue. Recurs on every onboarding whose ISO floor lags the managed floor.
Ordering-only fix (consume/install/persist internals + the 404-no-oracle
contract + the Consumer UNTOUCHED; R-71(b) rejected-by-design):
- New seam offsiteapply.SettleProvider.SettleState() + SettleFunc adapter over
the self-updater's own GetFloor()/IsUpdateRunning() (no second floor path).
- Bridge.AwaitSettle polls 10s BEFORE the 3-min Reconcile ctx: defers while an
update runs or the box is below the known floor; GOes at/above floor on the
first poll with zero added latency (B'). Bounds 90s floor sub-bound / 5min
overall, both GO+WARN (hub that can't serve a floor can't serve a consume ->
no burn risk; R-71c is the belt). ReconcileWhenSettled = gate then reconcile.
- main.go: bridge goroutine moved after the updater is built; wired only when an
updater exists (nil Settle = reconcile immediately, old behavior).
Finding: the floor is in-memory (report-ACK ~5-10s), NOT persisted -> unknown on
any restart until the first ACK; the 90s sub-bound is sized to that.
Tests (injectable clock, fake SettleState, recorded Consumer): A-E + nil-provider
+ cancelled-gate. Four red-proofs all observed FAIL then restored: gate removed /
updateRunning branch / floor sub-bound / overall bound. Deferral paths ship
unit-proven + red-proofed, NOT live-fired -- their precondition is now
structurally prevented by the v1.25.0 build gate. Layering: gate prevents, (a)
defers, (c) heals.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N7Drmtm2RzoqbkJZCNSFNQ
Network shares bind their share ROOT :rslave into FileBrowser — no
skeleton, no userdata scoping, nothing written toward the NAS. Gate is
the stub classifier (stub ⇒ excluded from mounts AND sources — an
exposed stub swallows uploads the real mount later shadows); idle autofs
is healthy and included. Drives byte-identical. Add/remove trigger the
debounced sync. Phase-0 probe on demo-hp: GO (in-container rslave access
wakes the idle trigger). Red-proofs A + B run.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UuFPHmHNrCJj1VhY6QdDMU
Leg A: „Hálózat" card on Beállítások → Rendszer — Helyi cím (LAN),
Hálózati név (only while Megosztás is enabled), Átjáró; live per render,
stored nowhere (S-5), „—" on unavailable.
Leg B: network section in the Debug system dump (interfaces/route/DNS/
lan_address), best-effort per item via the samba-netns door.
Leg C: NetBIOS trap named — Szerver field helper text + a purely lexical
hint on unreachable failures for single-label non-IP names.
Design note: all guest-net reads go through docker exec into the
host-networked felhom-samba container (stacks/guestnet.go, one seam) —
the controller's own netns is the docker bridge, so /proc/net/route etc.
would answer 172.x (the S-2 trap). Red-proofs: A2 gate-drop and C2
lexical-invert both failed as required.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UuFPHmHNrCJj1VhY6QdDMU
admin
Claude Code