felhom.eu/documentation/runbooks/logging-conventions.md

Logging conventions (all Felhom components)

Codified by the 2026-07-11 observability pass (agent v0.83.0 / controller v0.116.0 / hub v0.46.0). The capture layers below make DEBUG remotely readable WITHOUT config flips — these rules are what make that safe and useful.

Levels

Level	Meaning	Examples
TRACE (controller, v0.120.0)	A periodic job's ROUTINE success — "nothing happened, again". Dropped from the debug ring at write-time (fix-6, CAMPAIGN-3) so it never eats the finite post-incident window. NEVER used for a failure or a state change.	[TRACE] [scheduler] job status-refresh: finished in 2ms, [TRACE] [stacks] refreshStatusLocked: stack "radarr" → state=running
DEBUG	Flow detail: phase steps, per-call traces, parsed values, verdicts with their inputs	netverify: /proc/mounts verdict mounted=false, [agentapi] GET /storage -> 200 (12ms)
INFO	State changes and operations with durations ("X done in Yms")	network storage added + verified: … in 4200ms, operator log pull served
WARN	Degraded-but-continuing	journal unavailable — degraded classification, status persist failed
ERROR	A failed operation — always with the underlying error	rollback RemoveNetworkMount failed: …

Minimum standard per surface: entry, decisions, outcome + duration, errors. Every background job logs start → outcome → duration; every refusal logs the category it returned.

Language

Logs are operator-tier: English. The UI is Hungarian; logs are not UI. (Customer-facing messages derived FROM log categories are mapped to Hungarian at the UI layer, never in the log.)

Secrets

• Keys, never values. Log password=[REDACTED], a creds-file path, a token's length — never the credential, token, or key material itself.
• Journal-line content only as the bounded classification tail already established (netverify).
• Never log signatures, nonces, or one-time secrets — op class/host/expiry only.
• Belt-and-braces: the hub's bundle store runs a token-pattern grep gate before storing a pulled log bundle; a hit stores NOTHING and flags blocked: possible secret. [REDACTED] shapes and public checksums/fingerprints pass by design.

Capture layers (how DEBUG reaches the operator)

• Controller: the 5000-entry LogBuffer ring ALWAYS exists (raised from 1000 in v0.120.0 — fix-6); stdout respects logging.level via LevelFilterWriter. New leveled lines use internal/logx (Debugf/Infof/Warnf/Errorf). Legacy isDebug()-gated call sites are grandfathered. TRACE lines are dropped from the ring (periodic-noise policy). The ring spills to <DataDir>/debug-ring.log (SSD state dir, atomic JSON-lines) every 30 s + on shutdown and loads back on boot, so a restart / container recreation preserves the pre-restart window — the campaign's exact evidence-loss gap.
• Agent: applog.New fans out — stderr/journald at the configured level, the applog.Ring fixed at DEBUG. Served by GET /debug/logs (local API) and the heartbeat log pull.
• Hub: plain leveled Printf; the hub is directly operable, no ring needed.

Remote pulls (sovereignty posture)

Pull-only, box-pushed on its own cycle: controller ring via the report ACK (controller_log_requested → controller_log_tail), agent ring via the heartbeat envelope (log_tail_requested → log_tail). Consume-once on both ends; a failed push re-arms. Serving a pull logs the customer-visible operator log pull served INFO line — a pull is never silent. Bundles live 72 h on the hub, then purge.

Adding log lines (checklist)

1. Pick the level from the table — flow detail is DEBUG even if it feels important (the ring holds it).
2. English, keys-never-values, include duration on outcomes.
3. Controller: use logx.*; agent: the injected *slog.Logger.
4. For a NEW flow: add/extend the S7-style log-sequence smoke test (the ordered phase markers are the executable definition of "an operator can reconstruct this flow from the debug view").