admin
ebfd0967c1
gates / gates (push) Failing after 12s
Two whole_guest_backup_failed alerts at 04:30 and 04:32 CEST were one incident, and not on either customer box: ep0's proxmox-backup-proxy was active, holding its listening socket, and accepting nothing. Root cause: accept() returning EMFILE. The process held exactly 1024 fds -- its systemd-default soft RLIMIT_NOFILE -- of which 1016 were sockets and 547 connections sat in CLOSE-WAIT. The 1024-deep accept backlog had overflowed (Recv-Q 1025), so every client timed out. It was wedged from its own loopback too, which is what moved this from a network problem to a process problem. Fed by ~85k requests/day (a flat 3,538/hour) against an endpoint written to weekly, that leak reached the ceiling in 14 days of uptime. Fix: LimitNOFILE=65536 drop-ins for both PBS units, restart, verified from both boxes (200 in ~0.1s, felhom-pbs active), then re-drove the missed backups through the product path -- POST /backup?target=felhom-pbs on each agent's local API, not a hand-run vzdump. demo-felhom ct/9201/2026-08-18T03:57:43Z 4.10 GB 36.4s demo-hp ct/9201/2026-08-18T03:58:43Z 4.29 GB 41.5s Both host reports now carry felhom-pbs success=true, so the hub is green on the evidence rather than on a restart having been performed. No data lost, no backup skipped: the daily local tier was never affected and the PBS tier is weekly, so the window cost exactly one attempt. Evidence copied off ep0 BEFORE the restart, per standing rule 5. Filed: R-336 (the ~1 req/s poll rate is the real defect; the raised ceiling is mitigation, not a cure), R-337 (a status endpoint that trailed its own artifact by minutes then caught up -- WATCHING, downgraded from the defect I first wrote, because it self-corrected), R-338 (demo-hp is not on the R-50 island at all and nodes.md says it is; its local API is bound to the customer LAN). R-334 updated: still open, now one version wider (controller 0.216.0 vs golden 0.214.0). golden-currency is the only failing gate and is inherited -- it reads files this session did not touch -- so this push used --no-verify, stated per .claude/rules/gates.md. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016p1PTCzb8rF5G9Aa1qhpBN
Felhom — Documentation
Felhom is a managed home-server service for Hungarian households, built on a three-component model over Proxmox:
- Hub — operator backend on k3s (
hub.felhom.eu). Repo:felhom.eu/hub/. - Host agent — one per Proxmox host; operator-tier; owns all Proxmox interaction. Repo:
felhom-agent/. - In-guest controller — one per customer LXC; Docker-only; manages the customer's apps. Repo:
felhom-controller/.
This directory is the central, code-verified documentation home for all three components plus the platform and the security-audit record.
Sections
Controller (in-guest) — controller/
The Docker-only app-domain controller. Full per-area docs grounded in current source (v0.59.0).
→ controller/README.md: module map, deploy & stack lifecycle, backup
architecture, storage/monitoring/metrics, auth/hub/sync/integrations.
Where we stand — architecture/where-felhom-stands.*
The operator's one-page picture of what is proven, built, partial and missing.
architecture/where-felhom-stands.html— generated; do not hand-editarchitecture/where-felhom-stands.yaml— the data behind it; every claim cites its source. Gate:scripts/check_stands.py; regenerate withscripts/render_stands.pyarchitecture/where-felhom-stands-2026-08-09-snapshot.html— a dated snapshot, NOT maintained. The original React bundle, kept for the record; its statuses are those of 2026-08-09 before the verification pass
Host agent & platform — architecture/, proxmox-platform.md
The operator-tier agent and the Proxmox platform.
architecture/01-topology-and-trust.md— topology & trust modelarchitecture/03-host-agent.md— the host agent (Go; v0.29.1)architecture/04-control-plane-authorization.md— signing, escrow, authzarchitecture/02-controller-module-map.md— historical v0.33 planning map; the live map iscontroller/module-map.mdproxmox-platform.md— Proxmox platform reference
Hub (operator backend) — architecture/05
architecture/05-hub-architecture.md— hub architecture (v0.11.0)
Security audits & remediation — audits/
audits/deep-sweep-2026-06-13.md— cross-repo deep audit (controller + agent) with remediation statusaudits/bughunt-reconcile-2026-06-13.md— reconciliation of the v0.30.3 BUGHUNT against current code + merged fix list
Spike & test findings — tests/
Per-slice spike/validation findings (phases 0–5, slices 7–10). See tests/.
Conventions
- Code-verified, not memory-derived. Architectural claims here are checked against the actual current source; if a claim can't be verified it is omitted and flagged, not guessed.
- Per-repo operational working files (
CLAUDE.md,CONTEXT.md,CHANGELOG.md,BUGHUNT.md,REPORT.md,TASK.md) live in their own repos — they are operational, not published docs. - Authoritative versions at last refresh: controller v0.59.0, agent v0.29.1, hub v0.11.0.