admin
6dab0f6a2c
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NKSN3gSg4TKVBBqkwW2djR
132 KiB
ROADMAP — future features & open work
What this is: the prioritized decision log of planned/open work. Items are intentions, not claims about live behavior — the capability map (
architecture/00-capability-map.md) is the only place that states what the platform does today.Lifecycle: idea → spiked → spec'd → in-progress → shipped (item collapses to a one-liner with the version, and the corresponding capability-map row changes status with evidence). Items can also be killed (keep the one-liner + why — decisions are worth remembering).
Coupling rule: every item names the capability-map row(s) it flips. Every map gap row points back here by ID. Neither file duplicates the other's content.
Priorities: P1 = closed-alpha blocker · P2 = close during alpha · P3 = post-alpha. Existing loose notes in this folder (
FOLLOWUP-*,FIX-M*) are absorbed as references below.
P1 — closed-alpha blockers
| ID | Item | Size | Status | Notes / map rows flipped |
|---|---|---|---|---|
| R-1 | Peti convergence — the appliance half is DONE; this item is now Peti-only. Rehearsal EXECUTED 2026-07-18 (tests/VALIDATION-n100-rehearsal-2026-07-18.md): the full final-product flow ran on real metal in one pass (RESET → generic ISO → customer self-bind → day-0 → floor lift → escrow ceremony → offsite snapshots), which retires the "supervised rehearsal" dependency that R-13/R-21/R-23/R-24/R-27/R-28 were all parked behind. Surviving half: Peti's clean-slate proxmox2 reinstall + the parked publish trains on a REAL REMOTE customer — the one thing a demo box on the operator's own LAN can never prove. |
L | rehearsal DONE; Peti half open | Flips: publish train PARTIAL→PROVEN-LIVE; appliance/BYO/day-0 "real customer" notes; escrow ceremony. The single biggest unproven surface — an alpha where fixes can't ship remotely is dead. Reinstall arc SHIPPED hub v0.57.0 (2026-07-16): the clean-slate reinstall-of-existing-customer path is now first-class — claim re-issue (F2), offsite re-issue (F3), escrow-honesty-on-re-issue (2.3) all auto-fire on re-enrollment. Peti's proxmox2 clean-slate now walks a supported path |
| R-2 | ~hub/internal/notify/, store.go, hub/internal/claim/) |
S | killed (2026-07-16) | Not a real issue: the "foreign WIP" was in-flight code from a concurrent CC session on the customer-claim arc, snapshotted before it committed. All of it landed cleanly — notify/+claim/engine.go in 6b40eb8 (v0.50.0), store.go in a1d0450 (v0.54.0), plus follow-up e205a2d; v0.55.0 shipped. Working tree is clean, no stashes. Lesson already codified: never run two writing sessions on one felhom.eu clone (CLAUDE.md §git add -A) |
| R-3 | Friend-alpha onboarding runbook (generalized from pilot/RUNBOOK-peti-return-2026-07-13): hardware prep → golden → install → claim → ceremony → "first restore by the customer" scripted step |
M | idea | Flips: "customer performs a restore" MISSING row; produces the tester-agreement sibling of PETI-tester-agreement.md. Next from-scratch rehearsal to include customer DELETE + re-create — the ESCROW cascade is now DEFINED (hub v0.60.1): host delete DEMOTES escrow to retained custody (never destroys), customer Danger-zone delete PURGES it (the one true purge point). S6b (manual stale-host delete before re-enroll) is OBSOLETE — re-enrollment upserts the existing host row cleanly (store.UpsertHost ON CONFLICT DO UPDATE; handleAdminCreateHost no duplicate refusal) + the v0.57.0 arc auto-fires the re-issues; the rehearsal live-confirms it. NON-escrow offboarding NOW ANSWERED by the middle-tier Customer RESET (hub v0.61.0, LIVE): one operator action deprovisions the Hetzner sub-account/box (repo data destroyed), destroys the PBS namespace + backup groups + token, clears the DR recipe / one-time secret / claim state / retained escrow custody (separate ack) — identity + basic config survive. WG peer release rides host delete (peers are host-scoped, gone before RESET runs — RESET refuses while any host row exists). Remaining consistency gap: the customer Danger-zone DELETE still leaves host rows and does NOT run the offsite/PBS teardown (RESET is the teardown path; DELETE is escrow-purge + config-drop). Decide whether DELETE should require a prior RESET (or subsume it) — new item R-25b |
| R-4 | Claim-code deliverability: test-send to gmail.com / freemail.hu; tighten DMARC p=none → p=quarantine |
S | DONE 2026-07-21 (all three halves) | A claim code in spam bricks onboarding at step 1. DMARC now p=quarantine (RUNBOOK-publish-0.90-0.143-2026-07-18 Phase 4): _dmarc.felhom.eu TXT v=DMARC1; p=quarantine; rua=mailto:admin@felhom.eu set by Viktor (CF, Path B); dig gate green (1.1.1.1 + 8.8.8.8). Alignment safe (SES/Resend DKIM d=felhom.eu + relaxed send. SPF). gmail half CLOSED by the rehearsal (2026-07-18): the run's claim email was the first sent under p=quarantine and landed in the gmail Inbox, not spam (tests/VALIDATION-n100-rehearsal-2026-07-18.md). freemail.hu half CLOSED — operator-verified 2026-07-21: a [Felhom] Teszt értesítés was received at a freemail.hu address. With the gmail half closed by the 2026-07-18 rehearsal and the DNS half by p=quarantine, R-4 is COMPLETE |
P2 — during alpha
Sub-rank
P2-HIGH= close before the first REMOTE tester. These are the 2026-07-18 N100 rehearsal's findings (tests/VALIDATION-n100-rehearsal-2026-07-18.md). They are not P1 — the rehearsal proved the product flow works — but each one either misleads the operator, misleads the customer, or hides a failure, and all of that gets materially worse the moment the box is somewhere you cannot walk over to.
| ID | Item | Size | Status | Notes |
|---|---|---|---|---|
| R-39 | [P2-HIGH] The PBS DR tier can be applied and dead at the same time — and nothing notices. On the reborn N100 the descriptor auto-provisioned and the agent converged state=applied, yet pvesm status reports felhom-pbs: error fetching datastores - 401 Unauthorized / inactive and a direct probe with the stored credential 401s on every endpoint including /version (WG transport healthy: handshake 9 s, 27.9 ms RTT — so authentication, not ACL scope). Three compounding defects: (a) a mint/consume race — the hub minted a SECOND token secret at 16:47:52, two minutes after the agent applied the first, and consumed_at is still NULL; (b) the converged state machine will not re-apply, so the box is pinned to a stale secret; (c) the agent's 15-minute PBS verify loop cannot read the credential to detect any of it (open /etc/pve/priv/storage/felhom-pbs.pw: permission denied — the non-root agent writes that file through a root sudo wrapper, then reads it directly). |
M | CLOSED 2026-07-21 — PROVEN LIVE (hub 0.68.1 + agent 0.91.2) | DIAGNOSIS (2026-07-18, live on the N100 — supersedes the initial hypothesis). The brief guessed "the re-mint fails to bump the generation". That is FALSE and no hub fix was shipped: store.SetHostDesired bumps desired_generation unconditionally (it went 2→3 on the re-issue), and web/configs.go's applyPBSDR is likewise exonerated — its "no re-key, no second secret, no spurious generation bump" comment is accurate, guarded by the cur != nil && cur.Namespace != "" early return, and the hub log shows mint #2 came from the re-issue path, not from an Edit-tab Save. The real mechanism is a signal mismatch between the two tiers. The hub's re-consume signal is a generation bump + a poke; the agent's re-apply trigger is a change in the DESCRIPTOR CONTENT HASH (felhom-agent internal/pbsdr/manager.go ~L235: `if mk := m.loadMarker(); mk != nil && mk.Hash == h && (cf == nil |
| R-30 | [P2-HIGH] Liveness presence should come from the wait channel, not the report clock. The box was powered off at the start of the rehearsal, yet the hub carried it as healthy until the staleness threshold expired ~30 min later (host_stale 16:05:24 "no report for 30m"; cleared 16:33:24 "was stale for 27m"). The host-delete guard compounds it: RESET refuses while any host row exists, so a stale-but-"Online" host stalls a forced teardown. |
M | idea | Direction: derive presence from Dir-2 long-poll connectedness (~90 s grace), decoupled from notification hysteresis (the hysteresis is right for alerting, wrong for presence); an agent/ep0 analog can follow. Pairs with R-13/R-23 — the transport already exists, this is about believing it. (Discussed in-session as "R-29"; that number was already taken by the gate-rot item earlier the same day, so it is R-30.) |
| R-31 | [P2-HIGH] Offsite provisioning is synchronous with no status affordance. Save runs the Hetzner sync in-request, so the request can hit the nginx 504 while succeeding server-side: the operator cannot tell failed from slow, and a retry races the first attempt. | M | idea | Direction: make it async + a status card, reusing the proven awaiting-card/poll idiom (v0.138.0 escrow card). Interim mitigation belongs in R-3 as an operator note: click once, wait, verify — do not re-click. |
| R-32 | [P2-HIGH] RESET must purge the customer base dir; the orphan card must stay honest; unattributed bytes must be visible. The rehearsal's S7 said in advance that an orphan card would BE a finding — and one appeared (16:58:14). Cause: RESET's "hetzner":"ok" leg destroys the sub-account, but a Hetzner sub-account is an access-control object, not a data object — its directory survives, so re-enabling offsite recreated an account over the previous lifecycle's ciphertext, encrypted under a key that same RESET had destroyed. |
M | idea | Ruling from the run (three parts, deliberately separate): (1) because RESET destroys custody, the ciphertext it leaves behind is unrecoverable BY DESIGN → RESET gains a main-account purge of the customer base dir (the existing operator ack already covers it); (2) the move-aside guard STAYS for reinstall-without-RESET — there custody survives and the card's "history recoverable" promise is true (R-26 depends on exactly that); (3) the operator Restic tab shows per-customer directory bytes vs attributed snapshot bytes, so dead data cannot hide. Measured on the pool box that night: 49 M attributed (2 snapshots, 48.717 MiB) against 1.4 G + 3.0 M unattributed across TWO .orphaned-* dirs. Evidence restic-and-pool.txt |
| R-33 | [P2-HIGH] Bootstrap pairing-poll spams the customer-visible console. Each 30 s pairing poll is a separate felhom-bootstrap.service run that exits 1, so systemd prints Failed to start … on the screen the customer is watching — 52 FAILED lines in ~11 minutes while nothing was wrong (the box was correctly waiting to be bound). |
S | SHIPPED (scripts v1.21.0, 2026-07-18) | run_pairing() is now a while true loop sleeping POLL_INTERVAL (30s — the hub-side rate is unchanged) instead of exiting non-zero per poll, so the unit sits in activating and systemd prints nothing. Registration split into register_appliance(), whose transient failures the loop retries rather than taking the unit down. The journal is quiet without going dark: the wait is logged once on entry (worded so nobody reads it as an error), then on a 10-min heartbeat with elapsed minutes; 410 still exits non-zero on purpose (delivery consumed but no local env is a real crash window). Console banner every 5 min instead of every cycle, single accented spelling, plus the reassurance the rehearsal showed was missing („Ez a képernyő magától frissül"). The load-bearing half is TimeoutStartSec=infinity in the unit — a Type=oneshot ExecStart is killed at DefaultTimeoutStartSec (90s), so without it systemd would kill the new wait and Restart=on-failure would silently reinstate the exact spam this removes, after appearing to work for the first three polls. Verified behaviourally in a container against a stub hub answering 204 five times then delivering: one log line plus one heartbeat, zero exits between polls, then a clean fall-through to the direct install and exit 0 — where the old design produced 5 unit invocations and 5 Failed to start console lines. ISOs rebuilt at v1.21.0 and the shipped payload verified byte-identical to the committed source. Evidence bootstrap-journal.txt; ship commit bcdb042 |
| R-5 | Hub: offsite storage visibility — RESTIC box aggregate (v0.64.0) + PBS DR datastore (v0.65.0), each with fill + operator alert; Offsite tabs + dual dashboard gauges | M | SHIPPED (hub v0.64.0 + v0.65.0 + tenantsync v1.2.0, 2026-07-17) | Restic: monitor.OffsiteBoxChecker (throttled Hetzner GET; fill 80/90% + oversub 2.0×; scope "pool-box"). PBS DR (v0.65.0, Option A): read-only usage op on the ep0 felhom-tenantsync forced command (df, no mutation) → monitor.PBSDRBoxChecker (fill 80/90%, states ok/unavailable/degraded, scope "pbsdr-box"); /offsite Restic/PBS-DR tabs + two gauges; hub deploy ⟂ ep0 update (graceful "n/a"). Both data pipelines PROVEN-LIVE (restic 0.2% full; PBS 19.1% full, 7.1 GB of 37.2 GB after ep0 → v1.2.0). Thresholds pending Viktor (named keys; separate restic/PBS pairs). OPEN: UI render unit-only (password-gated), alert emails unit + red-proof only (both nominal). See hub v0.64.0 + v0.65.0 REPORTs |
| R-6 | Spike: LAN service discovery from the guest — SSDP multicast (UDP 1900, DLNA), WSD (Windows discovery), mDNS; host-network vs macvlan; is the customer LXC LAN-bridged in appliance deployments? | M | spiked (2026-07-18) | VERDICT: appliance guest IS LAN-bridged (own DHCP lease on the household /24); multicast discovery works ONLY in the guest netns — guest-direct or Docker --network host (SSDP/mDNS/WSD all PASS both ways); the default docker bridge is categorically DEAF to LAN multicast (WSD/mDNS RX FAIL, unicast-publish PASS). Real samba+wsdd on host-net → Windows 11 ProbeMatch + FELHOM-SPIKE renders in Explorer + 445 + authenticated SMB round-trip all PASS; real SSDP MediaServer:1 advert reaches both LAN clients. → R-7 SMB stack MUST be host-network LAN-bound; R-8 Jellyfin-DLNA plausible if host-network. Caveat: vmbr0 multicast_snooping=1 worked only because the household router is a live querier — customer LANs w/ snooping+no-querier, and Peti's BYO bridge, are UNTESTED gaps. S4b (human leg, the sharpest finding): wsdd makes the box VISIBLE but the Explorer double-click FAILS 0x80070035 — WSD gives no name resolution; the flat \\FELHOM-SPIKE resolved by no path. Adding nmbd (NetBIOS) fixed it live (flat name resolves + mounts). → R-7 needs smbd+wsdd+nmbd (+avahi/.local for modern clients), not wsdd alone. Doc: audits/SPIKE-lan-discovery-2026-07-18.md. |
| R-7 | SMB server share — LAN file sharing as an embedded controller feature („Megosztás") | L | SHIPPED slice 1 (controller v0.144.0 + felhom-samba:1.0.0, 2026-07-18) |
Flips map row Files from Windows Explorer / Mac Finder (SMB server) MISSING → IMPLEMENTED. Delivered as the FOURTH protected infra stack (host-net, smbd+nmbd+wsdd, bind interfaces only = lo eth0), NOT a catalog app. „Megosztás" → „Hálózati megosztás": enable + ONE household password (STDIN→smbpasswd, never persisted — only user_set), shares table, create flow (new folder under <storage>/shares/ or an existing folder via a guarded picker). Picker guard = absolute → EvalSymlinks → containment in a registered LIVE root → deny-listed system subtree (derived from ProtectedHDDPaths, provably a subset) → is-a-dir, with UNIFORM refusals (no filesystem oracle). Per-share read-only enforced in smb.conf AND as a :ro bind. force user = felhom → SMB writes land as uid 1000. Live-validated on demo through the real endpoints + a Win11 workstation (445 True, NetBIOS FELHOM resolves, write/read byte-compare PASS, write to a read-only share REFUSED, WSD ProbeMatch). Explorer leg PASSED 2026-07-18 (Viktor): both shares open from the Network view; an Explorer save into the writable share landed as uid 1000 and a write into the read-only share was refused, folder untouched. Slice 1 is fully PROVEN-LIVE. Docs: controller/sharing.md; ship report felhom-controller/REPORT.md. Slice 2 remainder: avahi/.local for modern+Apple clients, curated app-folder presets (one-click paperless consume/), per-share users + guest access, recycle bin. Backup gap split out → R-7b. |
| R-7b | Share backup EXECUTION — put share data into the live tier-2 + offsite runs (the design fork reported by R-7 slice 1) | M | SHIPPED (controller v0.145.0, 2026-07-18) | Viktor's ruling: Model B′ — a SIBLING shares source. New, additive job/leg code reusing the proven primitives (tier-2 mirror seam, restic wrappers, soft-quota/enlargement gate, status recorders) while leaving every per-app engine path byte-identical — NOT a synthetic recovery unit (breaks on multi-drive shares, wraps 1 KB of JSON in dump machinery) and NOT engine-loop surgery. The B′ invariant is enforced by test in both tiers, red-proofed. Tier 2 → RunSharesTier2 (legs grouped by SOURCE drive → backups/secondary/_shares/<driveKey>/<share>, payload at _payload/, layout marker LAST). Tier 3 → runOffboxSharesLeg: ONE extra restic backup --tag felhom-offbox --tag _shares placed after the app loop and BEFORE retention, so forget --group-by host,tags covers the new group with no flag change; a quota-blocked push degrades to the manifest only, never to nothing. Restore → „Megosztások" on /backups/restore: scratch, then a missing-only merge whose every destination is PREFIX-ASSERTED against live storage roots, definitions merged existing-wins, then ReconcileSamba, then the credential. The payload (_shares-manifest.json + a best-effort secret-bearing passdb.tar) is what makes DR return files + configuration + password rather than loose bytes. Fold-in: samba joins the liveness set — EffectiveProtected adds the CONTAINER felhom-samba exactly while sharing is on. FULLY PROVEN-LIVE on demo (2026-07-18), all four legs. (1) tier-2: real /api/backup/tier2 trigger → _shares tree + marker + payload on the cross-drive target, mirrored file md5-identical, payload 0600 preserved. (2) offsite: Viktor's manual run 12:18:16Z → snapshot e0b9d723 (tags felhom-offbox,_shares) with the payload dir + both share folders; a second run via the „Távoli mentés" button → 4e2b15ec, containing _shares-manifest.json (418 B) AND passdb.tar (855 040 B), both 0600, share files with uid 1000 preserved. (3) restore round-trip: probe file + the dokumentumok DEFINITION deleted via the real endpoints, then „Megosztások" restore + place → 1 file(s), 1 definition(s) re-added, 1 kept, 0 refused, credential=true; probe back md5-identical, the two pre-existing files NOT overwritten (missing-only proven on live data), definition back with its ORIGINAL flags and created_at, smb.conf re-rendered, filmek untouched. (4) liveness: samba stopped → health_critical pushed and hub-accepted (200) → self-healed. Remaining human leg: SMB positive auth with the real household password (never persisted by design). Correction: an earlier revision of this row and of the ship REPORT wrongly claimed the demo box had no offsite target — the verification read a guessed settings key (offbox_target) instead of the real one (offbox); root cause dissected in REPORT §7b. Findings: the reserved-name assumption was FALSE (nbNameRe accepted „_shares" as a share name — now refused); the alert/e-mail pipeline needed NO change and adds no new event type. Docs: controller/sharing.md; ship report felhom-controller/REPORT.md. |
| R-8 | DLNA (gate input now exists — R-6 spiked 2026-07-18: SSDP reaches LAN clients from host-net): validate Jellyfin's built-in DLNA server first; only add minidlna to the catalog if Jellyfin-DLNA fails | S | idea (unblocked) | Don't add catalog weight before proving the cheap path. R-6 confirmed the cheap path is physically viable — Jellyfin DLNA must run host-network (same multicast constraint as R-7) |
| R-9 | Uninstaller trio (from 07-15 Peti session): cluster-aware felhom_guests guard (node-local pct list deletes cluster-wide pveum objects); saferemove detection + time estimate + opt-in --quick-remove (never mutate storage.cfg); smarter restore_storage default for BYO clusters (shared storage, not local-lvm) |
M | idea | Second item's rejected alternative (temp-disable-and-restore) stays rejected — crash window silently downgrades cluster wipe policy |
| R-10 | T-6E-1: DB-dump dir-fsync asymmetry (LOW, confirmed in 6E) | XS | idea | One-line hardening; batch with the next controller task |
| R-11 | Tester-facing one-pager: what the box does, known limitations, how to report | S | RULED 2026-07-21 (channel); doc is the architect's | Pairs with R-3. operator ruling 2026-07-21: the reporting channel is DIRECT MESSENGER to the operator — no group, no tracker, no form. Rationale: the expected tester count at first invite is ~0–few, and a channel nobody is on is worse than a channel everyone already uses. The one-pager and the tester agreement are the ARCHITECT's deliverable, not CC's — CC does not write them; it may be asked to fact-check claims in them against the code. Revisit the channel only if tester count makes direct messages unmanageable |
| R-16 | Operator hygiene: campaign6 autofs orphan (clears on host reboot) + tied-CreatedAt flash duplicates (audiobookshelf/komga/romm) | XS | open (doc-drift bit CLOSED) | Viktor's own action items from 6D/6E. Doc-drift leftover CLOSED (host-install v1.17.0, 2026-07-17): the R-20-noted stale "EMPTY by default" operator-key comment corrected (keys are PINNED). Remaining = the two operator items above |
| R-22 | PBS-DR pre-check self-grant (F4). On a non-default storage id the token-auth GET /storage/<id> pre-check 403s (no ACL yet) and used to abort before the root-run grant that creates it. |
S | SHIPPED + PROVEN-LIVE agent v0.89.0 (2026-07-17) | On a 403 the reconcile self-grants via the root wrapper + re-reads, then converges. Red-proof TestSelfGrant_PreCheck403DoesNotAbortBeforeGrant; live-reproduced on the demo (marker aside + ACLs revoked → self-grant → converged state=adopted in ~3 s, ACLs restored, offsite active). Origin tests/VALIDATION-n100-baremetal-2026-07-16.md F4. |
| R-17 | Old-box archive (u629193-sub1) retirement — 9/9 byte-identical restores verified | XS | CLOSED 2026-07-22 — archive deleted (operator console) | operator ruling 2026-07-21: delete the archive now; the 9/9 byte-identical restores are the evidence it is no longer needed. CC spike-lite (2026-07-21) — it CANNOT be deleted through existing tooling, and CC did not improvise credentialed API calls: (a) felhom-tenantsync.sh does not touch Hetzner at all — it manages PBS namespaces/tokens only; (b) the hub CAN delete sub-accounts (hetznerapi.DeleteSubaccount), but its ONLY caller is offsite.Deprovision, which selects by the felhom-customer=<id> label and runs as part of a customer RESET — a legacy archive sub-account from the 2026-07-09/10 spikes carries no such label, so no tooling verb targets it; (c) the only in-repo references are TEST FIXTURES (hub/internal/hetznerapi/fake.go generates the u629193-sub%d pattern; felhom-controller/.../dr_recipe_test.go uses it as a literal) — nothing live points at it. Operator console steps: Hetzner Storage Box console → box u629193 → Sub-accounts → u629193-sub1 → confirm the directory holds only the old-box archive → Delete. Deleting the sub-account destroys its data; the 9/9 verification is the sign-off. CLOSED 2026-07-22: the operator executed the console action — u629193-sub1 deleted, confirmed no sub-accounts remain on PBS-storage-1 |
| R-19 | Internet-outage customer-experience drill: pull WAN on demo, verify lan_resolver path, document what the customer actually sees/does | S | idea | Flips map row E "LAN access" IMPLEMENTED→PROVEN-LIVE |
| R-20 | XS | closed (2026-07-16) | Confirmed against scripts/felhom-host-install.sh source (not changelog): keys resolve at L1181–1219 (script constants OPERATOR_KEY_*, populated, --operator-pubkey-file override), pinned automatically by step_agent_config() "STEP 6/8" (L2044; python builds authz.signers L2146–2156, reinstall preserves existing), verified at L2332–2337 ("authz signers: N … operator-signed self-update armed"). No interactive prompt or post-install hand-edit — fully automatic. Doc-drift note: the L193–197 "EMPTY by default" comment is stale vs the now-populated constants (→ R-16 hygiene) |
|
| R-23 | Immediate-sync Direction-2 follow-ups (hub v0.58 / controller v0.140, 2026-07-16): (a) — BANKED 2026-07-21 (both legs). The operator-UI save->apply round trip is PROVEN: the STOP-2 global-floor save (hub 18:56:27 CEST) released the controller's held wait in the same second (16:56:27Z wait woke: generation=1 - firing out-of-cycle report), with the report built 2 s later; the ring also shows wait baseline generation=0 at startup (baseline recorded WITHOUT firing, as designed) then generation=1, so the generation advanced past 0. RESTART LEG BANKED 2026-07-21 — the floor was moved to a version the box did NOT run, and the swap fired EXACTLY ONCE. Operator saved global floor 0.153.0 → v0.154.0 (a real version boundary, unlike the 2026-07-20 attempt which targeted an already-running version and therefore proved nothing). Timeline (guest UTC): 06:57:13 UpdateState pending written initiated_by=auto-floor → 06:57:17 agent controller swap requested 0.153.0 -> 0.154.0 → 06:57:19 image file written, restarting bootstrap → 06:57:21 container StartedAt + UpdateState completed_at → 06:57:29 agent new controller healthy. 16 s end to end. Assertions over the whole window (06:50 → 07:29, 39 min): controller swap requested = 1, agent-driven bootstrap restarts = 1, new controller healthy = 1, rollback/swap-failed/unhealthy = 0, container RestartCount = 0. VerifyStartup banked it on the next boot (Post-update startup: update successful (0.153.0 → 0.154.0)) and the 06:57:52 periodic check logged Current version 0.154.0 is up to date — the at/above-floor branch correctly doing nothing. No storm, no rollback, no second attempt. Caveat, disclosed: a hand-deploy of v0.155.0 at 07:17:10 sits inside the observation window and is what StartedAt shows after that point; it never touches SwapController, so the swap-count assertions above are uncontaminated across the full window. A second, unplanned confirmation of the at/above-floor branch came with it — after the hand-deploy the box ran 0.155.0 against a 0.154.0 floor and the updater logged Current version 0.155.0 is up to date and did nothing. Evidence: felhom-controller/REPORT.md §6 (2026-07-21). (Superseded note:) the self-restart single-fire leg was - the floor was set to a version the box ALREADY ran, so there was no work and no restart. Finish by bumping the floor to a version the box does NOT run, debug ring open, asserting EXACTLY ONE restart. Trap found while banking this: the wake is logx.Debugf, so it is INVISIBLE in docker logs at INFO and lives only in the debug ring (GET /api/debug/logs?level=DEBUG) - a hunter looking at stdout wrongly concludes the box never woke; arguably (b) generalised. (b) cosmetic: the Waiter's "recovered" INFO logs on the next hold completion (pollOnce blocks ~240 s), not at reconnect |
S | (a) BANKED in full; only (b) cosmetic remains | Map row "config/state change round-trips in seconds" flipped PARTIAL->PROVEN-LIVE 2026-07-21 on this evidence. Evidence: felhom-controller/REPORT.md 4f |
| R-34 | Backup data lifecycle management. An "inactive backups" section on „Távoli mentés": apps that have snapshots but no active backup — disabled OR uninstalled — listed with name / size / last snapshot / restorable, plus an explicit double-confirmed per-app delete via restic forget --tag + nightly prune. |
M | idea | RULING: the offsite toggle NEVER offers deletion — policy and destruction stay decoupled. Turning backups off must never be a data-destroying act, and deletion must never hide behind a toggle. Origin: 2026-07-18 rehearsal. Pairs with R-32 (that one is the operator's view of dead bytes; this one is the customer's) |
| R-35 | Config-apply should not end the customer's session. The offsite config push bumped config_version 10→11 at 16:54:58 and the controller self-restarted (container StartedAt 16:54:59Z, back up 16:55:02); in-memory sessions died with it and customer zero was force-logged-out mid-flow. |
S | idea | Direction: hot-apply the offbox target (no restart for a config the running process can adopt), or persist sessions across restart. The restart itself is by design — the collateral is not. Evidence controller-log-full.txt |
| R-36 | Post-RESET re-enroll leaves offsite "enabled but unprovisioned" — silently. The hub knows the state and says nothing on the customer page. | S | SHIPPED (hub v0.67.0, 2026-07-18) | Both halves delivered. (1) The warning: the customer page now names the state and the fix — enabled-but-unprovisioned raises an amber banner saying provisioning is Save-triggered (press Save once, then verify), reusing the exact enabled && type == "" predicate the offsite re-issue handler already refuses on. (2) The related sub-item, also done: the self-bind link is now auto-minted at customer creation AND at RESET completion, so the console banner's „e-mailben kapott link" is already true instead of true-once-the-operator-remembers. Extracting the shared mintAndSendSelfBindLink core keeps the button and the auto-mint callers on the same F1/F2 honesty rules, and the auto-mint never fails the operation it rides on. Gap found and closed while wiring it: PurgeCustomerResetDBState does NOT clear selfbind_tokens, so a link minted BEFORE a reset would have stayed live across it — the skip paths now clear stale tokens, giving the invariant "after auto-mint, the only live link is one we just issued, or none". Tests assert the banner is ABSENT in all three nominal cases too; red-proofed. — Original analysis: Source-cited behaviour, confirmed live in the rehearsal: provisioning is Save-triggered (configs.go applyOffsite) — which also answers S6's open question — and the re-enroll auto-re-issue correctly skips unprovisioned targets (handler.go). So nothing is broken; the gap is that nobody is told. Direction: flash it on the customer page. Interim: an R-3 step. Related sub-item: auto-mint the self-bind link on customer create/RESET, so the console banner's „e-mailben kapott link" is always already true instead of true-once-the-operator-remembers |
| R-27c | Customer self-bind, slice 2 — console-passphrase bind. Viktor's direction: bind using a passphrase shown on the box console, alongside (not instead of) the emailed capability link. | M | idea | Security constraints from the session ruling, all load-bearing: passphrase issued at customer creation; the global-lookup endpoint must be spray-hardened — per-appliance and per-IP caps, constant-time comparison, a single generic failure (no oracle), alerting on abuse; an accent-free wordlist (console keymaps are not Hungarian); the web capability-link path is RETAINED; claim-by-email is RETAINED as the delivery-channel proof. Also under this item: the self-bind email gains the public universal-ISO download link + two-line instructions (the DIY case). Secret-bearing per-customer ISOs are ruled OUT. Sibling of R-27b (second-box flow) — different axis, both build on the same /bind/ page |
| R-50 | [P2-HIGH] Island-bridge control plane — make controller↔agent independent of the LAN. The agent's localapi binds a LAN literal (listen_addr) and the guest dials that same literal from bootstrap.json. Move both onto a host-internal bridge with a fixed, private address that no router, DHCP lease, or site move can invalidate, so the control plane is location-independent and golden-image-stable. |
M | idea — SPIKE-FIRST | Origin: audits/AUDIT-vacation-remote-ops-2026-07-20.md (F1), where this failed for real. The demo box moved to a remote site, DHCP handed it .147 instead of .162, and the agent then could not start at all — bind: cannot assign requested address, systemd gave up after 4 retries — taking storage, PBS backup, quiesce, restore-test and DR down for as long as nobody noticed. Mitigated for that window by pinning vmbr0 static back to .162; that is a window mitigation, not the fix — it still depends on the site's subnet being 192.168.0.0/24 and free at that address. Spike-first is mandatory: validate end-to-end on the drill environment (agent bind + guest dial + TLS SAN/pin + reinstall/golden survival + the bootstrap-config migration for already-deployed boxes) BEFORE any production spec. Pin fact (verified 2026-07-20, agentapi/client.go L105-129 — supersedes the earlier "the SAN set must cover the new address" note in this entry, which was wrong): the controller-to-agent leg sets InsecureSkipVerify: true and replaces chain verification with a custom VerifyPeerCertificate that does a raw SHA-256 match on the leaf DER against the bootstrap fingerprint. Hostname/SAN therefore never enters verification on this leg, so moving the agent listen address most likely needs no cert re-issuance - only the endpoint the guest dials. The spike must still confirm this empirically rather than trust the read. Flips a future "box survives a site/network change" map row |
| R-50b | [P2] A root-owned privileged host artifact is delivered unversioned from main — "which wrapper is on this host?" is unanswerable. configs/felhom-pbs-apply installs to /usr/local/sbin/felhom-pbs-apply (0755 root:root) and is the pinned sudoers vector for create|reconcile|grant against /etc/pve/priv/storage. It is fetched by felhom-host-install.sh:1914 via fetch_raw, which hits raw/branch/main/<path> — no tag, no pin, no checksum, and no record in the Day-0 artifact manifest, unlike the agent binary (sha256-vouched) and the golden image. Three consequences: (1) two hosts installed a week apart can carry different privileged wrapper code while both reporting the same agent version; (2) a host hotfixed in place (felhom-pve, 2026-07-18) is indistinguishable from one that fetched the same content — the fleet has no inventory of it; (3) an accidental push to main reaches the next install of every host with no review gate between commit and root-owned deployment. |
S–M | (a) SHIPPED 2026-07-21; (b)/(c) open | Surfaced 2026-07-21 while stopping the R-39 v0.90.1 publish (felhom-controller/REPORT.md §5): the publish was cancelled precisely because the version number would have claimed to carry a fix that in fact rides this unversioned channel. Candidate shapes, in increasing cost: (a) record the wrapper's sha256 in the Day-0 artifact manifest beside the agent binary and have the agent report the installed file's hash, so drift is at least visible; (b) fetch_raw takes a pinned ref (tag or commit) supplied by the manifest rather than main; (c) the wrapper becomes a published generic-registry artifact with the same gate ladder as the agent binary. (a) is the cheap honest first step and would have caught this class already. Pairs with R-39 (whose remaining fleet half is specced separately) (a) SHIPPED 2026-07-21 — hub v0.68.0 + agent v0.91.2. ArtifactManifest.WrapperSHA256 + an operator field; agents report the installed wrapper's sha256 each cycle and the host page surfaces a mismatch. An unknown on EITHER side reads as quiet, never as drift — lighting every host amber on rollout day is how a warning becomes background noise. Live confirmation of exactly the problem: felhom-pve's July-18 in-place hotfix hashed 2888f2ea…, matching no commit anyone could name; it now reports 104db0a4… against a vouchable manifest value. (b)/(c) REMAIN OPEN: the wrapper is still fetched unversioned from raw/branch/main — this makes drift visible, it does not fix the channel. Also recorded: the 0440 sudoers file is not agent-readable, so its drift stays invisible. |
| R-51 | Dead-primary alerting — a multi-container app whose MAIN container is dead must alert. Aggregation currently classifies such a stack unhealthy, and IsDownState deliberately excludes unhealthy, so nothing fires. |
S | SHIPPED 2026-07-21 — controller v0.156.0 | Origin: AUDIT-vacation-remote-ops-2026-07-20.md (F4). Observed live: immich-server was Exited for 18 h with the app 100 % unreachable, and the box produced no dead-app banner and no app_start_failed hub event — while single-container Calibre-Web, down for the same reason, alerted correctly within 90 s. Constraint (load-bearing): do NOT simply fold unhealthy into down. That exclusion is deliberate (stacks/manager.go fix-3, downstate_test.go) and reverting it reintroduces the flapping it was added to stop. Direction: distinguish member-container-exited from healthcheck-failing in the aggregation, and treat a dead primary as down |
| R-52 | Boot desired-state reconciliation — a deployed: true app should be running after boot. The controller reports deployed-but-stopped apps (30 s deadapp-check) but never starts them, so an app that misses its boot start stays down until a human notices. |
M | SHIPPED 2026-07-21 — controller v0.156.0 | Origin: AUDIT-vacation-remote-ops-2026-07-20.md (F5). Observed live: the pre-transport shutdown left immich-server and calibre-web Exited; 10 sibling containers came back and those two did not, and they were still down ~18 h later. Includes root-causing why restart: unless-stopped did not resurrect them — both were stopped ~25 s before power-off, so Docker most likely recorded them as user-stopped; that hypothesis is untested because the guest journal is volatile and the controller's own logs were rotated by the container recreate. Direction: a bounded start-once reconciliation (N attempts, reusing the existing boot grace), never a restart loop. Pairs with R-51 — that one is the alarm, this one is the recovery |
| R-54 | [P2-HIGH] The guest's DHCP client is unsupervised — its death takes the box off the internet 1-2 hours later, invisibly. ifupdown starts dhclient once at guest boot and nothing restarts it. |
S-M | SHIPPED 2026-07-21 — agent v0.92.1 | Origin: audits/INCIDENT-guest-dhclient-killed-2026-07-20.md §5 "OPEN RISK" — this row closes it. On 2026-07-20 a cleanup step killed guest 9201's dhclient (visible in the HOST's pid namespace; §4's /proc/<pid>/cgroup rule exists because of it). The guest then kept working for another ~80 minutes on its unexpired lease; only at expiry did the address and default route vanish, taking the Cloudflare tunnel, hub reports, catalog sync and the controller→agent channel with them — 1h15m outage, and every observable signal said healthy for the first 80 minutes. The design consequence: liveness of the DHCP client is itself a probe. internal/guestnet flags a DHCP guest unhealthy on pgrep -x dhclient alone, while the lease is still live — waiting for the IP to disappear is waiting out precisely that silent window (red-proof: reverting to IP-presence-only makes the July-20 fixture report healthy with zero heals). Four fixed-shape pct exec probes (address / default route / /etc/network/interfaces mode / client liveness, parsers pinned to output captured live from 9201), the incident's restored invocation as the heal, verbatim, and dampers throughout: two CONSECUTIVE bad probes, ≥10 min between heals per guest, ≤3/hour, observe-only while guest or agent uptime < 3 min. Refuses to act on a static guest (dhclient must never fight a static config — reported loudly and left to R-50, which is where option 2 of the incident's three choices belongs), on an unknown mode, on an unprobeable guest, or on an ownership-unproven guest list (source is ListLXC ∩ the felhom pool, audit A1). Host-tier by necessity: a guest with no default route cannot repair its own default route. A live finding during deployment: the first sweep on felhom-pve logged dhclient liveness probe failed: sudo: a password is required and reported state=unknown — fail-safe, but blind. TASK-D assumed no sudoers change was needed; three of the four probes had no grant. FELHOM_GUESTNET + four guestnet-* capability rows shipped in v0.92.1 (v0.92.0 superseded, do not vouch). Healthy cycle PROVEN LIVE 2026-07-21 on felhom-pve: caps 68/68 ok, degraded=0 and level=DEBUG guestnet: guest network healthy vmid=9201 mode=dhcp has_route=true dhclient_alive=true. The HEAL leg (STOP-2, a deliberate replay of the incident) is operator-present and pending. Note the guest and the host still differ (host static since the F1 mitigation, guest DHCP) — choosing one for both remains R-50's call, not this row's. Evidence: felhom-agent/REPORT.md (2026-07-21) PROVEN LIVE 2026-07-21 (STOP-2, operator-present) — a deliberate replay of the incident. kill -9 on guest 9201's dhclient at 12:43:18 CEST (pid 336708, cgroup=0::/lxc/9201/ns/.lxc verified BEFORE the kill — the incident's own §4 rule, applied). 12:44:15 — 57 seconds later — detected on process liveness alone, with the address and default route still perfectly present and 83 minutes of lease remaining: unhealthy (first bad probe — not acting yet) bad_probes=1 required=2. 12:45:15 second consecutive bad probe → healing, cmd="pct exec 9201 -- dhclient -pf /run/dhclient.eth0.pid -lf /var/lib/dhcp/dhclient.eth0.leases eth0" (the incident's invocation, logged verbatim before it ran) → 12:45:18 guest network healed ip=192.168.0.104 has_route=true dhclient_alive=true heals_last_hour=1. Healed 120 s after the kill — ~80 minutes before the outage would have begun. The strongest evidence is what did not happen: cloudflared stayed Up 29 hours, DNS resolved, the hub answered 302 in 0.16 s and the public URL 302 in 0.25 s. The 2026-07-20 outage was prevented, not merely detected. Exactly ONE heal, zero re-heals. The damper then proved itself against a REAL transient, unplanned: the STOP-1 guest reboot at 12:53:16 produced unhealthy … no IPv4 address on eth0 bad_probes=1 — no action — then guest network recovered on the next cycle. A single blip really is not a diagnosis, and a booting guest is not injected with a dhclient. Negative leg run in full: 12:45:18 → 13:16:15 = 30 healthy Debug cycles, 0 heals, 0 WARN, 0 ERROR. Confirmed HUB-SIDE (read-only query of the hub store, after STOP-3): the guest_net stanza arrives complete, and the report history traces the whole session — 10:30:27Z agent 0.92.0 state=unknown (the sudoers-blind window, reported honestly rather than as a false healthy — independent proof that superseding 0.92.0 was right), then 10:48:12Z 0.92.1 healthy carrying last_heal_at=10:45:12Z heals_last_hour=1. Known limitation: the damping state is IN-MEMORY, so an agent restart resets the heal counters — the ≥10 min / ≤3 per hour ceilings hold within one agent lifetime, not across restarts (a restart does re-arm the 3-minute settle window, so the practical exposure is small). Do not quote the ceiling as a hard guarantee |
| R-55 | [P2] A customer's deliberate Stop does NOT survive a guest reboot for any drive-backed app — the boot bind gate recreated and started it unconditionally. | S | SHIPPED 2026-07-21 — controller v0.157.0 | Surfaced by STOP-1's R-52 leg, which was designed to prove the opposite property and instead proved it is not true end to end. internal/web/intermediary.go's boot gate ([gate] boot <id>: live bind confirmed — recreating drive-backed app <x>) selects on cfg.Deployed && HDD_PATH alone and then runs StopStack + StartStack on every match; the stack's state is passed in for the log line and is never consulted. So after pct reboot 9201, calibre-web and immich came back running at 10:53:18Z even though immich had been stopped from the UI seconds earlier (compose down, 0 containers) — while bookstack, which is NOT drive-backed, correctly fell through to R-52. R-52 is not implicated: its own gate behaved exactly as specified (immich, with zero containers, was never a candidate), and this behaviour predates it. Two consequences worth ruling on: (a) a customer who stops a drive-backed app finds it running again after any reboot, which is the same class of dishonesty as offering a control that cannot work; (b) R-52's practical scope on a real box is narrower than the brief assumed — the gate reaches every drive-backed app first, so the boot-orphan sweep only ever sees the rest. Likely shape: the gate should recreate only apps that were RUNNING (or deploying) at shutdown, which is the same running/stopped distinction R-52 already draws, rather than every deployed app. Needs a ruling, not a drive-by change — the gate exists to fix a real bind-ordering bug and must keep doing that |
14:29:28 [gate] boot 1784525102-11906045: live bind confirmed — recreating drive-backed app calibre-web (state=stopped) onto /mnt/felhom-drives/hdd_114:29:29 [gate] boot 1784525102-11906045: 1 drive-backed app(s) left stopped — zero containers means the customer stopped them on purpose
immich is absent from the recreate list and came back STOPPED (0 containers) — on 2026-07-21 before this fix, the identical fixture brought it back RUNNING. calibre-web recreated, bookstack back, [bootrecon] no boot-orphaned apps (consistent — nothing was left orphaned for it to adopt), ZERO alerts, whole convergence ~15 s from reboot to steady state. The left stopped INFO line fired in production for the first time, so the honoured path is observable rather than silent |
| R-56 | [P3] Apps do not say how technical they are, so a beginner can be ambushed by a config-heavy one. The catalog presents every app as equally approachable — one Telepítés button, the same Hungarian copy — but they are not. Glance needs a hand-written glance.yml before it does anything; some apps need a reverse-proxy or API concept to configure; others genuinely are install-and-use. A tester who picks the wrong first app concludes the PRODUCT is broken, not that they picked an advanced app. | S | idea (filed 2026-07-21) | Origin: TASK-E Part 3 — filed, deliberately not implemented. Shape: a difficulty: field in .felhom.yml (kezdő / haladó / technikás) surfaced as a catalog-card badge and repeated on the deploy screen. Cheap and incremental: one optional metadata field plus a badge, classifiable app-by-app with no migration — an app with no difficulty: simply shows no badge. This is the constructive half of the glance ruling: glance STAYS in the catalog (operator ruling 2026-07-21 — it is a legitimate app, not a broken one; its missing seeded glance.yml is a known pre-existing finding), and the honest fix is to LABEL it rather than hide it. Pairs with R-41: that gate proves an app CAN still deploy; this field tells a customer whether THEY should be the one deploying it. Badge plumbing is ALREADY BUILT (controller v0.158.0) — web.MetaBadge + the meta_badge template partial + the lifecycleBadge funcmap entry were written generic for exactly this: a difficultyBadge funcmap function returning the same *MetaBadge, plus a difficulty: field on stacks.Metadata, is the whole remaining job. No new markup, no new CSS. Re-sized accordingly |
| R-57 | An app can be withdrawn from the catalog without orphaning the customers already running it — .felhom.yml lifecycle: available / hidden / abandoned. | S | SHIPPED 2026-07-21 — controller v0.158.0 (+ v0.158.1 fix), LIVE-PROVEN | Motivating case: plant-it. Earlier the same day it was withdrawn by moving its directory to retired/ — which un-offers the app but ALSO makes the controller's orphan detector see the template as GONE for anyone running it, flagging their working install Elavult and offering a Törlés button. Withdrawing an app must never take a working app away from a customer, so the directory move was replaced by metadata. Operator requirements, verbatim (ruling 2026-07-21): states available / hidden / abandoned; abandoned apps are NOT offered to new installs (no badge-but-installable middle state); deployed instances of hidden/abandoned apps keep full function; an abandoned app shows a permanent notice that „Az alkalmazás fejlesztője felhagyott a fejlesztéssel. A telepített verzió továbbra is használható, de frissítések és biztonsági javítások már nem érkeznek hozzá." Design points that matter beyond this feature: (a) the deploy gate is server-side and fail-CLOSED before any mutation — hiding a button is not a gate, and a stale link or direct POST must be refused; (b) an unknown lifecycle value fails OPEN (→ available + one WARN), deliberately opposite, because a typo or a state from a newer catalog must never pull a working app out of every customer's list — both read the same EffectiveLifecycle, so they cannot disagree; (c) lifecycle NEVER reaches orphan detection, red-proofed. LIVE-PROVEN 2026-07-21 on 9201 through the real endpoints: lifecycle: abandoned arrived via the normal catalog sync; plant-it renders 0 times on the Alkalmazások page while the control app renders 10; a direct POST /api/stacks/plant-it/deploy returns HTTP 409 {"ok":false,"error":"Ez az alkalmazás jelenleg nem telepíthető."}; the app page carries the notice and no Telepítés button. v0.158.1 is a shipped-and-caught defect worth remembering: the three predicates were declared with POINTER receivers, and html/template cannot call those on the non-addressable value the handler passes — every /apps/<slug> returned 500, for every app, while compiling cleanly with a fully green suite, because no test rendered app_info. A template method call is only checked when the template runs. Follow-on: R-56's difficulty badge reuses this plumbing |
| R-58 | [P2] Assisted disk-picker install mode — the installer should let the operator CHOOSE the target disk instead of requiring the serial up front. Today an install is either unattended (the answer file pins one ID_SERIAL_SHORT, which you can only know by first booting the machine) or match-nothing safety (aborts by design). That forces a two-boot dance for every new box: boot the safety ISO to read the serial, rebuild the ISO armed, boot again. | S–M | idea — operator ruling 2026-07-21 | Operator's argument, verbatim: "the installer should list the available storage devices (excluding the installation media) and let us select one, and continue." Shape: a THIRD ISO mode alongside the two that exist — unattended-serial and match-nothing-safety. It enumerates candidate disks with size / model / serial, excludes the installation media itself, takes a selection plus a confirm, and proceeds. Unattended+serial REMAINS the appliance/factory mode — it is the right shape when the machine is provisioned in bulk and nobody is standing there; the picker is for the case where somebody is. Slice 1 (cheap, same code surface, do this first): improve the abort screen. On filter-no-match the installer currently just fails safe and says nothing useful — it should print the candidate table (size/model/serial) plus the one-line hint naming which serial to put in the profile. That alone collapses the two-boot dance from "boot, guess, go read docs, rebuild" to "boot, copy the serial off the screen, rebuild", and it is the same enumeration code the full picker needs. Why it matters beyond convenience: it is the BYO / reinstall flow — a customer's existing hardware, or a rebuild of a box whose disk layout nobody recorded, is exactly where the serial is unknown and a wrong guess is destructive. The current fail-safe is correct but mute. Origin: TASK-G, arming the HP install ISO — the serial had to be read off the board by hand between two boots |
| R-59 | [P1] A no-DHCP install must HARD-ABORT — instead it bakes the installer's fallback address as a STATIC config and completes, producing a box that can never call home. | S | SHIPPED v1.24.0 (2026-07-22) — as a FIRST-BOOT refuse-loudly gate, with a RECORDED DEVIATION: the install-time abort is out of scope (the 192.168.100.2 fallback is baked inside the Proxmox auto-installer itself, unreachable without an installer-initrd hook; operator-acked, not silently dropped). The screen + gate proven on the nested drill (SPIKE-firstboot-nic-sweep-2026-07-22.md); nested ≠ metal — metal proof rides the next real install | This is the worst silent onboarding failure shape there is: the install succeeds, the box looks finished, and it is permanently unreachable — no hub check-in, no pairing, no way in except a keyboard and monitor. Found on the HP t740's first install: the 4-port NIC got no DHCP lease at this site (see the t740 gotcha in scripts/iso/README.md), and rather than refusing, the installer wrote its 192.168.100.2 fallback as a static vmbr0 address into /etc/network/interfaces and carried on. The philosophy is already established one layer over — the disk filter refuses loudly and touches nothing when it cannot identify its target (spike S5c, proven twice on real boards). Networking deserves the identical treatment: no lease on any carrier-bearing NIC ⇒ abort with a legible screen, never invent an address. Slice: detect "DHCP produced no lease" in the answer/first-boot path and fail with the candidate NIC table (name / MAC / carrier / link speed) plus the one-line remedy, exactly as R-58 slice 1 does for disks — same refuse-loudly grammar, same screen shape. Pairs with R-60, which is the self-heal for the case where the cable simply moved |
| R-60 | [P2] First-boot NIC sweep self-heal: if the hub is unreachable, try DHCP across every carrier-bearing NIC before settling. | S | SHIPPED v1.24.0 (2026-07-22) — spike + nested drill proven (SPIKE-firstboot-nic-sweep-2026-07-22.md): cable move → sweep → heal + hub registration unaided in <1 min; sweep is structurally first-boot-only (state.json gate + the unit's done-flag condition); drill also surfaced and fixed the baked-fallback-default-route trap (flush before the bounded dhclient) | felhom-bootstrap currently accepts whatever addressing the installer left behind and, if the hub cannot be reached, simply stays broken. On demo-hp the fix was a human moving one cable from the 4-port card to the onboard port — a sweep would have healed it unaided: enumerate NICs with carrier=1, DHCP each in turn, and keep the first that reaches the hub. Cheap because the box has nothing to lose at first boot (no customer data, no running guests) and the failure it repairs is total. Deliberately scoped to FIRST BOOT and to the hub-unreachable condition only — a running box must never re-shuffle its own networking. Complements R-59: that one refuses to produce an unreachable box, this one repairs the case where the truth changed after the install (cable moved, switch port died, the installer guessed the wrong port) |
| R-61 | [P1] The baked root password must be knowable by the operator — the recurring console lockout. | S | slice 1 SHIPPED v1.24.0 (2026-07-22): the build emits the plaintext into a 0600 sibling <iso>.rootpw.txt (single record of truth — never logged/manifested/committed); drill-verified against the installed box's shadow hash. Follow-up (appliance-grade record-keeping) stays open | The ISO mints a fresh throwaway crypt hash per build and the plaintext is discarded, so nobody — including the person holding the machine — can log into the console of a box they just installed. Today that meant reaching demo-hp only through the G1 break-glass credential vaulted in the hub, which is the right mechanism for a lost password and the wrong one for a never-known password: it requires a working hub, a working network, and operator tooling, at exactly the moment the likely reason you need the console is that one of those is broken. Slice 1 (do this): the ISO build emits the baked root password into the build REPORT and the operator cheat-sheet alongside the sha256 — it is already a per-build value, so surfacing it costs nothing and closes the lockout. Follow-up (appliance-grade): keep it per-build random and treat the build output as the record of truth. A fixed well-known password is explicitly REJECTED (operator ruling 2026-07-21) — a pre-pairing box sits on a stranger's LAN with a predictable root credential, which is a far worse exposure than the lockout it would fix. Relates to G1 break-glass (the vault stays; this is about the window before/without it) |
| R-62 | [P3] Hub delete dialog: show the customer-id the operator must type, and reword the three acks for the ghost shape. | XS | idea (operator, 2026-07-22) | Cosmetic, hub-only, docs-only in the v1.24.0 train. The delete confirmation asks the operator to type the customer-id, but the id appears NOWHERE on the Edit page the dialog opens from — the operator has to fish it out of the URL or another tab. Also: for a GHOST customer (host already gone) the three acknowledgement checkboxes describe teardown steps that cannot happen; wording only — the server MUST keep requiring all three (the render-gate lesson of v0.70.1 stands: reachability and requirements are separate concerns). |
| R-63 | The install console learns ő/ű — the kernel default console font lacks the Hungarian double-acute glyphs, so the R-59 network screen and pairing banner rendered ő as blanks. | XS | SHIPPED (scripts v1.25.0, 2026-07-23) | felhom-bootstrap.sh loads a Latin-2 console font (Lat2-Terminus16 → Lat2-Fixed16 → Lat2-Terminus14) ONCE before the first paint — idempotent, best-effort (a missing font/ioctl never blocks boot). Lat2 ships in the trixie/PVE base (console-setup), so no copy rewording needed. Font names verified against the package. Nested-console capture proof rides the v1.25.0 drill. Also in the same train: F9 belt-everywhere RESOLVED (host-install default appliance leg + live on both boxes + login proven) and the R-71 build-gate (build-felhom-iso asserts golden ≥ managed floor, publish-train-rules.md rule 5). Live finding: golden 0.153.0 < floor 0.156.0 in production NOW — the gate catches it; the fix is the golden republish at 0.161.0 (Part 4, pending; the managed floor stays 0.156.0). See REPORT.md (2026-07-23). |
| R-64 | „Felhom↔Felhom media pairing blessed" — the two-box SMB pairing (one box shares, the other mounts it as NAS storage) becomes a supported, documented flow. | XS–S | idea (2026-07-22) | Origin: the operator ran the pairing drill on the live demo pair and it WORKS — the drill itself is the pending evidence leg (a written run-through with the R-66 surfaces in play). R-66 shipped the enabling visibility: the serving box's address is now on its own Beállítások → Rendszer „Hálózat" card, and the add form names the NetBIOS trap. Blessing = a short customer-facing recipe (documentation/controller/network-storage-nas.md naming-caveat paragraph is the seed) + one supported-path sentence in the capability map. Flips: would add a "Felhom↔Felhom media pairing" capability row (currently unlisted). Pairs with R-65 (same two-box topology, entirely different transport + guarantees) |
| R-66 | The box's own address becomes visible — „Hálózat" card, Debug network dump, NetBIOS hint. | XS | SHIPPED (controller v0.159.0, 2026-07-22) | Origin: the pairing drill — the serving box's IP was findable only as a hint buried on the OTHER box's Megosztás page, and the add form's failure for „FELHOM" taught nothing. Three legs: (A) „Hálózat" card on Beállítások → Rendszer (Helyi cím / Hálózati név only-while-sharing / Átjáró; live per render, stored nowhere — S-5; „—" when unavailable); (B) network section in the Debug dump (interfaces/route/DNS/lan_address, best-effort per item); (C) the NetBIOS trap named (Szerver helper text + a purely lexical hint on unreachable for single-label non-IP names). Design decision recorded: the controller is bridge-netns'd, so ALL guest-net reads go through the one netns door (docker exec into host-networked felhom-samba, stacks/guestnet.go) — with Megosztás off the card honestly shows „—" rather than the plausible-wrong 172.x answer. Deployed demo-felhom + demo-hp 2026-07-22; demo-hp live-shows the closed-door path (sharing off → dashes + in-place dump errors), demo-felhom the open one (real .104/.1/\FELHOM values). Flips no capability-map row (diagnosability/UX polish); enables R-64 |
| R-67 | The NAS share appears in FileBrowser — browse what you mounted. | S | SHIPPED (controller v0.160.0, 2026-07-22) | Origin: the R-64 pairing drill — the share said „Elérhető" and the customer had no way to BROWSE it (FileBrowser synced drives only). Couples to R-64: browsing was its missing UX half. A registered network storage now binds its share ROOT into FileBrowser (/mnt/felhom-drives/<name>:/srv/<name>:rslave) with its display label as the sidebar source; NAS add/remove trigger the same debounced sync. Two classes, two gates: drives keep the drive-absent gate byte-identically (proven live: the drives-only box logged a no-op sync); network shares gate on the STUB classifier instead — idle autofs is HEALTHY and included (Phase-0 probe on demo-hp: an in-container access through an rslave bind WAKES the idle trigger), while a stub verdict excludes the share from mounts AND sources with a WARN (an exposed stub swallows uploads the real mount later shadows). Nothing is ever written toward the NAS (no skeleton — red-proven). Live leg: cross-box upload round-trip demo-hp → demo-felhom + dead-NAS check (Host is down in seconds, unaided recovery after samba restart). Operator residual: the FileBrowser UI click-through (its admin credential is customer-held by design). Evidence: felhom-controller/REPORT.md (2026-07-22) |
| R-68 | Notification train: paired recovery mails + prefs seeding at claim + priority headers (power-outage audit F11+F12+F14-light). The dead-man's-switch fired perfectly on 07-22 and the customer who got „A szerver nem elérhető!" was never told it recovered (F11); a customer without a customer_notifications row is silently unnotifiable (F12, demo-hp live); delivered ≠ noticed (F14). | M | SHIPPED (hub v0.71.0, 2026-07-22) | Origin: AUDIT-power-outage-recovery-2026-07-22.md. Recovery = explicit eventType branch (severity semantics frozen; severityNotifies untouched): operator always hears both edges, customer iff PAIRED (customer-channel sent stale/down row newer than the last sent recovery — store.LastCustomerSentAt; enabled_events deliberately ignored for recovery; ties → no mail, flap-safe). Seed-at-claim: MarkClaimed → SeedNotificationPrefs (INSERT-if-absent, never upsert — red-proofed; empty email no-op; never fails the claim; default critical-only set). Hub-side empty-email no-clobber belt in handleSavePreferences (controller 0.160.0 already guards its own two push legs — latent, not live). X-Priority: 1 + Importance: high on error/critical via Resend headers (live-probed HTTP 200 before implementation); the test event now also mails the operator with those headers (one click proves both channels + rendering); latent sendTestEmail nil-prefs panic fixed. 17 tests + 4 red-proofs. Live legs pending: natural *_recovered mail on the next real staleness cycle (or the reboot-drill arc — NEVER fabricated by blocking reports, that is F9-bypass-shaped) and seed-at-claim on a real claim (Peti Friday reinstall is the natural candidate) |
| R-69 | F14-full: an operator push channel that actually interrupts (ntfy / Telegram / similar), beyond mail-client priority flags. F14-light (v0.71.0 headers + Gmail filter) nudges a mail client; a 15:29 node_down should reach the operator's pocket in seconds regardless of inbox hygiene. Needs: channel choice (self-hosted ntfy on k3s vs Telegram bot), dispatcher fan-out seam, per-severity routing, quiet hours. | M | idea | Origin: AUDIT-power-outage-recovery-2026-07-22.md F14. Deliberately NOT built in the v0.71.0 train (scope-forked per the task spec) |
| R-70 | [P2-HIGH] The offsite last mile is invisible on BOTH surfaces — the hub cannot tell "staged" from "delivered" from "applied". demo-hp sat 2 days with the hub customer page saying "Provisioned: … the transient password is delivered to the controller once" while the box said „Még nincs beállítva távoli mentési cél" — and a real customer would sit unprotected indefinitely believing otherwise. The hub HAS the signal (one_time_secrets.consumed_at + 153 consecutive reports carrying no offbox object) and reads none of it: the "Provisioned" line is static copy gated only on offsite.host in ConfigJSON (config_form_body.html:119–120). | S–M | SHIPPED (hub v0.72.0 + controller v0.161.0, 2026-07-23) — detector offsite.DeliveryStateFor (one impl, all consumers), customer-card state line with age (static "delivered once" copy GONE), offsite_delivery_stuck warning event (24h durable cooldown), controller truthful empty-state banner. Live validation on the two fixtures (demo-hp applied; demo-felhom applied + stale-staged info) recorded in felhom.eu/REPORT.md; the banner leg is unit-proven/live-pending (no box occupies the enabled+no-offbox window). | Origin: audits/DIAG-f10-demo-hp-offsite-2026-07-23.md. Two legs: hub customer card shows the real delivery state ("provisioned, awaiting box consumption" / "consumed, awaiting apply" / "applied" — consumed_at × report-offbox-presence is enough for all three), and controller banner when the descriptor is enabled but no target is configured („Felhom offsite készen áll — a beállítás automatikus, folyamatban"). Couple to R-31's async/status-card idiom (same surface likely serves both) and to the R-39 consumed_at honesty gauge precedent on the PBS side — a consumed secret + N report cycles with no offbox status is the same "disagreement no single tier can see" shape and deserves the same loud event. Supporting live datum: demo-felhom's 07-21 staged secret is still unconsumed today (key-auth-first path never consumes) — invisible for the same reason. |
| R-71 | [P1] Day-0 race: the managed floor-update kills the offsite apply-bridge between password-consume and persist — the one-shot credential is burned and the box lands in the silent consume-404 dead-end forever. Proven on demo-hp (07-21): consume 16:27:42 → managed update 0.153.0→0.156.0 replaces the container 16:28:17, ~35 s later, mid ssh-copy-id window; nothing persisted, no installed key ⇒ the key-auth-first recovery path can never engage, and every subsequent start logs the WARN and gives up until an operator Re-issue. This recurs structurally on every fresh onboarding whose ISO floor lags the managed floor — the update fires minutes after first boot, exactly when the bridge first runs. demo-felhom escaped by timing only. | M | **PARTIAL — (c) SHIPPED (hub v0.72.0, 2026-07-23): self-heal restage via the EXISTING Re-issue path (monitor.OffsiteDeliveryChecker), trigger = consumed ≥1h + ≥4 consecutive offbox-less reports + zero offbox evidence, one restage/customer/24h (durable via events table), every firing = offsite_credential_restaged warning; R-39(a) act-time guard red-proofed (clobber observed with guard removed). Ships unit-proven, NOT live-fired (no broken box existed; arms on next natural occurrence or a staged drill — never PROVEN-LIVE until then). (a) day-0 ordering stays OPEN → its own upcoming spec. | Origin: audits/DIAG-f10-demo-hp-offsite-2026-07-23.md (mechanism cites: bridge order offsiteapply.go:106–187 consume-then-persist, retry only on process start; the dead-end is even documented in source l.168–173 "the password is spent; reset it on the hub to retry"). Candidate directions, spec-first: (a) order — first-boot path lets the managed update settle before the bridge's consume step (cheapest; the race window is the update, not the reboot); (b) two-phase consume — hub marks consumed only on a controller ack-after-persist (touches the 404-no-oracle contract, design carefully); (c) hub-side self-heal — auto-restage a fresh secret when consumed_at is set but K consecutive reports show offsite enabled with no offbox status (the R-39(a) mint-race lesson applies: never restage on top of an UNCONSUMED secret). Pairs with R-70 (visibility) — but visibility alone is not the fix; the burned credential needs an unattended recovery path. |
| R-53 | app_export.html substituted the CSRF token where the customer domain belongs - the open-in-browser link was wrong for every app with a subdomain, and a session CSRF token landed in a URL. | XS | SHIPPED (controller v0.150.0, 2026-07-20) | One template token ({{$.CSRFToken}} -> {{$.Domain}}) plus the Domain key in exportPageHandler's data map - that handler does not go through baseData, which is where every other page gets it, so the template had no domain to read. Render tests assert the joined <sub>.<domain> and that the token appears nowhere in that line; red-proofed against the pre-fix template. Origin: audits/AUDIT-vacation-remote-ops-2026-07-20.md (F7) |
P3 — post-alpha
| ID | Item | Size | Status | Notes |
|---|---|---|---|---|
| R-26 | Guided old-history recovery via a retained superseded escrow + the recovery code. Enabled by hub v0.60.0 (Part B) which now RETAINS superseded escrow blobs (host_escrow_superseded, ListSupersededEscrow). Build the flow that, given the customer's recovery code, unwraps a retained old blob → recovers the old repo passphrase → mounts/reads the moved-aside .orphaned-<date> repo for restore. |
M | idea (enabled by v0.60.0) | Turns "history recoverable in principle" into a real customer-drivable path; pairs with the controller v0.142.0 orphaned-repo move-aside. Origin DIAGNOSE-offbox-repo-orphaned-2026-07-17 |
| R-27 | Customer-facing self-bind page (R-21 slice C follow-on). Today an unclaimed appliance is bound by the OPERATOR on the Hosts page (hub v0.62.0). Build the customer-facing flow so a customer can claim/bind their own freshly-installed box. | M | SHIPPED (slice 1, hub v0.66.0, 2026-07-17) | Slice 1 = the FIRST-box flow, DONE: operator "Send self-bind link" → 7-day tokenized capability link over Hungarian email → public two-factor /bind/<token> (console pairing code shown on the box screen + retrieval passphrase) → hub stages the bind via the same BindAppliance, no operator. Viktor's 3 rulings verbatim (console code not a list; operator-sent link; 5-attempt lockout→call support). No oracle; expiry falls back to operator-bind; THE TRAP /bind/ exemption tight (single predicate, 4 red-proofs). GC verdict §3 (no appliance GC → the 7-day TTL stands alone). Slice 1 PROVEN-LIVE 2026-07-18: in the rehearsal the operator minted + emailed the link at 16:28:55 and the customer bound their own box at 16:29:55 with zero failed attempts, appliance_bound carrying source customer_selfbind, credential delivered 26 s later with no operator action (tests/VALIDATION-n100-rehearsal-2026-07-18.md). The new-ISO console banner rendered on the real box. Slice 2 = console-passphrase bind → R-27c (below). Reused the appliance_registrations + one-shot delivery machinery; the capability link is the whole customer-auth surface (no login system built). Controller/agent untouched. See hub v0.66.0 REPORT. Multi-box per link is out of scope by design (single-active, one-shot → binding several boxes = repeated operator sends) |
| R-27b | Customer self-bind, second-box flow (controller side). For a customer who ALREADY has a bound box and installs another, the controller shows a dismissable "bind another box" prompt (and a bind-later entry under settings) that walks to the hub /bind/ page — so a returning customer isn't emailed a fresh operator-sent link for every box. Mechanism sketched in the hub v0.66.0 REPORT; NOT built (R-27 slice 1 deliberately did not touch the controller). |
M | idea (minted by hub v0.66.0) | Origin: hub v0.66.0 slice-1 ship (first-box only). Reuses the same /bind/ public page + tokenized-link machinery; adds a controller-side entry point + the operator "mint a link for an existing customer" affordance |
| R-25b | RULED: customer DELETE becomes a guided full-teardown cascade. The middle-tier Customer RESET (hub v0.61.0) runs the full external teardown (Hetzner sub-account/box + PBS namespace/groups/token) and refuses while any host row exists. The Danger-zone DELETE still (a) leaves host rows and (b) does NOT run that teardown. | M (was S) | SHIPPED hub v0.69.0 (2026-07-21) | operator ruling 2026-07-21: DELETE subsumes the whole cascade, behind explicit consent. Three separate acknowledgements, each its own checkbox — (1) the host(s) will be deleted, (2) the customer will be RESET including external teardown and offsite data destruction, (3) the customer record and escrow will be purged — plus a typed customer-name confirmation before the button arms. Internal order is host-delete → RESET → delete, which preserves every existing invariant rather than relaxing any: RESET keeps its no-hosts precondition (hosts are already gone by then), and escrow keeps its demote-then-purge custody rule (host delete DEMOTES to retained custody, the final delete PURGES — the one true purge point). Re-sized S → M: this is a multi-step destructive wizard with three acks and a typed confirmation, not a checkbox. Implementation is explicitly NOT part of TASK-E; the row carries the ruling and awaits its own spec. It no longer blocks R-3 — the model is decided, so the friend-alpha runbook can be written against it. IMPLEMENTED per the ruling (TASK-I, hub v0.69.0): POST /configs/{id}/delete now runs hosts → RESET → purge; three acks + typed customer-id + a stale-preview check + the ONLINE-host refusal, all gates before any write (zero side effects on refusal); custody purged exactly ONCE in leg 3 (leg 2 runs with purgeEscrow=false); ruling-3 preserved BY CONSTRUCTION and asserted from inside leg 2; failed legs retain the journal and the dialog offers Resume. Standalone RESET byte-identical. 5 red-proofs. Offboarding guidance: runbooks/RUNBOOK-onboarding-draft-v4.md §G. v0.70.0 follow-up (same day, found validating against the live hub): a completed delete still left the customer on the Customers list and still ALERTING, because GetCustomers() is report-derived and no tier ever deleted a report — new residue leg (reports/telemetry/log-tails/notif-prefs + the credential-bearing appliance_registrations/selfbind_tokens), and ghost customers are now deletable (404 = nothing here, not no-config-row). v0.70.1 (2026-07-22): the ghost delete was implemented but UNREACHABLE — the Danger-zone card (and the customerDeleteOpen script) sat inside {{if .HasConfig}}, so a ghost rendered no Delete button at all (the fourth inert-seam defect; handler tests POST directly and proved nothing about reachability). Render gate split: RESET stays HasConfig-gated, Danger zone gates on Deletable (the exact negation of the preview's 404 predicate), Block/Unblock stay config-only; render tests per branch + 2 red-proofs. Operator live leg: the demo-vm-felhom ghost delete click — PENDING (doubles as the v0.70.0+v0.70.1 live validation; expect residue=ok customer_delete=ok with skipped_no_config Hetzner/descriptor legs, staleness emails stop) |
| R-25 | Device-node TOCTOU hardening (drive init). Graduate the controller v0.141.0 Observation: the format → resolveEnrollUUID(path) → AssignDisk(uuid) sequence has a narrow /dev-re-enumeration window (agent-guarded on the destructive format via anti-retarget durable-id; benign fs-UUID mount). Bind resolve+assign to the format's durable-id so the mount can't target a moved node. |
S | idea | From the v0.141.0 F6 commit's security-review finding (felhom-controller REPORT). Low real risk (single-operator, agent-guarded), but cheap to close |
| R-24 | Guest RAM resize (live) — SHIPPED (agent v0.90.0 + controller v0.143.0, 2026-07-17). The customer right-sizes the guest's memory from the controller's Rendszer page; the agent enforces every bound fresh + applies via PVE SetConfig (live cgroup, no reboot, Phase-0 proven). Framing note: the original hub-desired-state framing is SUPERSEDED by Viktor's controller-direct ruling (2026-07-17) — the resize is controller→agent local-API, never through the hub. Memory only (cores stay observation). COMPLETE — the apply is proven in BOTH directions on metal (2026-07-18): customer zero shrank the guest 11675 → 8192 MB (16:50:22) and grew it 8192 → 12288 MB (17:02:17), each a live cgroup apply with no reboot, the new total rippling into the deploy page's memory math. The nested demo had proven the refusals; the N100 proved the applies. tests/VALIDATION-n100-rehearsal-2026-07-18.md ledger 9. Cores/live-resize as hub desired-state is NOT built (deferred, low demand). |
M | SHIPPED + PROVEN-LIVE | See felhom-agent + felhom-controller REPORTs; capability-map row "customer right-sizes guest RAM". |
| R-12 | Cluster mode: agent-follows-guest, bind-mount reconciliation on HA migration | XL | idea | Scoped 07-15; interim = HA-group pin to one node. Driven by Peti's two-node cluster |
| R-13 | OOB management arc: dual-use existing WireGuard + hub desired-state channel as mutual-repair | L | first slice PROVEN-LIVE (poke channel) | FIRST SLICE PROVEN-LIVE — the agent-plane poke channel (Direction-2a), agent v0.89.0 + hub v0.59.0 (2026-07-17): the ep0-relayed contentless poke (hub→ep0 felhom-poke forced-cmd→UDP→box WG /32:51822, peer-confined, zero ep0/box infra change) reaches the agent and fires an immediate desired-state cycle. Full path live: real operator manifest save → sync-poke delivered to 10.77.0.2; box → poke received → immediate desired-state cycle (~31 ms ep0→box, save→tick ≈ ~0.45 s). This is ONLY the listener+sender; the rest of the mutual-repair arc (self-heal actions over the channel) stays open. Per SPIKE-immediate-sync-transport-2026-07-16 P4. The controller-plane Direction-2 wait channel (hub v0.58 / controller v0.140) shipped the config-puller leg separately |
| R-28 | Agent fast-tick-until-first-convergence — SHIPPED (agent v0.90.0, 2026-07-17). Closes the immediacy leg a hub poke can't reach (the pre-tunnel WG-registration window): while ANY desired-state item is unapplied the agent pulses the out-of-band trigger every 30 s and self-disarms on convergence (state-based; four cached sources — desired-gen==0, reconcile Planned−Pending>0, pbsdr waiting_secret ONLY, wgtunnel desired-not-operational; LOUD states + pending_signature excluded). LIVE on both demo agents (startup line verified). Guests-0/0 passenger: diagnosed live and REFUTED as a bug — the 0/0 was the legitimate pre-provision reporting window; the fast-tick is exactly its mitigation (≤15 min → ~30 s). Unfired live: the real-onboarding proof (a fresh box converging in seconds without a poke) → the rehearsal S5. | M | SHIPPED | See felhom-agent REPORT (v0.90.0). Third immediacy leg alongside R-13 (poke) + R-23 (UI). |
| R-14 | Headscale/WireGuard spike: Minecraft/gaming port connectivity (CGNAT-proof, sovereign DERP fallback) | M | idea | |
| R-15 | Multi-user dashboard accounts (household members, roles) | L | idea | Single password is a stated alpha limitation (R-11) |
| R-21 | Bare-metal Felhom ISO — per-PVE-release auto-install ISO for blank customer hardware → first-boot wrapper (invokes felhom-host-install.sh) → universal secret-free / operator-bind (option C) |
XL | SHIPPED + PHYSICALLY CLOSED (slices A+B+C; rehearsal executed 2026-07-18) | PHYSICAL CLOSURE 2026-07-18 (tests/VALIDATION-n100-rehearsal-2026-07-18.md): the generic pairing ISO v1.20.0 (--loader mkimage, SB off) booted the very AMI board that F1 blocked, installed unattended, and the box self-registered as an unclaimed appliance the same second it first booted (16:17:14) → self-bind → credential → day-0 SUCCESS 16:32:32 → floor-lifted to current. F1 is closed on physical hardware and the rehearsal dependency on this item is discharged. Two residual notes stay open, neither blocking: PXE/network-boot is still unbuilt (the third F1 option, for boards where even USB-mkimage fails), and mkimage remains unsigned → SB must be OFF. The installer's GRUB menu still offers interactive installers → new item R-38. — Prior: PHYSICAL RUN 2026-07-16 (tests/VALIDATION-n100-baremetal-2026-07-16.md): demo N100 reinstalled clean-slate from a pipeline ISO → chain reached rc-0 first try on real hardware (closes slice A's operator-gated boundary), serial-filter safety proven on metal, PBS-DR reconciler self-healed on the reused peer, DMI verdict = key on MAC+UUID. F1 (HIGH, slice-B input): this cheap AMI AN3PLUS 0.01 firmware won't UEFI-boot the ISO's GRUB from USB (relocation 0x0) — SB-off/shim-bypass don't help; worked around live with a grub-mkimage loader built from the box's own GRUB. Pipeline must ship a firmware-compatible loader / PXE path. Reused-customer edges (F2 claim re-issue, F3 offsite re-issue, F4 non-default-storage-id ACL 403) feed R-1/Peti. UX: F6 drive-init doesn't mount+attach, F5 guest-RAM not configurable, F7 back-route. — Slice A (build pipeline + first-boot bootstrap) DONE + validated on VM 310: build gate/red-proof, disk-filter fail-safe, stub→retry-unit→real public-channel host-install fetch+invoke→retry, resume-decision, exactly-once, no-net retry+recovery all GREEN. Operator-gated remainder: host-install rc-0 terminal success (drill customer needs the password-gated create-UI). Slice B — SHIPPED (scripts v1.18.0, 2026-07-17): the F1 firmware fix is now a first-class pipeline mode `build-felhom-iso.sh --loader shim |
| R-29 | The design-v2 green gates are not enforced anywhere — one has been RED for 16 releases. controller/scripts/docker_run_volume_path_gate.py has failed continuously since 2026-07-14 (v0.129.0) and nobody noticed until R-7b's close-out ran it by hand at v0.145.0. Two separable parts. (a) The finding itself is benign and the fix is 3 lines. The flagged call is internal/appexport/estimate.go:179 docker run --rm -v <volumeName>:/vol:ro alpine du — a NAMED-VOLUME mount, i.e. daemon-side with no host path, which is the safe shape and byte-for-byte the same pattern as three entries already on the gate's ALLOWLIST (export.go volName+":/vol", backup.go volName+":/vol:ro", restore.go volName+":/vol"). It is NOT the v0.124.0 path-strand class the gate exists to catch — the author of the v0.129.0 F-A fix explicitly avoided that class (see the function's own comment) and simply never added the allowlist entry. So the fix is an ALLOWLIST addition WITH ITS WHY, not a docker-cp rewrite; anyone who 'fixes' this by rewriting the call has misread the gate. (b) The systemic half is the real item: the gates run only when a human remembers to run them, so a gate can sit red across 16 releases while every REPORT says 'green'. This is the SECOND instance of the class — cf. the v0.123.0 note 'Windows green gate silently red (read-only fsync)'. Decide where they run (pre-push hook, build.sh step, or a CI job) and make a red gate block the train the way the Go green gate does. |
S (a) / M (b) | idea | Origin: R-7b close-out, felhom-controller REPORT §4(f) — CC correctly left it alone as out-of-scope and pre-existing, and verified by stashing that it fails identically on the unmodified tree. Flips no capability-map row (engineering hygiene, no customer-visible behaviour). Affected gates to audit for the same rot: controller template_id_gate / emoji_gate / native_confirm_gate / offbox_rename_gate / mojibake_gate / app_row_dedup_gate / docker_run_volume_path_gate, hub hub_confirm_gate, manifests manifest_bearer_gate, website site_gates. Do not bundle (a) into an unrelated feature commit — it is a one-line behavioural claim about a mount's safety and deserves its own reviewed diff. 2026-07-18 rehearsal note: the run's finding list independently re-raised "assign the pre-existing docker_run_volume_path_gate failure its ID so red stops normalizing" — that is this item; no second ID was minted. |
| R-37 | Post-RESET health card shows stale pre-RESET warnings. After a RESET the card should read „RESET óta nincs adat" instead of carrying warnings about a lifecycle that no longer exists. | XS | SHIPPED (hub v0.67.0, 2026-07-18) | The customer page raises a banner when a RESET completed after the newest report, quoting „RESET óta nincs adat" and the reset timestamp, because until the box reports again every health figure describes a lifecycle that no longer exists. Deliberately narrow: an in-flight reset does not trigger it (only a completed one), and it clears itself on the first post-RESET report. Ties resolve to STALE — SQLite timestamps are second-resolution and a same-second report almost certainly arrived just before the reset destroyed what it describes; erring the other way would hide the banner exactly when it matters most. Red-proofed (neutering the predicate fails the assertion). — Origin: 2026-07-18 rehearsal. Same family as R-36 — the hub knows the state changed and the UI has not caught up |
| R-38 | Installer GRUB slice. A single default „Felhom telepítés" entry; the interactive installers REMOVED (safety: an interactive entry is how a wrong-disk manual install happens); felhom background. | S | SHIPPED (scripts v1.22.0, 2026-07-19) | Origin: 2026-07-18 rehearsal, alongside R-21's physical closure. Every ISO is repacked after prepare-iso: exactly one entry („Felhom telepítés", default, 5 s), with Graphical / Terminal UI / serial and the whole Advanced Options submenu (nomodeset ×2, three debug variants, Rescue Boot, memtest, UEFI settings) not emitted — not hidden, not password-gated. Boot behavior unchanged: the linux/initrd lines are lifted verbatim at repack time from the ISO's own automated entry, so a PVE bump tracks automatically; the build fails if they are missing, if the append line has lost proxmox-start-auto-installer, or if auto-installer-mode.toml is absent. Gated for 1 entry / 0 submenus / no live banned references, then re-verified by reading the menu back out of the finished ISO. Background generated at repack time from website/assets/og-image_2.png (one source, no second copy to drift), with the card's own grid phase-locked across the letterbox fill. Live-validated on the nested canary (UEFI/OVMF): renders, auto-fires, reaches the same match-nothing abort as v1.21.0. Squashfs/theme rebranding still DEFERRED — post-GRUB screens remain Proxmox-branded. GRUB's e/c keys also remain reachable (suppressing them needs a superuser password — a separate decision); the menu no longer advertises them |
| R-40 | [P2-HIGH] The update path cannot express a MULTI-HOP major upgrade. A template pin is a single value; the customer's update button pulls whatever the catalog now says. For apps whose upstream forbids version skipping this produces a broken upgrade. Nextcloud is explicit: "You cannot skip major releases. Please re-run the upgrade until you have reached the highest available release." Campaign 7 moved its template 31 → 34 (a fresh deploy validates fine — 302, 3/3 healthy), so an existing 31 customer pressing update would attempt a jump Nextcloud refuses. | M | idea | Origin: CAMPAIGN 7 (audits/CAMPAIGN-7-catalog-sweep-2026-07-19.md §7 F7). Not nextcloud-only — any app with sequential-major rules (gitea, tandoor, outline…) has the same shape. Directions: a per-app upgrade_path:/max_hop: in .felhom.yml that the update button walks in stages; or refuse-and-explain when the installed major is >1 behind; or pin an intermediate "stepping-stone" tag. Until this exists, a >1-major catalog bump is safe for NEW deploys and unsafe for the update button — which is exactly the asymmetry the campaign's MAJOR flag was meant to record but cannot enforce |
| R-41 | [SLICE 1 SHIPPED 2026-07-21] The catalog has no standing "does every template still deploy?" check. Campaign 7 was the first thing that ever tried to deploy all 53 apps, and found 5 that had NEVER been deployable: papra (missing required AUTH_SECRET), zipline (v4 renamed CORE_DATABASE_URL → DATABASE_URL), wishlist (Docker Hub image gone; upstream moved to ghcr.io), homebox (upstream dropped the v tag prefix + new required env), glance (needs a seeded glance.yml the template never provides — PROVEN pre-existing: the pre-campaign v0.7.4 pin fails identically). Plus 7 broken healthchecks and 2 apps whose images no longer resolve at all (plant-it, wanderer). | M | idea | Origin: CAMPAIGN 7 (§7 F5/F6). The repo already has the right pattern in scripts/check-image-pins.py — a mechanical gate run on every change. Cheap first slice: a resolvability gate (docker manifest inspect every pin) would alone have caught plant-it, wanderer, wishlist and homebox, and needs no box. Full slice: a periodic deploy-all sweep on the demo box reusing the campaign's engine. Silent rot is the real risk — an app can die upstream and nobody learns until a customer clicks Telepítés | SLICE 1 SHIPPED 2026-07-21 — app-catalog-felhom.eu/scripts/check-image-resolvable.py (+ 14 fixture tests, no network, resolver injected). Resolves every unique pin with docker manifest inspect, ONE image at a time; exit 0 / 1 (the registry says GONE) / 2 (inconclusive). Two traps encoded, both hit live while building it: (a) docker manifest inspect prints toomanyrequests: … and still exits 0 — the same exits-0-on-failure shape as the ISO tooling's validate-answer, so stderr is inspected even on rc=0; (b) the inverse and more dangerous one — the first full sweep called 24 of 65 pins dead, including postgres:16-alpine and redis:7-alpine, purely because Docker Hub throttled it partway through. Ambiguity therefore resolves to INCONCLUSIVE and never to an accusation: a gate that cries wolf gets ignored, and then it protects nothing. The full sweep is still OWED — DooPlex is not logged in to Docker Hub, so the 52-app table needs one re-run after docker login. Wired into CLAUDE.md + REUSE.md as a start-of-campaign / pre-publish-train step. It immediately paid for itself: it is what turned plant-it and wanderer from 'images do not resolve' into two DIFFERENT diagnoses (see the 2026-07-21 catalog entry). Full slice — the periodic deploy-all sweep on the demo box — remains open |
| R-42 | RULED: sidecar majors follow the APP, never the newest tag. Campaign 7 deliberately did NOT bump sidecar majors (postgres 16→18, redis 7→8, mariadb 11.6→12.3, postgis 16→17) while bumping ~40 app images to current. | S | RULED 2026-07-21 — option (a) | Origin: CAMPAIGN 7 §4. The case for not bumping: a DB major is a data-plane decision the application owns — immich proves it, upstream pins one specific tested postgres:14-vectorchord… build — and postgres:16-alpine/redis:7-alpine already track the newest patch inside their major, so they are not stale. The case for bumping: EOL majors eventually stop getting security patches, and "we never bump" silently becomes "we ship EOL databases". Suggested shape: per-app sidecar pin follows upstream's own compose where upstream publishes one, else stay within the current major and revisit at that major's EOL date |
| R-43 | [P2-HIGH] No offsite restore path can restore a database — offsite restore cannot reconstitute a DB-indexed app. All three offsite buttons are file-only. „Ellenőrző visszaállítás" and „Teljes visszaállítás" restore into a scratch/verify folder and never load a dump into postgres (controller/internal/backup/offbox_restore.go:171-232, :130-164); „Helyreállítás az élő adatok közé (csak a hiányzó fájlok)" is rsync -a --ignore-existing over mandatory userdata paths only — it never touches a DB and never stops/starts the stack (offbox_restore.go:346-421). The DB dump is faithfully carried into the snapshot and can never be replayed out of it by any offsite action; the only code that replays into a live DB is RestoreFromRecoveryUnit (backup/restore_unit.go:74), reachable solely from the local restore panel (web/handlers.go:1108). Compounding: the live recovery unit present — not overwriting guard (offbox_restore.go:400-405) skips the entire unit — compose, manifest and db-dumps — whenever a local backups/primary/<app> exists, which is true on any box whose local backup has run, so on a normal box the offsite dump is never even staged to disk. | M | SHIPPED controller v0.148.0 (2026-07-19) — live acceptance PENDING | SHIPPED: new „Teljes visszaállítás (fájlok + adatbázis)" (ReconstituteFromOffsite, /backup/offbox/reconstitute): safety dump → stop → files overwritten to the snapshot version → start → the snapshot's own dump replayed (reimportDBDumpsFrom, reading the SCRATCH unit — the live unit is still never overwritten) → health wait. Two invariants, both red-proofed: nothing is ever deleted (rsync -a, no --ignore-existing, no --delete; post-snapshot files survive as extras) and the undo exists before the act (the pre-restore- dump is verified ON DISK before anything is stopped/overwritten/replayed; if it cannot be taken the operation refuses with zero changes). Outcome-shaped flash; the old missing-only button now states what it does NOT do. Deployed to demo guest 9201, healthy; page renders, button correctly gated behind a prepared full scratch. STILL OPEN — the §9 live acceptance has NOT run (upload → push → empty the trash for real → one button → photos back in the timeline). Until it does, no capability-map flip, the customer-restore row stays MISSING and R-3 stays DRAFT. — Evidence: audits/DIAG-immich-restore-2026-07-19.md (live on demo-felhom). Viktor deleted 11 immich photos to test offsite restore; both place runs merged 0 files and flashed success. Root cause is two-layered — the UI delete only trashed the assets so nothing was missing to restore, but a valid test would also have failed: files return, the library stays empty, because immich's truth lives in postgres. The customer-visible failure mode: correct bytes on disk and an app that cannot see them. Severity case: this is the offsite tier — the one that survives losing the box — and most of the catalog is DB-indexed. Direction (needs a spec, do not improvise): either offsite gains a real DB-replay leg reusing RestoreFromRecoveryUnit behind a stop→restore→start sequence, or the offsite UI states plainly that it restores files only and routes the customer to the local recovery-unit flow for the DB. Pairs with R-44 (a replay leg is worthless while the dump is stale). Flips: the customer-restore map row stays MISSING and gains this as a note — the C6 evidence attempt found a product gap, which is the rehearsal system working |
| R-44 | [P2-HIGH] A manual offsite push ships an unrefreshed DB dump — "backed up now" is false for the DB half. offboxRunHandler → RunOffboxBackup goes straight to the restic push and never calls RunDBDumps / captureAllRecoveryUnits (controller/internal/web/offbox_handlers.go:203-227, backup/offbox.go:574-759); the recovery unit merely enumerates existing dump filenames via listFileNames, never creates them (backup/recovery_unit.go:105-106). Dumps come only from the separate local db-dump daily at 02:30 (cmd/controller/main.go:542), with the scheduled offsite at 04:15 — so a manual run at any other hour ships a dump up to ~24 h old. There is no freshness check and no RPO surface anywhere: zero RPO hits across controller/; offboxUnitTime is only a two-drive tiebreak (offbox.go:827-837); the DBValidationCache exists (backup.go:364-370) but no offsite or restore path reads it. | S–M | SHIPPED controller v0.148.0 (2026-07-19) | SHIPPED: every offsite run — manual AND nightly — now refreshes the DB/volume dumps and recovery units (offsitePreDump → runDBDumpsInternal) BEFORE the restic capture, so each snapshot is an internally coherent {DB@T, files@T} bundle and retention becomes a history of restorable points. Order is the mechanism and is red-proofed (moving the capture first yields [capture dump]): the gap can only ADD files the DB does not reference yet, never remove one it does. This also makes the nightly ordering structural rather than a coincidence of two scheduler entries at 02:30 and 04:15. Each unit manifest carries offsite_run_id + dumps_at, so a pair's coherence is verifiable at restore time instead of assumed; the periodic refresh carries a prior stamp forward and never invents one. A dump-leg failure is a loud WARN that does NOT abort the push (data-first: a degraded backup beats none). Honesty surfaces, all warn-level and none a gate: an unstamped pre-v0.148 pair reports its skew in the confirm, and ValidateDump gained an exact-match accounts-table sniff for customer-empty dumps (a substring match on "user" would flag user_metadata/album_user/user_audit on every healthy single-user box — red-proofed). — Evidence: audits/DIAG-immich-restore-2026-07-19.md. Today's unit dump immich-postgres.sql (51 954 452 B, mtime 02:30 CEST) probed to asset: 0 rows, user: 0 rows, album: 0 rows — the 52 MB is entirely immich's shipped geodata_places/naturalearth_countries reference data. It predates both the admin user (created 07:56:25) and the photos (07:57). Same for the unit's immich_immich_postgres_data.tar (323 MB, also 02:30). A dump that looks substantial by size can contain zero customer content — size is not a health signal, and nothing in the product says otherwise. Latent hazard: had a full restore actually loaded that dump it would have written an empty DB over the live one, destroying the trashed rows that were the only surviving recovery path. Direction: dump-before-push on manual runs (the honest fix), or an explicit RPO line in the UI („adatbázis-állapot: ") so the operator/customer can see what they are actually shipping. Cheap interim: surface dump mtime + row-count sanity from the existing DBValidationCache on /backups/restore |
| R-45 | [P2] Unified async-job feedback. Every long operation invents its own progress surface, or none. Tonight produced three more one-off cards (v0.147.x: samba bring-up, offsite progress, restore result) on top of two existing patterns (deploy 3-step panel; storage-init/netstorage status poll). They agree on nothing: some use {ok,data} envelopes and some raw JSON, some poll 1 s / 1.5 s / 3 s, some are in-memory-only and lie after a restart, and each re-implements single-flight + snapshot + phase→Hungarian mapping. | M | idea | Origin: 2026-07-19 feedback slice 1 (controller v0.147.0). The cases to generalise from are all in-tree: web/storage_init_job.go (the best shape — acquire/release/set/snapshot), web/netstorage_job.go, web/samba_ensure_job.go, backup/opstatus.go, backup/offbox_progress.go. Shape: one job registry + one poll endpoint + one client-side renderer, phases declared per job. Two lessons tonight that any framework must encode: (1) a terminal state must be probed, not inferred — compose up -d exits 0 on a crash-loop; (2) a progress source that reports nothing is normal, not broken — restic reports 0 bytes for a whole incremental run, and a bar that sits at 0% is worse than no bar. Also fixes the restart hole: in-memory job state currently vanishes and the card silently disagrees with reality 2026-07-20 — the first bill for NOT having this arrived, and it was customer-facing. The samba card's poll (web/samba_ensure_job.go + sharing.html) mixed a job EDGE and a service LEVEL on one JSON field, and /sharing reload-looped at ~1.2 s for every customer with sharing enabled until controller v0.151.0 (audits/DIAG-sharing-2026-07-20.md, S-1/S-4). v0.151.0 fixed THAT card's contract only — the framework is still this item. Third lesson for it to encode, beside the two already listed: a phase a client answers with a one-shot action must be an EDGE the registry SERVES ONCE, and must never be synthesised from a level; if it can be re-read, it will be re-acted on. |
| R-46 | [P2] Verification copies need a customer-visible browse surface and an expiry. v0.147.0 made them visible (listed with path/size/date, individually deletable) — but the customer still cannot LOOK INSIDE a verification restore to confirm the file they wanted is really there, which is the entire point of a verification restore, and nothing ever removes them. | S–M | idea | Origin: 2026-07-19 feedback slice 4a, registered as the explicit follow-up to it. Two gaps, deliberately designed together because they are the same object: (a) the invisible-result gap — a read-only browse of backups/offsite-restore/<app> (the FileBrowser infra stack already exists and already serves scoped roots, so this may be a mount rather than new code); (b) the disk-lifecycle gap — auto-expiry after N days with the count/size surfaced before it fires, so a drive is never quietly filled by verification restores nobody remembers taking. Pairs with R-43: a browse surface is also how a customer would discover that a DB-indexed app's files came back but the app still cannot see them |
| R-47 | [P2-HIGH] The DB replay races the application's own schema repair. | M | SHIPPED — controller v0.153.0, 2026-07-20 | Fixed on BOTH paths (it was a class defect, not an offsite-only one): offsite ReconstituteFromOffsite AND local RestoreFromRecoveryUnit now open a DB-only window — appbackup.DBServiceNames names the database SERVICE from the compose services: map, stacks.Manager.StartStackServices brings up only that service, the dump is replayed with the application still stopped, and the full start runs only after the replay exits 0. Fail-closed: a .sql dump with no identifiable DB service refuses BEFORE the first mutation. Every exit from the window (replay failure, DB-only start failure) still does a best-effort full start, so a failed restore never leaves a box with a database and no application. --clean --if-exists + ON_ERROR_STOP=1 untouched — the bug was the window, not the flags. Enablers: RedeployFromEnv split into PersistUnitRedeployConfig + its unchanged tail; StackDataProvider.RecreateStackFromUnit → RecreateStackDefinitionFromUnit (the hidden up -d inside the old name is what carried the defect locally). No agent coupling — MinAgent stays 0.90.0. 19 tests + 3 red-proofs, 23/23 green. Unblocks the capability-map offsite-restore (PARTIAL) and customer-restore (MISSING) rows, both of which now await only one clean acceptance run. LIVE-VALIDATED 2026-07-20 against the SAME snapshot that aborted in round 2 (49e7cb46): log shows Starting stack immich services only: [immich-postgres] → replay rc-0 in 20 s → full start; no already exists; immich's own DatabaseService logged No schema drift detected twice (round 2 left it reporting drift); 11 assets active, 4/4 containers healthy. Golden 0.153.0 baked + published the same day (sha256 15fdd191f3c660a6…) — the first golden carrying all FOUR infra images. Evidence: audits/DIAG-immich-restore-round2-2026-07-19.md (H4) + felhom-controller/REPORT.md §4b/§4c |
| R-48 | [P2-HIGH] Restore controls are separable only by layout — and the difference between them is whether the data comes back. The offsite restore row renders four buttons plus hint text into an overlapping, unreadable line, and the decisive second step („Teljes visszaállítás indítása") appears ONLY after „…előkészítése" was pressed, with no signposting that a second step exists or that the first one did nothing to live data. | M | idea | Evidence: audits/DIAG-immich-restore-round2-2026-07-19.md (finding 1) — this is not theoretical: it is the CAUSE of the round-2 incident. An operator who had read the code pressed the missing-only button instead of the full restore; the controller log shows /backup/offbox/reconstitute was never hit at all. The rule this establishes, worth stating once and applying beyond this page: two adjacent controls whose difference is "your data comes back" vs "your data cannot come back" must not be distinguishable only by layout. Direction (ruled in principle, spec rides v0.149): collapse to a single „Visszaállítás…" guided dialog — one intent, visible phases, the escrow-wizard precedent. Pairs with R-45 (the phases are exactly the async-feedback surface) and R-46 SHIPPED 2026-07-21 — controller v0.154.0 (3a9d744). Each app row on /backups/restore now carries ONE „Visszaállítás…" entry linking to a per-app wizard at GET /backups/restore/app?name=<app>: three intent CARDS each with a consequence sentence (ellenőrzés külön mappába / hiányzó fájlok visszahozása / teljes visszaállítás), a visible phase strip so the sequence is legible before the first click, danger styling on the destructive card, and the R-43 double-confirm carried over verbatim with its pair-honesty facts. deriveWizardStep is a PURE function of (op running, size-gate flash, scratch ready) — the step is never taken from the request, and a running op outranks a stale ?full_prep= so no commit button survives into a restore. While ANY op runs every mutation form is suppressed server-side rather than offered and then refused. No new mutation endpoint (one GET route; every card posts to the pre-existing /backup/offbox/* with unchanged field names and gates) and no R-45 graft — the wizard polls the two existing status surfaces as-is. Works with JavaScript disabled. Latent bug fixed on the way: offboxRedirectTo hardcoded "?" when appending its flash, which against the wizard's ?name=<app> target would have buried the flash inside the app name. 9 new tests + the Group-B red-proof (trivial always-INTENT impl → all 7 rows red). Live click-through + one non-destructive Ellenőrzés still PENDING (rides the operator's floor save). Evidence: felhom-controller/REPORT.md §3 (2026-07-21). |
| R-49 | [P2] The offsite capture set is ~90% cache and duplication — 1.1 GB of a 1.2 GB immich "photo backup". Measured 2026-07-19: immich_ml_cache.tar 823 660 032 B (~60%) — re-downloadable ML model weights; immich_postgres_data.tar 308 251 136 B (~23%) — a raw tar of the postgres data dir that DUPLICATES the logical .sql dump captured beside it; upload/backups/ 18 MB — immich's own nightly dump, a backup inside the backup, growing daily; plus the stranded pre-v3 dccc13fe… tree (~36 MB) no DB has ever referenced. Actual irreplaceable content: 72 MB of originals. | S–M | idea | Evidence: audits/DIAG-immich-restore-round2-2026-07-19.md §4 (full byte breakdown). This is the customer's offsite quota and transfer cost, and it lands on the Hetzner sub-account they are billed for. Recorded, deliberately not changed — a capture-set exclusion is a data-loss-shaped decision and gets its own ruling, not a drive-by edit. Candidates in priority order: (a) immich_ml_cache — pure cache, strongest case; (b) the postgres_data volume tar where a logical dump of the same DB is already captured (the dump is what the restore path actually replays); (c) upload/backups/. Likely generalises past immich into a template-classification rule about cache volumes and self-backup directories, so it should be specified against the catalog, not one app |
| R-65 | Buddy-box backup replication, cross-household — two Felhom boxes in different homes replicate backups to each other. | L | idea (post-alpha, spike-first, 2026-07-22) | The natural big sibling of R-64: two households each hosting the other's encrypted backup tier. Explicitly spike-first — the transport is NOT SMB (R-64's live-share protocol is wrong for backup replication across the internet: no auth story between households, no resumability, cleartext LAN assumptions); candidates to spike: restic rest-server / rclone / syncthing over the existing WG/tailnet plumbing, encryption keyed so the buddy can never read the payload. Sits on top of the offsite tier's FILL/OVERSUB thresholds thinking (R-5 aggregate). Flips: would add a "cross-household buddy replication" capability row (currently unlisted). Pairs with R-64 (same topology, different transport + guarantees) |
Pre-invite checklist — what stands between here and the first remote tester
Not roadmap items in their own right; the short list the 2026-07-18 rehearsal leaves behind. Everything here is remote-doable — the N100 is packed, and none of it needs hands on the box.
| Action | Owner | Note |
|---|---|---|
| Viktor (saves) | Golden 0.146.0 baked on the drill VM and published to gitea — felhom-golden/0.146.0/golden.tar.zst, sha256 4834c703162c5437467a329144b1a523019bf5693ab9d439558be7323587e955, 612 696 588 B (584 MB archive). All pass markers green: Result=success/ExecMainStatus=0, 0 FATAL/exclusions, docker OK (overlay2), all three mounts included (rootfs + mp0 /var/lib/docker + mp1 /mnt/sys_drive), pre-delete 404, upload HTTP 201; controller 0.146.0 confirmed baked in. Integrity round-trip independent of the build host: anonymous `GET |
|
| Golden ≥ 0.147.x carries ALL FOUR infra images | — (next bake) | build-golden.sh v2.1.0 (2026-07-19) now derives the pre-pull list from the controller binary it is about to bake (--print-infra-images) instead of a hand-maintained copy that had already drifted: felhom-samba was never added to it, so every golden so far baked 3 of 4 — which is why enabling Megosztás on a fresh box pulled from the registry with zero feedback. No golden rebuild for this alone; it takes effect at the next bake. Until then a fresh box still pulls felhom-samba at enable time, which controller v0.147.0's progress card now at least explains |
| freemail.hu test-send | Viktor | The open half of R-4; the gmail half closed on 2026-07-18 under p=quarantine |
| C6 — customer performs a restore, unassisted | Viktor as customer zero | The one open script step in R-3 and still MISSING as capability evidence. Remote-doable on the reborn box — the dashboard is remote |
| R-11 rulings | Viktor | Contact channel, tester agreement, alert thresholds (the R-5 gauge thresholds are still pending a ruling) |
Absorbed / superseded notes in this folder
FOLLOWUP-nas-automount-guest-reboot-reassert.md— shipped (agent v0.84/v0.85, CAMPAIGN-3); keep for historyFOLLOWUP-golden-default-controller-tag.md— verify against current golden flow; close or promote to an itemFIX-M18-NOTES.md,FIX-M19-NOTES.md,DIAGNOSIS-f9-storage-registration-gap-2026-06-14.md— historical diagnoses; superseded by shipped fixes