What we built, what happens when things go wrong, and what is still missing. Generated from where-felhom-stands.yaml, which cites the capability map, register row or evidence document behind every claim — and which is checked by scripts/check_stands.py.
Seven stages, left to right, as they happen to a person.
1 Getting a box
A blank machine installs itself from our own boot image and registers itself as unclaimed — proven on two different boardsTwo boards: N100 2026-07-18, HP t740 2026-07-21.capability-map Bare-metal Felhom ISO (blank hardware → zero-touch auto-ins… · evidence tests/VALIDATION-n100-rehearsal-2026-07-18.md
The installer is published rather than pushed: rolling it back is one actchanged 2026-08-09, was walkedGate 6 of hostinstall_gates asserts the manifest names an installer-v tag; ran green tonight.why it moved: gate 6 asserts the manifest names an installer tag, but no walk of a rollback is on filecapability-map The installer is PUBLISHED, not pushed · register R-110
Installing onto hardware the customer already owns — the path exists, the first real one has not happenedUPGRADED: a real --mode byo install completed on demo-hp 2026-08-09 (Day-0 provision SUCCESS, 3m49s). Still not a customer's own hardware, so not 'walked' — but 'has not happened' is now false.evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md
A machine that ends up on the wrong network port explains itself on screen and finds its way back — proven in a virtual drill, never on metalMap says the same: PROVEN-LIVE (nested drill — nested != metal).capability-map Box survives a wrong-NIC install · evidence audits/SPIKE-firstboot-nic-sweep-2026-07-22.md
Reinstalling a machine we previously installed refuses, because our own removal leaves a name service holding the port our own installer checksWarning card. Confirmed live 2026-08-09; dnsmasq restarted by --uninstall seizes :53.register R-272 · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md
2 Making it theirs
An emailed one-time code; the customer sets their own password and the operator never sees itExercised live 2026-08-09: reset code accepted at /claim, 302 + session.capability-map Customer claim: one-time emailed code → customer sets own p… · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md
The customer can bind their own machine from a link, with no operator present — done once, for realattempts=0, locked=0, source customer_selfbind, 2026-07-18.capability-map Customer binds their own appliance (self-service) · evidence tests/VALIDATION-n100-rehearsal-2026-07-18.md
It has never been done by a person who is not the operatorMap records MISSING (as evidence). Still true after 2026-08-09.capability-map A customer (not the operator) performs a restore via UI alone
Each of the two secrets has one name on every surface, and the re-enrol mail points at a page a rebuilt machine showsShipped (controller v0.211.0, hub v0.104.0/v0.105.0); no customer has typed the passphrase yet. Operator ruling 2026-10-12 (S13). The old defect cost a real code on 2026-08-09.register R-282 · register R-295 · register R-323 · register R-327
3 Using it
Apps installed from a catalog of about 52, with a memory guard and health-aware progress53 templates listed on the rebuilt box 2026-08-09; deploy driven live.capability-map Deploy an app from the catalog (env config, memory guard, h… · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md
Start, stop, restart, update, logs, remove — and the parts that must not be stopped cannot bechanged 2026-08-09, was builtWalked 2026-10-28 night on scratch 9202: install, stop, start, restart, logs; traefik and cloudflared refuse a stop (403); remove with data leaves no volume and no container. The update half: the same day's guarded updates on 9202 (bench + box + undo walks).why it moved: a walk document is now cited (2026-10-28 night)capability-map App lifecycle: start/stop/restart/update/logs/remove/redeploy · register R-108 · evidence audits/night-2026-10-28/part5-claims/lifecycle-walk.log · evidence audits/dockerhub-bigsteps-2026-10-28
Reachable from anywhere through a tunnel, per-app addressesVerified tonight: the rebuilt box answered on its public URL from outside.capability-map Remote access via Cloudflare Tunnel + Traefik (per-app subd… · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md
Reachable on the home network when the internet is downWalked 2026-11-03: with the internet gone the dashboard answers at the box's own address, and the box's DNS answers every app's name. By NAME it needs the home network's DNS to point at the box, and nothing tells the household; on a host whose FQDN is the dashboard's name (Tester 1) that one name answers the host and fails (R-1364).evidence audits/security-2026-11-02/part5/lan-resolver.txt · evidence audits/security-2026-11-02/part5/outage-wan.log · register R-1364 · capability-map LAN access when internet is down (lan_resolver)
Phone photos, documents with text recognition, files from Windows Explorer or a Maccapability-map Files from Windows Explorer / Mac Finder (SMB server) · evidence audits/SPIKE-lan-discovery-2026-07-18.md
A one-tap launcher, and a read-only guest link for visitorschanged 2026-08-09, was builtWalked 2026-10-28 night on scratch 9202 through the pages the browser receives: the launcher lists each openable app; the guest link opens with no login and no admin link, a wrong token is 404, rotate and switch-off kill the old link.why it moved: walked live 2026-10-28 night (map E8, E9 → PROVEN-LIVE)capability-map Indítópult (app launcher) — one-tap grid of the household's… · evidence audits/night-2026-10-28/part5-claims/launcher-walk.log
Media to a TVMap: MISSING.capability-map Media to TV via DLNA
Separate accounts per household memberMap: MISSING.capability-map Multiple household users / per-person accounts
4 Drives
A new drive is found, offered, formatted, mounted and enrolled — including on awkward older boot layoutsmoved twice: 2026-08-09 walked -> built, then 2026-08-10 back to walkedApplies to a NEW drive. Re-attaching an existing one after a reinstall is R-280 and fails.why it moved: receipt found 2026-08-10: a live throwaway drive on felhom-pve walked scan -> format -> mount -> PVE dir storage -> one-click Regisztralas, with the resulting storage.cfg entry and systemd mount unit recorded. The map already read PROVEN-LIVE; the dataset was behind it. Scope unchanged: this is a NEW drive. Re-attaching an existing one after a reinstall is R-280 and still fails.capability-map Drive wizard: scan/format/mount/enroll, incl. legacy-boot L… · register R-220 · evidence audits/SPIKE-raw-drive-enroll-2026-06-15.md
Moving data between drives, crash-safe; removing a drive safely; unplug detectedchanged 2026-08-09, was walkedwhy it moved: no walk document citedcapability-map Data migration between drives (all / per-app), crash-safe
A network drive can be browsed and hold bulk media, but may not hold an app's data — enforcedRefuseAsAppNamespace is the fail-closed predicate.capability-map Network storage (NAS) is browse + bulk-media only · register R-108 · evidence audits/R108-network-app-namespace-2026-07-30.md
After a reinstall the data drive cannot be re-attached through any dashboard routeWarning card. /api/disks/candidates returns empty; the restore page promises two clicks.register R-280 · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md
5 Backing up
App data on the machine, nightly database dumps, a copy on a second drivemoved twice: 2026-08-09 walked -> built, then 2026-08-10 back to walkedTrue for every app but one, and that one was found on 2026-08-21: paperless-ngx's 72-table PostgreSQL was dumped nightly and written into a folder named after an app that does not exist, so it never entered the recovery unit, the second-drive copy or the off-site copy. Fixed and proven live 2026-08-22 (R-355) — the dump is now in the app's own unit and in the off-site snapshot for the first time. A catalogue-wide sweep, itself proven able to convict a planted case, says 1 of 53 was affected.why it moved: receipt found 2026-08-10: an adversarial, destructive, unattended overnight campaign across both boxes and ep0, with A2 (one quiesce, two tiers) proven end to end. Map already read PROVEN-LIVE.capability-map Tier-2 secondary-drive copy: class-driven legs · register R-355 · evidence audits/CAMPAIGN-8-backup-restore-2026-07-27.md · evidence audits/DRILL-backup-truth-2026-08-21/evidence/phase4-paperl…
A whole-machine archive that lands off the guest's own disk — a single-drive machine is recorded as degraded rather than pretendingchanged 2026-08-09, was walkedwhy it moved: no walk document citedcapability-map Whole-guest backup lands OFF the guest's own physical device · register R-165
An encrypted off-site copy, sealed with a key the operator cannot readchanged 2026-08-09, was walkedDAILY and UNBROKEN, read from the repositories 2026-11-02: demo-hp 14–15 snapshots every day 10-21 → 11-02, Tester 1 4 every day 10-27 → 11-02. The 2026-08 gap's two causes were fixed (R-199, R-193). The key stays the household's.why it moved: the claim is about a CONTINUING daily copy; a 12-day silent gap was found on 2026-08-21 and nothing reported itevidence audits/security-2026-11-02/part5/offsite-demo-hp.txt · evidence audits/security-2026-11-02/part5/offsite-tester1.txt · capability-map Offsite (restic → Hetzner Storage Box) · register R-199 · register R-193 · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md · evidence audits/REPORT-DRILL-backup-truth-2026-08-21.md
The backups prove themselves: a restore is actually performed, unattended, on every tier, on both machineschanged 2026-08-09, was walkedRead 2026-11-02 from the journals of all three boxes: a scheduled restore-test passed on every tier in 10 days — demo-hp local and PBS, Tester 1 local and PBS, the N100 its USB drive and PBS, the newest on 2026-11-02 22:45. R-366's orphaned key was a one-off of the 2026-08 reinstall.proof decay: PROOF-DECAY RULE FIRED (first time it has). A receipt EXISTS - architecture/_recovery-inventory-2026-07-28.md carries live journal lines for scheduled restore-tests on both boxes and both tiers - but it is superseded by later observation: demo-hp logged restore_test_failed on 2026-08-05, and the box has since been wiped and reinstalled (2026-08-09). The claim is about a CONTINUING scheduled behaviour, so a 2026-07-28 observation cannot carry it. Stays grey until a scheduled restore-test is seen passing on the rebuilt box. THE CAPABILITY MAP STILL READS PROVEN-LIVE (2026-08-03) AND IS NOW THE THING OUT OF STEP.why it moved: no walk document cited, and the last recorded restore-test on demo-hp FAILED (2026-08-05)evidence audits/security-2026-11-02/part4/restore-test-demo-hp.txt · evidence audits/security-2026-11-02/part4/restore-test-tester1.txt · evidence audits/security-2026-11-02/part4/restore-test-n100.txt · capability-map Restore-proof is UNATTENDED — the scheduler covers EVERY tier · register R-86 · register R-366 · evidence audits/REPORT-DRILL-backup-truth-2026-08-21.md
The customer is warned before a drive fills, per drive, in their own languagechanged 2026-08-09, was walkedWalked 2026-11-03 on Tester 1: a data drive taken to 89 %; the watcher (every 10 minutes since R-363) warned at its next tick, 2 minutes later: the household's Hungarian mail names the drive and 5,2 GB free, once. The August gap (a daily check) is gone. The cadence leaves a drive up to 10 minutes.why it moved: a daily check cannot carry the word BEFORE; observed silent for the whole window a filesystem sat at 99%evidence audits/security-2026-11-02/part5/fill-walk.log · evidence audits/security-2026-11-02/part5/mail-fill-household.txt · capability-map The customer is warned BEFORE a filesystem fills · register R-167 · register R-363 · evidence audits/SPIKE-r165-mp1-merge-2026-08-02.md · evidence audits/REPORT-DRILL-backup-truth-2026-08-21.md
A backup that covered nothing still calls itself successfulWarning card — and the drill found two more of it, both live. (1) An off-site run with no app selected logs 'backup OK: 0 app(s) backed up'; the card does say 'nincs kijelölt alkalmazás' beside the green tick, so this one is honest if you read past the tick. (2) A restore that placed nothing reported success: '0 fájl visszaállítva', ok=true. The second is the one that matters and it is R-353, still open. The volume half of it is fixed (R-354): the message now names what came back.register R-240 · register R-353 · register R-354 · evidence audits/DRILL-backup-truth-2026-08-21/evidence/phase3-experi…
6 When something goes wrong
The machine watches itself and repairs some faults without telling anyone it had tochanged 2026-08-09, was builtWalked 2026-10-29 night on scratch 9202: a crashed app part (kill -9 of paperless-redis) ran again in 1 s and the health probe read the app healthy; a guest reboot brought every app back in under 2 minutes, the boot sweep found nothing to start; no event, no banner, no mail. Docker's restart policy did the repair in both legs; the controller's own boot repair had nothing to do (its last proof: 6 of 6 hard resets, 2026-08-02). R-264 records that the self-heal counters reach the hub and are decoded nowhere.why it moved: walked 2026-10-29 on scratch 9202 (a walk document is now cited)capability-map Box survives an unattended app or guest-network failure · register R-264 · evidence audits/night-2026-10-28/part5-claims/selfheal/WALK.md · evidence audits/DIAG-bootrecon-drive-absent-2026-08-02.md
Failures reach the operator by email, one mail per run, every failing app namedchanged 2026-08-09, was walkedSeen in the mailbox: „Tester4: 3 of 8 apps failed to back up (manual run)\" names immich, nextcloud and paperless-ngx in one mail (2026-10-20); seven of this shape in 60 days.why it moved: the digest is wired and source-verified, but no run of it has been observed deliveringevidence audits/security-2026-11-02/part5/mail-operator-digest.txt · capability-map A failed per-app Tier-1 backup reaches the OPERATOR — EVERY… · register R-182
Failures reach the customerSeen in the mailbox: the crash mail and the fill warning to tester1@ (2026-11-02/03), „A szerver nem elérhető!\" to tester4@ (2026-10-24), each in Hungarian. It needs an address on the household's page (Tester 1 had none).evidence audits/security-2026-11-02/part5/mail-crash-household.txt · evidence audits/security-2026-11-02/part5/mail-fill-household.txt · evidence audits/security-2026-11-02/part5/mail-operator-digest.txt · capability-map App crashes → customer notified (one event per transition, …
An already-paired box no longer tells anyone to pair it — its console says there is nothing to dochanged 2026-08-09, was partialWas a warning card (the console kept showing the pairing code). Fixed by ISO 1.28.0 (R-535; R-214, R-235 closed). Walked 2026-10-29 00:04 on Tester 1 (paired, claimed): the console reads, in Hungarian and English, that there is nothing to do on this machine and no need to log in — no pairing code.why it moved: the defect is closed and the console was read live on a paired box (2026-10-29)register R-214 · register R-235 · evidence audits/night-2026-10-28/part5-claims/tester1-console-2026-1…
7 Getting everything back
A rebuilt machine shows a full-page recovery screen without anyone looking for it, and says plainly that nobody can replace a lost recovery codeSeen unsought on the rebuilt demo-hp 2026-08-09, seal date matching host_escrow.created_at.register R-193 · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md
The customer's code opens the sealed package and the data returns byte for byte — including accented Hungarian filenames, verified as raw bytesStill true, and re-proven 2026-08-22 (5/5 byte-identical, both accented names as raw bytes) — but the SCOPE is narrower than the sentence sounds and was silently narrower still until v0.218.0. The drill found the off-site restore had NO named-volume leg at all: the archive sat in the unit, the snapshot and the checking folder and was never replayed, under a success message (R-354, fixed and proven 2026-08-22). AND 40 of the 53 catalogue apps STILL cannot run this route at all — it refuses first, saying a running app is not installed (R-356, open). So: proven for an app that declares a data drive; unproven and currently unreachable for the class whose entire dataset is a named volume.register R-201 · register R-354 · register R-356 · evidence tests/walk5-r201-2026-08-07/journal.md · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md · evidence audits/REPORT-DRILL-backup-truth-2026-08-21.md
Walked end to end with no command line inside the machine (2026-08-07)contestedCONTESTED-RESOLVED: not a contradiction. The walk proves the ROUTE needs no guest shell; the map's MISSING row is about a NON-OPERATOR doing it, which has still never happened. Two questions, one word 'customer'.register R-201 · evidence tests/walk5-r201-2026-08-07/journal.md · capability-map A customer (not the operator) performs a restore via UI alone
Putting restored files back where they belong is still manualWarning card. Confirmed 2026-08-09: the restore lands in a verification folder and says so.register R-213
The tripwire that says someone is opening this customer's backups does fireCONFIRMED tonight from the hub store: escrow_blob_served 2026-08-09 10:19:41Z = 12:19 CEST in the operator's mailbox. R-281's original claim of silence is WITHDRAWN.register R-281 · register R-285 · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md
19 situations, and what actually happens in each.
A disk starts failing
A disk starts failing — healthy path only; a genuinely failing disk has never been seenneeds-hardwareOnly a failing disk on a running box could settle it.capability-map Lemez-egészség felügyelet: per-disk SMART kártya · evidence audits/SPIKE-smart-coverage-2026-07-25.md
The backup drive is unplugged
The backup drive is unpluggedcapability-map An ABSENT backup-target drive raises its OWN alarm · evidence audits/R116-v0116-2026-07-30.md
A drive is filling up
A drive is filling upchanged 2026-08-09, was walkedSame walk as backup.fill-warning: warned before it filled; the file removed, nothing else was needed.why it moved: no walk document citedevidence audits/security-2026-11-02/part5/fill-walk.log · register R-167
An app crashes
An app crashes — the box stops it and both the operator and the household get a mailWalked 2026-11-02 on Tester 1: kill -9 of a throwaway app, 15 s apart; at 6 restarts in 10 minutes the box stopped and held it, the operator's mail arrived in 1 s; with an address on the household's page the Hungarian mail arrived too. The household's Start lifted it. One flaw: the mail ends with raw JSON (R-1363).evidence audits/security-2026-11-02/part5/crash-walk.log · evidence audits/security-2026-11-02/part5/crash-walk-customer.log · evidence audits/security-2026-11-02/part5/mail-crash-operator.txt · evidence audits/security-2026-11-02/part5/mail-crash-household.txt · register R-1363 · capability-map App crashes → customer notified (one event per transition, …
Power cut mid-backup
Power cut mid-backupevidence audits/AUDIT-power-outage-recovery-2026-07-22.md
The guest is destroyed
The guest is destroyedregister R-201 · evidence audits/DRILL-r201-night-run-2026-08-04.md · evidence audits/DRILL-r201-night-run-2026-08-04.md
The whole machine is wiped and reinstalled
The whole machine is wiped and reinstalled — the data comes backchanged 2026-08-09, was walkedThe 2026-08-09 rehearsal really did return 4/4 byte-identical, and that stands. What the next real reinstall showed (2026-08-21, demo-hp) is that the rebuild ORPHANS BOTH OFF-PREMISES TIERS at once, quietly: the restic target was lost and needed a self-heal plus a per-app re-enable before any copy resumed (R-193), and the PBS archives from before the reinstall cannot be opened by the rebuilt box at all, because it now presents a different key (R-366). The data came back in the rehearsal because the rehearsal restored it immediately; a machine left alone after a reinstall is not protected in the meantime and nothing says so.why it moved: a real reinstall on 2026-08-21 left both off-premises tiers broken - one silently for 12 days, the other unreadable - so 'the data comes back' holds only if someone restores it at onceregister R-193 · register R-366 · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md · evidence audits/REPORT-DRILL-backup-truth-2026-08-21.md
The whole machine is wiped and reinstalled
The whole machine is wiped and reinstalled — the journey needs a terminal twiceCONTESTED-RESOLVED against the map: the map's PROVEN-LIVE row is scoped to a controller-data-volume REBUILD (2026-08-04), not a whole-host reinstall. The map has no row for the host case, so there was no contradiction — only a gap.register R-273 · register R-280 · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md
The machine is stolen, and someone opens the backups
The machine is stolen, and someone opens the backups — the operator is toldConfirmed 2026-08-09 from the store and the mailbox.register R-281 · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md
The customer forgets their dashboard password
The customer forgets their dashboard passwordExercised 2026-08-09 via the reset code.evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md
The customer loses their recovery code
The customer loses their recovery code — by design, the data is unrecoverablemoved twice: 2026-08-09 walked -> built, then 2026-08-10 back to walkedThe recovery screen states it in Hungarian.why it moved: receipt found 2026-08-10, and then proven the hard way: the recovery screen states in Hungarian that nobody can replace a lost code, and on 2026-08-10 demo-felhom's pre-fix key was confirmed unrecoverable by construction (identity_blob NULL; the restic password lives only in that bundle). The by-design refusal is real.capability-map Escrow ceremony: customer-facing wizard, one-shot R claim, … · register R-198 · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md
The machine moves house / new network
The machine moves house / new networkcapability-map Box survives a site/network change (relocation, different s… · evidence audits/AUDIT-vacation-remote-ops-2026-07-20.md
The internet goes down
The internet goes downWalked 2026-11-03 on Tester 1: packets to the outside dropped on host and guest for 13 minutes, the line's route left in place. The dashboard answered on the home network, 10 apps ran, the public name gave 530; 60 s after the line came back the public name worked by itself, nothing to do. Slow: the Backups page took over 2 minutes (R-1362). An earlier try removed the route: the agent's self-heal put it back in about 100 s.evidence audits/security-2026-11-02/part5/outage-wan.log · evidence audits/security-2026-11-02/part5/outage-wan-3min.txt · register R-1362 · capability-map LAN access when internet is down (lan_resolver)
The hub is down
The hub is downchanged 2026-08-09, was walkedWalked 2026-11-03 on Tester 1 (the hub's address dropped for 13 minutes; DooPlex untouched): the box works on — dashboard, apps, an install from the cached catalog, a crash-loop stop. What fails: every event sent in that time is lost after 9 s and nothing re-sends it, so the operator never heard of the held app (R-1361).why it moved: no walk document citedevidence audits/security-2026-11-02/part5/outage-hub.log · register R-1361 · capability-map Config/state change round-trips in seconds (hub↔box immediacy
We ship a broken release
We ship a broken release — the guard is missingProven the hard way on 2026-08-09: a vouched agent version had no git tag and every install died at 5/8. Both guards still owed.register R-273 · register R-287
A fresh install picks up an old image
A fresh install picks up an old imageNarrowed 2026-08-09: the resume path fetched the vouched golden; a fresh install still takes the newest LOCAL archive with no manifest comparison.register R-274
The off-site provider account is deleted
The off-site provider account is deletedThe credential that holds the customer's documents can still delete.register R-95
The machine is switched off for an afternoon
The machine is switched off for an afternoon — no notion of expected downtimeConfirmed 2026-08-09: eight operator mails for deliberate, attended work.register R-285
A customer restores their own data with no help
A customer restores their own data with no helpWalked 2026-10-29 night on scratch 9202 through the household's restore button (the page's own POST): vikunja from the 02:30 local point in 79 s — the project made before the point came back, the one made after it is gone, the app runs, no hold; a restore that meets a busy volume puts the live data back (R-1256 H5). The route needs no help. Still open: no person who is not the operator has done it (claim.never-by-non-operator). The old off-site blocker (R-356) shipped in controller v0.219.0.capability-map A customer (not the operator) performs a restore via UI alone · register R-201 · register R-356 · evidence audits/REPORT-DRILL-backup-truth-2026-08-21.md · evidence audits/night-2026-10-28/part5-claims/selfrestore-walk.log