night 2026-10-28: R-1256 [DESIGN] + H5/H8 live facts in 07; fault.selfheal walked on 9202 (evidence)

documentation/architecture/07-backup-architecture.md

@@ -821,6 +821,32 @@ household's newest data („gen2") and its download on the drive were there, and
 nothing. Run 1 of the same proof found the journal left as `undo.json.tmp` (the rename lived only in the page cache) —
 the app was held for support, never started on the cut data; the folder sync above fixed it, run 2 passed.
 
+**`[DESIGN]` 2026-10-28 night — four restore edges (R-1256, review round 2 HUNT-1 H4/H5/H6/H8; controller `342a394`,
+unreleased — the morning list).** (H4) An off-site restore whose definition write fails after the compose was written
+writes the LIVE definition back before it starts the app; a write-back that fails too HOLDS the app (`restore_mixed`) —
+never a start on the snapshot's definition over the live data. (H5) A put-back whose stop fails pours nothing, keeps the
+copy and holds the app (off-site, local unit, and a restore cut by a power loss); a failed definition write-back keeps the
+copy (D20); a `docker volume rm -f` that fails (the volume is in use) fails that volume — the `volume create` after it was
+a no-op and the tar was unpacked ON TOP of the old content. (H6) A restore op begins in ONE step with the check for every
+other operation (a restore, the backup run, a second-drive copy — under the lock the mirror asks under); every restore
+button and the kept loads begin that way, and the install-time „use my kept data" asks about the second-drive copy too.
+(H8) An unused restore copy goes after 14 days (every 6 h): only when the app has no restore hold of any kind, the
+app-stop marker is readable and does not name the app, and no restore runs. **Decided by CC unattended 2026-10-28 —
+operator may reverse:** *How long does an unused restore copy stay?* (a) 14 days; (b) for ever (as before); (c) 3 days.
+(a) holds up to two weeks of disk per restore and gives support two weeks; (b) grows without end; (c) is shorter than a
+support reply. A held or waiting copy is never touched under any option. Tests `TestR1256_*`, each red first.
+**`[FACT]` 2026-10-29 night, scratch 9202 (`audits/night-2026-10-28/r1256-restore/h5-live/`):** vikunja's first volume
+held by a created-only container, the household's restore pressed through the page: „Failed to remove volume
+vikunja_vikunja_data before the restore … (not written into)", the restore failed, the put-back poured the live copy of
+both volumes, the app ran, no hold, and a project made before the restore point AND one made after it both read back. H4
+(off-site only, and a fault there races the write-back) and H5's failed stop are proven by tests only; their live proof
+is owed to an attended session.
+
+**`[FACT]` 2026-10-29 03:29 UTC, scratch 9202 (`audits/night-2026-10-28/r1256-restore/h8-live/`):** three restore
+copies planted by hand (dated 2026-10-01, 2026-10-05 and 2026-10-28); the first six-hour sweep after the controller start
+removed the two older than 14 days, one of them under the standing app paperless, and kept the young one; the job took
+8 ms. The young copy was then removed by hand.
+
 **[DESIGN] 2026-08-22 — the failure ladder of a database restore: replay → rollback → hold.**
 Recorded here rather than only in a closed register row, because a decision that survives only inside
 a closed work item is a decision nobody will find.

documentation/architecture/where-felhom-stands.html

@@ -7,7 +7,7 @@
 <h1>Where Felhom stands</h1>
 <div style="color:#8fa0b8;font-size:12.5px;margin:7px 0 4px;max-width:1100px;line-height:1.5">What we built, what happens when things go wrong, and what is still missing. <b style="color:#cbd7e8">Generated from <code>where-felhom-stands.yaml</code></b>, which cites the capability map, register row or evidence document behind every claim — and which is checked by <code>scripts/check_stands.py</code>.</div>
 <div style="color:#6b7a91;font-size:11.5px;margin-bottom:16px">Verified 2026-08-22 against felhom-agent <code>40d857b527</code>, felhom-controller <code>2da259af38</code>, hub <code>877fcd2a38</code>. <b style="color:#fbbf24">17 claim(s) carry a recorded status move</b> (7 re-checked in this pass) — each is marked on the page.</div>
-<div style="display:flex;gap:22px;flex-wrap:wrap;background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:11px 14px;margin-bottom:18px"><div style="display:flex;gap:8px;align-items:flex-start"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:11.5px"><b>Walked</b> <span style="color:#7c8aa3">(23)</span><span style="display:block;color:#6b7a91;font-size:10.5px">done end to end on real hardware, evidence on file</span></span></div><div style="display:flex;gap:8px;align-items:flex-start"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:11.5px"><b>Built</b> <span style="color:#7c8aa3">(13)</span><span style="display:block;color:#6b7a91;font-size:10.5px">shipped and tested, the real path never walked</span></span></div><div style="display:flex;gap:8px;align-items:flex-start"><span style="width:8px;height:8px;border-radius:50%;background:#fbbf24;flex:0 0 8px;margin-top:5px"></span><span style="font-size:11.5px"><b>Partial</b> <span style="color:#7c8aa3">(15)</span><span style="display:block;color:#6b7a91;font-size:10.5px">some walked, some not — the note says which</span></span></div><div style="display:flex;gap:8px;align-items:flex-start"><span style="width:8px;height:8px;border-radius:50%;background:#64748b;flex:0 0 8px;margin-top:5px"></span><span style="font-size:11.5px"><b>Missing</b> <span style="color:#7c8aa3">(4)</span><span style="display:block;color:#6b7a91;font-size:10.5px">does not exist</span></span></div><div style="display:flex;gap:8px;align-items:flex-start"><span style="font-size:11.5px"><b style="color:#7dd3fc">needs-hardware</b><span style="display:block;color:#6b7a91;font-size:10.5px">only a running box could settle it</span></span></div></div>
+<div style="display:flex;gap:22px;flex-wrap:wrap;background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:11px 14px;margin-bottom:18px"><div style="display:flex;gap:8px;align-items:flex-start"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:11.5px"><b>Walked</b> <span style="color:#7c8aa3">(24)</span><span style="display:block;color:#6b7a91;font-size:10.5px">done end to end on real hardware, evidence on file</span></span></div><div style="display:flex;gap:8px;align-items:flex-start"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:11.5px"><b>Built</b> <span style="color:#7c8aa3">(12)</span><span style="display:block;color:#6b7a91;font-size:10.5px">shipped and tested, the real path never walked</span></span></div><div style="display:flex;gap:8px;align-items:flex-start"><span style="width:8px;height:8px;border-radius:50%;background:#fbbf24;flex:0 0 8px;margin-top:5px"></span><span style="font-size:11.5px"><b>Partial</b> <span style="color:#7c8aa3">(15)</span><span style="display:block;color:#6b7a91;font-size:10.5px">some walked, some not — the note says which</span></span></div><div style="display:flex;gap:8px;align-items:flex-start"><span style="width:8px;height:8px;border-radius:50%;background:#64748b;flex:0 0 8px;margin-top:5px"></span><span style="font-size:11.5px"><b>Missing</b> <span style="color:#7c8aa3">(4)</span><span style="display:block;color:#6b7a91;font-size:10.5px">does not exist</span></span></div><div style="display:flex;gap:8px;align-items:flex-start"><span style="font-size:11.5px"><b style="color:#7dd3fc">needs-hardware</b><span style="display:block;color:#6b7a91;font-size:10.5px">only a running box could settle it</span></span></div></div>
 <div style="font-size:15px;font-weight:600;margin:0 0 9px">The journey</div>
 <div style="color:#6b7a91;font-size:11.5px;margin-bottom:10px">Seven stages, left to right, as they happen to a person.</div>
 <div class="grid7">
@@ -16,7 +16,7 @@
 <div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44"><span style="color:#4b5b76;font-family:ui-monospace,monospace">3</span> &nbsp;Using it</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Apps installed from a catalog of about 52, with a memory guard and health-aware progress<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">53 templates listed on the rebuilt box 2026-08-09; deploy driven live.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Deploy an app from the catalog (env config, memory guard, h… · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Start, stop, restart, update, logs, remove — and the parts that must not be stopped cannot be<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#3a2a12;color:#fbbf24;font-size:10px;white-space:nowrap">changed 2026-08-09, was built</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Walked 2026-10-28 night on scratch 9202: install, stop, start, restart, logs; traefik and cloudflared refuse a stop (403); remove with data leaves no volume and no container. The update half: the same day's guarded updates on 9202 (bench + box + undo walks).</span><span style="display:block;color:#9a7b3a;font-size:11px;margin-top:2px">why it moved: a walk document is now cited (2026-10-28 night)</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map App lifecycle: start/stop/restart/update/logs/remove/redeploy · register R-108 · evidence audits/night-2026-10-28/part5-claims/lifecycle-walk.log · evidence audits/dockerhub-bigsteps-2026-10-28</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Reachable from anywhere through a tunnel, per-app addresses<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Verified tonight: the rebuilt box answered on its public URL from outside.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Remote access via Cloudflare Tunnel + Traefik (per-app subd… · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Reachable on the home network when the internet is down<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#1b2740;color:#7dd3fc;font-size:10px;white-space:nowrap">needs-hardware</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Only an observation on a running box with WAN pulled could settle it. Boxes are off.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map LAN access when internet is down (lan_resolver)</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Phone photos, documents with text recognition, files from Windows Explorer or a Mac<span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Files from Windows Explorer / Mac Finder (SMB server) · evidence audits/SPIKE-lan-discovery-2026-07-18.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">A one-tap launcher, and a read-only guest link for visitors<sLine truncated
 <div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44"><span style="color:#4b5b76;font-family:ui-monospace,monospace">4</span> &nbsp;Drives</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">A new drive is found, offered, formatted, mounted and enrolled — including on awkward older boot layouts<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#3a2a12;color:#fbbf24;font-size:10px;white-space:nowrap">moved twice: 2026-08-09 walked -&gt; built, then 2026-08-10 back to walked</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Applies to a NEW drive. Re-attaching an existing one after a reinstall is R-280 and fails.</span><span style="display:block;color:#9a7b3a;font-size:11px;margin-top:2px">why it moved: receipt found 2026-08-10: a live throwaway drive on felhom-pve walked scan -&gt; format -&gt; mount -&gt; PVE dir storage -&gt; one-click Regisztralas, with the resulting storage.cfg entry and systemd mount unit recorded. The map already read PROVEN-LIVE; the dataset was behind it. Scope unchanged: this is a NEW drive. Re-attaching an existing one after a reinstall is R-280 and still fails.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Drive wizard: scan/format/mount/enroll, incl. legacy-boot L… · register R-220 · evidence audits/SPIKE-raw-drive-enroll-2026-06-15.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Moving data between drives, crash-safe; removing a drive safely; unplug detected<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#3a2a12;color:#fbbf24;font-size:10px;white-space:nowrap">changed 2026-08-09, was walked</span><span style="display:block;color:#9a7b3a;font-size:11px;margin-top:2px">why it moved: no walk document cited</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Data migration between drives (all / per-app), crash-safe</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">A network drive can be browsed and hold bulk media, but may not hold an app's data — enforced<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">RefuseAsAppNamespace is the fail-closed predicate.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Network storage (NAS) is browse + bulk-media only · register R-108 · evidence audits/R108-network-app-namespace-2026-07-30.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#fbbf24;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">After a reinstall the data drive cannot be re-attached through any dashboard route<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Warning card. /api/disks/candidates returns empty; the restore page promises two clicks.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">register R-280 · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md</span></span></div></div>
 <div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44"><span style="color:#4b5b76;font-family:ui-monospace,monospace">5</span> &nbsp;Backing up</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">App data on the machine, nightly database dumps, a copy on a second drive<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#3a2a12;color:#fbbf24;font-size:10px;white-space:nowrap">moved twice: 2026-08-09 walked -&gt; built, then 2026-08-10 back to walked</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">True for every app but one, and that one was found on 2026-08-21: paperless-ngx's 72-table PostgreSQL was dumped nightly and written into a folder named after an app that does not exist, so it never entered the recovery unit, the second-drive copy or the off-site copy. Fixed and proven live 2026-08-22 (R-355) — the dump is now in the app's own unit and in the off-site snapshot for the first time. A catalogue-wide sweep, itself proven able to convict a planted case, says 1 of 53 was affected.</span><span style="display:block;color:#9a7b3a;font-size:11px;margin-top:2px">why it moved: receipt found 2026-08-10: an adversarial, destructive, unattended overnight campaign across both boxes and ep0, with A2 (one quiesce, two tiers) proven end to end. Map already read PROVEN-LIVE.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Tier-2 secondary-drive copy: class-driven legs · register R-355 · evidence audits/CAMPAIGN-8-backup-restore-2026-07-27.md · evidence audits/DRILL-backup-truth-2026-08-21/evidence/phase4-paperl…</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">A whole-machine archive that lands off the guest's own disk — a single-drive machine is recorded as degraded rather than pretending<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#3a2a12;color:#fbbf24;font-size:10px;white-space:nowrap">changed 2026-08-09, was walked</span><span style="display:block;color:#9a7b3a;font-size:11px;margin-top:2px">why it moved: no walk document cited</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Whole-guest backup lands OFF the guest's own physical device · register R-165</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#fbbf24;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">An encrypted off-site copy, sealed with a key the operator cannot read<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#3a2a12;color:#fbbf24;font-size:10px;white-space:nowrap">changed 2026-08-09, was walked</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">The COPY is real and the key is still unreadable to us — what failed is DAILY and UNBROKEN. The 2026-08-09 note said '18 snapshots, daily, unbroken'; the next snapshot after 2026-08-09 08:30 was 2026-08-21 22:17, put there by hand during the drill. Twelve days, no alarm. Two causes in series: the 2026-08-21 rebuild lost the off-box target (R-193's shape), and after the self-heal restored it EVERY per-app switch was still off, so the first run logged 'backup OK: 0 app(s) backed up, 14s'.</span><span style="display:block;color:#9a7b3a;font-size:11px;margin-top:2px">why it moved: the claim is about a CONTINUING daily copy; a 12-day silent gap was found on 2026-08-21 and nothing reported it</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Offsite (restic → Hetzner Storage Box) · register R-199 · register R-193 · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md · evidence audits/REPORT-DRILL-backup-truth-2026-08-21.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">The backups prove themselves: a restore is actually performed, uLine truncated
-<div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44"><span style="color:#4b5b76;font-family:ui-monospace,monospace">6</span> &nbsp;When something goes wrong</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">The machine watches itself and repairs some faults without telling anyone it had to<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#3a2a12;color:#fbbf24;font-size:10px;white-space:nowrap">changed 2026-08-09, was walked</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">R-264 records that the self-heal counters reach the hub and are decoded nowhere.</span><span style="display:block;color:#9a7b3a;font-size:11px;margin-top:2px">why it moved: no walk document cited</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Box survives an unattended app or guest-network failure · register R-264</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Failures reach the operator by email, one mail per run, every failing app named<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#3a2a12;color:#fbbf24;font-size:10px;white-space:nowrap">changed 2026-08-09, was walked</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">CONFIRMED against source: backup_run_failures digest is allowlisted (api/handler.go:1837), operator-only (dispatcher.go:423), templated (templates.go:48); recovery_unit_capture_failed is record-only (dispatcher.go:376). NOTE: R-182's register row still describes the PRE-FIX behaviour — see R-289.</span><span style="display:block;color:#9a7b3a;font-size:11px;margin-top:2px">why it moved: the digest is wired and source-verified, but no run of it has been observed delivering</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map A failed per-app Tier-1 backup reaches the OPERATOR — EVERY… · register R-182</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Failures reach the customer<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">The customer leg is built; today's log shows customer-channel rows skipped as operator_only.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map App crashes → customer notified (one event per transition, …</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">An already-paired box no longer tells anyone to pair it — its console says there is nothing to do<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#3a2a12;color:#fbbf24;font-size:10px;white-space:nowrap">changed 2026-08-09, was partial</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Was a warning card (the console kept showing the pairing code). Fixed by ISO 1.28.0 (R-535; R-214, R-235 closed). Walked 2026-10-29 00:04 on Tester 1 (paired, claimed): the console reads, in Hungarian and English, that there is nothing to do on this machine and no need to log in — no pairing code.</span><span style="display:block;color:#9a7b3a;font-size:11px;margin-top:2px">why it moved: the defect is closed and the console was read live on a paired box (2026-10-29)</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">register R-214 · register R-235 · evidence audits/night-2026-10-28/part5-claims/tester1-console-2026-1…</span></span></div></div>
+<div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44"><span style="color:#4b5b76;font-family:ui-monospace,monospace">6</span> &nbsp;When something goes wrong</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">The machine watches itself and repairs some faults without telling anyone it had to<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#3a2a12;color:#fbbf24;font-size:10px;white-space:nowrap">changed 2026-08-09, was built</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Walked 2026-10-29 night on scratch 9202: a crashed app part (kill -9 of paperless-redis) ran again in 1 s and the health probe read the app healthy; a guest reboot brought every app back in under 2 minutes, the boot sweep found nothing to start; no event, no banner, no mail. Docker's restart policy did the repair in both legs; the controller's own boot repair had nothing to do (its last proof: 6 of 6 hard resets, 2026-08-02). R-264 records that the self-heal counters reach the hub and are decoded nowhere.</span><span style="display:block;color:#9a7b3a;font-size:11px;margin-top:2px">why it moved: walked 2026-10-29 on scratch 9202 (a walk document is now cited)</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Box survives an unattended app or guest-network failure · register R-264 · evidence audits/night-2026-10-28/part5-claims/selfheal/WALK.md · evidence audits/DIAG-bootrecon-drive-absent-2026-08-02.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Failures reach the operator by email, one mail per run, every failing app named<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#3a2a12;color:#fbbf24;font-size:10px;white-space:nowrap">changed 2026-08-09, was walked</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">CONFIRMED against source: backup_run_failures digest is allowlisted (api/handler.go:1837), operator-only (dispatcher.go:423), templated (templates.go:48); recovery_unit_capture_failed is record-only (dispatcher.go:376). NOTE: R-182's register row still describes the PRE-FIX behaviour — see R-289.</span><span style="display:block;color:#9a7b3a;font-size:11px;margin-top:2px">why it moved: the digest is wired and source-verified, but no run of it has been observed delivering</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map A failed per-app Tier-1 backup reaches the OPERATOR — EVERY… · register R-182</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Failures reach the customer<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">The customer leg is built; today's log shows customer-channel rows skipped as operator_only.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map App crashes → customer notified (one event per transition, …</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">An already-paired box no longer tells anyone to pair it — its console says there is nothing to do<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#3a2a12;color:#fbbf24;font-size:10px;white-space:nowrap">changed 2026-08-09, was partial</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Was a warning card (the console kept showing the pairing code). Fixed by ISO 1.28.0 (R-535; R-214, R-235 closed). Walked 2026-10-29 00:04 on Tester 1 (paired, claimed): the console reads, in Hungarian and English, that there is nothing to do on this machine and no need to log in — no pairing code.</span><span style="display:block;color:#9a7b3a;font-size:11px;margin-top:2px">why it moved: the defect is closLine truncated
 <div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44"><span style="color:#4b5b76;font-family:ui-monospace,monospace">7</span> &nbsp;Getting everything back</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">A rebuilt machine shows a full-page recovery screen without anyone looking for it, and says plainly that nobody can replace a lost recovery code<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Seen unsought on the rebuilt demo-hp 2026-08-09, seal date matching host_escrow.created_at.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">register R-193 · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">The customer's code opens the sealed package and the data returns byte for byte — including accented Hungarian filenames, verified as raw bytes<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Still true, and re-proven 2026-08-22 (5/5 byte-identical, both accented names as raw bytes) — but the SCOPE is narrower than the sentence sounds and was silently narrower still until v0.218.0. The drill found the off-site restore had NO named-volume leg at all: the archive sat in the unit, the snapshot and the checking folder and was never replayed, under a success message (R-354, fixed and proven 2026-08-22). AND 40 of the 53 catalogue apps STILL cannot run this route at all — it refuses first, saying a running app is not installed (R-356, open). So: proven for an app that declares a data drive; unproven and currently unreachable for the class whose entire dataset is a named volume.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">register R-201 · register R-354 · register R-356 · evidence tests/walk5-r201-2026-08-07/journal.md · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md · evidence audits/REPORT-DRILL-backup-truth-2026-08-21.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Walked end to end with no command line inside the machine (2026-08-07)<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#1b2740;color:#a78bfa;font-size:10px;white-space:nowrap">contested</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">CONTESTED-RESOLVED: not a contradiction. The walk proves the ROUTE needs no guest shell; the map's MISSING row is about a NON-OPERATOR doing it, which has still never happened. Two questions, one word 'customer'.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">register R-201 · evidence tests/walk5-r201-2026-08-07/journal.md · capability-map A customer (not the operator) performs a restore via UI alone</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#fbbf24;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Putting restored files back where they belong is still manual<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Warning card. Confirmed 2026-08-09: the restore lands in a verification folder and says so.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">register R-213</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">The tripwire that says someone is opening this customer's backups does fire<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">CONFIRMED tonight from the hub store: escrow_blob_served 2026-08-09 10:19:41Z = 12:19 CEST in the operator's mailbox. R-281's original claim of silence is WITHDRAWN.</span><span style="display:block;color:#4b5bLine truncated
 </div>
 <div style="font-size:15px;font-weight:600;margin:22px 0 9px">When things go wrong</div>

documentation/architecture/where-felhom-stands.yaml

@@ -418,17 +418,19 @@ claims:
     band: journey
     stage: 6
     title: "The machine watches itself and repairs some faults without telling anyone it had to"
-    status: built
-    note: "R-264 records that the self-heal counters reach the hub and are decoded nowhere."
+    status: walked
+    note: "Walked 2026-10-29 night on scratch 9202: a crashed app part (kill -9 of paperless-redis) ran again in 1 s and the health probe read the app healthy; a guest reboot brought every app back in under 2 minutes, the boot sweep found nothing to start; no event, no banner, no mail. Docker's restart policy did the repair in both legs; the controller's own boot repair had nothing to do (its last proof: 6 of 6 hard resets, 2026-08-02). R-264 records that the self-heal counters reach the hub and are decoded nowhere."
     sources:
       - capability-map: "Box survives an unattended app or guest-network failure"
       - register: "R-264"
+      - evidence: "audits/night-2026-10-28/part5-claims/selfheal/WALK.md"
+      - evidence: "audits/DIAG-bootrecon-drive-absent-2026-08-02.md"
     changed:
-      from: walked
-      reason: "no walk document cited"
+      from: built
+      reason: "walked 2026-10-29 on scratch 9202 (a walk document is now cited)"
     verified:
-      date: 2026-08-09
-      verdict: downgraded
+      date: 2026-10-29
+      verdict: upgraded
       depth: source-read
   - id: fault.operator-email
     band: journey