Commit Graph

271 Commits

Author SHA1 Message Date
admin ee48a48288 docs: migrate workflow to DooPlex-local execution 2026-07-19 12:16:52 +02:00
admin 13af252907 docs: scripts v1.22.0 + website grid CHANGELOGs, ROADMAP R-38 shipped, R-45/R-46, REPORT
R-38 flips to SHIPPED with the live-canary evidence. Two new items registered
from tonight's work: R-45 (unified async-job feedback — five one-off progress
surfaces now exist and agree on nothing; the two lessons any framework must
encode are "probe the terminal state, never infer it" and "a source reporting
nothing is normal, not broken") and R-46 (verification copies are now visible
but still cannot be browsed and never expire — the invisible-result gap and the
disk-lifecycle gap, designed together because they are the same object).

Pre-invite checklist gains the golden note: build-golden v2.1.0 makes goldens
>= 0.147.x carry all four infra images; no rebuild for this alone.

REPORT-<topic>.md per the parallel-session rule — another session is writing in
this clone tonight, so the shared REPORT.md is left alone.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nn3VgQk9iwEGgyx6QJ2NvE
2026-07-19 10:00:10 +02:00
admin f3d1ba3062 docs: DIAGNOSE immich offsite restore — findings + ROADMAP R-43/R-44
Viktor deleted 11 immich photos to test offsite restore; both place runs
merged 0 files and flashed success. Two-layered root cause:

- A UI delete means TRASH, not deletion — no file left the disk, so there
  was genuinely nothing missing to restore. The test proved nothing.
- A valid test would also have failed: no offsite path loads a database.
  All three buttons are file-only (offbox_restore.go:171-232, :346-421).
  Files return; the library stays empty. The only DB replay is
  RestoreFromRecoveryUnit, reachable solely from the LOCAL restore panel.

Compounding: the manual push never refreshes dumps (offbox_handlers.go:
203-227) — today's unit dump is from 02:30 and probes to asset:0 / user:0 /
album:0, predating both the admin user and the photos. Had a full restore
loaded it, it would have written an empty DB over the live one, destroying
the trashed rows that were the only surviving recovery path.

Answer to "can a customer trust same-day offsite?" for a DB-indexed app:
no — files come back, content does not.

No product code changed. Photos left in trash (recovery not wanted).
Capability map: customer-restore row stays MISSING by finding, not by
absence of attempt; offsite-restore row flagged scope-contested pending
a ruling on whether CAMPAIGN-6D exercised the DB half.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P9Nn14TWGzKoqAJAiVwC2s
2026-07-19 08:47:34 +02:00
admin 3e2078be8a CAMPAIGN 7 doc: MAJOR breaking-note table (§6.3)
Fills the gap flagged in §8.1. Upstream notes for bookstack, immich, nextcloud,
n8n, grafana, vikunja, privatebin, radarr, romm; plus the ones this sweep found
EMPIRICALLY on a real deploy (outline PGSSLMODE, tandoor port+ALLOWED_HOSTS,
zipline DATABASE_URL/healthcheck rename, homebox tag prefix + pepper, wger
DJANGO_DB_* + :8000) -- which are the more useful half, being what actually
happens rather than what a changelog remembered to mention.

Four majors (calcom, claper, gramps-web, onlyoffice) deployed clean but have no
retrieved upstream one-liner; recorded as not-retrieved, not fabricated.

Campaign 7 catalog sweep.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nn3VgQk9iwEGgyx6QJ2NvE
2026-07-19 03:30:50 +02:00
admin 13d8bffa96 CAMPAIGN 7 doc: offsite continuity across reinstall PROVEN (R-34 evidence)
An offsite run after the trio's fresh redeploy shows the snapshot TAGS are stable
across uninstall -> volume-destroy -> reinstall: calibre-web 243ad08c -> 9698ba74
and immich 6df12205 -> ea511336 both keep their exact tag sets, so a
'restic --tag <app>' query still spans the reinstall boundary and pre-uninstall
restore points stay addressable.

Side effect: the standing 'kimaradtak: bookstack' offsite warning is now CLEARED.
bookstack had no offsite snapshot because it had no tier-1 backup to push; the
fresh redeploy produced one, so its first-ever offsite snapshot landed (run ok,
1m13s, 6 snapshots, 211.6 MB, last_warning None).

Campaign 7 catalog sweep.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nn3VgQk9iwEGgyx6QJ2NvE
2026-07-19 03:29:00 +02:00
admin eee2c85321 docs: CAMPAIGN 7 catalog sweep — audit, ROADMAP R-40/41/42, REPORT convention
Adds the CAMPAIGN 7 audit doc (53/53 apps deployed and validated on the demo box
through the real controller endpoints; 45 pass, 13 template fixes, ranked
findings) and the uninstall-semantics map row it proves out: remove requires stop
first, named docker volumes are ALWAYS destroyed including the DB, HDD data and
backups survive unless explicitly ticked, and the per-app offsite toggle outlives
the app while tier-2 config is cleared.

ROADMAP +3:
  R-40 the update path cannot express a multi-hop major upgrade (nextcloud
       forbids skipping majors; the template moved 31 -> 34)
  R-41 no standing 'does every template still deploy?' check -- the sweep was
       the first thing to try, and found 5 apps that had NEVER been deployable
  R-42 ruling needed on DB/cache sidecar majors (deliberately not bumped)

CLAUDE.md: REPORT.md is overwritten, so a second concurrent session in this repo
writes REPORT-<topic>.md instead -- applied to this run's own report.

Campaign 7 catalog sweep.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nn3VgQk9iwEGgyx6QJ2NvE
2026-07-19 03:24:39 +02:00
admin 90ea253dc2 docs(roadmap): R-36 + R-37 collapse to SHIPPED (hub v0.67.0)
R-36: both halves delivered — the enabled-but-unprovisioned warning on the
customer page (reusing the same predicate the offsite re-issue handler refuses
on), and the related sub-item, auto-minting the self-bind link at customer
creation AND RESET completion so the console banner's promised email is already
true. Records the gap found while wiring it: PurgeCustomerResetDBState does not
clear selfbind_tokens, so a pre-RESET link would have survived the reset; the
skip paths now clear stale tokens.

R-37: the post-RESET staleness banner, narrow by design — an in-flight reset
does not trigger it, it clears itself on the first post-RESET report, and ties
resolve to STALE because SQLite timestamps are second-resolution and a
same-second report almost certainly predates the reset.

Both red-proofed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nn3VgQk9iwEGgyx6QJ2NvE
2026-07-18 21:49:27 +02:00
admin 1f4702fe50 docs: golden 0.146.0 baked + published (Phase 5); STOP for the operator saves
Golden 0.146.0 baked on the drill VM and published to gitea:
  felhom-golden/0.146.0/golden.tar.zst
  sha256 4834c703162c5437467a329144b1a523019bf5693ab9d439558be7323587e955
  612696588 B (584 MB archive), controller 0.146.0 confirmed baked in

All pass markers green: Result=success/ExecMainStatus=0, 0 FATAL/exclusions,
docker OK (overlay2), ALL THREE mounts included (rootfs + mp0 /var/lib/docker +
mp1 /mnt/sys_drive), pre-delete HTTP 404 (the pre-gate — version did not exist),
upload HTTP 201.

Integrity verified independently of the build host: anonymous GET | sha256sum
matches byte-for-byte, ranged GET 206, content-length matches the bake's bytes.
The version now appears in the hub dropdown (0.136.0, 0.143.0, 0.146.0).

Teardown per GL-1: log copied out as evidence first
(180:/mnt/5_hdd/felhom.eu/drill/bake-0.146.0.log), guest 9100 purged, token +
script + log shredded in-VM, VM off, qemu confirmed gone via `ps -eo comm` (not
the self-matching pgrep -f), drill disk reverted to the virgin snapshot exactly
as found. Token-leak grep = 0 against the LITERAL token value, on the bake log
and both ISO build logs from this session.

REMAINING is operator-only and password-gated: Day-0 manifest Golden -> 0.146.0
(Agent stays 0.90.0, MinAgent stays 0.90.0 — v0.146.0 declares no new agent
coupling), then the floor -> v0.146.0 saved LAST.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nn3VgQk9iwEGgyx6QJ2NvE
2026-07-18 21:20:50 +02:00
admin b14b5a65d9 docs: pre-travel train — R-39 diagnosis + heal, R-33 shipped, golden deferred
ROADMAP:
 - R-39 gains the full live diagnosis and REFUTES the brief's hypothesis. The
   generation IS bumped (SetHostDesired bumps unconditionally, 2->3) and
   applyPBSDR is exonerated, so no hub fix was shipped. The real mechanism is a
   signal mismatch: the hub's re-consume signal is a generation bump + poke,
   while the agent re-applies on a change of the DESCRIPTOR CONTENT HASH
   (manager.go ~L235). An ep0 re-issue re-keys the secret of an EXISTING token,
   so token_id/fingerprint are unchanged, the descriptor is byte-identical, the
   hash never moves, and the fresh secret is never consumed -> 401 forever.
   Proof: consumed-failed.json carries the same hash a4e5424... as the marker
   written two minutes before the re-issue.
   Records the second defect found while healing (wrapper reconcile passing
   --server, fixed in agent v0.90.1), marks the box HEALED with evidence
   (pvesm active, token 200, a real 9.7 GB encrypted backup listed PBS-side),
   and leaves the fleet fix explicitly pending its own spec.
 - R-33 collapses to SHIPPED (scripts v1.21.0), incl. why
   TimeoutStartSec=infinity is the load-bearing half.
 - Pre-invite checklist: golden target moves 0.145.x -> 0.146.0 and notes it is
   now MORE stale, since v0.146.0 is live on the demo box while the golden still
   bakes 0.143.0.

REPORT overwritten with the train: R-39 diagnosis verbatim + heal evidence, the
two ISO shas with the byte-identical-payload verification, the nav polish and
why the screenshot leg could not be done (the demo controller password is
customer-owned since the claim flow, so the build-server credentials are stale),
Phase 4 skipped cleanly, and Phase 5 deferred rather than half-run.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nn3VgQk9iwEGgyx6QJ2NvE
2026-07-18 21:02:34 +02:00
admin aa9c08f0dd docs(rehearsal): N100 S8 wrap — VALIDATION, map flips, ROADMAP R-30..R-39, R-3 draft
The 2026-07-18 N100 rehearsal ran the complete final-product flow on real metal
in one pass: RESET -> generic pairing ISO v1.20.0 -> customer self-bind -> day-0
-> managed-floor lift -> escrow ceremony -> offsite snapshots. No code changed;
every finding is recorded and ranked, none fixed.

VALIDATION-n100-rehearsal-2026-07-18.md — run context, a UTC-normalised timeline
built from the hub events stream / hub DB / controller log / bootstrap + agent
journals, per-ledger verdicts for S1-S8 + ledgers 8 and 9, 12 findings, the
not-exercised list, and 7 discrepancies against the brief.

Headline wall-clocks: bind -> credential 26 s; bind -> controller running the
current version 2 min 44 s; managed floor 0.143.0 -> 0.145.0 in 5 s unattended
(initiated_by: auto-floor); escrow ceremony -> offsite enabled 12 s; drive enrol
30.3 s. No post-bind leg stalled, which is the immediacy row's real-onboarding
proof.

Capability map (10 citations added):
- Bare-metal Felhom ISO           PARTIAL     -> PROVEN-LIVE (F1 closed on metal)
- Customer self-bind (slice 1)    IMPLEMENTED -> PROVEN-LIVE (customer_selfbind)
- Guest RAM resize (R-24)         IMPLEMENTED -> PROVEN-LIVE (shrink AND grow)
- Customer RESET                  two real firings + verified external teardown
- Escrow ceremony                 first live wizard firing
- Immediacy row                   "real-onboarding proof pending" cleared
- Publish train                   box-side floor lift proven on a fresh install
- Customer claim                  R-4 gmail half (Inbox under p=quarantine)
- Offsite orphan guard            staged live leg fired on its own
- DR tier by default              candidate PROVEN-LIVE upgrade WITHDRAWN (R-39)
Not flipped, as instructed: customer-performs-restore, BYO, DLNA, multi-user.

ROADMAP — collapsed R-1 (appliance half done, Peti half survives), R-21
(physically closed), R-24, R-27 slice 1, R-4. New ranked items:
  P2-HIGH  R-39 PBS DR applied-but-dead   R-30 liveness from the wait channel
           R-31 async offsite + status    R-32 RESET base-dir purge
           R-33 bootstrap quiet-poll
  P2       R-34 backup lifecycle          R-35 config-apply session survival
           R-36 post-RESET offsite prompt R-27c console-passphrase bind
  P3       R-37 post-RESET health card    R-38 installer GRUB slice
Plus a pre-invite checklist (golden 0.145.x rebuild, freemail.hu, C6, R-11).

R-39 is NEW and was not on the brief: the PBS DR descriptor auto-provisions and
the agent converges state=applied, but pvesm reports 401 Unauthorized/inactive
and a direct probe 401s on every endpoint including /version while WG is healthy.
The hub minted a second token secret two minutes after the agent applied the
first and consumed_at is still NULL; the converged state machine will not
re-apply, and the agent's verify loop cannot read the credential to notice it
(non-root read of a file it writes through a root wrapper). Rank is provisional
pending Viktor.

R-3 draft: all four [REFINE] slots filled, self-bind made the default path with
"send the link BEFORE the customer sees the console", the measured wall-clock
table added, and interim operator workarounds for R-31/R-36/R-39. C6 (renumbered
C7) is marked as the single unexecuted step and keeps the doc a DRAFT.

Evidence bundle: 180:~/n100-rehearsal/ (10 files + MANIFEST.md), collected before
the box was unplugged for travel. Secrets read only to run probes; recorded as
lengths and metadata, never values.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nn3VgQk9iwEGgyx6QJ2NvE
2026-07-18 19:56:17 +02:00
admin bbaa844f8c onboarding runbook 2026-07-18 19:32:00 +02:00
admin 2eccda4dd7 docs(roadmap): R-29 — design-v2 gates unenforced; docker_run_volume_path_gate red for 16 releases
Origin: R-7b close-out (felhom-controller REPORT section 4f). Two parts:
(a) the finding is benign — estimate.go:179 mounts a NAMED VOLUME (daemon-side, no
    host path), the same shape as three already-allowlisted entries, so the fix is a
    3-line ALLOWLIST addition with its WHY, NOT a docker-cp rewrite;
(b) the systemic half: the gates run only when a human remembers, so this one sat
    red from v0.129.0 (2026-07-14) to v0.145.0 while REPORTs said green. Second
    instance of the class after the v0.123.0 'Windows green gate silently red' note.
Lists the full gate inventory to audit for the same rot.
2026-07-18 15:17:11 +02:00
admin a03e142101 docs(R-7b): correction — all four legs PROVEN-LIVE, with snapshot evidence
Drops the false 'no offsite target on the demo box' clause from the ROADMAP row,
the capability-map SMB row and sharing.md. Cites offsite snapshots e0b9d723 /
4e2b15ec and the restore round-trip results. Root cause (guessed settings key) is
recorded in felhom-controller REPORT section 7b.
2026-07-18 14:32:39 +02:00
admin 4e3f6c48d4 docs(R-7b): share backup EXECUTION shipped (controller v0.145.0, Model B')
- capability map: SMB row KNOWN GAP cleared -> share data rides both tiers; the
  offsite leg + restore round-trip flagged as not-yet-live-exercised
- ROADMAP R-7b: idea -> SHIPPED, with the Model B' rationale and the live evidence
- controller/sharing.md: the KNOWN GAP block replaced by the execution contract;
  operator note corrected — samba IS liveness-monitored since v0.145.0
2026-07-18 13:35:00 +02:00
admin b76dad0c4e docs(R-7): Explorer leg PASSED — SMB server row MISSING -> PROVEN-LIVE
Viktor's human leg closed the last gate: both shares open from the Windows
Network view, an interactive Explorer save landed as uid 1000, and a write into
the read-only share was refused with the folder untouched. Capability map row
flipped to PROVEN-LIVE with that evidence; ROADMAP R-7 + sharing.md updated.
R-7b (shares classified but not in any live backup run) remains open.
2026-07-18 12:12:52 +02:00
admin d3a564cbff docs(R-7): SMB sharing slice 1 shipped — feature doc, map row, ROADMAP
controller/sharing.md (code-verified vs controller v0.144.0 + felhom-samba
1.0.0); capability map 'Files from Windows Explorer / Mac Finder (SMB server)'
MISSING -> IMPLEMENTED (PROVEN-LIVE pending Viktor's Explorer leg); ROADMAP R-7
-> shipped-slice-1 with the slice-2 remainder, and the backup design fork split
out as R-7b (shares are classified but not in any live backup run yet).
2026-07-18 12:05:15 +02:00
admin 7e0370f700 spike(R-6): human Explorer leg PASSED (nmbd) + stack torn down, report-clean
Viktor re-tried the double-click after the nmbd fix: Network -> FELHOM-SPIKE ->
spike-share opened, README.txt displayed. Full visible-and-openable one-click UX
works. Spike containers + /tmp scripts removed on guest and DooPlex; ports free;
verified report-clean. Spike COMPLETE.
2026-07-18 10:50:14 +02:00
admin d3eb8099a3 spike(R-6): S4b name-resolution finding — wsdd visible but flat name unresolvable
Human Explorer leg exposed the split: FELHOM-SPIKE renders (WSD PASS) but the
double-click fails 0x80070035 — flat name resolves by no path (DNS/LLMNR/NetBIOS
all silent; disable netbios=yes). By-IP mount works => SMB is healthy, the gap is
name resolution. Fix verified live: adding nmbd (NetBIOS) => nbtstat lists
FELHOM-SPIKE, ping resolves, \FELHOM-SPIKE\spike-share mounts by name. R-7 must
ship smbd+wsdd+nmbd (+avahi/.local), not wsdd alone.
2026-07-18 10:46:53 +02:00
admin 900d3041fa spike(R-6): LAN service discovery from the guest — SSDP/WSD/mDNS/SMB matrix
Verdict: appliance guest is LAN-bridged; multicast discovery works only in the
guest netns (guest-direct or docker --network host) — default bridge is deaf to
LAN multicast. Real samba+wsdd on host-net: Windows 11 ProbeMatch + 445 + SMB
round-trip PASS; SSDP MediaServer:1 reaches LAN clients. R-7 => host-network
LAN-bound; R-8 Jellyfin-DLNA plausible if host-network. ROADMAP R-6 -> spiked,
R-7/R-8 unblocked. S4.4 Explorer render pending human.
2026-07-18 10:38:46 +02:00
admin 22d04967b4 docs: relocate felhom build root to /mnt/5_hdd/felhom.eu (moved off DooPlex SSD 2026-07-18) 2026-07-18 09:51:29 +02:00
admin ee4e061d2b docs: publish train 0.90/0.143 COMPLETE — Viktor manifest/floor + DMARC quarantine, CC-verified 2026-07-18 09:21:12 +02:00
admin b2a1db1363 docs: publish train 0.90/0.143 + golden + N100 v1.20.0 ISOs run (2026-07-18); R-4 DMARC staged 2026-07-18 09:05:45 +02:00
admin 592818492c hub v0.66.0 + ISO v1.20.0: customer self-bind (R-27 slice 1)
Let a customer bind their own freshly-installed appliance without the
operator: operator "Send self-bind link" mints a 7-day tokenized
capability link, emailed (Hungarian, sibling sender) to the customer, who
opens a public /bind/<token> page and proves two factors — the console
pairing code shown on the box screen + their retrieval passphrase — and
the hub stages the bind via the same BindAppliance (provenance
customer_selfbind). The box's ~30s appliance poll delivers.

Viktor's three rulings verbatim: console pairing code (no appliance list
ever rendered), operator-sent tokenized link, 5-attempt lockout ->
"call support". Wrong code == wrong passphrase (one generic failure, no
oracle, both factors compared unconditionally); expiry falls back to
operator-bind unchanged.

THE TRAP: one public prefix /bind/, exempt from auth+CSRF at both /login
gate sites via a single isPublicBindPath predicate (tight trailing-slash
match; ServeMux ..-cleans; handler rejects '/' in token). 9 tests
(Scenarios A-F + F1/F2); 4 red-proofs verified red-then-green (lockout,
oracle, widened-prefix, single-active). GC verdict: no appliance GC ->
the 7-day TTL stands alone. Controller/agent untouched; R-27b deferred.

Green: full hub build/vet/test (17 ok) + bash -n + hub confirm gate.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017qDiBqKKQ5vPB5fXBqu7Kp
2026-07-17 23:56:53 +02:00
admin c6d7a69e6d runbook updated 2026-07-17 23:01:26 +02:00
admin a344dfdd54 runbook 2026-07-17 22:42:52 +02:00
admin a9cd308398 docs: hub v0.65.0 PBS DR visibility (R-5) — REPORT + CONTEXT + capability-map (PBS DR row, data pipeline PROVEN-LIVE) + ROADMAP (R-5 restic+PBS) 2026-07-17 21:20:34 +02:00
admin 7f11cfb36c hub v0.65.0 — PBS DR storage visibility (ep0 usage op) + Offsite tab split + dual dashboard gauges (R-5)
Makes PBS DR storage visible like the restic pool box (v0.64.0), differentiated. Scoping
correction: restic = subaccounts on the shared Hetzner Storage Box (Hetzner API); PBS DR =
the felhom-offsite PBS datastore on the ep0 endpoint VM (NO Hetzner API). Option A
(Viktor-ruled): a read-only `usage` op on the felhom-tenantsync ep0 forced command (twin of
fingerprint), polled by a new hub checker on the 15-min throttle. READ-ONLY throughout.

Phase-0 (gate PASSED): on ep0 (PBS 4.2.3), df -B1 --output=size,used,avail <datastore path>
yields bytes (39990112256/7627939840/... ~19%), read-only, existing sudo context, no admin token.

- scripts/felhom-tenantsync.sh -> v1.2.0: read-only `usage` short-circuit (df on the datastore
  path), no customer_id, no admin token, NO mutation. + a bash harness proving zero mutation.
- tenantsync.Client.Usage() + BoxUsage; unknown-op -> typed ErrUsageUnsupported (graceful).
- monitor.PBSDRBoxChecker: OffsiteBoxChecker clone over a usageReader seam; 15-min throttle,
  cached PBSBoxSnapshot, escalation-only pbsdr_box_fill on the "pbsdr-box" scope (operator only,
  no SaveEvent), recovery re-arm. Fill only. THREE states: ok / unavailable (ep0 <=v1.1.0,
  neutral no-alert) / degraded (exec failed, keep last).
- config: Alerting.PBSDRBoxFill{Warn,Crit}Percent (80/90); built with the tenantsync client,
  60s sweep, SetPBSDRBox. Hub deploy INDEPENDENT of the ep0 update (graceful degradation).
- web: /offsite splits into Restic + PBS DR hash tabs (endpoint cards under PBS DR); PBS panel;
  the single dashboard tile becomes two gauges (RESTIC pct.ratio, PBS DR pct / n/a).
- runbook offsite-endpoint.md 10: v1.2.0 update steps (no sudoers/authorized_keys change).

Tests: 10 Go + the harness; 3 red-proofs (usage mutation, escalation-only, unavailable-drives-band)
confirmed red then restored. go build/vet/test + bash -n + hub confirm gate all pass.
2026-07-17 21:13:30 +02:00
admin f2e51e676b docs: hub v0.64.0 offsite pool-box aggregate (R-5) — REPORT + CONTEXT + capability-map (IMPLEMENTED, data pipeline PROVEN-LIVE) + ROADMAP (R-5 SHIPPED) 2026-07-17 20:25:55 +02:00
admin 85a14192e7 docs: R-24 guest RAM resize + R-28 fast-tick SHIPPED (agent 0.90.0 + controller 0.143.0) — capability-map rows + ROADMAP collapse 2026-07-17 19:29:32 +02:00
admin 50c090eeaa docs: hub v0.63.0 system-initiated immediacy — REPORT + CONTEXT + capability-map (immediacy row) + ROADMAP (R-28 agent fast-tick SECONDARY) 2026-07-17 17:33:34 +02:00
admin 4c9b0e8706 docs: R-21 slice C — REPORT/CONTEXT + capability-map (universal-ISO chain, slice C live-verified) + ROADMAP (R-21 SHIPPED A+B+C, R-27 self-bind future) 2026-07-17 15:50:33 +02:00
admin 3172df1927 scripts v1.18.0 — R-21 slice B: firmware loader option --loader shim|mkimage (F1)
Closes N100 F1 (HIGH): cheap AMI (AN3PLUS 0.01-class) UEFI firmware can't
relocate the ISO's stock signed GRUB from USB (relocation 0x0). The run's live
grub-mkimage workaround is now a first-class pipeline mode.

- build-felhom-iso.sh: --loader shim|mkimage (default shim, byte-for-byte
  unchanged; profile-settable FELHOM_LOADER; --loader wins). Loud banner +
  manifest loader:/grub-mkimage: fields + -mkimage filename suffix.
- mkimage-surgery.sh (new): post-prepare-iso, in the assistant container. Builds
  a monolithic grub-mkimage loader from the ISO's own GRUB (module set from its
  grub.cfg; embedded search --fs-uuid -> configfile the real menu). Swaps it into
  the ISO9660 tree (real lowercase path) + the efi.img ESP; xorriso re-master
  preserves BIOS-hybrid + UEFI + GPT-ESP, drops only Apple HFS+/APM. Recipe from
  the N100 run evidence, not re-derived.
- Dockerfile.assistant: grub-common + grub-efi-amd64-bin + mtools + dosfstools.
  profiles/n100.profile (new, mkimage + SB-off note).
- Validated on nested VM 311 (RUNBOOK-B legs): leg1 shim boots+installs under
  OVMF SB-enforcing + SeaBIOS; leg2 mkimage boots+installs under SB-off; leg3
  (red-proof) mkimage under SB-enforcing FAILS Access Denied (unsigned -> SB must
  be OFF); leg4 surgery byte-identical payload. bash -n + shellcheck clean.
  Physical N100 closure folds into the rehearsal (n100-safety match-nothing ISO
  built + sha-recorded, unbooted). PXE stays a deferred R-21 note.
2026-07-17 14:28:10 +02:00
admin d86d2699c3 docs: ROADMAP R-25b (DELETE↔RESET consistency) + capability-map Customer RESET row (PROVEN-LIVE external teardown) 2026-07-17 13:17:56 +02:00
admin 7c9647327f docs: hub v0.61.0 Customer RESET — REPORT/CONTEXT + ep0 runbook §10 (tenantsync v1.1.0 deprovision, live-drilled) 2026-07-17 13:15:57 +02:00
admin 2752e12acc hub v0.60.1: host deletion demotes escrow custody (never destroys) + customer-delete purge point + S6b obsolete
- DeleteHost(deleteEscrow) demotes current host_escrow into host_escrow_superseded (copy-before-delete, same tx), spares existing; one shared demoteCurrentEscrowTx (reused by SaveHostEscrow). F-14 provenance/gate unchanged.
- DeleteCustomerConfig now purges both escrow tables for all the customer's hosts incl. already-deleted (F-14 provenance UNION) — the one true purge point.
- Wording: checkbox/refusal/Danger-zone → demotion. S6b OBSOLETE. Red-proofs TestDeleteHost_Demotes + TestDeleteCustomer_Purges + wording guard.
2026-07-17 11:25:38 +02:00
admin 106c3379b0 docs: offsite continuity Part B REPORT/CONTEXT; DIAGNOSE incident→fix pointer; capability map (offsite reinstall-continuity + escrow retention); ROADMAP R-25 (TOCTOU hardening) + R-26 (guided old-history recovery) 2026-07-17 10:55:17 +02:00
admin 1c737db4f4 docs: N100 F5/F6/F7 dispositions (fixed), capability map (fresh-USB wizard PROVEN-LIVE), ROADMAP (R-16 doc-drift closed, R-24 guest-resources-as-desired-state), iso README BIOS G3 note, REPORT/CONTEXT 2026-07-17 09:53:15 +02:00
admin 0c708a3bd0 docs: Direction-2a poke + F4/R-22 PROVEN-LIVE (real save→sync-poke delivered; ~31ms poke→tick; self-grant self-heal ~3s) — capability map + ROADMAP + REPORT/CONTEXT 2026-07-17 08:32:08 +02:00
admin 9d68d52fea docs: hub v0.59.0 + agent v0.89.0 — REPORT/CONTEXT overwrite, capability map (poke shipped, R-22 F4 closed, escrow live-reload), ROADMAP (R-22 shipped, R-13 first slice, R-3 deletion lifecycle) 2026-07-16 23:02:30 +02:00
admin eb227486d0 hub v0.59.0: Direction-2a agent-plane immediate-sync poke sender + ep0 felhom-poke surface
- internal/poke: pinned-host-key SSH poke sender (wgsync sibling) + fire-and-forget Notifier (PokeHost/PokeAllHosts). Poke refuses non-WG targets pre-dial; contentless via ep0 forced command to the box WG /32:51822.
- wiring: Server.SetPoke; applyPBSDR pokes the host after each descriptor gen-bump; handleSetArtifacts (MinAgent floor) pokes all hosts. main.go env POKE_SSH_KEY_FILE (reuses peersync endpoint/hostkey).
- scripts/felhom-poke.sh (non-root forced command) + offsite-endpoint.md §11; manifests/hub.yaml Secret/agent-poke + POKE_SSH_KEY_FILE (image tag bump follows the build).
2026-07-16 22:48:15 +02:00
admin bdb65a80e8 docs: capability-map row + ROADMAP R-23/R-13 + PROMPT-TEMPLATE N.5 for Direction-2 immediate-sync 2026-07-16 21:50:38 +02:00
admin 5c8e6494b7 docs 2026-07-16 21:47:55 +02:00
admin 10e07f5747 docs(spike): immediate-sync transport — poke-relay vs long-poll, both GO/complementary
Empirical spike (demo arc, docs-only, no product code). Measures the two hub->box
'sync now' transports. Verdict: (b) long-poll = primary for user config (hub-only,
reaches controller, no ep0/WG/agent coupling; binding ceiling = twin 60s nginx
proxy_read_timeout + hub WriteTimeout, both config lifts); (a) ep0-relayed poke
= agent-plane nudge (~0.42s, zero ep0/box infra change, EKEYREJECTED-confined).
Viktor Q2 answered: hub.felhom.eu is DNS-only, no Cloudflare. All live mutations
reverted; arc re-verified green.
2026-07-16 20:00:01 +02:00
admin 7747a16ff1 feat(hub): v0.57.0 reinstall-of-existing-customer arc — claim/offsite/escrow continuity
F2 claim re-issue on clean-slate re-enroll (ReissueForReenroll, host-enroll mint path,
single-bump, reset code; hub never stores the password so fork B). F3 offsite re-issue on
re-enroll (ReissueOffsiteForCustomer, same machinery as the manual button). 2.3 escrow honesty
(red-proofed): re-issuing offsite marks the escrow stale (MarkEscrowStale), withholds the
mismatched restic hash from auto-confirm, DR checklist shows stale not done. Events:
claim_reissued_reenroll / offsite_reissued / escrow_stale.

Controller + scripts unchanged (source contradicted both premises): the controller reads escrow
prereqs live from the agent; the installer can't know the descriptor-provisioned storage id. F4
root fix is agent-side -> ROADMAP R-22; demo unblocked live (Part 0 ACL grant). VALIDATION doc
F2 erratum + F3/F4 dispositions. Green gate + Scenario-C red-proof pass.
2026-07-16 18:00:13 +02:00
admin dd961a66bb docs(tests): N100 bare-metal physical validation run (R-21) — findings + current state
Supervised clean-slate reinstall of the demo N100 from a pipeline-built Felhom ISO +
onboarding rehearsal. Core objectives GREEN: first-boot chain reached rc-0 on the first
attempt on real hardware (closes slice A's operator-gated boundary), serial-filter safety
proven on metal, PBS-DR reconciler self-healed on the reused peer, DMI verdict -> key on
MAC+UUID. Hard obstacle was firmware not the pipeline: this cheap AMI AN3PLUS 0.01 board
won't UEFI-boot the ISO's GRUB from USB (relocation 0x0; SB-off + shim-bypass don't help),
worked around live with a grub-mkimage loader from the box's own GRUB (slice-B input).
7 findings logged (F1 firmware loader HIGH; F2/F3/F4 reused-customer edges -> R-1/Peti;
F5 guest-RAM, F6 drive-init mount, F7 back-route). Full doc in documentation/tests/;
ROADMAP R-21 + REPORT + CONTEXT updated. No production code changed.
2026-07-16 16:47:54 +02:00
admin 1fa3250aa3 feat(iso): R-21 slice A — bare-metal Felhom ISO pipeline + first-boot bootstrap
scripts/iso/: a DooPlex pipeline (build-felhom-iso.sh + Dockerfile.assistant) that
turns the official PVE ISO into a Felhom auto-install ISO whose first-boot stub
installs a retry-forever felhom-bootstrap unit which unattended-fetches
felhom-host-install.sh from the public felhom.eu/scripts channel and runs it until the
host is enrolled + a guest provisioned. host-install is UNMODIFIED (invoked only).

- build gates the answer on validate-answer OUTPUT text, never $? (spike S1 exit-0 trap)
- stub is from-iso, fully-up, exactly-once; retry unit owns all network work (S8a)
- retry-vs-resume encoded once: plain first, --resume when install state exists (v1.11.3)
- secret-bearing (embeds the retrieval passphrase): supervised/single-use; env shredded on success

Validated on VM 310: build gate + red-proof, disk-filter fail-safe, chain + retry,
resume-decision, exactly-once, no-net retry+recovery. Terminal host-install rc-0 success
operator-gated (drill customer needs the password-gated create-UI). scripts v1.16.0;
ROADMAP R-21 -> in-progress. Detail in REPORT.md.
2026-07-16 13:47:08 +02:00
admin 914d51aa5f docs(spike): bare-metal Felhom ISO — R-21 spiked (all mechanisms GREEN on nested virt)
Empirical spike of the Proxmox automated-installation path for R-21 (bare-metal
customer-hardware ISO). All legs proven on nested VM 310 (felhom-pve):
- S1 assistant 9.2.7 + ISO provenance; validate-answer exits 0 on failure (parse output)
- S2/S2b zero-touch install SeaBIOS + UEFI incl. Secure Boot enforcing (no keypress/MOK)
- S3 first-boot hook: fully-up = root + working pvesh/pct, exactly-once
- S4 webhook payload enumerated (SMBIOS-UUID + MAC + host SSH keys)
- S5 disk-filter installs only target (canary byte-identical), match-nothing fails safe
- S6 from-url + cert-fingerprint fails CLOSED (red-proof)
- S7 http answer-fetch identity; S8 no-net + invalid-answer failure modes
Findings doc + ROADMAP R-21 status=spiked. No production code.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017qDiBqKKQ5vPB5fXBqu7Kp
2026-07-16 11:29:03 +02:00
admin d202b43b95 docs(map): capability-map evidence audit — verify every status claim against source
Opened every citation in 00-capability-map.md and judged each of the 55 rows
against the specific scenario (not topic adjacency). 23 confirmed (incl. 5
MISSING proven-absent by code sweep), 26 citation-corrected, 6 demoted; no
upgrades. Status: 32 PROVEN-LIVE / 15 IMPLEMENTED / 3 PARTIAL / 5 MISSING
(was 38 PROVEN-LIVE).

Demotions PROVEN-LIVE->IMPLEMENTED: B6 (post-deploy config — T-PAGE-ALL is
render-only), E7 (FileBrowser access — no browse/download proof), F1 (health
email — cited P3-DELIVERY is the enlarge-block trigger), F4 (metrics — cited
legs are H1/H2 harness artifacts), F6 (operator alerting — no corpus doc),
G6 (offsite fill/staleness/freeze — no live-fired leg, freeze inconclusive).

Key corrections: D6 cited CAMPAIGN-3 F10/F11/F12 which are the CRITICAL
failures — fixes shipped agent v0.85 and were re-validated live in CAMPAIGN-4
/6A; C6 cited offbox-restic doc for a PBS-restore claim -> CAMPAIGN-2 T-P9;
C1/C5/B5 cited defect-discovery docs instead of the fix-validation legs.

Closed ROADMAP R-20: operator-key pinning is fully automatic in day-0
(felhom-host-install.sh step_agent_config, L2044/2146-2156/2332-2337).
Header draft caveat swapped for the verified stamp. Full per-row verdict
table in REPORT.md.
2026-07-16 09:19:51 +02:00
admin 4b18cc5208 docs(roadmap): kill P1 R-2 — foreign-WIP was committed concurrent-session work, not orphaned
The ~215 lines flagged as "unknown uncommitted code" (hub/internal/notify/,
claim/engine.go, store.go) were in-flight files from a parallel CC session on
the customer-claim arc, snapshotted before it committed. All landed cleanly
(6b40eb8 v0.50.0, a1d0450 v0.54.0, e205a2d); v0.55.0 shipped. Tree is clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017qDiBqKKQ5vPB5fXBqu7Kp
2026-07-16 08:14:11 +02:00
admin 95853e3dff docs 2026-07-16 08:10:33 +02:00