felhom.eu/CONTEXT.md

# CONTEXT — felhom.eu working state

> Created with the REUSE.md rollout (2026-07-03). Authoritative history: `hub/CHANGELOG.md` (hub),
> `website/CHANGELOG.md`, `scripts/CHANGELOG.md`; end-of-task detail in `REPORT.md`.

> **`documentation/backlog/OPEN-ITEMS.md` is the single source of truth for open work** (2026-07-27).
> Read it first; `ROADMAP.md` holds the full history and reasoning behind each item. **`REPORT.md` is
> per-session and overwritten — nothing durable may live only there.**

> **This file is NOT `STATUS.md`, and the two must not be consolidated** (2026-08-02). `CONTEXT.md`
> is technical working state written for Claude Code: rulings with their mechanisms, file paths,
> version numbers, the reasoning a session needs before it touches code. Root `STATUS.md` is written
> for the **operator** — technical, but not in the code and not a reader of the audits — in plain
> language, one screen, no identifiers in the prose. Same subjects, different readers; merging them
> would make one of the two audiences stop reading. `STATUS.md` is also a **view of `OPEN-ITEMS.md`**
> and holds nothing of its own; this file does hold its own content, namely the standing rulings below.

## Standing rulings

**DooPlex's own backup, surveyed 2026-08-06 (R-232, read-only).** Five sets nightly, 14/14 in 14
days, single-file restore proven byte-identical. Destination for **every** set is `/dev/sda1`;
`.claude-memory` and the PostgreSQL dumps are the two same-disk cases. **Nothing leaves the box** —
Longhorn's target is DooPlex itself over NFS, and the only outbound-looking cron pulls inbound.
`notify_failure` is a no-op (no webhook), so a failure is silent. The restic passphrase has an
**offline operator-held copy**, so disk loss is recoverable — it is not system-held escrow, which is
the only residual. Full survey: `audits/RECON-dooplex-backup-2026-08-06.md`.

**S-37 — A CLAIM IN AN INSTRUCTION FILE IS CHECKED, NOT TRUSTED (2026-08-06, R-229/R-230 close-out).**

1. **The workspace-root `CLAUDE.md` is a SYMLINK** to `documentation/runbooks/workspace-CLAUDE.md`.
   One file reachable by two paths cannot diverge; two files kept identical by hand must be policed
   forever. Check 5 accepts either shape — a clone elsewhere may still have two files — and for the
   link it asserts the target **resolves**, because a dangling link means the workspace instructions
   load nothing at all and there is no content left to notice is wrong.
2. **A citation that calls a register item OPEN is now gated** (check 7). The trigger is an
   **openness claim**, not any citation: policing every mention would fire on ~30 legitimate
   provenance citations and the gate would be switched off, which is R-29's own lesson.
3. **`MEMORY.md` is WARNed, never FAILed, on content.** Claude writes it between sessions; a hard
   failure would refuse a human's push over a line no human typed. The warning is aimed at the model
   that will next edit the file, which is what makes the loop self-correcting.
4. **A scan is not evidence until its false-positive class is known.** Three "expired statements"
   reported on 2026-08-06 were all filenames matched inside markdown link targets, while the one real
   expired claim carried no ISO date and was missed. Link targets are stripped before any content
   judgement now.
5. **A red-proof that does not go red is a finding about the instrument.** Two shipping bugs in check
   7 were found exactly that way — a state marker that is not self-closing (which made the check read
   its own founding case as open), and a CLOSED exemption applied line-wide instead of to the
   citation's clause.
6. **Correct a runbook only after measuring.** `target-selection.md` said demo-hp had no off-site
   tier; `pvesm list felhom-pbs` on the box returns two snapshots in its own namespace, newest
   2026-08-04. The claim was true when written and went stale when F10 resolved. This file decides
   which machine may be destroyed, so a wrong statement about what a machine holds is how a drill
   lands somewhere it should not.

**S-36 — THE AUTO-MEMORY STORE IS BACKED UP, NOT COMMITTED; AND THE WORKSPACE IS INSTALLABLE
(2026-08-06, R-229 part 2).**

1. **`.claude-memory/` is never committed.** It is **auto-written** — Claude writes it, so nobody
   reviews it before it lands — and may name hosts, paths and out-of-band secret locations that the
   secrets rule keeps out of committed files. A scan on 2026-08-06 found no credential *values* (535
   keyword mentions across 104 files, one `key: value`-shaped hit that was prose, zero private-key
   blocks), but "no secrets today" is not a property a directory keeps on its own when a machine
   writes to it unattended. It is protected by `dooplex-backup.service` instead
   (`CLAUDE_MEMORY_DIR`). **Reversing this needs a passing secrets scan first.**
2. **No memory file is ever deleted.** Set-aside means **moved** to `.claude-memory/archive/`. The
   part-2 reconciliation moved 40 and indexed 4, with the file count identical before and after.
3. **`MEMORY.md` is the index, and the index is what loads.** Topic files are read on demand, so
   detail belongs in them, not in the index. `instructions_gate` check 6 now enforces the limits —
   over-limit FAILS (auto-memory drops content past the limit with no error), an orphan WARNS (the
   store is outside git), an absent store PASSES **while printing its reason**.
4. **Correcting auto-written content is NOT yet ruled on.** The part-2 session diagnosed 21 lines
   carrying version literals, 5 carrying host addresses, and an entry still asserting a fact whose
   hand-written twin was deleted as expired — and **changed none of them**. R-230(a).
5. **The workspace is installable.** `scripts/install_workspace.py` lays down the workspace-root
   `CLAUDE.md` and the `InstructionsLoaded` hook from versioned copies. Its targets are **live
   configuration**, not disposable like the skills: it backs up before every write, **merges**
   `settings.json` owning exactly one key, and **refuses** an unparseable settings file rather than
   overwriting it. `documentation/runbooks/workspace-CLAUDE.md` remains the canonical source — a copy
   under `workspace/` would be a third copy of a file whose problem is that copies drift.
6. **A rule file created mid-session is NOT live in that session.** A directory whose instructions
   were already seeded is not re-scanned. Proven the hard way: after writing four rule files, reads
   that should have matched them produced no hook line at all. Verify new rules from a fresh session
   (`claude -p`), never from the frontmatter.

**S-35 — INSTRUCTION FILES ARE A SHORT CORE PLUS PATH-SCOPED RULES (2026-08-06, R-229).**

Decided while rightsizing the four `CLAUDE.md` files. The mechanisms were verified before being
relied on, and two of the three the task assumed turned out to need correcting:

1. **Shape.** A `CLAUDE.md` is a short always-loaded core: what the repo is, a "doing X → read Y"
   retrieval map, the gotchas that cost an incident, one command per surface, the fences, the
   end-of-session checklist. Everything path-bound goes to `.claude/rules/<topic>.md` with a
   `paths:` glob list, which Claude Code loads **only when a matching file is read** (confirmed
   against the installed 2.1.222 build). Procedures go to the skill that already covers them.
2. **Earned rationale goes in block-level HTML comments.** They are stripped before injection and
   never reach the model — **verified empirically**, not assumed: a control (two plain markers →
   both reported) against a treatment (one marker inside `<!-- -->` → not reported, twice). So the
   incident histories stay in the repo for human readers at zero cost. **This makes the raw line
   count the wrong measure** — the gate counts *effective* lines, and so should any future budget.
3. **What may NOT move into a lazily-loaded file:** irreversible fences and agent directives. Rules
   are not re-injected after `/compact`; the project-root `CLAUDE.md` is. That is why the
   destructive-target fences, the secrets rule and the clean-tree gate stay in the root file.
4. **Amnesty criterion.** A prohibition with no recorded production violation and a recoverable worst
   case may be deleted. Anything paid for in a real incident stays — and **a fence keeps its
   permitted target and its reason**, never reduced to a bare prohibition. Exactly one item met the
   bar (three generic code-quality bullets).
5. **No component version literal in any `CLAUDE.md`.** Versions change several times a day; ask the
   hub or the box. **A historical citation is not fleet state** — "fixed in hub v0.97.0" cannot go
   stale the way "this box runs agent 0.93.0" can, so citations moved into the HTML comment beside
   the rule they justify rather than earning a carve-out in the gate.
6. **Subagent rule, narrowed (operator, 2026-08-06):** read-only research, inventory and verification
   are permitted with a bounded digest; **no subagent may edit, commit, build, deploy or touch live
   hardware.** None was used for R-229.

Enforced by `scripts/instructions_gate.py`, registered in `controller_gates.py` and
`agent_gates.py`. **`felhom.eu/CLAUDE.md` is knowingly still over the ceiling (227 effective lines)**
and is therefore not yet gated — closing it needs the restructure R-229 defers.

**S-34 — UNLOCKING AND RESTORING ARE SEPARATE. The recovery screen shipped (2026-08-05, controller
v0.200.0, R-193 CLOSED). Read with S-32 and S-33; together they close the whole customer journey up to
the listing.**

**THE RULINGS (operator, 2026-08-05), recorded as given:**

1. **Unlocking and restoring are separate.** This screen unlocks only. Restore stays per-app, in the
   backups area. **A screen that unlocks and then offers to overwrite is two decisions wearing one
   button.** Putting files back is its own item (→ **R-213**), and its stated requirement is a
   live-versus-backup comparison — the customer must see what would change before anything is
   overwritten.
2. **"I do not want the old data" MOVES THE STORE ASIDE**, after double-checking with the customer.

*Mechanism facts a future session needs:*

- **ONE CORE, TWO CALLERS.** `backup.RecoverInstallCore` is the only fetch→unseal→compare→install path.
  `RecoverAndInstall` is now a thin CLI wrapper (exit codes and printed lines byte-identical — every
  pre-existing CLI test passed unchanged) and the web handler calls the same function. **Asserted from
  source by AST on BOTH sides**, plus a third test that the routes and the landing-page interception
  exist. Two implementations of the one operation that can permanently lose a customer's data would
  drift, and only one would be tested.
- **THE TRIGGER HAS TWO SHAPES, and the second is the one that matters in practice.**
  `backup.OffsiteRecoveryOffer` = the hub holds a package **AND** (no repository password **OR** the
  tier is orphaned). The literal "no repository password" alone is a window that CLOSES BY ITSELF:
  `WriteOffboxSecrets` auto-generates a password on re-apply — R-193's own orphaning mechanism — and
  since hub v0.96.0's credential self-heal the re-apply happens within ~15–30 minutes. Shape (b) is
  also the state the shipped move-aside requires, which is what lets the discard choice reach the
  existing handler instead of needing a new one.
- **CLAIMED IS PART OF THE PREDICATE.** A legacy-open box (no password anywhere) reaches `ServeHTTP`
  through `RequireAuth`'s pass-through, so without an explicit `authEnabled()` check the interception
  fires for an unauthenticated visitor. Caught by a test, not by review.
- **"Most nem" suppresses the FULL PAGE ONLY.** The backups-area entry point is bound to
  `recoveryOffer`, never to the postpone flag. A one-shot notice a flustered person clicks past is a
  notice that never happened.
- **The code is handled no more loosely than on the CLI:** POST body only (`PostFormValue`), never
  logged, never persisted, never echoed, cleared on every path, `no-store`, `autocomplete="off"`.
  **NO LOCKOUT, deliberately** — the code is a ten-word phrase and locking a customer out of their own
  data for a typo is worse than anything a lockout prevents; failures are logged locally without the code.
- **NOT YET PROVEN LIVE: a CORRECT code through the page.** No recovery code was kept for demo-felhom's
  orphaned history (`RECON-offsite-dr-chain-2026-08-04.md` §12.3) and demo-hp's is operator-held. The
  live run exercised handler → agent → hub fetch → age KDF and stopped at the unseal.

**S-33 — THE BOX DECLARES, THE HUB ANSWERS. R-204 item 4 / R-193's credential half closed
(2026-08-05, controller v0.199.0 + hub v0.96.0). Read with S-32; together they close all four of the
drill's manual interventions.**

**THE RULING AND ITS REASONING (operator, 2026-08-05).** Automate the rebuilt box's credential, and
**the trigger is a state the BOX DECLARES — never the hub inferring from a report.** From the hub an
ABSENT off-site object has FOUR meanings — never configured, mid-restart, a transient config read
failure, rebuilt-and-stranded — and the hub cannot tell them apart. The box can, from two local facts:
its data area is fresh (no repository password) AND the hub holds a sealed recovery package for it
(the ACK's `identity_blob_present`). **Both are required.** Freshness alone is a box that never had
off-site backups — declaring on it would make the whole fleet ask for credentials.

**SECOND RULING RECORDED:** the dashboard-password exposure on the future recovery-preview screen is
**metadata** (backup dates, app names), not content, and is **ACCEPTED**.

*Mechanism facts a future session needs:*

- **The ACK field used to be discarded.** `EscrowAutoConfirmer.Reconcile` returns early when the box is
  neither pending nor escrowed — exactly a rebuilt box — so `identity_blob_present` was thrown away
  every cycle. It is now recorded FIRST, before every gate, in the confirmer itself (already the one
  place the ACK's escrow object arrives, and already wired; a second consumer would be a second wiring
  point, and this project's built-but-never-wired count is six). Last-write-wins, not set-only: a
  customer RESET must be able to turn the declaration back off.
- **The declaration is inert to every existing reader, established from their code:** it carries
  `enabled:false` and zero sizes, and `OffsiteChecker.isStale` returns early on `!Enabled` while
  `fillBand` returns OK on a zero quota/size. A configured box's report JSON is byte-identical to
  v0.198.0's — no `state` key at all.
- **`reportHasOffsite` had to be tightened, and it is the ninth comment-vs-code entry — caught before
  it shipped.** Its comment asserted *"presence == applied-on-the-box"*. Left as bare presence it would
  have read a request for help as proof the tier was applied. It now requires `enabled:true`, provably
  a no-op for every pre-v0.199.0 report shape.
- **Restage before mint is possible because the stored value SURVIVES a consume** —
  `ConsumeOneTimeSecret` stamps `consumed_at` and nothing else, and `value` is `TEXT NOT NULL`. This was
  established from the schema, NOT inherited from the PBS analogy; the two secrets are different objects
  and assuming a shared shape is how two sessions confused the credentials.
- **The debounce is TWO DISTINCT REPORTS, derived from the ~15-minute report cadence** — one would act
  on a blip, three would leave a stranded customer waiting ~45 minutes. It counts fresh evidence, not
  reconciler ticks, so the 5-minute sweep cannot shorten it.
- **§8.4's deliberately-unhealed state exists and is excluded UPSTREAM, by construction:** the regressed
  shape (a box that lost its target while still holding its repository password) cannot reach the
  reconciler, because the declaration predicate requires the repository password to be absent.
- **R-192's counting guard is REPLACED, not repaired.** A declaration outranks both inferred shapes and
  the delivery checker stands down with a record — two mechanisms healing one customer would
  double-issue, and that one can only mint.
- **NEVER widen this to the ceremony. Credential automatic, key customer-present.**

**S-32 — THREE OF S-31's FOUR MANUAL STEPS ARE CLOSED (2026-08-05, R-204 items 1–3 / R-196).
controller v0.198.0 + hub v0.95.0. Read this BEFORE S-31 — it supersedes S-31's steps 2–5.**

- **Step 1 (the reset code) — CLOSED.** `web.effectiveClaimCode` now calls `settings.ReloadClaimCode()`
  before applying the settings-vs-config precedence, so a code minted by the SEPARATE
  `--print-reset-code` process is visible without a restart. **The precedence rule is unchanged** —
  the defect was the freshness of the settings value, not which source wins. It is a READ-THROUGH on
  the claim path only, deliberately **not** a watcher, a signal handler or a TTL: a TTL would leave a
  window in which a superseded code still works, which is worse than the bug. `effectiveClaimCode`
  now returns an error and **every caller fails closed** (`claimGateActive` keeps the gate UP; an
  absent settings file is NOT an error — a pre-first-save box legitimately falls back to the
  controller.yaml bake). **S-31's "restart the controller between minting and claiming" is obsolete.**
- **Steps 2–4 (the false staleness) — CLOSED.** `offsite.ReissueCredentials` no longer calls
  `MarkEscrowStale` and no longer emits `escrow_stale`; `offsite_reissued` is untouched. **The
  mechanism that made this expensive, recorded because it is not obvious:** `stale_at` makes
  `GetEscrowStatusForCustomer` WITHHOLD `restic_pw_sha256` — which is the exact value the controller's
  Scenario-F re-check (`report.EscrowAutoConfirmer.reconcileEscrowed`) compares against its live repo
  password on every ACK. **The precautionary mark was blinding the measurement that would have told
  the truth**, leaving the box able to report only the hash-LESS reason, which is false. Coverage
  after removal: that per-ACK comparison (continuous, box-side) plus R-197's `offsite_repo_key_changed`
  (edge-triggered, hub-side, at a supersession). **`MarkEscrowStale` is KEPT with no caller** — the
  flag is still read by the ACK, the operator card and the PBS-DR view, and the right way to set it is
  a future caller that has MEASURED a change; `TestEscrowStaleMechanism_StillWithholdsAndClears` stops
  it decaying meanwhile. **The manual `/backup/offbox/confirm-escrow` workaround is no longer needed
  after a Re-issue.**
- **Step 5 (the restore default) — CLOSED.** `restoreScratchOutcomeMsg` states, for `mode=unit`, what
  came back, that the customer's own files did NOT, and the step that gets them; the wizard's intent
  card 1 states its scope before the choice. **The `mode=full` size gate is untouched** and pinned as
  unchanged. **The default stays `unit`:** all three wizard forms set `mode` explicitly, so changing
  it would alter nothing the customer sees while silently changing a hand-crafted mode-less POST.
- **WHAT REMAINS is S-31's step 1-before-the-list: a rebuilt box still cannot obtain an off-site
  credential unaided** (the one-time password was consumed by its predecessor), so an operator
  Re-issue is still required. That is **R-193**, it needs an operator ruling, and it is why the
  capability-map recovery row keeps its *"with a person present"* qualifier.
- **Live-validated on demo boxes, endpoint-level** (no browser on DooPlex): reset codes on
  demo-felhom 9201 with `restarts=0` across the mints; the unit-restore outcome and the full-restore
  size-gate reveal on demo-hp 9201, using `privatebin` so the drill's `calibre-web` scratch was not
  touched. **The Part 2 change was NOT fired live on demo-hp** — a Re-issue there was out of scope.

**S-31 — THE DRILL PASSED: a customer's file survives a machine rebuild and comes back byte-identical.
The capability is proven; the customer JOURNEY is four undocumented manual steps (2026-08-04, R-201/R-204).**

**Proven on hardware, end to end:** demo-hp's controller data volume was destroyed and the sentinel
deleted from disk. The recovery code produced `8a9e33aa4da6…` (byte-identical to the pre-wipe on-disk
key AND the hub's independent record); it installed on the bare box; the **existing repository opened**
(`repo_state: null`, 3 snapshots, 42 026 B = the pre-wipe size — not a fresh history); and the customer
restore flow returned the sentinel at `643166269103a25c…`, **byte-identical**. `identity_blob` was
unchanged throughout and **no ceremony was run at any point** (superseded rows still 2).

**But it took four manual steps that are in no design document (R-204), all measured:**
1. a rebuilt controller cannot configure its off-site tier — the one-time password was consumed by its
   predecessor (`no unconsumed offsite password`, R-193);
2. the Re-issue that fixes that sets `stale_at` **while `restic_pw_sha256` is unchanged** (R-196);
3. a stale escrow makes the hub withhold the hash from the ACK → `EscrowAutoConfirmer` can never flip
   `pending → escrowed` → `OffboxRunnable` refuses every run;
4. it was cleared with the **manual** confirm (`/backup/offbox/confirm-escrow`), NOT a ceremony — a
   ceremony would have superseded the identity blob and destroyed the recovered key;
5. and the restore's **default `mode=unit` returns the recovery unit, not the customer's files** —
   `mode=full` is what returns documents. A customer following the obvious path gets the wrong thing
   and is told nothing.

**Before any of it, a rebuilt box is unclaimed** and the claim gate intercepts every controller
endpoint. **The local escape hatch does not work unaided:** `--print-reset-code` writes the new hash to
`settings.json` while the running controller keeps its old copy in memory, so the code never matches —
**restart the controller between minting and claiming.**

*Facts a future session needs:*
- **A guest rebuild in this fleet is a controller-DATA-VOLUME loss, not a guest reprovision.** The
  2026-08-03 incident R-193 is filed against ran with guest 9201 up throughout — no `pct destroy`, no
  `pct restore`, no `--selftest=provision`. Reproduce it that way.
- **A good snapshot is not durable against a later bad run on the same day.** `forget --keep-daily 7
  --group-by host,tags` keeps one per tag per day; a later, worse snapshot evicts a good one.
- **Never run a ceremony while a recovery is in flight** — it supersedes the identity blob. Under
  v0.93.0 the old blob is retained, but nothing serves a superseded blob back (R-199).

**S-30 — the R-201 drill was PREPARED and HALTED BEFORE THE WIPE (2026-08-04). Nothing was wiped.**

It stopped at step 4 because the sentinel file was **not in the off-site snapshot** while the run
reported `ok` (**R-203**): the deploy-time `${USERDATA_PATH}` root and the backup-time `NamespaceRoot`
disagree for an app whose `HDD_PATH` equals `system_data_path` — and the same compose used both roots.
Wiping would have destroyed the only copy of the sentinel and proven nothing.

*Established live on the way, all firsts, and all worth carrying:*
- **A rebuilt box's off-site run REFUSES** — `repo_state: orphaned`, the orphan card, and
  `offbox_repo_orphaned` at the hub. It does **not** silently start a fresh history. Closes R-193's Q3.
- **The orphan reset works**: move-aside to `/home/felhom-repo.orphaned-20260804`, never delete.
- **demo-hp's pre-rebuild off-site history is permanently unrecoverable** — its key is in superseded
  row id 3 with `identity_blob` NULL, superseded **four hours before** v0.93.0 fixed the retention.
- **Neither pre-existing off-site-toggled app has a restorable file leg** — both are named-volume-only,
  which the tier tars but the customer restore never unpacks. Any drill needing a recoverable file
  needs an app with a **mandatory userdata path**; `calibre-web` is now deployed on demo-hp as that
  fixture, with the sentinel in place.

*Still not established, unchanged:* **no file has ever been restored from an off-site backup after a
wipe**, and Part 0's install path (controller v0.196.0) has never run against a live recovery.

**S-29 — a box may fetch its OWN sealed recovery blob with its OWN credential; the operator-driven DR
path is a separate thing and stays gated (2026-08-04, R-199; hub v0.94.0 + agent v0.125.0 + controller
v0.195.0).**

**The decision, so it is never re-litigated from scratch.** Self-service recovery needs the box to
obtain its own sealed package. The only retrieval that existed was operator-gated behind recovery mode
and the global key (`dr.go`), which cannot serve a customer-present flow. `GET /hosts/{id}/escrow` now
serves that ONE object to its authenticated owner.

- **Why it is safe:** the blob is opaque; the hub has never held R and has no decrypt path; a wrong
  code fails closed at age's scrypt KDF (measured live — `exit status 1`, no plaintext, nothing
  written); the caller already authenticates as this host for its report, desired state, WG
  registration and PBS token, so this adds no new identity, only a new object.
- **What it changes, which is the whole of the trade:** obtaining the blob used to require the
  OPERATOR to arm recovery mode. It now needs only that box's credential. **Fewer parties are
  required.** The mitigation is that the capability is AUDITED, not silent: every retrieval raises
  `escrow_blob_served` (warning, operator-only), written before the bytes leave.
- **The override is one constant.** `api.escrowSelfServiceRetrieval` — false re-imposes the
  recovery-mode requirement on this endpoint and changes nothing else. Do not scatter the condition.
- **`handleReEnroll` / `handleGetRestoreDirective` are UNTOUCHED** and must stay so: they rotate the
  host API key and serve the K-escrow and the directive as well. Different object, different gate.
  Pinned by `TestEscrowGet_OperatorDRPathUnchanged`.

**Facts a future session needs before touching this area:**
- **The unseal lives in the AGENT and must stay there.** `age` is an agent runtime dependency
  (`/usr/bin/age`, present on both demo hosts) and is deliberately absent from the controller image;
  the blob is host-scoped and its only writer is the agent under the per-host key.
- **The agent returns ONE field.** Not the tunnel token, not the PBS token, not the WG key — the
  controller is a trust tier down. Narrowing cost nothing and is not recoverable later.
- **R's rules:** in memory for one call, cleared on every path, never on disk, never in argv, never
  logged, never echoed. The controller CLI reads it from STDIN for exactly that reason. Verified live:
  0 log lines, 0 files, 0 leftover staging dirs, with a positive control proving the search worked.
- **The chain today: links 1–8 walked, 9–11 not.** The KEY comes back. Nothing installs it, reopens a
  repository with it, or restores a file — R-200's remaining half and R-201.

**S-28 — the escrow retention now covers the OFF-SITE data key, and customer-present recovery is the
accepted design, which makes that retention load-bearing (2026-08-04, R-198/R-197; hub v0.93.0).**

**The contract changed, and this is the sentence to carry:** when an escrow blob supersedes another,
`demoteCurrentEscrowTx` retains **both** sealed keys — the K-escrow `blob` (PBS datastore key) **and**
`identity_blob` (the age-wrapped bundle carrying the restic **repository** password). Until v0.93.0 it
retained only the first, so the retention preserved the whole-guest key and destroyed the off-site data
key, and **the destroying act was the escrow ceremony a rebuilt box asks its customer to run.**

*Why this is load-bearing rather than tidy:* the operator ruled on 2026-08-04 that **candidate (c) is
refused — no repository password is retained on the Proxmox host.** With no host-retained copy, the
**customer-present recovery path is the only way back from a rebuild**, and it runs entirely through
the retained identity blob. A session that touches escrow custody is touching the single copy.

*Facts a future session needs before touching this area:*
- **The ordering is load-bearing and invisible from the routine.** `SaveHostDRBundle` writes
  `identity_blob` **after** `SaveHostEscrow` returns (the escrow PUT, `api/handler.go`), so at demote
  time `host_escrow` still holds the OLD identity blob. Invert that and the retained bytes become the
  NEW blob filed under the OLD hash — recoverable-looking and wrong. Pinned by
  `TestSaveHostEscrow_RetainsIdentityBlob`.
- **`demoteCurrentEscrowTx` is shared by re-escrow and host-delete.** Both have a test; fixing one
  caller instead of the routine is a demonstrated red-proof, not a hypothetical.
- **Nothing was backfilled and nothing could be** — pre-v0.93.0 retained rows have no identity blob and
  their source rows are already overwritten. Both demo boxes' pre-2026-08-04 repository passwords are
  gone permanently, recovery code or not.
- **A retained key is not a recovery.** The hub's blob-serving endpoints have no client (R-199), the
  injection seam has no form (R-200), and the path has never been exercised (R-201). v0.93.0 makes the
  key survive; it does not hand it back.
- **`offsite_repo_key_changed`** (R-197, warning, operator-only, edge-triggered on a supersession where
  both hashes are known and differ) is the evidential signal that a box's off-site data key moved. It
  carries **no hash value**. `MarkEscrowStale` is **precautionary**, not evidential — see S-26(a).

**S-27 — a customer with NO machine ever bound is UNKNOWN, silently; one that was bound and went quiet
still alarms (2026-08-04, R-195; hub v0.92.0).** Operator ruling, implemented as
`store.HasEverBoundHost` (live `hosts` row OR `host_deletions` tombstone) consulted once at the top of
`CheckBackupDeadlines`' loop. **The discriminator is "was a host EVER bound", never "has a report
arrived"** — a box that was installed and never phoned home is a real fault. Fail-**open** on a read
error; the deferral is logged with its own counter; the R-81 anchored-verdict structure is untouched.

*The mechanism is worth carrying, because the guard that should have covered this was keyed off the
wrong thing:* the existing down-skip reads `StalenessChecker.GetState()`, whose map is seeded from
`store.GetCustomers()` — **a query over the `reports` table**. A customer with zero reports is in no
row, gets no state, and `GetState()` returns `""` not `"down"`, so **the skip misses exactly the
customer it would most obviously cover.** `david` (created 2026-08-01, no machine) e-mailed an
`expected_dbdump_missed` ERROR three mornings running; `peti-felhom` — active, host deleted
2026-07-15 — does not, because its 482 old reports make it `down`. Generalise it: **a "skip the dead"
guard built on evidence of life cannot see something that was never alive.**

**S-26 — the one-shot secret is the recoverable one; the irreplaceable one is minted fresh on every
guest rebuild (2026-08-04, R-193 spike — `audits/SPIKE-offsite-credential-recovery-2026-08-04.md`).
No code shipped for it; the decision is the operator's.**

Two sentences, because they are what every future session needs before touching this area:

- **Q1 — what is one-shot.** Of the three secrets in the offsite path, the hub's **one-time
  provider password** is one-shot but freely re-issuable, the box's **SFTP key** is regenerated on
  every apply by design, and the **restic repository password** — the DATA key, which the agent's own
  source calls *"irreplaceable"* (`felhom-agent/internal/escrow/identity.go:35-39`) — is the only one
  nothing can restage.
- **Q2 — what a rebuild costs.** A rebuilt controller **mints a brand-new repository password**
  (`WriteOffboxSecrets`, `offbox.go:392`, generates whenever `<DataDir>/offbox/repo_password` is
  absent) and **no automatic path ever consults the escrowed one** — `InjectOffboxPassword` has exactly
  one caller in the whole repo, a web form a human pastes into — so **every guest rebuild orphans the
  previous off-site history.**

*Measured without touching a box, and the method is reusable:* `host_escrow.restic_pw_sha256` vs
`host_escrow_superseded.restic_pw_sha256`. demo-hp `8e03eddf…`→`8a9e33aa…` (15 snapshots / 40.9 MB
orphaned); demo-felhom `48741892…`→`c60c8bc7…` (**36 snapshots / 1.14 GB**). **demo-felhom is the half
that matters:** R-193 recorded it as having survived the rebuild by luck, and it did — its *delivery*
recovered in 76 s off a stale staged secret. **Its repository did not**, and nothing said so for 13 h.
*Luck restored the plumbing, not the data.*

*Three corrections that must not be re-inherited.* **(a)** `ReissueCredentials` does **not** rotate the
restic password — R-39's record, `offsite.go:198-201` and `api/handler.go:1067-1069` all say it does,
and all three are wrong (→ **R-196**, the eighth entry in `CLAUDE.md`'s table). **(b)** Candidate (b),
*"recoverable from escrow at re-bootstrap"*, is **not implementable** — the escrow is R-wrapped and
zero-knowledge (D6), so only a customer-present ceremony can open it, which is the manual form that
already exists. **(c)** Candidate (a), *"the hub restages automatically"*, **already exists** as
`reissueOnReenroll`'s F3 leg and is wired to the wrong event — it sits behind `handleHostEnroll`'s
mint-once-reuse short-circuit, and a **guest** rebuild leaves the `hosts` row intact. **Shipping it
first would have made both boxes look healthy on 2026-08-04 while their snapshots were orphaned** —
strictly worse than the current loud failure.

*The candidate nobody had named,* and the only one aimed at the actual harm: **the agent survives a
guest rebuild**, already receives the repo password over the pinned local API
(`POST /escrow/stage-secret`) and already writes it to a fixed 0600 path — it merely **wipes** it after
the ceremony. Retaining and serving it back needs no new seam. Its price is one real trade: a copy of
the data key at rest on the Proxmox host. **That trade is the operator's to make and the spike does not
make it.**

**S-24 — offsite retention is ep0's, and the box asks for none (2026-08-04, R-191; installer 1.25.0).**
R-89 moved offsite pruning server-side and box tokens stay write-only. The 2026-07-26 "two weeks"
ruling was not reversed — **where it is ENFORCED moved, and the installer's `keep_last: 2` did not
follow**, so every weekly run uploaded fine and then failed the job on a refused prune. The offsite
tier now writes `keep_last: 0`, which the agent's existing guard already reads as *never prune from
the box*; the local tier is untouched. A gate asserts it. **Before switching it off, ep0 was read:**
both namespaces have a prune job at 03:30 keep-last 2 that has run daily since 2026-07-27 (18 tasks,
all OK). **If that ever stops, `keep_last: 0` is unbounded growth** — check ep0's prune jobs before
assuming the offsite tier is retained.

**S-25 — a lost storage grant repairs itself, and the repair is REPORTED (2026-08-04, R-190; agent
v0.124.1).** On a missing grant the agent runs the existing root wrapper (`felhom-backup-target-apply
grant <id>`, already sudoers-permitted for any id) and re-reads once — the pbsdr R-22 shape. Bounded
at one attempt per tier per hour.

*The record is the point, and getting it wrong is easy.* A repair that leaves only `ok` behind
destroys the only evidence a permission vanished. v0.124.0 reported degraded for "one cycle" and
**that failed in production while every unit test passed**: `probeAll` is called independently by the
self-check log and by the collector, the repairing call was the log's, and the host-report three
seconds later sent `ok`. The fix is a latch on **time** (20 min > the 900 s report interval), so at
least one report must carry it. The explanation rides `Feature`, because that is the field the hub
interpolates into the operator's e-mail — `Reason` does not travel. **Nothing new was built:** the
hub's existing ok→degraded→ok edge is the channel.

*Caveat measured live:* **PVE caches permissions** (~40 s and ~16 min observed), so detection lags the
loss and a single permission read is a lagging indicator → R-194.

**S-23 — the host (on-box) whole-guest tier is restore-PROVEN, unattended, on both demo boxes
(2026-08-04). Scope: those two boxes, not the fleet.**

Four SCHEDULED runs overnight, none triggered by hand: demo-felhom host **83.8 s** / offsite
**540.4 s**; demo-hp host **109.3 s** / offsite **300.1 s**. Every one restored into a scratch guest,
booted, verified and destroyed itself.

*What this closes.* Until yesterday every live restore-proof this project held was on the OFFSITE
tier. The on-box tier — the one an ordinary recovery uses — had never been proven on either box, and
not because it failed: the agent could not read the storage it lives on (R-185), so it never saw an
archive there to test.

*What was observed for the first time.* Both boxes had BOTH tiers due simultaneously. Never-proven
sorts first, so each took its host tier, deferred the offsite one, and picked that up on the next
evaluation six hours later — R-86's ordering and the one-heavy-operation gate, working together,
unsupervised. The host-tier proof then reached the hub through R-189's merge, which is that path
carrying a host-tier entry for the first time.

*The asymmetry worth remembering:* a host-tier restore is **83–109 s**; an offsite one is
**300–540 s**. The tier that matters for an ordinary recovery is also the cheapest to prove.

**S-21 — an empty listing cannot distinguish FORBIDDEN from NEWBORN, so the box asks the permission
question directly (2026-08-03, R-185; agent v0.123.0 + installer 1.24.0).**

*The defect.* On both demo boxes the agent's token had `FelhomAgentStore` on `local`, `local-lvm` and
`felhom-pbs` and **not** on `felhom-backup` — the storage the same installer configured as
`local_backup_target`. The content API answered `{"data":[]}` through the token while root listed
three archives. `pickForThisRun` skipped the tier as *"no settled archive yet"*, which is exactly
what a brand-new tier reports, so the host tier was never restore-testable and nothing said so.

*The rule.* The permission question has a definite answer where the listing does not. `Permissions`
reads `/access/permissions?path=/storage/<target>` **as the agent's own token** — asking as root
answers a different question and always says yes — and one `capability.Status` per configured tier
reports it. The probed set comes from `BackupTiers()`, never a fixed list: a hardcoded probe list is
the defect reproduced inside the fix.

*The measured trap, because the obvious reading is wrong.* An ungranted path answers **neither empty
nor 403**: it carries the privileges inherited from the box-wide `/` grant
(`Sys.Audit, SDN.Use, Datastore.Audit`). Testing for path-presence, or for `Datastore.Audit`, reports
a blinded storage HEALTHY. The probe tests **`Datastore.AllocateSpace`**, and re-measuring is required
before that constant is ever changed.

*Criticality, weighed once.* Critical, because the hub alerts only on critical and a non-critical
entry would ride the report and alert nobody. **Except** the `local` fallback target, which
host-install's own comment calls the DEGRADED configuration: still probed, still reported, but it
does not page — turning an ordinary documented setup into an alert is how a signal becomes something
an operator archives unread. It never consults content, so it cannot alarm on a newborn tier by
construction, and it never reports ok when it could not ask.

**S-22 — the installer's Scenario-F arm must finish the job, not just leave the definition alone
(2026-08-03, R-185).** `configure_backup_target` has two arms. Case A creates the storage and grants
in the same breath. The reuse arm — *"the target already exists"* — returned **without granting**, and
that, not `PVE_STORAGES`, is where the drift came from: a box whose target pre-dated the install
(the vzdump-target-move runbook, or a reinstall) pointed `local_backup_target` at a storage its token
could not read. The reuse arm now ensures the ACL through the same guarded wrapper. **Scenario F is
unviolated** — the storage DEFINITION is untouched, and granting the role the agent is supposed to
have on the target this script is about to write into `agent.json` is finishing the job, not
retargeting the box. `$BACKUP_TARGET_ID` stays OUT of `PVE_STORAGES`: that list is granted a step
before the target is resolved, and `--acl-storages` entries are preflight-checked for existence.
A gate asserts every arm that resolves the target also grants on it.

**S-19 — a restore-test PROOF is durable and reportable; a FAILURE is neither, and that asymmetry is
the design (2026-08-03, R-189; agent v0.122.0).**

*The rule.* Only successful restore-tests are written to `RestoreTestState`, and that state is what
the host report carries after a restart. Failures live only in the in-memory `backup.Store`.

*Why, in one line each.* A **success suppresses future work** — under R-86's per-archive due-check a
proven archive is never re-tested, so a lost proof leaves the box quietly less tested than it
believes, for a whole archive generation (a week on the offsite tier). A **failure causes future
work** — a failing tier stays due and is retried at the next evaluation, so a lost failure heals
itself within one interval, while a *persisted* failure would outlive the fault it describes.

*What the report does with the two.* The collector merges them: **one entry per tier, newest by
`TestedAt` wins**. A fresh failure therefore beats a stored success (the failure is the news and
exists nowhere else), a stored success beats a stale in-memory entry after a restart, and a tier can
never appear twice — the hub would read that as two tests.

*It refuses to lie.* A persisted record missing the archive **or** the tier produces **no entry**: an
unproven tier reading as proven would be worse than the defect this closes. Run mechanics (scratch
VMID, duration) are not re-invented — an absent duration is not a claim, a fabricated one would be.
**Migration consequence, seen live:** a pre-R-189 record has no tier, so upgrading does not
retroactively make an old proof visible to the hub; the tier's next real proof fills it in.

**S-20 — the release order is build → tag LOCALLY → publish → push tag, and every step protects
something (2026-08-03, R-188 + R-186).**

The tag is created before the publish, so the build and the tag describe the same commit. It is
**pushed** after, because the push is what wakes CI (`on: [push]`) and a tag visible before its
package made `check-published-versions.py` correctly fail a *correct* release — measured on roughly
every second release, and R-168 mails those failures to the operator.

The invariant the old order protected is **asserted directly instead**: the gate now also refuses a
**published version with no tag**, as a bounded probe (frontier + patch gaps) that prints its own
coverage, because the package listing api is 401 without a token and absence cannot be enumerated.
A half-done release is loud: publish-then-failed-push dies naming the recovery command, and a failed
publish deletes the local-only tag so a retry is clean.

**A released binary is independently verifiable** — `-trimpath -buildvcs=false` means the same source
yields the same bytes with or without the tag; the verification command lives in
`felhom-agent/CLAUDE.md`. Both build paths (`release-agent.sh` and `publish-agent.sh`'s fallback) use
identical flags: they differed by `CGO_ENABLED=0` and produced binaries 74 KB apart for one version.

**S-17 — restore-testing is PER ARCHIVE GENERATION, and the hub's staleness window follows each
tier's own rhythm (2026-08-03, R-86; agent v0.121.0 + hub v0.91.0).**

*The rule.* Let **A** be the newest archive on a tier that has settled for at least the settle lag
(24 h). The tier is **DUE** when A exists and **A has not already been proven**. The daemon-start
ticker survives only as the **evaluation interval** (6 h). A daily tier is proved daily on yesterday's
archive; a weekly tier weekly on its own; a tier with no archive is UNKNOWN, never a fault.

*The trap, written down so it is not reintroduced.* The literal reading of R-86 — *"due when the
newest archive is ≥24 h old"* — is **never true on a daily tier**, because a new archive resets the
newest-archive age to zero long before it reaches the lag. It would have switched restore-testing off
for the tier that matters most, silently. Red-proved at 0 runs over 5 simulated days
(`felhom-agent/internal/backup/restoretest_due_test.go`).

*What the state holds now.* `RestoreTestState` records **which archive** was proven, not just when a
tier passed — a timestamp cannot answer *"have we proven THIS archive"*. A pre-R-86 file keeps its
time (rotation ordering survives a deploy) and yields no proven archive, so each tier is due exactly
once after the upgrade.

*The old config key.* `backup.restore_test_cadence_seconds` is DEPRECATED. **Negative still disables**
verbatim; a positive value now seeds the **settle lag** only, and the daemon WARNs once at start-up
naming `restore_test_eval_interval_seconds` (default 6 h) and `restore_test_settle_seconds`
(default 24 h). It is deliberately NOT carried into the evaluation interval.

*The hub half is not optional.* `restoreProvenStaleAfter` was a flat 7 days **derived from the cadence
R-86 removes**, and a healthy weekly tier's proof age reaches EXACTLY 168 h just before its next
proof — it sat ON the line. `restoreProvenWindow(tier, observed, ok)` now takes the tier's own
observed archive interval × 4 generations, floored at 7 days, capped at 12 days (strictly inside the
2-week offsite retention), falling back to the tier's **declared** rhythm (`backupStaleAfter` 26 h /
`offsiteBackupStaleAfter` 8 d — the backup-freshness checker's own thresholds) when history is too
short to observe one. Shipping Part 1 alone would have produced a nightly false alarm.

**S-18 — `ep0` is Tier 2, PROTECTED (operator ruling, 2026-08-03).** D-d named two protected machines
and did not name ep0 either way; `runbooks/target-selection.md` carried the question in writing for
two days. The ruling **extends D-d's protected list to three machines**: DooPlex, Peti's cluster,
**ep0**. It is a classification, not a new set of prohibitions — destroying datastores, prune jobs,
tunnel config or nftables rules was already forbidden by what it would destroy, and the ordinary
off-site READ a restore-test performs remains permitted.

**S-13 — the `mp1` merge landed, and the variant was chosen on measurement (2026-08-03, R-165 / D-a).**
The appliance's two data volumes are one. **Variant V-c**: the volume mounts at the NEUTRAL path
`/var/lib/felhom`, and both `/var/lib/docker` and `/mnt/sys_drive` are binds of subdirectories of it.
**Three shapes were built and rebooted before choosing** (`audits/SPIKE-r165-phase0-2026-08-03.md`) —
all three boot, reboot 3/3, give ONE `df` figure and keep a container's `statfs("/")` on the merged
volume, so **the ordering risk that motivated the probe was not what mattered.** They differ only in
which documented guarantee they break: volume-at-`/var/lib/docker` puts customer backups inside
Docker's data-root, so the ordinary "clear `/var/lib/docker`" reflex destroys every local unit;
volume-at-`/mnt/sys_drive` puts Docker's entire data-root under `/mnt`, which the controller container
mounts wholesale — **measured: it then sees `/mnt/sys_drive/docker`**, falsifying the bootstrap's own
comment that `/mnt` holds only Felhom's namespace mounts. **V-c breaks neither**, for one extra path.

**B2 is the bulkhead replacement, and "or prune the oldest" is REJECTED with its reason**, because the
question will be asked again: nothing on that filesystem is generational — a unit is ONE fixed path per
app (`backups/primary/<app>`) refreshed in place, and a DB dump is `<stack>-<dbtype>.sql`, also fixed —
so pruning could only mean deleting a **different** app's only local recovery unit.
`pruneStalePrimaryDirs` is an ORPHAN sweep with no notion of age and must never be repurposed.

**No migration exists, and that is a ruling not an omission:** every node is REINSTALLED. Both demo
boxes are Tier 0; the colleague's box carries none of our customer data and is clean-installed shortly.
So R-176's in-place migration rehearsal is **withdrawn**, not deferred.

**S-14 — prove first, then vouch (2026-08-03) — SPENT, and the ordering did not survive contact.** The
rule was: golden **0.192.0** stays UNVOUCHED until a box has been proven from it, because vouching is
what makes a fresh install pick a golden up. **In the event the golden was vouched at 07:23:26 CEST on
2026-08-03, before any box was reinstalled** (hub log `Artifact manifest set: agent=0.119.0
golden=0.192.0`), so the ordering was already spent when R-178's session opened; the operator elected
to accept it rather than revert the manifest. **Both boxes were then reinstalled and proven** (R-178,
`REPORT.md`), so the end state is the intended one and no unproven layout was ever in front of a real
install — but the rule protected nothing, because nothing enforced it. **The lesson is R-115's, one
layer up:** an ordering that lives only in a `CONTEXT.md` sentence and a runbook's §7 is a reminder,
and reminders do not hold. If prove-then-vouch is to be a rule it needs the shape R-120's gate has —
a refusal at `handleSetArtifacts`, the sole path to `SetArtifactManifest`, which runs without anyone
choosing to run it.

**S-15 — the merged layout is proven live, by two different supply paths (2026-08-03, R-178).** Both
demo boxes were wiped and reinstalled from golden 0.192.0 and taken through claim → deploy → back up →
**restore**. **demo-hp** was installed with `--golden <local volid>` (the layout proof) and
**demo-felhom** by the normal manifest route with `--force-gitea-golden` (the pipeline proof —
`verified sha256 54e2a4c431daf580… matches the hub manifest`), deliberately different so the session
proved the disk shape *and* the delivery route rather than one of them twice. Live shape on both:
`mp0` at `/var/lib/felhom`, `backup=1`, **no `mp1`**; `/var/lib/docker` and `/mnt/sys_drive` both real
mounts of its subdirectories via `/etc/fstab`; ONE `df` figure and one device id on all three paths;
3/3 reboots each with the binds surviving every time. B2 was **not** proven on that pass → **R-181**:
the floor guarded `captureAllRecoveryUnits` and not `runVolumeDumps`, the leg that fills the volume,
and its refusal's "the previous unit is untouched" was measured false. **R-181 CLOSED the same day
(controller v0.193.0 + v0.193.1), so R-165 is now PROVEN-LIVE in both halves** — see S-14.

**S-14 — the reserve is a per-app, per-run ADMISSION decision, not a capture check (2026-08-03, R-181;
controller v0.193.0 + v0.193.1).** B2 as first shipped was consulted in exactly one place —
`captureAllRecoveryUnits`, a few KB — while `RunDBDumps`' database leg and `runVolumeDumps` wrote the
bulk into the same `backups/primary/<app>` tree, first and unguarded. The reserve was therefore
consumed by the very write it exists to bound, and the refusal then claimed *"the previous unit is
untouched"* about a tree the earlier leg had already rewritten (182,272 B → 2,147,666,432 B under a
manifest that had not moved). **Sixth entry in `CLAUDE.md`'s table of shipped guarantees the code did
not provide, and the fourth of those found on live hardware rather than by review.**

- **`internal/backup/admission.go` — `admitApp` is now THE gate**, and every per-app write leg calls
  it. One verdict per app per run covers all three; they share one per-app root, which is what makes
  that honest.
- **Decided lazily at the app's first write, never once at run start** (app A's dump can put app B
  under the reserve), **never re-decided between an app's own legs** (that is the split it closes),
  and **reset per run**.
- **Ahead of `DumpAppVolumesSafe`**, which stops the stack as its first act — a refusal decided
  inside it has already bounced the app. **After** the volume-less check, which has no write to gate.
- **Size term added:** *would THIS app's write cross the reserve?*, estimated from the app's previous
  `.sql` + `.tar`. **No history → headroom-only**, deliberately — otherwise the first backup is the
  one that can never happen.
- **A container-based `du` was MEASURED and rejected**, not waved away: median **~355 ms/volume** over
  66 runs on demo-hp, on volumes holding tens of KB (container start-up, not the walk). Decisive on
  top: `docker run` needs the writable layer, so the instrument can fail under exactly the pressure
  the reserve handles.
- **The wording was NOT weakened; the behaviour moved so it became true**, and it is checked by
  sha256 tree fingerprint, never by reading the log line — the log line is what lied.
- **v0.193.1**, found by the proof run itself: a 178 KB estimate printed as `0.00 GiB`, which reads as
  *no estimate available*. Rendering moved to `humanizeBytes`; arithmetic still in GiB.
- **New finding, deliberately not fixed here → R-182**: `GetFullStatus`'s periodic capture sweep has
  no run scope, so a refused app re-alerts on every status refresh (measured: a second identical alert
  pair 13 s after the run's). Pre-existing in v0.192.0; R-181 changed neither caller.

**S-15 — publishing is an act, not a side-effect of pushing (2026-08-03, R-110 + R-115 + R-183).**
Two rulings, one shape: something became live because someone pushed, not because anyone decided.

- **The installer.** `/scripts/` now git-syncs the tag `installer-v<SCRIPT_VERSION>`; the **website
  keeps tracking `main`** in a second sync, because pinning both would make every copy edit a
  release. Publish = cut the next tag + bump the manifest `--ref` + sync. **Roll back = move the tag
  back**, which takes ~30 s and needs no ArgoCD sync at all — git-sync v4.4.0 follows a moved tag,
  and that half was measured before the manifest was touched because the whole model rests on it.
- **The sixteen run-time fetches were NOT what the spec described** — sixteen, not nine, and from
  `felhom-agent`, not this repo — so no tag here could cover them. They are pinned to
  `raw/tag/v$ART_AGENT_VER` instead, which is strictly better: the agent's configs now come from the
  same ref as the agent binary being installed. That closed a real skew (**R-183**), not just a
  channel.
- **The URL needed no change**, and that is worth knowing rather than re-deriving: it never carried
  a ref, so both producers follow the tag automatically — and no hub change means no hub bump.
- **The agent.** `scripts/release-agent.sh` is THE release path: build → tag → publish → **verify by
  an independent download**. It does not vouch. `check-published-versions.py` refuses a `v<semver>`
  tag with no downloadable package, and **CI now runs the full gate set** rather than `--fast`,
  without which that gate would have been registered and never run.
- **The gate's invariant is not the one specified, and P-C is why:** the hub manifest and Gitea's
  package listing are both **401** anonymously; the package download and the tags api are not. So CI
  can ask *is this installable* but not *what is vouched*. The residue is **R-184**.
- **Neither gate asserts "the newest version is published."** That would go red on the very push
  that bumps a version, before publishing — and a gate that fails on the normal path is one people
  learn to ignore.

**S-16 — a backup run NOTIFIES ONCE and RECORDS ALWAYS, and those are different things
(2026-08-03, R-182; controller v0.194.0 + hub v0.90.0/.1).** Measured: nine per-app capture failures
reached the hub, two were mailed, seven were dropped by a cooldown whose key carries no app
identifier — *before* `LogNotification`, so they left no row anywhere.

- **The record:** `recovery_unit_capture_failed`, per app, unconditionally, now routed
  **record-only** by the hub (`recordOnlyEvents`) — stored and logged every time, never competing
  for an e-mail slot.
- **The notification:** `backup_run_failures`, ONE per run, listing every failed app with its leg
  and reason plus the counts and free space. **A clean run emits nothing.**
- **A suppressed operator event now leaves a `suppressed` row** carrying the key that suppressed it —
  for every operator type. *"We chose not to e-mail you"* and *"nothing happened"* must never look
  identical; that is the whole finding, stated as a rule.
- **The periodic sweep gets a digest too, with NO `run_id`**, so it stays under the ordinary hourly
  cooldown. Without it the sweep's failures would be recorded and never notified — a new silence
  created while closing one. A real run's digest carries a unique `run_id` precisely so the cooldown
  can never collapse a manual run into the nightly one.
- **Why the silence is safe:** the hub's deadline check raises `expected_backup_missed` from report
  freshness, independently of any mail the box sends (`monitor/deadline.go:396,417`). **If that check
  is ever weakened, this design loses its footing.**
- **Not taken, and why:** putting `app` in the cooldown key. It fixes the swallowing by producing one
  mail per failing app — a dozen on a full disk.

**ep0 was rescaled by the operator to a CX33 (2026-08-03): 4 vCPU, 8 GB RAM, measured on the box, and
the 4 GiB swapfile survived. The 40 GB local disk is UNCHANGED** — a CPU/RAM resize only, so no disk
figure in any runbook needed correcting. That closed **R-90** and unblocked **R-86**.

**S-11 — D-c's routing, and why R-158's own proposal was overruled (2026-08-02, R-167 SHIPPED).**
Decision D-c splits two signals by AUDIENCE, and the split is the ruling: **a fill warning is the
CUSTOMER's** (they can free space, delete files, add a drive) and **a per-app backup capture failure
is the OPERATOR's** (they can do none of those things about it). R-158 proposed emitting the existing
`backup_failed` for the capture failure. **That was rejected and D-c wins**, because `backup_failed`
carries a `customerMessages` entry AND sits in `settings.DefaultEnabledEvents` — so reusing it emails
the customer, in Hungarian, that their backup failed, about something they cannot act on. It is
exactly the mistake R-97a avoided by minting `whole_guest_backup_failed`, and the reasoning is written
into `hub/internal/api/handler.go`'s allowlist. New type: `recovery_unit_capture_failed`, in
`allowedEventTypes` **and** `notify.operatorOnlyEvents` — **the second register is what makes it
operator-only; the first does not**, and v0.78.0 claimed otherwise and shipped the defect.

**The customer half reused the pair that already existed rather than minting a seventh type.**
`disk_warning`/`disk_critical` were allowlisted, carried Hungarian copy, sat in `DefaultEnabledEvents`
and had a UI checkbox — and **nothing in any repo emitted them**. A complete customer pipeline with no
producer: the **sixth** *built-but-never-wired* instance in this project. `internal/fillwatch` is now
that producer. Their generic `customerMessages` entries were **deleted**, because
`FormatCustomerEmail` PREFERS the entry over the message and a static template would discard the drive
label and the free-space figures — the same reason `offbox_enlarge_blocked` and `disk_health_degraded`
have none. `notify.IsOperatorOnly` was added so ONE test pins both registers; checked separately, an
allowlisted-but-not-operator-only type is invisible.

**S-12 — the monitoring landed BEFORE the merge, not with it (2026-08-02).**
D-a's condition (2) says R-167 ships in the same step as the `mp1`→`mp0` merge and never after,
because the merge removes a wall that currently fails safely. **This session landed it FIRST**, which
is strictly better and costs nothing: the warnings went in and were proven on real hardware while the
wall is still standing, so the merge session inherits a proven signal instead of an untested one.
**No disk layout was touched.** R-165's measurement is `audits/SPIKE-r165-mp1-merge-2026-08-02.md`,
which STOPS at a question for the operator (which merge shape; what replaces the bulkhead). Its two
load-bearing findings for anyone picking that up: **"the layout" is not one thing** (demo-felhom
`200G/50G`, demo-hp `50G/20G`, golden `16G/8G` — so §7.5's bound is one box's, → R-175), and **`mp1`
is also a BULKHEAD**, not only a ceiling — today an overflow cannot reach `/var/lib/docker`, and after
the merge it can.

**S-8 — CI detects; it does not block, and that is structural (2026-08-02, R-168).**
A Gitea Actions runner in `gitea-system` re-runs every repo's gate entry point on every push,
independent of who pushed and of what they typed. It **cannot refuse a push**: every felhom repo
pushes straight to `main` with no pull request, so there is no merge for a status check to stand at.
That is not a gap in the runner — there is no gate in the road. The refusing half is
`.githooks/pre-push` (per-clone, `--no-verify`-able); this half is what notices when that hook was
skipped or was never armed. Making CI blocking needs branch protection plus a PR workflow, which
changes how the operator works and is **their** call → R-169. Do not "fix" this by adding branch
protection.

**S-9 — a detector that tells no one is not finished (2026-08-02, R-168 probe P5).**
Probe P5 measured that a failed run produces **no mail, no notification row and no log line** from
Gitea. So the workflow sends its own alarm on the project's existing Resend path and **prints the
provider's accepted id**, which makes "a message left the machine" an observable rather than an
assumption. The acceptance test for this work was never a green pipeline — it was a red run with a
message in hand (`RESEND-ACCEPTED id=…`). Two traps found while building it, both worth keeping:
the runner image has **no `curl`** (deliberately — python3 and git only, so use `urllib`), and
`api.resend.com` sits behind **Cloudflare, which 403s the default `Python-urllib` User-Agent with
error 1010** — a failure that looks exactly like an auth failure and is not one.

**S-10 — the runner is unprivileged, and the reason is the host (2026-08-02).**
The usual `act_runner` recipe pairs it with a `docker:dind` sidecar and `privileged: true`. Rejected:
DooPlex is **Tier 2** and *is* the recovery chain — Gitea, the hub, the registry, PBS and
k3s + Longhorn all live on it and it cannot be rebuilt from anything else. Host execution mode costs
nothing here because every CI job is one Python command. Consequence to remember: **in host mode the
job sees exactly the runner image's tools**, which is why `python3` had to be baked in (probe P2:
stock `act_runner` carries git but not python3). If a future job genuinely needs Docker, that is a
conversation, not a patch.

**S-11 — CI reproduces the workspace's sibling layout, because two entry points depend on it
(2026-08-02).** `controller_gates.py` and `agent_gates.py` invoke the shared `reuse_refs_check.py`
that lives in the `felhom.eu` clone next door and is deliberately never copied, and both repos'
`REUSE.md` files cite a path that lives in the hub. Their workflows therefore clone `felhom.eu` as a
sibling; without it the gate fails **closed** — correctly, but for the wrong reason. Verified that CI
and the local hook then agree exactly (controller 126 exact / 6 suffix / 1 cross-repo).

**S-6 — the hub renders no host-install version, and the gate pins its absence (2026-08-02, R-94).**
The Setup tab's *"host-install 1.19.0"* label is **deleted, not derived**. Deriving it is not
achievable honestly: the Option-1 command downloads `felhom-host-install.sh` from the website **at
run time**, and the website git-syncs `main` every 30 seconds (R-110) — so the hub cannot know which
version a given box will run, at build time or at render time, and any literal there is a guess
wearing a version number's authority. The real one drifted to 1.19.0-vs-1.22.0 and stayed wrong for
nineteen days. `hostInstallVersion`, `pageData.ScriptVersion` and the rendered label are gone; a NOTE
sits where the const was so it is not helpfully re-added, and `scripts/hostinstall_gates.py` gate 1
**inverted** — it now asserts the hub carries no host-install version literal in any of six code
shapes across every `.go`/`.html` under `hub/`. Corollary that generalises past this row: the
tautological `render_test.go` assertion (`html contains hostInstallVersion`, where the same constant
put it there) **passed at `9.9.9`** — an assertion that compares a value to itself tests the
plumbing, never the claim.

**S-7 — gates run from ONE entry point per repo, and `reuse_refs_check` was fixed rather than the
convention it polices (2026-08-02, R-29).** Two rulings from the same census.

*Where gates run.* Thirteen gate scripts exist across the four repos. Measured 2026-08-02: **every
check a `CLAUDE.md` tells a person to run was passing, and two of the four nobody is told to run were
failing** — one since 14 July. The correlation was exact, so the fix is not more gates but one place
to run them from: `scripts/repo_gates.py`, `felhom-controller/controller/scripts/controller_gates.py`,
`felhom-agent/scripts/agent_gates.py`, `app-catalog-felhom.eu/scripts/catalog_gates.py` — each
mandated in its `CLAUDE.md`, each wired to `.githooks/pre-push` with `--fast`. The canonical shape is
`catalog_gates.py` (R-161), **not** `site_gates.py`, which is a gate (eight assertions in one file)
and not a runner; copying it produces another monolith nobody invokes. **A missing gate script is a
FAILURE with the path printed, never a skip.** The hook's limits are real and are written into the
hook: per-clone (`core.hooksPath` is local config) and `--no-verify`-able on purpose. The
unbypassable half is CI → **R-168**.

*Why the checker moved and the docs did not.* `reuse_refs_check.py` was RED on all four repos with
13 findings, of which a hand audit found **zero** genuine drift — twelve were package shorthand
(`appbackup/userdata.go` → `controller/internal/appbackup/userdata.go`) and one, `wgsync/reconciler.go`,
is cited by the controller and lives in the hub. `REUSE.md` cites by package shorthand and across
repos deliberately; that convention is the useful one. **Rejected, so they are not revisited:**
rewriting all four `REUSE.md` files to full paths (makes the docs worse to serve the tool), and
deleting the checker (drift across four repos is a live risk). The checker now resolves
exact → suffix → ambiguous → sibling repo → FAIL, **prints every non-exact hit and a per-rule tally**
(because "0 failures" alone cannot tell a working checker from a blind one), and lists every
resolution attempted on a failure. It stays in **one** place and is invoked across the workspace —
never copied, which would recreate the drift it detects.

**S-1 — N.5 gains a third leg: architecture docs are same-session coupled (2026-07-26, R-81).**
Any task that changes an **architectural contract** — tiers, targets, cadences, trust boundaries —
updates the owning `documentation/architecture/*.md` in the **same session**, under exactly the same
coupling rule that already binds the capability map and the ROADMAP. Origin: R-80/R-81 found
`07-backup-architecture.md` still describing a backup model that the shipped code does not implement
(single target, single cadence), while being cited as authoritative. A stale architecture doc is
worse than a missing one, because it is trusted.

**S-2 — architecture docs carry an honest status header (2026-07-26, R-81).**
Every `documentation/architecture/*.md` opens with the version it was **verified against** and the
date. A doc more than a few trains behind its subject is marked **STALE** *in that header*, so a
reader meets the warning before the content, not after acting on it. Origin:
`07-backup-architecture.md` — DRAFT since 2026-07-14, verified against controller v0.132.0, now 41
versions stale (live v0.173.0), and cited as authoritative throughout the R-80 diagnostic. Ratifying
or retiring it is → **R-83**.

**S-3 — the recovery model: six decisions, 2026-07-28.** Taken in an architecture discussion and
expressed in the `07-backup-architecture.md` full rewrite (which replaces the 2026-07-14 DRAFT
entirely — that doc was verified against controller v0.132.0, **51 versions stale**, while being
cited as authoritative). They are **decisions, not observations**; the rewrite labels every statement
`[DESIGN]` or `[FACT]` so a reader can tell which is which.

- **D1 — two lanes, different owners.** *Customer, unassisted:* files and app data, through the
  „Visszaállítás" surfaces. *Operator:* LXC guest and host recovery. This is a deliberate product
  decision, not a limitation to apologise for.
- **D2 — encryption follows the boundary, not the tier.** Data on the customer's own drives is
  **plaintext**, deliberately: someone who can take the second drive can take the first, local
  encryption adds a key-loss path that turns a working backup into a brick, and it would break
  FileBrowser and SMB browsing, which is a feature. Data **leaving the premises** is encrypted
  (restic, PBS) — the provider must not be able to read it.
- **D3 — recovery is a chain, not a set of tiers.** Every app-tier restore is conditioned on the
  whole-guest tier: Tier-1 needs the guest's secrets (`restore_unit.go:17-22`, *"Regenerate
  NOTHING"*), and Tier-2/Tier-3 need the app still deployed. **3-2-1 describes copies; it does not
  describe recovery.** This is the reason the rewrite exists.
- **D4 — three-part model.** *Recipe* (scaffolding, hub) · *Escrow* (identity key + restic repo
  password, hub, R-wrapped) · *Bytes* (the tiers). Losing one is a different problem from losing
  another, and the failure→recovery matrix is organised around that.
- **D5 — R's role narrows. CONDITIONAL, and currently BLOCKED.** The target is app secrets travelling
  with the LOCAL recovery unit, so Tier-1/Tier-2 restore work without the guest and without R,
  leaving R needed for offsite recovery and host identity only. **Not adopted:** the premise
  (the backup tree unreachable from every browsing/download surface) was tested and failed — see
  **R-108**. Every other surface verified clean; the exposure is FileBrowser's root-bind of network
  shares combined with the absence of an `IsNetwork()` filter on the deploy and per-app-migrate
  paths.
- **D6 — the trust model, stated honestly.** The operator holds root SSH on every box; *"the
  operator cannot read customer data"* was never the actual security property and no document may
  imply it. What R **does** provide, and must keep providing: **the hub alone is not enough** — a
  compromised hub yields blobs nobody can open, *provided the operator's key is never stored in the
  hub*. That proviso is why escrow custody is an open decision (`07` §11-A).

**S-4 — the hub session password alone now unlocks console root on every managed box (2026-07-31, hub v0.84.0).**
Retrieving a host's vaulted break-glass `root@pam` credential previously required the **global operator
API key**, a secret distinct from the hub login and kept out-of-band. The `Console access` card on the
host page puts that retrieval behind the **hub session** (`POST /hosts/{id}/reveal-recovery-credential`,
CSRF-gated, audited), so the hub login password is now sufficient. **Accepted deliberately**, and the
reasoning is the ruling: the hub is single-operator and geo-fenced to Hungary, and it already stores
these passwords **in plaintext at rest** — so the DB and the login were already jointly sufficient, and
this is a convenience trade rather than a new exposure. It is a real trade, not a free one, and it is
recorded here rather than only in a changelog. Two things follow. **(a)** The global-key path
(`api/handler.go handleAdminGetRecoveryCredential`) stays untouched and must acquire **no** dependency
on the session layer — it is the route for when the hub UI itself is what is broken, and coupling the
two would delete the independence that makes it a fallback. **(b)** The plaintext-at-rest half is now
the named open item **R-133**: every hub DB backup is a fleet-wide console-credential dump, and the fix
shape is envelope encryption under a KEK held outside the DB, the way escrow blobs are already opaque.

Five decisions were deliberately **left open for the operator** and are recorded, unanswered, in
`07` §11: escrow custody · lost-R policy · per-scenario RTO/RPO targets (**none have ever been
stated**) · Hetzner as a single failure domain · and `local` vzdump sharing a physical device with
the guest it backs up. Gaps minted the same session: **R-102 … R-108**.

**S-13 — boot recovery finished, and the lesson is about the DIAGNOSIS ORDER (controller v0.190.0,
2026-08-02, R-157 A · R-170 · R-171).**

**The session's most valuable half hour was spent NOT writing code.** A hole was reasoned out of the
v0.189.0 diff — replacing the container-count term with recorded intent should make a
drive-gate-stopped app read as a boot orphan — and the task's own rule was to CONFIRM it on hardware
before writing a fix. **The first attempt to confirm it produced a false negative**, and reporting
that as a disproof would have been wrong: unmounting only the parent bind is healed by the agent
within ~60 s, so the drive gate's startup reconcile restarted the apps **one second before** the
sweep looked. `no boot-orphaned apps` in that log is a race that went the safe way, not a mechanism.
Holding the drive genuinely absent reproduced it immediately. **"It didn't happen this time" is not a
disproof — name the mechanism or run it again.**

**The confirmation also changed the severity, in both directions.** The write hazard did NOT
materialise: compose failed `mkdir …/userdata: permission denied`, because the unbound mountpoint is
host-root-owned and the guest is unprivileged. **That protection is accidental** — no code chose it,
no test pins it, it is one `chown` or one privileged guest away from gone, and its removal would be
invisible until data landed on the wrong disk. Meanwhile the harm that DID occur was real on every
box and was not in the hypothesis: two wasted attempts and a **false dead-app alarm for an app the
drive gate is deliberately holding**. Diagnosing first is what produced both facts.

**The fix was already in the codebase, on another path.** The API's `startGatedByMissingDrive`
refuses a customer's start on an absent drive with a Hungarian message. The sweep bypassed it by
calling `Manager.StartStack` directly. **`StartStack` has no gate of its own** — that is the durable
fact worth carrying: every caller that is not the customer must decide for itself whether the app may
run, and there are now fourteen of them.

**Widening a window makes previously-unreachable overlaps reachable, and that is a design input, not
an afterthought.** The old T+5 s sweep never met a quiesce or an in-flight app-data operation; a
50 s window can. All three holders answer one seam rather than three, because they differ only in
the reason string.

**A test rejected my first constant, and the comment now says so.** `settle + budget + one retry`
must fit inside `deadAppBootGrace` or a successful recovery stops being silent; 60 s gave 95 s
against a 90 s grace. The budget is 50 s **because a test said so**, and the code records that rather
than presenting the number as taste. Widening the grace to fit was rejected outright: it hides a late
recovery instead of reporting one.

**AND THE FIX HAD ITS OWN DEFECT, FOUND BY LIVE VALIDATION AND NOT BY REVIEW.** The window sampled
`GetStacks()` — the Manager's in-memory map, refreshed by the scheduler every 10 s — every 5 s. Two
identical samples could therefore mean *the cache did not update*, not that the fleet had settled. It
surfaced as a container removed ~5 s before the window closed still being in the sampled fleet, with
the sweep logging `no boot-orphaned apps` for an app that had none. **Generalise it: a settle
detector is only as good as the freshness of what it samples — if the source is cached, refresh it or
you are watching the cache settle, not the system.**

**Live: 6/6 hard resets on the shipped build** (every app back; a customer-stopped app down in all
six), window settle times 10/40/10/10/15/15 s — routinely 2–8× the old fixed 5 s. The sharpest
evidence is a same-app before/after on one box: missed at 18:08:35, recovered at 18:18:50.

**S-12 — D-b is BUILT (controller v0.189.0, 2026-08-02, R-166).** The desired/in-flight/observed
split now exists; the S-1 contract lives in `architecture/02-controller-module-map.md` §0a.

**Both facts D-b said to establish first were established at source, and both changed the shape.**
(a) *Does the crash-safe journal in the backup code already cover the in-flight case?* The pattern
DID already exist — twice (`quiesce` marker+`Recover`, `migrate` journal+`RecoverMigration`) — and
covered **none** of the app-data path: `DumpAppVolumesSafe` stopped and restarted an app with no
marker, no journal and **not even a `defer`**. So the answer was neither "it exists, wire it" nor
"build it": copy the proven shape into its own file. It was the fifth time the question was worth
asking and the first time the answer was "the pattern, not the coverage". (b) *Is the SQLite store
reachable?* Reachable and **deliberately not used** — `metrics.db` is optional by design (the
controller runs with it absent), and operational state must not live in a store built to be dropped.

**The ruling that carried the design: absent means UNKNOWN, never "running".** Every `app.yaml` on
every box predates the field, so absent is what the whole fleet reads on upgrade; reading it as
running would have started every deliberately-stopped app on the first boot after the upgrade —
fleet-wide, silently. Where intent is unknown the box keeps the OLD inference rather than inventing
an answer. That is also why the backfill is **running-only**: "zero containers ⇒ stopped" is the
defect itself, so an ambiguous app stays ambiguous until a customer presses a button.

**The other load-bearing ruling: `StartStack`/`StopStack` are NOT writers of intent.** A census found
14 callers, of which exactly 2 are the customer. Recording intent in the primitive would make a
nightly backup indistinguishable from the customer pressing Stop — the confusion being removed.

**Found on the way, and it would have silently eaten the feature: `SaveAppConfig` rebuilt `AppConfig`
field-by-field.** That is the R-100 shape, which v0.181.0 shipped two live instances of. The literal
named five fields, so the sixth would have been dropped on every save across nine call sites — a
customer's Stop erased by the next unrelated `app.yaml` write. Copy-and-overlay is safe by
construction; the failure mode is generic, so **treat any field-by-field struct rebuild in a save
path as a defect on sight.** Measured and documented rather than assumed: `app.yaml` does NOT
round-trip YAML keys the struct does not model.

**Closes R-157 mechanism B; mechanism A (the sweep observes ~5 s after start and never re-checks) is
untouched and is now the whole of R-157** — and B's fix makes A cost more, since the sweep now has
more it could legitimately recover. **New: R-170** — `shouldRecreateOnBoot`
(`internal/web/intermediary.go:131`) still infers a Stop from `hasContainers`, i.e. the same defect
one gate over for drive-backed apps. Left deliberately: the task scoped `bootrecon`, and two boot
behaviour changes under one live validation is one too many.

**Live on 9201, three flows, each with a positive observable.** The interrupted-operation half is
**IMPLEMENTED, not PROVEN-LIVE** — unit-proven and red-proofed, but nobody killed the controller
mid-backup on real hardware; the capability map says so rather than rounding it up.

**S-5 — four operator decisions taken in discussion on 2026-08-02, recorded before anything is
built.** They existed only in conversation, which is the condition the standing rules were written
against. Labels are the ones used in the discussion (**D-a … D-d**) and are deliberately kept
distinct from S-3's `D1…D6`. **None of a, b or c is implemented by the session that recorded them**;
their work is carried as backlog rows (**R-165, R-166, R-167**), not as prose here.

- **D-a — the dedicated backup partition is MERGED AWAY, not resized.** The appliance's system drive
  carries the OS, a **50 G** docker area (`mp0`) and a separate **20 G** area holding local recovery
  units (`mp1`; demo-hp's guest 9201 ships exactly that). The decision is to **merge the backup area
  into the large data area** so local backups share space with app data and the ceiling stops
  existing — deliberately *not* to pick a larger number, because a larger number is the same wall
  further away. **Two conditions, recorded with the decision, not as commentary:** (1) it changes the
  **disk layout**, so it must land **before any external install** — after that, every deployed box
  is a migration; (2) it **removes a wall that currently fails safely**, so the storage monitoring of
  **D-c** must ship **with** it, never after. Closes the sizing question in **R-163**, which is
  **re-framed rather than answered with a ratio** and is not closed until the merge lands. Work:
  **R-165**.
- **D-b — app state gets a desired/observed model with its own store.** Three kinds of information,
  deliberately separated: **desired** (running, or stopped because the customer said so) stays in
  `app.yaml` with that app's other settings; **in-flight operations** ("a backup started and has not
  reported finishing") are **persisted**, written before the operation starts and after it ends, so
  an interruption is visible after a reboot; **observed** (unhealthy, restarting, drive gone) is
  **not** persisted and is rebuilt by looking. The last two live in a **state store separate from
  `app.yaml`**, so frequent writes never touch a file holding deploy settings and secrets.
  **Binding safety rule, verbatim:** *losing the state store must never cause an app to be deleted,
  restarted wrongly, or reported healthy when it is not — the worst acceptable outcome is re-running
  a backup that already ran.* The controller must also check **every container of an app**, so a
  partly-dead app cannot read as healthy (F-CRIT-1's shape). **Two things to establish before this is
  specced, not now:** whether the crash-safe journal already in the backup code covers the in-flight
  case — *it may already exist and merely be unwired, which has been the answer four times* — and
  whether the existing SQLite store is reachable from this code path. Work: **R-166**.
- **D-c — storage monitoring and backup alerts.** Two pieces: a warning to the **customer** as a
  drive fills, **before** anything fails; and a failure alert to the **hub/operator** when any backup
  fails for any reason, lack of space included. **Prerequisite for D-a** — see that decision's
  condition (2). Subsumes and widens **R-158**, which is the same alert one tier down; R-158 stays as
  the named seam and is not duplicated. Work: **R-167**.
- **D-d — which machines are protected: DooPlex and Peti's box, and nothing else.** DooPlex because
  it holds Gitea, the hub, the backups and the registry — everything else rebuilds from it. Peti's
  box because there is a real person behind it. **Every other box, both demo boxes included, is
  disposable and may be broken or reinstalled freely.** The earlier caution was costing sessions and
  pushing work onto DooPlex — the one machine that should never host it — so the ruling is a
  correction, not a relaxation. `documentation/runbooks/target-selection.md` updated the same session
  (2026-08-02); no backlog row, the decision *is* the change.

- **2026-07-27 — R-85: the DR tier is restore-tested UNATTENDED, and its failure is HEARD.** agent
  v0.104.0 + hub v0.77.0. R-82 proved a PBS restore round-trip **by hand**; three defects kept the
  unattended path from covering it. (a) `buildRestoreTestScheduler` built ONE runner on
  `cfg.Backup.BackupTarget()`, so the offsite tier's archives were never candidates — the DR tier
  could not be restore-tested unattended AT ALL. (b) the `Spec` was an immediately-invoked function
  evaluated once at daemon start, so nothing tier-varying was expressible and any config change
  stayed stale until restart. (c) a failed restore-test was a `[WARN]` line with **no event, no
  notification, no gauge** — true for the LOCAL tier that was already being tested, so the loudest DR
  signal this system produces was inaudible. **(c) was the one that mattered:** rotating tiers
  without it would only mean two tiers can fail silently instead of one.
  **Ruling (operator, 2026-07-26): oldest-first (Option 1)** — never-proven sorts first, no cadence
  knob; the per-tier last-proven timestamp is reported so the achieved interval can be **measured**
  before anyone decides a knob is needed.
  **Shipped:** per-run `SpecBuilder`; `PickRestoreCandidateOn` (a tier with no archive NO-OPS);
  persisted `RestoreTestState` (survives restart; credit only on SUCCESS); `backup.InFlight`, the
  host-wide one-heavy-op gate shared with the backup path (a LINK concern — an offsite restore pulls
  multi-GB over the tunnel a backup pushes one); hub `restore_test_failed` / `restore_test_stale`,
  kept DISTINCT and anchored on R-81, operator-tier only.
  **Also fixed a time bomb I planted in Slice C:** a deadline test hard-coded the incident timestamp
  `2026-07-18T18:31:06Z` while comparing against the REAL clock — harmless under one 26h threshold,
  but once the offsite tier got an 8-day limit it passed all day and began failing at exactly 18:31
  UTC, 8 days later. A test that passes at commit time and fails hours later is worse than one that
  fails immediately.
  **NOT yet proven:** rotation has not been observed selecting both tiers across consecutive
  UNATTENDED cadences — a multi-day window at a 24h cadence. Manual restore-proof IS proven (R-82).
  Spec: `documentation/backlog/SPEC-r85-phase4-5-2026-07-26.md`.

- **2026-07-26 — R-82 SHIPPED: the backup target split (local daily + offsite weekly).** agent
  v0.97→v0.102.0, controller v0.174→v0.175.0, hub v0.76.0, host-install 1.20.0. `BackupTarget()`
  returned ONE string and `BackupCadence()` ONE 24h window, so "local daily AND PBS weekly" was not
  expressible — which is why the DR tier was `applied` since 07-21 with **one** snapshot on
  demo-felhom and **zero, ever** on demo-hp. Now: `backup_targets[]` per-tier cadence+retention;
  ONE quiesce window for both due tiers (never two app outages for one night); per-tier hub
  thresholds (host 26h / offsite 8d); fresh-install default; an unprovisioned tier DEFERS.
  **Operator rulings:** 2-week offsite retention, first backup runs as long as it needs, one backup
  at a time per guest, drill box dropped from the rollout.
  **RETENTION IS A COMMERCIAL ATTRIBUTE — the hub decides, ep0 executes (operator ruling 2026-07-27,
  R-89).** A paid tier may buy longer retention, so the policy belongs with customer config on the
  hub, never in ep0's PBS config and never in a box's config. Execution stays server-side: a
  reconciler writes a **PBS prune job** and PBS's own scheduler runs it, so hub downtime leaves the
  last-known policy running rather than silently stopping retention. **Box tokens stay write-only
  (`DatastoreBackup`) — never widen a grant to fix a prune error:** a compromised box must not be
  able to delete its own offsite backups, which is the scenario offsite DR exists to survive.
  Increment 1 shipped 2026-07-27 (boxes stop attempting prune via `keep_last: 0`; per-namespace prune
  jobs on ep0, daily 03:30 UTC) — `runbooks/RUNBOOK-pbs-prune-serverside-2026-07-27.md`. This closed a
  live false-negative: **every** demo-hp PBS backup since 07-26 reported `job errors` while the data
  landed correctly, because `DatastoreBackup` carries no `Datastore.Prune`.
  **Four defects found by RUNNING it, not reviewing it** — a 30-min wait bound against a 41-min
  backup (the agent recorded `success:false` while the backup was still going); the restore tier read
  from the configured target instead of the archive (**a silent regression of the S4.1 fix** — the
  mechanism was never removed, its INPUT changed when `local_backup_target` was retargeted to
  `local`); a leaked scratch keeping `onboot:1`; and a tier firing at a not-yet-provisioned storage.
  **A correction I had to make mid-arc:** I reported that the restore-test would boot a
  network-conflicting clone and break the control plane. It does not — step 2 link-downs every NIC
  before boot, and it is unit-tested. I read a config artifact and inferred behaviour without reading
  the code path that consumes it, then escalated before finishing the check.
  **NOT done:** the offsite tier is never AUTOMATICALLY restore-tested (the scheduled test only ever
  selects the primary tier); the hub infers "PBS ⇒ weekly" from storage TYPE rather than a reported
  cadence. The installer-default fleet flip waits on a full weekly cycle.
  **R-84 SHIPPED (agent v0.103.0)** — the cold in-memory store no longer causes a redundant backup:
  the due-check asks the STORAGE (ground truth) instead of persisting state, so a pruned archive
  correctly stops counting. Proven live on both boxes with the store cold. **demo-hp's FIRST EVER
  offsite backup landed** (4.25 GB) — the R-82 finding closed on the box where it was worst.
  Controller v0.175.0 deployed to both boxes. Evidence: `audits/SPIKE-r82-phase0-2026-07-26.md`, `REPORT.md`.

- **2026-07-26 — hub v0.75.0: R-81 SHIPPED — "no signal" is not "bad signal".** Third instance of
  one class (hub v0.12.0 `expected_backup_missed` daily-for-everyone; hub v0.73.0 `offsite_stale`
  minutes after a healthy repair; now this), fixed **as a class**, with the invariant written at the
  head of `assessBackupFreshness` naming all three. `assessBackupFreshness` returns
  OK/UNKNOWN/MISSED instead of `missed bool`; absence is UNKNOWN until it outlives an anchored
  window. **The anchor was free** — Phase-0 probe found the hub retains 90 days of host-reports, so
  `GetHostReportsSince` + `newestBackupEvidence` (7-day bounded lookback, early-exit on fresh
  evidence) answer *"when did I last SEE evidence of a backup?"* rather than *"what does the latest
  report say?"*; absence is graded from first contact (`GetFirstHostReportAt`) reusing the existing
  26 h threshold as the grace — **no new knob, no agent change, no new persisted state**.
  **Rulings:** absence is never silent (a genuinely dead box must still alarm — that is the half the
  naive fix breaks, and `TestBackupFreshness_NoEvidenceBeyondAnchor_Alarms` is what makes the
  suppression safe); reason strings stay distinct (the whole 07-26 diagnosis turned on reading the
  exact string); deferrals are logged so quiet ≠ not-checked. Tests 493→508; red-proofs A/B/C
  observed — A reproduces `newest backup is 176h0m0s old` **verbatim the mail demo-felhom sent**.
  **Replayed against the real 03:00 reports** (600/417/77 rows): all three now silent. `backupStaleAfter`
  deliberately untouched — ⚠️ it will alarm on a healthy WEEKLY PBS snapshot, landmine recorded,
  owned by **R-82**. The real finding behind R-80 stands: the **PBS/offsite-DR tier has no schedule
  at all** → R-82. Cause-fix (persist the agent store) → R-84. Evidence:
  `documentation/audits/DIAG-backup-missed-2026-07-26.md`, `hub/CHANGELOG.md` v0.75.0.

- **2026-07-23 — hub v0.72.0 + controller v0.161.0: R-70 SHIPPED + R-71(c) PARTIAL (the offsite
  last mile is visible; burned credentials self-heal).** One detector
  (`offsite.DeliveryStateFor` — secret-row timestamps × report offsite-presence), four consumers:
  operator card (static "delivered once" copy DELETED; state+age, amber past 30 min, stale-staged
  info line), `offsite_delivery_stuck` warning (≥1 h, 24 h durable cooldown via `LastEventAt` on
  the events table — restart-proof), R-71c self-heal through the EXISTING Re-issue path
  (`monitor.OffsiteReissuer` ← webServer, armed only with a provisioner; consumed ≥1 h + ≥4
  offbox-less reports + zero offbox evidence; one restage/24 h; every firing = a warning event),
  and the controller's truthful empty-state banner (v0.161.0, both boxes). **Rulings:** applied
  wins (stale-staged is a flag, never a downgrade); cooldowns durable via events table; both new
  events operator-only (no customerMessages, not in allowedEventTypes — pbsdr_* precedent);
  **R-39(a) guard in the CALLER** — the heal re-reads the secret row at act time and refuses over
  an unconsumed row; the store's clobber semantics stay (Re-issue depends on supersede). Clobber
  red-proof observed (`reissue calls = 1` + row clobbered with the guard removed). Self-heal +
  banner ship unit-proven, NOT live-fired (no broken box; F9 rule) — arms on the next natural
  occurrence. Live: checker silent on the healthy fleet; demo-felhom's stale-staged specimen
  intact (its cleanup is the operator's, post-validation). R-71(a) day-0 ordering OPEN → next
  spec. Evidence: `REPORT.md` (2026-07-23).

- **2026-07-23 — ISO train v1.25.0 + hub v0.73.0 (four rulings + rider).** host-install
  1.17.0→1.18.0, build-felhom-iso 1.24.0→1.25.0, felhom-bootstrap R-63, hub v0.73.0. **Done +
  live-verified:** (1) **F9 belt everywhere** — host-install installs the H1 OOB belt by default in
  appliance mode (`--no-oob` opts out; byo still refuses), `oob.enabled` written so the agent
  activates it; belt LIVE on both boxes, **login PROVEN on felhom-pve** (`felhom-op@demo-felhom`);
  the orphaned operator identity was re-anchored to the operator's real machine (WG peer + SSH key
  re-registered, ep0 forward rules persisted). (2) **apt** — `align_apt_repos` (appliance-only)
  switches enterprise→no-subscription; LIVE-aligned both boxes (`apt-get update` clean). (3) **R-63**
  — Lat2 console font before first paint. (4) **R-71 build-gate** — `assert_golden_ge_floor` in
  build-felhom-iso + rule 5; red-proofed. (6) **root vault** — felhom-pve root rotated+vaulted+
  verified over :22 (G1 parity). (7) **hub v0.73.0** — `offsite_stale` never-ran branch ANCHORED
  (applied-only + consumed_at/escrow anchor; one-state-one-owner with `offsite_delivery_stuck`),
  shipped+deployed, red-proofed. **CRITICAL live finding:** golden **0.153.0 < floor 0.156.0** in
  production now — the R-71 gap is real; the gate correctly blocks a v1.25.0 build. golden republished at 0.161.0 + **vouched** (gap CLOSED), v1.25.0 ISO **built**
  (gate live-passed). **Nested drill aborted on the N100** (install crawled — the N100 was
  contending with live guest 9201; KVM-confirmed, not memory) → cleaned up (9201 untouched);
  **operator steer: future drills on the HP**. R-63 font code-verified; the live ő/ű console
  capture rides the next HP drill (belt/apt already proven live on both boxes; no-floor-update is
  structurally guaranteed post-vouch). Evidence: `REPORT.md`, `scripts/CHANGELOG.md`,
  `operations/nodes.md`.

- **2026-07-23 — F10 offsite leg RESOLVED on demo-hp (diagnostic + designed-path repair + tier
  proof; no code changed).** The hub said "Provisioned" while the box said „Még nincs beállítva" —
  root cause: the day-0 managed update (0.153.0→0.156.0, 07-21 16:28) killed the offsite
  apply-bridge ~35 s after it consumed the one-time password, before key-install/persist; the box
  then hit the consume-404 dead-end at every start, silently, for 2 days. Repair = operator
  **Re-issue** click (converged in 8 s), escrow ceremony via the real wizard (auto-confirm 3 s),
  then the full proof: paperless-ngx toggled into scope, first offsite run (repo initialized on
  sub3, 79.8 MB, snapshots=1), probe deleted → `mode=full` restore (snapshot `2bf7f2e1`) → place
  merged 1 file missing-only → **md5-identical**. Minted **R-70** (last-mile invisible on both
  surfaces; hub never reads `one_time_secrets.consumed_at`) and **R-71 [P1]** (the race recurs on
  every fresh onboarding whose ISO floor lags the managed floor). Record:
  `documentation/audits/DIAG-f10-demo-hp-offsite-2026-07-23.md`. F10's PBS-DR half stays open
  (F13 + ceremony R-moment).

- **2026-07-22 — hub v0.71.0: the notification train (audit F11+F12+F14-light).** Four rulings now
  standing: **(1) recovery pairing** — `*_recovered` notifies the operator always and the customer
  **iff the customer was mailed the paired stale/down** (evidence = `notification_log` customer-
  channel sent rows via `store.LastCustomerSentAt`; `enabled_events` deliberately ignored for
  recovery; ties → no mail). Severity semantics FROZEN — recovery stays `info`, routed by an
  explicit eventType branch before the severity gate. **(2) seed-at-claim** — `MarkClaimed` seeds
  `customer_notifications` from the registered email, INSERT-if-absent only (never touches an
  edited row; empty email = no-op; seed failure never fails the claim); default set is
  critical-only: node_down, backup_failed, disk_critical, host_disk_critical,
  storage_fill_critical, offbox_repo_orphaned. **(3) empty-email no-clobber** — a prefs push with
  empty email preserves a stored address (hub-side belt; controller 0.160.0 already guards its two
  push legs). **(4) priority headers** — error/critical mails carry `X-Priority: 1` +
  `Importance: high` (Resend `headers`, live-probed); the `test` event now also mails the operator
  with those headers (one click proves both channels + rendering). Latent nil-deref fixed:
  `sendTestEmail` panicked on a customer with no prefs row. NOT yet live: a natural `*_recovered`
  mail (next real staleness cycle / reboot drill — never fabricate one by blocking reports) and
  seed-at-claim on a real claim (Peti's Friday reinstall is the natural candidate).

- **2026-07-22 — hub v0.70.1: the ghost customer's Delete button exists now.** The v0.70.0
  ghost-delete path was fully implemented and fully unreachable — the **fourth inert-seam defect**,
  this time a TEMPLATE GATE: the Danger-zone card (and the `customerDeleteOpen` script) sat inside
  `{{if .HasConfig}}`, so a ghost rendered no Delete button (found live on `demo-vm-felhom`).
  Fix: `Deletable` page flag = the exact negation of the delete preview's 404 predicate; RESET
  stays HasConfig-gated, Block/Unblock config-only. Render tests per branch + 2 red-proofs.
  **The seam-wiring rule now covers template gates** (see CLAUDE.md) — handler tests that POST
  directly prove nothing about reachability. **Operator live leg PENDING: the demo-vm-felhom ghost
  delete click** (doubles as v0.70.0+v0.70.1 live validation). Same day: **agent 0.93.0 published
  + deployed fleet-wide** (`pilot/RUNBOOK-publish-agent-0.93-2026-07-22.md`), **R-17 CLOSED**
  (u629193-sub1 deleted at the console).

- **2026-07-21 — THE FLEET IS TWO HOSTS.** A second Proxmox node exists: **`demo-hp-bb76ea`**, an
  **HP t740 Thin Client** (Ryzen V1756B, 30 GiB, PVE node `felhom-host`, customer `demo-hp`), installed
  from the armed universal ISO the same day. Both hosts run **agent 0.92.1**; the N100 is
  `demo-felhom-8363b5`. **Full inventory: `documentation/operations/nodes.md`** — read it before
  touching the HP, because two things there are load-bearing: the box's **1TB Toshiba NVMe is present
  and UNENROLLED** (still carrying its old NTFS partition; a future Tárhely candidate that must join
  through the storage flow, never the installer), and there is **no operator SSH key baked** on it, so
  access is the hub-vaulted G1 break-glass password (R-61 is the fix for that lockout).
  **Both nodes are at the VACATION site and travel home ~2026-08-02.** `demo-hp` joined the tailnet as
  `100.76.96.79` — an **operator-lab exception**, not product shape; real customer boxes never get
  tailscale. **Second-hardware proof, worth keeping:** the pairing/day-0 chain ran end to end on virgin
  hardware it had never seen, the **shim loader booted with Secure Boot ENABLED** (so SB-off was an
  N100-firmware workaround, not a Felhom requirement), and the box **self-lifted the floor
  0.153.0 → 0.156.0 during day-0**. The install was not clean, and the failures are filed: **R-59**
  (no DHCP on the 4-port NIC → the installer baked a static fallback instead of aborting), **R-60**
  (first-boot NIC sweep self-heal), **R-61** (baked root password unknowable).

- **2026-07-19 — N100 MADE LOCATION-INDEPENDENT via Tailscale.** `felhom-pve` (the N100) joined the
  tailnet as a host package (`100.70.170.35`, expiry disabled); DooPlex already advertised
  `192.168.0.0/24` via its GitOps k3s `admin-system/tailscale` pod (`100.107.87.53`). `ssh felhom-pve`
  now targets the tailnet IP (direct-over-LAN at home, tunnel when away); `felhom-pve-lan` =
  `192.168.0.162` fallback. `--accept-dns=false` on the host (pi-hole split-horizon preserved). The
  host was converted **static→DHCP** (reservation MAC `68:1d:ef:5d:a6:64`→`.162` keeps home behaviour).
  **Measured spike:** `--accept-routes` on the N100 *while on `192.168.0.0/24`* hairpins the local
  subnet (table 52 overrides `main`) → kept OFF, it's a travel-only opt-in. **PBS is offsite** (its own
  WireGuard tunnel to Hetzner, not the home LAN) so it needs none of this. **Recorded, NOT fixed:** the
  controller↔agent plane is hard-pinned to `192.168.0.162` (`felhom-agent` `listen_addr` +
  guest `bootstrap.json`), so the box is reachable + backs up off-LAN but its local operator plane is
  not yet portable. Full doc: `documentation/operations/tailscale.md`. No code changed.

- **2026-07-19 — CC WORKFLOW MIGRATED TO DOOPLEX.** Claude Code now runs natively on DooPlex
  (192.168.0.180, Debian 13, `kisfenyo`) in a tmux session, working directly in
  `/mnt/5_hdd/felhom.eu/git/<repo>`. **DooPlex is now the canonical CC environment; the Windows
  workstation trees are stale** as of felhom.eu `ee48a48288`, felhom-agent `d81fddc7b1`,
  felhom-controller `ac7323dc9a`, app-catalog-felhom.eu `a14c97db65` (unchanged — it had no
  environment-specific content). Builds are local commands; felhom-pve is one `ssh` hop; `kubectl`
  is local. **`claude-in-chrome` is NOT available here** — endpoint-level validation is the
  standard method. New **clean-tree gate** before any build (`git status --porcelain` empty AND
  `HEAD` == `origin/main`), because the CC working tree is now the tree `build.sh` builds from.
  Windows knowledge is preserved in per-file "Legacy: Windows workstation" sections and the
  versioned workspace file `documentation/runbooks/workspace-CLAUDE.md` (live copy:
  `/mnt/5_hdd/felhom.eu/git/CLAUDE.md`).

- **2026-07-17 — CUSTOMER SELF-BIND shipped (hub v0.66.0 + ISO scripts v1.20.0, R-27 slice 1).** A
  customer binds their OWN freshly-installed appliance without the operator. Operator clicks **"Send
  self-bind link"** on the customer Setup tab → hub mints a **7-day tokenized capability link** →
  emails it (Hungarian, sibling sender, NOT via the claim engine) → customer opens the **public,
  Hungarian `/bind/<token>`** page (no login — the URL token IS the auth) → enters the **console
  pairing code** (shown on the box screen) + their **retrieval passphrase** → hub stages the bind via
  the same `BindAppliance` (provenance `customer_selfbind`) → the box's ~30 s appliance poll delivers.
  **Viktor's three rulings, verbatim:** (a) *"only their own visible"* → console pairing code, **no
  appliance list ever rendered**; (b) *first-box entry* → operator-sent tokenized capability link over
  Hungarian email; (c) *lockout after 5 failed attempts* → token locks, "call support". Wrong code and
  wrong passphrase = **one identical generic failure** (no oracle; both factors compared
  unconditionally); expiry falls back to operator-bind unchanged. **THE TRAP (§9.2):** one public
  prefix `/bind/`, exempted from auth+CSRF at both `/login` gate sites via a single `isPublicBindPath`
  predicate (tight trailing-slash match; ServeMux `..`-cleans; handler rejects `/` in the token) — 4
  red-proofs verified red-then-green (lockout→C1, oracle→B, widened-prefix→E, single-active→C4).
  **GC verdict:** no appliance-staleness GC exists (`applianceStaleAfter` is a display badge only) → the
  7-day token TTL stands alone, single-active-per-customer, no reaper needed. **Controller/agent
  untouched; R-27b (controller second-box dismissable prompt) DEFERRED, mechanism sketched.** Green:
  full hub `build/vet/test` (17 ok, +9 self-bind tests) + `bash -n`. **DEPLOYED + LIVE (hub 0.66.0 on
  k3s, Synced/Healthy).** Live through the real ingress: `GET /bind/<garbage>` → 200 Hungarian expired
  page (public, logged-out, no-oracle — the 200-not-500 proves `selfbind_tokens` migrated live); `/` +
  `/hosts` → 302 /login (gate intact); `POST /bind/<garbage>` no-CSRF → 200 (exempt) while
  `POST /customers/acme/block` no-CSRF → 302 /login (exemption is `/bind/`-only). **PENDING
  (operator-gated, NOT-live):** operator "Send self-bind link" + the customer's real two-factor success
  (hub UI password-gated → CC can't log in) + the new-ISO console banner on a re-imaged box → fold into
  the R-1 rehearsal. See REPORT.md.

- **2026-07-17 — PBS DR STORAGE VISIBILITY + Offsite tab split + dual gauges shipped (hub v0.65.0 +
  tenantsync v1.2.0, R-5), LIVE.** Scoping correction (Viktor): "restic box" and "PBS box" are NOT two
  Hetzner Storage Boxes — restic = subaccounts on the shared Hetzner box (Hetzner API, v0.64.0); **PBS DR =
  the `felhom-offsite` PBS datastore on ep0** (NO Hetzner API; the hub had no read of its fill). **Option A
  (ruled):** a read-only `usage` op on the `felhom-tenantsync` ep0 forced command (twin of `fingerprint`,
  `df` on the datastore path — no customer_id, no admin token, NO mutation), polled by `monitor.PBSDRBoxChecker`
  (OffsiteBoxChecker clone over a `usageReader` seam; 15-min throttle; three states ok/unavailable/degraded;
  escalation-only `pbsdr_box_fill` on the `"pbsdr-box"` operator scope). `/offsite` now has **Restic** +
  **PBS DR** tabs (endpoint cards under PBS DR — the endpoint IS the PBS host); the dashboard has **two
  gauges** (restic %·ratio, PBS %). **Graceful degradation: the hub deploy is INDEPENDENT of the ep0 update**
  — an ep0 ≤ v1.1.0 → `ErrUsageUnsupported` → the gauge shows "n/a" until updated (no hub redeploy).
  **PROVEN-LIVE (data pipeline, both gauges):** ep0 updated to v1.2.0 this session; the hub polled it —
  `PBS-DR box refreshed: 19.1% full (7.1 GB of 37.2 GB)`. **DECISION:** PBS thresholds are separate keys
  (`PBSDRBoxFill*`) defaulting to restic's 80/90 (independently tunable; Viktor's call for a different pair);
  no oversubscription for PBS (namespaces, not quotas). **OPEN:** UI render unit-verified only (hub UI
  password-gated); the PBS fill alert email is unit + red-proof verified, NOT fired live (datastore nominal
  at 19%). READ-ONLY against ep0 + Hetzner. See REPORT.md.

- **2026-07-17 — OFFSITE POOL-BOX AGGREGATE shipped (hub v0.64.0, R-5).** The operator now sees the
  shared pool box's real state on the hub: total fill vs capacity, Σ(shared soft quotas)/capacity (the
  oversubscription ratio), per-customer usage/quota bars, and a box-level operator alert (fill 80/90%,
  oversub 2.0×). New `monitor.OffsiteBoxChecker` (OffsiteChecker-sibling): fetch-throttled to one Hetzner
  GET / 15 min, cached `BoxSnapshot`, escalation-only + recovery re-arm; two INDEPENDENT signals (fill +
  oversub). Capacity = `storage_box_type.size`, usage = `stats.size` (Phase-0-pinned, bytes); Σ(quota)
  from the ConfigJSON `Descriptor` (`offsite.ReadDescriptor`, new — NEVER the report echo; dedicated +
  disabled excluded); per-customer USAGE from the report echo. Events scope `"pool-box"` → operator channel
  only, no SaveEvent. Web: Offsite-tab panel + compact dashboard tile, reading the cached snapshot only
  (never fetches). **PROVEN-LIVE (data pipeline):** the in-cluster checker computed the real box —
  `0.2% full (2.6 GB of 1.00 TB), Σ shared quota 150 GB, oversub 0.15x`. **DECISION: thresholds
  (80/90/2.0×) are Claude's encoding — Viktor's ruling pending; named `Alerting.OffsiteBox*` keys = the
  one-line flip.** **OPEN:** the UI render is unit-verified only (hub UI password-gated → CC can't
  screenshot); the alert emails are unit + red-proof verified but NOT fired live (real pool nominal — a
  live-fire would email Viktor; offered on demand). READ-ONLY against Hetzner throughout. See REPORT.md.

- **2026-07-17 — SYSTEM-INITIATED IMMEDIACY wired per-plane (hub v0.63.0).** The immediate-sync arc
  covered only OPERATOR-initiated changes; **system-initiated** desired-state mutations bumped the
  generation silently, so a freshly onboarded box waited a full agent tick for state the hub had already
  minted (observed live at slice-C onboarding). Decision: wire the existing, live-proven notifiers into
  every system site on the CORRECT plane — agent-plane `poke.Notifier` at `PBSDRAutoProvision` (the exact
  lag), `ReissuePBSDR` (which also lifts the pbsdrheal reconciler's escalation with zero reconciler
  changes), `handlePBSDRReissue`, and the two admin api desired-state writers (new nil-safe `Poker` seam);
  controller-plane `intent.Hub.Bump` at `reissueOnReenroll`. All fire AFTER the store write only. The
  **pbsdrheal Restage path is deliberately poke-free** (no generation bump → the agent's 60 s pbsdr ticker
  is its pickup path — a poke there is a verified no-op); the WG register/delete bumps stay poke-free
  (undeliverable pre-tunnel / transport removed — the agent fast-tick SECONDARY owns the register leg).
  The **first-hold-baseline race** is still OPEN by design (a bump landing during a fresh controller's
  FIRST wait is recorded as baseline without firing — the ruled fix is carrying `intent_gen` in the report
  ACK, its own future task). Unit-tested + red-proofed; live legs fire only on real system events (fold
  into the rehearsal bind sequence). See REPORT.md + hub/CHANGELOG.md v0.63.0.

- **2026-07-17 — R-21 SLICE C SHIPPED: the universal secret-free ISO (hub v0.62.0 + scripts v1.19.0, LIVE).**
  The distributed ISO carries NO customer secret. A box booted from `build-felhom-iso.sh --pairing`
  self-registers as an UNCLAIMED APPLIANCE (`POST /api/v1/appliance/register`, keyed by SMBIOS-uuid +
  MAC set — the DMI-verdict tiebreaker; token = sha256-stored, 256-bit), the operator BINDS it to a
  customer on the Hosts page (new "Unclaimed appliances" section, SSH-key fingerprints, host-count
  display-only), and the hub delivers customer-id + retrieval passphrase ONCE (one-shot poll: 404
  no-oracle / 204 unbound / 200 deliver / 410 gone). The bootstrap is **one unit, two modes** — direct
  (env-baked, byte-identical, regression-proven zero-appliance-calls) vs pairing (register→poll→write
  env→fall through to direct). Red-proofs: one-shot delivery + register idempotency (both proven red).
  Live-verified through the public ingress (register→token, 204, bogus→404). Artifact proven secret-free
  (baked env = hub URL only; manifest `secret-bearing: no`). **Bind is operator-password-gated** → the
  live boot→bind→day-0 composition (+ physical N100) folds into the supervised rehearsal, which now runs
  the COMPLETE final flow in one pass (RESET demo → generic ISO → bind → day-0). SSH-key pinning =
  stored + fingerprints displayed (host-attach on bind deferred, no clean surface). Self-bind page = R-27.
  R-21 now SHIPPED (A+B+C). See REPORT.md.

- **2026-07-17 — R-21 SLICE B SHIPPED: firmware loader option (scripts v1.18.0).** Closes N100 **F1
  (HIGH)** — cheap AMI (`AN3PLUS 0.01`-class) firmware can't USB-boot the ISO's stock GRUB
  (`relocation 0x0`). `build-felhom-iso.sh --loader shim|mkimage` (default **shim**, unchanged;
  profile-settable via `FELHOM_LOADER`). **mkimage** = a monolithic `grub-mkimage` loader built from
  the ISO's own GRUB (module set from its `grub.cfg`, embedded `search --fs-uuid`→`configfile`), swapped
  into the ISO9660 tree + `efi.img` ESP by `mkimage-surgery.sh` **after** `prepare-iso` (payload
  provably untouched), re-mastered with xorriso preserving BIOS-hybrid + UEFI + GPT-ESP (drops only the
  irrelevant Apple HFS+/APM). **Unsigned → target Secure Boot MUST be OFF.** Loud: banner + manifest
  `loader:` + `-mkimage` filename. New `profiles/n100.profile`; Dockerfile gains grub/mtools.
  **RUNBOOK-B legs (nested VM 311, OVMF):** shim boots+installs under SB-enforcing + SeaBIOS; mkimage
  boots+installs under SB-off; **mkimage under SB-enforcing FAILS `Access Denied`** (red-proof); surgery
  byte-identical payload. Physical N100 boot still pending → supervised rehearsal (an `n100-safety`
  match-nothing ISO built + sha-recorded, unbooted). PXE stays a deferred R-21 note. See REPORT.md.

- **2026-07-17 — CUSTOMER RESET SHIPPED: hub v0.61.0 + felhom-tenantsync v1.1.0 (LIVE).** The middle
  lifecycle tier (host delete < RESET < customer Delete). One operator action → pre-first-install: all
  OPERATIONAL state dies (offsite repo, PBS namespace+backups, DR recipe, one-time secret, claim state,
  retained escrow custody); IDENTITY + basic config (incl. the offsite tier CHOICE) + provenance + events
  SURVIVE. Rulings: separate escrow-custody ack; clears claim (fresh code next onboarding); REFUSES while
  any host row exists; live-counted confirm inventory. Discipline: external teardown FIRST, DB purge LAST,
  every leg idempotent → partial run re-runs from the top (purge withheld until externals ok). New:
  `store/customer_reset.go` (journal+inventory+ack-gated purge), `claim.ResetToUnclaimed`,
  `offsite.{Deprovision,OffsiteIdentifier,ClearProvisionedDescriptor}`, `tenantsync.Deprovision` +
  `felhom-tenantsync.sh` **deprovision** op (v1.1.0, destroys ns+groups+token, shared user untouched),
  `web/customer_reset.go` (GET inventory JSON / POST orchestration) + an **amber** RESET card distinct
  from the red Danger-zone Delete. Red-proofs: ack-gate + partial-failure resumability (both proven red).
  **Live-drilled on ep0** (throwaway `drill-reset-01` with a real backup): deprovision `deleted:true`,
  idempotent re-run `deleted:false`, all 3 real tenants survived. Hetzner-delete + the password-gated web
  POST covered by tests (offsite Deprovision mirrors live-proven ReissueCredentials). See REPORT.md.

- **2026-07-17 — HOST-DELETE DEMOTES ESCROW: hub v0.60.1 (LIVE).** Closes the v0.60.0 review gap:
  `DeleteHost(deleteEscrow=true)` now DEMOTES the current escrow blob into `host_escrow_superseded`
  (copy-before-delete, same tx) + spares existing superseded rows — never destroys custody. The
  customer Danger-zone Delete (`DeleteCustomerConfig`, which before touched ONLY `customer_configs`)
  is now the one true purge point: cascades both escrow tables for all the customer's hosts incl.
  already-deleted (via F-14 `host_deletions`). One shared copy routine `demoteCurrentEscrowTx` (reused
  by SaveHostEscrow). Checkbox/refusal/Danger-zone wording → demotion. **S6b VERDICT: OBSOLETE** —
  re-enroll upserts cleanly (`UpsertHost` ON CONFLICT DO UPDATE; `handleAdminCreateHost` no dup
  refusal) + v0.57.0 arc auto-fires re-issues (ROADMAP R-3 refined). Red-proofs
  `TestDeleteHost_DemotesEscrowNeverDestroys` + `TestDeleteCustomer_PurgesEscrowCustody`. Observation:
  customer-delete still doesn't touch host rows / the broader offboarding (rehearsal's question). REPORT.md.

- **2026-07-17 — OFFSITE CONTINUITY Part B SHIPPED: hub v0.60.0 (LIVE).** Superseded-escrow
  retention (data-first, Viktor's ruling). `SaveHostEscrow`'s destructive `ON CONFLICT` overwrite
  destroyed the old passphrase's only copy (the reinstall-orphan incident); now a new
  different-sha blob RETAINS the old into the new `host_escrow_superseded` history table (tx, before
  overwrite); same-sha = idempotent. ACK/restore read the current row unchanged. `escrow_superseded`
  event + operator retained-count on host detail; registered `offbox_repo_orphaned`/`offbox_repo_reset`.
  Red-proof `TestSaveHostEscrow_RetainsSuperseded`. Pairs with controller v0.142.0 (Part A
  orphaned-repo guard). Guided old-history recovery = R-26 (not built). See REPORT.md.

- **2026-07-17 — N100 polish (F5 scripts + F6/F7 controller).** F5 (host-install v1.17.0):
  appliance mode auto-sizes the guest (RAM=clamp(host-4096,min 4096,max host-2048,ceil host-1024) +
  cores=host-1 min 2) when no explicit cap; explicit `--memory`/`--cores` win. Harness red-proof
  (8/16/32 GB). Same commit: the stale "EMPTY by default" operator-key comment fixed (R-16 doc-drift,
  keys are PINNED). F6/F7 landed in controller v0.141.0 (LIVE on demo; fresh-USB wizard enroll+format
  now PROVEN-LIVE — see the felhom-controller REPORT). New ROADMAP R-24 (guest resources as
  desired-state / live resize = F5 follow-on). Operator sidebar #1 (BIOS State-After-G3→Power-On)
  added to `scripts/iso/README.md`. See REPORT.md.

- **2026-07-16 — DIRECTION-2a AGENT-PLANE POKE SHIPPED: hub v0.59.0 (LIVE).** The agent-plane
  sibling of the v0.58.0 wait channel: `internal/poke` — a pinned-host-key SSH sender (wgsync
  sibling) + fire-and-forget `Notifier` (`PokeHost`/`PokeAllHosts`, nil-safe, detached) that nudges a
  box in seconds via a CONTENTLESS UDP poke relayed hub → ep0 `felhom-poke` forced-cmd → wg0-origin →
  the box WG /32:51822. Wired at `applyPBSDR` (per-host) + `handleSetArtifacts` (fleet). ep0 surface
  (`scripts/felhom-poke.sh`, non-root, no sudoers; runbook §11) + `Secret/agent-poke` +
  `manifests/hub.yaml` (tag 0.59.0). **Deployed live** (ArgoCD Synced/Healthy, `agent-plane poke
  enabled`); ep0 leg live-validated (`poke-fired`; non-WG + `rm -rf /` refused). Source note: the
  MinAgent-floor save does not bump per-host generation (self-update rides signed-ops) → the fleet
  poke accelerates the next report, not a delta. Pairs with felhom-agent v0.89.0 (the listener).
  **PROVEN-LIVE 2026-07-17:** the operator's real manifest save logged `poke: sync-poke delivered to
  10.77.0.2` (+ floor HELD for the below-MinAgent box); the demo (now 0.89.0) received a poke in
  ~31 ms and fired the immediate desired-state cycle (save→tick ≈ ~0.45 s). All three live legs
  (Scenario 4 A/B train, Scenario 1 R-22 self-heal, Scenario 3 latency) done; capability-map poke +
  F4 rows upgraded to PROVEN-LIVE. See REPORT.md.

- **2026-07-16 — DIRECTION-2 IMMEDIATE-SYNC SHIPPED: hub v0.58.0.** The hub→box "sync now" wait
  channel (option b of `documentation/audits/SPIKE-immediate-sync-transport-2026-07-16.md`). New
  `internal/intent` in-memory per-customer generation notifier (Bump/Wait/Close; coalesce-to-latest;
  race-closer; in-memory by design → a restart costs one harmless full-state report, never a storm).
  `GET /api/v1/wait` long-poll: per-customer only (global key 400; A can't see B), 240 s hold with a
  25 s heartbeat newline that defeats nginx's 60 s `proxy_read_timeout` **without any ingress
  annotation** (WriteTimeout lifted per-connection via `ResponseController`; global timeouts
  untouched). Every operator-intent web handler bumps AFTER its store write (config CRUD, claim
  resend, offsite re-issue/freeze, password regen, block/unblock, floors [global bumps all managed],
  controller log-tail + log-bundle). `intentHub.Close()` before `server.Shutdown`. Pairs with
  controller v0.140.0 (the long-poll client). The immediate-sync arc is COMPLETE for the controller
  plane (Direction 1 = ctrl v0.139.0, Direction 2 = hub v0.58.0 / ctrl v0.140.0); the agent-plane
  ep0-poke stays PARKED in the OOB arc. Detail: `hub/CHANGELOG.md` v0.58.0, `REPORT.md`.

- **2026-07-16 — REINSTALL ARC SHIPPED: hub v0.57.0 (F2/F3/2.3).** The clean-slate
  reinstall-of-existing-customer path is now first-class (Peti R-1 prerequisite). On host-enroll's
  mint path (single-bump-per-re-enroll): claim re-issue for a claimed customer (`ReissueForReenroll`,
  reset code — the hub never stores the password, so fork B not A), offsite re-issue to the fresh box
  (`ReissueOffsiteForCustomer`), and escrow-honesty — re-issuing offsite marks the escrow STALE
  (`MarkEscrowStale`), withholds the mismatched restic-hash from auto-confirm, and shows "stale" not
  "ceremony done" (2.3, red-proofed). **Part 0** unblocked the demo live (ACL grant on
  `/storage/felhom-offsite` → PBS-DR converged, `escrow.pbs_storage_id` seeded). **Controller +
  scripts unchanged** — both spec premises were contradicted by source (the controller reads escrow
  prereqs live from the agent; the installer can't know the descriptor-provisioned storage id). **F4**
  root fix is agent-side → **ROADMAP R-22** (agent train: the reconcile pre-check 403 must not abort
  before the root-run apply grants the ACL). Detail: `hub/CHANGELOG.md` v0.57.0, `REPORT.md`.

- **2026-07-16 — R-21 PHYSICAL RUN on the demo N100 (`tests/VALIDATION-n100-baremetal-2026-07-16.md`).**
  demo-felhom host reinstalled clean-slate from a pipeline-built ISO; first-boot chain reached
  **rc-0 on the first attempt on real hardware** (closes slice A's operator-gated boundary),
  serial-filter safety proven on metal (external-HDD canary byte-identical), PBS-DR reconciler
  self-healed on the reused peer. **The obstacle was firmware, not us:** the cheap AMI `AN3PLUS 0.01`
  board can't UEFI-boot the ISO's GRUB from USB (`relocation 0x0`; SB-off + shim-bypass don't help) —
  worked around live by rebuilding the stick's loader with `grub-mkimage` from the box's own working
  GRUB. Findings (to tackle): F1 firmware boot loader (slice-B), F2 claim-re-issue on reinstall,
  F3 offsite-re-issue, F4 PBS-DR ACL on non-default storage id, F5 guest-RAM configurability,
  F6 drive-init doesn't mount+attach, F7 back-route. Box healthy, BookStack deployed, demo live.

- **2026-07-16 — R-21 SLICE A SHIPPED: bare-metal Felhom ISO pipeline (`scripts/iso/`, scripts
  v1.16.0).** A DooPlex pipeline (`build-felhom-iso.sh` + `Dockerfile.assistant`) turns the official
  PVE ISO into a Felhom auto-install ISO whose first-boot stub installs a retry-forever
  `felhom-bootstrap` unit that unattended-fetches `felhom-host-install.sh` from the public
  `felhom.eu/scripts/` channel and runs it (host-install **unmodified**). Validated on VM 310:
  build gate + red-proof, disk-filter fail-safe, stub→unit→real-fetch→invoke→retry, resume-decision,
  exactly-once, no-net retry+recovery — all GREEN. **Rulings settled:** stub is from-iso (no
  cert-rotation risk); ordering fully-up; the retry unit owns all network work. **§4.4 finding
  (shapes slice C):** the distribution URL carries no secret, but the unattended install needs the
  customer retrieval passphrase → **the slice-A ISO is secret-bearing** (supervised/single-use; the
  bootstrap shreds the env on success). OPEN: host-install rc-0 terminal success (hub-enrolled +
  guest-provisioned) is operator-gated — a drill customer needs the password-gated create-UI (a
  live-`hub.db` insert is unsafe). Slice B = physical N100 RUNBOOK; slice C = claim-pairing to
  de-secret the distributed ISO. Detail: `REPORT.md`, `documentation/audits/SPIKE-baremetal-iso-2026-07-16.md`.

- **2026-07-13 eve — ESCROW CEREMONY WIZARD SHIPPED (agent v0.88.0 + controller v0.127.0; this
  repo: host-install v1.16.0 + RUNBOOK rewrite).** The recovery-code ceremony is now
  customer-driveable from the controller (`/backup/escrow`): one-shot R reveal, re-stage-first,
  Scenario-F stale-blob re-check (fired LIVE on both boxes' hash-less blobs at first ACK; drill's
  repaired via a real ceremony — hub hash now covers). **Ruling F1 (2026-07-13), settled:** *R
  transiting the Cloudflare tunnel once at reveal is an accepted risk — same trust class as the
  claim code/login password; LAN-direct delivery is PARKED.* (threat model:
  runbooks/RUNBOOK-escrow-ceremony.md). host-install v1.16.0 = FELHOM_ESCROW rides the canonical
  sudoers fetch; hub `hostInstallVersion` synced (gate green), **hub NOT redeployed** (const rides
  the next train). OPEN: supervised full-browser wizard pass (Viktor's drill login; also demo, to
  clear its legacy stale warning); agent 0.88.0 publish + Day-0 vouch at the next train.

- **2026-07-13 — CLOSING BUNDLE SHIPPED: hub v0.53.0** — the last engineering items on the
  pre-tester board: F-14 gated auto-Reissue (deletion provenance `host_deletions` in-tx +
  `token_exists` gate → the EXISTING Reissue op + `pbsdr_auto_reissue` audit event; no record /
  un-acked → refusal byte-unchanged), dead-host roll-up honesty (customer status = worst of
  controller-derived + per-host `hostStatus` on dashboard//configs/detail, cause chip names the
  host; the Peti 23h-masking shape is the pinned fixture), operator bearer OUT OF GIT
  (`Secret/report-api` secretKeyRef non-optional + `REPORT_API_KEY` env override +
  `scripts/manifest_bearer_gate.py`; VALUE ROTATION = supervised step in
  documentation/runbooks/secrets.md — the git-history copy is dead only after it).
  **Two operator rulings (2026-07-13), recorded verbatim — future sessions must treat these as
  settled:**
  1. ***F-14:** auto-Reissue of offsite credentials is permitted ONLY when the hub's own deletion
     record shows the tenancy's owning host was removed through the escrow-ack flow — acknowledged
     destruction is not silent re-keying. No record → manual Re-issue stays the only path.*
  2. ***Roll-up:** a customer's status may never look better than its worst expected host. Single
     staleness definition (`hostStatusOf`) — no second threshold anywhere.*
  Gate additions: `python scripts/manifest_bearer_gate.py` after ANY manifests/ change (64-hex
  bearer shapes; felhom.secret.yaml carve-out = visible KNOWN-BACKLOG line). GOTCHA: the repo
  .gitignore `*secret*` pattern silently un-tracks any "secret"-named script — hence the
  "bearer" name. The controller repo's controller.yaml.example carried the REAL bearer literal —
  scrubbed (felhom-controller 51c871a).

- **2026-07-13 — POLISH BATCH SHIPPED: hub v0.52.0 (+ controller v0.123.0)** — take-two F-15
  CLOSED (reset-request response carries the rotated code hash — ACK shape/guarantee — emailed
  codes work immediately; live 1 s apply + first-try accept on Viktor's exact failure path) and
  F-16 CLOSED (zero native `confirm()` in the hub UI — `inline_confirm.html` partial, Igen/Mégse
  inline two-step, asserted by `scripts/hub_confirm_gate.py` — **which exists but is NOT enforced:
  it is invoked by nothing** (R-29); danger-zone typed-confirm untouched;
  live: offsite re-issue completed under browser automation without freezing). Operator resend
  keeps the ACK lag — flash says so. Take-two §4 dispositions updated (F-15/F-16/zero-toggle obs
  → SHIPPED). Remaining from take-two: **F-14** (needs operator ruling: auto-Reissue re-adopt vs
  never-silently-re-key) + S5 DR-restore drill; hub bearer key rotation still DUE.

- **2026-07-12 — DR-TIER-BY-DEFAULT BATCH SHIPPED: installer v1.15.0 + hub v0.51.0 (+ agent
  v0.86.0 in felhom-agent)** — closes DRILL-day0-vm F-1/F-2/F-3/F-7/F-8/F-9/F-10 and F-6 (by
  policy). **The five operator decisions (2026-07-12), recorded verbatim — future sessions must
  treat these as settled:**
  1. *Every install deploys the full DR plumbing uniformly (pbs-apply wrapper, `age`, WG enabled,
     full ACL). **Capability is baked; activation is a hub flag.***
  2. ***DR tier defaults ON for new customers.** Disabling is the per-customer exception — and it
     removes the Felhom-side cost entirely (no PBS namespace, no endpoint dependency).*
  3. ***Offsite app backup requires the DR tier ON** (fork-4 needs the ceremony; the ceremony
     needs PBS). Enforced hub-side by extending the drill-proven dependency guard.*
  4. ***Identity-only escrow is formally PARKED** — closed by policy, not implemented. Revisit
     only if a customer class demands lean-tier offsite. Record in the ceremony runbook + CONTEXT
     so future sessions know it was a decision, not an oversight.*
  5. ***WG is base infrastructure, not a DR component** — always-on like cloudflared; it is also
     the future OOB-management transport. BYO consent: a disclosure line lands in the
     tester-agreement doc; per-customer hub disable remains possible.*
  Mechanics: installer ships `felhom-pbs-apply` + `age` + `wg_tunnel.enabled: true` (byo wg-off
  assert RETIRED — consent = typed disclosure ack + tester-agreement line), single VERSION source
  with a cross-check against the hub Setup-tab const written as `scripts/hostinstall_gates.py` —
  **but NOT enforced: the script exists, is RED today (1.19.0 != 1.22.0) and is invoked by nothing**
  (tracked as R-94 leg (b), class R-29) — default ACL
  keeps felhom-pbs (doc advice REVERSED — narrowing caused the drill 403). Hub: per-customer
  `dr_tier` flag (new default ON; legacy backfilled from descriptor reality, one-time),
  cascade-not-error semantics + per-stage status on the edit form, WG-registration
  auto-provision hook (scenario A hands-free), offsite-requires-DR guard (exact F-6 message),
  host-page capability chips (agent 0.86.0 `inactive` = neutral; pre-1.15 "binary not found" →
  migration hint). felhom-pve's latent F-3 state fixed live; Peti's box unreachable (his F-3
  check + D/E/G deferred).
- **2026-07-12 — HUB v0.50.0 SHIPPED + LIVE: customer-claim password arc (closes DRILL-day0-vm
  F-4/F-5).** The customer OWNS the dashboard password via a one-time claim code the hub emails to
  the registered address (`bcrypt(code)` stored, plaintext only in the send). `internal/claim`
  engine (EnsureIssued idempotent Day-0/live-box issue+email, Resend, RequestReset 3/day,
  MarkClaimed set-only) + `store.customer_claims` (RotateClaimCode bumps generation + preserves
  claimed_at) + delivery (config bake `web.claim_code_*`, report-ACK `claim` object, `POST
  /api/v1/claim/reset-request`) + Setup-tab chip/resend + 3 Hungarian emails. Pairs with controller
  **v0.122.0** (the gate). **LIVE-proven on drill box demo-vm-felhom: gate ON via the real edge (/ →
  302 claim page, /api → 401); code emailed to the registered address.** Deployed felhom-hub:0.50.0
  (ArgoCD synced/healthy). Floor raise 0.120→0.122 = operator's supervised LAST step. Observed: the
  hub issues a claim code for EVERY reporting customer on first report (benign for pre-0.122 boxes —
  no gate, ACK ignored). Report: documentation/audits/DRILL-day0-vm-2026-07-12.md (F-4/F-5 RESOLVED).
- **2026-07-12 — HUB v0.49.0 SHIPPED: Edit tab merge (edit-a) + scoped auto-refresh + riders**
  (CHANGELOG v0.49.0; commits `e740147` → `2e03de1` → `1d94b1a` + docs/manifest). The standalone
  customer edit page merged into the customer page's **Edit** tab (renamed from Settings): shared
  `config_form_body` sub-template + `configFormData` builder (REUSE.md rows added), Block/Delete
  relocated to a danger-zone card, header cleaned, `GET /configs/{id}/edit` → 302 `#tab=edit`,
  post-action redirects anchored to their tab, `settings→edit` legacy-hash alias. Update handler
  gained server-side required-field validation (error path preserves SUBMITTED values —
  red-proofed). The 60s auto-refresh is now scoped: fires only on live tabs
  (`data-live-tabs="overview,applications,events,host"`) and never while a form is dirty; muted
  `(paused)` hint. Riders: `/style.css?v={{hubVersion}}` cache-bust on every template (closes the
  v0.47.0 max-age=3600 gotcha) + the CLAUDE.md `git add -A` ban codified. **NOTE: the task spec
  said v0.48.0, but v0.48.0 (app_start_failed) had already shipped — published tags are never
  re-pointed, so this train is v0.49.0.** Out of scope (observed, not done): type-to-confirm for
  customer delete (still `confirm()`, relocated as-is); the workspace-root `E:\git\CLAUDE.md` and
  other repos' CLAUDE.md files still carry `git add -A` snippets — flag for their own sessions.
- **2026-07-12 — Day-0 VM DRILL COMPLETE (auto-confirm FIRST LIVE FIRING): full arc proven on a
  fresh nested-PVE box** — appliance Day-0 → floor-at-first-report → ceremony → **auto-confirm
  pending→escrowed in ~7.5 min, zero clicks** → offsite backup + restore round-trip. Two HIGH gaps:
  **F-4 no operator password-set path exists (G10 unclosable, dashboards born OPEN — blocks tester
  gate)** and **F-6 identity-only ceremony never implemented (no-PBS appliance can't escrow — drill
  forked to PBS DR tier = full Peti-sequence rehearsal, all green)**. Installer fresh-box gaps:
  felhom-pbs-apply not shipped (F-7), `age` missing (F-10), root-owned guests/ parents (F-3 —
  check demo for the latent copy), silent root@pam rotation UX (F-8). Runbook fixes committed
  (day0 A.2 anonymous-fetch; escrow-ceremony identity-only claim CORRECTED + age prereq). Report:
  `documentation/audits/DRILL-day0-vm-2026-07-12.md`. Drill VM qm 300 kept (3 snapshots) for
  re-drills; teardown list in report §9.
- **2026-07-12 — CAMPAIGN-3 Task A SHIPPED: agent v0.85.0 boot/recovery plane + appliance self-heal.**
  Fixes F12 (CRITICAL boot ordering cycle — templates drop network-online, `MigrateNetworkUnits`
  repairs installed units), F11/F10/F9 (reassert: fstype-driven classify, reset-failed+rearm, per-unit
  verdict, + re-arm-on-guest-blind matrix-correction), F2/F1 (zero residue: reset-failed + rmdir), the
  guest-hook rc255 belt (recover+timeout, wrapper no longer execs), and the appliance node self-heal
  (`internal/selfheal`: host networking recovery, gated on `deployment_mode` — host-install `--mode`
  templates it; byo remedy unreachable). 3 sudoers grants added (reset-failed, rmdir, start networking).
  **Live matrix on demo felhom-pve: host reboot ×2 = 0 ordering-cycle lines both boots (F12 dead);
  appliance self-heal recovered networking on attempt 1; F10 reset-failed+rearm sweep loud; zero
  residue; guest-reboot heals.** **NEW finding F13 (HIGH, deferred):** active nfs4 under mp8 can fail
  PVE rbind (rc255); NAS-down-at-boot → apps `Created` (fix-3). Peti's box (agent 0.81) unaffected
  until his rollout — carries the F12 coin flip if it has a NAS share (interim drop-in in the agent
  REPORT). host-install v1.14.0; scripts CHANGELOG. Details: `documentation/controller/network-storage-nas.md`.
- **2026-07-12 — CAMPAIGN-3 NIGHT RUN DONE; host recovered, RCA closed.** Unattended chaos campaign
  22:09→04:27 vs demo box (ctrl 0.117.0/agent 0.84.0): data plane solid (refusals/verify/rollback/
  restore byte-identical/EIO instant/stub badge + deploy-409 live-validated). The NAS automount
  plane produced 2 CRITICAL + 3 HIGH: **F12 CRITICAL (= the overnight host loss, RCA'd next
  morning): the agent automount template's `After/Wants=network-online.target` creates a boot
  ordering cycle — systemd deletes an arbitrary job to break it; boot -1 sacrificed networking
  (host dark 7h, hardware exonerated), boot 0 sacrificed the automount (NAS dead). Every host
  boot with an enrolled share = coin flip until the template drops that ordering (`_netdev` on
  the .mount suffices).** F10 CRITICAL (start-limit never re-armed, once blocked guest start),
  F11/F9 HIGH (post-start reassert "skip-active" skips the healing automount restart; agent-restart
  sweep silent) → 4 NAS apps dead-at-boot on 3/3 guest reboots; F7 HIGH (in-place dump write →
  0-byte tar replaced last good backup during outage window). Fix-6 numbers: ring horizon 55min
  idle / **6.5min under load**, wiped on every restart. Box left in defined state (6 wave apps
  healthy, temps removed, NAS baseline-identical). Full ledger/evidence:
  `documentation/audits/CAMPAIGN-3-2026-07-11.md` + `180:~/campaign3/`.
- **2026-07-11 — HUB v0.47.0 UI REORGANIZATION SHIPPED** (CHANGELOG v0.47.0; commits `9f29bf3` →
  `0daddcd` + docs). Five deliverables: **(1)** `.data-table td a:not(.btn)` button-contrast fix;
  **(2)** customer page = **8 hash tabs** (`#tab=…`, sticky summary strip, Events error badge,
  no-JS graceful degradation — panels hide only via a JS-added body class); **(3)** shared
  `host_detail_body` sub-template rendered by `/hosts/{id}` AND the new per-customer **Host tab**
  (a list by design; `store.ListHostsByCustomer` + `hostDetailData` builder); **(4)** **stale host
  removal** — `GET /hosts/{id}/delete-impact` (counts/booleans only) + `POST /hosts/{id}/delete`
  behind type-to-confirm; ONLINE → 409 always (no override), escrow needs an explicit checkbox
  (`ErrHostEscrowPresent`, tx never starts), one-tx cascade incl. the bound wg peer (wgsync's 5-min
  declarative push converges the endpoint; log bundles die by `scope_id == host_id` only);
  **(5)** **/offsite multi-endpoint management UI** — all `wg_endpoints` rows as cards +
  add/edit/delete with 409 guards (peers-in-subnet pins subnet + delete). **Deferral (explicit):
  peer allocation, the wgsync reconciler push and the desired-state merge stay lowest-endpoint-id
  (`GetWGEndpoint` untouched); the future arc is a `wg_peers.endpoint_id` migration + per-endpoint
  allocation/reconciler/desired-state.** Five red-proofs ran (online gate, escrow ack, bundle
  scope, endpoint-delete guard, subnet-change guard). *Resolved:* the earlier ⚠️ about `146d165`
  sweeping the Part-4 WIP — the sweep caught `hosts.go` mid-red-proof (escrow ack bypassed);
  `068427a` restored the gate; the arc is now complete and author-reviewed. **Stale-doc note:** the
  workspace-root CLAUDE.md's hub deploy wording predates GitOps — deploys are manifest-tag bumps +
  deliberate ArgoCD sync, never `kubectl set image`.

- **2026-07-11 — RCA FIXES 1+2+4 SHIPPED + Q1c GREEN (agent v0.84.0 + controller v0.117.0 LIVE on
  demo).** Reboot survival: automatic since agent 0.84.0, live-proven 2026-07-11 21:20 (`pct reboot
  9201` under operator GO — the hook re-armed + verified nas-media DURING the start task, CWA
  auto-started healthy, health ok first cycle, no badge). Agent: `ReassertNetworkMounts` (hook
  post-start + startup; stop+enable-now on existing sudoers verbs — NO sudoers widening needed;
  never in periodic paths). Controller: statfs consuming-namespace classifier → probe
  `not_network_fs` (stub can never verify), deploy-time stub refusal (409; idle autofs deploys
  fine), distinct stub badge (wins over unreachable; unreachable line byte-identical), deploy view
  selects stored HDD_PATH (RCA fix 4). MinAgent unchanged 0.81.0. All red-proofs recorded in repo
  REPORTs. Feature doc: `documentation/controller/network-storage-nas.md` §Reboot survival; audit §7
  follow-up added. Queued: RCA fixes 3/5/6; Peti publish train now carries 0.84 + 0.117.

- **2026-07-11 — NAS/CWA RCA (diagnostic, no fixes):** `documentation/audits/AUDIT-nas-cwa-rca-2026-07-11.md`.
  The nas-media re-add verified legitimately (new host automounts DO propagate into running
  guests); a second `pct reboot 9201` at 15:39 severed the trigger → guest-side stub; CWA
  deployed correctly to the NAS (app.yaml + binds + data all nas-media) but is DOWN since 15:39
  (docker can't recreate the bind source; perms luckily blocked any system-drive writes; zero data
  loss). UI "felhom-usb" = display bug (deploy.html selects by IsDefault, never stored HDD_PATH);
  hub device-ID WARN was CORRECT and the only guard that fired; NAS badge masks because it asks
  the agent (host ns). 6 prioritized fix tasks in the audit §5. **Remediation §6 PERFORMED on
  operator GO (~20:14–20:20 CEST): automount-unit restart re-planted the trigger in the running
  guest instantly (fix-1 mechanism live-proven), CWA restarted healthy on the NAS library,
  uid-1000 write OK, controller health back to status=ok at 20:18. Demo fully healed.**

- **2026-07-11 — OBSERVABILITY PASS (hub v0.46.0 LIVE Synced/Healthy; agent v0.83.0 + controller
  v0.116.1 on demo).** Remote pull-only log access to both box components' always-DEBUG capture
  rings. Hub: `log_bundle_requests`/`log_bundles` store (gzip, newest-3, **72 h TTL** on the 60 s
  sweep), the **secret gate fail-closed** before storing (`blocked: possible secret`; `[REDACTED]`/
  checksums pass), report-ACK `controller_log_requested` + heartbeat-envelope `log_tail_requested`
  channels (consume-once; pre-0.83 agents stay visibly pending), host-detail Diagnostics section
  (Request controller/agent logs + View/Download; the hosts read-only invariant amended: these two
  forms are the only actions). Conventions codified:
  `documentation/runbooks/logging-conventions.md` + CLAUDE.md one-liners in all three repos.
  **OPEN: operator clicks both request buttons on hub.felhom.eu/hosts/<demo> to close the live
  round-trip** (UI password-gated; the flow is fully API-test-proven). Hub bearer key rotation
  still DUE (pre-existing).

- **2026-07-11 — PUBLISH-TRAIN RULES CODIFIED** (`documentation/runbooks/publish-train-rules.md`):
  1. manifest before floor; 2. the manifest screen carries the LIVE DB floor
  (`hub_settings.min_controller_version` overrides env, acts immediately) — save the floor field
  LAST; 3. MinAgent gate — **now HUB-ENFORCED PER-BOX (hub v0.45.0, `store.ResolveManagedFloor`),
  the manual fleet check retired**; 4. box-level backstop = the controller's agent-capability gate
  (v0.114.0 route probe → v0.115.0 version compare). (Floor-UI separation + per-box gating SHIPPED
  in hub v0.45.0 — see the backlog-clear entry above; this rules-doc note supersedes the original
  "hub untouched / option-3 deferred" wording.)

- **2026-07-11 — NAS/coupling backlog CLEARED (hub v0.45.0 LIVE + agent v0.82.0/ctrl v0.115.0 on demo).**
  Composite A–F. Hub v0.45.0 (Synced/Healthy): floor-UI separation + effective-floor source line
  (DB `hub_settings` vs env, `store.ResolveGlobalFloor`) + type-to-confirm w/ live below-floor count;
  per-box MinAgent conditional floor (`store.ResolveManagedFloor` HOLDS the controller floor for a box
  whose `hosts.agent_version` < the manifest's MinAgent, flags it on the Hosts dashboard) — manual
  fleet check retired (publish-train-rules.md rule 3). THE one comparator extracted to
  `internal/semver`. Agent v0.82.0 = `X-Felhom-Agent-Version` header (demo-only, Peti stays 0.81 =
  probe fallback); ctrl v0.115.0 = version-aware Supports + DSM-validated NFS guidance. **DSM spike**
  (real DSM 7.2 via virtual-dsm): consumer recipes validated E2E, hardlink caveat closed for Synology
  (record: documentation/audits/SPIKE-nas-dsm-2026-07-11.md); QNAP caveat stands. **Q1c (supervised
  reboot) FAILED**: a NAS automount trigger does NOT survive a guest reboot (empty-dir; agent lacks a
  network-mount reassert) — fix spec'd at documentation/backlog/FOLLOWUP-nas-automount-guest-reboot-reassert.md.
  NOT published (agent 0.82 demo-only). Record: felhom.eu/REPORT.md + each repo's REPORT.

- **2026-07-11 — PUBLISH TRAIN 0.81/0.113 COMPLETE — PETI LIVE on agent 0.81.0 + controller 0.113.0.**
  Record: `documentation/pilot/RUNBOOK-publish-0.81-0.113-2026-07-11.md`. Agent 0.81.0 published
  (sha `914bb3b5…30df0b`, live-bytes, anon-verified) + golden 0.113.0 baked/published (sha
  `66de5e64…82054c`, drill VM, virgin-reverted); manifest + signed `agent_update`
  (key-id `felhom-op-1`) by Viktor → Peti 0.79.0→0.81.0 clean (no rollback); **floor auto-confirm
  proof CAPTURED** (10:41 0.112.0 → 10:42 0.113.0, anonymous pull proven on HIS box) — KEY LEARNING:
  the hub UI manifest screen ALSO sets the GLOBAL floor as a DB `hub_settings.min_controller_version`
  row that OVERRIDES the env and acts immediately (this is how Peti reached 0.112 and now 0.113
  before the GitOps env bump `16390d0`); a transient forbidden skew (ctrl 0.113 + agent 0.79,
  ~10 min) was benign only because zero NAS shares existed — next train: save the floor field LAST.
  OPEN: Peti journal-group one-liner (E), temp Git Sync creds deletion (G1), **hub bearer key
  rotation now DUE** (committed in manifests/hub.yaml + screenshot-exposed this session).

- **2026-07-11 — NAS verify-before-commit SHIPPED (agent v0.81.0 + controller v0.113.0 LIVE on demo;
  host-install v1.13.0).** The spike's production task landed same-day: add = agent verify (retry=0
  unit, journal-classified, auto-rollback) → controller uid-1000 write probe (--netprobe re-exec) →
  register LAST; full rollback on any failure; §3.2 Hungarian error map (nfs_export merged);
  storage_network.html redesigned (SMB-first, Route-A guidance); orphan rows; systemd-journal group
  (NO sudoers). Live A–E validated vs an isolated sim NAS, Route A proven in production (alien-uid
  1060 server-side). Feature doc: `documentation/controller/network-storage-nas.md`. NOT published
  (no Gitea/Day-0 bump; Peti pending with the usermod one-liner). Record:
  `felhom-controller/REPORT.md`.

- **2026-07-11 — SPIKE: NAS verify mechanisms + NFS squash matrix — ALL FIVE QUESTIONS ANSWERED, no
  blocker.** Record: `documentation/audits/SPIKE-nas-verify-2026-07-11.md`. Q1 in-guest access DOES
  wake the host automount (make-or-break PASS; container bind-mount too); Q2 controller drops to uid
  1000 and writes (setpriv proof; SETUID/SETGID caps present); Q3 **Route A ACCEPTED-with-caveats**
  (`all_squash` to ANY server uid = full rw+hardlink for the guest app; guest sees nobody; chown
  EPERM-clean, chmod persists; June's `anonuid=1000` WRONG case pinned = anonuid without all_squash
  is a no-op for non-root); Q4 error taxonomy captured (NFSv4 makes no-export vs not-permitted
  INDISTINGUISHABLE; systemd 90 s start-timeout is the real ceiling; **`retry=0` → production unit**:
  dead-NAS access 91 s→3.8 s; verify must NOT ride the 15 s agentapi timeout → own path/status-poll,
  ~120 s budget); Q5 plain-user SMB PASS incl. hardlink. Teardown verified to pre-counts. Production
  TASK is the operator side's next move.

- **2026-07-11 — PBS DR tier SLICE 2 SHIPPED (agent v0.80.0): the apply-bridge — demo
  ADOPTION-PROVEN live + PUBLISHED; the supervised Peti phase PENDING.** Full chain fired on the
  demo: hub enable (storage-id `felhom-offsite`) → tenantsync provisioned ns `demo-felhom` on ep0
  → agent ADOPTED the manual entry (tenancy entry-owned, "entry wins"; staged secret asserted
  UNCONSUMED; DR green; `escrow.pbs_storage_id` seeded). Laws red-proof-verified: set-only (K
  destruction guard), secret-on-stdin, verify-pin-before-consume, loud consumed-failed. Part 0
  evidence: write path needs the §4b dual-grant exactly (403 `Datastore.AllocateSpace` → re-assert
  → token vzdump OK); demo grants restored. Live finding fixed: escrow seed = in-place write
  (/etc/felhom-agent dir is root-owned; rename impossible for the non-root agent). 0.80.0
  published (`f2ba62ca…f06d2`, == deployed bytes). **NEXT = RUNBOOK-peti-pbsdr-2026-07-11.md**
  (Peti box prep: sudoers+wrapper+wg_tunnel enable — no tunnel/OOB on his box today; Viktor opsign
  0.80.0; hub enable → FRESH path, K born, degradation clears; ceremony → auto-confirm → first
  offsite run = the arc closes). Detail: felhom-agent/REPORT.md + CHANGELOG v0.80.0.

- **2026-07-10 — PBS DR tier SLICE 1 SHIPPED+LIVE (hub v0.44.0): ep0 tenantsync surface + hub
  provisioning.** `felhom-tenantsync` v1.0.0 installed on ep0 (second forced-command key, peersync
  untouched — hashes asserted; smoke caught + fixed the client `{"data":[…]}` json wrap; orphan
  `root@pam!spike` cleaned). Hub: `internal/tenantsync` (pinned-SSH twin of wgsync; errors never
  embed stdout), config-form "PBS DR tier (ep0)" section → WG-peer check (fail-closed) → provision →
  `host_pbs_secrets` consume-once (agent fetches `POST /api/v1/hosts/{id}/pbs/consume-token` —
  note: /hosts/, not the task's /host/) → `pbs_dr` descriptor into HOST desired_json + gen bump;
  idempotent re-save; Re-issue button (F4 twin). Red-proofs: consume-once / fail-closed /
  idempotency all mutation-proven. **NOT live-provisioned — slice 2 (agent bridge) + supervised
  Peti enable pending.** Runbook §10 addendum in offsite-endpoint.md. Commits ce6a566 + 32ca1f6.

- **2026-07-10 — SPIKE: PBS DR tier auto-provisioning SLICE 0 — all three mechanisms SETTLED.** Record:
  `documentation/audits/SPIKE-pbs-tier-provisioning-2026-07-10.md`. (1) PVE token storage-create **impossible**
  (create/modify/delete all check `/storage` root) → **sudoers vector proven** (pinned-id alias, cycle green as
  the agent user, unpinned denied; secret must ride a stdin-wrapper, never sudo argv). (2) ep0 tenancy op-set +
  one-time-secret-over-SSH-stdout **proven live on ep0** (🛑 Viktor-approved; read/write cross-tenant 403s);
  current peersync channel can't carry it → slice 1 ships a second forced-command surface (`felhom-tenantsync`).
  (3) `pvesm add pbs --encryption-key autogen` births K at `PBSEncKeyPath` + vzdump lands ciphertext (doc-06
  property) + escrow-create finds K. **⚠ `pvesm remove` DELETES K** → re-apply must be set-only. All spike
  artifacts torn down; flags: orphan `root@pam!spike` on ep0 scratch; demo's §4b step-3 storage grants absent.
  Epic slice map in the record §5.

- **2026-07-10 — new-box restore round-trip VALIDATED + old-box archive RETIRED.** Record:
  `documentation/audits/VALIDATION-newbox-restore-2026-07-10.md`. Both toggled apps restored from the
  production pool box via the real endpoint — **9/9 sha256-identical** (immich 44MB db-dump exact,
  `d78466fcbf…4526b`, agreeing with the campaign's independent round). ~10s/app. **Viktor's decision:
  RETIRE the old archive** — he deletes `u629193-sub1` in the old project's console (the PBS-storage-1 box
  itself stays for dev). C2 fixed same day as controller v0.110.0 (crash-lock self-heal, live-proven).
  Offsite arc: build-complete + restore-proven both directions; remaining = the Peti onboarding runbook.

- **2026-07-10 — offsite arc overnight soak (unattended) — PASS + 1 HIGH finding.** Record:
  `documentation/audits/CAMPAIGN-offsite-overnight-2026-07-10.md`. F4 re-issue proved LIVE (reset invalidates
  the old password); restore round-trip 9/9 byte-identical new-box↔USB (immich db-dump exact → old-box
  archive `u629193*` safe to retire, Viktor's decision); single-flight + crash-restart (zero consume, escrow
  untouched) PASS. **FINDING C2 (HIGH, OPEN):** a controller crash MID-PRUNE leaves a stale restic exclusive
  lock the controller can't self-heal → offbox runs fail until manual `restic unlock` (fix candidate:
  `ensureOffboxRepo` auto-unlocks a self-owned stale lock). Track B (live quota-crossing) + hub-orchestrated
  Track A blocked — the hub operator config surface is session/CSRF-gated (unattended can't reach it) → Peti
  runbook. Demo restored to start-state. One expected `backup_failed` email (the crash test).

- **2026-07-10 — SLICE 4 soft-quota + offsite health SHIPPED+LIVE (hub v0.41.0 + controller v0.109.1) —
  the offsite arc is BUILD-COMPLETE.** Controller: `quota_gb` rides the descriptor into the target (hash
  includes it → quota changes re-apply via key-auth-first, no consume); `repo_size_bytes` from restic
  stats; pre-run gate (≥100% refuses NEW backups Hungarian-loudly, **prune still runs** — red-proofed;
  restore never gated; ≥80% warns); /backups usage bar. Hub: `monitor.OffsiteChecker` (fill 90/95 of
  quota + `offsite_stale` >48h for enabled+escrowed only — red-proofed; nil-safe on old reports) +
  **manual-only Freeze/Unfreeze lever** (readonly on the exactly-1 labelled sub-account, SSH preserved).
  **Live:** key-auth-first proven against the real box (the deploy's own re-apply); that re-apply exposed
  + fixed v0.109.1 (re-apply must preserve EscrowState + runtime status — it demoted the escrowed demo);
  demo repaired (manual confirm + run: escrowed, 45 MB / 50 GB, checker healthy 0 events, usage bar
  renders). Freeze + 100%-refusal NOT live-exercised (STOP; red-proofs cover). NEXT: Peti onboarding
  runbook, NEW-box restore round-trip → old-box retirement.

- **2026-07-09 — SLICE 3 hub-verified escrow auto-confirm SHIPPED+LIVE (hub v0.40.0 + controller v0.108.0 +
  agent v0.79.0).** Replaces the operator-trust confirm with a verified fact: ceremony hashes the sealed
  repo password (agent, `restic_pw_sha256,omitempty` on the escrow upload — contract test moved in lockstep)
  → hub stores it (additive `host_escrow` migration, NULL on legacy rows) + serves
  `escrow:{identity_blob_present,restic_pw_sha256,created_at}` in the report ACK → controller
  (`report.EscrowAutoConfirmer`) flips pending→escrowed ONLY on hash match with its local repo password.
  **Red-proofed core: blob-presence alone never confirms** (stale blob = false custody claim); mismatch
  warns once per hash naming the ceremony; never un-confirms; flip wipes the agent-staged secret. Hashers
  pinned by the same cross-repo vector. Manual confirm = deprecated fallback (legacy blobs — demo included,
  hash-NULL verified live; the escrowed demo produced zero confirm activity = Scenario E live). Full happy
  chain proves out at the next real enable (Peti). NEXT: SLICE 4 soft-quota, Peti runbook.

- **2026-07-09 — offsite hardening bundle SHIPPED+LIVE (hub v0.39.0 + controller v0.107.0 + agent v0.78.0).**
  The e2e's sharp edges closed before slices 3–4: **F4** "Re-issue offsite credentials" (explicit operator
  recovery for consumed-password dead-ends — exactly-1 label scope, fresh one-time secret, ConfigVersion
  bump; `hetznerapi.ResetBoxPassword` added for dedicated); **F2** host-key scan retry (~60s ladder — first
  save survives fresh-subaccount DNS lag); **F5** submit-disable + in-flight notice; **key-auth-first bridge**
  (working installed key → re-pin+reconfigure WITHOUT consuming; fingerprint verify never bypassed; kills the
  stale-descriptor consume-404 loop); **staged-secret wipe** (agent `DELETE /escrow/stage-secret`; controller
  calls it on every flip to `escrowed`; the e2e's lingering staged file wiped live through the new endpoint,
  idempotency proven). All red-proofed. F4 reset NOT exercised live (demo credential untouchable — throwaway
  customer under supervision when needed). NEXT: SLICE 3 auto-confirm, SLICE 4 soft-quota, Peti runbook.

- **2026-07-09 — offsite provisioning VALIDATED LIVE end-to-end (hub v0.38.1 + controller v0.106.1) — PASS.**
  Full loop through the real pipeline on `demo-felhom` against the new dedicated-project pool box (611714 /
  `u629488`, project 15282031): UI save → sub-account 269008 provisioned → descriptor v6 + one-time password →
  controller config-refresh (≤15 min) → apply-bridge (verify-pin → consume-once → pinned `ssh-copy-id` →
  `EscrowState=pending`) → run-gate refusal proven → escrow confirm (Option A: the repo password was already
  escrowed — `WriteOffboxSecrets` preserves it) → first run: **restic repo + 2 snapshots on the new box,
  key-only**. Two live bugs found+fixed mid-run: **F1** hub v0.38.1 (`applyOffsite` on `context.WithoutCancel` —
  a re-click canceled the provision between create and password-store, stranding the sub-account) and **F3**
  controller v0.106.1 (`ssh-copy-id -s` needs `~/.ssh` to exist in the container — died AFTER consuming the
  password). OPEN: F2 (fresh sub-account DNS lags → first-save scan 502s; retry-with-backoff), F4 (hub has no
  password-reset path — recovery today is delete-subaccount + re-save), F5 (no save spinner → re-click bait).
  Phase-0 wiring: `manifests/hub.yaml` now injects `HETZNER_TOKEN`/`HETZNER_POOL_BOX_ID` from
  Secret/storagebox; scoped-token check = list boxes with it (must see ONLY 611714). Old-box offsite kept as
  frozen archive until a NEW-box restore round-trip. Record:
  `documentation/audits/VALIDATION-offsite-provisioning-e2e-2026-07-09.md`. NEXT: SLICE 3 (auto-confirm),
  SLICE 4 (soft-quota), F2/F4/F5, Peti onboarding runbook.

- **2026-07-09 — offsite provisioning SLICE 2 (hub v0.38.0 + controller v0.106.0).** The controller apply-bridge:
  on startup it reconciles the hub-served `offsite:` descriptor into a key-only offbox target
  (`controller/internal/offsiteapply.Bridge`) — **verify-pin the box host key against the hub-captured
  `host_fingerprint` (no blind TOFU)** → consume the one-time password (single-use) → `sshpass ssh-copy-id -s -f`
  install → configure offbox → `EscrowState="pending"` → persist a descriptor-hash marker. Idempotent +
  fail-safe; both red-proofs green. Hub v0.38.0 adds `Descriptor.HostFingerprint` captured via an
  `x/crypto/ssh` keyscan (fail-closed). **NOT yet live-applied** — supervised end-to-end (hub provisions on
  the new pool box → controller apply) is the next runbook, gated on the hub's new scoped `HETZNER_TOKEN`.
  NEXT: SLICE 3 (escrow auto-confirm), SLICE 4 (soft-quota).

- **2026-07-09 — hub offsite provisioning SLICE 1 (hub v0.37.0).** The hub can now provision the offsite tier
  on operator enable: `internal/hetznerapi` (typed client, base **api.hetzner.com/v1**, `CloudAPI` interface +
  exported `Fake`, `WaitAction`), `internal/offsite` (`Provisioner.ProvisionOffsite` — idempotent by label
  `felhom-customer=<id>`, shared sub-account / dedicated box, generates the transient password, builds the
  NON-SECRET descriptor, **fail-closed**), `one_time_secrets` store (single-use `Save/ConsumeOneTimeSecret`),
  `POST /offsite/consume-password/{id}` (customer-API-key auth, once), config-form Offsite section →
  `applyOffsite` (502 + no-save on error) → descriptor in ConfigJSON → ConfigVersion bump. Token from
  `HETZNER_TOKEN` (out-of-band); password NEVER in ConfigJSON/logs. Unit-tested vs a faked Cloud API (+ a
  fail-closed red-proof). **NOT yet live-provisioned** — awaiting the **dedicated-project scoped token** (the
  current token can delete ep0). NEXT: SLICE 2 (controller apply-bridge consuming the descriptor + one-time
  password), SLICE 3 (escrow auto-confirm), SLICE 4 (soft-quota). Deployed via GitOps.

- **2026-07-09 — Hetzner API provisioning spike (offsite tier).** Both offsite models are API-provisionable
  (FIT). **#1 correction: the Storage Box API is `https://api.hetzner.com/v1`, NOT `api.hetzner.cloud/v1`**
  (Cloud API 404s for storage boxes); same Bearer token. Proven live on `spike-*` (torn down): sub-account
  create/reset/readonly/delete + a billable bx11 box create/rescale(`change_type`)/delete. Create takes a
  **password (no ssh-key param)** → transient-password→`ssh-copy-id -s -f`→restic-reach→`reset_subaccount_password`
  (transient pw invalidated, key survives) = key-only steady state. RepoPath `/home/<repo>` (chroot). Quota:
  dedicated=box type/`change_type` (hard); shared=no native quota → Felhom soft-quota + `update_access_settings{readonly}`
  freeze. Idempotency = **label_selector** (box names not unique). Rate 3600/h. **SECURITY: the token is
  project-wide unscopeable Read+Write — it can also touch ep0 (the prod offsite server) + 3 SSH keys →
  move customer boxes to a DEDICATED Hetzner project before the hub holds a write token.** Record:
  `documentation/audits/SPIKE-hetzner-api-provisioning-2026-07-09.md`. NEXT: the hub-provisioning impl.

- **2026-07-09 — fork-4 custody VALIDATED end-to-end live (supervised).** The full custody cycle proven on
  the demo (agent v0.77.0 + controller v0.105.0): enable → controller pushes the repo password to the agent →
  **real escrow ceremony** (`--selftest=escrow-create`, K + restic-pw under a fresh R, uploaded to the hub,
  staged secret wiped, K byte-unchanged) → **atomicity proof:** a run **skipped** before `confirm-escrow` and
  **proceeded** after (2 apps, 2 snapshots) — the gate opens exactly on confirm. **DR money shot:** the restic
  password recovered from the **production hub blob** (`host_escrow.identity_blob` for `demo-felhom-01`, via
  `identity-consume` with R) is byte-identical to the live password (`sha 24c44f49…`) and **decrypted the real
  box repo**; the real `inject-password`→`restore` endpoints reproduced a 45 MB db-dump byte-perfect. Demo
  offbox left **escrowed** (un-paused); Viktor holds R out-of-band; no secret in any log; live `repo_password`
  unchanged. Record: `documentation/audits/VALIDATION-fork4-custody-2026-07-09.md`. **NEXT:** hub-provisioning +
  soft-quota spec (hub-verified auto-confirm replacing operator-trust; both storage models); per-customer
  onboarding runbook; the `identity-consume` OK-message fix (omits the recovered restic field).

- **2026-07-09 — offbox v0.104.0 VALIDATED live against the Storage Box (supervised).** Closed the offsite
  arc: controller `:0.104.0`@`908e4b9` ran the real `/backup/offbox/*` endpoints against a fresh box
  **sub-account** (`u629193-sub1`, RepoPath `/home/felhom-demo-repo`). The capstone proof — **immich**
  (undeployed, recovery unit only on the USB drive, the exact case v0.103.0 mis-resolved to `systemDataPath`)
  → discovery found the USB unit → `ok`, 1 snapshot; 45 MB db-dump restore sha256-matched. No-silent-success
  confirmed on the live box: 0/N → `error` + `backup_failed` operator alert; partial → `ok` +
  Hungarian `LastWarning` on `/backups`. Persists across restart. **Forks:** (1) direct topology + (2)
  sub-account-per-customer (RepoPath `/home/<name>`, `ssh-copy-id -p 23 -s -f` — console SSH-key field still
  broken, F3) confirmed; (3) PBS/WG tier deferred; **(4) custody surface RECORDED, escrow design OPEN** —
  the per-customer restic repo password + SSH key are 0600 in the controller data dir, **NOT independently
  recoverable** (ride DR only via the PBS whole-CT) → the load-bearing follow-up before real reliance.
  **KEPT** as a standing demo-offsite (€4.06/mo). Record: `documentation/audits/VALIDATION-offbox-storagebox-2026-07-09.md`.

- **2026-07-09 — TASK GL-7 SHIPPED: customer-page passphrase hardening + install-command generator
  (hub v0.36.0, `844fbfa7`; deploy `02c748eb`; LIVE, ArgoCD Synced/Healthy)** — go-live G7/G9/G10/G11.
  Two coupled, security-first changes to the operator customer page (`customer_unified.html` +
  `configs.go`). **(1) Passphrase hardening:** the per-customer retrieval passphrase was cleartext
  twice (visible `#retrieval-pw` + baked into the Option-3 debug curl `X-Retrieval-Password:` header);
  now masked-by-default with reveal (`toggleSecret`) + copy (`copySecret`), value in `data-secret`
  (existing model), and the Option-3 command carries a `<YOUR-RETRIEVAL-PASSWORD>` placeholder — no
  secret in any copyable command. (Zero-secret-in-DOM fetch = noted future follow-up, NOT scoped.)
  **(2) Install-command generator:** the three hard-coded `<code>` blocks became a client-side builder
  (vanilla JS, no framework/CDN/network) emitting ONLY real host-install v1.12.0 flags, download-then-
  run, byo-requires-caps enforced client-side, CustomerID prefilled from the server, JS-off static
  fallback; the seven dangerous/operator-only flags (`--force`/`--rotate-recovery`/`--enable-oob`/
  `--remove-golden`/`--uninstall`/`--adopt-pool`/`--rescope-acl`) are never offered as controls. Tests
  `TestTemplates_PassphraseHardened` + `TestTemplates_InstallGenerator` (green; passphrase test
  red-proofs). **Tester agreement written** (`documentation/pilot/PETI-tester-agreement.md`): BYO
  trust boundary, honest limitations (no offsite backup, F2 removable-drive, F7 pool-reassert, F6
  `:53`-free), exit rights, Peti box facts (80c/128 GB → caps 12/32768, `sajatfelhom.hu` re-point),
  byo install command, onboarding sequence. GO-LIVE-PACKAGE: GL-7 ✔; G9/G10/G11 → done. **One open
  pilot question (owned by the agreement §6):** Peti's local backup TARGET — does his box have a
  second disk/pool? → sets `--acl-storages` at onboarding. Deploy gotcha: first build was stale
  (local commits not pushed before `build.sh` pulls origin) — pushed then rebuilt. **NEXT: Peti day-0
  onboarding (operator-run).**
- **2026-07-08 — TASK GL-8 SHIPPED: BYO coexistence hardening (host-install v1.12.0, `18a556a5`)** —
  the GL-6 findings F3/F6/F1. **F3:** the populated-host leaf guard fires only when a `felhom_guests`
  guest exists (not `pct list`) → a fresh byo install on a host running only the OWNER's guests needs
  no `--allow-new-leaf`; the pin protection for real Felhom guests is unchanged. **F6:** byo preflight
  REFUSES a foreign process on `:53` with remediation and mutates NOTHING (Viktor's ruling: refuse+
  instruct only — the dnsmasq analog of the break-glass no-touch rule; appliance untouched). **F1:**
  uninstall purges the config `.bak*` siblings (a GL-6 residue held a live hub api_key). F6
  live-proven on felhom-pve (byo preflight refuses at :53 — the exact GL-6 scenario, read-only die).
  Harness 36/36 (+GL8-F3/F6/F1 static, GL8-F1b behavioural, GL8-F6 live) + red-proofs RP-F3/F6/F1;
  GL-2/GL-4 regression green; shellcheck clean. Shipped BEFORE GL-7 so Peti's command needs no
  workaround. **F2/F7 remain open (non-blocking). NEXT: GL-7.**
- **2026-07-08 — TASK GL-5 SHIPPED: DR bring-up bind overrides (agent v0.75.0, LIVE + scratch-DR
  validated)** — go-live G8; docs-only here (GO-LIVE-PACKAGE G8/G6/G1 flips + decision log; the
  code + full report live in felhom-agent `b3446213`). Guest-loss DR restore WORKS now: the two
  structural binds (mp8/mp9) are overridden with throwaways at restore and swapped for the REAL
  binds pre-start (root pct via the new Engine HostRunner seam), displaced unusedN deleted. TWO
  live-discovered PVE rules beyond the spike: explicit-params restore needs an explicit rootfs AND
  silently DROPS unlisted mountpoints — the full param set now derives from the archive's embedded
  config (`ExtractArchiveConfig`, 200 under the scoped token). Live proof: scratch 9310 from a real
  9201 archive — mp0 200G + mp1 50G restored WITH content (7m23s), real binds, zero residue, clean
  teardown; the same op failed outright on v0.74.0. OPERATOR: **publish agent 0.75.0 + bump the
  manifest to it** (vouching 0.74.0 ships broken DR). Follow-ups: restore-test boot-verifies
  without mp0/mp1 (same PVE rule, reverse direction — candidate reuse of drRestoreOverrides);
  DR-selftest KeepMAC duplicates a live source's MAC briefly; no host-loss DR runbook exists yet
  (GL-6/S5 writes it).
- **2026-07-08 — TASK GL-4 SHIPPED: uninstaller gap-closure + operator-key day-0 fold
  (host-install v1.11.0)** — go-live G6 + the G1 key-pin follow-up; **awaiting GL-6** for the real
  (non-dry) teardown + armed-pin install. Uninstall now removes the self-update artifacts (4b4:
  guarded wrapper, .prev/.new.* slots, rollback unit, limits drop-in — derived from
  configs/felhom-selfupdate-guarded), unmounts enrolled drives under /mnt/felhom-drives (plain
  umount ONLY, busy = warn+guidance, root-bind guarded), and ends with a KEPT-vs-WIPED statement
  (drives/PBS/hub record/escrow/vaulted recovery credential live on; guest-only mode prints the
  vmid's bind-store drives). Key-pin: OPERATOR_KEY_* constants (EMPTY until the operator ceremony)
  + `--operator-pubkey-file` (validated at argv, comment=key_id required) → authz.signers written
  at step 6 per the agent SignerKey schema; **no-keys-resolved KEEPS preserved signers** (never
  un-pin a manually-pinned box); verify reports armed/dormant (dormant = WARN). Harness 28/28 on
  felhom-pve incl. the NEW GL4 H-U full-uninstall DRY transcript vs live 9201; red-proofs RP-1..3
  green. **GO-LIVE-PACKAGE.md was ABSENT AGAIN** (spec said the operator attaches it; not present
  in the repo or workspace) — G6/G1 status recorded here, doc still pending its operator commit.
  Key CEREMONY (real keypairs + felhom-pve pin) = operator step.
- **2026-07-07 — RUNBOOK GL-1 EXECUTED: agent 0.74.0 + golden 0.103.0 PUBLISHED** — go-live G1
  (partial): the published-artifact chain is current. Agent = the LIVE felhom-pve bytes (sha
  `1ec3f588…76af05`, provenance preserved, publish-agent.sh round-trip verified); golden baked
  clean-room in the drill VM (build-golden.sh v2.0.0 @ ced60ddf, controller 0.103.0 baked, all
  integrity gates green, sha `8481e8a1…6026e`, 583MB) and published HTTP 201. Both artifacts
  fetch-verified **ANONYMOUSLY** (Gate 3c finding: generic packages are world-readable → G3
  git.token read-only rotation CANNOT break fetches; but the INSTALLER dies on an empty token —
  script-side requirement, not Gitea's). Drill VM restored to `virgin`; evidence
  `/mnt/5_hdd/felhom.eu/drill/bake-0.103.0.log` on 180 + `documentation/pilot/RUNBOOK-GL1-publish-2026-07-07.md`.
  **OPERATOR NEXT: bump the hub Day-0 manifest** to agent 0.74.0 / golden 0.103.0 (the four values
  are in the execution record) — until then fresh installs land 0.63.0/0.98.3. Also recorded:
  operator-key pin follow-up (fold into GL-4); GO-LIVE-PACKAGE.md still not in the repo (G1 status
  recorded here instead).
- **2026-07-07 — TASK GL-2 SHIPPED: `--mode appliance|byo` install profile (host-install v1.10.0 +
  harness)** — go-live items G2/G4/G5 implemented, **awaiting the supervised GL-6 drill** for live
  provision/uninstall + C7 verify-drift. DECISIONS: the flag is MANDATORY (no default; the old
  `--mode provision` value retired with a specific die, `dr` stub kept); break-glass gated at its
  CALL SITE (byo never touches root@pam); byo requires `--cores`+`--memory`, refuses
  `--enable-oob`/`--rotate-recovery`/non-9.x-PVE/missing `--acl-storages`; disclosure + typed-hostname
  ack; **the config write's `lan_resolver` default flips to OFF in byo** (the spec assumed
  default-off — it was default-ON; appliance unchanged) + post-write asserts refuse
  lan_resolver/wg_tunnel/oob=true; pool+ACL verify asserts in BOTH modes (R2 lesson);
  `--preflight-only` (no state, PASS/FAIL verdict, proven live on felhom-pve). NEW
  `scripts/hostinstall-mode-harness.sh` (static C1–C4 + invariants everywhere; PVE tier: C5 +
  A/B dry transcripts) — 16/16 PASS on felhom-pve incl. C5 live; red-proofs RP-1..3 run→fail→revert.
  FINDING: demo-felhom's controller.yaml has EMPTY git.username/git.token → any step-5
  (re)install for that customer dies until the operator sets real Gitea read creds (relates to the
  standing scope-down+rotate follow-up). `documentation/pilot/GO-LIVE-PACKAGE.md` is NOT in the
  repo and no operator copy was provided — the G2/G4/G5 status flip is recorded here instead;
  add the doc when the operator supplies it. Next: **GL-6 supervised drill** (appliance + byo
  end-to-end, uninstall/re-provision, C7, the ack prompt interactively).
- **2026-07-05 — TASK H1 SHIPPED: OOB operator access (hub v0.35.0 + installer + endpoint; agent
  v0.72.0)** — merged E1+H1. Hub: operator OOB peer (`store/wg_operator.go`, `PUT/GET
  /admin/wg/operator-peer` global key) + `oob_peer_ip`/`oob_operator_ssh_key` in the desired-state
  wireguard block + `monitor/host_oob` (oob_degraded/recovered). Installer: `--enable-oob`/`install_oob`
  (static felhom-sshd + belt). Doc 06 §4.5/§4.6 amended (forwarding per-pair allow-listed, box↔box
  drop explicit, oob health stanza); endpoint runbook §9. Live-validated: operator→box SSH as
  felhom-op with scoped sudo; OF-1 /32 self-heal survival; coexistence (:22 PID unchanged); belt
  drops LAN + box↔box; PBS unaffected. Operator key = hub report_api_key.
- **2026-07-05 — TASK G1 SHIPPED: management-plane break-glass (hub v0.34.1 + installer; agent
  v0.71.0)** — prereq for felhom-sshd (H1). Hub: `store.host_recovery` vault (per-host root@pam
  console password; `PUT /hosts/{id}/recovery-credential` self-scoped + `GET /admin/hosts/{id}/
  recovery-credential` global-only, secret never logged) + `host_mgmtplane` monitor raising
  `mgmt_plane_healed`. Installer: `step_break_glass` (generate+set+vault root@pam, `--rotate-recovery`)
  + `install_mgmt_watchdog` (the 3 host artifacts, RuntimeDirectory-guarded). Live-validated:
  agent-independent `/run/sshd` auto-heal in 30.0s; warning fired end-to-end; break-glass
  vault→retrieve→PVE-ticket(200). felhom-pve root@pam is now the vaulted value.
- **2026-07-04 — S3 SHIPPED: agent WG tunnel (felhom-agent v0.64.0; docs-only here)** — the
  doc-06 §3.3 handshake is now END-TO-END LIVE on demo: agent keygen → `POST /hosts/{id}/wg` →
  desired-state `wireguard` block → agent-managed `wg-quick@wg-felhom` → PBS page over
  10.77.0.1:8007 in 3 s. **`wg_tunnel.enabled` defaults FALSE agent-side** (rollouts to other
  boxes are no-ops until the production endpoint exists). Revocation semantics completed (§3.5):
  removed peer → agent disables + keeps its marker + NEVER re-registers; operator re-adds via the
  pubkey the heartbeat report now carries. Escrow: `IdentityBundle.WGPrivateKey` auto-injected on
  new escrow-create runs (pre-S3 blobs can't be retrofitted — honest fallback = fresh-key
  re-register, /32 kept). OPEN: CGNAT/mobile-hotspot smoke (operator-assisted appendix, deferred
  non-blocking); S4 = PBS over the tunnel; hub-side S6 = tunnel-health alerting on the new report
  stanza. Detail: felhom-agent CHANGELOG v0.64.0 + REPORT.
- **2026-07-04 — S2 SHIPPED: box-facing WG registration + wireguard desired-state block + /offsite UI (hub v0.33.0)** —
  doc-06 row S2. DECISIONS: the WG assignment is **hub-owned state, merged into served
  desired-state at READ time** — stored `desired_json` stays a pure operator blob; admin PUT with
  a top-level `wireguard` key → 400 (no copy-paste clobber); **re-key-in-place keeps the /32**
  (stable tunnel addressing across rotation/DR); one bound peer per host (partial unique index);
  generation bumps ONLY on real change (register/re-key/adopt/bound-delete — idempotent paths
  move nothing). `POST /hosts/{id}/wg` self-scoped per-host key; NEW golden
  `desired-state-wireguard.golden.json` = the S3 cross-repo contract (byte-identical agent copy
  rule). peersync **v1.0.1** (strip out of process substitution — the S1 exit-swallow finding;
  sandbox red-proofed) redeployed to the dev endpoint. Live-proven on the REAL demo-felhom-01
  record via the real per-host key: gen 2→3, served block = golden shape, endpoint `wg show`
  carried the peer, v0.63.0 agent tolerated the unknown key (fetch clean, guests untouched),
  then removed → gen 4, all baselines restored. `/offsite` page unit-proven (live view is
  operator-password-gated — eyeball after next login). Next: **S3** (agent keygen +
  registration + `wg-felhom` service + escrow join + CGNAT smoke — risky/supervised).
- **2026-07-04 — S1 SHIPPED: offsite endpoint live + hub peer-sync (hub v0.32.1)** — the doc-06
  roadmap's first slice, done-criterion met live. Peer-sync design point RESOLVED: **hub pushes
  over SSH** to a forced-command `felhom-peersync` script (pull rejected: weakens immediate
  revocation; HTTPS API rejected: new binary + third public port). Endpoint = the dev
  `felhom-hetzner` box, provisioned by the NEW runbook `documentation/runbooks/offsite-endpoint.md`
  (WG on **443/udp**, subnet 10.77.0.0/24, PBS datastore `felhom-offsite` at `/mnt/pbs-datastore`
  — a 100 GiB Hetzner Cloud Volume attached 2026-07-27 06:29:40 UTC; the pre-migration copy at
  `/srv/pbs-felhom` is 13 G of dead weight on `/` awaiting R-91's go-ahead —
  nftables drop-all + wg0-only 8007, `ep0.felhom.eu` grey-cloud). Hub: `wg_endpoints`/`wg_peers` +
  /32 allocator + `/admin/wg/*` (global-key, pubkey-in-body) + `internal/wgsync` (FixedHostKey pin
  + **pinned-type HostKeyAlgorithms — live-caught bug**: multi-hostkey sshd presented ECDSA vs the
  ed25519 pin; fixed v0.32.1 + regression test) + full-list declarative reconciler (5-min drift
  repair). Live: add/remove→wg show, malformed→state-untouched, reboot→persist+converge, client
  tunnel via ep0:443→PBS login page, public 8007 closed. INCIDENT handled: first WG server key
  leaked via `wg show dump` into the session log → rotated immediately (warning now in the
  runbook). OPERATOR follow-up: **fix the AAAA record** (currently `…7aa1::`, must be `…7aa1::1`).
  Next: **S2** (hub schema host-join + `wireguard` block on WireDesiredState + box-facing
  registration + UI).
- **2026-07-03 — OFFSITE-CONNECTIVITY DESIGN-OF-RECORD WRITTEN (architecture Part 06)** —
  `documentation/architecture/06-offsite-connectivity.md`: records the six settled decisions
  (plain WG / host-side agent-managed `wg-felhom` as the agent-managed-unit PILOT / one shared
  hub-driven endpoint VM running WG+PBS, NO agent on it / hub = source of truth, `wireguard`
  block rides `WireDesiredState` on the existing DesiredGeneration channel / one datastore +
  per-customer namespaces / PBS ON the VM, relay-through-DooPlex REJECTED as non-scaling),
  the Day-0 join handshake, robustness set (WG-roaming-not-DynDNS, DNS re-resolve watchdog, MTU
  1420, per-/32 isolation, tunnel-health via storage-target model), trust conformance (outbound
  only; operator-inbound + game-ingress out of scope), the honest ledger (CGNAT still unmeasured
  — mobile-hotspot smoke closure; peer-sync push-vs-pull = slice-1 design point), and the S1–S6
  slice roadmap (**MVP = S1 endpoint + S2 hub schema/desired-state + S3 agent WG service, then S4
  PBS-over-tunnel**; S5 merges with DR-completeness). day0-install backlog line now cites spike +
  design doc. Docs-only; slices are separate future tasks.
- **2026-07-03 — OFFSITE-BACKUP TRANSPORT DECIDED (connectivity spike, GO at the lowest rung)** —
  plain WireGuard, host-side, box-dials-out to a small public endpoint; **no Headscale/DERP needed
  for backup** (separable fleet layer, adoptable later). Live-proven on demo-felhom → throwaway
  Hetzner: 11.4-min idle hold, PBS API through the tunnel with the exact-cert pin intact
  (positive + negative), real 2 GiB worst-case backup at 4.26 MiB/s (= the home uplink; tunnel
  overhead ~5%), UDP 51820 *and* 443 both pass. P0 honesty: the operator's line is plain-NAT
  (fixed public IP, One Hungary), NOT CGNAT, and has zero IPv6 — CGNAT confirmation deferred to
  Peti's VM 110 (second ISP). P7 surprise: kernel WG *works* inside the unprivileged guest (no TUN
  needed) — host placement stands on architecture (backups originate host-side; cloudflared
  precedent), not infeasibility. Evidence + architecture recommendation (endpoint VM shapes, key
  custody via IdentityBundle escrow, per-peer /32 isolation):
  `documentation/audits/SPIKE-connectivity-wireguard-2026-07-03.md`. **This unblocks the
  DR-completeness work: next task = production connectivity spec** (endpoint VM, agent-managed WG
  service, hub peer registry, escrow join). Throwaway Hetzner box parked powered-off (secrets
  shredded); demo-felhom verified residue-free.
- **2026-07-03 — 4th skill: `felhom-app-catalog`** (catalog authoring workflow; points at
  app-catalog REUSE.md §1–2, no duplicated tables; auto-discovered by `install_skills.py`, 4/4
  found fresh-session). Proven by finalizing SparkyFitness live on demo (image-inspected
  healthchecks, probe-container naming rule discovered + REUSE row added, UI-applied, controller
  probe healthy). App-count drift recorded: website says "45+", per-category spans sum 49, real
  catalog 53 — website fix is a separate task.
- **2026-07-03 — GOLDEN 0.98.3 LIVE + VOUCHED; guide D.1b RETIRED (drill B1+B5 FIXED)** —
  `build-golden.sh` v2.0.0 (mandatory controller tag + baked bootstrap `.path` unit); golden baked
  with controller 0.98.3, clean-room validated (drill VM) through all four scenarios incl. the
  published-artifact `--force-gitea-golden` path, published + operator-vouched (Day-0 manifest now:
  agent 0.63.0 + golden 0.98.3). Fresh installs land CURRENT and self-manage — D.1b is a one-line
  check; the prior "vouch agent 0.63.0" operator follow-up is DONE. Evidence:
  `documentation/audits/DRILL-golden-098-2026-07-03.md`. NEW operator follow-ups: (1) SECURITY —
  the customer-config `git.token` is a Gitea admin token with package-WRITE (proven during the
  bake); scope down + rotate. (2) optionally raise the global controller floor to 0.98.3.
- **2026-07-03 — drill findings B2+B3 FIXED (agent v0.63.0, live on felhom-pve + published)** —
  token reload-on-miss (no more fresh-install 401/manual restart) + guesthook snippets-dir mkdir.
  Guide D.1b restart step narrowed to "agent < v0.63.0"; drill ledger updated. **OPERATOR: vouch
  agent 0.63.0 in the Day-0 manifest UI** (until then fresh installs fetch 0.62.0).
- **2026-07-03 — GO-LIVE PACKAGE SHIPPED (host-install v1.9.1 + day0-install runbook + clean-room drill)** —
  `documentation/runbooks/day0-install.md` is the operator Day-0 guide, **every command executed
  verbatim** on a throwaway nested PVE 9.2.2 (build server): install → verify → app deploy →
  uninstall → **zero-residue diff** → reinstall. v1.9.1 = residue-free uninstall (agent config w/
  hub api_key, shared-parent unit/script/mount, mkfs wrapper, hook snippet, dnsmasq snippets) +
  post-provision guest reboot (without it the controller never deploys on slower hardware) + header
  sync. Evidence + go/no-go for Peti (**GO**, 3 install-day items):
  `documentation/audits/DRILL-day0-cleanroom-2026-07-03.md`. Key OQ-2 finding: golden bakes a
  pre-floor controller 0.85.1 → **fresh installs need the one-time D.1b update** (agent restart →
  update button → 0.98.3); follow-ups recorded: rebuild golden ≥0.86.0 + re-vouch (operator), agent
  `mkdir -p` for the guesthook snippet dir, fresh-install local-API 401 (pre-existing), hub has no
  host-delete path. Drill VM parked virgin-snapshotted on 180 (`/mnt/5_hdd/felhom.eu/drill/` — moved off the SSD 2026-07-18).
- **2026-07-03 — A1 CLOSED (host-install v1.9.0 + agent v0.62.0)** — `Pool.Audit` added to
  `FelhomAgentGuest`; the agent's stale-lock reaper now intersects its scan with
  `GET /pools/felhom` membership (fail-safe skip on read failure). Spike:
  `documentation/audits/SPIKE-a1-pool-membership-read-2026-07-03.md`. Deploy order on any box:
  rescope ACL first, then the agent. STILL PENDING (operator): rotate + de-git the 3 committed
  secrets in `manifests/felhom.secret.yaml` (spike appendix; REUSE §2 errata added).
- **2026-07-03 — hub v0.31.0 LIVE** (critical-severity ingest fix + UI badges; ArgoCD Synced/Healthy;
  live probe: critical event stored + operator email delivered).
- **2026-07-03 — Felhom skills exist**: `skills/{felhom-build-deploy,felhom-ui-design,felhom-testing}`,
  installed via `scripts/install_skills.py` — **junction mode** (repo edits live immediately; no
  re-install needed after editing). Fresh-session discovery verified.
- **2026-07-03 — CLAUDE.md refreshed** (all four repos): no version-pinned state in CLAUDE.md —
  current state lives here / CHANGELOG / REUSE.md.

- **2026-07-03 — `REUSE.md` exists at the repo root** (hub helpers / website conventions / gate scripts / manifest patterns, code-verified); maintenance rule active: update it in the same commit that changes a shared helper. `scripts/reuse_refs_check.py` machine-checks its citations across all four felhom repos.