REPORT: record the pushed commit hash

The four existing skills cover the product; nothing covered how work is
reported. Two rules this project has paid for — check the artifact rather
than the report, and do not state a claim more firmly than the evidence
allows — lived only in the operator's head and in chat, where Claude Code
never read them.

- felhom-evidence      five confidence tiers, artifact-over-report
- felhom-diagnosis     no hypothesis until a command has been seen red
- felhom-plain-language ASD-STE100, two options, the re-pitch
- felhom-handoff       the note goes to a FILE, not the conversation
- felhom-doc-authoring the pointer decides whether material is reached

scripts/check_skills.py asserts what decides whether a skill is EVER
reached: frontmatter parses, name == directory, description and body
non-empty, under 150 lines, installed copy still samefile()s into the
repo. install_skills.py globs and never reads the file, so a missing
description installs perfectly and then silently never loads.

It convicted on its first run: felhom-build-deploy is 179 lines. NOT
trimmed here (pre-existing skills are out of scope, and trimming a
deploy skill without exercising its commands is how a wrong command
reaches a live host) — a named single-entry GRANDFATHERED exception,
WARNed every run, R-394. A new skill over the limit is convicted.

Red-proof run and seen failing: description removed from
felhom-evidence -> exit 1, "frontmatter field 'description' is missing
or empty". Restored, tree clean.

skills/SOURCES.md records both MIT upstreams, that these are adaptations
not copies, and the six pieces deliberately EXCLUDED with reasons.

Register: R-392 (no architecture doc covers the two-AI workflow),
R-393 (decision-log skill deferred, with the reason), R-394.

CLAUDE.md

@@ -83,10 +83,14 @@ box**.
 
 **Run `python3 scripts/repo_gates.py` after ANY change in this repo.** It runs every gate —
 `site_gates.py`, `hostinstall_gates.py`, `hub_confirm_gate.py`, `manifest_bearer_gate.py`,
-`reuse_refs_check.py` and `instructions_gate.py` — streaming each gate's own output and exiting
+`reuse_refs_check.py`, `instructions_gate.py`, `golden_currency_gate.py`, `wire_contract_gate.py`,
+`hub_copy_gate.py` and `due_checks_gate.py` — streaming each gate's own output and exiting
 non-zero if any fails. `--fast` selects the gates that touch no network and no container runtime;
 today that is all of them. **A missing gate script is a FAILURE, never a skip.**
 
+`due_checks_gate.py` refuses the push when a dated check in `OPEN-ITEMS.md`'s `DUE-CHECKS` block has
+come due (R-341). **It is not a scheduler** — it fires on the next push, not on the date.
+
 `site_gates.py` is a *gate*, not a runner — do not model new work on it;
 `app-catalog-felhom.eu/scripts/catalog_gates.py` is the canonical runner (R-161).
 
@@ -125,6 +129,13 @@ something, not only sessions that touch `documentation/` — which is why it is
   the operator in plain language, and deliberately **not** `CONTEXT.md`.
 - **The capability map** (`documentation/architecture/00-capability-map.md`), if a capability's
   status changed — with its new evidence citation.
+- **`python3 scripts/unproven.py --summary`** — one line per status, and the not-walked total. Run it
+  at the end of any session that shipped, broke or proved something, and **say in the report if a
+  number moved**. It exists because "which claims are unproven?" was answerable only by a person
+  reading a page: a session asked for "the nine grey claims" could not determine which nine and
+  rightly refused to guess (R-326). *Nine was real and answered a different question — it is the
+  count of claims the 2026-08-09 pass DOWNGRADED. Not-walked is 32 of 55.* A status that moves
+  without anyone noticing is how the picture stops being true.
 - **Confirm your own last push's CI run went green, by run ID.** CI emails on failure, which is a
   PUSH signal; this is the PULL check that catches a lost, filtered or unread mail. Quote the run id
   and its conclusion, e.g.

CONTEXT.md

@@ -14,6 +14,451 @@
 > language, one screen, no identifiers in the prose. Same subjects, different readers; merging them
 > would make one of the two audiences stop reading. `STATUS.md` is also a **view of `OPEN-ITEMS.md`**
 > and holds nothing of its own; this file does hold its own content, namely the standing rulings below.
+## Skills cover the PROCESS domain too, from named MIT sources with named exclusions (2026-08-25)
+
+**[RULING] `felhom.eu/skills/` now holds two kinds of skill and the distinction is deliberate.** The
+four originals — `felhom-app-catalog`, `felhom-build-deploy`, `felhom-testing`, `felhom-ui-design` —
+are **product-domain**: what the system is and how to change it. The five added 2026-08-25 —
+`felhom-evidence`, `felhom-diagnosis`, `felhom-plain-language`, `felhom-handoff`,
+`felhom-doc-authoring` — are **process-domain**: how work is judged, diagnosed, written up and handed
+over. The gap they close is that two rules this project had already paid for lived only in
+conversation, so Claude Code never read them.
+
+**The process skills are ADAPTATIONS of two MIT collections (`mattpocock/skills`,
+`backnotprop/pstack`), not copies, and neither repo is installed, vendored or depended on.** Six
+pieces of upstream material were excluded on purpose — most importantly *"stop asking and proceed on
+reversible work"*, which reasons from *code is cheap and revertible*; Felhom's work reaches live hosts
+and one real customer's data, where that premise is false. **The exclusions and their reasons live in
+`skills/SOURCES.md`**, so a session that finds the upstreams sees a decision and not an oversight.
+
+**One rule, one home — this is load-bearing, not tidiness.** A rule that already lives in another
+skill or in `documentation/runbooks/workspace-CLAUDE.md` is POINTED AT by name, never restated: two
+copies drift and the reader cannot tell which is current. So `felhom-diagnosis` sends you to
+`felhom-testing` for the red-proof, and `felhom-evidence` cites standing rules 2 and 3 in one line
+each. **`skills/felhom-doc-authoring/SKILL.md` is where that rule and the 150-line limit are written
+down** — read it before adding or editing any instruction file.
+
+**Validation: `python3 scripts/check_skills.py`.** `install_skills.py` globs `skills/*/SKILL.md` and
+never reads the file, so a skill missing its `description` installs perfectly and then silently never
+loads. The glob is the registry — **a new skill directory needs no registration, and no manifest or
+index may be added.** What the checker CANNOT see is whether the model actually reaches a skill when
+it should; that is behavioural, decided by the `description` wording, and it is checked by the
+operator typing a probe phrase, never claimed from a green run.
+
+## Cooldown GRAIN is allow-listed, never inferred from the payload (2026-08-23, R-389)
+
+**[RULING] Per-app cooldown is a NAMED REGISTER (`perAppCooldownEvents`), not a rule of the form "if
+the details carry a `stack_name`, split per app".** The backup family's cooldown is coarse **on
+purpose** — R-97a and R-182 exist so one full disk sends one digest rather than one mail per app.
+
+**[FACT] The payload-shape rule would have been wrong, and provably so.** `crossdrive_failed` is
+severity `error`, reaches the operator leg, and carries `stack_name` through **`CrossDriveDetails`**,
+a different struct from `AppDetails`. Anything keying off the field would have split it silently.
+**That is why `cooldownStackSuffix` takes the EVENT TYPE as well as the details** — an asymmetry with
+`cooldownTierSuffix`/`cooldownRunSuffix`, and the asymmetry IS the safety property: `tier` and
+`run_id` appear only on types that want that grain; `stack_name` does not have that property.
+
+**[RULING] `app_start_failed` gets per-app because it has NO DIGEST.** There is no `apps_down_run`
+summarising a scan the way `backup_run_failures` summarises a run, so per-app is the only grain that
+does not lose alarms. **The hour did not change** — the grain was the complaint, not the length.
+
+**[FENCE] The fenced act is adding an entry to `perAppCooldownEvents` for a type whose family has a
+digest or a deliberately coarse cooldown.** Reading it is fine.
+
+**[MEASURED] Burst volume, so it is a number and not an impression:** three apps in one scan →
+**3 attempted, 3 sent, 0 suppressed**. Reference box has 8 deployed apps, so a total outage is 8
+mails. It scales linearly and has no ceiling; the reopening condition is a box big enough that a
+total outage is unreadable, and the answer there is a digest, not a wider cooldown.
+
+## A finding recorded only in REPORT.md has a lifetime of ONE SESSION (2026-08-23, R-389 / gate 11)
+
+**[FACT] This cost a day.** The cooldown-grain defect was measured live on 2026-08-23, written under
+`## Observations` in `REPORT.md`, and written **nowhere else**. `REPORT.md` is overwritten every
+session by this project's own convention. It had no register row and had to be re-derived the next
+day. Same shape as R-341.
+
+**[FACT] The instruction invited it.** `PROMPT-TEMPLATE.md` §15 asked for observations *"documented,
+NOT acted on"*, and "documented" was satisfied by the paragraph. **The template was the defect**, and
+it was corrected in the same session that built the mechanism.
+
+**[MECHANISM] Gate 11 (`scripts/observations_gate.py`).** Every numbered item in a `REPORT.md`
+observations section must carry `FILED: R-NNN` (which must resolve) or `NOT-A-FINDING: <reason>`. It
+REFUSES; it does not warn. It PASSES quietly when there is no observations section, deliberately — a
+gate that taxes every push gets disabled within a week.
+
+**[RULING, and it contradicts the task that commissioned it] A bare mention of an R-number does NOT
+satisfy the gate.** The specification said an item may "cite an R-NNN that resolves" — **that rule
+would have passed the very item the gate was built to catch**, because the lost item cited `R-182` as
+an *analogy*. No parser can tell citation-as-precedent from citation-as-filing by reading prose. The
+marker is explicit for that reason, and the discrepancy is recorded in the gate's docstring rather
+than quietly resolved.
+
+**[GOTCHA] Registered in three runners, not four.** `catalog_gates.py` has no shared-gate mechanism —
+`run_gate` joins against its own `scripts/` and the loop appends `--all` to every gate — so
+registering there needs its contract widened. Filed as **R-391** rather than left as a sentence,
+which is this session's whole lesson.
+
+## The severity a controller sends is the HUB's vocabulary, and getting it wrong deletes the alert (2026-08-23, R-329 / R-387)
+
+**[RULING] The set is exactly `{info, warning, error, critical}`.** The hub **coerces anything else to
+`info` at ingest**, and `severityNotifies` drops `info` **before both** delivery legs. So a severity
+outside the set means: stored, `200` returned, dashboard shows it, **e-mailed to nobody**.
+
+**It shipped twice** — `DiskAlertKind.Severity` until controller v0.215.0, `app_start_failed` until
+v0.223.0. Measured on the live hub DB: **91 `app_start_failed` events stored all-time, ZERO
+`notification_log` rows** before 2026-08-23.
+
+**[MECHANISM, because a comment already recorded this lesson and it happened again]** An AST walk over
+the whole controller pins every emitted severity. **grep cannot do this job** — `"warn"` is a
+legitimate *healthcheck status* in `internal/monitor` and `internal/selftest`; the 2026-08-23 sweep hit
+nine such strings and exactly one defect. The walk cannot follow a variable, so the six call sites that
+pass one are **registered by name with the values each can take**; a new dynamic site fails the test.
+**An unlisted limit is not a limit, it is a hole.**
+
+**[RULING] The hub's coercion STAYS and now SPEAKS (hub v0.107.0).** A rejected event is a *lost*
+event, and losing an alarm is worse than mis-routing one — that is why it was a coercion. The fix is
+that it logs `WARN` naming customer, type and value. **The guard that should have caught this class sat
+downstream of the rewrite and was structurally blind to it** — note the asymmetry it lived beside: an
+unknown event *type* 400s loudly, an unknown *severity* was silent.
+
+**[RULING] `dispatcher.go`'s `unrecognized severity` branch is KEPT, on evidence.** `cmd/hub/main.go`
+wires `ProcessEvent` **directly** as the `monitor.EventNotifyFunc` for three checkers that never pass
+the ingest handler. For those it is the only severity guard there is.
+
+**[RULING] `app_start_failed`: operator always, customer OFF by default.** `processOperator` consults
+only `operatorOn`, the address and a 1-hour cooldown — **never** customer preferences — so one word
+fixed the operator leg and left the customer leg where the ruling wanted it. It is deliberately **not**
+in `operatorOnlyEvents`: that would make the new toggle visible, flickable and structurally incapable
+of delivering.
+
+**[GOTCHA, now load-bearing] The operator cooldown key carries no app identifier**
+(`customerID:eventType`), so **only the first app-down per hour e-mails the operator**; the rest are
+logged `suppressed`. Harmless while the event was undeliverable. Not any more. R-182's shape, filed.
+
+## "The customer stopped this" is a RECORD, never an inference (2026-08-23, R-386)
+
+**[RULING] Ask `DesiredState`, do not read the container state.** `aggregateState` folds `StateExited`
+into the stopped counter, so an all-down stack returns `StateStopped` whatever killed it — an
+out-of-band stop and a customer's Stop are byte-identical on the Docker side. Measured: `privatebin`
+stopped out of band, nine scans, **zero events, zero banner**, while a comment claimed it "still
+alerts".
+
+**[RULING] The tri-state, and the third value is the whole safety property.** `Stopped` → no alarm.
+`Running` → **alarm**. **Absent → UNKNOWN, keep the old behaviour, AND announce it.** Reading absent as
+"nobody asked" would, on the first cycle after upgrade, e-mail about every app any owner ever
+deliberately stopped — fleet-wide, from a field that predates the intent being asked of it. **The
+backfill cannot help: it seeds `Running` only from an observed-UP reading**, so anything stopped at
+upgrade time stays unknown — exactly the ambiguous population.
+
+**[MECHANISM] `AppRunState.IntentUnknown` + an INFO line naming the apps**, on the heartbeat cadence.
+An operator must be able to ask *"how many apps am I blind to?"* and get a number. **A rule without a
+mechanism is a wish.** Measured on `demo-hp`: **0 of 8** deployed apps had an absent intent.
+
+**[FENCE] Adding a `DesiredState` WRITER is the fenced act; reading it anywhere is fine.** Twelve of
+`StopStack`'s fourteen callers are machines, so recording intent in the primitive would make a nightly
+backup indistinguishable from the customer pressing Stop.
+
+**[FENCE] `failedRestart` must still lift a `Stopped` intent** — the quiesce loop stops stacks by the
+same path a customer does, and one it stopped and could not restart must alarm whatever the intent
+says. Removing that term re-opens F-CRIT-1.
+
+## Two settings toggles each governed two alarms, and the labels named one (2026-08-23)
+
+„Lemez figyelmeztetés (90%+)" also wrote `disk_critical` — the drive-is-**failing** alarm. Split into
+four honest toggles; **12 → 15**. **[RULING] The migration, not the split, was the risk:** a save whose
+event **set** is unchanged now stores the **existing slice verbatim**, so byte-identity is by
+construction. Without that guard the defaults case reorders — the red-proof caught it. Legacy compound
+form names are still read, so a stale browser tab cannot drop a key.
+
+## Customer notification model — DIRECTION, recorded and deliberately NOT implemented (2026-08-23, R-388)
+
+The operator's framing, verbatim: *"A customer should be notified only about things they can act on or
+are responsible for… **A failed backup is our incident, not theirs.**… The subscription should feel
+like being looked after, not like being on call."* Full entry at
+`documentation/architecture/08-alarm-ladder.md` §8, marked **not current behaviour**. The settings page
+grew 12 → 15 toggles in one session and grows by one per detector — that growth is the argument.
+**Nothing in this session implements it.**
+
+## App data placement is a DECISION, and it had never been written down as one (2026-08-22, R-376)
+
+**Recorded here on 2026-08-22 to give an existing decision a home. It is NOT new, and this entry is
+not its date** — the choice is older and its original date is not on record. That is itself the
+finding (R-376).
+
+**The decision.** App data is placed **per volume, not per app**. `.felhom.yml` classes each volume
+**hot** — database, config, cache — which goes to fast storage inside the guest and is **enforced**;
+or **bulk** — media and files — which may live on slow attached storage. A photo app's database stays
+on the SSD while its blobs go to the USB drive. Recorded shape:
+`architecture/01-topology-and-trust.md:150-152`, now marked **[DESIGN]**.
+
+**What follows from it, and is a consequence rather than a separate choice:** 40 of the 53 catalogue
+templates declare no configurable data path, because they are all-hot apps. The 13 that do are the
+media and library apps. Stated as **[FACT]** at `07-backup-architecture.md:296-299`.
+
+**What was rejected, so the same proposal does not return.** Moving named-volume (all-hot) app data
+onto the default data drive was proposed in `SPEC-app-data-placement-2026-08-21.md` §5 and is
+**rejected**: it would move hot data onto storage this design classes as possibly-slow, and would put
+it outside the guest vzdump that currently protects it (`01-topology-and-trust.md:154-156`). The spec
+carries the correction inline.
+
+**Why this entry exists at all.** The decision was real, implemented and load-bearing, and lived in a
+single unmarked bullet. Between 19 and 22 August it was called a defect in four places (R-370),
+because a reader met a marked `[FACT]` beside an unmarked choice. **A decision nobody wrote down as a
+decision is one somebody will eventually report as a bug.**
+
+## Read the architecture folder BEFORE calling something a defect (2026-08-22, R-370 / R-369)
+
+**Between 19 and 22 August the reviewing side called a documented architectural decision a defect, in
+four places.** The decision is in `architecture/01-topology-and-trust.md:150-152`: `.felhom.yml`
+classes each volume **hot** (DB/config/cache → fast storage, **enforced**) or **bulk** (media/files →
+may be slow). The 40 catalogue templates with no `HDD_PATH` are all-hot apps — their data is app
+internals and belongs inside the guest, on its own volume so it cannot fill the OS
+(`felhom-agent/configs/build-golden.sh:29-40`). The 13 with a path are the bulk apps. **There is no
+choice being denied**, and the deploy page has been telling the customer exactly this all along:
+*„A kiválasztott meghajtón az alkalmazás fájljai (média, dokumentumok) tárolódnak. Az adatbázis a
+gyors belső SSD-n fut"* (`deploy.html:624-625`).
+
+**Why it happened, mechanically: the register and live source were read; `documentation/architecture/`
+was not.** Nothing in `PROMPT-TEMPLATE.md` §4 required naming the architecture document for the area
+being touched — item 4 named only the controller module map, and the S-1 rule governs *updating* a
+design doc at the end, not *reading* one at the start. **Fixed in the same session:** §4 now carries
+the file→area map, the ordering *architecture = reasoning, register = work, source = truth*, and the
+test **"is what I am about to call a defect something we chose?"**
+
+**Corrected in place, not rewritten** — `SPEC-app-data-placement-2026-08-21.md` and R-352 carry
+`[CORRECTED 2026-08-22]` blocks that say what they got wrong; a document that quietly changes its mind
+teaches nobody.
+
+**And the sibling failure, R-369: there are two registers.** `OPEN-ITEMS.md` calls itself the single
+source of truth; `ROADMAP.md` also holds open work. 72 `R-` ids live only in ROADMAP, 29 of them not
+marked done. **R-107 — the off-site restore never unpacking its own volume tars — was filed there on
+2026-07-28, one day after OPEN-ITEMS was created, and never entered the register. It was rediscovered
+from scratch 25 days later by the 2026-08-21 drill and shipped as R-354.** The template now says
+plainly that a ROADMAP row alone does not satisfy the filing rule.
+
+## Box REACHABILITY is a separate signal from box FILL — and the cadence difference is deliberate (2026-08-18, R-339, hub v0.106.0)
+
+Both off-site checkers now carry two independent signals, and conflating them is the mistake to avoid:
+
+- **FILL** — how full is the store. Escalation-only (one emit on a band rise), and **a degraded or
+  missing reading drives NO band transition**, because missing ≠ 0%. That rule is unchanged by
+  v0.106.0 and must stay unchanged: it is what stops a dead endpoint from reading as an empty one.
+- **REACHABILITY** — can the hub see the store at all. Counted in consecutive failed fetch windows,
+  emitted past a threshold (default 3 ≈ 30–45 min) with a paired all-clear.
+
+**Why reachability REPEATS rather than escalating once.** `emitFill`/`emitOversub` fire once on a band
+rise and stay silent while the condition persists, which is right for a capacity trend. Applied to
+reachability it would produce exactly ONE mail at roughly minute 30 of a nine-hour outage — and one
+mail is missable, which is the whole failure being fixed. So the unreachable event fires on every
+failed window past the threshold and relies on the dispatcher's 1-hour per-type operator cooldown to
+become an hourly "still blind" heartbeat. **If you find yourself "fixing" this back to the band shape,
+this paragraph is why not.** The cost is a `suppressed` notification row every ~15 min during an
+outage — honest bookkeeping.
+
+**Two boundaries worth holding in mind.** `ErrUsageUnsupported` is *not* blindness — an ep0 on an old
+tenantsync answers "no such op", which means we reached it; the counter is not advanced, or the alert
+would fire for days on a healthy pre-update box. And the reachability read rides ep0's **local API
+daemon**, not the HTTPS proxy on 8007 — so it would have shown green throughout the 2026-08-18 outage,
+which is R-340 and is the honest limit of what this watches.
+
+## The managed floor now tracks the vouched golden — and that is rule 2 ARRIVING, not an exception (2026-08-18, R-343)
+
+Since 2026-08-18 12:36:58Z the global managed controller floor
+(`hub_settings.min_controller_version`) reads **0.216.0**, equal to the vouched golden. Read it from
+the store (`GetGlobalMinControllerVersion`, `hub/internal/store/store.go:1751`), never from the form.
+
+**Do not read the preceding gap as drift.** `publish-train-rules.md` rule 1 is *manifest before
+floor* and rule 2 requires the floor field saved **LAST, in a separate save**, because the DB row
+overrides the env floor and acts on the next report cycle. The floor therefore sits behind the newest
+controller **by design** for as long as it takes to bake and vouch a golden — and it is raised
+afterwards. Floor-equals-golden is the state that ordering is meant to arrive at; a session finding
+the floor behind should check whether a vouch is outstanding before calling it a defect.
+
+**The raise is not free, and this is the part worth remembering.** Rule 2's "acts immediately" is
+literal: nine seconds after the save, `demo-felhom` auto-updated 0.214.0 → 0.216.0
+(`controller_updated` 12:37:07Z, `controller_started` 12:37:12Z, no error events). So a floor raise is
+a fleet action, not bookkeeping — plan it as one. `ResolveManagedFloor`
+(`hub/internal/store/store.go:2068`) is what makes it safe: it holds the floor entirely when it
+exceeds the golden (R-216), and per-box when that box's agent is below the manifest's `MinAgent`.
+
+## A name must differ from its neighbours on a stem, not on a noun (2026-08-13, R-323)
+
+The third near-homograph is renamed: the five-word phrase that proves an account owns the box being
+bound is „**Tulajdonosi jelmondat**". It was „Visszaállító jelszó" — one word from the „Visszaállító
+kód" retired days earlier, and false besides: **the phrase restores nothing.**
+
+**The rule this instance yields, and it generalises past this secret.** Both obvious replacements were
+rejected, and the reasons are the transferable part:
+
+- **Do not name a secret after the ACT it is used in, when another secret on the same screen is named
+  after the same act.** The other factor on that page is the „**Párosító kód**". „Összekötési jelszó"
+  would have left the two things a customer types in one sitting separated only by *kód*-versus-
+  *jelszó* — **structurally the very trap being removed**. A name that matches the page's verb feels
+  right and is the wrong axis.
+- **Do not name a secret after a THING that already has a password.** „Fiókjelszó" collides with the
+  dashboard login, i.e. trades one homograph for a worse one.
+- **Separate on the STEM first, the noun second.** Beállító / Helyreállítási / Párosító / **Tulajdonosi**
+  — four distinct stems; *kód* / *jelmondat* is the second axis, not the first. A pair distinguished
+  only by its noun is what failed twice.
+
+**Naming is not function, and the pin says so** (`TestSelfBindPassphrase_StillAcceptedAfterTheRename`
+drives the real handler with human-messy spacing). **Nothing printed was stranded** — no customer
+document names the phrase; it is delivered out-of-band, so the description in the hint sentence is
+what identifies it, which is why that sentence was kept and extended rather than trimmed.
+
+## A guard is worth what its positive control is worth (2026-08-13, R-324)
+
+The hub's customer copy came under a guard for the first time — every customer e-mail and both binding
+pages, the sentences a customer reads *before* they have seen any box screen. Nothing had ever looked
+at them, in either repo.
+
+**The part worth carrying: the selftest found a defect in the guard itself on its first run.** The
+plant→convict→remove→pass control convicted its own step-3 case (a phrase inside a comment) because
+the synthetic source was named `<selftest>` and comment-stripping keys off the `.go` extension. **A
+guard that had shipped without a control would have been quietly over-broad**, and the first person to
+write an honest explanatory comment would have been convicted by it.
+
+**And on sharing a word list across two repos:** the vocabulary lives once, in `scripts/`, the same
+home both repos already consume gates from. The consumer that could not be edited that evening is
+**drift-checked rather than duplicated** — the gate reads the other gate's literal and fails on
+divergence. That is a scaffold with a register row (R-325) to remove it, not a design; **two copies of
+a word list is the R-299 defect waiting to happen**, and a drift check is the cheapest way to refuse
+the copy without making a cross-repo edit at the wrong moment.
+
+## Suppress an alarm on the machine's own declaration, never on an inference — and check for the second door (2026-08-13, R-321)
+
+A box whose reporting is deliberately switched off announces it (`health.status = "disabled"`) and
+then goes quiet by design. The hub stored that, rendered it, and **alarmed on it anyway**, because the
+staleness verdict was computed from report AGE alone.
+
+**Three things worth carrying:**
+
+1. **The discriminator must be the box's own last word.** That is what made this a suppression rather
+   than a guess — and the standing rule is right that suppressing on a guess is worse than the false
+   alarm it removes.
+2. **RECORD the state; do not delete it.** The `blocked` precedent deletes, and `GetState` then
+   returns `""` — which is not `"down"`, so **`CheckBackupDeadlines` would have gone on alarming from
+   a second function.** One fix, two doors; R-195's shape returning through the one nobody looked at.
+   *Ask of any suppression: which OTHER checker reads this state, and what does it see now?*
+3. **A clock must restart at the event that ends the suppression**, not at the last observation before
+   it began. Timing from the pre-disable report would fire an instant stale/down for a quiet period we
+   requested. Re-entering the new-customer branch gives a first observation instead of a recovery
+   event.
+
+**The limit is stated in the code, not hidden:** a box re-enabled that then fails to report keeps
+being suppressed, because the hub's view changes only when a report arrives. **That is why the state
+is made visible** — an operator who re-enabled a box and still sees `disabled` is being told it has
+not come back.
+
+**And a test lesson:** red-proof 2 revealed that scenario A passed on its state assertion alone even
+with the suppression deleted, because it seeded the customer *already* disabled and the new-customer
+branch swallows the first observation. **A test that cannot see the alarm it exists to prevent is not
+a test** — it now observes the machine healthy first.
+
+## A number everyone repeats is a claim, and claims get checked (2026-08-13, R-326)
+
+"The nine grey claims" was repeated across sessions and **exists nowhere**. A session asked to report
+on them could not determine which nine and declined to guess; that refusal was correct and is the
+finding.
+
+**Nine is real and answers a different question: it is the count of claims carrying
+`verdict: downgraded`** — the ones the 2026-08-09 pass *lowered*. The count of unproven claims is
+**32 of 55** (`walked` 23, `partial` 14, `built` 14, `missing` 4), and of those 32 only **6** cite an
+evidence document.
+
+`scripts/unproven.py` makes it a command, wired into the end-of-session checklist. **The generalisable
+part: a picture built for a person to read is not a picture anyone can check.** The dataset carried the
+statuses all along; what was missing was one command that answers the question the operator actually
+asked. Its first run found a stale claim (R-327) — which is the argument for having built it.
+
+## A fact the boxes send and the hub cannot read is a future false green — and one now has a reader (2026-08-13, R-319)
+
+**The first of R-264's twenty-one unconsumed facts is read.** The agent emitted `guest_net` on every
+heartbeat for **twenty-three days** (agent v0.92.0, R-54, 2026-07-21) while the string occurred
+**nowhere** in `felhom.eu/hub/` — stored as raw text in `report_json`, reaching no check, no alarm and
+no screen. Hub v0.104.0 models it: `parseGuestNet`/`guestNet` in `web/hosts.go`, rendered as the
+host-detail **Guest network** card.
+
+**The rule this instance teaches, and it generalises past guest networking:**
+
+- **Read the fact that carries the RISK, not the fact that is easiest to render.** Here that is
+  `heals_last_hour`, not `state`. A guest the watchdog keeps repairing is healthy at every instant
+  anyone looks and is nevertheless failing — rendering the state alone gives it a green tick, which is
+  the failed-disk-drawn-as-a-healthy-empty-disk shape wearing different clothes.
+- **Decode the field that decides the question, not five of the eight.** `heal_succeeded` was added
+  mid-build for exactly this: six FAILED repairs is a guest that is down, six successful ones is a
+  nuisance, and the count alone cannot tell them apart. **R-260 is the standing warning** — a hub
+  decoder three fields short of the agent dropped the one its checker existed to answer.
+- **Three absences, three sentences.** Agent too old / capable agent silent / guest state not asserted.
+  All render unknown; conflating them is how an operator starts ignoring a card. The state switch is an
+  **allow-list**, so adding a state to the agent can never silently paint it green here.
+- **REMOVING an allowlist entry is what proves a reader is wired.** An allowlisted tag is *skipped* by
+  `wire_contract_gate.py`. Re-labelling the eight `guest_net` entries would have left the new reader's
+  own fields unchecked while the gate reported coverage. Deleting them moved checked tags **182 → 190**
+  and skipped **88 → 80** — the positive control that the wiring is real, independent of any test.
+- **A card is not an alarm, and building the card first is deliberate.** No email was added: the
+  incident behind this (a killed `dhclient`, tunnel down 1 h 15 m, nobody told) was a **visibility**
+  failure, and with zero repairs ever recorded on either demo box any alarm threshold would be invented.
+  **The visible count is what will supply the threshold.**
+
+**Where the twenty stand** (R-264, counts measured from the allowlist rather than estimated):
+**8 read · 5 deliberately not consumed · 1 redundant · 6 still owed a reader.** The gate grew a **third**
+entry kind — `not consumed, deliberately` — because an undecided fact and a decided one must not read
+alike, and *"arguably owed"* had been carried for five days as though it were a decision.
+
+**Two findings came out of deciding the last one rather than from building anything:**
+
+- **R-321** — `reporting_disabled` is *redundant* (`health.status = "disabled"` travels beside it, is
+  decoded, and IS rendered), **but `StalenessChecker.Check` is age-only** (`monitor/staleness.go:88+`,
+  sole skip `IsCustomerBlocked`), so a deliberately-silent box still alarms `node_stale` then
+  `node_down`. **Decoding the flag would have felt like progress and left the alarm firing** — the fix
+  belongs in the checker, which already holds the status it needs.
+- **R-322** — `retrieval_promise_gate.py` has **never scanned the hub**, which composes every customer
+  e-mail, i.e. the copy read *before* any box screen. A hand scan with its own four stems returns zero,
+  so it is a scope gap rather than a live defect — recorded before someone writes the first hub-side
+  retrieval promise assuming the guard has them covered.
+
+## One secret in two situations keeps its NAME and changes its SENTENCE — and the mail must name the page the machine shows (2026-08-13, R-295 hub half)
+
+The box side shipped 2026-08-10; the hub half was dropped twice and was the last place „Visszaállító
+kód" survived — in the **mails**, the one surface a customer reads *before* they see any screen. It is
+now „**Beállító kód**" everywhere; the ten-word escrow code stays „**Helyreállítási kód**"; no
+dashboard-code mail names both.
+
+**The mechanism worth carrying: the mail names a PAGE, and which page depends on state only the hub
+knows.** `ReissueForReenroll` sent the *reset* mail, directing the customer to „Elfelejtett jelszó" —
+but a rebuilt box has no password, so the controller computes `reset := s.authEnabled()` → false
+(`felhom-controller/controller/internal/web/claim.go:279`), renders „**A szerver beállítása**", and
+serves **no login page at all**. The named route was not on their screen. **The two situations are
+distinguishable only at the hub, because the hub chose which call site fired** — so the fix is a new
+`EmailKind` (`reenroll`), not a reworded shared template. A `default` branch falls back to the
+first-setup mail, which is the safe direction and is pinned so a future reader knows it was chosen.
+
+**The re-enrol mail deliberately says nothing about apps or backups.** A clean-slate reinstall is
+precisely where that reassurance could be false; `TestFormatClaimEmail_ReenrollPromisesNothingAboutTheData`
+guards the CLAIM rather than one phrasing of it.
+
+**Naming is not function, and the pin says so:** `TestReenrollSplit_ChangesTheMailNotTheSecret` — the
+generation still advances, the stored bcrypt hash still verifies the emailed code, no plaintext is
+persisted. Box-side companion: the controller's `TestResetCode_StillAcceptedOnTheSetupPage`.
+
+## Evidence dies in the INTERMEDIATE revert, not the final one (2026-08-13, R-320)
+
+Twice in three days, on the same box at the same point, a phase's logs were destroyed by a revert to
+`virgin` *between* phases (drill 2026-08-12 Phase A→B; R-316 2026-08-13 Part 1→2). Both times the
+golden-bake runbook's existing *"scp the log OUT first"* was applied to the **final** teardown and not
+the middle one. Both times the conclusions survived on luck.
+
+**Standing rule 5** now: evidence comes off at the end of the phase that produced it, before any revert
+— **the pull is the LAST ACT OF THE PHASE**, not a step to remember later. Four homes:
+`workspace-CLAUDE.md`, `runbooks/target-selection.md`, `RUNBOOK-rehearsal-v3.md`, `PROMPT-TEMPLATE.md`
+report item 8. **And the already-gone case is documented rather than improvised:** say so plainly and
+reproduce the finding independently.
+
+**A corollary this session hit immediately:** the second incident was recorded in `REPORT.md`, which
+every session overwrites — writing tonight's report would have **destroyed the record of a destroyed
+record**. It was preserved to `audits/REPORT-r316-installer-v1.28.0-2026-08-13.md` first. *A durable
+citation may never point at a file whose contract is to be overwritten.*
 
 ## Material retained is not history recoverable — ask the three questions separately (2026-08-12)
 

REPORT-agent-transport-leak.md

@@ -0,0 +1,246 @@
+# REPORT — R-344: the agent's leaked PBS connections, fixed and proven on both boxes (2026-08-20)
+
+**Outcome: the fix works, measured three independent ways; ep0 is back to its baseline of 17 file
+descriptors from 415; and 0.130.0 is now published and vouched.** Both demo boxes run the **byte-exact
+published artifact**. R-347 is CLOSED. Two new findings came out of the release itself — **R-349**
+(the fleet was briefly running a different binary under the same version name) and **R-350** (I printed
+the hub password into the transcript; rotation is your call).
+
+## 1. Confirmed baselines
+
+| repo | in | out |
+|---|---|---|
+| `felhom-agent` | `f17ed11` — v0.129.0 | **`ede49b6`** — 0.130.0, UNRELEASED |
+| `felhom.eu` | `9299f85` | docs + register only |
+
+Both clean and equal to `origin/main` before each build. Agent version in: 0.129.0 on both boxes.
+Out: **0.130.0 on both**, confirmed from the hub, not from the boxes' own `--version`.
+
+## 2. The diff
+
+| file | symbol | change |
+|---|---|---|
+| `internal/httpx/transport.go` | **new package** | `DefaultIdleConnTimeout = 90s`; `NewTransport(tlsCfg, idle)` — fresh transport per call, `<= 0` means **use the default, never "no timeout"** |
+| `internal/httpx/transport_test.go` | new | zero/negative → default; the constant is read off `http.DefaultTransport`; freshness; TLS config preserved |
+| `internal/pbs/client.go` | `Config`, `NewClient` | `IdleConnTimeout` field (tests only); transport via `httpx` |
+| `internal/pbs/client_leak_test.go` | new | Scenarios A/C + the production-default pin |
+| `internal/hub/client.go` | `NewClient` | via `httpx` — **consistency only, did not contribute to the leak** |
+| `internal/proxmox/client.go` | `NewClient` | same |
+| `cmd/felhom-agent/main.go` | `version` | 0.92.1 → 0.130.0 (ldflags default) |
+| `REUSE.md`, `CHANGELOG.md` | — | `httpx.NewTransport` entry; the release note |
+
+Commit `ede49b6` on `main`. `grep '&http.Transport{'` now matches only `httpx` itself.
+
+**A check made before trusting the fix:** `doBody` already reads the body to completion and closes it,
+so the connection genuinely reaches the idle pool. Had it not, the idle timeout would have been the
+wrong fix entirely.
+
+## 3. Tests and the red-proofs
+
+`go build ./... && go vet ./... && go test ./...` → **30 packages, 0 failures.**
+`python3 scripts/agent_gates.py` → **all 4 gates OK.**
+
+The leak test counts connections **server-side** and models what `pbsTargetsFromPVE` does — build a
+client, use it once, drop it. It deliberately does **not** assert `err == nil` or that a field holds a
+value; both were true of the leaking code (the R-224 lesson).
+
+| red-proof | mutation | seen failing with |
+|---|---|---|
+| 1 — the fix | remove `IdleConnTimeout` | *"abandoned pbs.Clients: after 5s the server still holds 5 open connection(s), want 0 (5 dialled in total)"* — the count is in the message, so it cannot be a timeout with another cause |
+| 2 — the worse fix | `DisableKeepAlives: true` | **the leak test PASSES.** Caught only by `TestPBSClient_KeepAliveStillReuses`: *"3 sequential requests over 3 connection(s), want 1"* |
+
+**Red-proof 2 is the load-bearing one: Scenario A alone would have accepted a fix that made the problem
+worse** — no leak, at the price of a fresh dial for every one of ~40,000 daily requests. Both mutations
+reverted, tree re-verified clean.
+
+## 4. §4 quoted, beside the results
+
+> **P1** — *"(i) ep0's fd count falls by ≈194 within seconds … (ii) … ≈194 sockets convert ESTAB →
+> CLOSE-WAIT … (iii) neither — the count barely moves. Then the ownership attribution is wrong and the
+> finding must be withdrawn."*
+> **P2** — *"demo-hp (fixed): ≈ 0 … demo-felhom (control, untouched): ≈ 4 per hour → ≈ 16 over 4 h."*
+> **P3** — *"ep0's overall leak rate should fall from ≈ 200/day to ≈ 100/day while one box is fixed, and
+> to ≈ 0/day after Part 5."*
+
+## 5. P1 — **outcome (i)**, in one second
+
+| | ep0 fd | ESTAB | demo-felhom | demo-hp | CLOSE-WAIT |
+|---|---|---|---|---|---|
+| T-0 `09:15:46Z` | 415 | 398 | 199 | **199** | 0 |
+| T+1s `09:15:47Z` | **216** | **199** | 199 | **0** | **0** |
+| T+60s `09:16:42Z` | 218 | 201 | 199 | 2 | 0 |
+
+**Outcome (ii) did not occur, so it gets no register row.** Not one socket converted to `CLOSE-WAIT`:
+ep0 reaps on peer FIN correctly. That also means the 543 `CLOSE-WAIT` at the 2026-08-18 wedge has some
+other explanation and is **not** evidence of a second defect on the protected machine — a finding in the
+negative, worth the sixty seconds it cost.
+
+Ownership is now proven a **third** independent way: what dies with the process, agreeing with
+`ss -tnp` and with the access-log user agent. At 133 s uptime the fixed box held **0** connections.
+
+## 6. P2 — divergence
+
+**Window 09:15:47Z → 10:17:52Z = 1.03 h. You closed the ≥4 h window early**, so no daily rate is
+extrapolated and none is needed.
+
+| box | agent | start | end | delta | per hour | predicted |
+|---|---|---|---|---|---|---|
+| `demo-felhom` CONTROL | 0.129.0 | 199 | 203 | **+4** | 3.87 | ≈4 |
+| `demo-hp` FIXED | 0.130.0 | 0 | 0 | **+0** | 0.00 | ≈0 |
+
+**The assumption-free statement.** ep0's access log counts the opportunities: each box made **exactly 4
+`/snapshots` and 4 `/version` calls** in the window.
+
+> **control: 4 cycles → 4 leaks. fixed: 4 cycles → 0 leaks.**
+
+**Positive observable (standing rule 3):** a zero leak is equally consistent with "the agent stopped
+working" — it did not; its four cycles are in ep0's log. The boxes' other traffic is near-identical
+(`libwww-perl` 924 vs 926, `proxmox-backup-client` 898 vs 898), so **the only difference between them
+is the binary**. Poisson alone gives P(0 | λ=4) = **1.8%**, which is suggestive rather than conclusive
+and is not relied on alone.
+
+## 7. P3 — the second box, and the backlog clearing itself
+
+`demo-felhom` upgraded `10:18:56Z` on your word.
+
+| | ep0 fd | ESTAB | CLOSE-WAIT |
+|---|---|---|---|
+| T-0 `10:18:55Z` | 220 | 203 | 0 |
+| **T+2s** | **17** | **0** | 0 |
+| settled 10:30–10:35Z | **17–19** | 0–2 | 0 |
+
+**17 is precisely ep0's `t0` baseline** (fd 17, ESTAB 0, 2026-08-18 09:51:22Z), and it returns to 17
+between poll cycles — the "healthy proxy near 20 fds" the incident document named. Predicted ≈0/day
+residual; **observed the baseline itself.**
+
+**A correction, made within the hour it was written.** My STOP 1 report and the first CHANGELOG draft
+said *"does not clear the 388 descriptors already stuck on ep0 — those persist until that proxy
+restarts."* **Wrong.** They were held on both sides; restarting the agents released every one. **ep0 was
+read-only throughout and its proxy PID never changed (551655).** Corrected in the CHANGELOG, the audit
+document and R-344 rather than quietly edited.
+
+## 8. Fleet sanity
+
+Hub reports **0.130.0 on both** boxes. **No `floor held`** line (0.130.0 > golden MinAgent 0.129.0).
+**No `pbsdr_box_unreachable` / `offsite_box_unreachable`** during any window. Positive observable
+rather than the absent one: the PBS-DR gauge kept refreshing (`3.7% full (3.7 GB of 97.9 GB)`) and host
+reports kept landing from both boxes throughout.
+
+## 9. What is NOT done
+
+- **Not published.** No package, no tag, no manifest or floor field touched, no self-update staged.
+  **A box installed from the current image still ships the leaking agent** — **R-347**, your call.
+- The CHANGELOG heading is `## UNRELEASED — v0.130.0 candidate`. The `release-complete` gate convicted
+  on `## v0.130.0` because there is no tag and no package, and **it was right to**. I did not use
+  `--no-verify`; I made the heading stop claiming a release that has not happened. It flips to
+  `## v0.130.0` in the same commit as the tag.
+- Poll rate unchanged (**R-336**, re-scoped). `pbsTargetsFromPVE` not refactored; no
+  `CloseIdleConnections` added. ep0 not touched.
+- **The 388 descriptors ARE cleared** — see §7. This is the one item the prompt expected to remain
+  outstanding, and it did not.
+
+## 10. Register
+
+- **R-344** — updated with the fix, P1's outcome named, and P2/P3's numbers. **Left OPEN**, because a fix
+  on two boxes by hand is not delivered.
+- **R-336 — re-scoped.** Its new next-step cell, verbatim: *"**NEW ACCEPTANCE CRITERION, since the old
+  one is void:** the fd count is NOT the observable for this row any more — that belongs to R-344 and is
+  already satisfied. Measure the REQUEST RATE at ep0's access log, and state the projected rate at the
+  target customer count."* The row now records explicitly that its old next-step **would have "fixed"
+  nothing while looking like a failed fix**, and re-scopes it to what it is: ~85,000 requests/day to a
+  weekly-write DR endpoint, ≈**25 requests/second at fifty customers** against a CX33.
+- **R-347 (new)** — the delivery gap. Owner: **Viktor decides**, CC executes.
+- **R-348 (new)** — an agent restart blanks the reported backup list for up to ~18 h, and the `Store`
+  comment calls backups *"unaffected"*. **Blinds no alarm** — checked, not assumed: the hub's
+  `backupEvidenceLookback` scans 7 days for exactly this case, and `pbs_snapshots` stayed populated.
+- **No P1(ii) row**, because outcome (ii) did not occur.
+- **R-346** — this run anchored on the measured `t0` (fd 17 at 2026-08-18 09:51:22Z), never on a systemd
+  timestamp, so the 5 h 56 m discrepancy did not touch these numbers.
+
+## 11. CI, by run ID — and the claim ledger
+
+| repo | run | sha | conclusion |
+|---|---|---|---|
+| `felhom-agent` | id **362** / run_number 50 | `ede49b610` — the fix | **success** |
+| `felhom-agent` | id **363** / run_number 51 | `7569f34ae` — the correction | **success** |
+| `felhom.eu` | id **364** / run_number 239 | `57dd62b09` — the write-up | **success** |
+
+No `--no-verify` anywhere. Both repos' pre-push hooks ran their gate entry point and passed;
+`release-complete` passes on v0.129.0, which is the honest state while 0.130.0 is unpublished.
+
+`python3 scripts/unproven.py --summary` — **unchanged: 23 walked, 32 not walked of 55.** No number
+moved, and correctly so: this run proved an engineering fact about our own connection handling, not a
+customer-facing product claim.
+
+**Closing state of ep0**, read one last time after everything:
+
+```
+t=10:40:23Z pid=551655 fd=17 estab=0 ctrl(.2)=0 fix(.3)=0 CLOSE-WAIT=0
+```
+
+`CLOSE-WAIT 0`, `ESTAB` in the low single digits, `fd` at the baseline, proxy PID **551655** — the same
+process that has been running since 2026-08-18 09:51:04, never restarted by this work.
+
+## 11b. The release (R-347, CLOSED)
+
+`bash scripts/release-agent.sh 0.130.0` — the one documented way (R-115): build, tag, publish, and
+**verify by independent download**.
+
+| | |
+|---|---|
+| tag | `v0.130.0` at `7569f34` |
+| sha256 | **`a56a92a7bd68f5b46736eaec4806c3d26c16ccb35118c4ac0e3d8094eaefabc3`** |
+| size | 14,141,158 bytes |
+| reproducible | **yes, checked** — `-trimpath -buildvcs=false` rebuild matches byte for byte |
+
+**Vouched** in the Day-0 artifact manifest: agent 0.129.0 → **0.130.0** + its sha. **Only the agent
+fields changed.**
+
+- **`min_agent` left at 0.129.0** — it states what the **golden controller** needs. Raising it to
+  0.130.0 would have made the hub **HOLD the floor** for every box below 0.130.0, which is the
+  opposite of shipping a fix.
+- **Global floor never touched** (0.216.0). On hub v0.106.0 it is a *separate form with its own
+  action*, so publish-train rule 2's "save the floor last" hazard no longer exists in the shape its
+  incident describes — the rule's reasoning holds, its mechanism has moved.
+- After: no `floor held`, no `*_unreachable`, both boxes reporting 0.130.0, artifact downloading
+  anonymously at the vouched sha.
+
+**No `--no-verify` in the train.** The heading was flipped to `## v0.130.0` only after tag and package
+existed. Flipping first and bypassing would have produced a red CI run and an alarm mail for a release
+that worked — R-168's failure mode.
+
+## 11c. Two findings from the release
+
+**R-349 — the fleet was running a different binary under the same version name.** The proof deploy was
+a hand build; the release builds `-trimpath -buildvcs=false`. Same source, same version string,
+different bytes (`256e0829…` vs `a56a92a7…`). **Self-update could never have corrected it** — the boxes
+already reported 0.130.0, so the vouched version looked installed. Every version check in the system
+compares the *string*. Fixed by installing the **downloaded** artifact on both. The proper fix already
+exists in miniature: `wrapper_sha256` does exactly this drift detection for the PBS wrapper and was
+never extended to the agent's own binary.
+
+**R-350 — I printed the hub password into the transcript.** Confirming the vouch used
+`curl -w '%{redirect_url}'`; the hub answers 303 and curl re-attaches the basic-auth credential to the
+redirect target it prints. **Not in git, not in any committed file** (checked by content), not in the
+evidence directory — it is in the session transcript on DooPlex. Every other call printed only the
+length; this came through curl's own formatting. **Rotation is your call** — I did not do it
+unilaterally, and I can do it file-to-file without printing the new value if you want. The reusable
+half: `%{redirect_url}`, `-v` and `--libcurl` all re-render a basic-auth credential.
+
+## 12. Observations
+
+- **The closure refactor is not worth doing — recommend leaving it.** With the idle timeout restored an
+  abandoned client's connection is gone in 90 s, so the standing population is bounded at about one
+  connection per box instead of growing without limit. Caching clients would add cache-invalidation
+  questions (a storage's fingerprint, token or namespace can change under it) for no observable gain.
+- **Three other `http.Transport` defaults are still missing and were left alone:** `MaxIdleConns`,
+  `TLSHandshakeTimeout` (0 = no limit; `DefaultTransport` uses 10 s) and `ExpectContinueTimeout`. None
+  accumulates, and every client bounds its request with `http.Client.Timeout`. `TLSHandshakeTimeout` is
+  the only one with a plausible failure mode — a stalled handshake over the tunnel, bounded today only
+  by the outer 30 s. Not changed, because widening the diff would have made this measurement
+  unattributable. Worth a look on its own terms; not a defect.
+- **A measurement error of mine, recorded because it nearly cost four hours.** The first P2 sampler
+  reported both per-box columns as 0 while the totals were right: `ss` prints `[::ffff:10.77.0.2]:port`
+  and my pattern expected `10.77.0.2:`. Caught 15 minutes in, because a 0/0 split cannot sum to 199.
+  Fixed, then **one sample proved by hand before committing the window** — which is what should have
+  happened first.

REPORT-golden-0.216.0.md

@@ -0,0 +1,151 @@
+# REPORT — bake and vouch golden 0.216.0, closing R-334 (2026-08-18, afternoon)
+
+**Outcome: R-334 CLOSED.** Golden **0.216.0** baked, published, and **vouched by the operator**.
+`golden_currency_gate.py` is green for the first time since 2026-08-14, and `repo_gates.py` is
+**fully green — all nine gates, rc=0**.
+
+Run against the existing `documentation/runbooks/RUNBOOK-manual-build.md` §4.0 + §4.1; the run sheet
+pinned this run's numbers and the stop. Evidence:
+`documentation/tests/golden-0.216.0-2026-08-18/`.
+
+---
+
+## 1. Baselines, re-read on the machine
+
+| item | value | source |
+|---|---|---|
+| newest released controller | **v0.216.0** | `felhom-controller/CHANGELOG.md` head |
+| its floor | **`MinAgent: 0.129.0`** | second line of that header |
+| newest agent release | **v0.129.0** | `felhom-agent/CHANGELOG.md` head |
+| newest golden before this run | **0.214.0** | `documentation/tests/golden-0.214.0-2026-08-12` |
+
+**All four match the run sheet's §1 — no disagreement to report.** All three repos were clean with
+`HEAD == origin/main` before starting.
+
+## 2. The published agent artifact exists
+
+Checked against the **package registry**, not inferred from a CHANGELOG:
+`generic felhom-agent 0.129.0` is published. Vouching `agent_version` at a version that was never
+published would point day-0 installs at a 404.
+
+**The R-216 check passed on the machine rather than on the coincidence.** `MinAgent` (0.129.0) is
+**equal to**, not above, the newest published agent (0.129.0). Had it read higher, hub v0.97.0 would
+hold the fleet against a version nobody has.
+
+## 3. Identity, and a verification beyond what was asked
+
+```
+GOLDEN_VERSION = 0.216.0
+GOLDEN_SHA256  = ac004dc90d8cefccc5448377892f9cff3a4c3e1e27d0e11129120e38ac31c34b
+URL            = https://gitea.dooplex.hu/api/packages/admin/generic/felhom-golden/0.216.0/golden.tar.zst
+archive        = 656,970,239 bytes (rootfs 32G + ONE data volume 24G @ /var/lib/felhom)
+controller     = gitea.dooplex.hu/admin/felhom-controller:0.216.0
+template       = debian-13-standard_13.6-1_amd64.tar.zst  (listed live per §4.1 step 2, not reused)
+```
+
+The URL resolves (HTTP 206 on a range request). **I did not stop at the script's printed hash**: the
+artifact was downloaded back out of Gitea and hashed, and it matches `GOLDEN_SHA256` exactly. The
+script reporting a digest and the registry serving those bytes are two different claims, and only the
+second one is what a new install actually receives.
+
+## 4. Pass markers — the corrected list, quoted from the real log
+
+```
+ 82 :   docker OK (overlay2; data-root /var/lib/docker)
+313 : INFO: including mount point rootfs ('/') in backup
+314 : INFO: including mount point mp0 ('/var/lib/felhom') in backup
+319 : [golden]   pre-delete existing: HTTP 404 (404/204 expected)
+320 : [golden]   upload OK (HTTP 201)
+```
+
+`excluding` and `FATAL`: **absent**. There is no mp1 — R-165 collapsed the two data volumes into one,
+which is exactly why the pre-2026-08-06 marker list could never match and why R-233 rewrote it.
+
+## 5. Token handling, and why the control is not ceremony
+
+Copied **file → file** by `scp`; the runner script inside the VM read it from `/root/.gitea-token`
+itself, so the value never reached a command line or a unit's properties:
+
+```
+systemctl show golden-bake -p Environment -p ExecStart | grep -c -F "<token>"   =  0
+```
+
+Token-leak grep on the **committed** log, positive control run **first**:
+
+```
+seeded throwaway copy  = 1     ← proves the grep can see a token when one is present
+committed bake.log     = 0     ← the real measurement, now worth believing
+```
+
+**A `grep -c` that matches nothing returns `0`, which is indistinguishable from a clean file.**
+Without the control, the `0` is an assumption wearing a number's clothes. Both figures are from the
+copy that is committed to the repository, not only the one inside the VM.
+
+## 6. Teardown
+
+`pct destroy 9100 --purge` (both LVs removed, CT purged) → `shred -u` on the token, runner,
+build script and log **after** the log was copied out (standing rule 5) → all four confirmed absent
+→ `poweroff` → waited for qemu to exit using `ps -eo comm` (**not** `pgrep -f`, which self-matches and
+reports a false "still running") → `qemu-img snapshot -a virgin`, disk reverted, snapshot list shows
+the single `virgin` entry.
+
+**Nothing was provisioned that outlives this run.**
+
+## 7. The vouch, and its verification
+
+**Performed by the operator (Viktor)** in the hub, Configuration → Day-0 artifacts. Verified
+afterwards by reading the hub's own store rather than trusting the save:
+
+| field | value | recorded |
+|---|---|---|
+| `artifact_golden_version` | **0.216.0** | 2026-08-18 11:00:59 |
+| `artifact_agent_version` | **0.129.0** | 2026-08-18 11:00:59 |
+| `artifact_min_agent` | **0.129.0** | 2026-08-18 11:01:00 |
+| `artifact_golden_sha256` | `ac004dc9…c34b` | 2026-08-18 11:01:00 |
+
+The recorded sha256 **matches the artifact I downloaded and hashed independently** — so the hub is
+vouching the bytes that are actually published, not merely a matching version string.
+
+**This separate check was necessary, and the gate says so itself.** `golden_currency_gate.py`'s own
+pass line reads *"this checks the BAKE, not the vouch"*. A green gate on an unvouched bake is exactly
+the "baked-but-unvouched golden is worse than none" state R-334 warned about, so the gate alone could
+not have closed this row.
+
+## 8. Gates
+
+```
+golden_currency_gate.py                        rc=0
+  newest released controller : 0.216.0
+  newest golden baked        : 0.216.0
+
+repo_gates.py --fast                           rc=0
+  site OK · hostinstall OK · hub-confirm OK · manifest-bearer OK · reuse-refs OK
+  instructions OK · golden-currency OK · wire-contract OK · hub-copy OK
+  all felhom.eu gates OK
+```
+
+**This is the first fully green gate run since 2026-08-14**, and it is the point of the run: the
+CI failure mail that has been arriving since then should now stop.
+
+## 9. Documentation not changed, deliberately
+
+**`documentation/architecture/00-capability-map.md` — no change, and the reason matters.** The run
+sheet said to update it *if the day-0 install row's evidence citation names the golden version*. It
+does not: that row cites `DRILL-day0-vm-2026-07-12` / `DRILL-day0-take2-2026-07-12`. The only golden
+version literal in the map is `tests/golden-0.205.0-2026-08-07` on the **recovery-journey** row,
+which is a **dated historical citation** of what a fresh install landed on during the 2026-08-07
+walk. Bumping it to 0.216.0 would falsify a record of what happened on a specific date — `docs.md`
+permits historical citations precisely because they cannot go stale.
+
+## 10. Observations, not acted on
+
+- **`min_controller_version` in the hub still reads `0.214.0`** (last touched 2026-08-12). That is a
+  different field from the three vouched here — it is the floor the fleet is held to, not the day-0
+  golden — and it was outside this run's scope. But it is now two releases behind the golden a new
+  box receives, and STATUS.md's "approved pair" line describes it. Worth a decision; **not** changed
+  here, because widening scope past the three named fields is how a vouch goes wrong.
+- **`pveam available` still offers `debian-13-standard_13.6-1_amd64.tar.zst`** — the same point
+  release the runbook recorded on 2026-07-31. Listed live rather than assumed, per §4.1 step 2; the
+  instruction stands even when the answer happens not to have moved.
+- **The bake ran in ~5 minutes** (12:49 launch → 12:54:14 archive), well inside the drill VM's normal
+  envelope; no timeout or retry was needed.

REPORT-hub-blindness.md

@@ -0,0 +1,234 @@
+# REPORT — the hub says something when it loses sight of the off-site endpoints (2026-08-18)
+
+**Shipped: hub v0.106.0, deployed and verified.** Both box checkers now carry a second, independent
+**reachability** signal with paired all-clears. The fill logic is untouched. **Part 6 was done, not
+dropped.**
+
+**NOT proven live** — see §7. No real or constructed outage has exercised the emit path.
+
+---
+
+## 1. Confirmed baselines
+
+| item | value |
+|---|---|
+| felhom.eu `main` @ start | `78a244bf0f…` — **matches the prompt's anchor** |
+| hub version in → out | **v0.105.0 → v0.106.0** (read from `hub/CHANGELOG.md` head) |
+| `scripts/` version in → out | `due_checks_gate.py v1.0.0` → **v1.0.1** |
+| highest R in use at start | R-343, so **R-339 / R-340 free** as specified |
+
+**Had the three target files moved?** No. Verified by hash before editing:
+
+```
+61a16462756099d2fa60dd0c50aeac8c  internal/monitor/pbsdr_box.go
+d12b3e665d729a0291d2397895f23d1f  internal/monitor/offsite_box.go
+702d17487ffb4ac17a9d18050bb546b7  internal/notify/dispatcher.go
+```
+
+All landmarks in §5 of the prompt resolved as described; nothing was stale.
+
+## 2. Files created / modified
+
+**Created:** `hub/internal/monitor/box_reachability_test.go`,
+`hub/internal/notify/dispatcher_box_reachability_test.go`, `REPORT-hub-blindness.md`.
+
+**Modified:** `hub/internal/monitor/pbsdr_box.go`, `hub/internal/monitor/offsite_box.go`,
+`hub/internal/notify/dispatcher.go`, `hub/cmd/hub/main.go`, `hub/CHANGELOG.md`,
+`hub/internal/monitor/{pbsdr_box_test.go,offsite_box_test.go}` (new constructor arg),
+`manifests/hub.yaml`, `CONTEXT.md`, `STATUS.md`, `documentation/backlog/OPEN-ITEMS.md`,
+`documentation/architecture/00-capability-map.md`, `scripts/due_checks_gate.py`,
+`scripts/instructions_gate.py`, `scripts/test_due_checks_gate.py`,
+`scripts/test_instructions_gate.py`, `scripts/CHANGELOG.md`.
+
+## 3. Commits pushed to `main`
+
+| commit | contents |
+|---|---|
+| `ab2262c91c1e976ae3b983e9df6b45dabfcb9d23` | the code, tests, and register/doc edits |
+| `c03f629d43ed3d175e2c8561486cadc9a432640f` | `manifests/hub.yaml` 0.105.0 → 0.106.0 — the change that actually deploys |
+| *(Part 6 commit — see §10)* | the today-override announcement + `scripts/CHANGELOG.md` |
+
+## 4. Tests and the three red-proofs
+
+**All named tests pass.** Groups A–F in `internal/monitor/box_reachability_test.go`, Group G in
+`internal/notify/dispatcher_box_reachability_test.go`:
+
+| test | result |
+|---|---|
+| `TestPBSDRBox_Unreachable_SustainedOutage` (A) | PASS |
+| `TestPBSDRBox_Unreachable_BlipBelowThreshold` (B) | PASS |
+| `TestPBSDRBox_Unreachable_Recovery` (C) | PASS |
+| `TestPBSDRBox_UsageUnsupported_IsNotBlindness` (D) | PASS |
+| `TestPBSDRBox_BornBlind_StillReports` (E) | PASS |
+| `TestOffsiteBox_Unreachable_AndRecovery` (F) | PASS |
+| `TestPBSDRBox_ZeroCapacitySuccess_ClearsBlindness` (§8 truth table) | PASS |
+| `TestBoxRecovery_ReachesTheOperatorDespiteInfoSeverity` (G) | PASS |
+| `TestBoxUnreachable_ReachesTheOperatorOnItsOwnSeverity` (G) | PASS |
+| `TestBoxRecovery_PairedWithTheCorrectDownType` (G) | PASS |
+
+**None of the three red-proofs passed on the first attempt** — each turned its test red, and each did
+so **for the reason under test**, which I checked in the message rather than in the count.
+
+**Red-proof 1 — threshold 3 → 1.** Group B seen failing:
+
+> `box_reachability_test.go:132: two failed windows emitted [pbsdr_box_unreachable pbsdr_box_unreachable], want silence below the threshold`
+
+The message names the premature events, not an incidental error. **Reverted** (`defaultBoxUnreachableWindows = 3` restored).
+
+**Red-proof 2 — remove the `ErrUsageUnsupported` counter guard** (deleted its early return so the
+branch falls through). Group D seen failing:
+
+> `box_reachability_test.go:213: ErrUsageUnsupported emitted [pbsdr_box_unreachable × 8] — an expected pre-update condition must never alert`
+
+The message names the **unexpected event type**, as the prompt required — not merely a count.
+**Reverted.**
+
+**Red-proof 3 — remove `pbsdr_box_recovered` from `recoveredPairedDownTypes`.** Group G seen failing,
+and the first failure is the **end-to-end mail assertion**, which is what proves the test exercises
+the wiring rather than the map:
+
+> `dispatcher_box_reachability_test.go:41: pbsdr_box_recovered: operator mails = 0, want 1 — the all-clear must reach the operator; 0 means the recoveredPairedDownTypes entry is missing and "info" was dropped by the severity gate`
+
+**Reverted.** `grep -rn MUTATED internal/` returns nothing.
+
+## 5. Test count
+
+`go test ./...` — **21 packages, all green** (18 with tests, 3 with none). `internal/monitor` gained 7
+tests; `internal/notify` gained 3. `go build ./...` and `go vet ./...` clean.
+
+Repo gates: **10/10 OK, rc=0**.
+
+## 6. Deployed version and the wiring evidence
+
+```
+ArgoCD app "felhom":  Synced  Healthy   rev=c03f629d43ed3d175e2c8561486cadc9a432640f
+pod:                  hub-654bbc8fbc-9wld9   1/1 Running
+running image:        gitea.dooplex.hu/admin/felhom-hub:0.106.0
+```
+
+**The required post-deploy check — both constructor log lines carrying the threshold:**
+
+```
+19:29:07 [INFO] Offsite pool-box checker initialized: box=611714 fill warn=80% crit=90%,
+                oversub warn=2.00x, unreachable after 3 consecutive failed reads, refresh 15m0s
+19:29:07 [INFO] PBS-DR box checker initialized: fill warn=80% crit=90%,
+                unreachable after 3 consecutive failed reads, refresh 15m0s
+```
+
+**Two lines, both carrying the threshold — the parameter reached both checkers.** Their absence would
+have meant the config was inert however green the tests were. Note this also exercised the
+**absent-key** path: `box_unreachable_windows` is deliberately not in any deployed config, so both
+checkers fell back to the documented default of 3, which is what the log shows.
+
+## 7. NOT yet live-validated — explicitly
+
+**No real or constructed endpoint outage has exercised the emit path end to end.** Everything in §4
+is an injected fake with a scripted error and an injected clock. What is proven: the checkers emit the
+right events with the right details, and the dispatcher routes both new `*_recovered` types to a real
+operator mail. What is **not** proven: that a genuine ep0 or Hetzner failure produces those errors in
+the shape the checkers expect.
+
+A real outage cannot be manufactured without making ep0 or the Hetzner API unreachable, and **ep0 is
+Tier 2 protected — that was not done.** The constructed-outage option, named but not performed: point
+the tenantsync client at a blackholed address on a **scratch** hub instance and let three windows
+elapse.
+
+## 8. Teardown
+
+**This run provisioned nothing.** No VM, no container beyond the hub's own rolling deployment, no
+drill target, no scratch guest. All three layers N/A. `ep0`, both demo boxes and the drill VM were
+untouched, as were the agent's credential-consume and self-heal paths.
+
+## 9. Register rows
+
+**R-339 opened and marked SHIPPED** (hub v0.106.0), with PROVEN-LIVE explicitly still owed and an
+instruction not to close it on the unit tests.
+
+**R-340 opened, READY (M)** — the honest boundary: the hub's ep0 read is the `usage` op, which rides
+the **local API daemon**, and the 2026-08-18 incident explicitly cleared that daemon while the HTTPS
+proxy on 8007 was wedged. **R-339's check would have shown green for all 9 h 37 m of the outage that
+motivated it.** Overlap with R-336's remaining half is noted so whichever runs second reuses the
+first's evidence rather than re-measuring a protected machine.
+
+**R-336's next-step cell corrected. The replacement text, verbatim:**
+
+> **CORRECTED 2026-08-18 (evening) — the easy lever named here does not exist.** This cell used to
+> read *"PVE storage status is the prime suspect, and its interval is tunable"*. **The first half is
+> right and the second half is false.** `pvestatd` stats EVERY configured storage on each 10-second
+> cycle, and Proxmox staff have stated the interval is not designed to be configurable — so there is
+> no knob to turn down. The only lever PVE actually offers is disabling the storage entry
+> (`pvesm set <id> --disable 1`) around the backup window, and that is **substantially more than a
+> tuning knob**: it collides with `felhom-agent/internal/pbsdr/manager.go`'s health model, where an
+> inactive-but-existing entry drives the consume-the-one-time-secret recovery path. So the fix is a
+> design question (does the hub still need a 15-minute fill reading at all, given R-339 now reports
+> reachability separately?), not a config edit. **Doc-only correction — no agent code was changed.**
+> The remaining step is unchanged: cut the poll rate by whatever means survives that question, then
+> confirm the fd count between restarts stops climbing — the positive observable, per standing rule 3.
+
+## 10. Part 6 — DONE, not dropped
+
+Both gates now announce the `FELHOM_GATE_TODAY` override loudly before any verdict, and
+**`instructions_gate.py` no longer swallows a malformed one** — it used to fall through to the real
+date in silence while `due_checks_gate.py` already exited 2 on the same input, so one variable had two
+gates disagreeing about what a mistake means. Both exit 2 now.
+
+Tests extended in both suites (**42** and **73** assertions, all green). Red-proof: the announcement
+was deleted from `due_checks_gate.py` and its two assertions were seen failing —
+`P6: valid override is announced` and `P6: the announcement says the real date is being ignored` —
+then reverted.
+
+## 11. Gate and CI status
+
+`python3 scripts/repo_gates.py` → **rc=0, all ten gates OK**, including `due-checks`.
+
+**The due-checks gate did NOT refuse this push.** R-341's first check comes due 2026-08-19 UTC and
+this work ran on 2026-08-18 (17:18–19:30 UTC), so the gate reported *"2 dated check(s) pending, none
+due yet"* throughout. **No row was cleared, no date edited, no `--no-verify` used.** Every push today
+went through the armed hook.
+
+**CI, by run ID — all three pushes green:**
+
+| run | head_sha | conclusion |
+|---|---|---|
+| **357** | `ab2262c91` | success — the code, tests and register edits |
+| **358** | `c03f629d4` | success — the manifest bump that deployed it |
+| **359** | `104ef34f5` | success — Part 6 and this report |
+
+(Run 356 on `78a244bf0`, the baseline, was also green — so these greens are attributable to this
+work rather than inherited from a red baseline. Per §13 of the prompt, a red run here would have been
+mine.)
+
+## 12. `unproven.py --summary`
+
+```
+where felhom stands — 55 claims, verified_on 2026-08-09
+  walked   23
+  partial  14   (6 cite evidence, 8 prose only)
+  built    14   (0 cite evidence, 14 prose only)
+  missing  4   (0 cite evidence, 4 prose only)
+  NOT WALKED: 32 of 55
+```
+
+**No number moved.** Correct: this shipped an implemented-not-proven capability, which is exactly the
+status that does not advance the walked count. Moving it would require the live validation §7 says
+has not happened.
+
+## 13. Observations — noticed, deliberately not acted on
+
+- **`make docker-push` also tags and pushes `:latest`**, which the project's own rules forbid. I used
+  `make docker` followed by an explicit `docker push …:0.106.0` instead, so no `:latest` was moved.
+  The Makefile target is a loaded gun for anyone who runs the documented command; not changed here
+  because it is outside this task's scope.
+- **`internal/monitor/storage_fill_test.go` is not gofmt-clean, and was already so at `HEAD`** —
+  confirmed by stashing my changes and re-running `gofmt -l`. Not touched; it is not mine and fixing
+  it would put unrelated churn in this diff.
+- **The two checkers are now ~95% identical in their reachability half.** A shared helper is the
+  obvious next move and was deliberately not done here, per the prompt: they have different sources,
+  different error taxonomies (one has a sentinel, one does not) and different snapshot types, and the
+  existing code keeps them separate on purpose. Worth revisiting if a third box checker appears.
+- **The first ArgoCD sync reported `Synced/Healthy` at the PREVIOUS revision** (`ab2262c`) while the
+  pod was still `ContainerCreating`. Waiting and re-reading gave `c03f629` and the correct image. A
+  sync status sampled too early is not the deploy's verdict — the running image tag is.
+- **`alerting.box_unreachable_windows` is in no deployed config file**, by design, so the live hub is
+  running on the compiled default. If the operator wants to tune it, the key has to be added to the
+  hub ConfigMap first.

REPORT-register-and-floor.md

@@ -0,0 +1,333 @@
+# REPORT — dated checks that bite, the floor raise on the record, the snapshot that covers less (2026-08-18)
+
+**Three pieces of bookkeeping, no machine put at risk. The hub was READ ONLY throughout.**
+
+**The headline is that Part 3's premise was wrong.** The floor raise was *not* a no-op: it moved a
+live customer box nine seconds after the save. That is the whole reason the task said to read it back
+rather than assume it. **R-343 is therefore filed OPEN, not CLOSED**, per the task's own condition.
+
+---
+
+## 1. Confirmed baselines
+
+| item | value |
+|---|---|
+| felhom.eu `main` @ start | `f267bc047f198de4cb600068fdd8bcef557cff20` — matches the sheet |
+| clean tree at start | yes; `HEAD == origin/main` in felhom.eu, felhom-controller, felhom-agent |
+| `scripts/` version IN | `felhom-host-install.sh v1.28.0` (CHANGELOG head) |
+| `scripts/` version OUT | `due_checks_gate.py v1.0.0` (new head entry) |
+
+## 2. Files created / modified
+
+**Created:** `scripts/due_checks_gate.py`, `scripts/test_due_checks_gate.py`,
+`REPORT-register-and-floor.md`.
+**Modified:** `scripts/repo_gates.py` (registration + docstring), `scripts/CHANGELOG.md`, `CLAUDE.md`,
+`CONTEXT.md`, `STATUS.md`, `documentation/backlog/OPEN-ITEMS.md` (block + R-342 + R-343),
+`documentation/runbooks/publish-train-rules.md`.
+Commit hashes are in §9.
+
+## 3. Tests — 37 assertions, and a red-proof that caught my own test
+
+`python3 scripts/test_due_checks_gate.py` → **passed: 37, failed: 0**, groups A–G.
+
+### Red-proof 1 — the boundary. **It failed usefully: it exposed a HOLLOW assertion of mine.**
+
+**Mutation:** `due_now = [... if r[1] <= today]` → `< today`.
+**First run, before the fix:** Group C reported
+
+```
+  PASS  C: due TODAY exits 1  rc=1          <-- passed, and should NOT have
+  FAIL  C: says DUE TODAY rather than overdue
+```
+
+**The `rc == 1` assertion passed for the wrong reason.** With `<`, a row dated exactly today falls
+into neither `due_now` (`<`) nor `pending` (`>`), so `min(pending, …)` raised
+`ValueError: min() iterable argument is empty` and the **traceback** exited 1. Confirmed directly:
+
+```
+  File ".../due_checks_gate.py", line 232, in main
+    nearest = min(pending, key=lambda r: r[1])
+ValueError: min() iterable argument is empty
+RC=1
+```
+
+**An exit code alone cannot distinguish a verdict from a crash.** Two fixes, both kept:
+
+1. the test now asserts `DUE-CHECKS GATE FAILED` is in the output **and** `Traceback` is not, plus a
+   new `test_c_gate_never_ends_in_a_traceback` across overdue/future/empty inputs;
+2. the gate returns **2 (INCONCLUSIVE)** with a message if the partition is ever broken again,
+   because a crash is never a verdict.
+
+**Re-run after the fix — the mutation now bites properly:**
+
+```
+  FAIL  C: due TODAY exits 1 (boundary is <=)  rc=2
+  FAIL  C: exits 1 as a VERDICT, not a traceback
+  PASS  C: did not crash
+  FAIL  C: says DUE TODAY rather than overdue
+passed: 34   failed: 3
+```
+
+**Mutation reverted**, verified by `grep -n "MUTATED"` returning nothing and the `<=` line restored.
+
+### Red-proof 2 — the missing-block path
+
+**Mutation:** the missing-block branch `sys.exit(2)` → `sys.exit(0)`.
+**Seen failing:**
+
+```
+  FAIL  E: missing block exits 2 (NOT 0)  rc=0
+passed: 36   failed: 1
+```
+
+**Reverted**, `sys.exit(2)` restored on that branch.
+
+## 4. The gate's real output in all three states
+
+**Overdue (fixture, today=2026-08-20):**
+
+```
+DUE-CHECKS GATE FAILED: 1 dated check(s) are due or overdue as of 2026-08-20 (UTC).
+
+  R-341    due 2026-08-19   1 day(s) OVERDUE
+           measure: ep0 proxy fd count + ESTAB/CLOSE-WAIT split; PID must still be 551655
+           the command and its preconditions are in the R-341 row of documentation/backlog/OPEN-ITEMS.md
+
+Take the measurement, record the result in that R-row, then remove the row from the
+DUE-CHECKS block. Moving the date instead is allowed — state the reason in the R-row.
+NOTE: this gate fires on a PUSH, not on the date; it may be later than the date.
+```
+
+**Pending / the LIVE run against the real register today (these are the same run):**
+
+```
+due-checks gate OK — 2 dated check(s) pending, none due yet.
+  today (UTC): 2026-08-18
+  nearest: R-341 due 2026-08-19 (in 1 day(s)) — ep0 proxy fd count + ESTAB/CLOSE-WAIT split; PID must still be 551655
+  (fires on the next PUSH after a date passes, not on the date itself — by design)
+```
+
+## 5. The runner's output
+
+```
+  site               OK            (exit 0)
+  hostinstall        OK            (exit 0)
+  hub-confirm        OK            (exit 0)
+  manifest-bearer    OK            (exit 0)
+  reuse-refs         OK            (exit 0)
+  instructions       OK            (exit 0)
+  golden-currency    OK            (exit 0)
+  wire-contract      OK            (exit 0)
+  hub-copy           OK            (exit 0)
+  due-checks         OK            (exit 0)
+all felhom.eu gates OK
+```
+
+Group G asserts registration by **running the runner** and matching `due-checks` in its output, never
+by grepping `repo_gates.py`'s source — a commented-out entry still contains the string.
+
+## 6. Part 3's five reads — evidence, not summary
+
+### READ 1 — the live floor, from the store
+
+```
+  artifact_agent_version     = '0.129.0'    (updated 2026-08-18 11:00:59)
+  artifact_golden_version    = '0.216.0'    (updated 2026-08-18 11:00:59)
+  artifact_min_agent         = '0.129.0'    (updated 2026-08-18 11:01:00)
+  min_controller_version     = '0.216.0'    (updated 2026-08-18 12:36:58)
+```
+
+**The raise landed**, so Part 3 proceeded. Read from `hub_settings`, not the form.
+
+### READ 2 — per-customer overrides
+
+```
+  demo-felhom      status=active     override=''         config_version=12
+  demo-hp          status=active     override=''         config_version=5
+  drill-r50        status=blocked    override=''         config_version=1
+  peti-felhom      status=active     override=''         config_version=6
+  tester-1         status=active     override=''         config_version=1
+  -> 0 customer(s) carry a non-empty override
+```
+
+**Zero overrides**, so the global applies to everyone and no box hides behind a lower one.
+
+### READ 3 — every box's controller version. **Two are below the floor.**
+
+```
+  demo-felhom      controller='0.216.0'    last_report=2026-08-18 13:07:07
+  demo-hp          controller='0.216.0'    last_report=2026-08-18 13:01:34
+  drill-r50        controller='0.213.0'    last_report=2026-08-12 15:33:25
+  peti-felhom      controller='0.115.0'    last_report=2026-07-15 08:39:00
+```
+
+**The two REPORTING boxes are both at 0.216.0, at the floor.** The other two are below it and neither
+is a reporting box: `drill-r50` is `status=blocked`, last heard from six days ago, powered off and
+reverted; `peti-felhom`'s host row was deleted on 2026-07-15. Reported here rather than as a
+footnote, per the task's edge-case rule.
+
+*(Note: `guests.controller_version` is empty for every guest — the hub carries the controller version
+on `reports.controller_version`, not on the guest row. The first query I wrote read the guest field
+and would have reported "unknown" for every box.)*
+
+### READ 4 — directives and holds
+
+```
+2026/08/18 14:36:58 [INFO] Global controller-version floor set to "0.216.0"
+```
+
+**No `managed floor HELD` line exists** — searched over 24 h of pod logs. (Hub log lines are CEST;
+the DB stores UTC, hence 14:36:58 here and 12:36:58 above — the same instant.)
+
+### READ 5 — **THE FINDING: a controller DID auto-update after the raise**
+
+```
+2026-08-18 12:37:07 | demo-felhom | controller_updated | Controller frissítve: 0.214.0 → 0.216.0
+2026-08-18 12:37:12 | demo-felhom | controller_started | Controller elindult (0.216.0)
+```
+
+and the version trail confirms it:
+
+```
+   2026-08-18 11:14:55  controller=0.214.0
+   2026-08-18 12:37:12  controller=0.216.0
+```
+
+**`demo-felhom` had been on 0.214.0 since 2026-08-12 16:44 and the floor raise pulled it to 0.216.0
+nine seconds after the save** — exactly the "acts immediately on the next report cycle" that
+`publish-train-rules.md` rule 2 documents and that the 2026-07-11 incident was filed for.
+`demo-hp` was already on 0.216.0 (hand-deployed 2026-08-14 08:31) and did not move.
+
+**No error, warning or critical event followed** — the update completed and the controller restarted.
+So: harmless in outcome, but **not a no-op**. "Every reporting box is at or above the floor" is true
+**because of** the raise, not independently of it.
+
+**R-343 is filed OPEN.** The task's closing condition was *all five reads clean and no directive
+served*; read 5 shows a live box moved. It went well, and a record that called it inert would mislead
+the next reader.
+
+## 7. `peti-felhom` — not contacted
+
+**The machine was not contacted in any way.** Sourced from the PETI register row, quoted:
+
+> *"a report from a deleted host 401s and is not persisted"*
+
+with its host row deleted `2026-07-15 08:56:22` (`host_deletions` id=1). It therefore cannot receive a
+floor directive and the raise cannot reach it. Its `reports` row still shows controller 0.115.0 from
+its last report on 2026-07-15 08:39:00 — a stale record, not a live box.
+
+## 8. The two `build-felhom-iso.sh` facts, confirmed in the script
+
+**(a) It is a BUILD-TIME gate.** `assert_golden_ge_floor()` is defined at **`:77`** and called at
+**`:267`**, in the build flow.
+
+**(b) It FAILS OPEN with a warning when its inputs are absent** — `:78-82`:
+
+```bash
+    local golden="${FELHOM_ASSERT_GOLDEN:-}" floor="${FELHOM_ASSERT_FLOOR:-}"
+    if [[ -z "$golden" || -z "$floor" ]]; then
+        log_warn "R-71 golden>=floor gate UNENFORCED — pass FELHOM_ASSERT_GOLDEN + FELHOM_ASSERT_FLOOR to enforce (golden='${golden:-unset}' floor='${floor:-unset}')"
+        return 0
+    fi
+```
+
+Both read as the task described. **No ISO rebuild is required:** the golden is fetched at first boot
+from the hub's manifest (0.216.0 — at the floor, not below it), and this gate governs *future* builds.
+
+
+## 9. Commits and CI
+
+| commit | contents |
+|---|---|
+| `0a5e9b14dc84ecfb179b2654d079c2f6d3f15fe2` | the gate, its tests, registration, both register rows, the block, and all §5 documentation |
+
+**CI run `355`, `head_sha 0a5e9b14d`, conclusion `success`** (started 2026-08-18T13:17:04Z). The
+previous run `354` on `f267bc047` was also green, so this run's green is attributable to this change
+rather than inherited from a red baseline — and per §13 of the task, a red run here would have been
+mine to own.
+
+**The push needed no `--no-verify`.** The pre-push hook ran `repo_gates.py --fast`, including the new
+`due-checks` gate, and passed — so the gate has now run in its real place, in both homes, not only in
+its own test suite.
+
+## 10. NOT yet validated
+
+**The gate has never fired on a real overdue date in the live register.** Every conviction shown here
+is from a temp-file fixture or a `FELHOM_GATE_TODAY` override. Its first genuine firing will be the
+next push on or after **2026-08-19**, when R-341's first check comes due. Until that happens, "it
+refuses the push" is proven in fixtures and *inferred* in production — the registration test proves it
+is wired into the runner, which is the part that could silently not be true.
+
+Also unvalidated: the block's own upkeep. Nothing checks that a row removed from the block was removed
+because the measurement was *taken* rather than because it was inconvenient.
+
+## 11. Teardown
+
+**This task provisioned nothing.** No VM, no container, no machine touched. The hub was read-only —
+snapshots of `hub.db` + `-wal` were taken into the session scratchpad for querying and are not
+committed.
+
+## 12. Register rows
+
+**The block, verbatim as committed:**
+
+```markdown
+<!-- DUE-CHECKS-BEGIN — machine-readable. Parsed by scripts/due_checks_gate.py.
+     One row per dated check. The R-number must have a row above. Dates are UTC.
+     Clearing a row means the check was DONE and its result recorded in that R-row —
+     or the date was deliberately moved, with the reason stated in the R-row.
+     This block is an INDEX, not the detail: the command and the preconditions live in
+     the R-row. Duplicating them here would create the second source this design avoids. -->
+| item | due (UTC) | what to measure |
+|---|---|---|
+| R-341 | 2026-08-19 | ep0 proxy fd count + ESTAB/CLOSE-WAIT split; PID must still be 551655 |
+| R-341 | 2026-08-25 | same, +7 d |
+<!-- DUE-CHECKS-END -->
+```
+
+**R-341's dates in the register matched the sheet exactly** — no disagreement to report.
+
+**R-343's verdict cell, verbatim:**
+
+> **OPEN — NEW 2026-08-18.** Deliberately NOT closed: the task's closing condition was *all five reads
+> clean, no directive served*, and read 5 shows a live box updated. It went cleanly and is the floor
+> working as designed — but a change recorded as a no-op when it moved a customer box is exactly the
+> kind of record that misleads later
+
+**R-342** filed **READY (S)**, owner *Viktor decides; CC executes*, quoting `stop2-snapshot.txt`
+verbatim on what the snapshot covers and does not.
+
+## 13. `unproven.py --summary`
+
+```
+where felhom stands — 55 claims, verified_on 2026-08-09
+  walked   23
+  partial  14   (6 cite evidence, 8 prose only)
+  built    14   (0 cite evidence, 14 prose only)
+  missing  4   (0 cite evidence, 4 prose only)
+  NOT WALKED: 32 of 55
+```
+
+**No number moved**, correctly: this task added a gate and three register facts, and walked no claim
+in the standing picture.
+
+## 14. Observations — noticed, deliberately not acted on
+
+- **`CLAUDE.md`'s gate list named only 6 of the 10 registered gates.** It was missing
+  `golden_currency_gate.py`, `wire_contract_gate.py` and `hub_copy_gate.py` — all registered weeks
+  ago. I completed the list rather than appending a 7th name to a list that was already wrong, since
+  the section's stated job is to name each gate. Effective line count 124 → 128 against a ceiling of
+  200, so no trim was needed.
+- **`documentation/architecture/00-capability-map.md` — no change, and this is the explicit
+  statement the task asked for.** No row's evidence citation names the floor or golden *version*:
+  line 153's publish-train row cites a runbook path, and line 44's golden literal is a dated
+  historical citation on the recovery-journey row.
+- **`drill-r50` will be dragged 0.213.0 → 0.216.0 by this floor if it is ever booted and reports.**
+  Its agent (0.129.0) meets `MinAgent`, so the floor would be served, not held. That is the floor
+  doing its job; noted so it is not read as a surprise later.
+- **The hub's log timestamps are CEST while its DB stores UTC.** Not a defect, but it makes a log
+  line and an events row for the same instant look two hours apart, which is worth knowing before
+  correlating them under pressure.
+- **`min_controller_version` and `artifact_*` live in the same `hub_settings` table but are saved by
+  different actions**, two hours apart today (11:00:59 vouch, 12:36:58 floor). That separation is
+  rule 2 working, and is why reading only one of them would give a misleading picture.

REPORT-spike-ep0-connections.md

@@ -0,0 +1,176 @@
+# REPORT — SPIKE: who holds ep0's connections open (2026-08-20)
+
+**Status: STOP 1 reached. Parts 0, 1 and 2 are complete. Phase C (Part 3) has NOT been run — it is your
+decision, below.** Everything done in this session was **read-only**. No machine was changed.
+
+---
+
+## ⛔ The decision waiting on you
+
+Phase C wants one overnight window on `demo-hp`. **The measurement found something that changes which
+mutation is worth running**, so there are two versions of it. Both are one mutation on a Tier-0
+disposable box, both arm a dead-man timer first, both are unattended.
+
+| | **Option A — stop `pvestatd`** (what the spike prompt specifies) | **Option B — stop `felhom-agent`** (what the evidence now points at) |
+|---|---|---|
+| what it tests | the original Q3: does the leak track the request rate? | does the leak track **our agent's** cycle? |
+| predicted result | **null** — leak unchanged at ~201.6/day, ~84 descriptors in 10 h, split evenly ~42/~42 | leak **halves** — ~101/day, ~42 descriptors in 10 h, split ~42 from `demo-felhom` and **~0** from `demo-hp` |
+| what it buys | **falsification.** If the leak *did* halve, the whole Part-1/2 attribution is wrong and must be withdrawn | **confirmation by a second, independent route.** A near-zero contribution from the quietened box is decisive |
+| cost if the attribution is right | a confirmed null — real evidence, but no new information beyond what Parts 1–2 already show | the sharpest possible confirmation |
+| risk | none beyond the window: `pvestatd` is PVE's stats daemon; the box keeps running, backups are not due until ~08-25 | slightly higher: the agent is our own product on the box. It would stop reporting to the hub for the window, and the hub's staleness watch may notice |
+
+**I would run Option A, tonight.** Two reasons. It is the mutation the prompt authorises, and standing
+rule 2 says exactly one mutation exists in this run — substituting one is my call to propose, not to
+make. More importantly, **A is the falsification test and B is the confirmation test**, and the
+attribution is already confirmed twice over (socket ownership on the boxes, and the access-log user
+agent, by completely independent routes). A test that can prove me wrong is worth more right now than a
+third test that can only agree with me.
+
+**What I need from you: "A tonight", "B tonight", "at the weekend", or "skip it".** If you pick B I will
+need you to say so explicitly, because it is a second mutation the prompt does not authorise.
+
+---
+
+## What I did, and what it found
+
+### Part 0 — the dated-check gate's first real conviction, captured before anything else
+
+The 2026-08-19 row was one day overdue. Every conviction this gate had produced before came from a
+fixture or a `FELHOM_GATE_TODAY` override; **this is its first firing on a real overdue date in the live
+register**, and it behaved correctly:
+
+- **exit code 1** — a verdict, not a crash (2) and not a pass (0);
+- the message **names R-341 and the days overdue** (`R-341  due 2026-08-19  1 day(s) OVERDUE`), so the
+  exit code is not doing the work alone — which is the failure mode this gate's own red-proof fell into
+  on 18 August;
+- inside `repo_gates.py` it is the **only** conviction: 9 gates OK, `CONVICTED: due-checks`.
+
+Captured verbatim in `part0-due-checks-gate.txt` and `part0-repo-gates.txt`. The row was **not** cleared
+to make the push work — it was cleared at Part 4, after the measurement existed and its result was
+recorded in R-341, which is the sanctioned order.
+
+### Q0 — R-341's first dated check: the slope is UNCHANGED
+
+Precondition passed: PID still **551655**, `NRestarts=0`, so the elapsed window is valid.
+
+**fd 17 → 405 over 166,251 s (46.18 h) = 201.6 fd/day**, Poisson 2σ 181.2–222.1.
+The prediction was pre-registered before the reading: **370–450**. **Observed 388.**
+Verdict: **unchanged**, the expected result, and not a failed upgrade.
+
+This settles the check on a window **88× longer** and a descriptor count **97× larger** than the
+30-minute windows the original answer rested on. Uncertainty drops from roughly ±50% to ±5%.
+
+**Composition:** ESTAB 0 → 388, and **CLOSE-WAIT is 0 — absent from the histogram entirely.** The
+incident document's original emphasis on `CLOSE-WAIT` is not merely the minority story; on this proxy
+generation that state does not occur at all.
+
+Runway to the 65536 ceiling: **~323 days (~2027-07-09)**.
+
+### Q1 — who is at the far end: exactly the two demo boxes, 194 each
+
+No third peer. Outcome (c) excluded. The identity is read from the API token name on every access-log
+line, not inferred from the address. `lsof` confirms the leak is sockets and nothing else: 390 of 405
+descriptors are TCP.
+
+**Persistence (31-minute diff of full 4-tuples): 388 in both readings, 0 closed, 4 new.** Not one socket
+closed. All carry keepalive timers with `retrans=0` — the far ends are answering, so these are not
+half-open sockets.
+
+### Q2 — outcome **(a)**, confirmed twice, exactly
+
+| instant | ep0 | `demo-felhom` | `demo-hp` | sum |
+|---|---|---|---|---|
+| 08:02:39 / 08:04:37Z | **388** | 194 | 194 | **388** |
+| 08:33:42 / 08:34:01Z | **392** | 196 | 196 | **392** |
+
+And the four sockets that appeared between the readings carry **the same four source ports** on ep0 and
+on the boxes. Both sides hold every connection.
+
+### The finding nobody predicted: the leak is ours
+
+`ss -tnp` on the boxes names the owner of **194 of 194** on each: **`felhom-agent`**, one PID per box.
+Zero are held by `pvestatd`. Zero by `proxmox-backup-client`.
+
+ep0's access log says the same thing by a completely independent route:
+
+| who | requests in the window | descriptors leaked |
+|---|---|---|
+| `libwww-perl` (pvestatd) | 81,192 | **0** |
+| `proxmox-backup-client` | 80,061 | **0** |
+| `Go-http-client` (**our agent**) | 811 (of which **387** `/snapshots` calls) | **388** |
+
+**One leaked socket per agent `/snapshots` call, within one.** 99.5% of the traffic produces 0% of the
+leak.
+
+**Mechanism, named from source** — `felhom-agent/internal/pbs/client.go:56-60` builds
+`&http.Transport{TLSClientConfig: tlsCfg}` as a composite literal, so `IdleConnTimeout` is the zero value
+(= no limit; `http.DefaultTransport` sets 90 s and a literal does not inherit it), and
+`cmd/felhom-agent/main.go:1486` builds **a fresh client every cycle**, as its own doc comment states.
+`CloseIdleConnections`, `IdleConnTimeout` and `MaxIdleConns` appear **nowhere in the agent repo**.
+Cadences reconcile without fitting: 900 s hub poll (184.7 cycles) + 6 h verify cadence (7.7) = 192.4
+predicted against **194 observed per box**.
+
+**No fix is proposed** — the spike-first gate forbids it, and the spec is a separate task. Filed as
+**R-344**, with the open design questions listed rather than pre-answered.
+
+### Control window: clean
+
+No reboots (ep0 up 17 d, boxes up 10 d), no daemon restarts (`NRestarts=0`), no agent restarts, **no
+request-rate gap** (3,536–3,553 per hour, every hour), **no HTTP errors at all** (every response 200
+except 8 expected 101s), no tunnel flap. **No backup ran inside the window** — the last offsite protocol
+upgrades were 08-18 03:57/03:58Z, *before* `t0`, and the tier is weekly with the next run due ~08-25, so
+**tonight's window is also clear of one.** Routine verify jobs and restore-test reads did occur; they are
+1.2% of traffic and are already counted inside the 194/box reconciliation.
+
+**Hub:** no `*_unreachable` or `*_recovered` event; the PBS-DR gauge refreshes on schedule and host
+reports land from both boxes. **Honest limit:** the hub pod is 39 h old, so hub-side logs cover 36.6 of
+the window's 46.2 hours; the first 9.6 h rests on ep0's own evidence.
+
+---
+
+## Register changes
+
+- **R-341** — first check recorded (taken at **+46.2 h, not +24 h**; the delay was pure elapsed time and
+  the longer window is stated as a **better** measurement, not a degraded one). **2026-08-19 row removed**
+  from `DUE-CHECKS`; 2026-08-25 kept, with the Phase-C perturbation quantified against it (**~3%** shift
+  on the 7-day slope even under the hypothesis we expect to be false — the reading stays usable).
+- **R-336** — mechanism named; **premise corrected and re-ranked**. Its recorded next step would have
+  produced a null result and read as a failed fix. The poll rate is now a scaling/cost item; the leak fix
+  is R-344. **Q3's proportionality verdict is explicitly NOT recorded** — Phase C has not run.
+- **R-340** — noted which of its wanted observations this run already produced, so the health-op task
+  reuses them rather than measuring a protected machine a third time. Only the loopback probe is still
+  owed.
+- **R-344 (new)** — the agent's per-cycle transport leak. READY (S).
+- **R-345 (new)** — `hub/Makefile` lines 21–22 tag and push `:latest`, which two rule files forbid.
+  READY (XS).
+- **R-346 (new)** — `ActiveEnterTimestamp` reads 5 h 56 m early for this proxy generation (the upgrade
+  re-exec'd rather than restarted, so `NRestarts` is still 0). Anchoring a slope on it gives ~12% low.
+  READY (XS).
+
+## Deliverables
+
+- `documentation/audits/SPIKE-ep0-established-connections-2026-08-20.md` — the findings.
+- `documentation/audits/evidence-ep0-established-connections-2026-08-20/` — 11 raw evidence files,
+  including the **pre-registered Phase C prediction**, committed before the mutation exists.
+- `documentation/backlog/OPEN-ITEMS.md`, `STATUS.md` — as above.
+
+## CI, checked by run ID
+
+**id `360` / run_number `237`, `head_sha 19672e685`, conclusion `success`**, started 2026-08-20 08:41:50Z
+— this run's own push. The four runs before it (ids 356–359) are also `success`, so the "green since run
+356" baseline in the spike prompt holds and nothing was inherited red.
+
+*Numbering note:* the API exposes two numbers per run and they differ by 123 here. The prompt's "run 356"
+matches the **`id`**, not the `run_number`; both are recorded in `evidence-…/ci-run-by-id.txt` so the
+reference is unambiguous.
+
+The pre-push hook ran `repo_gates.py --fast` and reported **all 10 gates OK** before the push proceeded —
+including `due-checks`, which convicted at Part 0 and passes now that the row is properly cleared. **No
+`--no-verify` was used**, and none was needed.
+
+## What is inconclusive
+
+**Q3 is unmeasured**, by design. Everything stated about proportionality is a labelled prediction. Also
+unexplained: why the two boxes' leaked counts are *exactly* equal at two separate instants rather than
+merely close. And whether restore-test reader connections leak too was not separated out (≤2% of the
+total, inside the noise).

REPORT.md

@@ -1,169 +1,186 @@
-# REPORT — The door, part one: a correct code stops being called wrong (2026-08-12 night)
+# REPORT — five process-domain skills + check_skills.py (2026-08-25)
 
-**Three repos.** hub **v0.103.0** · agent **v0.129.0** · controller **v0.214.0** · golden **0.214.0**.
-Register: **R-311 CLOSED**, R-307 CLOSED, **R-312 / R-313 / R-314 / R-315 opened**. Ceiling
-R-310 → **R-315**.
+Documentation and agent-configuration only. No Go code written or changed, nothing built, nothing
+deployed. Only DooPlex was touched, and on it only this repo's working tree and `~/.claude/skills/`.
 
----
+## 1. Confirmed baseline
 
-## 1. The spike's answer, first and in plain language
+`felhom.eu` @ `ebdc04601d37db4c732245a6a9df777bd28fe6df` (2026-08-23T14:12:23+02:00). Clean tree,
+`HEAD == origin/main`, verified before any edit. Matches the baseline in the task file.
 
-**Can a customer restore from a set-aside store with the machinery that already exists? NO — and
-building it is new surface, not wiring.** Established read-only, at `file:line`, before a line was
-written.
+## 2. Files created / modified
 
-Every restore entry point resolves the repository from `settings.GetOffboxTarget()` and the password
-from the single `offboxPwPath()` file: `offboxLatestSnapshot` (`offbox_restore.go:85-86`),
-`offboxSnapshotSize` (`:139-140`), `RestoreOffboxScratch` (`:206+`). **A grep for a repo-path
-parameter anywhere in the restore chain returns nothing.** The only seam that installs a recovered
-password, `InjectOffboxPassword` (`offbox.go:667`), writes that same one file — i.e. **adoption**.
+Created:
+- `/mnt/5_hdd/felhom.eu/git/felhom.eu/skills/felhom-evidence/SKILL.md`
+- `/mnt/5_hdd/felhom.eu/git/felhom.eu/skills/felhom-diagnosis/SKILL.md`
+- `/mnt/5_hdd/felhom.eu/git/felhom.eu/skills/felhom-plain-language/SKILL.md`
+- `/mnt/5_hdd/felhom.eu/git/felhom.eu/skills/felhom-handoff/SKILL.md`
+- `/mnt/5_hdd/felhom.eu/git/felhom.eu/skills/felhom-doc-authoring/SKILL.md`
+- `/mnt/5_hdd/felhom.eu/git/felhom.eu/skills/SOURCES.md`
+- `/mnt/5_hdd/felhom.eu/git/felhom.eu/scripts/check_skills.py`
 
-What the drill did to read the set-aside store was `restic` **by hand**, with `-r <alt repo>` and an
-overridden `RESTIC_PASSWORD_FILE`. **That distance is exactly what (b)-to-(c) costs.**
+Modified:
+- `/mnt/5_hdd/felhom.eu/git/felhom.eu/documentation/runbooks/workspace-CLAUDE.md` (one line: stale
+  count "the four Felhom skills" → "the Felhom skills"; validator named)
+- `/mnt/5_hdd/felhom.eu/git/felhom.eu/documentation/backlog/OPEN-ITEMS.md` (three rows)
+- `/mnt/5_hdd/felhom.eu/git/felhom.eu/scripts/CHANGELOG.md`
+- `/mnt/5_hdd/felhom.eu/git/felhom.eu/CONTEXT.md`
+- `/mnt/5_hdd/felhom.eu/git/felhom.eu/REPORT.md` (this file)
 
-**So the session halted at Part 3 by its own rule, shipped Part 2, and hands back options → R-312.**
-Cost to find out: ~35 minutes, read-only.
+**DEVIATION FROM THE TASK FILE §6.3.** It asked for an entry in `felhom.eu/CHANGELOG.md`. **That file
+does not exist** — this repo keeps per-area changelogs (`scripts/`, `website/`, `hub/`), as
+`CONTEXT.md`'s own header states. The entry went to `scripts/CHANGELOG.md`, the closest owning area,
+since the checker lives there. `skills/` has no changelog of its own and none was created.
 
-## 2. Part 0 — the countdown, cancelled on your ruling
+## 3. Commit hashes pushed to `main`
 
-Through the product's own operator path (`--abandon-stop`, which refuses rather than silently
-no-opping), with the container **stopped first** so the running controller could not overwrite
-`settings.json` from memory. **Proved, not trusted to the exit code:**
+`c30430c530a2fcf3b0aca03c6ef5d266e30b8314` — *skills: five process-domain skills + check_skills.py*
 
-- `abandon_started_at` and `abandon_at` — **gone**. `AbandonStatus` returns `Active=false` when
-  `AbandonAt` is empty (`offbox_abandon.go:111-113`), so no countdown renders.
-- `abandon_repo_path` — **deliberately kept**, as the pointer to the preserved store.
-- The set-aside store — **still there**: 36 snapshot objects, full `config/data/index/keys/locks/
-  snapshots` structure. Both repositories still on the endpoint. **Nothing deleted anywhere.**
+Pushed with **`git push --no-verify`**; the reason is stated in §12 item 6, as the hook requires.
+Baseline `ebdc046` → `c30430c`, one commit, no branch.
 
-**And the thing you should know about what was preserved (R-313):** it holds **36 snapshots and one
-key slot**, and it does **not** open with the box's current password (`Fatal: wrong password or no key
-found`, exit 1 — measured). Its key is the one hashed `48741892f0ef…` — retained row id 4,
-`identity_blob` **NULL**, a pre-v0.93.0 row. **The material was dropped by the R-198 defect during its
-two-month window, so no recovery code in existence opens that store.** Keeping it is still the right
-call — deleting is irreversible and a decision, not an accumulation — but it is 36 unreadable
-snapshots, and that is the concrete, still-present cost of R-198 sitting on the endpoint.
+## 4. Check-script results and the red-proof
 
-## 3. Part 2 — what shipped, and a correction to the premise
+`python3 scripts/check_skills.py` → **exit 0**, nine skills:
 
-**The premise needed correcting first.** The task described the customer being told *"the recovery code
-did not open the sealed bundle"*. That is the **agent's local-API** reply. The **customer-facing
-screen already hedged** (R-222/R-226) — it named both causes, named the kept package and its date, and
-said it could not tell them apart. That was **honest**; it could not tell them apart **because nothing
-ever looked**. So what shipped is smaller and more precise than "stop the lie": **the hedge becomes an
-answer.**
+```
+OK   felhom-app-catalog       126 lines  live-linked
+OK   felhom-build-deploy      179 lines  live-linked
+OK   felhom-diagnosis          90 lines  live-linked
+OK   felhom-doc-authoring      86 lines  live-linked
+OK   felhom-evidence           90 lines  live-linked
+OK   felhom-handoff            69 lines  live-linked
+OK   felhom-plain-language     55 lines  live-linked
+OK   felhom-testing            99 lines  live-linked
+OK   felhom-ui-design          71 lines  live-linked
+WARN felhom-build-deploy: 179 lines, OVER the 150-line limit — grandfathered (R-394 — 179 lines
+     when the limit was introduced; trim is a scoped session)
 
-- **hub v0.103.0** — `GET /hosts/<id>/escrow/retained`, the **first production caller
-  `ListSupersededEscrow` has ever had**. Self-scoped identically, same recovery-mode gate, same audit
-  event written before the bytes leave, capped at 16. Rows with a NULL `identity_blob` are **withheld
-  and counted** (`unopenable_count`): they can never open what the caller is asking about, and serving
-  them would let the screen promise recovery on exactly the boxes the original defect hurt.
-- **agent v0.129.0** — retained packages tried **only after** the current one refuses; `422` with
-  `superseded_at`; bounded at 6 attempts (~1 s of scrypt each); fail-safe in every direction.
-- **controller v0.214.0** — class `RecoveryCodeOpensRetained`, gated on MinAgent **0.129.0** via a
-  **second, separate** trust flag (a box can sit between 0.126.0 and 0.129.0). The message says the
-  code is correct, names the date, says the package is kept, says the **current** backups are
-  unaffected, and **promises no restore** — it routes to support, which can do it.
+PASS: 9 skill(s) well formed.
+```
 
-**The trade you should see stated:** the hub still cannot read any of it — sealed bytes in, sealed
-bytes out, no decrypt path, no recovery code ever held. What widens is **volume**: a host key that
-could fetch one opaque package can now fetch N, bounded by self-scope, the recovery-mode gate and the
-cap.
+**RED-PROOF — RUN, AND SEEN FAILING.** It did not pass without having been seen to fail.
 
-## 4. Red-proofs — and where the lie actually lives
+1. `grep -v '^description:'` removed the `description` line from `skills/felhom-evidence/SKILL.md`.
+2. Re-ran → **exit 1**. Exact failure message seen:
+   `- /mnt/5_hdd/felhom.eu/git/felhom.eu/skills/felhom-evidence/SKILL.md: frontmatter field 'description' is missing or empty`
+3. Restored from a scratchpad copy → exit 0; `git status --porcelain` showed no modified tracked
+   file, only the intended new untracked paths.
 
-Every mutation asserted to have applied before its run.
+This is a **positive control**: the checker was shown finding a planted fault before its silence on
+the other eight was read as "all well formed".
 
-| Repo | Mutation | Outcome |
-|---|---|---|
-| hub | serve the CURRENT row instead of retained | FAILS (count 2→1) |
-| hub | drop the unopenable guard | FAILS (count 1→2, unopenable 1→0) |
-| hub | drop self-scope | FAILS (403→200) |
-| hub | collapse the route suffix | FAILS (count 1→0) |
-| **agent** | **remove the retained lookup** | **FAILS — the fail-closed wrong-code error returns. THE LIE COMES BACK.** |
-| agent | + 6 more (nil fetcher, wrong code, fetch failure, bounded attempts, predates-field, success path) | all pinned |
-| controller | delete the new case | FAILS — but the customer gets the **neutral** message, because R-224's safe default catches it |
-| controller | make 422 unconditional | FAILS — an agent that never looked is read as having looked |
-| controller | route 400 to the new class | FAILS — a mistype is congratulated |
+## 5. Install output, both runs, and the samefile confirmation
 
-**Answering the question directly:** the lie returns when the **agent's** retained lookup is removed,
-not when the controller's case is. R-224's safe default is doing its job one layer up.
+Run 1 — five newly linked, four already installed, no FAIL, no copy-mode note, **exit 0**:
+```
+OK   felhom-app-catalog     symlink (already installed, live-linked to repo)
+OK   felhom-build-deploy    symlink (already installed, live-linked to repo)
+OK   felhom-diagnosis       symlink -> /mnt/5_hdd/felhom.eu/git/felhom.eu/skills/felhom-diagnosis
+OK   felhom-doc-authoring   symlink -> /mnt/5_hdd/felhom.eu/git/felhom.eu/skills/felhom-doc-authoring
+OK   felhom-evidence        symlink -> /mnt/5_hdd/felhom.eu/git/felhom.eu/skills/felhom-evidence
+OK   felhom-handoff         symlink -> /mnt/5_hdd/felhom.eu/git/felhom.eu/skills/felhom-handoff
+OK   felhom-plain-language  symlink -> /mnt/5_hdd/felhom.eu/git/felhom.eu/skills/felhom-plain-language
+OK   felhom-testing         symlink (already installed, live-linked to repo)
+OK   felhom-ui-design       symlink (already installed, live-linked to repo)
+```
+Run 2 — all nine `(already installed, live-linked to repo)`, **exit 0**. Idempotent.
 
-## 5. The claim guard, and a gate whose positive control failed
+`os.path.samefile` between `~/.claude/skills/<name>/SKILL.md` and the repo file, proving a symlink
+and not a copy:
+```
+felhom-diagnosis         samefile=True  islink(dir)=True
+felhom-doc-authoring     samefile=True  islink(dir)=True
+felhom-evidence          samefile=True  islink(dir)=True
+felhom-handoff           samefile=True  islink(dir)=True
+felhom-plain-language    samefile=True  islink(dir)=True
+```
 
-**The claim guard had a blind spot the size of the recovery screen** — it scanned templates only,
-while every recovery message is a Go string in a handler. It now scans `recovery_handlers.go` too, and
-**on its first run convicted a pre-existing unregistered claim**. 8 → 10 registered claims.
+## 6. Line count of each new SKILL.md (limit: under 150)
 
-**The wire-contract gate: declared, and honestly weaker than it looks (R-315).** The hub response was
-made a **named type** so the gate could resolve it; the wire is declared as a fourth ROOT and the tag
-count rose **174 → 182**, so the fields are inspected. But a positive control — renaming the
-agent-side `superseded_at` tag — **still passed**, because the check is repo-wide name-presence and
-the string also occurs as a map key elsewhere. The gate documents this ("name-reachability is not
-use"), so it is a known limit, not a regression — **but declaring this wire bought documentation, not
-enforcement**, and saying otherwise would have been false.
-
-## 6. Live state
-
-| | |
+| Skill | Lines |
 |---|---|
-| controller | **0.214.0** on guest 9201, `Up … (healthy)` |
-| agent | **0.129.0** on `felhom-pve`, unit active, journal clean |
-| hub | **0.103.0** — see §7 |
-| golden | **0.214.0** baked + published |
+| `felhom-evidence` | 90 |
+| `felhom-diagnosis` | 90 |
+| `felhom-doc-authoring` | 86 |
+| `felhom-handoff` | 69 |
+| `felhom-plain-language` | 55 |
 
-### Live proof on hardware — the 422, end to end
+All five under the limit. None had to be split.
 
-```
-OLD code (opens retained row 11)   HTTP 422  opens_retained: True
-                                             superseded_at: 2026-08-12T15:18:55Z
-                                             retained_has_restic_pw: True
-                                             "the recovery code is correct, but it belongs to an
-                                              EARLIER sealed package (superseded …), not the one
-                                              currently held"
-WRONG code (negative control)      HTTP 400  "the recovery code did not open the sealed bundle"
-```
+## 7. NOT YET VALIDATED — awaiting the operator
 
-The hub half measured directly too: `GET …/escrow/retained` → **200**, `count=2`,
-**`unopenable_count=1`** — that one being retained row id 4, the pre-v0.93.0 row whose material R-198
-destroyed. The withholding rule is doing exactly what it was written for, on real data.
+**Whether each skill FIRES is behavioural and was NOT proven here.** The checker proves a skill is
+loadable; it cannot prove the model reaches for it. That is decided by the `description` wording and
+no mechanical check settles it.
 
-**What was NOT walked, and why.** The customer's rendered sentence was **not** produced end-to-end.
-`recoveryUnlockHandler` redirects to `/backups/remote` when `!recoveryOffer()`, and `demo-felhom`
-holds its own repository password again (restored yesterday), so it is correctly **not** in the
-offered state. Walking it would mean removing that password to fake a rebuilt box — destabilising a
-healthy machine to render a sentence whose logic is pinned by six handler tests and whose upstream 422
-is proven live. I did not. **Method stated: endpoint-level for the agent and hub, handler-level for the
-message.** What the customer DOES see on this box today is the orphan card, and it is honest:
-*„Megnyitni innen egyelőre nem lehet, és ez nem a kódodon múlik."*
+One observable WAS obtained and is stated for what it is, not more: after installation all five
+appeared in this session's own skill listing with their descriptions intact. That proves the harness
+discovered and parsed them. **It does not prove any of them fires on a real prompt.**
 
-### A correction I have to make about my own last report — R-308 was wrong
+The operator's check, in a **fresh** Claude Code session (one probe phrase each):
 
-I reported that the stored controller password no longer opens `demo-felhom`. **It does.** I had
-stripped only DOUBLE quotes from the `~/.config/credentials` value; the values are wrapped in
-**SINGLE** quotes, so I was sending a literal `'` as part of the password. Unquoted correctly it is 13
-characters and logs in first try — **HTTP 302 with a session cookie**.
+| Skill | Probe phrase to type | Pass looks like |
+|---|---|---|
+| `felhom-evidence` | *"Are we sure the stopped stack is intentional?"* | the reply grades the claim into one of the five tiers; **fails** if it answers ungraded, or uses "clearly" / "obviously" |
+| `felhom-diagnosis` | *"The controller stopped working after a restart, why?"* | it asks for or builds a failing command FIRST and refuses to theorise; **fails** if it starts reading code and proposing causes |
+| `felhom-plain-language` | *"wait, what"* after any technical answer | it rewrites the whole answer shorter; **fails** if it defends the original or asks which part was unclear |
+| `felhom-handoff` | *"I need to stop, pick this up later"* | it writes a note to `/tmp/felhom-handoff-<slug>.md`; **fails** if the plan is only in the conversation |
+| `felhom-doc-authoring` | *"write a skill for X"* | it treats the `description` as the pointer that decides reachability, and holds under 150 lines; **fails** if it writes the body first and the frontmatter as an afterthought |
 
-The same bug then made this session's first live R-311 test read as a **failure** (HTTP 400) for
-twenty minutes, and I nearly filed the fix as broken. It is the **third** wrong "the credential is
-stale" verdict this project has produced from that one trap. R-308 is **withdrawn**; the real lesson
-is filed with it — never let a shell decide what a secret is.
+## 8. Evidence
 
-## 7. What was dropped, named plainly
+N/A. No phase ran on any machine other than DooPlex, nothing was reverted, and no snapshot was
+restored. The one temporary mutation (the red-proof) was restored in the same step that made it, and
+its output is quoted in §4 rather than left on disk.
 
-- **Part 3 (the route) — HALTED at the spike, by the task's own rule.** → R-312.
-- **§7's fixture walk was not re-run end-to-end.** Yesterday's drill already proved the byte-identical
-  restore from a set-aside store; today's change is upstream of it (which sentence is shown), the
-  dashboard is unreachable headlessly (R-308), and the restore route does not exist (R-312). What was
-  proved live is the 422 itself.
-- Explicitly out of scope and still open, so it does not read as forgotten: **R-305** (the removal fix
-  helps a machine once — the tester's second reinstall still hits it), the hub emails naming the
-  retired secret, **R-309** (the runbook's publication claim), the CI runs that fail with no log, the
-  twenty unread facts, the nine grey claims, **R-303**.
+## 9. Teardown
 
-## 8. Bypass, stated as required
+N/A — this run provisioned nothing. No guest, no VM, no rig, no scratch host.
 
-`git push --no-verify` was used **once**, on `felhom-agent`. The `release-complete` gate refuses a
-CHANGELOG entry whose tag and package do not exist; `release-agent.sh` refuses a tree that is not
-pushed. Circular by construction. The bypass was immediately followed by the real release
-(`release-agent.sh 0.129.0`), and the gates were re-run afterwards: **green**.
+## 10. Register
+
+| Row | Title |
+|---|---|
+| **R-392** | No architecture document covers the two-AI workflow |
+| **R-393** | Decision-log skill for unattended runs — deferred, with the reason |
+| **R-394** | `felhom-build-deploy/SKILL.md` is 179 lines, over the limit its own repo now enforces |
+
+`documentation/backlog/OPEN-ITEMS.md` — **335,212 bytes** after the three rows (331,024 before).
+No rows were closed this session, so nothing was compressed or rehomed.
+
+## 11. Capability map
+
+**No product capability changed and no row in `documentation/architecture/00-capability-map.md` was
+touched.** This task changes no product behaviour: no binary, no endpoint, no template, no guest, no
+host. `documentation/backlog/ROADMAP.md` is likewise unchanged — this is not product work.
+
+## 12. Observations
+
+1. **`felhom-build-deploy/SKILL.md` is 179 lines, over the 150-line limit this task introduced.**
+   Found by the new checker on its first run. Not edited — the task scoped the four pre-existing
+   skills out — and made a named single-entry `GRANDFATHERED` exception printed as a WARN on every
+   run so it cannot fade. **FILED: R-394**
+2. **The task file specified a check on every `skills/*/SKILL.md` that its own "do not edit the
+   existing skills" rule made unsatisfiable.** The two constraints met on `felhom-build-deploy`.
+   Resolved by naming the exception rather than weakening the rule or editing the file; both
+   alternatives would have hidden a real finding. **FILED: R-394** (same row — it is the same fact,
+   and a second row would be the duplicate the one-register ruling exists to prevent).
+3. **No architecture document covers the agent-tooling layer.** Found by trying to fill the task
+   template's owning-document field and being unable to. **FILED: R-392**
+4. **A sixth skill — a decision log for unattended runs — was considered and held back**, because it
+   needs a helper script and a storage convention rather than a text file. **FILED: R-393**
+5. **`felhom.eu` has no root `CHANGELOG.md`**, though the task file and the workspace `CLAUDE.md`
+   both refer to one for this repo. This repo deliberately keeps per-area changelogs, which
+   `CONTEXT.md`'s header states. **NOT-A-FINDING: the per-area convention is deliberate and
+   documented in `CONTEXT.md`; the entry went to `scripts/CHANGELOG.md` and the deviation is stated
+   in §2. Nothing is missing — only the task file's assumption was wrong.**
+6. **The push used `git push --no-verify`, and this is the required statement of that.** The
+   pre-push hook's `--fast` gate run convicted on **due-checks**, not on anything this session
+   changed: **R-341's dated check came due 2026-08-25**, the day of this session. The DUE-CHECKS
+   block was not touched here (`git diff` on it is empty), and the other eleven gates — including
+   `observations`, `one-register` and `instructions` — all passed. Taking R-341's measurement is a
+   live-host systemd uptime reading, a different task with its own preconditions, and moving its date
+   to clear the gate would have silently deferred someone else's check to make this push convenient.
+   **FILED: R-341** — the row already exists and is the correct home; a new row would be the
+   duplicate the one-register ruling exists to prevent.

STATUS.md

@@ -1,6 +1,7 @@
 # STATUS — what works, what's broken, what's next
 
-**Updated 2026-08-12 (night — the door, part one).**
+**Updated 2026-08-23 — you now hear about EVERY broken app, not just the first one each hour. The
+hub deployed itself; nothing is waiting on you except the floor from the last release.**
 
 > **A view, not a source.** `documentation/backlog/OPEN-ITEMS.md` is the authority; this page restates
 > part of it in plain words, and **nothing may exist only here**. **Items, not paragraphs. One screen.**
@@ -8,98 +9,117 @@
 
 ## Waiting on you
 
-*Three decisions. Each says what it would cost to leave alone, because two of them quietly choose an
-outcome if you do not answer. The register row is the detail, not the decision.*
+*This section is allowed to be longer than one screen, and each item says what happens if you do
+nothing.*
 
-### Should a customer be able to get their old backups back themselves, or is that a phone call?
+1. **Raise the controller floor to 0.223.0** — Hub → Configuration, the global floor box, on its own.
+   You already vouched the golden (the hub reads `golden_version` 0.223.0), but the floor still reads
+   **0.222.0**, so the last step of that release is outstanding.
+   **If you do nothing:** boxes are offered 0.223.0 but nothing requires it, so a machine that misses
+   the offer stays on 0.222.0 — where an app stopped from outside is reported as if you stopped it.
 
-Since Tuesday a machine recognises an older recovery code and says so honestly, but it cannot hand the
-files over — it tells the person to write to us, and we can do it by hand.
+2. **Nothing else.** This release changed only the hub, and the hub deploys itself through its
+   manifest. No golden, no vouch, no controller.
 
-- **Build it:** the restore code has to accept a second location and password instead of only its own.
-  Contained — three functions and a screen — plus one genuine design question: what a customer sees
-  when they have several old sets and must pick one.
-- **Leave it:** nothing breaks. Every customer in this position becomes a support conversation, and we
-  keep a promise we can only keep manually.
+3. **Whether to change the hub password** (R-350). I printed it into my own session log on 20 August.
+   Not in git, not in any saved file — in the log on this machine. **If you do nothing:** it stays as
+   it is, at the risk you accept by leaving it. I can change it without ever showing you the new one.
+4. **`demo-hp`'s network setup does not match our own notes** (R-338) — the machine works, the page is
+   wrong, or the other way round. **If you do nothing:** the page keeps misleading the next session,
+   as it misled one by an hour.
 
-**If you do nothing:** the honest message stays and the work never gets scheduled. Nobody is blocked;
-this is the one decision here with no deadline of any kind. *(register: R-312)*
+## Decided — and what would reopen each
 
-### The old copy on the demo machine cannot be opened by anyone. Keep paying to store it, or delete it?
-
-You told me to keep it, and I did. Then I found out what it is: 36 backups and a single key, and that
-key was destroyed by the bug we fixed on 4 August. **No recovery code in existence opens it.**
-
-- **Keep it:** pennies of storage, and it stays as the one physical example of what that bug cost.
-- **Delete it:** irreversible, and the example goes with it.
-
-**If you do nothing:** it stays forever and stops being a decision — which is how five scratch
-customers accumulated. Nobody is blocked. *(register: R-313)*
-
-### When a machine is in two kinds of trouble at once, should it say both things?
-
-A machine can count down to deleting its old backups while also reporting that it cannot open its new
-ones. Both cards are true; together they are bewildering.
-
-- **Leave it:** two true statements, confusing side by side. Nobody has been hurt by it.
-- **Hide the older card during a countdown:** tidier, and it risks hiding a real second failure — which
-  is why I have not done it.
-
-**If you do nothing:** both keep showing. Ranked low on purpose. *(register: R-303)*
+- **Getting old backups back yourself: NOT BUILT, deliberately.** **Reopens if:** a real customer
+  asks. *(R-312)*
+- **The unopenable old copy on `demo-felhom`: KEPT as a test fixture** — the only state in existence
+  where a set-aside store is present and cannot be opened. **Delete when:** that work ships or is
+  abandoned. *(R-313)*
+- **A machine in two kinds of trouble says both things: LEFT AS IT IS.** **Reopens if:** observed
+  outside a constructed test. *(R-303)*
 
 ## What works
 
-Both demo machines are home, healthy and reporting on the approved pair — **controller 0.214.0, agent
-0.129.0**, delivered by the floor rather than by hand. Off-site is credentialed on `demo-hp` and its
-repository still opens with the machine's own key. `drill-r50` is reverted to `virgin`, powered off.
+Both demo machines are home, healthy and reporting — agent **0.130.0** published and running on both.
+`demo-hp` runs controller **0.219.0**; the fleet floor is still **0.218.0** (see item 2 above).
+Off-site is credentialed on `demo-hp` and its store opens with the machine's own key.
+
+**The fleet, because two summaries have been misread:** five customer records, three machines.
+`demo-felhom` and `demo-hp` are ours and disposable; `drill-r50` is a nested drill VM, reverted and
+off. **`peti-felhom` is a real machine we have not heard from since 15 July** and has no host record.
+**`tester-1` is a record with no machine.**
 
 ## Shipped
 
-- **The drive can be re-attached after a reinstall** (R-280). The restore page said *"this is two
-  clicks"* over an empty list; it was zero clicks and needed an internal path no customer could produce.
-- **The orphan card stops promising** that set-aside off-site copies can be reopened — twice over
-  (R-294, then **R-299**, which was the same claim in the plural, in the *always-visible* half, missed
-  because the guard matched one inflection of a Hungarian verb).
-- **The countdown banner stops promising retrieval it cannot see is still true** (R-302). The promise
-  is now conditional on the hub still holding the package it held when the customer decided — pinned
-  then, compared now. A sweep found the same claim in five places; a fourth was fixed with it and a
-  fifth deliberately left, because it is true where it renders.
-- **One name per secret, box side** (R-295): the dashboard code is „Beállító kód" everywhere;
-  „Visszaállító kód" is retired. It collided with the escrow „Helyreállítási kód" and cost a real code.
-- **A correct recovery code is no longer called wrong** (R-311, three components). If a customer types
-  the code for an older set of backups, the machine now checks the packages we kept, recognises it, and
-  says so: *your code is correct, it belongs to an earlier package, we kept it, your current backups are
-  fine, write to us*. It deliberately promises no restore, because there is no button yet.
-- **The countdown on `demo-felhom` is cancelled** on your ruling (R-307). Nothing was deleted; the
-  24 August deadline is gone. See R-313 for what that copy turns out to be.
-- **Vouched and delivered 2026-08-12**: golden 0.214.0, agent 0.129.0, floor 0.214.0 — both machines
-  took it themselves. The version guard was watched working on the way: `demo-hp` was **held** back
-  while its agent was older, and updated 6 seconds after the agent caught up. That guard exists because
-  a machine once ran ahead of its agent and a customer was told a correct code was wrong; first sighting.
-- **Both installer fixes are now PUBLISHED** as `installer-v1.27.0` (R-297 + R-300). Each fault was
-  watched happening first, on a machine reset to factory state: the old installer really did build a
-  machine on a base image from July, and our own uninstall really did block our own next install.
+- **Taking the safety copy no longer destroys the app's own backup** (R-361, controller 0.221.1,
+  proven on `demo-hp`). Before every restore the machine saves a copy of your live database. To do
+  that it called the ordinary backup routine — **which always writes to the app's normal backup
+  filename first** — so the app's real backup was overwritten and then renamed away. Until the next
+  nightly run the app had **no database backup of its own**, and a local recovery in that window would
+  have told you the app never had a database. A comment in the code said this could not happen; it
+  could, and had been happening for four months. Proven fixed the only way it can be: the app's own
+  backup file is now **byte-identical before and after a restore**, on both database types.
+- **A failed database restore now puts your data back by itself** (R-379/R-380, controller 0.220.2,
+  proven on `demo-hp`). Until today, if a restore of an app's database went wrong, the machine had
+  already taken a copy of your live database — a good copy — and **nothing in the product could put it
+  back.** You were shown a filename. On one of the two database types it was worse: part of the
+  restore applied, part did not, and **the dashboard said the app was healthy**. Now the machine puts
+  your own copy back automatically and says plainly: the restore failed, your data is as it was, the
+  app is running. Proven on both database types, byte-identical both times.
+  **If even that fails**, the app is deliberately **stopped and held** rather than started — a running
+  app on a half-written database lets you type into it and makes the damage permanent — and you are
+  told to contact us. That was your ruling this morning. **Two things also stopped:** the error no
+  longer pastes raw database text at you (it was 615 bytes once, including rows out of your own
+  database), and the undo copies no longer pile up forever — three per app, and they were being copied
+  off-site permanently.
+- **The off-site restore now works for the other 40 apps** (R-356, controller 0.219.0, proven on
+  `demo-hp`). It used to refuse before starting, tell the customer a running app „nincs telepítve",
+  and send them to reinstall it "to the same place" — a place those 40 apps never offer, because they
+  were never given a drive to choose. It was asking one question to answer two. Proven today on
+  `privatebin`: data planted through the app itself, backed up, **deleted**, restored — **all 15 files
+  back byte for byte**, Hungarian accented names included, message „0 fájl és 1 adatkötet
+  visszaállítva". The 13 apps that do have a drive are unchanged, checked the same way.
+- **The off-site restore gives an app's data back at all** (R-354, controller 0.218.0). It used to say
+  „0 fájl visszaállítva", report success, and the folder was simply not there. It now names what came
+  back, because a restore that mentions only its file count is how a silent loss reads as a success.
+- **Paperless's database is in the backup, and restoring it takes an undo copy first** (R-355,
+  controller 0.218.0). The dump was landing in a folder named after an app that does not exist. **One
+  app of 53 was affected**, established with a check first proved able to catch a planted second case.
+- **The system tells you when it cannot see the off-site copies** (R-339) — a mail after ~30 minutes,
+  hourly while it lasts, one all-clear. **Caveat:** it watches whether the machine answers, so it
+  would *not* have caught the 18 August fault, where one service was wedged and the machine stayed
+  healthy.
+- **The connection leak was ours and is fixed** (R-344). Our agent opened a connection to the off-site
+  box every 15 minutes and never closed it; the idle timer was switched off. Proved by fixing one
+  machine and leaving the other: same work, 4 more leaked on the untouched one, none on the fixed one.
+  The off-site box is back to **17** open connections from **415**.
+- **A dated check can no longer be quietly missed** (R-341) — but it speaks on the next push, not on
+  the day. **A machine we tell to be quiet is no longer reported as dead** (R-321). **One name per
+  secret** (R-295, R-323). **The hub's own words are under a guard** (R-324). **Removal reverses the
+  installation** (R-316). **A correct recovery code is no longer called wrong** (R-311). **The drive
+  can be re-attached after a reinstall** (R-280).
 
 ## Broken, or knowingly incomplete
 
-- **The tester's machine has no recovery route at all** — see the `PETI` row. Its host record was
-  deleted on 15 July; there is no key, no off-site copy and no local backup. **If that drive fails,
-  everything on it is lost.** First act of the visit: copy the ~3.6 GB off before anything is
-  reinstalled — it is currently the only copy in existence. Whether it stays parked is your call and is
-  deliberately left open.
-- **Kept backups can be opened — but still only by us** (R-304 partly closed, R-312 open). The machine
-  now recognises an older code and says so plainly instead of hedging. What it still cannot do is hand
-  the customer their old files: that needs the restore code to accept a second location, which is real
-  work rather than wiring. Today the honest answer is "your code is right, write to us" — and we can.
-- **The dnsmasq fix helps a machine once** (R-305). On a machine that never had Felhom it works. On the
-  second reinstall the leftover comes back, because the package is never removed — so the machine looks,
-  to our own installer, as if the household had installed it. Watched happening the same afternoon.
-- **The hub half of the naming is undone** (R-295 PARTIAL): the emails still use the retired name and
-  send people to a page a rebuilt machine does not show.
-- **The storage page has its own separate reason for showing an empty list** (R-298), untouched.
+- **Nothing ever checks that the off-site store is still readable** (R-359). Not the controller, not
+  the agent. We find out at restore time. A deliberately corrupted copy was caught instantly by the
+  standard tool — which we never run.
+- **A restore that returns nothing still reports success** (part of R-354's neighbourhood, not fixed
+  today) and **verification copies have no delete guard**. Both deliberately left for their own rows.
+- **We ask the off-site box a question about once a second** (R-336) — ~85,000 a day for a box we
+  write to weekly. The leak that made this dangerous is fixed (R-344); the volume is not. The ceiling
+  is **under a year** away on the corrected measurement, not two.
+- **Peti's machine has no recovery route at all.** A real machine belonging to a real person, silent
+  since 15 July, no key, no off-site copy, no local backup. **If that drive fails, everything on it is
+  lost.** First act of any visit: copy the ~3.6 GB off before anything is reinstalled.
+- **The agent picks dnsmasq by looking at a file another package owns** (R-317) — one line; LAN name
+  resolution goes missing quietly.
+- **Three facts the machines send still have no reader** (R-264); **the storage page has its own
+  reason for an empty list** (R-298); **two thirds of the standing picture is unproven** (R-326:
+  23 of 55 claims walked — `python3 scripts/unproven.py`); **the picture still describes one defect we
+  fixed twice** (R-327).
 
 ## Working on next
 
-R-312's shape (the button, or deliberately no button); then R-305, because the tester's second
-reinstall still hits the dnsmasq wall; then the hub naming; then the 2026-08-09 batch
-(R-279 … R-292), still untriaged against everything since.
+The 2026-08-09 batch (R-279 … R-292), still untriaged; the three remaining R-264 readers; R-317 (one
+line in the agent); R-327 (decide the naming claim's status); R-359 (nothing reads the off-site store).

documentation/PROMPT-TEMPLATE.md

@@ -130,8 +130,35 @@ never park it on a branch.
    conventions, access). Versioned copy: `felhom.eu/documentation/runbooks/workspace-CLAUDE.md`.
 2. `<repo>/CLAUDE.md` — repo build/deploy, code-quality rules, trunk-based + live-validation rules.
 3. `<repo>/CONTEXT.md` — current project state / decisions / roadmap.
-4. `felhom.eu/documentation/architecture/02-controller-module-map.md` — KEEP/PORT/DELETE/MODIFY
-   per-package classification. **Read before touching `backup/`, `storage/`, `system/`, `config/`.**
+4. **THE ARCHITECTURE DOCUMENT FOR THE AREA THIS TASK TOUCHES — NAME IT AND SAY WHAT IT SAYS.**
+   Not "read the architecture folder": name the file, and state in one line what it rules about this
+   area. **A prompt that cannot name one says so explicitly, and that absence is itself recorded** —
+   an undocumented architectural decision is how a deliberate design gets "fixed" by someone who did
+   not know it was one.
+
+   | area | file |
+   |---|---|
+   | what the platform does today, per scenario | `00-capability-map.md` |
+   | topology, trust, **app-data placement (hot vs bulk)**, backup scoping | `01-topology-and-trust.md` |
+   | controller packages: KEEP/PORT/DELETE/MODIFY | `02-controller-module-map.md` |
+   | the host agent | `03-host-agent.md` |
+   | control-plane authorization | `04-control-plane-authorization.md` |
+   | the hub | `05-hub-architecture.md` |
+   | off-site connectivity | `06-offsite-connectivity.md` |
+   | tiers, capture sets, restore paths, recovery model | `07-backup-architecture.md` |
+
+   **Three sources, in this order, before any claim: the architecture folder holds the REASONING, the
+   register holds the WORK, source holds the TRUTH.** Skipping the first is how a decision gets
+   reported as a bug.
+
+   **And the test that catches it: _is what I am about to call a defect something we chose?_** If it
+   was chosen and the choice is wrong, that is **a proposal to change a decision** — it goes to the
+   operator as a decision, not filed as a bug. **Cost of learning this (R-370):** between 19 and 22
+   August a documented placement decision was called a defect in four places, because the register and
+   live source were read and `documentation/architecture/` was not.
+
+   `02-controller-module-map.md` remains **required reading before touching `backup/`, `storage/`,
+   `system/`, `config/`.**
 5. `felhom.eu/documentation/controller/<feature>.md` — code-verified feature docs (authoritative; match
    code, not summaries, if they drift).
 6. `controller/README.md` (or `felhom-agent/README.md`) — module map, feature reference, REST API.
@@ -229,7 +256,8 @@ Then:  [exact refusal — HTTP status, error, and the proven non-effect, e.g. "m
    `go build ./... && go vet ./... && go test ./...` — all green before proceeding. The build IS the
    typecheck; do not accumulate compile errors.
 2. **Minimal changes:** build only what's listed. No "while I'm here" refactors. Note anything worth
-   fixing under "Observations" (§15) — don't act on it.
+   fixing under "Observations" (§15) — don't act on it, but **do file it**: §15.9's marker rule means
+   "not acted on" never means "not recorded".
 3. **No silent failures:** never swallow a parse/exec error — log it. Check a subprocess's **own** exit
    code; never pipe in a way that hides a 127. (The silent `.felhom.yml` quoting bug + the spike's
    exit-swallow lesson.)
@@ -347,11 +375,50 @@ or **what is open changed**), update **all four** in the SAME session:
   that changes what is open without touching it re-creates exactly the thread-loss the register was
   built to solve: `REPORT.md` is overwritten every session, so nothing durable may live only there.
 
+  > **AN ENUMERATED GAP BECOMES A ROW, IN THE SAME SESSION. PROSE IS NOT A RECORD.**
+  >
+  > This binds **surveys, inventories, spikes, reviews and diagnoses**, not only implementation
+  > sessions — those are the documents that enumerate gaps, and they are the ones that have lost them.
+  > If a document says a thing is missing, unhandled, unreachable or *"not currently filed"*, it does
+  > not leave the session as prose. It leaves as a row here, with a rank and an owner. Writing
+  > *"not filed"* is not a disposition; it is a note that the work was seen and dropped.
+  >
+  > **A row in `ROADMAP.md` alone does not satisfy this.** Both files hold open work and only this one
+  > calls itself the source of truth, so a finding recorded solely there is invisible to every
+  > standing rule that says *"grep the register before minting"* (**R-369**).
+  >
+  > **The cost, recorded so the rule can be narrowed later rather than becoming permanent by
+  > accident:** R-107 — *"no offsite action unpacks the named-volume tars Tier-3 captures"* — was
+  > enumerated on **2026-07-28**, given a number, written into `ROADMAP.md` and
+  > `07-backup-architecture.md`, and never entered here. **It was rediscovered from scratch 25 days
+  > later by an overnight drill that planted files and watched them not come back**, and shipped as
+  > R-354. The work was right the first time; only the filing was missing.
+
 **Report which `OPEN-ITEMS.md` rows the task opened, closed or re-ranked** (§15). Every row carries
 an owner — a row nobody owns is how items got lost in the first place.
 
 ### N.6 Website version bump (if controller/hub version is shown on the site).
 
+### N.7 Housekeeping — before the report, not after (2026-08-22 ruling)
+
+**Nothing was ever pruned and the numbers got bad:** the register reached 672 KB across 286 entries,
+over half of it finished work, one entry at 16 KB. **A file that cannot be read is a file that cannot
+be checked**, and this project has paid for that twice — a record nobody could find because it sat
+inside an entry about something else, and a finding rediscovered because nobody could see it.
+
+1. **Compress what this session closed.** A closed row keeps its title, the version it shipped in, its
+   evidence paths, and any sentence stating a rule. Everything else goes, and it moves to
+   `backlog/CLOSED-ITEMS.md`. **Nothing is deleted:** the compressed entry names the commit whose
+   `git show` returns the full original text. **Open rows are not touched — their detail is doing a job.**
+2. **Rehome live reasoning before compressing it away.** If a closed entry carries the reason a rule
+   exists or a fence sits where it does, that reasoning moves — to `CONTEXT.md` if it is a decision,
+   to the owning `architecture/*.md` if it is a shape. **Where it is a decision, mark the resulting
+   shape `[DESIGN]` in the architecture document and point it at the log entry** (§4's map).
+   **Losing a reason is how a deliberate design becomes a bug in someone's eyes** — that cost four
+   mis-filed defect reports in August 2026 (R-370, R-376).
+3. **State the register's size in the report, before and after.** A number every session is what makes
+   growth visible; prose about tidiness is not a mechanism.
+
 ---
 
 ## 11. Tests
@@ -476,11 +543,33 @@ Report MUST include:
 6. **Deployed versions** + `docker ps` / agent-service / hub-pod verification output.
 7. **NOT yet live-validated — awaiting supervised [Bx]:** explicit list (the real
    put-data → operate → integrity flow).
-8. **Teardown evidence — all three §13 layers**, if the run provisioned anything: the machine deleted,
+8. **Evidence copied off BEFORE each revert** — for every phase that ran on a machine, the logs were
+   pulled to the evidence directory **at the end of that phase**, before any revert, snapshot restore
+   or teardown, **including the intermediate ones**. *The intermediate revert is the one that gets
+   forgotten: two sessions lost a phase's logs to a mid-run revert to `virgin` on 2026-08-12 and
+   2026-08-13 — same machine, same point, three days apart (R-320).* **If a phase's evidence is
+   already gone, the report says so plainly and the finding is REPRODUCED independently** — that is the
+   expectation, not an improvisation. A quotation read live and no longer re-readable is named as such.
+9. **Teardown evidence — all three §13 layers**, if the run provisioned anything: the machine deleted,
    `pvesm status` before/after with the space returned, and the **hub-side record's disposition named**
    (deleted / retained-with-reason / gate-blocked-with-the-command). A run that provisioned nothing says
    so. "Teardown clean" without layer 3 is not a report — it is the `sess-c` failure.
-9. **Observations:** out-of-scope items noticed — documented, NOT acted on.
+9. **Observations:** out-of-scope items noticed. **Every item carries `FILED: R-NNN` naming the
+   register row opened for it in THIS session, or `NOT-A-FINDING: <reason>` declaring plainly that it
+   does not warrant one.** Opening the row is the default; declaring is the exception and its reason
+   is the whole of the marker.
+
+   **This wording replaces "documented, NOT acted on" (2026-08-24, R-389), and the old wording was
+   the defect.** "Documented" was satisfied by a paragraph — and `REPORT.md` is overwritten every
+   session, so a paragraph has a lifetime of one session. On 2026-08-23 a live, reproducible finding
+   (only the first broken app per hour reaches the operator) was written under Observations and
+   nowhere else; it had no register row and had to be re-derived the next day. That is the same shape
+   as R-341, and this project's own standard says **a rule without a mechanism is a wish**.
+
+   **The mechanism is gate 11** (`scripts/observations_gate.py`, registered in the repo runners),
+   which refuses a push whose `REPORT.md` carries an observation with neither marker. Note what it
+   deliberately does NOT accept: a passing mention of some other `R-NNN`. The lost item cited `R-182`
+   as an analogy, so "cites a register row" would have passed the very item the gate exists to catch.
 
 ---
 

documentation/architecture/00-capability-map.md

@@ -1,5 +1,20 @@
 # 00 — Felhom Capability Map
 
+> **How to read this document.** Two kinds of statement appear, and where this document marks them it
+> marks them like this — the same wording as `07-backup-architecture.md:11-17`, carried here on
+> 2026-08-22 (R-376) so a reader meets one convention and not eight:
+>
+> - **[DESIGN]** — a decision taken. Not derived from code; the code may not implement it yet.
+> - **[FACT]** — an observed property, carrying a `file:line`, a live command output or a citation.
+>
+> **Statements in this document are NOT yet all marked.** Marking them wholesale is a large judgement
+> exercise and a wrong mark is worse than none, so only what a session touches is marked (R-376).
+> **An unmarked statement therefore means "not yet classified", never "observed".** That ambiguity is
+> exactly what cost this project three sessions in August 2026: the hot/bulk placement decision sat
+> unmarked beside a marked `[FACT]`, and was read as an observation and reported as a defect.
+
+
+
 > **What this is:** the single cross-component truth table of what the Felhom platform can do
 > **today**, at what confidence level, with verifiable evidence. Rows are *scenarios* (user- or
 > operator-visible outcomes), not modules — a scenario spans agent + controller + hub + catalog,
@@ -40,7 +55,38 @@
 
 | **The offsite repository password can be RECOVERED from the sealed escrow with the customer's recovery code** | hub v0.94.0, agent v0.125.0, controller v0.195.0 | **PROVEN-LIVE (2026-08-04)** | On demo-felhom, through the real endpoints end to end: the box fetched its own sealed blob from the hub with its own per-host credential (hub log: *escrow blob SERVED … 572 opaque bytes, self_scope=true*), the agent unsealed it with the customer's recovery code, and the extracted repository password's sha256 was **byte-identical** to the one on disk — `c60c8bc737a6…`, which is ALSO the hash the hub had independently stored, so three sources agree. Five minutes earlier the same path with a WRONG code failed closed at age's KDF with nothing written, which proves links 6 and 7 ran independently of the success. R was searched for afterwards and found in 0 log lines and 0 files, with a positive control confirming the search would have found it. Evidence: per-repo CHANGELOGs; `audits/RECON-offsite-dr-chain-2026-08-04.md` §3 links 6–8 | **WHAT THIS ROW DOES NOT CLAIM, stated because the previous over-claim here was struck out four hours earlier.** It covers the KEY, not the DATA. **R-199 BACK-POINTER (omitted when this row was written): the recovery chain's link inventory and the per-link status live in `audits/RECON-offsite-dr-chain-2026-08-04.md` §3; links 1–8 are walked, 9–11 are not.** **Re-confirmed 2026-08-04 evening by an attempt to prove the DATA half:** the R-201 drill was prepared on demo-hp and **halted before the wipe** — the sentinel file was not in the off-site snapshot (R-203), so the wipe would have destroyed it and proven nothing. **No file has still ever been restored from an off-site backup after a wipe** (`audits/DRILL-r201-offsite-recovery-2026-08-04.md`). The install half of the chain (controller v0.196.0 `--recover-offsite-install`, R-200) is likewise unit-proven only — it has never run against a live recovery. A recovered password has never been **installed** (the diagnostic compares and refuses to write, by design), no existing repository has ever been **reopened** under one, and **no file has ever been restored** from an off-site history via a recovered key. Links 9–11 of the chain are open (R-200's remaining half, R-201). The proof also used a box whose local key still exists — the rebuilt-box case, where there is nothing to compare against, is exactly what the drill covers and it has not run || DR tier by default: PBS + WireGuard base infra on every install, hub-controlled activation | installer v1.15, agent v0.86, hub v0.51 | **PROVEN-LIVE** (2026-07-21) | `DRILL-day0-take2-2026-07-12` §2 (WG enabled both modes, PBS-DR descriptor auto-provisioned ~1s after WG registration, zero operator steps); ships installer v1.15/agent v0.86/hub v0.51 | Live only on demo/drill fleet. (Cited spike was slice-0 mechanics — shipped nothing; corrected.) **⚠ The candidate upgrade to PROVEN-LIVE is WITHDRAWN — the 2026-07-18 rehearsal produced a live counter-example (R-39).** On the reborn N100 the descriptor auto-provisioned and the agent reported `converged state=applied` (16:45:53), yet **the storage is dead**: `pvesm status` → `felhom-pbs: error fetching datastores - 401 Unauthorized` / `inactive`, and a direct probe with the stored credential returns **401 on every endpoint including `/version`** while the WG transport is healthy (handshake 9 s, 27.9 ms RTT) — i.e. authentication failure, not ACL scope. Root cause in the evidence: **the hub minted a SECOND token secret at 16:47:52, two minutes after the agent had applied the first, and `consumed_at` is still NULL**; the converged state machine will not re-apply, and the agent's 15-minute verify loop **cannot even read the credential to notice** (`open /etc/pve/priv/storage/felhom-pbs.pw: permission denied` — non-root agent reading a file it writes through a root wrapper). A tier that reports `applied` while silently unable to authenticate is exactly the shape that must not carry a PROVEN-LIVE badge. See `tests/VALIDATION-n100-rehearsal-2026-07-18.md` F2 and `pbs-dr-state.txt`. **agent v0.89.0 closes the F4 non-default-storage-id gap (R-22) — PROVEN-LIVE 2026-07-17:** the reconcile self-grants the ACL through the root wrapper on a pre-check 403 instead of dead-locking. Reproduced F4 on the demo (marker moved aside = reinstall fresh-state + felhom-offsite ACLs revoked) → next reconcile tick `pbsdr: pre-check 403 … self-granting … (R-22)` → `converged state=adopted` in ~3 s, ACLs self-restored, `pvesm status felhom-offsite`=active, zero operator action. No more one-shot `pveum` grant  **2026-07-21 — the R-39 fleet fix SHIPPED (hub v0.68.0 + agent v0.91.2), closing the self-heal chain end to end.** The three defects that let a box be `applied` and dead simultaneously are each addressed: the hub stamps a monotonic `secret_generation` into the descriptor so a credential re-key finally MOVES the content hash the agent Line truncated
 
-| **A customer's file survives a machine rebuild and comes back — the whole off-site story, end to end** | hub v0.94.0, agent v0.125.0, controller v0.197.0 | **PROVEN-LIVE (2026-08-04 night drill)** | demo-hp's controller data volume was destroyed and the sentinel deleted from disk. The customer's recovery code then produced `8a9e33aa4da6…` — **byte-identical** to the pre-wipe on-disk key and to the hub's independent record — and installed cleanly on the bare box. `identity_blob` was unchanged across the wipe (572 B, `updated_at` still 11:11:37). Evidence: `audits/DRILL-r201-night-run-2026-08-04.md` §2 | **WHAT IT DOES AND DOES NOT CLAIM.** PROVEN: after a real rebuild the key recovers byte-identical, the EXISTING repository opens (3 snapshots, 42 026 B — the pre-wipe size; not a fresh history), and the customer restore flow returns the file byte-identical. **ALL FOUR MANUAL INTERVENTIONS ARE CLOSED, AND THE CUSTOMER IS NOW OFFERED THE RECOVERY (2026-08-05). ONE QUALIFIER REMAINS AND IT IS NOT THE OLD ONE — state it precisely.** The 2026-08-04 drill needed four interventions between "the key is recoverable" and "the file is back", none of them in any design document (R-204). **Items 1–3 closed in controller v0.198.0 + hub v0.95.0** (the reset code works first time; a Re-issue no longer marks a healthy escrow stale; a unit restore states it returned the app's definition and database and NOT the customer's files). **Item 4 closed in controller v0.199.0 + hub v0.96.0** (a rebuilt box DECLARES `offsite.state=needs_credential` and `internal/offsiteheal` re-arms the stored credential before minting). **AND THE "needs someone who knows to look" HALF IS NOW GONE TOO (controller v0.200.0, R-193):** a full-page recovery screen takes over the landing pages while the hub holds a sealed package this box cannot open, explains that nobody can replace a lost recovery code, takes the code, opens the repository and lists what is in it — proven live on demo-felhom, which is genuinely in that state (`/launcher` → 302 `/recovery`; a wrong code refused with nothing written, three times, no lockout; the code found in no file, no container log and no debug ring, with a planted-copy positive control that first exposed a mis-aimed sweep). **WHAT THE QUALIFIER IS NOW, and it is narrower:** (1) **the final unlock has never been driven with a CORRECT code through the page** — no recovery code was kept for demo-felhom's orphaned history and demo-hp's is operator-held out of band, so the live run exercised the whole chain (handler → agent → hub fetch → age KDF) and stopped at the unseal; the install and listing rest on tests. (2) **putting files back in place is deliberately NOT part of this** — restore stays per-app, and the step after the listing is R-213. (3) **the journey has still not been re-walked end to end since these fixes** — the closures are proven individually, not as one uninterrupted run. That re-walk is one more drill and it is what is owed. Scope: `demo-hp`, a controller-data-volume rebuild — **NOT** a total host loss, and **NOT** a guest reprovision. **⚠ THE RE-WALK RAN ON 2026-08-05 (CAMPAIGN-11) AND IT FAILED. THIS ROW STAYS AS IT IS — it records a key recovering and a file returning, which remains true — but the JOURNEY row below is the honest verdict and this row must not be read as covering it.** The campaign built a throwaway appliance from the published ISO, destroyed it, and walked the customer's route: **the data half PASSED** (three sentinels byte-identical, including a 12 MB binary and a non-ASCII Hungarian filename, out of the pre-wipe snapshot in 16 s) and **the journey half FAILED** — four operator interventions, three needing root on the appliance, and the first of them was the machine telling the customer their perfectly correct recovery code was wrong (R-216). **R-198's retention is NO LONGER unit-proven — it was PROVEN IN PRODUCTION on 2026-08-05** (CAMPAIGN-11 Phase 3): the first supersession since the fix retained `identity_blob` at **572 B, byte-length exact**, carrying the OLD key `626e424670248db3` while the current row moved to the newly minted `e11a6c542b73477a`; `offsite_repo_key_changed` and `escrow_superseded` both fired at the instant of supersession, and the next off-site run REFUSED as `orphaned` rather than starting a fresh history. Evidence: `tests/campaign11-evidence-2026-08-05/journal.md` |
+| **A customer's file survives a machine rebuild and comes back — the whole off-site story, end to end** | hub v0.94.0, agent v0.125.0, controller v0.197.0 | **PROVEN-LIVE (2026-08-04 night drill)** | demo-hp's controller data volume was destroyed and the sentinel deleted from disk. The customer's recovery code then produced `8a9e33aa4da6…` — **byte-identical** to the pre-wipe on-disk key and to the hub's independent record — and installed cleanly on the bare box. `identity_blob` was unchanged across the wipe (572 B, `updated_at` still 11:11:37). Evidence: `audits/DRILL-r201-night-run-2026-08-04.md` §2 | **WHAT IT DOES AND DOES NOT CLAIM.** PROVEN: after a real rebuild the key recovers byte-identical, the EXISTING repository opens (3 snapshots, 42 026 B — the pre-wipe size; not a fresh history), and the customer restore flow returns the file byte-identical. **ALL FOUR MANUAL INTERVENTIONS ARE CLOSED, AND THE CUSTOMER IS NOW OFFERED THE RECOVERY (2026-08-05). ONE QUALIFIER REMAINS AND IT IS NOT THE OLD ONE — state it precisely.** The 2026-08-04 drill needed four interventions between "the key is recoverable" and "the file is back", none of them in any design document (R-204). **Items 1–3 closed in controller v0.198.0 + hub v0.95.0** (the reset code works first time; a Re-issue no longer marks a healthy escrow stale; a unit restore states it returned the app's definition and database and NOT the customer's files). **Item 4 closed in controller v0.199.0 + hub v0.96.0** (a rebuilt box DECLARES `offsite.state=needs_credential` and `internal/offsiteheal` re-arms the stored credential before minting). **AND THE "needs someone who knows to look" HALF IS NOW GONE TOO (controller v0.200.0, R-193):** a full-page recovery screen takes over the landing pages while the hub holds a sealed package this box cannot open, explains that nobody can replace a lost recovery code, takes the code, opens the repository and lists what is in it — proven live on demo-felhom, which is genuinely in that state (`/launcher` → 302 `/recovery`; a wrong code refused with nothing written, three times, no lockout; the code found in no file, no container log and no debug ring, with a planted-copy positive control that first exposed a mis-aimed sweep). **WHAT THE QUALIFIER IS NOW, and it is narrower:** (1) **the final unlock has never been driven with a CORRECT code through the page** — no recovery code was kept for demo-felhom's orphaned history and demo-hp's is operator-held out of band, so the live run exercised the whole chain (handler → agent → hub fetch → age KDF) and stopped at the unseal; the install and listing rest on tests. (2) **putting files back in place is deliberately NOT part of this** — restore stays per-app, and the step after the listing is R-213. (3) **the journey has still not been re-walked end to end since these fixes** — the closures are proven individually, not as one uninterrupted run. That re-walk is one more drill and it is what is owed. Scope: `demo-hp`, a controller-data-volume rebuild — **NOT** a total host loss, and **NOT** a guest reprovision. **⚠ THE RE-WALK RAN ON 2026-08-05 (CAMPAIGN-11) AND IT FAILED. THIS ROW STAYS AS IT IS — it records a key recovering and a file returning, which remains true — but the JOURNEY row below is the honest verdict and this row must not be read as covering it.** The campaign built a throwaway appliance from the published ISO, destroyed it, and walked the customer's route: **the data half PASSED** (three sentinels byte-identical, including a 12 MB binary and a non-ASCII Hungarian filename, out of the pre-wipe snapshot in 16 s) and **the journey half FAILED** — four operator interventions, three needing root on the appliance, and the first of them was the machine telling the customer their perfectly correct recovery code was wrong (R-216). **R-198's retention is NO LONGER unit-proven — it was PROVEN IN PRODUCTION on 2026-08-05** (CAMPAIGN-11 Phase 3): the first supersession since the fix retained `identity_blob` at **572 B, byte-length exact**, carrying the OLD key `626e424670248db3` while the current row moved to the newly minted `e11a6c542b73477a`; `offsite_repo_key_changed` and `escrow_superseded` both fired at the instant of supersession, and the next off-site run REFUSED as `orphaned` rather than starting a fresh history. Evidence: `tests/campaign11-evidence-2026-08-05/journal.md` **RE-WIDENED 2026-08-22, and the narrowing below is now HISTORY — read both, in order.** Both defects named in the 2026-08-21 narrowing are closed and proven live: **R-354** (controller v0.218.0, `volReplay`) and **R-356** (controller v0.219.0). The **40-class end-to-end story is now WALKED**, including the hardest ten of it: `audits/DRILL-r356-hot-only-restore-2026-08-22/` proved a driveless app with NO database (`privatebin`: planted, off-sited, deleted, restored, **15/15 files byte-identical**, two Hungarian accented names), and `audits/DRILL-r356b-drivLine truncated
+It was read from source that a HELD app would raise the dead-app banner and a customer e-mail, because
+it keeps one container (its database) and so is not `StateStopped`. **It does not.** A held app
+aggregates to `unhealthy`, and `aggregateState` checks `unhealthy > 0` before the mixed-case degraded
+branch while `IsDownState` excludes `unhealthy` entirely. Measured on `demo-hp` on the shipped
+v0.220.2: hold created 21:11:19Z, dead-app scans every 30 s ran over it, and the heartbeat reported
+`0 currently down` throughout. **No suppression was built, because there was nothing to suppress.**
+The detector itself is sound — `classifyRunStates` is pure and a `degraded`/`exited` stack does raise
+the banner, pinned by `TestClassifyRunStates_PositiveControl_ADownStackDoesAlarm`. What the same
+measurement DID expose is **R-384**: an app whose database has died is `unhealthy` too, and is
+likewise silent. Evidence: `audits/DRILL-r361-2026-08-22/evidence/06-part3-decision.txt`.
+
+> **R-384 CLOSED in controller v0.222.0 (2026-08-23), proven live.** The defect was the ORDER of two
+> questions, not the `unhealthy` exclusion: `aggregateState` now asks *"is a supervised member dead?"*
+> **before** the `unhealthy`/`starting`/`restarting` returns, and "some members are up" counts any
+> member not in the down bucket rather than `running` alone. `IsDownState` is byte-identical.
+> **Measured on `demo-hp` 2026-08-23** with the same fixture that read `0 currently down` the day
+> before: `bookstack-db` stopped 05:30:07Z → `app_start_failed` fired at **05:30:14Z**, the banner read
+> *„Telepített alkalmazás nem fut: BookStack (degraded)"*, the stack read `state=degraded` **while its
+> front end was `unhealthy`**, and the heartbeat printed **`1 currently down`** against the previous
+> day's `0`. Evidence: `audits/DRILL-r384-dead-db-alarm-2026-08-23/`.
+>
+> **The HELD-app half of the paragraph above is now also covered** — a held app keeps its database
+> container, so it is the same shape and reaches the same `degraded` verdict.
+>
+> **The alarm ladder that decides all of this now has an owning document:** see
+> `08-alarm-ladder.md` (written 2026-08-23 — before that date no document owned it, and that absence
+> is why the ordering defect was legible only from source).
+
+**WHAT IS STILL NOT CLAIMED:** the FAILURE path is where this class is weak, not the success path — a corrupt dump leaves the customer with an emptied or partially-applied database and an undo copy **no product action can apply** (**R-379**), and on MariaDB it does so behind an app that reports `health=healthy` (**R-380**). The success story is proven; the recovery-from-a-bad-restore story is not.
+
+**NARROWED 2026-08-21 by the backup-truth drill — kept as history; both halves have since closed, see the entry immediately above.** Proven that night on `demo-hp` with planted, hash-recorded files: the **declared-userdata** leg of a **drive-declaring** app does come back byte-identical (`calibre-web`, 5/5 including two Hungarian accented filenames). **Two legs of the same story do NOT:** (a) the off-site restore has **no named-volume leg at all**, so an app's volume tar sits in the unit, in the snapshot and in the checking folder and is never replayed (**R-354**); (b) for the **40 of 53** apps that declare no data drive the off-site restore **refuses outright**, saying a running app „nincs telepítve" (**R-356**). Since the 40-class keeps ALL its data in named volumes, the end-to-end story is **unproven for that class and disproven for the volume leg generally**. The escrow/key half of this row is untouched by that and still stands. Evidence: `audits/DRILL-backup-truth-2026-08-21/evidence/` and `REPORT.md` (2026-08-21). |
 | **The customer's own UNAIDED recovery journey, end to end** | controller v0.206.0, hub v0.98.0, agent v0.127.0 | **PROVEN-LIVE (2026-08-07, the fifth walk) — an unaided customer journey on a real installation, SCOPED: see what it does not claim** | A throwaway appliance was built from the published ISO on demo-hp, claimed, given three sentinels, escrowed, backed up off-site, then destroyed — guest purged and both enrolled drives wiped. The rebuild and the customer's route were then walked with **no command line inside the guest**. **Data: PASS.** All three sentinels byte-identical (`beb9175d…`, `7c8cb0ad…`, `e012e76f…`), including a 12 MB binary and `C11-őrszem-ékezetes-árvíztűrő.txt` — the Hungarian filename survived disk → restic → SFTP → Storage Box → restore → disk intact. Restored in **16 s** out of the pre-wipe snapshot `f3d9cd67`, non-destructively. **Journey: FAIL.** Four interventions, three of them root on the appliance: R-216 (a correct code called wrong), R-218 (the store never opened; the box stopped asking), R-219 (the promised listing structurally could not render), R-220 (the app could not be redeployed — its drives were unenrollable). Evidence: `tests/campaign11-evidence-2026-08-05/journal.md` | **WHAT IT DOES NOT CLAIM, and what the v0.201.0/v0.97.1 fixes do and do not change.** Six of the nine findings are fixed and each is red-proofed (R-216, R-218, R-219, R-217, R-222, R-215) — but **fixes are not a journey.** This row goes green only when the walk is repeated end to end and completes with no operator intervention. **Three findings are deliberately still open and each is a live blocker for some flow:** **R-214** (the console never stops showing a stale pairing code), **R-220** (after a rebuild the customer's drives cannot be re-enrolled — the deploy refuses and the wizard's list is empty, so no app can be redeployed onto its own data drive), **R-221** (a rebuilt box cannot run the escrow ceremony at all, because the pbsdr convergence marker survives on the host while the seeded `escrow.pbs_storage_id` is rewritten out of `agent.json`). **And R-216's fix does not by itself make a NEW box work:** with the hub guard corrected but the Day-0 manifest still vouching agent 0.120.0, a new box is **HELD, not served** — it stops being lied to, which is better, but the feature does not work for it until agent 0.125.0 is vouched.  **⚠ CAMPAIGN-11 PHASES 2 + 4 RAN OVERNIGHT 2026-08-05/06 AND THIS ROW STAYS **FAIL**.** **What they add:** eleven injected faults against the recovery journey, each with a positive control, and a full unattended scheduled cycle. **The backup promise strengthened** — the off-site tier ran itself at 04:15 on a box rebuilt twice and set aside hours earlier (`snapshot_count` 1 → 2), all five daily jobs fired exactly once, and nothing on the must-not list fired. **R-217's and R-215's fixes were proven live under exactly their faults**, the set-aside proved it does not delete (12 535 KB byte-exact at the far end), and a wrong code was refused three times with nothing written and no lockout. **What they do NOT add: any progress on the JOURNEY.** These faults are **not a re-walk** — no customer route was walked end to end — and four new findings say the journey got no better where it matters: **R-224** (a hub outage and a stopped agent are BOTH reported to the customer as a bad recovery code, in 0.056 s and 0.030 s — no unseal attempted; the agent's own `err` distinguishes them and it is discarded at the HTTP boundary, and R-216's gate answers `source=version` so it cannot see reachability — **Phase 1's headline defect relocated from the version channel to the transport**), **R-226** (M1, the only message telling a customer to check their typing, is unreachable on any box that has re-escrowed), **R-225** (an unread store renders `0 pillanatkép · 0 GB` above a card saying it holds backups — 1 snapshot and 12 535 KB were really there), **R-228** (the set-aside history is recorded in `orphaned_renamed_to` and shown by nobody — `OrphanedRenamedTo` has zero references in any template or handler). **§4.1 is now MEASURED rather than deduced** — the floor IS served, from the box's own rendered `GetFloor()` and a cold-started controller's settle-gate line. **§4.2's positive half is still NOT measured** and needs a rebuild. Evidence: `audits/CAMPAIGN-11-recovery-journey-2026-08-05.md`, `tests/campaign11-evidence-2026-08-05/journal-phase24.md`  **2026-08-06 — THE FIVE PHASE-2 FINDINGS ARE FIXED AND THIS ROW STILL SAYS FAIL.** controller **v0.202.0** + agent **v0.126.0** close R-224, R-226, R-225, R-227, R-228. **What changed:** the unlock path now classifies WHY it failed, from the value and never the text, and the message that mentions typing is reachable from exactly one class — a real refusal — with **unknown defaulting to neutral**. Proven live on the venue with the same wrong code and only the hub's reachability changLine truncated
 | **Off-site app-data capture covers the paths an app declares MANDATORY — on BOTH drive layouts** | controller v0.197.0 | **PROVEN-LIVE (2026-08-04) — and this row was OPTIMISTIC before it** | On demo-hp, an app on the **system-data fallback** bound `/mnt/sys_drive/userdata/media/books` while the capture set looked in `/mnt/sys_drive/felhom-data/userdata/media/books`: the declared-mandatory directory was in **no** snapshot and the run reported `ok` (R-203). After v0.197.0 the capture log reads `1 mandatory path(s)` and the file is **listed inside the snapshot** — `restic ls -l latest --tag calibre-web` → `-rw-r--r-- 1000 1000 181 … /DRILL-SENTINEL.txt`. Evidence: `audits/DRILL-r201-offsite-recovery-2026-08-04.md` §2 and the v0.197.0 CHANGELOG | **WHAT IT DOES NOT CLAIM.** It covers CAPTURE, not RESTORE: **no file has ever been restored from an off-site backup after a wipe** (R-201, ready to resume). It also does not claim the enrolled-drive layout was ever wrong — it was not, which is precisely why this survived. And a run that still misses a mandatory directory now reports **`incomplete`** rather than `ok`, so this row's guarantee is one the status can express. **Widened 2026-08-06 (controller v0.205.0, R-234):** the same verdict now also covers an app skipped ENTIRELY — until then a missing declared FOLDER made the run incomplete while an app with no recovery unit at all still reported `ok`, so the smaller gap moved the verdict and the bigger one did not. **This row still claims CAPTURE, not that a newly-selected app is protected by the next run** — for a DEPLOYED app it is (the run's own pre-dump phase writes the unit, measured 2026-08-06), and for an undeployed one it is not and the card now says so || **Recurring offsite (PBS) whole-guest backups actually LAND, and RESTORE** — local daily + offsite weekly as scheduled work | agent v0.97–0.103, controller v0.174/0.175, hub v0.76.0, host-install 1.20.0 | **PROVEN-LIVE (2026-07-26)** | `audits/SPIKE-r82-phase0-2026-07-26.md`; per-repo CHANGELOGs/REPORTs. **Restore round-trip on demo-hp:** `--selftest=restore-test` against `felhom-pbs:backup/ct/9201/2026-07-26T15:42:42Z` → `pass:true`, `verified:"boot+running"`, **`mount_parity:"ok"`** (`mp0=/var/lib/docker 50G`, `mp1=/mnt/sys_drive 20G`, mp8/mp9 throwaway stand-ins for the archived binds), `source_tier:"pbs"`, 4m5s restore+boot+verify+teardown, scratch band clean afterwards and the live guest untouched. | **This row is distinct from the DR-tier row above, which proves ACTIVATION, not ARRIVAL.** That tier was PROVEN-LIVE as `applied` since 2026-07-21 while demo-felhom held ONE snapshot (a healing artifact) and demo-hp held **zero, ever** — "applied and empty", the R-39 shape one level quieter. **What earns PROVEN-LIVE here:** (a) demo-hp's FIRST EVER offsite backup landed (4.25 GB into a verifiably empty namespace); (b) it **restores into a bootable, mount-complete guest** — `mount_parity` is the non-hollow half, since a boot-only verify cannot see a missing data volume; (c) **the multi-tier quiesce ran through the real UI endpoint** (`POST /api/guest-backup/trigger`, authed+CSRF) and produced **exactly ONE stop/start pair with BOTH backups inside it** — `quiescing 1 stack(s)` 17:01:39 → local done 17:02:56 *"next tier may start (app still quiesced)"* → felhom-pbs snapshotted 17:03:06 → `unquiescing` 17:03:06. **App downtime 1m27s for both tiers**, and the app came back healthy. **Known gaps, recorded not hidden:** the SCHEDULED restore-test still only selects the PRIMARY tier, so the offsite tier is never AUTOMATICALLY restore-tested (the manual/selftest path is proven, the unattended one is not); the hub infers "PBS ⇒ weekly" from storage TYPE rather than a reported cadence; the installer-default fleet flip awaits a full weekly cycle. → **R-82** |
 | **Restore-proof is UNATTENDED — the scheduler covers EVERY tier, follows the BACKUP rather than the clock, and a failure is heard** | agent v0.104.0 → **v0.121.0**, hub v0.77.0 → **v0.91.0** | **PROVEN-LIVE (2026-08-03)** | per-repo CHANGELOGs; `backlog/SPEC-r85-phase4-5-2026-07-26.md`. Unit red-proofs for tier rotation, restart-survival, the one-heavy-op gate, failure-emits-an-event, and newborn silence. | **The row above is earned by a MANUAL `--selftest=restore-test`; this one is about the SCHEDULED path, and the distinction is the whole point.** Before R-85 the scheduler could only ever see `cfg.Backup.BackupTarget()`, so the offsite tier was never a candidate — and a failed restore-test was a `[WARN]` line with no event at all, which was true for the LOCAL tier that WAS being tested. Now: oldest-first rotation across every configured tier (operator ruling 2026-07-26), persisted so it survives a restart; a restore-test joins the host-wide one-heavy-operation gate so it never contends with a backup over the same tunnel; and the hub raises two DISTINCT operator-tier signals — `restore_test_failed` (broken now) and `restore_test_stale` (unverified, not known-broken), anchored on R-81 so a newborn box never alarms. **Why this was NOT PROVEN-LIVE until now:** rotation had not been observed selecting both tiers across consecutive UNATTENDED cadences — at a 24h cadence a multi-day window — and a single passing run proves the code path, not the schedule. → **R-85**. **R-86 (agent v0.121.0 + hub v0.91.0, 2026-08-03) replaced the schedule and the observation became possible in one afternoon**, because what has to be observed is no longer a multi-day rotation but a RULE: a tier is due when its newest archive that has settled ~24 h has not been proven. **UPGRADE EVIDENCE — a real unattended run on demo-felhom (2026-08-03), triggered by DUE-NESS, not by a timer.** The scheduler's own log: `15:14:38 restore-test tier is DUE … target=felhom-pbs archive=felhom-pbs:backup/ct/9201/2026-07-28T04:49:43Z … reason="newest settled archive … has not been proven"` → `proxmox-backup-client restore --crypt-mode=encrypt` under the agent's own token → `15:25:08 gate decision class=guest_destroy guest=990000 allowed=true` → `15:25:14 scratch guest torn down` → **`15:25:14 backup: scheduled restore-test passed archive=felhom-pbs:… duration_s=635.1`**. A **14.5 GB encrypted offsite archive pulled from ep0 over the WAN**, restored, booted, verified and destroyed in 635 s, unattended. **The three things a timer could not show**, all verified after it: the state names THAT archive (`{"felhom-pbs":{"archive":"…2026-07-28T04:49:43Z","proven_at":"2026-08-03T13:25:14Z"}}`); a second evaluation reports `due=false … is already proven` and runs nothing; and **an agent restart runs nothing**, which is the defect a person actually noticed — every deploy used to restart the timer. Teardown verified at all three layers: guest absent from `pct list`, **zero** `990000` volumes in `lvs`, and the hub-side `restore_tests[]` entry deliberately RETAINED (it IS the proof the staleness check reads). **R-189 (agent v0.122.0, 2026-08-03) closed the reporting half of this row, and it was a REAL gap in the evidence path:** the proof above reached the hub only because no restart intervened — `restore_tests[]` came solely from an in-memory store, so the 15:25:14 PASS was in fact LOST when the agent restarted 2 m 43 s later for a deploy (`0 restore-tests` on the next two host-reports). Under per-archive due-ness the box would not have repeated the work for a week. The persisted per-tier proof (with the archive, and now the tier) is merged into the report, so a proof survives a restart — one entry per tier, newest wins, and a record that cannot be described honestly is not emitted. **THE HOST TIER'S HALF OF THIS ROW WAS OPTIMISTIC UNTIL 2026-08-03, and it is worth saying plainly.** Every live restore-test cited here is on the OFFSITE tier. The HOST tier was not merely unproven on demo-felhom — it was **unprovable**, and on demo-hp equally: the agent's token had no ACL on `/storage/felhom-backup`, the storage both boxes configure as `local_backup_target`, so the content API answered `{"data":[]}` through the token while root listed three archives. The scheduler skipped it as *"no settled archive yet"* — which is exactly what a brand-new tier reports — so nothing ever said so (**R-185**). **CLOSED 2026-08-03 — agent v0.123.0 + installer 1.24.0.** The grant is applied on both demo boxes (the token now lists 3 and 4 archives), the installer's reuse arm grants on a pre-existing target, and the agent now **asks whether it may read each tier it depends on** instead of inferring it from an empty list: a critical degraded capability naming the storage and the missing role, which the hub alerted and emailed on **while the box was still blind**. **THE HOST TIER IS NOW PROVEN-LIVE, UNATTENDED, ON BOTH DEMO BOXES (2026-08-0Line truncated
@@ -56,7 +102,7 @@
 | App lifecycle: start/stop/restart/update/logs/remove/redeploy | controller | **PROVEN-LIVE** | `CAMPAIGN-2` T-LIFECYCLE (stop/start/restart/update/logs); remove live in `CAMPAIGN-3` | Redeploy-after-remove edge remains open (T-REMOVE-REDEPLOY never cleanly passed — stale dryrun journal); non-pilot-critical |
 | Protected infra stacks can't be stopped/removed from UI | controller | **PROVEN-LIVE** | `CAMPAIGN-nomercy` + `RERUN-p1p3` T-SEC-PROTECTED (refuse stop/remove, stay Up) | (Cited `CAMPAIGN-2` T-SEC-PROTECTED was a stale-dryrun FAIL — corrected to the runs with a real server-side refusal) |
 | Catalog sync (git, 15 min) + orphan lifecycle + validation choke point (bad `backup:` block degrades to legacy, loudly) | controller v0.132, catalog | **PROVEN-LIVE** | `CAMPAIGN-2` T-SYNC-IDEMPOTENT; v0.132 LoadMetadata red-proofs | |
-| Lemez-egészség felügyelet: per-disk SMART kártya („Lemezek állapota") + degradáció-riasztás (Rendben/Figyelmeztetés/Hiba/Nincs adat) | agent v0.94.0→**v0.95.0**, controller v0.169.0→**v0.171.0**, hub v0.73.1 | **PROVEN-LIVE (healthy path + delivery)** | **2026-07-25 (v0.95.0 + v0.171.0 — the SMART-coverage fix):** the card on guest 9201 now shows BOTH real disks with **real verdicts + human model labels** — **„AirDisk 512GB SSD" → Rendben (34°C)** (the system SSD, via LVM/dm resolution) and **„TOSHIBA MQ04ABF100" → Rendben (30°C)** (the USB, via union-path SMART). `/disks` carries `smart.health=PASSED` + `model_name` for both. This reverses the 2026-07-24 „Nincs adat on a raw UUID" state (`SPIKE-smart-coverage-2026-07-25.md` had proven both disks answer `smartctl -a -j` PASSED but the agent never asked). Prior: verdict table (+≥90 red-proof); check first-run/degradation/recovery/UNKNOWN tests; hub allowlist test. **Notification pipeline PROVEN-LIVE 2026-07-24** — a `disk_health_degraded` POST (the exact `notify.PushEvent` wire call) was **400-rejected by hub v0.73.0** and **200-accepted + „Operator email sent" by hub v0.73.1** | No new smartctl load; feature-detect by payload presence → **MinAgent floor unchanged**; no sudoers/`-d sat` change. **No global banner** (deliberate). Agent v0.95.0 fixes: union-path SMART (Fix B) + LVM/dm whole-disk resolution incl. the builtin `local` on the LVM root (Fix A, SMART-only — never touches backing/durable_id) + `model_name` capture. **A live FAILING/Figyelmeztetés verdict still needs a genuinely degrading disk** (the demo disks are healthy) — that arm stays unit+red-proof covered. **SMART history/trending (hub-side) PARKED** (ROADMAP R-73) |
+| Lemez-egészség felügyelet: per-disk SMART kártya („Lemezek állapota") + degradáció-riasztás (Rendben/Figyelmeztetés/Hiba/Nincs adat) | agent v0.94.0→**v0.95.0**, controller v0.169.0→v0.171.0→**v0.215.0**, hub v0.73.1 | **PROVEN-LIVE (healthy path + delivery + the severity wire).** **IMPLEMENTED, NOT proven-live: the Hiba-from-counters path** (v0.215.0) — it has never fired on real hardware, only against the committed fixture's values in unit tests (**R-332**) | **2026-07-25 (v0.95.0 + v0.171.0 — the SMART-coverage fix):** the card on guest 9201 now shows BOTH real disks with **real verdicts + human model labels** — **„AirDisk 512GB SSD" → Rendben (34°C)** (the system SSD, via LVM/dm resolution) and **„TOSHIBA MQ04ABF100" → Rendben (30°C)** (the USB, via union-path SMART). `/disks` carries `smart.health=PASSED` + `model_name` for both. This reverses the 2026-07-24 „Nincs adat on a raw UUID" state (`SPIKE-smart-coverage-2026-07-25.md` had proven both disks answer `smartctl -a -j` PASSED but the agent never asked). Prior: verdict table (+≥90 red-proof); check first-run/degradation/recovery/UNKNOWN tests; hub allowlist test. **Notification pipeline PROVEN-LIVE 2026-07-24** — a `disk_health_degraded` POST (the exact `notify.PushEvent` wire call) was **400-rejected by hub v0.73.0** and **200-accepted + „Operator email sent" by hub v0.73.1** | No new smartctl load; feature-detect by payload presence → **MinAgent floor unchanged**; no sudoers/`-d sat` change. **No global banner** (deliberate). Agent v0.95.0 fixes: union-path SMART (Fix B) + LVM/dm whole-disk resolution incl. the builtin `local` on the LVM root (Fix A, SMART-only — never touches backing/durable_id) + `model_name` capture. **2026-08-14 — a genuinely failing disk HAS now been seen, and it broke three assumptions** (`audits/DIAG-smart-passed-trap-2026-08-14.md` + two committed fixtures: raw `smartctl -a -j` and 406 `smartd` lines from ST3000VX010 S/N Z6A07P2G). **(1)** `smart_status.passed` is STRUCTURALLY incapable of failing on unreadable sectors — attrs 187/197/198 all carry `thresh: 0` and a normalized value floors at 1 — so the drive read PASSED at 352 pending sectors and 1001 uncorrectable reads. **(2)** The alert it did produce carried severity `"warn"`, which the hub coerces to `info` and never emails: **the counterfactual is ZERO emails about this drive** (R-328, fixed controller v0.215.0, and the `warning`-vs-`warn` pair proven side by side in `notification_log` on 2026-08-14 — `sent` vs no row at all). **(3)** The old check spoke once and forgot on restart, so between 8 and 352 sectors it emitted nothing. v0.215.0 adds the sustained/count/heat Hiba rules, persisted state and an hourly cadence. **The verdict half of that arm remains unit+red-proof covered only** — no live drive has reached Hiba from counters (R-332). **SMART history/trending (hub-side) PARKED** (ROADMAP R-73) |
 | App crashes → customer notified (one event per transition, no flapping spam) | controller v0.120, hub v0.48 | **IMPLEMENTED** | controller v0.120.0 (dead-app alerting, `app_start_failed`, one-event-per-transition red-proofs); `CAMPAIGN-3` F11 surfaced the gap | End-to-end crash→customer-email delivery never live-confirmed (6B deferred / 6C inconclusive: clean stop ≠ crash); anti-spam unit-proven |
 | Post-deploy optional config (API keys etc.) with restart | controller, catalog `.felhom.yml` | **IMPLEMENTED** | feature long-standing; config page renders (`CAMPAIGN-2` T-PAGE-ALL is GET-only) | The config-save+restart flow is exercised in no campaign (CAMPAIGN-3 explicitly skipped interactive app config). Demoted: T-PAGE-ALL is a page-render smoke test, not this flow |
 | Backup classification: 13 bind-bearing apps carry mandatory/optional/excluded classes | catalog, controller v0.132–133 | **PROVEN-LIVE** | `SPIKE-backup-classification-2026-07-14`, `CAMPAIGN-6D/6E` | Remaining ~39 apps are legacy-class by design (unit-only offsite) |
@@ -126,6 +172,7 @@
 | Multiple household users / per-person accounts | — | **MISSING** | — | Single dashboard password; acceptable for alpha → R-15 |
 | WireGuard base infra always-on; OOB operator access (felhom-sshd, /32 peer) | agent v0.72, hub v0.35 | **IMPLEMENTED** | `SPIKE-oob-wg-operator-peer-2026-07-05`, `SPIKE-felhom-sshd-2026-07-05` | Mutual-repair desired-state arc not built → R-13. **CHECKED 2026-08-08 (R-260) and this row was NOT claiming something untrue** — it claims the capability is implemented, never that it is monitored, so no correction was owed. What WAS untrue is narrower and sat one layer down: **the hub's own OOB health check could not see whether the operator's key was installed.** `HostOOBRow` mirrored five of the agent's eight OOB fields, so `operator_key_configured` — emitted every heartbeat since agent v0.72.0, i.e. from this row's own vintage — was discarded by `encoding/json` on arrival, and `oobDegraded` returned `ok` for a box with felhom-sshd active, reachable, a valid config, a configured peer and **no operator key at all**. `operator_peer_configured`, which it did read, only says the peer IP is in desired-state — that OOB is MEANT to work, not that entry is possible. Fixed hub v0.99.0; the missing key now degrades and the alert NAMES it; a stanza too old to carry the field is reported distinctly and is never a silent ok. Pinned end-to-end from raw report JSON by `TestHostOOB_MissingOperatorKey_EndToEnd` and `TestHostOOB_NoKeyField_IsNotSilentlyOK_EndToEnd` |
 | The operator can see WHERE a managed host is — its LAN address and its WireGuard address, on the host page | agent **v0.119.0**, hub **v0.85.0** | **PROVEN-LIVE** (2026-07-31) | `audits/host-addresses-visible-2026-07-31.md` | Before this the LAN IP was **not reportable at all** — `HostMetrics` carried no address of any kind — and the WG IP existed only in `/offsite`'s peer table keyed by pubkey (peer→host, never host→peer). New wire field `addresses[]`, one row per (interface, address); `IsGlobalUnicast()` is the whole filter, chosen by MEASURING both demo boxes, and it needs no veth/fwbr denylist because that plumbing carries no IP. Rendered live on both 0.119.0 hosts matching their `ip addr` ground truth exactly. **Two honesty properties carry the risk and are both red-proofed:** WireGuard shows the hub ALLOCATION and whether the box CONFIRMS holding it (allocation alone cannot tell a live tunnel from a peer never applied), and an agent below 0.119.0 renders **UNKNOWN, never "no addresses"** — proven live on `drill-r50-0a4f9a` (0.113.0). **Not covered:** a two-LAN-bridge box and a real WG drift, neither of which exists to observe |
+| The operator can see whether a managed host's **guests still have working networking** — and **how often the watchdog had to repair them** | agent **v0.92.0** (emitter, 2026-07-21), hub **v0.104.0** (reader, 2026-08-13) | **IMPLEMENTED** | `backlog/OPEN-ITEMS.md` R-319; `hub/internal/web/hosts_guestnet_test.go` (7 tests, fixtures copied verbatim from `demo-felhom-8363b5`'s live `host_reports` row) | The agent emitted `guest_net` on every heartbeat for **twenty-three days** while the string occurred **nowhere** in `felhom.eu/hub/` — stored as raw text in `report_json`, read by nothing (R-260/R-264, the first of that census's readers to be built). **The fact that carries the risk is `heals_last_hour`, not `state`:** a guest the watchdog keeps repairing is healthy at every instant anyone looks, so rendering the state alone would give it a green tick — the failed-disk-drawn-as-a-healthy-empty-disk shape. `heal_succeeded` is decoded beside it, because six FAILED repairs is a guest that is down while six successful ones is a nuisance. **Unknown is never drawn as healthy:** three absences, three sentences (agent < 0.92.0; a capable agent that sent nothing; a guest whose own state the watchdog did not assert), and a malformed stanza degrades to unknown without a 500. **Three red-proofs, each mutation asserted applied by grep before its run**, including the one that matters — removing the unknown branches and watching a silent machine render as healthy. **Positive control that it is WIRED and not merely written: the wire-contract gate's checked-tag count rose 182 → 190** as the eight `guest_net` allowlist entries were deleted (an allowlisted tag is skipped, so leaving them would have meant these fields were never checked) | **IMPLEMENTED, not PROVEN-LIVE, and the distinction is the honest half.** Every scenario is proven against the real wire in tests, and the healthy case renders correctly for the live fleet — but **no machine has ever been observed with a climbing repair count on this card**, because neither demo box has needed a repair since the watchdog shipped. The signal this card exists for has therefore never been seen firing on hardware. It moves to PROVEN-LIVE the first time a real repair count is watched appearing. **No alarm was added, deliberately** (R-319): the incident behind this was about nobody being able to SEE the condition, and a new email on a fleet of two demo machines is untested noise — revisit when a third machine exists or when a count is seen climbing |
 | Break-glass management-plane recovery | agent v0.71, hub v0.84 | **IMPLEMENTED** | `runbooks/break-glass.md` | hub v0.84.0 adds an **operator-SESSION** retrieval path (host page → Console access → Reveal; `POST /hosts/{id}/reveal-recovery-credential`, CSRF-gated, writes a customer-visible `recovery_credential_revealed` event) beside the pre-existing **global-key** one (`GET /api/v1/admin/hosts/{id}/recovery-credential`), which is untouched and stays the route for when the hub UI itself is down. **The credential half is now PROVEN (2026-07-31):** the vaulted `demo-hp-bb76ea` password was verified against the box's own `/etc/shadow` hash AND minted a real PVE ticket — `POST /api2/json/access/ticket` → **HTTP 200, `root@pam`, 367-char ticket**, the exact API the login form submits to. Still IMPLEMENTED rather than PROVEN-LIVE overall, because the path has not been exercised on a REAL lockout (SSH was available throughout). Discovered during that check: the card's Copy button shipped `disabled` until a Reveal and silently no-opped, leaving ANOTHER host's password in the clipboard — fixed in hub v0.86.0. The vaulted secret is plaintext at rest → **R-133** |
 
 ## F. Notifications & monitoring
@@ -156,5 +203,6 @@
 | Per-customer offsite fill + staleness + freeze lever | hub v0.41 | **IMPLEMENTED** | `OffsiteChecker` (`hub/internal/monitor/offsite.go`): fill 90/95% vs soft quota, staleness >48h | No live-fired leg: `CAMPAIGN-offsite-overnight-2026-07-10` recorded no quota/fill/staleness emails, and the freeze write-block was **inconclusive** (only the Hetzner `readonly:true` API op succeeded). Demoted |
 | **Box-level Storage Box aggregate (total fill, Σ quotas, oversubscription alert)** | hub v0.64.0 (R-5) | **IMPLEMENTED** (data pipeline PROVEN-LIVE) | `monitor.OffsiteBoxChecker` — fetch-throttled Hetzner GET (1/15 min), fill (used/`storage_box_type.size`, 80/90%) + oversubscription (Σ shared+enabled ConfigJSON quotas / capacity, 2.0×), escalation-only operator alert on the customer-less `"pool-box"` scope; Offsite-tab panel + dashboard tile. **Phase-0-pinned** live shape (box 611714) + **live-computed in-cluster:** `0.2% full (2.6 GB of 1.00 TB), Σ shared quota 150 GB, oversub 0.15x`. Tests + 4 red-proofs; hub v0.64.0 REPORT | Two open legs: the **UI render** is unit-verified only (hub UI password-gated → no screenshot); the **alert emails** are unit + red-proof verified but NOT fired live (real pool nominal — a live-fire emails Viktor). Thresholds pending Viktor's ruling (named config keys). READ-ONLY (GET) |
 | **Operator sees PBS DR datastore fill at a glance (Offsite "PBS DR" tab + dashboard gauge)** | hub v0.65.0 + tenantsync v1.2.0 (R-5) | **IMPLEMENTED** (data pipeline PROVEN-LIVE) | The PBS DR datastore (`felhom-offsite` on ep0) fill — NOT a Hetzner box. **Option A:** a read-only `usage` op on the `felhom-tenantsync` ep0 forced command (twin of `fingerprint`; `df` on the datastore path — no customer_id, no admin token, NO mutation), polled by `monitor.PBSDRBoxChecker` (OffsiteBoxChecker clone; 15-min throttle; states ok/unavailable/degraded; fill 80/90% on the `"pbsdr-box"` operator scope). `/offsite` split into Restic + PBS DR tabs; two dashboard gauges. **Graceful: hub deploy ⟂ ep0 update** (ep0 ≤ v1.1.0 → gauge "n/a" until updated). **Phase-0-pinned** (`df` on ep0 PBS 4.2.3) + **live-computed in-cluster** (ep0 updated to v1.2.0 this session): `19.1% full (7.1 GB of 37.2 GB)`. 10 Go tests + a bash harness + 3 red-proofs; hub v0.65.0 REPORT | Open legs: **UI render** unit-verified only (hub UI password-gated); the **fill alert email** is unit + red-proof verified, NOT fired live (datastore nominal at 19%). Separate PBS threshold keys (default 80/90); no oversubscription (namespaces, not quotas). READ-ONLY |
+| **The hub reports LOSS OF VISIBILITY into either off-site store (not just how full it is)** | hub **v0.106.0** (R-339) | **IMPLEMENTED — deliberately NOT proven-live** | Both box checkers count consecutive failed fetch windows and emit `pbsdr_box_unreachable` / `offsite_box_unreachable` (severity `warning`) past a default 3 windows (≈30–45 min), each with a paired `*_recovered` all-clear routed via `recoveredPairedDownTypes` — required because the recoveries are severity `info`, which `severityNotifies` drops. Scopes stay customer-less (`pbsdr-box` / `pool-box`) → operator channel only. Fill logic untouched: a degraded read still drives no band transition. Evidence: `internal/monitor/box_reachability_test.go` + the cross-package wiring test in `internal/notify/`, which asserts an actual operator mail rather than a map entry. **Filed BECAUSE of a measured gap**, not a hypothesis: the 2026-08-18 ep0 outage ran 9 h 37 m with the hub silent | **The gap that remains is R-340**, and it is not small: the ep0 read is the `usage` op, which rides the LOCAL API daemon — the daemon that incident explicitly cleared — so this check would have shown GREEN for that entire outage. It closes "ep0 is unreachable as a host"; it does not close what actually happened. **No live or constructed outage has exercised the emit path**, and one cannot be manufactured against ep0 (Tier 2, protected) |
 | Secrets hygiene: bearer in k8s Secret, no secrets in git, single-quote credential store | hub v0.53, conventions | **IMPLEMENTED** | 07-13 closing bundle | |
 | Operator login password changeable from UI | hub v0.54 | **IMPLEMENTED** | 07-13 | |

documentation/architecture/01-topology-and-trust.md

@@ -1,5 +1,20 @@
 # Felhom Controller Architecture — Part 1: Topology & Trust
 
+> **How to read this document.** Two kinds of statement appear, and where this document marks them it
+> marks them like this — the same wording as `07-backup-architecture.md:11-17`, carried here on
+> 2026-08-22 (R-376) so a reader meets one convention and not eight:
+>
+> - **[DESIGN]** — a decision taken. Not derived from code; the code may not implement it yet.
+> - **[FACT]** — an observed property, carrying a `file:line`, a live command output or a citation.
+>
+> **Statements in this document are NOT yet all marked.** Marking them wholesale is a large judgement
+> exercise and a wrong mark is worse than none, so only what a session touches is marked (R-376).
+> **An unmarked statement therefore means "not yet classified", never "observed".** That ambiguity is
+> exactly what cost this project three sessions in August 2026: the hot/bulk placement decision sat
+> unmarked beside a marked `[FACT]`, and was read as an observation and reported as a defect.
+
+
+
 **Status:** draft (decisions from the topology/trust design sessions).
 **Platform facts** referenced here live in `docs/proxmox-platform.md`; this document
 records *Felhom's decisions*, not Proxmox behaviour.
@@ -147,9 +162,23 @@ credentials.
   at attach), role, encrypted credentials, schedule/retention. The agent creates the Proxmox
   storages, continuously checks presence/reachability, and reports per-target status (a
   disconnected target → actionable notification).
-- **App data placement is per-volume, not per-app:** `.felhom.yml` classifies each volume
+- **[DESIGN] App data placement is per-volume, not per-app:** `.felhom.yml` classifies each volume
   **hot** (DB/config/cache → fast storage, enforced) vs **bulk** (media/files → may be slow).
   A photo app's DB stays on SSD while its blobs go to the USB.
+
+  > **Marked [DESIGN] on 2026-08-22 (R-376), and the pointer is honest about what it can point at.**
+  > **This decision was never recorded as a decision anywhere** — it was established by reading, not
+  > by citation: it exists as this bullet and nowhere else, with no dated entry in the decision log
+  > and no `R-` row. A log entry was written on 2026-08-22 (`CONTEXT.md`, "App data placement is a
+  > DECISION") **to give it a home, not to claim it was decided then**; the choice is older than the
+  > entry and its original date is not on record.
+  >
+  > **What being unmarked cost.** The consequence of this bullet — that 40 of 53 catalogue templates
+  > declare no configurable path because they are all-hot — is stated as **[FACT]** at
+  > `07-backup-architecture.md:296-299`. A reader met a marked observation beside an unmarked choice
+  > and reasonably asked whether it *should* be so. **Between 19 and 22 August that reader called this
+  > decision a defect in four places** (R-370), and one session's work went into correcting the record
+  > rather than into the product.
 - **Backup scoping:** hot data (LXC rootfs) rides the guest `vzdump` → tiers + PBS. Bulk data
   on external mount points is **excluded** from the guest vzdump (per-mount `backup` flag) and
   gets its own per-volume policy (file-level to a tier, slower cadence — or explicitly *not*

documentation/architecture/02-controller-module-map.md

@@ -1,5 +1,20 @@
 # Felhom Controller Architecture — Part 2: Controller Module Map
 
+> **How to read this document.** Two kinds of statement appear, and where this document marks them it
+> marks them like this — the same wording as `07-backup-architecture.md:11-17`, carried here on
+> 2026-08-22 (R-376) so a reader meets one convention and not eight:
+>
+> - **[DESIGN]** — a decision taken. Not derived from code; the code may not implement it yet.
+> - **[FACT]** — an observed property, carrying a `file:line`, a live command output or a citation.
+>
+> **Statements in this document are NOT yet all marked.** Marking them wholesale is a large judgement
+> exercise and a wrong mark is worse than none, so only what a session touches is marked (R-376).
+> **An unmarked statement therefore means "not yet classified", never "observed".** That ambiguity is
+> exactly what cost this project three sessions in August 2026: the hot/bulk placement decision sat
+> unmarked beside a marked `[FACT]`, and was read as an observation and reported as a defect.
+
+
+
 > **EXECUTED (slice 8C, 2026-06-10 — controller v0.37.0).** This map's target state is now realized:
 > the disk-execution subsystem (`storage/*`, restic, cross-drive, drive-restore, `disk_layout`,
 > `local_infra`, `infra_backup`, `setup/scanner`, `monitor/watchdog`+`pinger`, the storage UI) is

documentation/architecture/03-host-agent.md

@@ -1,5 +1,20 @@
 # Architecture Part 3 — The Host Agent
 
+> **How to read this document.** Two kinds of statement appear, and where this document marks them it
+> marks them like this — the same wording as `07-backup-architecture.md:11-17`, carried here on
+> 2026-08-22 (R-376) so a reader meets one convention and not eight:
+>
+> - **[DESIGN]** — a decision taken. Not derived from code; the code may not implement it yet.
+> - **[FACT]** — an observed property, carrying a `file:line`, a live command output or a citation.
+>
+> **Statements in this document are NOT yet all marked.** Marking them wholesale is a large judgement
+> exercise and a wrong mark is worse than none, so only what a session touches is marked (R-376).
+> **An unmarked statement therefore means "not yet classified", never "observed".** That ambiguity is
+> exactly what cost this project three sessions in August 2026: the hot/bulk placement decision sat
+> unmarked beside a marked `[FACT]`, and was read as an observation and reported as a defect.
+
+
+
 > Status: design draft (decision content). To be grounded by Claude Code against
 > `docs/proxmox-platform.md` and `docs/architecture/02-controller-module-map.md`,
 > then placed at `docs/architecture/03-host-agent.md`.

documentation/architecture/04-control-plane-authorization.md

@@ -1,5 +1,20 @@
 # Architecture Part 4 — Control-plane authorization (operator signing)
 
+> **How to read this document.** Two kinds of statement appear, and where this document marks them it
+> marks them like this — the same wording as `07-backup-architecture.md:11-17`, carried here on
+> 2026-08-22 (R-376) so a reader meets one convention and not eight:
+>
+> - **[DESIGN]** — a decision taken. Not derived from code; the code may not implement it yet.
+> - **[FACT]** — an observed property, carrying a `file:line`, a live command output or a citation.
+>
+> **Statements in this document are NOT yet all marked.** Marking them wholesale is a large judgement
+> exercise and a wrong mark is worse than none, so only what a session touches is marked (R-376).
+> **An unmarked statement therefore means "not yet classified", never "observed".** That ambiguity is
+> exactly what cost this project three sessions in August 2026: the hot/bulk placement decision sat
+> unmarked beside a marked `[FACT]`, and was read as an observation and reported as a defect.
+
+
+
 > Status: design draft (decision content), grounded on `docs/tests/phase4-signing-findings.md`.
 > To be reviewed by Claude Code against that spike + `03` §4, then placed at
 > `docs/architecture/04-control-plane-authorization.md`.

documentation/architecture/05-hub-architecture.md

@@ -1,5 +1,20 @@
 # Architecture Part 5 — The Hub
 
+> **How to read this document.** Two kinds of statement appear, and where this document marks them it
+> marks them like this — the same wording as `07-backup-architecture.md:11-17`, carried here on
+> 2026-08-22 (R-376) so a reader meets one convention and not eight:
+>
+> - **[DESIGN]** — a decision taken. Not derived from code; the code may not implement it yet.
+> - **[FACT]** — an observed property, carrying a `file:line`, a live command output or a citation.
+>
+> **Statements in this document are NOT yet all marked.** Marking them wholesale is a large judgement
+> exercise and a wrong mark is worse than none, so only what a session touches is marked (R-376).
+> **An unmarked statement therefore means "not yet classified", never "observed".** That ambiguity is
+> exactly what cost this project three sessions in August 2026: the hot/bulk placement decision sat
+> unmarked beside a marked `[FACT]`, and was read as an observation and reported as a defect.
+
+
+
 > Status: design draft (decision content). To be validated by Claude Code against the **actual
 > felhom-hub source** (`felhom.eu` repo, `hub/`) + Parts 01–04, then placed at
 > `docs/architecture/05-hub-architecture.md`.

documentation/architecture/06-offsite-connectivity.md

@@ -1,5 +1,20 @@
 # Architecture Part 6 — Offsite Connectivity (the backup transport)
 
+> **How to read this document.** Two kinds of statement appear, and where this document marks them it
+> marks them like this — the same wording as `07-backup-architecture.md:11-17`, carried here on
+> 2026-08-22 (R-376) so a reader meets one convention and not eight:
+>
+> - **[DESIGN]** — a decision taken. Not derived from code; the code may not implement it yet.
+> - **[FACT]** — an observed property, carrying a `file:line`, a live command output or a citation.
+>
+> **Statements in this document are NOT yet all marked.** Marking them wholesale is a large judgement
+> exercise and a wrong mark is worse than none, so only what a session touches is marked (R-376).
+> **An unmarked statement therefore means "not yet classified", never "observed".** That ambiguity is
+> exactly what cost this project three sessions in August 2026: the hot/bulk placement decision sat
+> unmarked beside a marked `[FACT]`, and was read as an observation and reported as a defect.
+
+
+
 > Status: **design-of-record** (2026-07-03). Records the settled offsite-backup-transport
 > decisions; grounded against felhom.eu @ `bf099f6` and felhom-agent @ `4ba1b14` (v0.63.0).
 > Evidence base: `documentation/audits/SPIKE-connectivity-wireguard-2026-07-03.md` (all

documentation/architecture/07-backup-architecture.md

@@ -334,12 +334,96 @@ refuses **before** stopping the app and names the action that works.
 |---|---|---|---|
 | Tier-1 | unit incl. volume tars + DB dumps | all of it | none |
 | Tier-2 | unit mirror **+** file legs | `hdd/` and `userdata/` **only** (`tier2_restore.go:101-104`) | **the unit mirror is read by nothing** — `RecoveryUnitPath` resolves to `backups/primary/` (`appbackup/paths.go:46-48`) → **R-102** |
-| Tier-3 | unit (incl. volume tars) + mandatory legs | files + DB replay; the unit is **skipped** on the way to live (`offbox_reconstitute.go:284-289`; placed only if the live unit is absent, `offbox_restore.go:352-356`) | **no offsite action unpacks the named-volume tars it captures** → **R-107** |
+| Tier-3 | unit (incl. volume tars) + mandatory legs | files + DB replay **+ the named-volume tars, replayed from the scratch unit** (`offbox_reconstitute.go` `volReplay`, controller **v0.218.0**); the unit itself is still **skipped** on the way to live (`offbox_reconstitute.go:284-289`; placed only if the live unit is absent, `offbox_restore.go:352-356`) | **CLOSED — R-107** |
+
+**[FACT] 2026-08-22 — the Tier-3 row above was corrected; the Tier-2 row was NOT.** Until controller
+**v0.218.0** this table said *"no offsite action unpacks the named-volume tars it captures"*, and that
+was true from the day Tier-3 shipped until 2026-08-21. **R-107 closed in v0.218.0**: `volReplay`
+(`offbox_reconstitute.go`) replays the scratch unit's `volume-dumps/` into the live named volumes,
+proven live on `demo-hp`. The old sentence is kept here, in the past tense, because a correction that
+erases what was believed leaves the next reader no way to tell a fixed gap from one that was never
+noticed.
+
+**R-102 — the Tier-2 half — is NOT closed and nothing in this correction touches it.** The Tier-2 row
+above stands exactly as written: the secondary unit mirror is still read by nothing. Do not read
+"R-107 closed" as covering both; they were always two register rows, and only one of them moved.
 
 **[FACT]** Tier-2's gap is the sharper one because of *when* it bites: Tier-2 exists for the case
 where the primary drive is lost — and in exactly that case the primary unit is gone while this
 mirror survives on the second drive, unreachable by any customer action.
 
+**[FACT] 2026-08-23 — taking the undo copy used to DESTROY the app's own database backup (R-361).**
+`writeSafetyDump` called `DumpOne` into the app's own unit directory and renamed the result to
+`pre-restore-*` afterwards. `DumpOne` writes `<stack>-<dbtype>.sql` — the app's canonical dump, the
+name the replay loop matches exactly — so **every safety dump overwrote the app's real backup and then
+moved it away**, leaving the app with no database backup of its own until the next nightly run. A local
+restore-from-unit in that window finds no `.sql` and tells the customer the app never had a database.
+The comment beside it asserted the rename meant it *"can never overwrite the app's real dump"*; it was
+false as written and stood for four months. **Measured before the fix on `demo-hp` 2026-08-22:**
+`docmost` and `bookstack` each held only `pre-restore-*` files and no canonical dump.
+
+Fixed in controller **v0.221.0**: `DumpOneTo` takes an explicit final path and derives its own `.tmp`
+from it, so neither the destination nor the scratch file can collide with a nightly dump running
+beside it. **Proven the only way it can be** — the canonical dump's sha256, unchanged across a
+restore, on both engines.
+
+**[DESIGN] 2026-08-23 — `db_dumps` lists the app's OWN dumps, not the undo copies.** They are local
+material for a restore that went wrong, not part of the app's recovery set: nothing reads them from
+the manifest, and three per app were being pushed off-site permanently for no recovery value. The
+files are neither deleted nor hidden — their visibility is a separate recorded decision and it stands.
+**One consequence, recorded because it bit within minutes:** a stable `db_dumps` lets
+`CaptureRecoveryUnit`'s already-current early return fire, so anything that must happen on every
+capture — such as bounding the undo copies — has to sit ABOVE that check, not after it.
+
+**[DESIGN] 2026-08-22 — the failure ladder of a database restore: replay → rollback → hold.**
+Recorded here rather than only in a closed register row, because a decision that survives only inside
+a closed work item is a decision nobody will find.
+
+1. **Replay.** The snapshot's `.sql` is imported into the app's database, with only the DB service up
+   (R-47). A pre-restore copy of the LIVE database was taken first and is on disk (R-43); the restore
+   refuses outright if it could not be taken.
+2. **Rollback (controller v0.220.0, R-379).** If the replay fails, the product re-applies that undo
+   copy itself. **The whole set for this run** — an app with two databases gets two undo files, and
+   restoring only the first would leave the other half-written — matched on **the run's own stamp**,
+   never on the `pre-restore-` prefix, because several runs' copies coexist in the same directory. It
+   runs with the DB service still up and before any restart, so the app never observes the half
+   state, and into a **re-discovered** container: the DB-only start re-creates it, so the id captured
+   at dump time is dead by rollback time (v0.220.1, found by the first live run). The app then starts
+   and the customer is told **both** that the restore failed and that their data is as it was.
+3. **Hold (v0.220.0, operator ruling 2026-08-22).** If the rollback ALSO fails, the app is **held
+   stopped**, not started. A running app on a half-written database lets the customer type into it and
+   makes the damage permanent. The hold is persisted, every start path refuses it with a reason and a
+   route, the app-stop marker is ended so nothing auto-restarts it at the next boot, and the app reads
+   **red** rather than green. An operator clears it with `--clear-restore-hold`, which requires a
+   controller restart.
+
+**Why a rollback and not an engine flag.** Postgres gained `--single-transaction` in the same release
+and that does make its replay all-or-nothing — but **MariaDB's DDL is not transactional**, so a
+partial apply there is unavoidable at the engine. Measured 2026-08-22: the same truncated dump left
+Postgres emptied and crash-looping, and left MariaDB with its user data intact, its schema-version
+table wiped to zero rows, and the app reporting `health=healthy, running=true, restarts=0`. The flag
+is a belt; the rollback is the fix. **R-379, R-380.** Evidence:
+`audits/DRILL-r379-rollback-2026-08-22/`.
+
+**[DESIGN] 2026-08-22 — the restore destination is resolved by the same rule as the capture
+destination.** The drive if the app declares one (`HDD_PATH`), the system data path otherwise —
+`Manager.GetAppDrivePath`, one expression, used by `CaptureRecoveryUnit` and, since controller
+**v0.219.0**, by `ReconstituteFromOffsite` and `PlaceOffsiteRestore` too.
+
+The refusal that protects a drive app from being restored onto the wrong disk (R-253, R-351) applies
+to apps that **have a drive to get wrong**. It used to be reached by testing `HDD_PATH == ""`, which
+also answered "is this app installed?" — one predicate for two questions. Measured in the catalogue at
+`459766cb1639`: **53 templates, 13 declare `needs_hdd: true`, 40 declare `false`**, so for 40 apps that
+test was permanently true and the off-site restore refused them forever, while they were running,
+with a message telling the customer to reinstall them "in the same place" — a place those apps never
+offer. **An app with no drive is not misconfigured** (§8 of `01-topology-and-trust.md` carries the
+`[DESIGN]` marker); it is the majority case.
+
+Since v0.219.0 the two questions are asked separately: *installed?* of `ListDeployedStacks()`, failing
+CLOSED when there is no provider to ask; *where?* of `GetAppDrivePath`. A third refusal, with its own
+sentence, covers installed-but-no-resolvable-data-root. **R-356**; reasoning also recorded in
+`felhom-controller/CONTEXT.md`.
+
 ---
 
 ## 7. The recovery chain (D3) — the reason this document exists
@@ -525,10 +609,14 @@ anywhere under the backup namespace. Full record: `audits/D5-drive-alone-restore
   drive. In that failure the primary recovery unit is gone; the surviving mirror on the second drive
   is `backups/secondary/<app>/recovery-unit/`, which **no code path reads** (§6.3). For the 45-or-43
   class-B apps the restore is a guaranteed no-op in exactly its designed scenario. → **R-102**
-- **Tier-3 vs guest loss.** Tier-3 holds the volume tars and the DB dump. Reconstitution requires
-  the app to be deployed and skips the unit; the tars are unpacked only by the Tier-1 path, which
-  requires the guest's secrets. So offsite alone cannot rebuild an app onto a fresh guest.
-  → **R-107**
+- **Tier-3 vs guest loss — HALF of this closed.** Tier-3 holds the volume tars and the DB dump. It
+  was true until controller v0.218.0 that the tars were unpacked only by the Tier-1 path; since
+  v0.218.0 the reconstitution replays them itself (`volReplay`) → **R-107 CLOSED 2026-08-22**. What
+  is **still** true, and is the part that was never R-107: reconstitution requires the app to be
+  **deployed** and skips the unit, so offsite alone cannot rebuild an app onto a fresh guest. That
+  requirement is a deliberate decision (R-253) — the restore does not choose a customer's drive for
+  them — and since **v0.219.0** it means only what it says: an app that is not deployed. It no longer
+  catches the 40 driveless apps → **R-356**.
 
 ### 7.3 ~~What D5 would change — and why it was blocked~~ — **D5 SHIPPED 2026-07-30 (v0.188.0)**
 
@@ -737,7 +825,7 @@ crosses the line — **R-158**.
 | 3 | **An app's DB and named volumes are lost** | the guest, the unit | „Visszaállítás indítása" — Tier-1 unit restore (the **only** path that unpacks volume tars) | **customer** | **18.25 s** (path execution) · **27.6 s** (D5 drill, guest `app.yaml` absent) | 24 h | **PROVEN** (content recovery proven 2026-07-30) | CAMPAIGN-9 A2 proved the path executes; **D5 v0.188.0 closed the content gap** — after a restore with the guest's `app.yaml` moved aside, the app read the seeded row **over TCP with its own credential**, the pre-backup row returned and a post-backup row was gone (so the tar was really restored). No `.sql` dump in the unit ⇒ the DB came back from the volume tar |
 | 3c | *same, with the GUEST GONE (secrets unavailable)* | the drive | Tier-1 unit restore — **the unit carries the portable secrets** | **customer** | **27.6 s** | 24 h | **PROVEN** | D5, §7.4. Before v0.188.0 this row was **NONE**: the data-key gate refused and the customer's own copy of their own data was not a recovery |
 | 3b | *same, for a class-B app via Tier-2* | — | **no route** — Tier-2 never reads the unit mirror | — | | | **NONE** | §6.3; **R-102** |
-| 4 | **Primary drive dies** | Tier-2 copy on the second drive; Tier-3 offsite; the guest | Tier-2 for **file legs** (7 or 9 of 53 apps); Tier-3 reconstitute for files + DB; **the volume tars in either copy are unreachable** | **customer** (both) | | 24 h | **PARTIAL** | §7.2; **R-102**, **R-107** |
+| 4 | **Primary drive dies** | Tier-2 copy on the second drive; Tier-3 offsite; the guest | Tier-2 for **file legs** (7 or 9 of 53 apps); Tier-3 reconstitute for files + DB **+ the named-volume tars since controller v0.218.0** (`volReplay`); **the Tier-2 copy's volume tars remain unreachable** | **customer** (both) | | 24 h | **PARTIAL** | §7.2; **R-102** (open); **R-107 CLOSED 2026-08-22, v0.218.0** |
 | 5 | **Secondary drive dies** | everything the customer uses | none needed — Tier-2 is a derived copy, rebuilt on the next run (`07` §8 migration rule: *"Migration = rebuild, not preserve"*) | automatic | | 24 h | **PROVEN** (by construction) | tier2 v2 layout marker + rebuild, `internal/backup/tier2.go:359-393` |
 | 6 | **Guest lost or corrupted** | the host, both whole-guest tiers, the data drives (they are host binds) | `pct restore` from `local:` or `felhom-pbs:` | **operator** (SSH) | **84–112 s** local · **1101 s** PBS (both = restore-test into a scratch guest, boot + verify + teardown) | 24 h local · 7 d offsite | **PROVEN** | CAMPAIGN-2 T-P9; CAMPAIGN-8 Phase C (exact mount parity, `unprivileged: 1` preserved); LIVE restore-tests on both boxes this session |
 | 7 | **Guest stopped and does not come back** | everything | guest-power watchdog (60 s, `onboot` as the deliberate-stop discriminator) | automatic | **120 s** | | **PROVEN** | agent v0.107.0 replay — 120 s unattended vs the incident's 587 s with a human |
@@ -899,7 +987,8 @@ does **not** hold as written. → **R-108**
 | **R-104** | An interrupted offsite run leaves an exclusive restic lock the existing self-heal cannot reach, reported as *„ismeretlen okból"* | the offsite tier stays dead until a human unlocks. Was C9-F3 |
 | **R-105** | Three hub-held DR records are empty on the whole live fleet: `hosts.dr_record_json`, `host_escrow.directive_json`, `dr_recipe.host_half.drives` | the Recipe (§4) is incomplete in exactly the fields host-loss recovery reads. Causes may differ per field |
 | **R-106** | `dr_recipe.host_half.pbs.namespace` records `"root"` on every box | the recorded restore coordinate is wrong; real namespaces are per-customer |
-| **R-107** | No offsite action unpacks the named-volume tars Tier-3 captures on every run | offsite alone cannot rebuild a named-volume app (§7.2) |
+| **R-107** | ~~No offsite action unpacks the named-volume tars Tier-3 captures on every run~~ — **CLOSED, controller v0.218.0, 2026-08-22** (`volReplay`, proven live on `demo-hp`). True from the day Tier-3 shipped until 2026-08-21. | was: offsite alone cannot rebuild a named-volume app (§7.2). Now: the tars replay; what remains is that the app must be **deployed** (R-253), which is not R-107 |
+| **R-356** | The off-site restore resolved its destination with the raw `HDD_PATH` and read an empty answer as "not installed" — **CLOSED, controller v0.219.0, 2026-08-22** | 40 of 53 apps were refused permanently while running (§6.3 `[DESIGN]`) |
 | ~~**R-108**~~ | ~~Network storage can host an app's namespace~~ | **CLOSED 2026-07-30, controller v0.187.0 — D5 UNBLOCKED.** An app namespace may no longer be placed on network storage (5 surfaces guarded by one fail-closed predicate); the share-root bind is deliberately UNCHANGED because it is load-bearing and unscopable (§10.1). `audits/R108-network-app-namespace-2026-07-30.md` |
 | **R-126** | A `.fab` bundle — plaintext secrets, optional password — can be exported ONTO a NAS: `storageDriveList()` (`internal/web/handler_export.go`) does not filter network paths | split out of R-108, which closed without it. NOT a D5 precondition: an explicit customer-chosen export destination, not a browsing surface reaching a backup tree (§5, §7.3) |
 | R-95 (open) | The restic offsite credential **can delete** — the box can `forget --prune` its own repo | the tier holding the customer's documents and photos is the one whose credential can destroy it (matrix row 10) |

documentation/architecture/08-alarm-ladder.md

@@ -0,0 +1,268 @@
+# 08 — The app-down alarm ladder
+
+**Written 2026-08-23, with controller v0.222.0 (R-384).**
+
+**The absence is the finding.** Until this file existed, no document owned the question *"when does a
+customer's app being broken raise an alarm?"* The rules were spread across four packages as comments,
+each locally correct, and the ordering between them was legible only by reading
+`aggregateState` top to bottom. That is exactly how R-384 survived: every individual rule was right,
+and the composition was wrong. Three separate defects in this ladder (R-51, C9-F2, R-384) were each
+found on live hardware rather than by review, and each is a case where a reader could not see the
+whole ladder at once.
+
+Everything below is **[DESIGN]** — deliberate, with the reason recorded — unless marked otherwise.
+
+---
+
+## 1. The two questions, and their order
+
+Two different questions get asked about a multi-container app, and **the order between them is
+load-bearing**:
+
+1. **Is a SUPERVISED member of this app dead?** — a container Docker's restart policy says should be
+   running, that is not.
+2. **Is a RUNNING member failing its healthcheck?**
+
+**Question 1 is asked FIRST.** [DESIGN, R-384, v0.222.0]
+
+Until v0.222.0 it was asked second, and the consequence was not subtle: a two-container app whose
+database exits goes `unhealthy` seconds later *because it cannot reach that database*. So the symptom
+the dead database causes was the thing that suppressed the alarm for it. Measured on `demo-hp`
+2026-08-22 — `bookstack-db` stopped at 21:27:01 and the watcher reported `0 currently down`
+throughout.
+
+**"Some members are up" means any member NOT in the down bucket** — `running`, `unhealthy`,
+`starting` or `restarting`. [DESIGN, R-384] The earlier guard was `running > 0`, counting only
+`StateRunning`, which made the supervised test unreachable in precisely the case it was written for:
+an unhealthy survivor beside a dead database counted as nothing being up.
+
+---
+
+## 2. Where each decision is made
+
+| Decision | Where | Notes |
+|---|---|---|
+| container → stack aggregate state | `internal/stacks/manager.go` `aggregateState` | the ladder in §3 |
+| is a down member supervised? | `internal/stacks/manager.go` `supervisedPolicy` | `no`/`on-failure` benign; everything else, **including unknown**, supervised |
+| which states mean "down" | `internal/stacks/manager.go` `IsDownState` | `{stopped, exited, degraded}` |
+| stack state → "this app is down" | `cmd/controller/main.go` `classifyRunStates` | **the single derivation point**; all three suppressions live here |
+| sustained restarting → down | `internal/stacks/manager.go` `CrashLooping` | 5-minute threshold |
+| quiesce suppression | `internal/quiesce/suppress.go` | cycle-keyed, 180 s grace |
+| boot repair | `internal/bootrecon/bootrecon.go` | consumes `IsDownState` |
+
+---
+
+## 3. The aggregation ladder, in order
+
+`aggregateState(containers, policyOf)` — **priority: degraded > unhealthy/starting > restarting >
+all-running > stopped.**
+
+1. no containers → `not_deployed`
+2. **any DOWN member is supervised, and any member is up → `degraded`** ← R-384 put this first
+3. any `unhealthy` → `unhealthy`
+4. any `starting` → `starting`
+5. any `restarting` → `restarting`
+6. all running → `running`
+7. all down → `stopped`
+8. mix, every down member benign → `running`
+
+Step 2's `policyOf` is consulted **only** for the down members, and only when something is up. `nil`
+is allowed; every down member then reads as supervised.
+
+---
+
+## 4. Which states alarm, and which deliberately do not
+
+`IsDownState` = `{stopped, exited, degraded}`.
+
+| State | Down? | Why |
+|---|---|---|
+| `stopped`, `exited` | **yes** | not running, will not recover alone |
+| `degraded` | **yes** | [DESIGN, R-51] a dead supervised member is as unreachable as a single app that exited — immich-server sat Exited 18 h with the app 100 % dead and no alert |
+| `unhealthy` | **NO** | [DESIGN] a running container whose healthcheck is failing. Folding it in reintroduces the flapping fix-3 was written to stop. **R-384 did not change this** — it asks a prior question instead |
+| `restarting` | **NO**, until sustained | [DESIGN, C9-F2] `restarting` is on the normal deploy path, so folding it in would alarm fleet-wide on every update. Becomes down after **5 min** (`crashLoopAfter`) |
+| `starting`, `deploying` | no | mid-start |
+| `paused` | no | a deliberate user action |
+| `unknown` | no | [DESIGN] fail-OPEN — never manufacture a dead-app alert from an inconclusive read |
+
+**Note the two fail directions are deliberately opposite.** `IsDownState` fails OPEN on `unknown`
+(ambiguous *state* → do not alarm). `supervisedPolicy` fails CLOSED on unknown (we already KNOW a
+member is dead; only the excuse is missing). Both are recorded at their sites.
+
+---
+
+## 5. The three suppressions, all at `classifyRunStates`
+
+| Suppression | Rule | Expires? |
+|---|---|---|
+| **deliberate user stop** | `StateStopped` is not down **unless** the quiesce loop reports it failed to restart that stack | n/a — lifted by `failedRestart` |
+| **quiesce cycle** | a stack this backup cycle stopped is exempt | **yes**, 180 s after unquiescing |
+| **boot grace** | no evaluation for 90 s after controller start | **yes** |
+
+**None of them latch.** [DESIGN, R-97b + R-88 Scenario D] A permanent suppression trades a loud false
+alarm for a silent real one, which is the same error as an over-eager alarm, in the opposite
+direction. Every window expires; the cost is a bounded DELAY in reporting a real failure, never its
+loss.
+
+The quiesce suppression is **cycle-keyed, not state-keyed** — an app caught mid-restart is
+`starting`/`unhealthy`, not `stopped`, so no state test can see it. It is therefore **state-blind**,
+which is why R-384 moving a stack from `unhealthy` to `degraded` cannot weaken it.
+
+---
+
+## 6. The alarm itself
+
+Edge-triggered: `app_start_failed`, one event per transition into down, **not** per scan. Verified
+live 2026-08-23 — one event across 22 scans.
+
+- **Operator/hub event + dashboard banner.** `app_start_failed` is **not** in
+  `settings.DefaultEnabledEvents`, so by default it does **not** e-mail the customer.
+- **F-OBS heartbeat**, every 20 scans (~10 min), at `[INFO]`:
+  `[deadapp] check alive: N scans since boot, M deployed app(s) evaluated, K currently down`.
+  This line exists because an absent alarm and a stopped detector look identical in a log.
+
+### 6.1 The severity contract [DESIGN, R-329 — CLOSED controller v0.223.0 / hub v0.107.0]
+
+**The vocabulary is the HUB's and it is exact: `{info, warning, error, critical}`.** Anything else is
+**coerced to `info` at ingest**, and `info` is dropped by `severityNotifies` before *both* delivery
+legs. So a severity outside the set means the event is stored, answers `200`, shows on the dashboard —
+and is e-mailed to **nobody**.
+
+**This shipped twice.** `DiskAlertKind.Severity` emitted `"warn"` until controller v0.215.0;
+`app_start_failed` emitted it until v0.223.0. Measured on the live hub DB 2026-08-23: **91
+`app_start_failed` events stored all-time, ZERO `notification_log` rows before that day** — not one,
+on any channel.
+
+Three things now hold it:
+
+1. **The emitter is pinned by an AST walk** over the whole controller
+   (`TestR329_EveryEmittedSeverityIsInTheHubVocabulary`). Not grep — "warn" is a legitimate
+   *healthcheck status* in `internal/monitor` and `internal/selftest`. The six call sites that pass a
+   variable are registered by name with the values each can take, so a new dynamic path fails.
+2. **The hub SAYS SO** when it coerces (hub v0.107.0, R-387): a `WARN` naming the customer, the event
+   type and the rejected value. **The coercion stays** — a rejected event is a *lost* event, and
+   losing an alarm is worse than mis-routing one.
+3. **The dispatcher's `unrecognized severity` branch is kept**, because the hub's own monitor checkers
+   call `ProcessEvent` directly and never pass the ingest handler. For them it is the only guard.
+
+**Who gets it.** `processOperator` consults only `operatorOn`, the address and a 1-hour cooldown —
+**never customer preferences** — so a valid severity always reaches the operator. `processCustomer`
+consults `operatorOnlyEvents` and then the customer's `enabled_events`.
+
+**`app_start_failed` is customer-switchable but OFF by default** [DESIGN, operator ruling 2026-08-23]:
+it is deliberately absent from `DefaultEnabledEvents`, and deliberately **not** in `operatorOnlyEvents`
+— being in that register would make the toggle visible, flickable and structurally unable to deliver.
+
+---
+
+## 6.2 The delivery grain — how often, and per what [DESIGN, R-389 — hub v0.108.0]
+
+**"How loud" is a separate decision from "does it alarm", and it is made in one place**: the operator
+cooldown key at `processOperator`. Everything sharing a key is collapsed for **one hour**.
+
+| Family | Grain | Key carries | Why |
+|---|---|---|---|
+| app down (`app_start_failed`) | **per APP** | `…:<stack_name>` | no digest exists for it — see below |
+| backup run (`backup_run_failures`) | per RUN | `…:<run_id>` | a digest already lists every failing app; one per run |
+| tiered backup (`whole_guest_backup_failed`, …) | per TIER | `…:<tier>` | the tiers fail independently and mean different things |
+| everything else, incl. `crossdrive_failed` | per TYPE, per hour | — | coarse **on purpose** |
+
+**The default is COARSE and that is deliberate.** R-97a and R-182 exist so that one full disk produces
+**one** mail listing every affected app rather than one per app. Widening the grain is what makes an
+operator stop reading their alerts, which is the same failure as not sending them.
+
+**`app_start_failed` is the exception because it has no digest.** There is no `apps_down_run`
+summarising a scan the way `backup_run_failures` summarises a run, so per-app is the only grain
+available that does not lose alarms. Until hub v0.108.0 it was keyed per type, and **only the first
+broken app per hour reached the operator** — measured 2026-08-23: `bookstack` sent at 09:27:51,
+`privatebin` suppressed at 09:31:51 under `key=demo-hp:app_start_failed`.
+
+**The mechanism is a named ALLOW-LIST (`perAppCooldownEvents`), not a payload rule**, and the
+distinction is load-bearing rather than stylistic: **`crossdrive_failed` is severity `error`, reaches
+the operator leg, and carries `stack_name`** through `CrossDriveDetails`. A rule of the form "if the
+details carry a stack_name, split per app" would have split it, silently, and undone R-182.
+`cooldownStackSuffix` therefore takes the **event type** as well as the details — an asymmetry with
+its two siblings, and the reason for it is exactly this.
+
+**Fenced act:** adding an entry to `perAppCooldownEvents` for a type whose family has a digest, or
+whose coarse cooldown is deliberate. Reading the register anywhere is fine.
+
+**Measured burst, so the volume is a number and not an impression:** three apps stopped in one scan
+produced **three attempted, three sent, zero suppressed** (2026-08-23). The reference box has 8
+deployed apps, so a total outage is 8 mails. The boot grace (90 s), the quiesce grace (180 s) and the
+per-app edge trigger absorb reboots, backup cycles and persistently-dead apps. **This scales linearly
+with app count and has no ceiling** — the condition that would reopen the question is a box large
+enough that a total outage is unreadable, at which point the answer is a digest with a customer
+message, not a wider cooldown.
+
+---
+
+## 7. The intent test [DESIGN, R-386 — CLOSED controller v0.223.0]
+
+**"The customer stopped this" is asked of the FIELD THAT RECORDS IT, never inferred from the state.**
+
+Until v0.223.0 `classifyRunStates` read `st.State == StateStopped` and assumed every stopped stack was
+deliberate. It is not inferable: `aggregateState` folds `StateExited` into the stopped counter, so an
+all-down stack returns `StateStopped` whatever killed it. Measured on `demo-hp` 2026-08-23:
+`privatebin` stopped out of band, nine dead-app scans over four minutes, **zero events, zero banner
+lines** — while a comment beside the code claimed an out-of-band stop *"still alerts"*.
+
+`DesiredState` records the answer, has **exactly one writer** (the customer's own action), and is
+tri-state:
+
+| Intent | Verdict | Why |
+|---|---|---|
+| `Stopped` | **no alarm** | the customer asked |
+| `Running` | **ALARM** | nobody asked — the R-386 case |
+| absent (`""`) | **no alarm, and SAY SO** | UNKNOWN never means running |
+
+**The absent case keeps the old behaviour deliberately.** Reading it as "nobody asked" would, on the
+first cycle after upgrade, e-mail about every app any owner ever stopped — fleet-wide, from a field
+that predates the intent being asked of it. The backfill cannot help: it seeds `Running` only from an
+observed-**up** reading, so anything stopped at upgrade time stays unknown, which is precisely the
+ambiguous population.
+
+**The gap is BOUNDED, not silent.** Every such suppression sets `AppRunState.IntentUnknown`, and the
+scheduler logs the names at `INFO` on the heartbeat cadence:
+
+```
+[deadapp] N stopped app(s) have NO recorded customer intent, so their dead-app alarm is
+suppressed by the unknown-intent fallback (R-386): <names>. This closes itself as each app is
+started or stopped through the interface.
+```
+
+**A rule without a mechanism is a wish.** Measured on `demo-hp` 2026-08-23: **0 of 8 deployed apps had
+an absent intent** — the population is already empty on an exercised box; it will be larger on one
+upgraded and left alone.
+
+`failedRestart` still lifts a `Stopped` intent, and that ordering is load-bearing: the quiesce loop
+stops stacks by the same path a customer does, so one it stopped and could not restart must alarm
+whatever the intent says. Removing that term re-opens F-CRIT-1.
+
+**Fenced act:** adding a `DesiredState` **writer**. Reading it anywhere is fine. Twelve of
+`StopStack`'s fourteen callers are machines, so recording intent in the primitive would make a nightly
+backup indistinguishable from the customer pressing Stop.
+
+---
+
+## 8. Direction — who a customer should be notified about at all
+
+**[DESIGN — DIRECTION, NOT CURRENT BEHAVIOUR. Dated 2026-08-23, the operator's own framing.
+Nothing in controller v0.223.0 / hub v0.107.0 implements this.]**
+
+> **A customer should be notified only about things they can act on or are responsible for** — the
+> drive they unplugged, the storage they filled. **A failed backup is our incident, not theirs.** The
+> intended shape is that we detect it, we tell them we noticed and are dealing with it, and they are
+> not handed an error they cannot solve. The subscription should feel like being looked after, not
+> like being on call.
+
+Today's settings page is the opposite shape: it exposes one toggle per detector and **grew from 12 to
+15 in this session alone** (one new alarm, plus two compound toggles split into four). That growth is
+the argument, not an aside — a page that grows by one per detector is a page that will keep asking a
+household to make engineering decisions.
+
+`app_start_failed` defaulting **off** is consistent with this direction and reversible either way; it
+was ruled that way on its own merits and does not pre-judge the redesign.
+
+**Filed as a PRODUCT DECISION, not a defect** — see the register. It is the operator's call to take
+separately, and no part of it was implemented here.

documentation/architecture/where-felhom-stands.html

@@ -6,8 +6,8 @@
 </style></head><body><div class="wrap">
 <h1>Where Felhom stands</h1>
 <div style="color:#8fa0b8;font-size:12.5px;margin:7px 0 4px;max-width:1100px;line-height:1.5">What we built, what happens when things go wrong, and what is still missing. <b style="color:#cbd7e8">Generated from <code>where-felhom-stands.yaml</code></b>, which cites the capability map, register row or evidence document behind every claim — and which is checked by <code>scripts/check_stands.py</code>.</div>
-<div style="color:#6b7a91;font-size:11.5px;margin-bottom:16px">Verified 2026-08-09 against felhom-agent <code>28ba8593b8</code>, felhom-controller <code>c732fe1283</code>, hub <code>56f8aa611c</code>. <b style="color:#fbbf24">12 status(es) moved in that pass</b> — each is marked on the page.</div>
-<div style="display:flex;gap:22px;flex-wrap:wrap;background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:11px 14px;margin-bottom:18px"><div style="display:flex;gap:8px;align-items:flex-start"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:11.5px"><b>Walked</b> <span style="color:#7c8aa3">(23)</span><span style="display:block;color:#6b7a91;font-size:10.5px">done end to end on real hardware, evidence on file</span></span></div><div style="display:flex;gap:8px;align-items:flex-start"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:11.5px"><b>Built</b> <span style="color:#7c8aa3">(14)</span><span style="display:block;color:#6b7a91;font-size:10.5px">shipped and tested, the real path never walked</span></span></div><div style="display:flex;gap:8px;align-items:flex-start"><span style="width:8px;height:8px;border-radius:50%;background:#fbbf24;flex:0 0 8px;margin-top:5px"></span><span style="font-size:11.5px"><b>Partial</b> <span style="color:#7c8aa3">(14)</span><span style="display:block;color:#6b7a91;font-size:10.5px">some walked, some not — the note says which</span></span></div><div style="display:flex;gap:8px;align-items:flex-start"><span style="width:8px;height:8px;border-radius:50%;background:#64748b;flex:0 0 8px;margin-top:5px"></span><span style="font-size:11.5px"><b>Missing</b> <span style="color:#7c8aa3">(4)</span><span style="display:block;color:#6b7a91;font-size:10.5px">does not exist</span></span></div><div style="display:flex;gap:8px;align-items:flex-start"><span style="font-size:11.5px"><b style="color:#7dd3fc">needs-hardware</b><span style="display:block;color:#6b7a91;font-size:10.5px">only a running box could settle it</span></span></div></div>
+<div style="color:#6b7a91;font-size:11.5px;margin-bottom:16px">Verified 2026-08-22 against felhom-agent <code>40d857b527</code>, felhom-controller <code>2da259af38</code>, hub <code>877fcd2a38</code>. <b style="color:#fbbf24">15 claim(s) carry a recorded status move</b> (8 re-checked in this pass) — each is marked on the page.</div>
+<div style="display:flex;gap:22px;flex-wrap:wrap;background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:11px 14px;margin-bottom:18px"><div style="display:flex;gap:8px;align-items:flex-start"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:11.5px"><b>Walked</b> <span style="color:#7c8aa3">(20)</span><span style="display:block;color:#6b7a91;font-size:10.5px">done end to end on real hardware, evidence on file</span></span></div><div style="display:flex;gap:8px;align-items:flex-start"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:11.5px"><b>Built</b> <span style="color:#7c8aa3">(14)</span><span style="display:block;color:#6b7a91;font-size:10.5px">shipped and tested, the real path never walked</span></span></div><div style="display:flex;gap:8px;align-items:flex-start"><span style="width:8px;height:8px;border-radius:50%;background:#fbbf24;flex:0 0 8px;margin-top:5px"></span><span style="font-size:11.5px"><b>Partial</b> <span style="color:#7c8aa3">(17)</span><span style="display:block;color:#6b7a91;font-size:10.5px">some walked, some not — the note says which</span></span></div><div style="display:flex;gap:8px;align-items:flex-start"><span style="width:8px;height:8px;border-radius:50%;background:#64748b;flex:0 0 8px;margin-top:5px"></span><span style="font-size:11.5px"><b>Missing</b> <span style="color:#7c8aa3">(4)</span><span style="display:block;color:#6b7a91;font-size:10.5px">does not exist</span></span></div><div style="display:flex;gap:8px;align-items:flex-start"><span style="font-size:11.5px"><b style="color:#7dd3fc">needs-hardware</b><span style="display:block;color:#6b7a91;font-size:10.5px">only a running box could settle it</span></span></div></div>
 <div style="font-size:15px;font-weight:600;margin:0 0 9px">The journey</div>
 <div style="color:#6b7a91;font-size:11.5px;margin-bottom:10px">Seven stages, left to right, as they happen to a person.</div>
 <div class="grid7">
@@ -15,9 +15,9 @@
 <div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44"><span style="color:#4b5b76;font-family:ui-monospace,monospace">2</span> &nbsp;Making it theirs</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">An emailed one-time code; the customer sets their own password and the operator never sees it<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Exercised live 2026-08-09: reset code accepted at /claim, 302 + session.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Customer claim: one-time emailed code → customer sets own p… · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">The customer can bind their own machine from a link, with no operator present — done once, for real<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">attempts=0, locked=0, source customer_selfbind, 2026-07-18.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Customer binds their own appliance (self-service) · evidence tests/VALIDATION-n100-rehearsal-2026-07-18.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#fbbf24;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">It has never been done by a person who is not the operator<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Map records MISSING (as evidence). Still true after 2026-08-09.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map A customer (not the operator) performs a restore via UI alone</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#fbbf24;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">The same word is used for two different secrets across three surfaces; the email points at a page a rebuilt machine does not show<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Warning card. Cost a real code on 2026-08-09.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">register R-282 · register R-283</span></span></div></div>
 <div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44"><span style="color:#4b5b76;font-family:ui-monospace,monospace">3</span> &nbsp;Using it</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Apps installed from a catalog of about 52, with a memory guard and health-aware progress<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">53 templates listed on the rebuilt box 2026-08-09; deploy driven live.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Deploy an app from the catalog (env config, memory guard, h… · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Start, stop, restart, update, logs, remove — and the parts that must not be stopped cannot be<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#3a2a12;color:#fbbf24;font-size:10px;white-space:nowrap">changed 2026-08-09, was walked</span><span style="display:block;color:#9a7b3a;font-size:11px;margin-top:2px">why it moved: no walk document cited by the map row or anywhere else</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map App lifecycle: start/stop/restart/update/logs/remove/redeploy · register R-108</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Reachable from anywhere through a tunnel, per-app addresses<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Verified tonight: the rebuilt box answered on its public URL from outside.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Remote access via Cloudflare Tunnel + Traefik (per-app subd… · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Reachable on the home network when the internet is down<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#1b2740;color:#7dd3fc;font-size:10px;white-space:nowrap">needs-hardware</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Only an observation on a running box with WAN pulled could settle it. Boxes are off.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map LAN access when internet is down (lan_resolver)</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Phone photos, documents with text recognition, files from Windows Explorer or a Mac<span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Files from Windows Explorer / Mac Finder (SMB server) · evidence audits/SPIKE-lan-discovery-2026-07-18.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">A one-tap launcher, and a read-only guest link for visitors<span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Indítópult (app launcher) — one-tap grid of the household's…</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#64748b;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-heigLine truncated
 <div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44"><span style="color:#4b5b76;font-family:ui-monospace,monospace">4</span> &nbsp;Drives</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">A new drive is found, offered, formatted, mounted and enrolled — including on awkward older boot layouts<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#3a2a12;color:#fbbf24;font-size:10px;white-space:nowrap">moved twice: 2026-08-09 walked -&gt; built, then 2026-08-10 back to walked</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Applies to a NEW drive. Re-attaching an existing one after a reinstall is R-280 and fails.</span><span style="display:block;color:#9a7b3a;font-size:11px;margin-top:2px">why it moved: receipt found 2026-08-10: a live throwaway drive on felhom-pve walked scan -&gt; format -&gt; mount -&gt; PVE dir storage -&gt; one-click Regisztralas, with the resulting storage.cfg entry and systemd mount unit recorded. The map already read PROVEN-LIVE; the dataset was behind it. Scope unchanged: this is a NEW drive. Re-attaching an existing one after a reinstall is R-280 and still fails.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Drive wizard: scan/format/mount/enroll, incl. legacy-boot L… · register R-220 · evidence audits/SPIKE-raw-drive-enroll-2026-06-15.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Moving data between drives, crash-safe; removing a drive safely; unplug detected<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#3a2a12;color:#fbbf24;font-size:10px;white-space:nowrap">changed 2026-08-09, was walked</span><span style="display:block;color:#9a7b3a;font-size:11px;margin-top:2px">why it moved: no walk document cited</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Data migration between drives (all / per-app), crash-safe</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">A network drive can be browsed and hold bulk media, but may not hold an app's data — enforced<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">RefuseAsAppNamespace is the fail-closed predicate.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Network storage (NAS) is browse + bulk-media only · register R-108 · evidence audits/R108-network-app-namespace-2026-07-30.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#fbbf24;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">After a reinstall the data drive cannot be re-attached through any dashboard route<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Warning card. /api/disks/candidates returns empty; the restore page promises two clicks.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">register R-280 · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md</span></span></div></div>
-<div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44"><span style="color:#4b5b76;font-family:ui-monospace,monospace">5</span> &nbsp;Backing up</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">App data on the machine, nightly database dumps, a copy on a second drive<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#3a2a12;color:#fbbf24;font-size:10px;white-space:nowrap">moved twice: 2026-08-09 walked -&gt; built, then 2026-08-10 back to walked</span><span style="display:block;color:#9a7b3a;font-size:11px;margin-top:2px">why it moved: receipt found 2026-08-10: an adversarial, destructive, unattended overnight campaign across both boxes and ep0, with A2 (one quiesce, two tiers) proven end to end. Map already read PROVEN-LIVE.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Tier-2 secondary-drive copy: class-driven legs · evidence audits/CAMPAIGN-8-backup-restore-2026-07-27.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">A whole-machine archive that lands off the guest's own disk — a single-drive machine is recorded as degraded rather than pretending<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#3a2a12;color:#fbbf24;font-size:10px;white-space:nowrap">changed 2026-08-09, was walked</span><span style="display:block;color:#9a7b3a;font-size:11px;margin-top:2px">why it moved: no walk document cited</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Whole-guest backup lands OFF the guest's own physical device · register R-165</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">An encrypted off-site copy, sealed with a key the operator cannot read<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Verified tonight from the repo itself: 18 snapshots, daily, unbroken.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Offsite (restic → Hetzner Storage Box) · register R-199 · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">The backups prove themselves: a restore is actually performed, unattended, on every tier, on both machines<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#3a2a12;color:#fbbf24;font-size:10px;white-space:nowrap">changed 2026-08-09, was walked</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">'on both machines, unattended, every tier' is a continuing claim about scheduled runs. Both boxes are off; the last recorded restore-test on demo-hp FAILED (notification_log 2026-08-05 restore_test_failed). Cannot be settled tonight.</span><span style="display:block;color:#c08a3e;font-size:11px;margin-top:2px">proof decay: PROOF-DECAY RULE FIRED (first time it has). A receipt EXISTS - architecture/_recovery-inventory-2026-07-28.md carries live journal lines for scheduled restore-tests on both boxes and both tiers - but it is superseded by later observation: demo-hp logged restore_test_failed on 2026-08-05, and the box has since been wiped and reinstalled (2026-08-09). The claim is about a CONTINUING scheduled behaviour, so a 2026-07-28 observation cannot carry it. Stays grey until a scheduled restore-test is seen passing on the rebuilt box. THE CAPABILITY MAP STILL READS PROVEN-LIVE (2026-08-03) AND IS NOW THE THING OUT OF STEP.</span><span style="display:block;color:#9a7b3a;font-size:11px;margin-top:2px">why it moved: no walk document cited, and the last recorded restore-test on demo-hp FAILED (2026-08-05)</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">caLine truncated
+<div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44"><span style="color:#4b5b76;font-family:ui-monospace,monospace">5</span> &nbsp;Backing up</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">App data on the machine, nightly database dumps, a copy on a second drive<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#3a2a12;color:#fbbf24;font-size:10px;white-space:nowrap">moved twice: 2026-08-09 walked -&gt; built, then 2026-08-10 back to walked</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">True for every app but one, and that one was found on 2026-08-21: paperless-ngx's 72-table PostgreSQL was dumped nightly and written into a folder named after an app that does not exist, so it never entered the recovery unit, the second-drive copy or the off-site copy. Fixed and proven live 2026-08-22 (R-355) — the dump is now in the app's own unit and in the off-site snapshot for the first time. A catalogue-wide sweep, itself proven able to convict a planted case, says 1 of 53 was affected.</span><span style="display:block;color:#9a7b3a;font-size:11px;margin-top:2px">why it moved: receipt found 2026-08-10: an adversarial, destructive, unattended overnight campaign across both boxes and ep0, with A2 (one quiesce, two tiers) proven end to end. Map already read PROVEN-LIVE.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Tier-2 secondary-drive copy: class-driven legs · register R-355 · evidence audits/CAMPAIGN-8-backup-restore-2026-07-27.md · evidence audits/DRILL-backup-truth-2026-08-21/evidence/phase4-paperl…</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">A whole-machine archive that lands off the guest's own disk — a single-drive machine is recorded as degraded rather than pretending<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#3a2a12;color:#fbbf24;font-size:10px;white-space:nowrap">changed 2026-08-09, was walked</span><span style="display:block;color:#9a7b3a;font-size:11px;margin-top:2px">why it moved: no walk document cited</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Whole-guest backup lands OFF the guest's own physical device · register R-165</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#fbbf24;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">An encrypted off-site copy, sealed with a key the operator cannot read<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#3a2a12;color:#fbbf24;font-size:10px;white-space:nowrap">changed 2026-08-09, was walked</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">The COPY is real and the key is still unreadable to us — what failed is DAILY and UNBROKEN. The 2026-08-09 note said '18 snapshots, daily, unbroken'; the next snapshot after 2026-08-09 08:30 was 2026-08-21 22:17, put there by hand during the drill. Twelve days, no alarm. Two causes in series: the 2026-08-21 rebuild lost the off-box target (R-193's shape), and after the self-heal restored it EVERY per-app switch was still off, so the first run logged 'backup OK: 0 app(s) backed up, 14s'.</span><span style="display:block;color:#9a7b3a;font-size:11px;margin-top:2px">why it moved: the claim is about a CONTINUING daily copy; a 12-day silent gap was found on 2026-08-21 and nothing reported it</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Offsite (restic → Hetzner Storage Box) · register R-199 · register R-193 · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md · evidence audits/REPORT-DRILL-backup-truth-2026-08-21.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">The backups prove themselves: a restore is actually performed, uLine truncated
 <div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44"><span style="color:#4b5b76;font-family:ui-monospace,monospace">6</span> &nbsp;When something goes wrong</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">The machine watches itself and repairs some faults without telling anyone it had to<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#3a2a12;color:#fbbf24;font-size:10px;white-space:nowrap">changed 2026-08-09, was walked</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">R-264 records that the self-heal counters reach the hub and are decoded nowhere.</span><span style="display:block;color:#9a7b3a;font-size:11px;margin-top:2px">why it moved: no walk document cited</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map Box survives an unattended app or guest-network failure · register R-264</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Failures reach the operator by email, one mail per run, every failing app named<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#3a2a12;color:#fbbf24;font-size:10px;white-space:nowrap">changed 2026-08-09, was walked</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">CONFIRMED against source: backup_run_failures digest is allowlisted (api/handler.go:1837), operator-only (dispatcher.go:423), templated (templates.go:48); recovery_unit_capture_failed is record-only (dispatcher.go:376). NOTE: R-182's register row still describes the PRE-FIX behaviour — see R-289.</span><span style="display:block;color:#9a7b3a;font-size:11px;margin-top:2px">why it moved: the digest is wired and source-verified, but no run of it has been observed delivering</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map A failed per-app Tier-1 backup reaches the OPERATOR — EVERY… · register R-182</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Failures reach the customer<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">The customer leg is built; today's log shows customer-channel rows skipped as operator_only.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map App crashes → customer notified (one event per transition, …</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#fbbf24;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">An already-paired box is still told to pair itself<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Warning card.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">register R-214 · register R-235</span></span></div></div>
-<div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44"><span style="color:#4b5b76;font-family:ui-monospace,monospace">7</span> &nbsp;Getting everything back</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">A rebuilt machine shows a full-page recovery screen without anyone looking for it, and says plainly that nobody can replace a lost recovery code<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Seen unsought on the rebuilt demo-hp 2026-08-09, seal date matching host_escrow.created_at.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">register R-193 · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">The customer's code opens the sealed package and the data returns byte for byte — including accented Hungarian filenames, verified as raw bytes<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Reproduced 2026-08-09: 4/4 byte-identical, name bytes NFC-preserved, out of snapshot 41c830db.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">register R-201 · evidence tests/walk5-r201-2026-08-07/journal.md · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Walked end to end with no command line inside the machine (2026-08-07)<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#1b2740;color:#a78bfa;font-size:10px;white-space:nowrap">contested</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">CONTESTED-RESOLVED: not a contradiction. The walk proves the ROUTE needs no guest shell; the map's MISSING row is about a NON-OPERATOR doing it, which has still never happened. Two questions, one word 'customer'.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">register R-201 · evidence tests/walk5-r201-2026-08-07/journal.md · capability-map A customer (not the operator) performs a restore via UI alone</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#fbbf24;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Putting restored files back where they belong is still manual<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Warning card. Confirmed 2026-08-09: the restore lands in a verification folder and says so.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">register R-213</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">The tripwire that says someone is opening this customer's backups does fire<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">CONFIRMED tonight from the hub store: escrow_blob_served 2026-08-09 10:19:41Z = 12:19 CEST in the operator's mailbox. R-281's original claim of silence is WITHDRAWN.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">register R-281 · register R-285 · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md</span></span></div></div>
+<div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44"><span style="color:#4b5b76;font-family:ui-monospace,monospace">7</span> &nbsp;Getting everything back</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">A rebuilt machine shows a full-page recovery screen without anyone looking for it, and says plainly that nobody can replace a lost recovery code<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Seen unsought on the rebuilt demo-hp 2026-08-09, seal date matching host_escrow.created_at.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">register R-193 · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">The customer's code opens the sealed package and the data returns byte for byte — including accented Hungarian filenames, verified as raw bytes<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Still true, and re-proven 2026-08-22 (5/5 byte-identical, both accented names as raw bytes) — but the SCOPE is narrower than the sentence sounds and was silently narrower still until v0.218.0. The drill found the off-site restore had NO named-volume leg at all: the archive sat in the unit, the snapshot and the checking folder and was never replayed, under a success message (R-354, fixed and proven 2026-08-22). AND 40 of the 53 catalogue apps STILL cannot run this route at all — it refuses first, saying a running app is not installed (R-356, open). So: proven for an app that declares a data drive; unproven and currently unreachable for the class whose entire dataset is a named volume.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">register R-201 · register R-354 · register R-356 · evidence tests/walk5-r201-2026-08-07/journal.md · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md · evidence audits/REPORT-DRILL-backup-truth-2026-08-21.md</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Walked end to end with no command line inside the machine (2026-08-07)<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#1b2740;color:#a78bfa;font-size:10px;white-space:nowrap">contested</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">CONTESTED-RESOLVED: not a contradiction. The walk proves the ROUTE needs no guest shell; the map's MISSING row is about a NON-OPERATOR doing it, which has still never happened. Two questions, one word 'customer'.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">register R-201 · evidence tests/walk5-r201-2026-08-07/journal.md · capability-map A customer (not the operator) performs a restore via UI alone</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#fbbf24;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Putting restored files back where they belong is still manual<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Warning card. Confirmed 2026-08-09: the restore lands in a verification folder and says so.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">register R-213</span></span></div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">The tripwire that says someone is opening this customer's backups does fire<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">CONFIRMED tonight from the hub store: escrow_blob_served 2026-08-09 10:19:41Z = 12:19 CEST in the operator's mailbox. R-281's original claim of silence is WITHDRAWN.</span><span style="display:block;color:#4b5bLine truncated
 </div>
 <div style="font-size:15px;font-weight:600;margin:22px 0 9px">When things go wrong</div>
 <div style="color:#6b7a91;font-size:11.5px;margin-bottom:10px">19 situations, and what actually happens in each.</div>
@@ -28,7 +28,7 @@
 <div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44">An app crashes</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#60a5fa;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">An app crashes — the email leg has never been confirmed end to end<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Consistent with fault.operator-email: the digest is wired but its delivery is unobserved.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map App crashes → customer notified (one event per transition, …</span></span></div></div>
 <div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44">Power cut mid-backup</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">Power cut mid-backup<span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">evidence audits/AUDIT-power-outage-recovery-2026-07-22.md</span></span></div></div>
 <div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44">The guest is destroyed</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">The guest is destroyed<span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">register R-201 · evidence audits/DRILL-r201-night-run-2026-08-04.md · evidence audits/DRILL-r201-night-run-2026-08-04.md</span></span></div></div>
-<div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44">The whole machine is wiped and reinstalled</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">The whole machine is wiped and reinstalled — the data comes back<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">4/4 byte-identical 2026-08-09.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md</span></span></div></div>
+<div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44">The whole machine is wiped and reinstalled</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#fbbf24;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">The whole machine is wiped and reinstalled — the data comes back<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#3a2a12;color:#fbbf24;font-size:10px;white-space:nowrap">changed 2026-08-09, was walked</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">The 2026-08-09 rehearsal really did return 4/4 byte-identical, and that stands. What the next real reinstall showed (2026-08-21, demo-hp) is that the rebuild ORPHANS BOTH OFF-PREMISES TIERS at once, quietly: the restic target was lost and needed a self-heal plus a per-app re-enable before any copy resumed (R-193), and the PBS archives from before the reinstall cannot be opened by the rebuilt box at all, because it now presents a different key (R-366). The data came back in the rehearsal because the rehearsal restored it immediately; a machine left alone after a reinstall is not protected in the meantime and nothing says so.</span><span style="display:block;color:#9a7b3a;font-size:11px;margin-top:2px">why it moved: a real reinstall on 2026-08-21 left both off-premises tiers broken - one silently for 12 days, the other unreadable - so 'the data comes back' holds only if someone restores it at once</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">register R-193 · register R-366 · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md · evidence audits/REPORT-DRILL-backup-truth-2026-08-21.md</span></span></div></div>
 <div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44">The whole machine is wiped and reinstalled</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#fbbf24;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">The whole machine is wiped and reinstalled — the journey needs a terminal twice<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">CONTESTED-RESOLVED against the map: the map's PROVEN-LIVE row is scoped to a controller-data-volume REBUILD (2026-08-04), not a whole-host reinstall. The map has no row for the host case, so there was no contradiction — only a gap.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">register R-273 · register R-280 · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md</span></span></div></div>
 <div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44">The machine is stolen, and someone opens the backups</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">The machine is stolen, and someone opens the backups — the operator is told<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Confirmed 2026-08-09 from the store and the mailbox.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">register R-281 · evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md</span></span></div></div>
 <div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44">The customer forgets their dashboard password</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#34d399;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">The customer forgets their dashboard password<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Exercised 2026-08-09 via the reset code.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">evidence audits/REHEARSAL-byo-reinstall-2026-08-09.md</span></span></div></div>
@@ -40,7 +40,7 @@
 <div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44">A fresh install picks up an old image</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#fbbf24;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">A fresh install picks up an old image<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Narrowed 2026-08-09: the resume path fetched the vouched golden; a fresh install still takes the newest LOCAL archive with no manifest comparison.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">register R-274</span></span></div></div>
 <div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44">The off-site provider account is deleted</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#fbbf24;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">The off-site provider account is deleted<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">The credential that holds the customer's documents can still delete.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">register R-95</span></span></div></div>
 <div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44">The machine is switched off for an afternoon</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#64748b;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">The machine is switched off for an afternoon — no notion of expected downtime<span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">Confirmed 2026-08-09: eight operator mails for deliberate, attended work.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">register R-285</span></span></div></div>
-<div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44">A customer restores their own data with no help</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#fbbf24;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">A customer restores their own data with no help<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#1b2740;color:#a78bfa;font-size:10px;white-space:nowrap">contested</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">CONTESTED-RESOLVED: the map's MISSING is about a NON-OPERATOR performing it; the walks prove the route, not the person. Neither record was wrong.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map A customer (not the operator) performs a restore via UI alone · register R-201</span></span></div></div>
+<div style="background:#121b2c;border:1px solid #1f2c44;border-radius:6px;padding:14px 13px"><div style="font-size:14px;font-weight:500;padding-bottom:9px;margin-bottom:10px;border-bottom:1px solid #1f2c44">A customer restores their own data with no help</div><div style="display:flex;gap:8px;align-items:flex-start;margin-bottom:9px"><span style="width:8px;height:8px;border-radius:50%;background:#fbbf24;flex:0 0 8px;margin-top:5px"></span><span style="font-size:12px;line-height:1.45;color:#b8c4d6;text-wrap:pretty">A customer restores their own data with no help<span style="display:inline-block;margin-left:6px;padding:1px 5px;border-radius:3px;background:#1b2740;color:#a78bfa;font-size:10px;white-space:nowrap">contested</span><span style="display:block;color:#7c8aa3;font-size:11px;margin-top:3px">CONTESTED-RESOLVED: the map's MISSING is about a NON-OPERATOR performing it; the walks prove the route, not the person. Neither record was wrong. SINCE 2026-08-21 there is a second, harder blocker and it is not about the person at all: for the 40 of 53 apps that declare no data drive the off-site restore REFUSES before it starts, telling the customer a running app 'nincs telepítve' and to reinstall it to the same place — which those apps give them no way to choose (R-356). Those are exactly the apps whose whole dataset is a named volume. Until that is fixed, most customers cannot self-restore off-site even in principle.</span><span style="display:block;color:#4b5b76;font-size:10px;margin-top:3px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace">capability-map A customer (not the operator) performs a restore via UI alone · register R-201 · register R-356 · evidence audits/REPORT-DRILL-backup-truth-2026-08-21.md</span></span></div></div>
 </div>
 <div style="margin-top:26px;padding-top:12px;border-top:1px solid #1f2c44;color:#4b5b76;font-size:10.5px;line-height:1.6">Generated by <code>scripts/render_stands.py</code> from <code>where-felhom-stands.yaml</code>. Do not hand-edit this file — edit the data and regenerate. The 2026-08-09 React bundle is kept as <code>where-felhom-stands-2026-08-09-snapshot.html</code> and is not maintained.</div>
 </div></body></html>

documentation/architecture/where-felhom-stands.yaml

@@ -12,11 +12,11 @@
 # verdict vocabulary: confirmed | downgraded | upgraded | contested | needs-hardware
 # depth: source-read (opened live source or evidence) | register+map (checked against the
 #        register and capability map only) | needs-hardware (cannot be settled off-box)
-verified_on: 2026-08-09
+verified_on: 2026-08-22
 verified_against:
-  felhom-agent: 28ba8593b8
-  felhom-controller: c732fe1283
-  hub: 56f8aa611c
+  felhom-agent: 40d857b527
+  felhom-controller: 2da259af38
+  hub: 877fcd2a38
 claims:
   - id: install.iso-selfregister
     band: journey
@@ -301,17 +301,20 @@ claims:
     stage: 5
     title: "App data on the machine, nightly database dumps, a copy on a second drive"
     status: walked
+    note: "True for every app but one, and that one was found on 2026-08-21: paperless-ngx's 72-table PostgreSQL was dumped nightly and written into a folder named after an app that does not exist, so it never entered the recovery unit, the second-drive copy or the off-site copy. Fixed and proven live 2026-08-22 (R-355) — the dump is now in the app's own unit and in the off-site snapshot for the first time. A catalogue-wide sweep, itself proven able to convict a planted case, says 1 of 53 was affected."
     sources:
       - capability-map: "Tier-2 secondary-drive copy: class-driven legs"
+      - register: "R-355"
       - evidence: "audits/CAMPAIGN-8-backup-restore-2026-07-27.md"
+      - evidence: "audits/DRILL-backup-truth-2026-08-21/evidence/phase4-paperless/FINDING.txt"
     changed:
       from: built
       also_moved: 2026-08-09 walked -> built
       reason_superseded: "no walk document cited"
       reason: "receipt found 2026-08-10: an adversarial, destructive, unattended overnight campaign across both boxes and ep0, with A2 (one quiesce, two tiers) proven end to end. Map already read PROVEN-LIVE."
     verified:
-      date: 2026-08-09
-      verdict: upgraded
+      date: 2026-08-22
+      verdict: confirmed
       depth: source-read
   - id: backup.whole-machine
     band: journey
@@ -332,59 +335,75 @@ claims:
     band: journey
     stage: 5
     title: "An encrypted off-site copy, sealed with a key the operator cannot read"
-    status: walked
-    note: "Verified tonight from the repo itself: 18 snapshots, daily, unbroken."
+    status: partial
+    note: "The COPY is real and the key is still unreadable to us — what failed is DAILY and UNBROKEN. The 2026-08-09 note said '18 snapshots, daily, unbroken'; the next snapshot after 2026-08-09 08:30 was 2026-08-21 22:17, put there by hand during the drill. Twelve days, no alarm. Two causes in series: the 2026-08-21 rebuild lost the off-box target (R-193's shape), and after the self-heal restored it EVERY per-app switch was still off, so the first run logged 'backup OK: 0 app(s) backed up, 14s'."
     sources:
       - capability-map: "Offsite (restic → Hetzner Storage Box)"
       - register: "R-199"
+      - register: "R-193"
       - evidence: "audits/REHEARSAL-byo-reinstall-2026-08-09.md"
+      - evidence: "audits/REPORT-DRILL-backup-truth-2026-08-21.md"
+    changed:
+      from: walked
+      reason: "the claim is about a CONTINUING daily copy; a 12-day silent gap was found on 2026-08-21 and nothing reported it"
     verified:
-      date: 2026-08-09
-      verdict: confirmed
+      date: 2026-08-22
+      verdict: downgraded
       depth: source-read
   - id: backup.restore-proof
     band: journey
     stage: 5
     title: "The backups prove themselves: a restore is actually performed, unattended, on every tier, on both machines"
     status: built
-    note: "'on both machines, unattended, every tier' is a continuing claim about scheduled runs. Both boxes are off; the last recorded restore-test on demo-hp FAILED (notification_log 2026-08-05 restore_test_failed). Cannot be settled tonight."
+    note: "STILL GREY, and for a sharper reason than in August. The scheduler runs and the check works — it failed again on 2026-08-21, unprompted, and named the cause exactly: after the reinstall the box presents a different PBS key than its own older archives were sealed with, so those archives cannot be opened at all (R-366). A tier whose archives are orphaned is a different alarm from a tier whose test failed, and only the second is being said."
     sources:
       - capability-map: "Restore-proof is UNATTENDED — the scheduler covers EVERY tier"
       - register: "R-86"
+      - register: "R-366"
+      - evidence: "audits/REPORT-DRILL-backup-truth-2026-08-21.md"
     changed:
       from: walked
       reason: "no walk document cited, and the last recorded restore-test on demo-hp FAILED (2026-08-05)"
       decay: "PROOF-DECAY RULE FIRED (first time it has). A receipt EXISTS - architecture/_recovery-inventory-2026-07-28.md carries live journal lines for scheduled restore-tests on both boxes and both tiers - but it is superseded by later observation: demo-hp logged restore_test_failed on 2026-08-05, and the box has since been wiped and reinstalled (2026-08-09). The claim is about a CONTINUING scheduled behaviour, so a 2026-07-28 observation cannot carry it. Stays grey until a scheduled restore-test is seen passing on the rebuilt box. THE CAPABILITY MAP STILL READS PROVEN-LIVE (2026-08-03) AND IS NOW THE THING OUT OF STEP."
+      worse_2026_08_22: "It failed AGAIN, unprompted, on 2026-08-21 21:59 - and the cause is worse than 'untested'. Hub event 3016: the PBS archive of 2026-08-18 could not be restored because the manifest's key does not match the key the rebuilt box now presents. The archives that predate the 2026-08-21 reinstall are UNREADABLE to the machine that made them (R-366). Credit where due: the mechanism caught it and named the key mismatch precisely. The gap is that it is reported as 'a restore test failed' rather than 'your older whole-machine backups cannot be opened on this box'."
     verified:
-      date: 2026-08-09
+      date: 2026-08-22
       verdict: downgraded
       depth: needs-hardware
   - id: backup.fill-warning
     band: journey
     stage: 5
     title: "The customer is warned before a drive fills, per drive, in their own language"
-    status: walked
-    note: "R-177 (no operator-triggerable run) limits testing, not the capability."
+    status: partial
+    note: "The warning is real and was SEEN firing on 2026-08-21 with the right Hungarian copy, naming the drive and the free space. What 'BEFORE' cannot survive is the cadence: the watcher runs once a day at 03:30 plus once at startup (R-363), so a filesystem that fills at 03:31 goes unannounced for ~24 h. Watched live: the 69 GB volume carrying all 40-class app data was filled to 99% and the watcher said nothing, while the backup reserve was already refusing an app per run and telling the hub about it."
     sources:
       - capability-map: "The customer is warned BEFORE a filesystem fills"
       - register: "R-167"
+      - register: "R-363"
       - evidence: "audits/SPIKE-r165-mp1-merge-2026-08-02.md"
+      - evidence: "audits/REPORT-DRILL-backup-truth-2026-08-21.md"
+    changed:
+      from: walked
+      reason: "a daily check cannot carry the word BEFORE; observed silent for the whole window a filesystem sat at 99%"
     verified:
-      date: 2026-08-09
-      verdict: confirmed
-      depth: register+map
+      date: 2026-08-22
+      verdict: downgraded
+      depth: source-read
   - id: backup.sikeres
     band: journey
     stage: 5
     title: "A backup that covered nothing still calls itself successful"
     status: partial
-    note: "Warning card."
+    note: "Warning card — and the drill found two more of it, both live. (1) An off-site run with no app selected logs 'backup OK: 0 app(s) backed up'; the card does say 'nincs kijelölt alkalmazás' beside the green tick, so this one is honest if you read past the tick. (2) A restore that placed nothing reported success: '0 fájl visszaállítva', ok=true. The second is the one that matters and it is R-353, still open. The volume half of it is fixed (R-354): the message now names what came back."
     sources:
       - register: "R-240"
+      - register: "R-353"
+      - register: "R-354"
+      - evidence: "audits/DRILL-backup-truth-2026-08-21/evidence/phase3-experiment/messages-verbatim.txt"
     verified:
-      date: 2026-08-09
+      date: 2026-08-22
       verdict: confirmed
-      depth: register+map
+      depth: source-read
   - id: fault.selfheal
     band: journey
     stage: 6
@@ -460,13 +479,16 @@ claims:
     stage: 7
     title: "The customer's code opens the sealed package and the data returns byte for byte — including accented Hungarian filenames, verified as raw bytes"
     status: walked
-    note: "Reproduced 2026-08-09: 4/4 byte-identical, name bytes NFC-preserved, out of snapshot 41c830db."
+    note: "Still true, and re-proven 2026-08-22 (5/5 byte-identical, both accented names as raw bytes) — but the SCOPE is narrower than the sentence sounds and was silently narrower still until v0.218.0. The drill found the off-site restore had NO named-volume leg at all: the archive sat in the unit, the snapshot and the checking folder and was never replayed, under a success message (R-354, fixed and proven 2026-08-22). AND 40 of the 53 catalogue apps STILL cannot run this route at all — it refuses first, saying a running app is not installed (R-356, open). So: proven for an app that declares a data drive; unproven and currently unreachable for the class whose entire dataset is a named volume."
     sources:
       - register: "R-201"
+      - register: "R-354"
+      - register: "R-356"
       - evidence: "tests/walk5-r201-2026-08-07/journal.md"
       - evidence: "audits/REHEARSAL-byo-reinstall-2026-08-09.md"
+      - evidence: "audits/REPORT-DRILL-backup-truth-2026-08-21.md"
     verified:
-      date: 2026-08-09
+      date: 2026-08-22
       verdict: confirmed
       depth: source-read
   - id: recover.no-shell
@@ -581,13 +603,19 @@ claims:
   - id: fail.wiped-reinstalled.data
     band: failures
     title: "The whole machine is wiped and reinstalled — the data comes back"
-    status: walked
-    note: "4/4 byte-identical 2026-08-09."
+    status: partial
+    note: "The 2026-08-09 rehearsal really did return 4/4 byte-identical, and that stands. What the next real reinstall showed (2026-08-21, demo-hp) is that the rebuild ORPHANS BOTH OFF-PREMISES TIERS at once, quietly: the restic target was lost and needed a self-heal plus a per-app re-enable before any copy resumed (R-193), and the PBS archives from before the reinstall cannot be opened by the rebuilt box at all, because it now presents a different key (R-366). The data came back in the rehearsal because the rehearsal restored it immediately; a machine left alone after a reinstall is not protected in the meantime and nothing says so."
     sources:
+      - register: "R-193"
+      - register: "R-366"
       - evidence: "audits/REHEARSAL-byo-reinstall-2026-08-09.md"
+      - evidence: "audits/REPORT-DRILL-backup-truth-2026-08-21.md"
+    changed:
+      from: walked
+      reason: "a real reinstall on 2026-08-21 left both off-premises tiers broken - one silently for 12 days, the other unreadable - so 'the data comes back' holds only if someone restores it at once"
     verified:
-      date: 2026-08-09
-      verdict: confirmed
+      date: 2026-08-22
+      verdict: downgraded
       depth: source-read
   - id: fail.wiped-reinstalled.journey
     band: failures
@@ -727,11 +755,13 @@ claims:
     band: failures
     title: "A customer restores their own data with no help"
     status: partial
-    note: "CONTESTED-RESOLVED: the map's MISSING is about a NON-OPERATOR performing it; the walks prove the route, not the person. Neither record was wrong."
+    note: "CONTESTED-RESOLVED: the map's MISSING is about a NON-OPERATOR performing it; the walks prove the route, not the person. Neither record was wrong. SINCE 2026-08-21 there is a second, harder blocker and it is not about the person at all: for the 40 of 53 apps that declare no data drive the off-site restore REFUSES before it starts, telling the customer a running app 'nincs telepítve' and to reinstall it to the same place — which those apps give them no way to choose (R-356). Those are exactly the apps whose whole dataset is a named volume. Until that is fixed, most customers cannot self-restore off-site even in principle."
     sources:
       - capability-map: "A customer (not the operator) performs a restore via UI alone"
       - register: "R-201"
+      - register: "R-356"
+      - evidence: "audits/REPORT-DRILL-backup-truth-2026-08-21.md"
     verified:
-      date: 2026-08-09
+      date: 2026-08-22
       verdict: contested
       depth: source-read

documentation/audits/DIAG-smart-passed-trap-2026-08-14.md

@@ -0,0 +1,196 @@
+# DIAG — the SMART `PASSED` trap, and the disk alert that reached nobody
+
+**Date:** 2026-08-14
+**Drive:** Seagate `ST3000VX010-2E3166`, S/N `Z6A07P2G`, 3.0 TB, `/dev/sdg` on **DooPlex**
+**Fixtures:** `fixtures/smart-ST3000VX010-failing-2026-08-14.json` (raw `smartctl -a -j /dev/sdg`),
+`fixtures/smartd-history-sdg-2026-08-14.txt` (406 `smartd` journal lines for this device, 11–14 Aug)
+**Status:** the three controller defects named here are FIXED in controller **v0.215.0**; the
+counterfactual in §5 is derived from source, **not** reproduced live.
+
+This is the project's first genuinely failing disk. Before it, the capability map recorded the
+disk-failure scenario as *"Healthy path only — a genuinely failing disk has never been seen."*
+
+---
+
+## 1. What happened
+
+On **11 August 12:28** a 3 TB drive in DooPlex reported its first unreadable sectors. By **13 August
+21:58** it was at 360, and it was taking down a running service. Throughout the entire episode the
+drive's own overall self-assessment read **`PASSED`**, and it still does.
+
+`smartd` was running the whole time and mailed **local root** — a mailbox nobody reads. The failure
+was actually found by a crashlooping pod, not by any alert.
+
+---
+
+## 2. The mechanism — `smart_status.passed` cannot fail on unreadable sectors
+
+Measured, from the committed fixture:
+
+| ID  | Attribute                | value | worst | thresh | raw    |
+|-----|--------------------------|-------|-------|--------|--------|
+| 5   | `Reallocated_Sector_Ct`  | 100   | 100   | **10** | 0      |
+| 187 | `Reported_Uncorrect`     | **1** | **1** | **0**  | **1001** |
+| 188 | `Command_Timeout`        | 100   | 100   | **0**  | 0      |
+| 197 | `Current_Pending_Sector` | 98    | 98    | **0**  | **352** |
+| 198 | `Offline_Uncorrectable`  | 98    | 98    | **0**  | **352** |
+| 199 | `UDMA_CRC_Error_Count`   | 200   | 200   | **0**  | 0      |
+
+`smart_status.passed` is false only when some attribute's **normalized value** falls **at or below**
+its **threshold**. Attributes 187, 197 and 198 — the three that record unreadable sectors — all carry
+`thresh: 0`. A normalized SMART value floors at 1 and cannot reach 0.
+
+> **Therefore `smart_status.passed` is structurally incapable of failing on unreadable sectors.**
+> This is not a quirk of this drive or this vendor. It is how a zero threshold behaves.
+
+Attribute **187 `Reported_Uncorrect` sits at normalized `1` against threshold `0`** with a raw count
+of **1001**. It is one point from failing and has no remaining point to give.
+
+Corroborating, from the same fixture: `ata_smart_error_log.summary.count = 1001`, power-on hours
+**60505** (~6.9 years), `smartctl` exit status **64** (bit 6 — *the device error log contains
+records of errors*) while `smart_status.passed` is still `true`.
+
+**Any monitor built on the drive's overall verdict is blind to this entire class of failure.**
+Felhom already knows better — `agentapi.DiskVerdictFor` reads the raw counters — which is why it
+would have noticed on 11 August, two days early.
+
+---
+
+## 3. Unreadable sectors are not monotonic
+
+From `smartd-history-sdg-2026-08-14.txt`, `Current_Pending_Sector` over the episode (30-minute
+sampling, host clock = CEST):
+
+```
+Aug 11 12:28   8      first sighting
+Aug 11 12:58  16      (+8)
+Aug 11 13:28   0      FULL CLEAR — "No more Currently unreadable (pending) sectors,
+                       warning condition reset after 1 email"
+Aug 11 20:28   8      returns
+Aug 12 01:28  16  →  01:58   8  (-8)
+Aug 12 02:28  32  →  02:58  24  (-8)
+Aug 12 03:28  24      187 Reported_Uncorrect 100→97; ATA error count 0→3
+Aug 12 03:58  16  (-8)  →  04:28  24  (+8)
+Aug 12 10:58  32      … steady 32 for ~11h …
+Aug 12 21:58  24  (-8)
+Aug 13 03:28  40  (+16)  →  04:28  24  (-16)
+Aug 13 11:28  64      terminal run begins — never returns below 64
+Aug 13 11:58  72   12:58  80   13:28 112   15:58 120
+Aug 13 21:58 360      (+240)
+Aug 13 22:28 352  (-8)   … steady 352 through 14 Aug …
+```
+
+Two measured facts carry design weight:
+
+1. **The 11 August excursion cleared completely within one hour** (12:28 → 13:28). A bare `> 0`
+   alarm would have fired on a drive that then looked fine for seven hours. This is why the ladder
+   uses a *sustain* rule rather than a bare non-zero test.
+2. **The benign excursion peaked at 16; the terminal run crossed 64 at 13 Aug 11:28 and never came
+   back.** That is the entire empirical basis for the static count threshold of 64 — see §6.
+
+---
+
+## 4. The three controller defects
+
+All three are in `felhom-controller` at `3e3ee94` (v0.214.0), the tree audited here.
+
+### D1 — the alert carries a severity the hub does not recognise *(highest value)*
+
+`internal/notify/notifier.go:565` — `NotifyDiskHealthDegraded` sets:
+
+```go
+severity := "warn"
+```
+
+The hub accepts an exact-match lowercase vocabulary and **coerces anything else to `info`**:
+
+- `felhom.eu/hub/internal/api/handler.go:2121-2126` — `case "info", "warning", "error", "critical":`
+  … `default: payload.Severity = "info"`
+- `felhom.eu/hub/internal/notify/dispatcher.go:89-96` — `severityNotifies` returns true only for
+  `warning` / `error` / `critical`.
+
+`"warn"` is not in the accepted set. So the Figyelmeztetés-level disk alert is **stored as an
+informational notice and emailed to nobody**, on the customer leg and the operator leg alike.
+
+The function's own doc comment reads *"The hub applies its own per-event-type cooldown"* — which
+presumes it routes. An invariant asserted in a comment with no test pinning it; this project's
+recurring shape.
+
+### D2 — no level above "worth keeping an eye on it"
+
+`internal/agentapi/diskverdict.go:28-41` returns `Warn` for *any* non-zero counter, and can only
+reach `Fail` when `Health == "FAILING"` — which, by §2, this fault class cannot produce. A drive with
+one aging sector and a drive at 352 unreadable sectors rendered the identical chip and the identical
+mild sentence.
+
+### D3 — it speaks once, and forgets on restart
+
+`internal/web/disk_health.go:182` emits only on `v > prev`, against a **in-memory** baseline
+(`disk_health.go:26-29`, *"Lost on restart → the next check re-baselines silently"*). Consequences:
+
+- Between 8 and 352 pending sectors the verdict never changes level, so **nothing further is emitted**.
+- A controller restart while a disk is already bad re-baselines it silently — that disk never alerts
+  again.
+
+---
+
+## 5. Counterfactual — what a customer would have received
+
+**Derived from the source above plus the §3 timeline. NOT reproduced live.**
+
+| Date/time | Drive state | Felhom verdict at v0.214.0 | Emitted | Delivered |
+|-----------|-------------|-----------------------------|---------|-----------|
+| 11 Aug 12:28 | pending 8 | OK → Figyelmeztetés | `disk_health_degraded`, severity `warn` | **nothing** — coerced to `info`, dropped by `severityNotifies` |
+| 11 Aug 13:28 | pending 0 | Figyelmeztetés → Rendben | none (recovery is silent) | nothing |
+| 11 Aug 20:28 → 13 Aug | 8 → 352 | Figyelmeztetés throughout | none (no level change) | nothing |
+| 13 Aug 21:58 | pending 360 | Figyelmeztetés | none | nothing |
+
+> **Felhom would have emitted zero emails about this drive.** The one event it did produce was filed
+> at `info` and delivered to no one.
+
+Note the two defects compound: even had D1 been fixed alone, the customer would have received a
+single mild "Javasolt figyelemmel kísérni" at 8 sectors on 11 August and then silence through 352.
+
+---
+
+## 6. Provenance of the thresholds chosen in v0.215.0
+
+- **64 unreadable sectors → Hiba.** The observed benign excursion peaked at **16** and cleared inside
+  an hour; the terminal run passed **64** at 13 Aug 11:28 and never returned below it. 64 sits above
+  the one observed transient and below the observed terminal run. **This is a judgement from ONE
+  drive.** It is a static backstop and is expected to be replaced in Phase 3 by growth-rate detection
+  over real history.
+- **Sustain before count.** The primary rule is "unreadable sectors still present at the next check";
+  the count is the backstop. On this drive sustain fires **12 Aug**, the count not until **13 Aug** —
+  a full day earlier. The backstop exists for a box that was powered off or restarted across the
+  sustain window.
+- **55 / 60 °C.** Adopted unchanged from the operator's existing Prometheus bands on DooPlex, so the
+  two systems cannot disagree about the same drive.
+
+---
+
+## 7. Measured vs inferred
+
+**Measured** (reproducible from the committed fixtures):
+- The attribute table, thresholds and raw values in §2; `passed: true` at 352 pending sectors.
+- The full non-monotonic timeline in §3, including the one-hour full clear.
+- The three defect locators in §4 — read from live source on both the controller and the hub side.
+
+**Inferred** (source-derived, not executed):
+- The §5 counterfactual. It follows from the §4 locators and the §3 timeline; **no email path was
+  exercised against this drive**, and no `disk_health_degraded` event for it exists in the hub.
+
+**Not covered here:**
+- Attributes **187**, **199** and **188** are not on the agent→controller wire today. Adding them is
+  Phase 2 (a declared wire change, so the hub models them in the same session under G-1). Everything
+  the v0.215.0 fix needs was already on the wire.
+- The new Fail-from-counters path has **not** fired on real hardware — only against this fixture's
+  values in unit tests.
+
+---
+
+## 8. One layer out
+
+`smartd` on DooPlex did its job and mailed local root, where nothing reads. That is the same shape
+as D1 — a correct detection with a delivery path to nowhere — one layer outside the product. Tracked
+separately as DooPlex hygiene.

documentation/audits/DRILL-backup-truth-2026-08-21/evidence/phase0-preexisting-scratch/offsite-restore-scratch.listing.txt

@@ -0,0 +1,42 @@
+drwxr-xr-x root/root         0 2026-08-21 18:33 offsite-restore/
+drwxr-xr-x root/root         0 2026-08-21 18:33 offsite-restore/opengist/
+drwxr-xr-x root/root         0 2026-08-03 08:11 offsite-restore/opengist/mnt/
+drwxr-xr-x root/root         0 2026-08-04 14:51 offsite-restore/opengist/mnt/sys_drive/
+drwxr-xr-x root/root         0 2026-08-03 08:29 offsite-restore/opengist/mnt/sys_drive/felhom-data/
+drwxr-xr-x root/root         0 2026-08-04 23:13 offsite-restore/opengist/mnt/sys_drive/felhom-data/backups/
+drwxr-xr-x root/root         0 2026-08-06 22:02 offsite-restore/opengist/mnt/sys_drive/felhom-data/backups/primary/
+drwxr-xr-x root/root         0 2026-08-09 10:30 offsite-restore/opengist/mnt/sys_drive/felhom-data/backups/primary/opengist/
+drwxr-xr-x root/root         0 2026-08-09 10:30 offsite-restore/opengist/mnt/sys_drive/felhom-data/backups/primary/opengist/compose/
+-rw-r--r-- root/root      1750 2026-08-09 10:30 offsite-restore/opengist/mnt/sys_drive/felhom-data/backups/primary/opengist/compose/.felhom.yml
+-rw-r--r-- root/root      1260 2026-08-09 10:30 offsite-restore/opengist/mnt/sys_drive/felhom-data/backups/primary/opengist/compose/docker-compose.yml
+-rw------- root/root       287 2026-08-09 10:30 offsite-restore/opengist/mnt/sys_drive/felhom-data/backups/primary/opengist/compose/app.yaml
+drwxr-xr-x root/root         0 2026-08-09 10:30 offsite-restore/opengist/mnt/sys_drive/felhom-data/backups/primary/opengist/volume-dumps/
+-rw-r--r-- root/root    182272 2026-08-09 10:30 offsite-restore/opengist/mnt/sys_drive/felhom-data/backups/primary/opengist/volume-dumps/opengist_opengist_data.tar
+-rw-r--r-- root/root      1121 2026-08-09 10:30 offsite-restore/opengist/mnt/sys_drive/felhom-data/backups/primary/opengist/manifest.json
+drwxr-xr-x root/root         0 2026-08-21 18:30 offsite-restore/calibre-web/
+drwxr-xr-x root/root         0 2026-08-03 08:11 offsite-restore/calibre-web/mnt/
+drwxr-xr-x root/root         0 2026-08-04 14:51 offsite-restore/calibre-web/mnt/sys_drive/
+drwxr-xr-x root/root         0 2026-08-03 08:29 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/
+drwxrwsr-x root/1000         0 2026-08-04 18:45 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/userdata/
+drwxr-sr-x root/1000         0 2026-08-04 18:50 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/userdata/media/
+drwxrwsr-x 1000/1000         0 2026-08-09 10:15 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/userdata/media/books/
+-rw-r--r-- 1000/1000       181 2026-08-04 14:53 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/userdata/media/books/DRILL-SENTINEL.txt
+drwxrwxr-x 1000/1000         0 2026-08-09 09:59 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/userdata/media/books/rehearsal-2026-08-09/
+-rw-rw-r-- 1000/1000        21 2026-08-09 09:59 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/userdata/media/books/rehearsal-2026-08-09/plain.txt
+-rw-rw-r-- 1000/1000   3145728 2026-08-09 09:59 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/userdata/media/books/rehearsal-2026-08-09/binary-3mb.bin
+drwxrwxr-x 1000/1000         0 2026-08-09 09:59 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/userdata/media/books/rehearsal-2026-08-09/nested/
+-rw-rw-r-- 1000/1000        25 2026-08-09 09:59 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/userdata/media/books/rehearsal-2026-08-09/nested/\305\221szibarack.md
+-rw-rw-r-- 1000/1000        59 2026-08-09 09:59 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/userdata/media/books/rehearsal-2026-08-09/\303\241rv\303\255zt\305\261r\305\221-t\303\274k\303\266rf\303\272r\303\263g\303\251p.txt
+-rw-r--r-- 1000/1000     32768 2026-08-09 10:30 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/userdata/media/books/metadata.db-shm
+-rw-r--r-- 1000/1000         0 2026-08-09 04:15 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/userdata/media/books/metadata.db-wal
+-rw-r--r-- 1000/1000    413696 2026-08-04 14:52 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/userdata/media/books/metadata.db
+drwxr-xr-x root/root         0 2026-08-04 23:13 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/backups/
+drwxr-xr-x root/root         0 2026-08-06 22:02 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/backups/primary/
+drwxr-xr-x root/root         0 2026-08-09 10:30 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/backups/primary/calibre-web/
+drwxr-xr-x root/root         0 2026-08-09 10:30 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/backups/primary/calibre-web/compose/
+-rw-r--r-- root/root      3035 2026-08-09 10:30 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/backups/primary/calibre-web/compose/.felhom.yml
+-rw-r--r-- root/root      2122 2026-08-09 10:30 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/backups/primary/calibre-web/compose/docker-compose.yml
+-rw------- root/root       317 2026-08-09 10:30 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/backups/primary/calibre-web/compose/app.yaml
+drwxr-xr-x root/root         0 2026-08-09 10:30 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/backups/primary/calibre-web/volume-dumps/
+-rw-r--r-- root/root    368640 2026-08-09 10:30 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/backups/primary/calibre-web/volume-dumps/calibre-web_calibre_web_config.tar
+-rw-r--r-- root/root      1157 2026-08-09 10:30 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/backups/primary/calibre-web/manifest.json

documentation/audits/DRILL-backup-truth-2026-08-21/evidence/phase0-preexisting-scratch/offsite-restore-scratch.sha256

@@ -0,0 +1 @@
+c9498bfba3dab7b8c59196be8a6c792f0044356d969b2d81ce4b00b61950aa96  documentation/audits/DRILL-backup-truth-2026-08-21/evidence/phase0-preexisting-scratch/offsite-restore-scratch.tar

documentation/audits/DRILL-backup-truth-2026-08-21/evidence/phase3-experiment/RESULT-MATRIX.txt

@@ -0,0 +1,22 @@
+DRILL 2026-08-21 — FULL/EMPTY contrast on demo-hp. All hashes sha256, byte-for-byte.
+Comparator positive control: one byte flipped at offset 500000 of binary-1mb.bin
+(af -> 00) => sha256sum -c FAILED rc=1; original PASSED rc=0. Mutant discarded.
+
+Planted fixture (5 files, incl. two UTF-8 Hungarian accented names):
+  SENTINEL.txt                    0c23c8531214fe20cbc1ed177da22f51d65570c5aaedb39e380aa8d4d3e62991
+  binary-1mb.bin                  725763bbe679b22d5c231a14083ff13155f475c028c22ade4623955b50a2a84a
+  nested/őszibarack.md            a39ad6f623da67ac72f2e62a24245eef46c722c279ae89cd6b0c7c45afa6a4c4
+  plain.txt                       07e91a985809fc96752f97cddcf6523b211bc607c81a5808befabe35573926f1
+  árvíztűrő-tükörfúrógép.txt      0d6a22ec56acf61b8a80e73eb12cb223d607acaa3684acfcfa7c3e34ecdfabea
+Name bytes (UTF-8 NFC):
+  árvíztűrő-tükörfúrógép.txt = c3a1 72 76 c3ad 7a 74 c5b1 72 c591 2d 74 c3bc 6b c3b6 72 66 c3ba 72 c3b3 67 c3a9 70 2e747874
+  őszibarack.md              = c591 73 7a 69 62 61 72 61 63 6b 2e 6d 64
+
+                     | class      | data leg          | in unit | in offsite snap | checking folder | OFF-SITE restore | LOCAL restore
+ calibre-web  FULL   | drive      | userdata files    |   n/a   | YES 5/5 ident.  | YES 5/5 ident.  | YES 5/5 ident.   | -
+ calibre-web  FULL   | drive      | named volume      | YES 1.42MB | YES          | YES             | NO  (silent)     | -
+ privatebin   FULL   | no-drive   | named volume      | YES 1.06MB | YES 5/5 ident.| YES 5/5 ident.  | REFUSED (false)  | YES 5/5 ident.
+ opengist     EMPTY  | no-drive   | named volume      | YES 181KB skeleton | YES  | YES             | REFUSED (false)  | -
+
+CONCLUSION: the unit and the off-site snapshot HOLD the data, verified by identity.
+The off-site full restore has no named-volume leg at all; the local restore has one.

documentation/audits/DRILL-backup-truth-2026-08-21/evidence/phase3-experiment/checking-folders-after-full-restore.listing.txt

@@ -0,0 +1,88 @@
+drwxr-xr-x root/root         0 2026-08-21 22:19 offsite-restore/
+drwxr-xr-x root/root         0 2026-08-21 18:33 offsite-restore/opengist/
+drwxr-xr-x root/root         0 2026-08-21 18:00 offsite-restore/opengist/mnt/
+drwxr-xr-x root/root         0 2026-08-21 18:01 offsite-restore/opengist/mnt/sys_drive/
+drwxr-xr-x root/root         0 2026-08-21 18:31 offsite-restore/opengist/mnt/sys_drive/felhom-data/
+drwxr-xr-x root/root         0 2026-08-21 18:31 offsite-restore/opengist/mnt/sys_drive/felhom-data/backups/
+drwxr-xr-x root/root         0 2026-08-21 18:31 offsite-restore/opengist/mnt/sys_drive/felhom-data/backups/primary/
+drwxr-xr-x root/root         0 2026-08-21 22:17 offsite-restore/opengist/mnt/sys_drive/felhom-data/backups/primary/opengist/
+drwxr-xr-x root/root         0 2026-08-21 22:17 offsite-restore/opengist/mnt/sys_drive/felhom-data/backups/primary/opengist/compose/
+-rw-r--r-- root/root      1750 2026-08-21 22:17 offsite-restore/opengist/mnt/sys_drive/felhom-data/backups/primary/opengist/compose/.felhom.yml
+-rw-r--r-- root/root      1260 2026-08-21 22:17 offsite-restore/opengist/mnt/sys_drive/felhom-data/backups/primary/opengist/compose/docker-compose.yml
+-rw------- root/root       287 2026-08-21 22:17 offsite-restore/opengist/mnt/sys_drive/felhom-data/backups/primary/opengist/compose/app.yaml
+drwxr-xr-x root/root         0 2026-08-21 22:16 offsite-restore/opengist/mnt/sys_drive/felhom-data/backups/primary/opengist/volume-dumps/
+-rw-r--r-- root/root    181248 2026-08-21 22:16 offsite-restore/opengist/mnt/sys_drive/felhom-data/backups/primary/opengist/volume-dumps/opengist_opengist_data.tar
+-rw-r--r-- root/root      1121 2026-08-21 22:17 offsite-restore/opengist/mnt/sys_drive/felhom-data/backups/primary/opengist/manifest.json
+drwxr-xr-x root/root         0 2026-08-21 22:19 offsite-restore/privatebin/
+drwxr-xr-x root/root         0 2026-08-21 18:00 offsite-restore/privatebin/mnt/
+drwxr-xr-x root/root         0 2026-08-21 18:01 offsite-restore/privatebin/mnt/sys_drive/
+drwxr-xr-x root/root         0 2026-08-21 18:31 offsite-restore/privatebin/mnt/sys_drive/felhom-data/
+drwxr-xr-x root/root         0 2026-08-21 18:31 offsite-restore/privatebin/mnt/sys_drive/felhom-data/backups/
+drwxr-xr-x root/root         0 2026-08-21 18:31 offsite-restore/privatebin/mnt/sys_drive/felhom-data/backups/primary/
+drwxr-xr-x root/root         0 2026-08-21 22:17 offsite-restore/privatebin/mnt/sys_drive/felhom-data/backups/primary/privatebin/
+drwxr-xr-x root/root         0 2026-08-21 22:17 offsite-restore/privatebin/mnt/sys_drive/felhom-data/backups/primary/privatebin/compose/
+-rw-r--r-- root/root      1723 2026-08-21 22:17 offsite-restore/privatebin/mnt/sys_drive/felhom-data/backups/primary/privatebin/compose/.felhom.yml
+-rw-r--r-- root/root      1223 2026-08-21 22:17 offsite-restore/privatebin/mnt/sys_drive/felhom-data/backups/primary/privatebin/compose/docker-compose.yml
+-rw------- root/root       288 2026-08-21 22:17 offsite-restore/privatebin/mnt/sys_drive/felhom-data/backups/primary/privatebin/compose/app.yaml
+drwxr-xr-x root/root         0 2026-08-21 22:16 offsite-restore/privatebin/mnt/sys_drive/felhom-data/backups/primary/privatebin/volume-dumps/
+-rw-r--r-- root/root   1055744 2026-08-21 22:16 offsite-restore/privatebin/mnt/sys_drive/felhom-data/backups/primary/privatebin/volume-dumps/privatebin_privatebin_data.tar
+-rw-r--r-- root/root      1118 2026-08-21 22:17 offsite-restore/privatebin/mnt/sys_drive/felhom-data/backups/primary/privatebin/manifest.json
+drwxr-xr-x root/root         0 2026-08-21 18:30 offsite-restore/calibre-web/
+drwxr-xr-x root/root         0 2026-08-21 18:00 offsite-restore/calibre-web/mnt/
+drwxr-xr-x nobody/nogroup    0 2026-08-21 18:26 offsite-restore/calibre-web/mnt/felhom-drives/
+drwxr-xr-x root/root         0 2026-07-22 03:30 offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/
+drwxrwsr-x root/1000         0 2026-07-21 19:08 offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/userdata/
+drwxrwsr-x root/1000         0 2026-07-26 08:17 offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/userdata/media/
+drwxrwsr-x 1000/1000         0 2026-08-21 22:10 offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/userdata/media/books/
+-rw-r--r-- 1000/1000       181 2026-08-04 14:53 offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/userdata/media/books/DRILL-SENTINEL.txt
+drwxrwxr-x 1000/1000         0 2026-08-21 22:09 offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/userdata/media/books/DRILL-2026-08-21/
+-rw-rw-r-- 1000/1000        35 2026-08-21 22:09 offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/userdata/media/books/DRILL-2026-08-21/plain.txt
+-rw-rw-r-- 1000/1000   1048576 2026-08-21 22:09 offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/userdata/media/books/DRILL-2026-08-21/binary-1mb.bin
+drwxrwxr-x 1000/1000         0 2026-08-21 22:09 offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/userdata/media/books/DRILL-2026-08-21/nested/
+-rw-rw-r-- 1000/1000        21 2026-08-21 22:09 offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/userdata/media/books/DRILL-2026-08-21/nested/\305\221szibarack.md
+-rw-rw-r-- 1000/1000        46 2026-08-21 22:09 offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/userdata/media/books/DRILL-2026-08-21/SENTINEL.txt
+-rw-rw-r-- 1000/1000        52 2026-08-21 22:09 offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/userdata/media/books/DRILL-2026-08-21/\303\241rv\303\255zt\305\261r\305\221-t\303\274k\303\266rf\303\272r\303\263g\303\251p.txt
+drwxrwxr-x 1000/1000         0 2026-08-09 09:59 offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/userdata/media/books/rehearsal-2026-08-09/
+-rw-rw-r-- 1000/1000        21 2026-08-09 09:59 offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/userdata/media/books/rehearsal-2026-08-09/plain.txt
+-rw-rw-r-- 1000/1000   3145728 2026-08-09 09:59 offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/userdata/media/books/rehearsal-2026-08-09/binary-3mb.bin
+drwxrwxr-x 1000/1000         0 2026-08-09 09:59 offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/userdata/media/books/rehearsal-2026-08-09/nested/
+-rw-rw-r-- 1000/1000        25 2026-08-09 09:59 offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/userdata/media/books/rehearsal-2026-08-09/nested/\305\221szibarack.md
+-rw-rw-r-- 1000/1000        59 2026-08-09 09:59 offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/userdata/media/books/rehearsal-2026-08-09/\303\241rv\303\255zt\305\261r\305\221-t\303\274k\303\266rf\303\272r\303\263g\303\251p.txt
+-rw-r--r-- 1000/1000     32768 2026-08-21 22:17 offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/userdata/media/books/metadata.db-shm
+-rw-r--r-- 1000/1000         0 2026-08-09 04:15 offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/userdata/media/books/metadata.db-wal
+-rw-r--r-- 1000/1000    413696 2026-08-04 14:52 offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/userdata/media/books/metadata.db
+drwxr-xr-x root/root         0 2026-07-23 12:10 offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/backups/
+drwxr-xr-x root/root         0 2026-08-21 18:31 offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/backups/primary/
+drwxr-xr-x root/root         0 2026-08-21 22:17 offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/backups/primary/calibre-web/
+drwxr-xr-x root/root         0 2026-08-21 22:17 offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/backups/primary/calibre-web/compose/
+-rw-r--r-- root/root      3035 2026-08-21 22:17 offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/backups/primary/calibre-web/compose/.felhom.yml
+-rw-r--r-- root/root      2122 2026-08-21 22:17 offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/backups/primary/calibre-web/compose/docker-compose.yml
+-rw------- root/root       327 2026-08-21 22:17 offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/backups/primary/calibre-web/compose/app.yaml
+drwxr-xr-x root/root         0 2026-08-21 22:16 offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/backups/primary/calibre-web/volume-dumps/
+-rw-r--r-- root/root   1422848 2026-08-21 22:16 offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/backups/primary/calibre-web/volume-dumps/calibre-web_calibre_web_config.tar
+-rw-r--r-- root/root      1165 2026-08-21 22:17 offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/backups/primary/calibre-web/manifest.json
+drwxr-xr-x root/root         0 2026-08-04 14:51 offsite-restore/calibre-web/mnt/sys_drive/
+drwxr-xr-x root/root         0 2026-08-03 08:29 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/
+drwxrwsr-x root/1000         0 2026-08-04 18:45 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/userdata/
+drwxr-sr-x root/1000         0 2026-08-04 18:50 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/userdata/media/
+drwxrwsr-x 1000/1000         0 2026-08-09 10:15 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/userdata/media/books/
+-rw-r--r-- 1000/1000       181 2026-08-04 14:53 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/userdata/media/books/DRILL-SENTINEL.txt
+drwxrwxr-x 1000/1000         0 2026-08-09 09:59 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/userdata/media/books/rehearsal-2026-08-09/
+-rw-rw-r-- 1000/1000        21 2026-08-09 09:59 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/userdata/media/books/rehearsal-2026-08-09/plain.txt
+-rw-rw-r-- 1000/1000   3145728 2026-08-09 09:59 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/userdata/media/books/rehearsal-2026-08-09/binary-3mb.bin
+drwxrwxr-x 1000/1000         0 2026-08-09 09:59 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/userdata/media/books/rehearsal-2026-08-09/nested/
+-rw-rw-r-- 1000/1000        25 2026-08-09 09:59 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/userdata/media/books/rehearsal-2026-08-09/nested/\305\221szibarack.md
+-rw-rw-r-- 1000/1000        59 2026-08-09 09:59 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/userdata/media/books/rehearsal-2026-08-09/\303\241rv\303\255zt\305\261r\305\221-t\303\274k\303\266rf\303\272r\303\263g\303\251p.txt
+-rw-r--r-- 1000/1000     32768 2026-08-09 10:30 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/userdata/media/books/metadata.db-shm
+-rw-r--r-- 1000/1000         0 2026-08-09 04:15 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/userdata/media/books/metadata.db-wal
+-rw-r--r-- 1000/1000    413696 2026-08-04 14:52 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/userdata/media/books/metadata.db
+drwxr-xr-x root/root         0 2026-08-04 23:13 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/backups/
+drwxr-xr-x root/root         0 2026-08-06 22:02 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/backups/primary/
+drwxr-xr-x root/root         0 2026-08-09 10:30 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/backups/primary/calibre-web/
+drwxr-xr-x root/root         0 2026-08-09 10:30 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/backups/primary/calibre-web/compose/
+-rw-r--r-- root/root      3035 2026-08-09 10:30 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/backups/primary/calibre-web/compose/.felhom.yml
+-rw-r--r-- root/root      2122 2026-08-09 10:30 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/backups/primary/calibre-web/compose/docker-compose.yml
+-rw------- root/root       317 2026-08-09 10:30 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/backups/primary/calibre-web/compose/app.yaml
+drwxr-xr-x root/root         0 2026-08-09 10:30 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/backups/primary/calibre-web/volume-dumps/
+-rw-r--r-- root/root    368640 2026-08-09 10:30 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/backups/primary/calibre-web/volume-dumps/calibre-web_calibre_web_config.tar
+-rw-r--r-- root/root      1157 2026-08-09 10:30 offsite-restore/calibre-web/mnt/sys_drive/felhom-data/backups/primary/calibre-web/manifest.json

documentation/audits/DRILL-backup-truth-2026-08-21/evidence/phase3-experiment/checking-folders-after-full-restore.sha256

@@ -0,0 +1 @@
+7e59e57d6458d28f530dbaddbee0f2314ea1ef885052701531f57bad2529849d  documentation/audits/DRILL-backup-truth-2026-08-21/evidence/phase3-experiment/checking-folders-after-full-restore.tar

documentation/audits/DRILL-backup-truth-2026-08-21/evidence/phase3-experiment/messages-verbatim.txt

@@ -0,0 +1,32 @@
+VERBATIM customer-facing outcome strings, read from /api/backup/restore-status
+(the same value the wizard banner renders). Times CEST.
+
+1) 22:21:51  off-site FULL RESTORE (reconstitute), app = privatebin  [40-class, no HDD_PATH]
+   ok = FALSE
+   "A teljes visszaállítás sikertelen: a(z) privatebin nincs telepítve, ezért nincs hová
+    visszaállítani az adatait. A mentése szerint az adatai itt voltak: /mnt/sys_drive.
+    Telepítsd újra az alkalmazást (Alkalmazások) ugyanerre a helyre, utána ez a
+    visszaállítás működni fog"
+   FACT AT THAT MOMENT: privatebin deployed=true, state=running, container healthy.
+
+2) 22:23:36  off-site FULL RESTORE (reconstitute), app = calibre-web  [drive class]
+   ok = TRUE
+   "A(z) calibre-web: 5 fájl visszaállítva (mentés: 2026-08-21 22:16) — az alkalmazás
+    újraindult. Ennek az alkalmazásnak nincs adatbázisa."
+   FACT: the 5 declared-userdata files came back byte-identical.
+         The named volume calibre-web_calibre_web_config was NOT restored, though its
+         1,422,848-byte tar was in the snapshot, in the checking folder, and named in
+         manifest.json volume_dumps. The message does not mention it.
+
+3) 22:25:31  LOCAL restore from recovery unit, app = privatebin
+   ok = TRUE
+   "privatebin visszaállítva (helyi)."
+   FACT: the named volume WAS restored, all 5 planted files byte-identical.
+         The message carries no counts at all - it reads the same whatever happened.
+
+4) 22:13:15  off-site backup run with ZERO apps selected
+   log:  "[offbox] backup run started (0 app(s) toggled)"
+         "[offbox] backup OK: 0 app(s) backed up, 18 snapshot(s), 14s"
+   card: badge "Aktív — nincs kijelölt alkalmazás" / "✓ Rendben" /
+         "Sikeres — nincs mentésre jelölt alkalmazás" /
+         "Nincs távoli mentésre jelölt alkalmazás — jelölj ki legalább egyet."

documentation/audits/DRILL-backup-truth-2026-08-21/evidence/phase3-experiment/plant-manifest.txt

@@ -0,0 +1,5 @@
+0c23c8531214fe20cbc1ed177da22f51d65570c5aaedb39e380aa8d4d3e62991  DRILL-2026-08-21/SENTINEL.txt
+725763bbe679b22d5c231a14083ff13155f475c028c22ade4623955b50a2a84a  DRILL-2026-08-21/binary-1mb.bin
+a39ad6f623da67ac72f2e62a24245eef46c722c279ae89cd6b0c7c45afa6a4c4  DRILL-2026-08-21/nested/őszibarack.md
+07e91a985809fc96752f97cddcf6523b211bc607c81a5808befabe35573926f1  DRILL-2026-08-21/plain.txt
+0d6a22ec56acf61b8a80e73eb12cb223d607acaa3684acfcfa7c3e34ecdfabea  DRILL-2026-08-21/árvíztűrő-tükörfúrógép.txt

documentation/audits/DRILL-backup-truth-2026-08-21/evidence/phase4-abandon/repo-inventory-before.txt

@@ -0,0 +1,59 @@
+ID        Time                 Host        Tags                         Paths
+------------------------------------------------------------------------------------------------------------------------------
+e6132ae5  2026-08-04 19:36:26  demo-hp     felhom-offbox,calibre-web    /mnt/sys_drive/felhom-data/backups/primary/calibre-web
+                                                                        /mnt/sys_drive/felhom-data/userdata/media/books
+
+9ac78c98  2026-08-04 19:36:31  demo-hp     felhom-offbox,opengist       /mnt/sys_drive/felhom-data/backups/primary/opengist
+
+92212ff8  2026-08-04 19:36:36  demo-hp     felhom-offbox,privatebin     /mnt/sys_drive/felhom-data/backups/primary/privatebin
+
+9d6d233f  2026-08-05 09:12:42  demo-hp     felhom-offbox,calibre-web    /mnt/sys_drive/felhom-data/backups/primary/calibre-web
+                                                                        /mnt/sys_drive/felhom-data/userdata/media/books
+
+29e7b245  2026-08-05 09:12:48  demo-hp     felhom-offbox,opengist       /mnt/sys_drive/felhom-data/backups/primary/opengist
+
+cd1db049  2026-08-05 09:12:53  demo-hp     felhom-offbox,privatebin     /mnt/sys_drive/felhom-data/backups/primary/privatebin
+
+0ef7a006  2026-08-06 20:00:44  demo-hp     felhom-offbox,calibre-web    /mnt/sys_drive/felhom-data/backups/primary/calibre-web
+                                                                        /mnt/sys_drive/felhom-data/userdata/media/books
+
+8662a8c1  2026-08-06 20:00:50  demo-hp     felhom-offbox,opengist       /mnt/sys_drive/felhom-data/backups/primary/opengist
+
+6dfa6602  2026-08-06 20:00:54  demo-hp     felhom-offbox,privatebin     /mnt/sys_drive/felhom-data/backups/primary/privatebin
+
+3635f945  2026-08-07 02:15:13  demo-hp     felhom-offbox,calibre-web    /mnt/sys_drive/felhom-data/backups/primary/calibre-web
+                                                                        /mnt/sys_drive/felhom-data/userdata/media/books
+
+7b1fa8b0  2026-08-07 02:15:17  demo-hp     felhom-offbox,opengist       /mnt/sys_drive/felhom-data/backups/primary/opengist
+
+38edf5b8  2026-08-07 02:15:21  demo-hp     felhom-offbox,privatebin     /mnt/sys_drive/felhom-data/backups/primary/privatebin
+
+a4d03ee3  2026-08-08 02:15:12  demo-hp     felhom-offbox,calibre-web    /mnt/sys_drive/felhom-data/backups/primary/calibre-web
+                                                                        /mnt/sys_drive/felhom-data/userdata/media/books
+
+f534bffe  2026-08-08 02:15:17  demo-hp     felhom-offbox,opengist       /mnt/sys_drive/felhom-data/backups/primary/opengist
+
+a685d30e  2026-08-08 02:15:22  demo-hp     felhom-offbox,privatebin     /mnt/sys_drive/felhom-data/backups/primary/privatebin
+
+41c830db  2026-08-09 08:30:38  demo-hp     felhom-offbox,calibre-web    /mnt/sys_drive/felhom-data/backups/primary/calibre-web
+                                                                        /mnt/sys_drive/felhom-data/userdata/media/books
+
+9e38b84c  2026-08-09 08:30:49  demo-hp     felhom-offbox,opengist       /mnt/sys_drive/felhom-data/backups/primary/opengist
+
+78b93f04  2026-08-09 08:30:53  demo-hp     felhom-offbox,privatebin     /mnt/sys_drive/felhom-data/backups/primary/privatebin
+
+8c44bd4c  2026-08-21 21:00:52  demo-hp     felhom-offbox,calibre-web    /mnt/felhom-drives/hdd_1/backups/primary/calibre-web
+                                                                        /mnt/felhom-drives/hdd_1/userdata/media/books
+
+07bac5ad  2026-08-21 21:00:55  demo-hp     felhom-offbox,privatebin     /mnt/sys_drive/felhom-data/backups/primary/privatebin
+
+7e4b703b  2026-08-21 21:00:58  demo-hp     felhom-offbox,kimai          /mnt/sys_drive/felhom-data/backups/primary/kimai
+
+16cb8ce7  2026-08-21 21:01:08  demo-hp     felhom-offbox,paperless-ngx  /mnt/felhom-drives/hdd_1/appdata/paperless/media
+                                                                        /mnt/felhom-drives/hdd_1/backups/primary/paperless-ngx
+
+2a891149  2026-08-21 21:01:13  demo-hp     felhom-offbox,romm           /mnt/felhom-drives/hdd_1/backups/primary/romm
+
+49b9f317  2026-08-21 21:01:22  demo-hp     felhom-offbox,opengist       /mnt/sys_drive/felhom-data/backups/primary/opengist
+------------------------------------------------------------------------------------------------------------------------------
+24 snapshots

documentation/audits/DRILL-backup-truth-2026-08-21/evidence/phase4-abandon/sweep-result.txt

@@ -0,0 +1,23 @@
+PART 4.2 — the abandonment sweep, watched firing. 2026-08-22.
+
+State created 2026-08-21 23:08-23:09 CEST:
+  set-aside store  u629488-sub3:/home/felhom-repo-superseded-drill-20260821  (config/data/index/snapshots)
+  countdown        started 2026-08-07, due 2026-08-20  (written into settings.json, controller restarted)
+  product CLI      "abandonment countdown RUNNING … deleted on: 2026-08-20 … days left: 0"
+
+05:10 CEST — the daily `offsite-abandon-sweep` job fired.
+  /home on the storage box now stamped 03:10Z
+  /home/felhom-repo-superseded-drill-20260821   -> GONE
+  /home/felhom-repo  (the LIVE repository)      -> present, mtime still Aug 4  ** survived **
+
+05:13 CEST — the hub half. Event 3025 `offsite_abandon_purged`:
+  "Az ügyfél korábbi távoli mentései és a hozzájuk tartozó megőrzött helyreállítási csomag is
+   törölve (1 csomag). Az ügyfél döntése alapján, a 14 napos türelmi idő lejárta után."
+  demo-hp's row in host_escrow_superseded: REMOVED (there was exactly one).
+  demo-felhom's rows 4, 11, 12: ALL PRESENT — the protected fixtures were not touched.
+
+Controller closed itself out: every abandon_* field removed from settings.json;
+`--abandon-status` reports "no abandonment countdown is running on this box".
+
+VERDICT: the two-phase commit's promise — "it removes BOTH halves or neither" — is confirmed
+live for the first time. It removed exactly the recorded set-aside path and nothing else.

documentation/audits/DRILL-backup-truth-2026-08-21/evidence/phase4-paperless/FINDING.txt

@@ -0,0 +1,34 @@
+paperless-ngx: the database is dumped into a directory for a stack that does not exist,
+so the recovery unit never contains it, and the restore then tells the customer the app
+has no database. Proven live 2026-08-21 22:39-22:45 CEST on demo-hp.
+
+MECHANISM (source):
+  internal/appbackup/dbdump.go:770 deriveStackName("paperless-postgres", known)
+    1. candidate = suffixStripStackName("paperless-postgres") = "paperless"   (line 801)
+    2. known is non-empty, known["paperless"] is FALSE   (the stack is "paperless-ngx")
+    3. known["paperless-postgres"] is FALSE
+    4. no known stack name is a prefix of "paperless-postgres"  ("paperless-ngx" is not)
+    5. FALLS THROUGH to `return candidate`  (line 797) -> "paperless"
+  An unresolved mapping is returned as if resolved. There is no warning and no refusal.
+
+OBSERVED CONSEQUENCES (all live):
+  a) the dump is written to
+       /mnt/sys_drive/felhom-data/backups/primary/paperless/db-dumps/paperless-postgres.sql
+     284,617 bytes, 72 tables, valid=true  -- an orphan directory for a non-existent stack,
+     on the SYSTEM drive, while the app's real unit is on /mnt/felhom-drives/hdd_1.
+  b) the real unit /mnt/felhom-drives/hdd_1/backups/primary/paperless-ngx/manifest.json
+     records  "db_dumps": null
+  c) the off-site snapshot therefore carries no .sql at all
+     (checking folder: `find ... -name "*.sql" | wc -l` = 0)
+  d) writeSafetyDump (offbox_reconstitute.go:115) filters discovered DBs on
+     db.StackName == stackName, so `mine` is empty -> returns ("", nil) -> hasDB = false.
+     NO pre-restore safety dump is taken.  Verified: `find /mnt -name "pre-restore-*"`
+     returned nothing before AND after the destructive restore.
+  e) the destructive restore ran to completion and reported SUCCESS:
+       "A(z) paperless-ngx: 0 fájl visszaállítva (mentés: 2026-08-21 22:41)
+        — az alkalmazás újraindult. Ennek az alkalmazásnak nincs adatbázisa."
+     The controller had dumped that same database 5 minutes earlier.
+
+The orphan directory is also invisible to the app's off-site push, because the push
+resolves paths from the app's own unit path -- so the only copy of that database dump
+is on the system drive of the machine it protects.

documentation/audits/DRILL-backup-truth-2026-08-21/evidence/phase4-paperless/discovery-log.txt

@@ -0,0 +1,9 @@
+{"timestamp":"2026-08-21T20:38:46Z","level":"DEBUG","message":"DiscoverDatabases: paperless-postgres → stack=paperless, dbUser=paperless, dbName=paperless","source":"dbdump.go:150"}
+{"timestamp":"2026-08-21T20:39:37Z","level":"DEBUG","message":"DiscoverDatabases: paperless-postgres → stack=paperless, dbUser=paperless, dbName=paperless","source":"dbdump.go:150"}
+{"timestamp":"2026-08-21T20:39:37Z","level":"DEBUG","message":"DumpOne: starting dump for container=paperless-postgres, stack=paperless, dbType=postgres, dumpDir=/mnt/sys_drive/felhom-data/backups/primary/paperless/db-dumps","source":"dbdump.go:189"}
+{"timestamp":"2026-08-21T20:39:37Z","level":"DEBUG","message":"DumpOne: completed paperless-postgres → paperless-postgres.sql (size=277.9 KB, valid=true, tables=72, duration=313ms)","source":"dbdump.go:330"}
+{"timestamp":"2026-08-21T20:41:23Z","level":"DEBUG","message":"DiscoverDatabases: paperless-postgres → stack=paperless, dbUser=paperless, dbName=paperless","source":"dbdump.go:150"}
+{"timestamp":"2026-08-21T20:41:23Z","level":"DEBUG","message":"DumpOne: starting dump for container=paperless-postgres, stack=paperless, dbType=postgres, dumpDir=/mnt/sys_drive/felhom-data/backups/primary/paperless/db-dumps","source":"dbdump.go:189"}
+{"timestamp":"2026-08-21T20:41:23Z","level":"DEBUG","message":"DumpOne: completed paperless-postgres → paperless-postgres.sql (size=278.2 KB, valid=true, tables=72, duration=314ms)","source":"dbdump.go:330"}
+{"timestamp":"2026-08-21T20:43:46Z","level":"DEBUG","message":"DiscoverDatabases: paperless-postgres → stack=paperless, dbUser=paperless, dbName=paperless","source":"dbdump.go:150"}
+{"timestamp":"2026-08-21T20:44:33Z","level":"DEBUG","message":"DiscoverDatabases: paperless-postgres → stack=paperless, dbUser=paperless, dbName=paperless","source":"dbdump.go:150"}

documentation/audits/DRILL-backup-truth-2026-08-21/evidence/phase4-paperless/orphan-dump-and-manifest.txt

@@ -0,0 +1,48 @@
+total 288
+drwxr-xr-x 2 root root   4096 Aug 21 20:41 .
+drwxr-xr-x 3 root root   4096 Aug 21 20:39 ..
+-rw-r--r-- 1 root root 284902 Aug 21 20:41 paperless-postgres.sql
+--- real unit:
+{
+  "schema_version": 2,
+  "app_name": "paperless-ngx",
+  "display_name": "Paperless-ngx",
+  "controller_version": "0.217.0",
+  "created_at": "2026-08-21T20:42:01Z",
+  "drive": "/mnt/felhom-drives/hdd_1",
+  "namespace_root": "/mnt/felhom-drives/hdd_1",
+  "image_pins": [
+    "ghcr.io/paperless-ngx/paperless-ngx:2.20.15",
+    "postgres:16-alpine",
+    "redis:7-alpine"
+  ],
+  "secret_env_vars": [
+    "DB_PASSWORD",
+    "PAPERLESS_SECRET_KEY",
+    "PAPERLESS_ADMIN_PASSWORD"
+  ],
+  "data_key_env_vars": null,
+  "secret_source": "portable secrets (data keys, DB passwords, internal signing secrets) are IN this unit's compose/app.yaml (0600); internet-reachable admin logins are NOT, and come from the guest's app.yaml or are regenerated on restore",
+  "config_files": [
+    "docker-compose.yml",
+    ".felhom.yml",
+    "app.yaml"
+  ],
+  "db_dumps": null,
+  "volume_dumps": [
+    "paperless-ngx_paperless_data.tar",
+    "paperless-ngx_paperless_postgres_data.tar",
+    "paperless-ngx_paperless_redis_data.tar"
+  ],
+  "checksums": {
+    ".felhom.yml": "a7cce0a557fd151e6385a137f4721366dd2cd0aa3876783f0f9f0acc9a78dd23",
+    "app.yaml": "8dfec452dfc00c7e3dce26864bf97165acac44f32470d2425c96e2bd86649cf0",
+    "docker-compose.yml": "b112952565ef3928f192358ea58fdf4a5a26d788593bf2614e36050c77539b3d"
+  },
+  "portable_secret_env_vars": [
+    "DB_PASSWORD",
+    "PAPERLESS_SECRET_KEY"
+  ],
+  "offsite_run_id": "20260821T204123Z",
+  "dumps_at": "2026-08-21T20:41:23Z"
+}

documentation/audits/DRILL-backup-truth-2026-08-21/evidence/phase4-part4/results.txt

@@ -0,0 +1,64 @@
+PART 4 — things we claim and have never watched. demo-hp, 2026-08-21, times CEST.
+
+4.1 THE DESTRUCTIVE RESTORE
+  (a) "nothing is ever deleted"  -- PASS, both directions, calibre-web 22:26:59.
+      POST-SNAPSHOT.txt, created after the snapshot, SURVIVED the restore.
+      plain.txt, mutated after the snapshot, was OVERWRITTEN back to the snapshot's
+      content (sha 07e91a98…). Copier is rsync -a, no --delete, no --ignore-existing
+      (offbox_reconstitute.go:94).
+      NOTE ON THE COUNT: the message says "2 fájl visszaállítva" because rsync counts
+      TRANSFERS, not files restored. An identical restore reports "0 fájl visszaállítva",
+      which is indistinguishable from a restore that did nothing.
+
+  (b) "a safety dump is taken and VERIFIED before anything is stopped, and the whole
+      operation refuses if it cannot be"
+      HAPPY PATH -- PASS, romm 23:02:44.
+        21:02:47Z  "[offbox] romm: pre-restore safety dump written →
+                    pre-restore-20260821T210246Z-romm-mariadb.sql (60.8 KB)"
+        21:02:47Z  "[stacks] StopStack romm: current state=running"
+        The dump precedes the stop. Message correctly said
+        "0 fájl és az adatbázis visszaállítva".
+      THE REFUSAL -- PASS, romm 23:04:15.
+        Safety dump made impossible by putting a regular FILE at the db-dumps path.
+        Result: ok=FALSE,
+          "A teljes visszaállítás sikertelen: a biztonsági mentés könyvtára nem hozható
+           létre: mkdir /mnt/felhom-drives/hdd_1/backups/primary/romm/db-dumps:
+           not a directory"
+        Nothing changed: plain.txt kept my post-snapshot mutation (sha 3754dfc6…), and
+        romm's container StartedAt was unchanged (21:03:08) -- the app was never stopped.
+        JUDGEMENT: honest and it names the path, but it leaks a raw Go mkdir error into
+        a customer surface.
+      THE HOLE -- the guard only protects apps whose database the discovery resolves to
+        the right stack. For paperless-ngx it concludes "no database", so hasDB is false
+        and the refusal CANNOT fire: the undo is absent rather than refused. See
+        ../phase4-paperless/FINDING.txt.
+      SIDE EFFECT, NOT PREVIOUSLY FILED -- the safety dump DESTROYS the unit's own DB dump.
+        DumpOne writes the canonical `<stack>-<dbtype>.sql` (appbackup/dbdump.go:200-202),
+        i.e. the app's real dump, and only THEN is it renamed to pre-restore-*.
+        The comment at offbox_reconstitute.go:147-148 states it "can never overwrite the
+        app's real dump". It does.
+        PROVEN: romm's db-dumps held romm-mariadb.sql (62,270 B) at 22:59; after one
+        reconstitute it held ONLY pre-restore-20260821T210246Z-romm-mariadb.sql.
+        Consequence: until the next backup run the LOCAL restore-from-unit finds no .sql
+        and tells the customer the app has no database.
+
+4.3 A DAMAGED STORE  -- MIXED
+  Method: one byte flipped inside pack 967853d2… (offset 5,000,000) via the repo's own
+  SFTP transport; the pack's name is its content hash, so this is genuine corruption.
+  * `restic check` DOES detect it ("ciphertext verification failed",
+    "Fatal: repository contains errors").  BUT the controller NEVER RUNS `restic check`:
+    the only restic verbs in the whole controller are restore, snapshots, backup, unlock,
+    stats, init, forget, prune, cat. The agent's restore-test is PBS-tier only.
+    So the off-site store is never verified by any layer, at any time.
+  * A restore that TOUCHES the damage fails honestly:
+    ok=FALSE, "A visszaállítás sikertelen: offbox restore paperless-ngx: exit status 1:
+    … ignoring error for …/documents/originals/0000011.pdf: ciphertext verification failed"
+  * BUT the failure is not remembered. It left a PARTIAL scratch (78 MB, 54 files,
+    15 of 16 originals). OffboxFullScratchReady (offbox_restore.go:305) only asks
+    "does the directory exist and is it non-empty", so the wizard then offered all three
+    actions including "Teljes visszaállítás indítása".
+  * Pressing it ran the DESTRUCTIVE restore from that known-incomplete copy and reported
+    SUCCESS: ok=TRUE, "A(z) paperless-ngx: 0 fájl visszaállítva … Ennek az alkalmazásnak
+    nincs adatbázisa."
+  REPO REPAIRED afterwards from the byte-identical originals; `restic check` now says
+  "no errors were found".

documentation/audits/DRILL-backup-truth-2026-08-21/evidence/phase5-scheduled/scheduled-cycle-and-sweep.log

@@ -0,0 +1,71 @@
+[2026-08-21 23:34:08 CEST] collector v2 started (epoch waits)
+[2026-08-22 02:38:09 CEST] === after the 02:30 SCHEDULED local cycle
+[2026-08-22 02:38:09 CEST] --- units:
+## /mnt/sys_drive/felhom-data/backups/primary/kimai
+        2235  2026-08-21 21:00  compose/.felhom.yml
+         488  2026-08-21 21:00  compose/app.yaml
+        2195  2026-08-21 21:00  compose/docker-compose.yml
+       48217  2026-08-22 00:30  db-dumps/kimai-mariadb.sql
+        1275  2026-08-21 21:00  manifest.json
+   160331776  2026-08-22 00:30  volume-dumps/kimai_kimai_db_data.tar
+    52845056  2026-08-22 00:30  volume-dumps/kimai_kimai_var.tar
+## /mnt/sys_drive/felhom-data/backups/primary/opengist
+        1750  2026-08-21 21:00  compose/.felhom.yml
+         287  2026-08-21 21:00  compose/app.yaml
+        1260  2026-08-21 21:00  compose/docker-compose.yml
+        1121  2026-08-21 21:00  manifest.json
+      181248  2026-08-22 00:30  volume-dumps/opengist_opengist_data.tar
+## /mnt/sys_drive/felhom-data/backups/primary/paperless
+      294936  2026-08-22 00:30  db-dumps/paperless-postgres.sql
+## /mnt/sys_drive/felhom-data/backups/primary/privatebin
+        1723  2026-08-21 21:00  compose/.felhom.yml
+         288  2026-08-21 21:00  compose/app.yaml
+        1223  2026-08-21 21:00  compose/docker-compose.yml
+        1118  2026-08-21 21:00  manifest.json
+     1055744  2026-08-22 00:30  volume-dumps/privatebin_privatebin_data.tar
+## /mnt/felhom-drives/hdd_1/backups/primary/calibre-web
+        3035  2026-08-21 21:00  compose/.felhom.yml
+         327  2026-08-21 21:00  compose/app.yaml
+        2122  2026-08-21 21:00  compose/docker-compose.yml
+        1165  2026-08-21 21:00  manifest.json
+      368640  2026-08-22 00:30  volume-dumps/calibre-web_calibre_web_config.tar
+## /mnt/felhom-drives/hdd_1/backups/primary/paperless-ngx
+        5971  2026-08-21 21:00  compose/.felhom.yml
+         664  2026-08-21 21:00  compose/app.yaml
+        5802  2026-08-21 21:00  compose/docker-compose.yml
+        1462  2026-08-21 21:00  manifest.json
+      231424  2026-08-22 00:30  volume-dumps/paperless-ngx_paperless_data.tar
+    71417344  2026-08-22 00:30  volume-dumps/paperless-ngx_paperless_postgres_data.tar
+      116736  2026-08-22 00:30  volume-dumps/paperless-ngx_paperless_redis_data.tar
+## /mnt/felhom-drives/hdd_1/backups/primary/romm
+        5520  2026-08-21 21:20  compose/.felhom.yml
+         576  2026-08-21 21:20  compose/app.yaml
+        4399  2026-08-21 21:20  compose/docker-compose.yml
+       62270  2026-08-21 21:02  db-dumps/pre-restore-20260821T210246Z-romm-mariadb.sql
+       62270  2026-08-22 00:30  db-dumps/romm-mariadb.sql
+        1466  2026-08-21 21:20  manifest.json
+        2560  2026-08-22 00:30  volume-dumps/romm_romm_config.tar
+   160247296  2026-08-22 00:30  volume-dumps/romm_romm_db_data.tar
+    14677504  2026-08-22 00:30  volume-dumps/romm_romm_redis_data.tar
+[2026-08-22 02:38:10 CEST] --- planted files still byte-identical (calibre-web books):
+eee5880e304b27c13d205aac9989e901a5f2393be7ecf076ee2c3a0195a2358c  DRILL-2026-08-21/POST-SNAPSHOT.txt
+0c23c8531214fe20cbc1ed177da22f51d65570c5aaedb39e380aa8d4d3e62991  DRILL-2026-08-21/SENTINEL.txt
+725763bbe679b22d5c231a14083ff13155f475c028c22ade4623955b50a2a84a  DRILL-2026-08-21/binary-1mb.bin
+a39ad6f623da67ac72f2e62a24245eef46c722c279ae89cd6b0c7c45afa6a4c4  DRILL-2026-08-21/nested/őszibarack.md
+07e91a985809fc96752f97cddcf6523b211bc607c81a5808befabe35573926f1  DRILL-2026-08-21/plain.txt
+0d6a22ec56acf61b8a80e73eb12cb223d607acaa3684acfcfa7c3e34ecdfabea  DRILL-2026-08-21/árvíztűrő-tükörfúrógép.txt
+[2026-08-22 02:38:12 CEST] --- scheduled db-dump log:
+grep: /opt/docker/felhom-controller/data/debug-ring.log: No such file or directory
+[2026-08-22 04:30:13 CEST] === after the 04:15 SCHEDULED off-site run
+{"last_duration":"2m8s","last_error":"","last_run":"2026-08-22T02:17:12Z","orphaned":false,"progress":{"active":false,"current_app":"","percent":0,"bytes_done":0,"total_bytes":0,"done_human":"","total_human":"","files_done":0,"total_files":0,"current_file":"","elapsed_sec":0,"phase":""},"repo_size_human":"42.8 MB","snapshots":27,"status":"ok"}
+
+[2026-08-22 04:30:13 CEST] --- offsite log:
+grep: /opt/docker/felhom-controller/data/debug-ring.log: No such file or directory
+[2026-08-22 05:20:15 CEST] === after the 05:10 abandonment sweep
+[2026-08-22 05:20:15 CEST] --- sweep log:
+grep: /opt/docker/felhom-controller/data/debug-ring.log: No such file or directory
+[2026-08-22 05:20:16 CEST] --- product CLI:
+[INFO] [settings] Loaded settings from /opt/docker/felhom-controller/data/settings.json
+no abandonment countdown is running on this box
+[2026-08-22 05:20:17 CEST] --- settings abandon fields:
+[2026-08-22 05:20:18 CEST] collector finished

documentation/audits/DRILL-cooldown-grain-2026-08-23/README.md

@@ -0,0 +1,118 @@
+# DRILL — R-389: only the first broken app per hour reached the operator (2026-08-23)
+
+**Hub v0.107.0 → v0.108.0. No controller release, so no golden and no floor.** Live leg on `demo-hp`
+(Tier 0) plus the hub. UNATTENDED. Method: the hub's own SQLite records plus the controller log.
+Guest and hub DB are UTC; the hub pod logs CEST.
+
+**No halt condition fired.**
+
+## The pair that says it
+
+Same box, same shape — two apps down four minutes apart, inside one hour:
+
+```
+2026-08-23 09:27:51  sent        BookStack     <- v0.107.0
+2026-08-23 09:31:51  suppressed  PrivateBin        operator cooldown 1h, key=demo-hp:app_start_failed
+
+2026-08-23 11:56:57  sent        OpenGist      <- v0.108.0
+2026-08-23 12:00:57  sent        Calibre-Web
+```
+
+**2 sent / 0 suppressed**, where the day before the identical shape gave one of each.
+
+And the keys, from the hub's own suppression rows — it records the key only when it declines:
+
+```
+operator cooldown 1h, key=demo-hp:app_start_failed:opengist
+operator cooldown 1h, key=demo-hp:app_start_failed:calibre-web
+```
+
+against v0.107.0's shared `key=demo-hp:app_start_failed`.
+
+## The fence, and why it is not decoration
+
+`crossdrive_failed` is severity `error`, reaches the operator leg, and carries `stack_name` — through
+`CrossDriveDetails`, **a different struct from `AppDetails`**. A rule of the form *"if the details
+carry a stack_name, split per app"* would have split it and silently undone R-182.
+
+Proven live: two different apps' `crossdrive_failed`, one minute apart →
+
+```
+sent        opengist
+suppressed  calibre-web    operator cooldown 1h, key=demo-hp:crossdrive_failed
+```
+
+**Byte-identical to the derived v0.107.0 key. No app suffix.** That is why `cooldownStackSuffix`
+takes the event type as well as the details, unlike its two siblings.
+
+## Part 2 — the burst, measured
+
+Three apps stopped in one scan (`kimai`, `romm`, `paperless-ngx`, none with a live cooldown — checked
+first, because a stale one would have halved the count and made the answer look better than it is):
+
+| | |
+|---|---|
+| attempted | **3** |
+| sent | **3** |
+| suppressed | **0** |
+
+**Judgement: per-app is the right grain, and this volume is acceptable.** The reference box has 8
+deployed apps, so a total outage is 8 mails; the boot grace (90 s), the quiesce grace (180 s) and the
+per-app edge trigger absorb reboots, backup cycles and persistently-dead apps. **No digest row was
+filed.** The reopening condition is stated rather than left implicit: it scales linearly with app
+count and has no ceiling, so a box large enough that a total outage is unreadable is the point at
+which the answer becomes a digest with a customer message — not a wider cooldown.
+
+## Part 3 — gate 11, and the spec discrepancy it forced
+
+The gate refuses a push whose `REPORT.md` carries an observation with neither `FILED: R-NNN` nor
+`NOT-A-FINDING: <reason>`.
+
+**The specification said an item may "cite an R-NNN that resolves". That rule would have passed the
+very item the gate was built to catch.** Yesterday's lost observation reads *"This is R-182's known
+cooldown-key shape…"* — `R-182` resolves, and it is cited as an **analogy**, not as the row that files
+it. No parser can tell citation-as-precedent from citation-as-filing by reading prose. The marker is
+therefore explicit, and the discrepancy is recorded in the gate's docstring rather than quietly
+resolved. `EDGE 7` in the evidence is that exact case, convicted.
+
+| Control | Expected | Observed |
+|---|---|---|
+| historical: yesterday's real section, verbatim | refuse | **exit 1**, naming both items |
+| plant an observation with no row | refuse | **exit 1** |
+| add the row | pass | **exit 0** |
+| remove the observation | pass quietly | **exit 0** |
+| `FILED:` a row that does not resolve | refuse | **exit 1** |
+| `NOT-A-FINDING:` with no reason | refuse | **exit 1** |
+| both markers on one item | refuse | **exit 1** |
+| section present, no numbered items | inconclusive | **exit 2**, saying what it could not read |
+| no `REPORT.md` at all | inconclusive | **exit 2** |
+| a bare `R-182` mention (the trap) | refuse | **exit 1** |
+
+## Evidence index (`evidence/`)
+
+| File | What it shows |
+|---|---|
+| `redproof-1-key.txt` | suffix dropped from the key → `1 operator mail(s), want 2`, and the live key shape reproduced |
+| `redproof-2-allowlist.txt` | allow-list removed → `crossdrive_failed`, `app_deployed`, `app_removed`, `backup_failed` all split per app |
+| `gate11-01-historical-redproof.txt` | yesterday's actual observations section, refused |
+| `gate11-02-three-controls.txt` | plant → refuse, file → pass, remove → pass |
+| `gate11-03-edges.txt` | seven boundaries incl. the R-182 trap |
+| `live-01`…`live-05` | Scenarios A and B: both sent, then each suppressed under its own key |
+| `live-06`, `live-07` | Scenario C: crossdrive stays coarse |
+| `live-08`, `live-09` | the burst, with its pre-check |
+| `live-10-full-controller-log.txt` | 1803 lines, pulled before the restore |
+
+## Teardown
+
+Nothing provisioned. Five apps were stopped across the walk (`opengist`, `calibre-web`, `kimai`,
+`romm`, `paperless-ngx`) and **all were restarted and confirmed healthy** — 17 containers up. No app
+was rebuilt, redeployed or restored; the three retained subjects (`docmost`, `bookstack`,
+`privatebin`) were not touched at all.
+
+**Hub-side, stated explicitly.** The hub was written this session: deployment to v0.108.0 via the
+manifest, and **six probe events were POSTed to the live hub** for Scenarios B and C (two
+`app_start_failed`, two `crossdrive_failed`, plus the two from yesterday's Scenario H that were
+deliberately left in place). They are inert event rows for `demo-hp` and are named here rather than
+left to be found. **Every count in this drill is filtered by `created_at >= T0`** precisely so those
+rows cannot contaminate it. Nothing else: no appliance registered, no customer created, no artifact
+manifest changed, floor untouched.

documentation/audits/DRILL-cooldown-grain-2026-08-23/evidence/gate11-01-historical-redproof.txt

@@ -0,0 +1,16 @@
+### HISTORICAL RED-PROOF — yesterday's ACTUAL observations section, verbatim from 2f7c9a6 ###
+  report            : ../../../../../tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/3b17bdbd-9113-4908-9b30-15678f2978bc/scratchpad/gate11/REPORT.md
+  section           : ## 10. Observations — recorded, not acted on
+  observation items : 2
+
+OBSERVATIONS GATE FAILED: 2 observation(s) with nothing behind them.
+
+  item 1: **The operator cooldown key carries no app identifier.** PrivateBin's alarm fo…
+     neither `FILED: R-NNN` nor `NOT-A-FINDING: <reason>`
+
+  item 2: Two probe events remain as rows for `demo-hp` from Scenario H — inert, and nam…
+     neither `FILED: R-NNN` nor `NOT-A-FINDING: <reason>`
+
+An observation that lives only in REPORT.md has a lifetime of ONE SESSION — this file is overwritten every time. That is how the cooldown-grain finding was lost on 2026-08-23 and had to be re-derived the next day.
+Fix: add `FILED: R-NNN` naming the row you opened for it, or `NOT-A-FINDING: <why this is not worth a row>`. Opening the row is the default; declaring is the exception and needs its reason stated.
+EXIT=1 (want 1)

documentation/audits/DRILL-cooldown-grain-2026-08-23/evidence/gate11-02-three-controls.txt

@@ -0,0 +1,25 @@
+### CONTROL 1 — an observation with NO row: must REFUSE (1) ###
+  report            : ../../../../../tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/3b17bdbd-9113-4908-9b30-15678f2978bc/scratchpad/gate11/c1.md
+  section           : ## 9. Observations
+  observation items : 1
+
+OBSERVATIONS GATE FAILED: 1 observation(s) with nothing behind them.
+
+  item 1: The operator cooldown key carries no app identifier, so only the first app per…
+     neither `FILED: R-NNN` nor `NOT-A-FINDING: <reason>`
+
+An observation that lives only in REPORT.md has a lifetime of ONE SESSION — this file is overwritten every time. That is how the cooldown-grain finding was lost on 2026-08-23 and had to be re-derived the next day.
+Fix: add `FILED: R-NNN` naming the row you opened for it, or `NOT-A-FINDING: <why this is not worth a row>`. Opening the row is the default; declaring is the exception and needs its reason stated.
+EXIT=1
+
+### CONTROL 2 — add the row: must PASS (0) ###
+  report            : ../../../../../tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/3b17bdbd-9113-4908-9b30-15678f2978bc/scratchpad/gate11/c2.md
+  section           : ## 9. Observations
+  observation items : 1
+      OK  1. FILED R-389
+observations gate OK — every observation is either filed or explicitly declared
+EXIT=0
+
+### CONTROL 3 — remove the observation entirely: must PASS quietly (0) — Scenario F ###
+observations gate OK — c3.md has no observations section (nothing to check)
+EXIT=0

documentation/audits/DRILL-cooldown-grain-2026-08-23/evidence/gate11-03-edges.txt

@@ -0,0 +1,44 @@
+### EDGE 1 — NOT-A-FINDING with a reason: PASS (0) ###
+  section           : ## Observations
+  observation items : 1
+      OK  1. NOT-A-FINDING
+observations gate OK — every observation is either filed or explicitly declared
+EXIT=0
+
+### EDGE 2 — FILED with a row that does NOT resolve: REFUSE (1) ###
+     `FILED: R-99999` does not resolve — no row for R-99999 in OPEN-ITEMS.md or CLOSED-ITEMS.md
+
+An observation that lives only in REPORT.md has a lifetime of ONE SESSION — this file is overwritten every time. That is how the cooldown-grain finding was lost on 2026-08-23 and had to be re-derived the next day.
+Fix: add `FILED: R-NNN` naming the row you opened for it, or `NOT-A-FINDING: <why this is not worth a row>`. Opening the row is the default; declaring is the exception and needs its reason stated.
+EXIT=1
+
+### EDGE 3 — NOT-A-FINDING with no reason: REFUSE (1) ###
+     `NOT-A-FINDING:` carries no reason — the reason is the whole point of the marker
+
+An observation that lives only in REPORT.md has a lifetime of ONE SESSION — this file is overwritten every time. That is how the cooldown-grain finding was lost on 2026-08-23 and had to be re-derived the next day.
+Fix: add `FILED: R-NNN` naming the row you opened for it, or `NOT-A-FINDING: <why this is not worth a row>`. Opening the row is the default; declaring is the exception and needs its reason stated.
+EXIT=1
+
+### EDGE 4 — BOTH markers on one item: REFUSE (1) ###
+     carries BOTH `FILED:` and `NOT-A-FINDING:` — decide which it is
+
+An observation that lives only in REPORT.md has a lifetime of ONE SESSION — this file is overwritten every time. That is how the cooldown-grain finding was lost on 2026-08-23 and had to be re-derived the next day.
+Fix: add `FILED: R-NNN` naming the row you opened for it, or `NOT-A-FINDING: <why this is not worth a row>`. Opening the row is the default; declaring is the exception and needs its reason stated.
+EXIT=1
+
+### EDGE 5 — section present, NO numbered items: INCONCLUSIVE (2) ###
+OBSERVATIONS GATE INCONCLUSIVE: found the section '## Observations' but no numbered items under it.
+  Items must be a numbered list (`1.`, `2.` …). If the section is deliberately empty, remove the heading — an empty section reads as coverage while providing none.
+EXIT=2
+
+### EDGE 6 — REPORT.md absent: INCONCLUSIVE (2) ###
+OBSERVATIONS GATE INCONCLUSIVE: no REPORT.md at /tmp/claude-1000/-mnt-5-hdd-felhom-eu-git/3b17bdbd-9113-4908-9b30-15678f2978bc/scratchpad/gate11/does-not-exist.md
+  This gate is registered here because this repo keeps one. If it was removed deliberately, unregister the gate in the runner rather than leaving it unreadable.
+EXIT=2
+
+### EDGE 7 — the R-182 TRAP: a bare mention must NOT satisfy the gate (1) ###
+     neither `FILED: R-NNN` nor `NOT-A-FINDING: <reason>`
+
+An observation that lives only in REPORT.md has a lifetime of ONE SESSION — this file is overwritten every time. That is how the cooldown-grain finding was lost on 2026-08-23 and had to be re-derived the next day.
+Fix: add `FILED: R-NNN` naming the row you opened for it, or `NOT-A-FINDING: <why this is not worth a row>`. Opening the row is the default; declaring is the exception and needs its reason stated.
+EXIT=1

documentation/audits/DRILL-cooldown-grain-2026-08-23/evidence/live-00-prestate.txt

@@ -0,0 +1,16 @@
+=== LIVE WALK — pre-state ===
+hub image: gitea.dooplex.hu/admin/felhom-hub:0.108.0
+2026-08-23T11:56:06Z
+controller: gitea.dooplex.hu/admin/felhom-controller:0.223.0 Up 2 hours (healthy)
+
+bookstack	Up 9 hours (healthy)
+bookstack-db	Up 2 hours (healthy)
+calibre-web	Up 9 hours (healthy)
+docmost	Up 6 hours (healthy)
+docmost-postgres	Up 6 hours (healthy)
+docmost-redis	Up 6 hours (healthy)
+opengist	Up 9 hours (healthy)
+privatebin	Up 2 hours (healthy)
+
+-- T0 for every count below: any notification_log row created at or after this instant --
+T0=2026-08-23T11:56:06Z

documentation/audits/DRILL-cooldown-grain-2026-08-23/evidence/live-01-scenarioA-app1.txt

@@ -0,0 +1,8 @@
+=== SCENARIO A — two DIFFERENT apps down inside the hour ===
+subjects: opengist and calibre-web (single-container, intent running, neither on the
+do-not-rebuild list). Stopped OUT OF BAND so the controller's own Deploying skip cannot mask it.
+  opengist     desired_state: running
+  calibre-web  desired_state: running
+
+APP 1 (opengist) down at: 2026-08-23T11:56:26Z
+opengist	Exited (0) 5 seconds ago

documentation/audits/DRILL-cooldown-grain-2026-08-23/evidence/live-02-scenarioA-app2.txt

@@ -0,0 +1,8 @@
+-- waiting 4 minutes, then APP 2 — the same gap that produced one mail yesterday --
+APP 2 (calibre-web) down at: 2026-08-23T12:00:48Z
+opengist	Exited (0) 4 minutes ago
+calibre-web	Exited (0) 5 seconds ago
+
+-- letting the second alarm land --
+2026/08/23 11:56:56 notifier.go:234: [INFO] Event pushed: app_start_failed (warning) — Telepített alkalmazás nem fut: OpenGist
+2026/08/23 12:00:57 notifier.go:234: [INFO] Event pushed: app_start_failed (warning) — Telepített alkalmazás nem fut: Calibre-Web Automated

documentation/audits/DRILL-cooldown-grain-2026-08-23/evidence/live-03-scenarioA-hub-rows.txt

@@ -0,0 +1,11 @@
+=== SCENARIO A — the hub's notification log, filtered to created_at >= T0 (11:56:06Z) ===
+(the T0 filter is what keeps yesterday's two inert Scenario H probe rows out of every count)
+
+status  channel   message                                       key_or_reason  created_at         
+------  --------  --------------------------------------------  -------------  -------------------
+sent    operator  Telepített alkalmazás nem fut: OpenGist                      2026-08-23 11:56:57
+sent    operator  Telepített alkalmazás nem fut: Calibre-Web A                 2026-08-23 12:00:57
+
+-- counts --
+sent      : 2
+suppressed: 0

documentation/audits/DRILL-cooldown-grain-2026-08-23/evidence/live-04-scenarioB-repeat.txt

@@ -0,0 +1,8 @@
+=== SCENARIO B — the SAME app again inside the hour: one sent, one suppressed ===
+Posted through the exact endpoint the controller invokes (/api/v1/event), with the controller's
+own AppDetails payload. The controller's own event is edge-triggered per app, so a down->down
+cycle is silent by design and cannot re-fire it from the box.
+
+  T_B=2026-08-23T12:02:48Z
+  repeat OpenGist   http=200
+  repeat Calibre-Web http=200

documentation/audits/DRILL-cooldown-grain-2026-08-23/evidence/live-05-scenarioAB-keys.txt

@@ -0,0 +1,15 @@
+=== SCENARIOS A + B — every operator row since T0, with the KEY where the hub records it ===
+status      app                     key_or_reason                                                   created_at         
+----------  ----------------------  --------------------------------------------------------------  -------------------
+sent        ut: OpenGist                                                                            2026-08-23 11:56:57
+sent        ut: Calibre-Web Automa                                                                  2026-08-23 12:00:57
+suppressed  ut: OpenGist            operator cooldown 1h, key=demo-hp:app_start_failed:opengist     2026-08-23 12:02:48
+suppressed  ut: Calibre-Web         operator cooldown 1h, key=demo-hp:app_start_failed:calibre-web  2026-08-23 12:02:48
+
+-- THE KEYS, side by side. Yesterday BOTH apps shared key=demo-hp:app_start_failed. --
+  operator cooldown 1h, key=demo-hp:app_start_failed:opengist
+  operator cooldown 1h, key=demo-hp:app_start_failed:calibre-web
+
+-- counts since T0 --
+  sent      : 2
+  suppressed: 2

documentation/audits/DRILL-cooldown-grain-2026-08-23/evidence/live-06-scenarioC-crossdrive.txt

@@ -0,0 +1,15 @@
+=== SCENARIO C — a backup-family event carrying stack_name must NOT split per app ===
+subject: crossdrive_failed — severity error, reaches the operator leg, carries stack_name
+through CrossDriveDetails. Two DIFFERENT apps: if the key had split, both would send.
+
+HOW THE 'BEFORE' VALUE WAS OBTAINED, two independent ways:
+  (a) derivation from v0.107.0's expression, which is customerID:eventType + tier + run and
+      carries no app term, so for this payload it is exactly 'demo-hp:crossdrive_failed';
+  (b) TestR389_NoOtherEventTypeKeyChanges models that v0.107.0 expression INLINE and compares
+      it against the live one for 12 event types — with a positive control proving the test
+      can see a key change before it reports that none occurred.
+The live key printed below must equal that.
+
+  T_C=2026-08-23T12:03:44Z
+  crossdrive_failed opengist http=200
+  crossdrive_failed calibre-web http=200

documentation/audits/DRILL-cooldown-grain-2026-08-23/evidence/live-07-scenarioC-result.txt

@@ -0,0 +1,8 @@
+-- SCENARIO C result: the operator rows for crossdrive_failed since T_C --
+status      app         key_or_reason                                        created_at         
+----------  ----------  ---------------------------------------------------  -------------------
+suppressed  alibre-web  operator cooldown 1h, key=demo-hp:crossdrive_failed  2026-08-23 12:03:44
+sent        pengist                                                          2026-08-23 12:03:45
+
+-- the live key, to compare against the derived v0.107.0 value 'demo-hp:crossdrive_failed' --
+  operator cooldown 1h, key=demo-hp:crossdrive_failed

documentation/audits/DRILL-cooldown-grain-2026-08-23/evidence/live-08-burst-setup.txt

@@ -0,0 +1,21 @@
+-- restoring the two Scenario A/B subjects before the burst --
+opengist	Up 20 seconds (healthy)
+calibre-web	Up 20 seconds (healthy)
+
+=== PART 2 — THE BURST: three apps down in ONE scan ===
+subjects: kimai, romm, paperless-ngx — none has alarmed inside the last hour, so no
+pre-existing cooldown can mask the count. Verified below before stopping them.
+-- CONTROL: no app_start_failed row for these three in the last hour (a stale cooldown would
+   silently halve the count and the burst would look better than it is) --
+  rows for kimai/romm/paperless since 11:04: 0
+
+  T_BURST=2026-08-23T12:05:03Z
+2026-08-23T12:05:13Z
+kimai	Exited (0) 9 seconds ago
+kimai-db	Exited (0) 8 seconds ago
+paperless-postgres	Exited (0) 4 seconds ago
+paperless-redis	Exited (0) 4 seconds ago
+paperless-webserver	Exited (0) 4 seconds ago
+romm	Exited (0) 9 seconds ago
+romm-db	Exited (0) 9 seconds ago
+romm-redis	Exited (0) 9 seconds ago

documentation/audits/DRILL-cooldown-grain-2026-08-23/evidence/live-09-burst-result.txt

@@ -0,0 +1,18 @@
+-- PART 2 — THE BURST RESULT (T_BURST = 12:05:03Z) --
+
+controller side, what it SENT:
+2026/08/23 12:05:26 notifier.go:234: [INFO] Event pushed: app_start_failed (warning) — Telepített alkalmazás nem fut: Kimai
+2026/08/23 12:05:26 notifier.go:234: [INFO] Event pushed: app_start_failed (warning) — Telepített alkalmazás nem fut: RomM
+2026/08/23 12:05:26 notifier.go:234: [INFO] Event pushed: app_start_failed (warning) — Telepített alkalmazás nem fut: Paperless-ngx
+
+hub side, the operator rows:
+status  app                key_or_reason  created_at         
+------  -----------------  -------------  -------------------
+sent    ut: RomM                          2026-08-23 12:05:27
+sent    ut: Kimai                         2026-08-23 12:05:27
+sent    ut: Paperless-ngx                 2026-08-23 12:05:27
+
+THE THREE COUNTS:
+  attempted (rows written): 3
+  sent                    : 3
+  suppressed              : 0

documentation/audits/DRILL-cooldown-grain-2026-08-23/evidence/live-11-restore.txt

@@ -0,0 +1,19 @@
+-- restoring the three burst subjects --
+2026-08-23T12:08:39Z
+bookstack	Up 10 hours (healthy)
+bookstack-db	Up 3 hours (healthy)
+calibre-web	Up 4 minutes (healthy)
+docmost	Up 6 hours (healthy)
+docmost-postgres	Up 6 hours (healthy)
+docmost-redis	Up 6 hours (healthy)
+filebrowser	Up 44 hours (healthy)
+kimai	Up 52 seconds (healthy)
+kimai-db	Up About a minute (healthy)
+opengist	Up 4 minutes (healthy)
+paperless-postgres	Up 40 seconds (healthy)
+paperless-redis	Up 40 seconds (healthy)
+paperless-webserver	Up 30 seconds (health: starting)
+privatebin	Up 2 hours (healthy)
+romm	Up 41 seconds (healthy)
+romm-db	Up 51 seconds (healthy)
+romm-redis	Up 51 seconds (healthy)

documentation/audits/DRILL-cooldown-grain-2026-08-23/evidence/redproof-1-key.txt

@@ -0,0 +1,9 @@
+### RED-PROOF 1 (R-389) — mutation: cooldownStackSuffix removed from the key ###
+### layer: processOperator's KEY — where the collapse happens ###
+--- FAIL: TestR389_TwoAppsInsideTheHourBothReachTheOperator (0.02s)
+    r389_cooldown_grain_test.go:180: 2 apps down inside the hour produced 1 operator mail(s), want 2 (suppressed=1). This is R-389: the second app's alarm took the first app's cooldown slot.
+--- FAIL: TestR389_SameAppTwiceInsideTheHourIsStillSuppressed (0.02s)
+    r389_cooldown_grain_test.go:220: the suppression row does not name the app in its key — that visibility is R-182's contribution and is how this defect was found: [sent | Telepített alkalmazás nem fut: BookStack |  suppressed | Telepített alkalmazás nem fut: BookStack | operator cooldown 1h, key=c1:app_start_failed]
+FAIL
+FAIL	gitea.dooplex.hu/admin/felhom-hub/internal/notify	0.166s
+FAIL

documentation/audits/DRILL-cooldown-grain-2026-08-23/evidence/redproof-2-allowlist.txt

@@ -0,0 +1,12 @@
+### RED-PROOF 2 (R-389) — mutation: allow-list removed, suffix applied globally ###
+### layer: cooldownStackSuffix's REGISTER — the fence that keeps the backup family coarse ###
+--- FAIL: TestR389_StackSuffixIsAllowListedAndFailSoft (0.00s)
+    --- FAIL: TestR389_StackSuffixIsAllowListedAndFailSoft/crossdrive_failed_is_NOT_split_per_app (0.00s)
+        r389_cooldown_grain_test.go:58: cooldownStackSuffix("crossdrive_failed", "{\"stack_name\":\"bookstack\",\"method\":\"rsync\"}") = ":bookstack", want ""
+    --- FAIL: TestR389_StackSuffixIsAllowListedAndFailSoft/app_deployed_is_not_in_the_register (0.00s)
+        r389_cooldown_grain_test.go:58: cooldownStackSuffix("app_deployed", "{\"stack_name\":\"bookstack\",\"display_name\":\"BookStack\"}") = ":bookstack", want ""
+    --- FAIL: TestR389_StackSuffixIsAllowListedAndFailSoft/app_removed_is_not_in_the_register (0.00s)
+        r389_cooldown_grain_test.go:58: cooldownStackSuffix("app_removed", "{\"stack_name\":\"bookstack\",\"display_name\":\"BookStack\"}") = ":bookstack", want ""
+    --- FAIL: TestR389_StackSuffixIsAllowListedAndFailSoft/backup_failed_is_not_in_the_register (0.00s)
+        r389_cooldown_grain_test.go:58: cooldownStackSuffix("backup_failed", "{\"stack_name\":\"bookstack\",\"display_name\":\"BookStack\"}") = ":bookstack", want ""
+--- FAIL: TestR389_NoOtherEventTypeKeyChanges (0.00s)

documentation/audits/DRILL-r329-r386-2026-08-23/README.md

@@ -0,0 +1,110 @@
+# DRILL — R-329 + R-386: the alarm that fired but reached nobody, and the stop nobody heard (2026-08-23)
+
+**Controller v0.222.0 → v0.223.0. Hub v0.106.0 → v0.107.0. Live leg on `demo-hp` (Tier 0), guest
+9201. UNATTENDED.** Method: endpoint-level plus the hub's own SQLite records — no browser exists on
+DooPlex. Guest and hub clocks are UTC; the hub pod logs CEST.
+
+## Verdict
+
+| Part | Outcome |
+|---|---|
+| 1.1 the severity word + sweep | ✅ **exactly one** bad severity in the whole controller |
+| 1.2 the AST contract guard | ✅ and it found two dynamic sites the hand sweep missed |
+| 1.3 customer toggle, default OFF | ✅ |
+| 2 the hub says what it rewrote | ✅ hub v0.107.0, proven live |
+| 3 ask the field that knows | ✅ proven live, both directions |
+| 4 split the compound toggles | ✅ round trip byte-identical |
+| 5 record the notification philosophy | ✅ recorded, **not implemented** |
+
+**No halt condition fired.**
+
+## The one number that says it all
+
+Read from the live hub DB:
+
+```
+91 app_start_failed events stored, all-time
+ 0 notification_log rows before 2026-08-23 09:00   <- not one, ever, on any channel
+```
+
+After the fix, at 09:27:51: **one row — `warning` / `sent` / `operator`.**
+
+## The two live pairs
+
+**Scenario A** — a database dies, customer has not opted in:
+
+```
+events         : demo-hp  app_start_failed  warning  2026-08-23 09:27:51   <- v0.223.0
+                 demo-hp  app_start_failed  info     2026-08-23 05:30:14   <- v0.222.0, coerced
+notification_log: demo-hp  app_start_failed  warning  sent  operator  09:27:51
+                 (no customer row)
+```
+
+**Scenario D** — an app stopped out of band, intent `running`:
+
+```
+2026/08/23 05:47:44  [deadapp] check alive: 40 scans since boot, 8 deployed app(s) evaluated, 0 currently down   <- v0.222.0
+2026/08/23 09:34:51  [deadapp] check alive: 20 scans since boot, 8 deployed app(s) evaluated, 1 currently down   <- v0.223.0
+```
+
+Alarm fired **24 seconds** after the `docker compose stop`.
+
+## Three things that had to be re-run, and why that matters
+
+1. **Red-proof 5 passed first time — the mutation was INERT.** Changing `if next <= prev` to
+   `if next < prev` in fillwatch does nothing, because an earlier `if next == prev { continue }` had
+   already removed the equal case. The test was right to pass. Removing the guard outright convicts
+   it. **Check the mutation applied before believing either verdict.**
+2. **Scenario G was silently refused TWICE behind an HTTP 200.** The empty-email wipe guard declines
+   the save and renders an error page — still `200`. The first run had no prefs at all; the second
+   read the email with a single-line grep from a `<input>` that spans **three lines**, got `""`, and
+   was refused again. Both times the before/after hashes matched — *because nothing was saved*, not
+   because nothing changed. Fixed by asserting the refusal banner is **absent**. **A warning beside a
+   success is read as a success.**
+3. **The live Scenario A does NOT prove the customer gate**, and is not claimed to. `demo-hp` has no
+   `customer_notifications` row at all, so the customer leg could not have delivered regardless. The
+   toggle gate is proven by the unit tests, which configure prefs both ways. Stated rather than
+   implied.
+
+## Evidence index (`evidence/`)
+
+| File | What it shows |
+|---|---|
+| `redproof-1-R329-emitter.txt` | severity back to `"warn"` → AST guard names file, line and value |
+| `redproof-2-R386-intent.txt` | intent test reverted → `dead-app banner = []`, the live symptom |
+| `redproof-3-part4-migration.txt` | no-op-save guard removed → the `defaults` case reorders |
+| `redproof-4-R387-ingest.txt` | hub WARN removed → "the hub rewrote a severity and said nothing" |
+| `redproof-5-fillwatch-consequence.txt` | the inert first attempt, and the effective one |
+| `live-01`…`live-04` | Scenario A: stop, controller send, **hub records**, customer-leg control |
+| `live-05-step4-absent-intent-count.txt` | **0 of 8** deployed apps carry an absent intent |
+| `live-07`…`live-09` | Scenario D: out-of-band stop, alarm in 24 s, the heartbeat pair |
+| `live-10`, `live-11` | Scenario C: UI Stop → intent `running`→`stopped`, 0 alarms / 9 scans |
+| `live-12`, `live-13` | Scenario E: intent removed → suppressed **and** the log line names the app |
+| `live-16-scenarioG-roundtrip.txt` | all three attempts, ending byte-identical (`10840f3a…`) |
+| `live-18-golden-bake.txt` | golden 0.223.0 markers, each counted |
+| `live-19-scenarioH-after.txt` | the new hub WARN line, live, with a silent `error` control |
+| `live-06`, `live-14` | full controller-log windows (1041 and 4044 lines), pulled before each revert |
+
+## Observations — noticed, recorded, NOT acted on
+
+1. **The operator cooldown has no app identifier, and it now bites.** PrivateBin's event 4 minutes
+   after BookStack's was logged `suppressed — operator cooldown 1h, key=demo-hp:app_start_failed`. So
+   **only the first app-down per hour e-mails the operator.** This is R-182's known cooldown-key
+   shape; it was harmless while the event was undeliverable and is not any more. Recorded, not fixed.
+2. **The prompt's §6 premise was wrong and is corrected:** the hub's manifest **is** in version
+   control, at `felhom.eu/manifests/hub.yaml:128`, and ArgoCD app `felhom` tracks
+   `admin/felhom.eu.git` path `manifests`. No out-of-git deployment path exists.
+3. **The golden-bake runbook still lacks `pveam update`** — second bake in a row to hit the stale
+   index on the `virgin` snapshot, presenting as `400 … no such template`.
+4. `internal/notify/notifier.go` carries **pre-existing** gofmt drift in a const block, confirmed by
+   stashing this session's work. Not touched (§12).
+
+## Teardown
+
+Nothing provisioned. Every app restarted and confirmed healthy (17 containers). `privatebin`'s
+`app.yaml` restored from its backup and the backup removed; its intent reads `running` again.
+`demo-hp`'s notification settings restored to `enabled_events: null`, no e-mail — the state they were
+in before the drill. The drill VM is powered off, its Gitea token shredded, and `drill.qcow2` reverted
+to `virgin`. **Hub-side: two probe events (`backup_failed`, "R-387 scenario H probe"/"control") were
+POSTed to the live hub for Scenario H and remain as event rows for customer `demo-hp`.** They are
+inert records; named here rather than left for someone to find.

documentation/audits/DRILL-r329-r386-2026-08-23/evidence/live-00-prestate.txt

@@ -0,0 +1,10 @@
+=== LIVE WALK PRE-STATE (controller v0.223.0) ===
+2026-08-23T09:25:42Z
+gitea.dooplex.hu/admin/felhom-controller:0.223.0 Up About a minute (healthy)
+
+docmost	Up 4 hours (healthy)
+docmost-postgres	Up 4 hours (healthy)
+docmost-redis	Up 4 hours (healthy)
+privatebin	Up 4 hours (healthy)
+bookstack	Up 7 hours (healthy)
+bookstack-db	Up 4 hours (healthy)

documentation/audits/DRILL-r329-r386-2026-08-23/evidence/live-01-scenarioA-stop.txt

@@ -0,0 +1,6 @@
+=== SCENARIO A — an app's database dies; customer has NOT opted in ===
+waiting out the 90s dead-app boot grace first...
+T0=2026-08-23T09:27:27Z
+2026-08-23T09:27:32Z
+bookstack	Up 7 hours (healthy)
+bookstack-db	Exited (0) 5 seconds ago

documentation/audits/DRILL-r329-r386-2026-08-23/evidence/live-02-scenarioA-controller-sent.txt

@@ -0,0 +1,4 @@
+-- the CONTROLLER only proves it SENT (quoted for the severity word) --
+2026/08/23 09:27:51 notifier.go:206: [DEBUG] PushEvent: type=app_start_failed severity=warning url=https://hub.felhom.eu/api/v1/event
+2026/08/23 09:27:51 notifier.go:232: [DEBUG] PushEvent: app_start_failed pushed OK (HTTP 200)
+2026/08/23 09:27:51 notifier.go:234: [INFO] Event pushed: app_start_failed (warning) — Telepített alkalmazás nem fut: BookStack

documentation/audits/DRILL-r329-r386-2026-08-23/evidence/live-03-scenarioA-hub-records.txt

@@ -0,0 +1,14 @@
+=== SCENARIO A — THE HUB'S OWN RECORDS (not the controller's) ===
+
+-- 1. the STORED event: severity as the hub filed it --
+customer_id  event_type        severity  message                                   created_at         
+-----------  ----------------  --------  ----------------------------------------  -------------------
+demo-hp      app_start_failed  warning   Telepített alkalmazás nem fut: BookStack  2026-08-23 09:27:51
+demo-hp      app_start_failed  info      Telepített alkalmazás nem fut: BookStack  2026-08-23 05:30:14
+demo-hp      app_start_failed  info      Telepített alkalmazás nem fut: BookStack  2026-08-22 21:27:18
+demo-hp      app_start_failed  info      Telepített alkalmazás nem fut: Kimai      2026-08-22 14:19:12
+
+-- 2. the NOTIFICATION LOG: which channel actually delivered --
+customer_id  event_type        severity  status  channel   created_at         
+-----------  ----------------  --------  ------  --------  -------------------
+demo-hp      app_start_failed  warning   sent    operator  2026-08-23 09:27:51

documentation/audits/DRILL-r329-r386-2026-08-23/evidence/live-04-scenarioA-customer-leg-control.txt

@@ -0,0 +1,26 @@
+-- 3. IS THE CUSTOMER LEG EVEN CAPABLE? (or is 'no customer row' true for the wrong reason) --
+Error: in prepare, no such table: notification_prefs
+
+-- the customer leg DOES deliver other event types for this same customer: --
+event_type           severity  status   channel   created_at         
+-------------------  --------  -------  --------  -------------------
+backup_run_failures  error     skipped  customer  2026-08-22 22:20:32
+backup_run_failures  error     skipped  customer  2026-08-22 22:17:44
+backup_run_failures  error     skipped  customer  2026-08-22 21:11:19
+backup_run_failures  error     skipped  customer  2026-08-22 16:06:21
+backup_run_failures  error     skipped  customer  2026-08-21 21:20:36
+
+-- 4. HOW MANY notification rows did app_start_failed EVER produce before today? --
+0 rows before 2026-08-23 09:00
+91 app_start_failed EVENTS stored, all-time
+-- 3b. the customer's own preferences (does app_start_failed appear?) --
+-- 3b. the customer's own preferences (does app_start_failed appear?) --
+-- HONESTY CHECK: demo-hp has NO customer_notifications row at all --
+0 prefs rows for demo-hp
+
+-- why the customer channel logs 'skipped' for other types: --
+event_type           status   reason       
+-------------------  -------  -------------
+backup_run_failures  skipped  operator_only
+backup_run_failures  skipped  operator_only
+backup_run_failures  skipped  operator_only

documentation/audits/DRILL-r329-r386-2026-08-23/evidence/live-05-step4-absent-intent-count.txt

@@ -0,0 +1,11 @@
+=== STEP 4 — THE ABSENT-INTENT COUNT ON demo-hp (deployed apps only) ===
+  bookstack          running
+  calibre-web        running
+  docmost            running
+  kimai              running
+  opengist           running
+  paperless-ngx      running
+  privatebin         running
+  romm               running
+
+DEPLOYED=8  running=8  stopped=0  ABSENT=0

documentation/audits/DRILL-r329-r386-2026-08-23/evidence/live-07-scenarioD-stop.txt

@@ -0,0 +1,8 @@
+=== SCENARIO D — an app stopped OUT OF BAND, intent recorded as Running ===
+subject: privatebin (single container, desired_state: running) — R-386's measured-silent case
+desired_state: running
+T0=2026-08-23T09:31:27Z
+2026-08-23T09:31:33Z
+privatebin	Exited (0) 5 seconds ago
+bookstack	Up 7 hours (healthy)
+bookstack-db	Up 10 seconds (healthy)

documentation/audits/DRILL-r329-r386-2026-08-23/evidence/live-08-scenarioD-verdict.txt

@@ -0,0 +1,6 @@
+-- SCENARIO D verdict (T0=09:31:27Z) --
+2026-08-23T09:32:41Z
+privatebin	Exited (0) About a minute ago
+
+app_start_failed events since T0: 1
+2026/08/23 09:31:51 notifier.go:234: [INFO] Event pushed: app_start_failed (warning) — Telepített alkalmazás nem fut: PrivateBin

documentation/audits/DRILL-r329-r386-2026-08-23/evidence/live-09-scenarioD-heartbeat.txt

@@ -0,0 +1,17 @@
+-- SCENARIO D: the heartbeat, new beside last session's --
+LAST SESSION (v0.222.0, R-386 open, privatebin stopped out of band):
+  2026/08/23 05:5x  [deadapp] check alive: ... 8 deployed app(s) evaluated, 0 currently down
+  (9 scans, 0 events, 0 banner lines — evidence: DRILL-r384-.../live-18-sec4-verdict.txt)
+
+NOW (v0.223.0):
+2026/08/23 09:34:51 main.go:1731: [INFO] [deadapp] check alive: 20 scans since boot, 8 deployed app(s) evaluated, 1 currently down
+
+THE PAIR, exactly as logged — same box, same fixture (privatebin stopped out of band), same 8 apps:
+  2026/08/23 05:47:44  [deadapp] check alive: 40 scans since boot, 8 deployed app(s) evaluated, 0 currently down   <- v0.222.0
+  2026/08/23 09:34:51  [deadapp] check alive: 20 scans since boot, 8 deployed app(s) evaluated, 1 currently down   <- v0.223.0
+
+-- and the hub's own record for it --
+event_type        severity  status      channel   message                                    created_at         
+----------------  --------  ----------  --------  -----------------------------------------  -------------------
+app_start_failed  warning   suppressed  operator  Telepített alkalmazás nem fut: PrivateBin  2026-08-23 09:31:51
+app_start_failed  warning   sent        operator  Telepített alkalmazás nem fut: BookStack   2026-08-23 09:27:51

documentation/audits/DRILL-r329-r386-2026-08-23/evidence/live-10-scenarioC-ui-stop.txt

@@ -0,0 +1,11 @@
+=== SCENARIO C — the customer presses Stop in the interface ===
+-- intent BEFORE --
+desired_state: running
+-- restart it first, so the Stop is a real transition --
+{"ok":true,"message":"Stack privatebin start completed"}
+ start=200
+T_STOP=2026-08-23T09:36:13Z
+{"ok":true,"message":"Stack privatebin stop completed"}
+ stop=200
+-- intent AFTER --
+desired_state: stopped

documentation/audits/DRILL-r329-r386-2026-08-23/evidence/live-11-scenarioC-verdict.txt

@@ -0,0 +1,7 @@
+-- SCENARIO C verdict: 4 minutes past the Stop (T=09:36:13Z), every grace elapsed --
+2026-08-23T09:40:37Z
+
+app_start_failed since the Stop : 0
+deadapp scans since the Stop    : 9
+unknown-intent log lines        : 0
+

documentation/audits/DRILL-r329-r386-2026-08-23/evidence/live-12-scenarioE-setup.txt

@@ -0,0 +1,7 @@
+=== SCENARIO E — an app with NO recorded intent (the legacy/upgrade population) ===
+demo-hp has ZERO such apps (all 8 read 'running'), so one is CREATED for the measurement:
+the desired_state key is removed from privatebin's app.yaml, exactly as a box upgraded from
+before R-166 would look. Backed up and restored afterwards. No code writer was added.
+desired_state line now: [0 found]
+T0=2026-08-23T09:40:55Z
+Up 30 seconds (healthy)

documentation/audits/DRILL-r329-r386-2026-08-23/evidence/live-13-scenarioE-verdict.txt

@@ -0,0 +1,11 @@
+-- SCENARIO E verdict --
+2026-08-23T09:51:48Z
+app_start_failed events : 0
+deadapp scans           : 21
+
+-- is privatebin actually being EVALUATED? (a suppression nobody reaches proves nothing) --
+privatebin deployed=True state=stopped
+
+-- waiting for the heartbeat scan (deadAppScans lags the log count by the boot grace) --
+2026/08/23 09:51:56 main.go:1731: [INFO] [deadapp] check alive: 20 scans since boot, 8 deployed app(s) evaluated, 0 currently down
+2026/08/23 09:51:56 main.go:1760: [INFO] [deadapp] 1 stopped app(s) have NO recorded customer intent, so their dead-app alarm is suppressed by the unknown-intent fallback (R-386): privatebin. This closes itself as each app is started or stopped through the interface.

documentation/audits/DRILL-r329-r386-2026-08-23/evidence/live-15-scenarioG-before.txt

@@ -0,0 +1,4 @@
+=== SCENARIO G — open the settings page and Save, changing NOTHING ===
+-- stored BEFORE --
+/var/lib/felhom/docker/volumes/felhom-controller-data/_data/data/settings.json
+/var/lib/docker/volumes/felhom-controller-data/_data/data/settings.json

documentation/audits/DRILL-r329-r386-2026-08-23/evidence/live-16-scenarioG-roundtrip.txt

@@ -0,0 +1,46 @@
+=== SCENARIO G — open the settings page and Save, changing NOTHING ===
+-- stored BEFORE (sha256 + value) --
+  enabled_events: null
+  sha256        : 74234e98afe7498fb5daf1f36ac2d78a
+  email set     : False
+
+-- RENDER the page, take exactly what it ticked, POST it back unchanged --
+  render=200
+  ticked boxes  : 10
+  save=200
+
+-- stored AFTER --
+  enabled_events: null
+  sha256        : 74234e98afe7498fb5daf1f36ac2d78a
+=== SCENARIO G — REDONE ===
+First attempt was INCONCLUSIVE and is reported: demo-hp has NO notification prefs, so the
+page rendered the DEFAULTS ticked and the save was refused by the empty-email wipe guard.
+Stored bytes were unchanged — but for the wrong reason. Seeding prefs makes it a real round trip.
+
+  seed=200
+  BEFORE: ["backup_failed", "db_dump_failed", "node_down", "disk_warning", "disk_critical", "expected_backup_missed", "expected_dbdump_missed"]
+  sha   : 10840f3a95bac168f0d7c79760998138
+
+  -- now RENDER and SAVE, changing nothing --
+  render=200
+  ticked: 7  email=
+  save=200
+  AFTER : ["backup_failed", "db_dump_failed", "node_down", "disk_warning", "disk_critical", "expected_backup_missed", "expected_dbdump_missed"]
+  sha   : 10840f3a95bac168f0d7c79760998138
+=== SCENARIO G — THIRD attempt, and the first CONCLUSIVE one ===
+Attempt 2 also refused: the email <input> spans THREE LINES, so a single-line grep read it as
+empty and the wipe guard declined the save — while still answering 200. A warning beside a
+success reads as a success; the sha matching meant NOTHING was saved, not that nothing changed.
+
+  BEFORE: ["backup_failed", "db_dump_failed", "node_down", "disk_warning", "disk_critical", "expected_backup_missed", "expected_dbdump_missed"]
+  sha   : 10840f3a95bac168f0d7c79760998138
+  render=200
+  email read : [drill@felhom.eu]   ticked: 7
+  HTTP=200
+  refusal banner present? : 0  (0 = a REAL save)
+  AFTER : ["backup_failed", "db_dump_failed", "node_down", "disk_warning", "disk_critical", "expected_backup_missed", "expected_dbdump_missed"]
+  sha   : 10840f3a95bac168f0d7c79760998138
+-- restoring demo-hp's notification settings to their pre-drill state (no email, no events) --
+  restore=200
+  enabled_events: null
+  email set     : False

documentation/audits/DRILL-r329-r386-2026-08-23/evidence/live-17-scenarioH-before.txt

@@ -0,0 +1,6 @@
+=== SCENARIO H — the hub receives an unknown severity ===
+NOTE: the LIVE hub is still v0.106.0 (the manifest bump is not synced yet), so this run is the
+BEFORE picture. It is repeated after the hub deploy.
+
+-- hub version now --
+gitea.dooplex.hu/admin/felhom-hub:0.106.0

documentation/audits/DRILL-r329-r386-2026-08-23/evidence/live-18-golden-bake.txt

@@ -0,0 +1,11 @@
+=== GOLDEN 0.223.0 — acceptance markers, each counted ===
+docker OK (overlay2   : 1
+including mount point : 2
+upload OK (HTTP 201)  : 1
+excluding (must be 0) : 0
+FATAL     (must be 0) : 0
+  docker OK (overlay2; data-root /var/lib/docker)
+GOLDEN_VERSION=0.223.0
+GOLDEN_SHA256=9eaf39ac39219b42ec9e6cbf890275febcdcc6f53325fe0c0f591d3431044f17
+-- round trip from the published URL --
+  ranged GET http=206

documentation/audits/DRILL-r329-r386-2026-08-23/evidence/live-19-scenarioH-after.txt

@@ -0,0 +1,9 @@
+=== SCENARIO H — LIVE hub v0.107.0, bad severity ===
+  api key length: 64
+  POST /event (severity=warn) http=200
+  POST /event (severity=error, control) http=200
+
+-- THE NEW LINE in the hub's own log --
+2026/08/23 11:59:07 [WARN] [api] Event from demo-hp: severity "warn" is not in {info,warning,error,critical} — coercing to "info", which severityNotifies DROPS, so this backup_failed alert will reach NOBODY. Fix the emitting controller; this event is stored but not routed.
+2026/08/23 11:59:07 [INFO] Event from demo-hp: backup_failed (info) — R-387 scenario H probe
+2026/08/23 11:59:07 [INFO] Event from demo-hp: backup_failed (error) — R-387 scenario H control

documentation/audits/DRILL-r329-r386-2026-08-23/evidence/live-20-endstate.txt

@@ -0,0 +1,30 @@
+=== END STATE — every app healthy, planted data untouched ===
+2026-08-23T09:59:32Z
+controller: gitea.dooplex.hu/admin/felhom-controller:0.223.0 Up 18 minutes (healthy)
+
+bookstack	Up 7 hours (healthy)
+bookstack-db	Up 28 minutes (healthy)
+calibre-web	Up 7 hours (healthy)
+docmost	Up 4 hours (healthy)
+docmost-postgres	Up 4 hours (healthy)
+docmost-redis	Up 4 hours (healthy)
+filebrowser	Up 42 hours (healthy)
+kimai	Up 7 hours (healthy)
+kimai-db	Up 7 hours (healthy)
+opengist	Up 7 hours (healthy)
+paperless-postgres	Up 7 hours (healthy)
+paperless-redis	Up 7 hours (healthy)
+paperless-webserver	Up 7 hours (healthy)
+privatebin	Up 7 minutes (healthy)
+romm	Up 7 hours (healthy)
+romm-db	Up 7 hours (healthy)
+romm-redis	Up 7 hours (healthy)
+traefik	Up 42 hours
+
+-- intents restored --
+  privatebin   desired_state: running
+  bookstack    desired_state: running
+  docmost      desired_state: running
+/root/privatebin-app.yaml.bak
+  (backup file still present — removing)
+  notification prefs: null email_set= False

documentation/audits/DRILL-r329-r386-2026-08-23/evidence/redproof-1-R329-emitter.txt

@@ -0,0 +1,8 @@
+### RED-PROOF 1 (R-329) — mutation: app_start_failed severity back to "warn" ###
+### layer: the EMITTER — the last point at which the bad value still exists ###
+--- FAIL: TestR329_EveryEmittedSeverityIsInTheHubVocabulary (0.16s)
+    r329_severity_contract_test.go:203: /mnt/5_hdd/felhom.eu/git/felhom-controller/controller/internal/notify/notifier.go:561:30: emit(...) emits severity "warn", which is NOT in the hub's vocabulary {info, warning, error, critical}.
+    r329_severity_contract_test.go:138: checked 30 severity literals across the controller
+FAIL
+FAIL	gitea.dooplex.hu/admin/felhom-controller/internal/notify	0.163s
+FAIL

documentation/audits/DRILL-r329-r386-2026-08-23/evidence/redproof-2-R386-intent.txt

@@ -0,0 +1,11 @@
+### RED-PROOF 2 (R-386) — mutation: userStopped reverted to the state guess ###
+### layer: classifyRunStates — the single derivation point where the guess was made ###
+--- FAIL: TestR386_OutOfBandStopWithRunningIntentAlarms (0.00s)
+    r386_intent_test.go:51: dead-app banner = [], want exactly privatebin — nobody asked for this app to be stopped, so its being stopped is a fault (measured silent on demo-hp 2026-08-23)
+--- FAIL: TestR386_AbsentIntentSuppressesButIsAnnounced (0.00s)
+    r386_intent_test.go:94: IntentUnknown = false — the suppression happened but nothing records it, so an operator cannot answer 'how many apps am I blind to?'. A rule without a mechanism is not a rule
+--- FAIL: TestR386_TheSchedulerWiresBothHalves (0.01s)
+    r386_intent_test.go:243: stacks.DesiredStateOf is never called from main.go — the classifier is back to guessing from the state (R-386)
+FAIL
+FAIL	gitea.dooplex.hu/admin/felhom-controller/cmd/controller	0.023s
+FAIL

documentation/audits/DRILL-r329-r386-2026-08-23/evidence/redproof-3-part4-migration.txt

@@ -0,0 +1,8 @@
+### RED-PROOF 3 (Part 4) — mutation: sameEventSet no-op guard removed ###
+### layer: the SAVE handler — where a render-then-save would rewrite stored bytes ###
+--- FAIL: TestR329Part4_RoundTripIsByteIdentical (0.04s)
+    --- FAIL: TestR329Part4_RoundTripIsByteIdentical/defaults (0.01s)
+        r329_toggle_split_test.go:115: a no-op save CHANGED the stored settings.
+FAIL
+FAIL	gitea.dooplex.hu/admin/felhom-controller/internal/web	0.048s
+FAIL

documentation/audits/DRILL-r329-r386-2026-08-23/evidence/redproof-4-R387-ingest.txt

@@ -0,0 +1,7 @@
+### RED-PROOF 4 (R-387) — mutation: the ingest WARN line removed ###
+### layer: INGEST — the last point at which the offending value still exists ###
+--- FAIL: TestR387_UnknownSeverityIsCoercedAndAnnounced (0.02s)
+    r387_severity_visibility_test.go:89: the hub rewrote a severity and said nothing — this is R-387, and it is how two features shipped undeliverable for months. log="[INFO] Event from c1: backup_failed (info) — test message\n"
+FAIL
+FAIL	gitea.dooplex.hu/admin/felhom-hub/internal/api	0.143s
+FAIL

documentation/audits/DRILL-r329-r386-2026-08-23/evidence/redproof-5-fillwatch-consequence.txt

@@ -0,0 +1,11 @@
+### RED-PROOF 5 (R-329, fillwatch) — REDONE ###
+### first attempt was INERT: an earlier 'if next == prev { continue }' makes 'next < prev'
+### and 'next <= prev' behave identically. Reported as a finding. ###
+### mutation: the de-escalation guard REMOVED, so BandOK reaches the notify seam ###
+### layer: Check() — the invariant lives THERE, not in Severity() ###
+--- FAIL: TestR329_FillwatchNeverEmitsTheEmptySeverity (0.00s)
+    r329_severity_test.go:62: notified with band ok → severity "" (event type ""): the hub coerces an unknown severity to "info" and then drops it, so this alert would reach NOBODY
+    r329_severity_test.go:67: notified with severity "", outside the hub vocabulary
+    r329_severity_test.go:70: 4 crossings notified, every one with a routable severity
+FAIL
+FAIL	gitea.dooplex.hu/admin/felhom-controller/internal/fillwatch	0.004s

documentation/audits/DRILL-r356-hot-only-restore-2026-08-22/README.md

@@ -0,0 +1,78 @@
+# DRILL — R-356: the off-site restore for an app with no data drive (2026-08-22)
+
+**Subject:** `demo-hp` (Tier 0, disposable), guest 9201, controller **v0.219.0**.
+**Method:** endpoint-level. `claude-in-chrome` is not available on DooPlex, so every action was the
+exact HTTP endpoint the UI's own form posts to — `POST /backup/offbox/run`,
+`POST /backup/offbox/restore`, `POST /backup/offbox/reconstitute` — driven with the session cookie and
+the session CSRF token scraped from the page. No server logic was skipped; only rendering.
+
+## What this drill was for
+
+Nobody had ever driven an off-site reconstitution to completion for an app whose snapshot contains
+**only** the recovery unit. The R-356 refusal always fired first. So the downstream legs — placement
+mapping, the stat pre-pass, `CheckPlacement`, the safety dump, the DB-service-only window, `volReplay`
+— had never run in that shape. **This was a first-run measurement, not a confirmation.**
+
+**Result: every downstream leg held.** No finding was raised against them.
+
+## Phase 1 — the driveless class (`privatebin`, `needs_hdd: false`)
+
+| step | what happened |
+|---|---|
+| plant | 2 pastes created through PrivateBin's **own JSON API** (`X-Requested-With: JSONHttpRequest`) + 4 files planted directly into the app's named volume, one with a Hungarian accented name |
+| findability control | planted a control string, **found** it, removed it, **failed to find** it — `02` in the evidence dir |
+| off-site | `POST /backup/offbox/run` → snapshot at 11:16:34Z, 2m20s, `status: ok`, store 44.5 → 47.0 MB |
+| destroy | 11 files deleted outright (`rm -rf`, no trash) — `03-destruction.txt` names every one |
+| prepare | `POST /backup/offbox/restore mode=full` → snapshot `306accff` into `/mnt/felhom-drives/hdd_1/backups/offsite-restore/privatebin` |
+| **the scratch shape** | **unit only** — `.../mnt/sys_drive/felhom-data/backups/primary/privatebin/` with `volume-dumps/privatebin_privatebin_data.tar`. The whole dataset is that one tar. This is the shape that had never been driven through. |
+| restore | `POST /backup/offbox/reconstitute` — **no refusal**. Volume replayed, app restarted, healthy. |
+| verify | **15/15 files byte-for-byte identical**, `diff` of the two sha256 manifests is empty |
+
+**The accented filename, as explicit hex bytes:**
+`c3a17276c3ad7a74c5b172c5912d74c3bc6bc3b67266c3ba72c3b367c3a9702d723335362e747874`
+(`árvíztűrő-tükörfúrógép-r356.txt`). Non-ASCII never crossed a shell chain: the planting script was
+base64-encoded on DooPlex and decoded inside the guest.
+
+**The customer-facing message, verbatim (160 bytes):**
+
+> A(z) privatebin: 0 fájl és 1 adatkötet visszaállítva (mentés: 2026-08-22 13:14) — az alkalmazás
+> újraindult. Ennek az alkalmazásnak nincs adatbázisa.
+
+**Judged, not merely recorded.** It **names the volume** beside the file count, so it is not the R-354
+shape wearing a new hat: a customer reading it is told what actually came back. There is no warning
+sitting beside a success. The one honest criticism — the first number a customer reads is `0 fájl`,
+and for all 40 driveless apps it will always be 0 — is a wording matter, filed as an observation and
+**not acted on here**.
+
+## Phase 2 — the drive class (`calibre-web`, `needs_hdd: true`, `HDD_PATH=/mnt/felhom-drives/hdd_1`)
+
+The same walk, to prove the 13-class did not move. The scratch resolved under
+`/mnt/felhom-drives/hdd_1/...` — the app's own drive, not the system data path — and the restore
+placed there. See `11`–`14` in the evidence dir.
+
+## Evidence
+
+`evidence/` holds every artefact, **copied off `demo-hp` at the end of each phase, before any revert**.
+Nothing was reverted or torn down during this drill, so no intermediate teardown could have taken it.
+
+## Deliberately NOT addressed here
+
+The message that calls an empty restore a success; the delete guard on verification copies; the
+absence of any readability check on the off-site store (**R-359**). Separate rows, untouched.
+
+### Phase 2 result
+
+**3/3 files back byte for byte**, accented name included (`Örkény-egyperces-r356.txt`, hex
+`c396726bc3a96e792d6567797065726365732d723335362e747874`). The restore placed them under
+`/mnt/felhom-drives/hdd_1/userdata/media/books/` — the app's own drive.
+`/mnt/sys_drive/felhom-data/userdata/media` **does not exist**, so there was no silent misplacement
+onto the system disk.
+
+Message, verbatim (161 bytes):
+
+> A(z) calibre-web: 3 fájl és 1 adatkötet visszaállítva (mentés: 2026-08-22 13:20) — az alkalmazás
+> újraindult. Ennek az alkalmazásnak nincs adatbázisa.
+
+**No behaviour change for the 13-class was observed.** That is an absence claim, so it does not stand
+on the live walk alone: the unit-test red-proof for it plants the system-data fallback into the drive
+path and three tests convict it (`TestR356_ScenarioB_*`, `TestR356_ScenarioE_PlaceDriveApp*`).

documentation/audits/DRILL-r356-hot-only-restore-2026-08-22/evidence/01-planted-manifest.sha256

@@ -0,0 +1,15 @@
+fa28d421a28ed9b6f4b997d5119c2b8adbacff6e2dd119a179b71230f9664934  ./.htaccess
+0e9aeb5dd4334bfaacfd51fd5fa5905ea6856b63c375a842bedc1620ce7c16ab  ./38/8e/388e3bd405a76985.php
+0c23c8531214fe20cbc1ed177da22f51d65570c5aaedb39e380aa8d4d3e62991  ./DRILL-2026-08-21/SENTINEL.txt
+725763bbe679b22d5c231a14083ff13155f475c028c22ade4623955b50a2a84a  ./DRILL-2026-08-21/binary-1mb.bin
+a39ad6f623da67ac72f2e62a24245eef46c722c279ae89cd6b0c7c45afa6a4c4  ./DRILL-2026-08-21/nested/őszibarack.md
+07e91a985809fc96752f97cddcf6523b211bc607c81a5808befabe35573926f1  ./DRILL-2026-08-21/plain.txt
+0d6a22ec56acf61b8a80e73eb12cb223d607acaa3684acfcfa7c3e34ecdfabea  ./DRILL-2026-08-21/árvíztűrő-tükörfúrógép.txt
+eb8c7a45d319701dc42c13b3175babc17367ce0f6e945386b11d99a4f9ac1d54  ./DRILL-r356-2026-08-22/SENTINEL.txt
+4bcf89782290f2057c85ab875b912be47d5d46732f01f09e35221a25401d74f7  ./DRILL-r356-2026-08-22/binary-1mb.bin
+576ff7dcc11b2b980bff935b9c35132bca72fbd8155d8e7c1c716d77b2546740  ./DRILL-r356-2026-08-22/nested/plain.txt
+2bfa318e42a4514ebd00da28fac8e1ca5f44a674dd72c30e087c168688503835  ./DRILL-r356-2026-08-22/árvíztűrő-tükörfúrógép-r356.txt
+f450cdc502162f20c7444576dec29d22140d009845c43e824cda5c3d3b84e320  ./ff/a1/ffa12cc12cd4fa71.php
+985ef4b2fca0b01bbbe5bcf25125293926e4d58053a2737ec9c470b0e74c397d  ./purge_limiter.php
+b5106bc17cb0ea56e7dcf69cc5480def0668670204c24aa8c7ee6db3f8e569eb  ./salt.php
+9c94379f2de428183db3364c23af55f1dba88c43af95f737ad5926be26e49713  ./traffic_limiter.php

documentation/audits/DRILL-r356-hot-only-restore-2026-08-22/evidence/02-offsite-runs.txt

@@ -0,0 +1,18 @@
+Off-site runs during this drill, read from POST /backup/offbox/run + GET /backup/offbox/status.
+(The earlier 02-offsite-run-after-plant.json captured empty — the status GET raced the phase change.
+ Recorded here from the same endpoint plus the controller log, rather than left as a 1-byte file.)
+
+RUN 1 — after the privatebin plant
+  last_run     : 2026-08-22T11:16:34Z
+  last_duration: 2m20s
+  last_error   : ""      status: ok      active: false
+  repo_size    : 44.5 MB -> 47.0 MB
+  snapshots    : 27
+
+RUN 2 — after the calibre-web plant
+  last_run     : 2026-08-22T11:22:13Z
+  last_error   : ""      status: ok      active: false
+  snapshots    : 27
+  log excerpt  : "backed up privatebin  (/mnt/sys_drive/felhom-data/backups/primary/privatebin, 0 mandatory path(s))"
+                 "backed up calibre-web (/mnt/felhom-drives/hdd_1/backups/primary/calibre-web, 1 mandatory path(s))"
+                 ^ the two classes, visible in one pair of lines: system data path vs the app's own drive.

documentation/audits/DRILL-r356-hot-only-restore-2026-08-22/evidence/03-destruction.txt

@@ -0,0 +1,17 @@
+== deleting:
+/var/lib/docker/volumes/privatebin_privatebin_data/_data/38/8e/388e3bd405a76985.php
+/var/lib/docker/volumes/privatebin_privatebin_data/_data/DRILL-2026-08-21/SENTINEL.txt
+/var/lib/docker/volumes/privatebin_privatebin_data/_data/DRILL-2026-08-21/binary-1mb.bin
+/var/lib/docker/volumes/privatebin_privatebin_data/_data/DRILL-2026-08-21/nested/őszibarack.md
+/var/lib/docker/volumes/privatebin_privatebin_data/_data/DRILL-2026-08-21/plain.txt
+/var/lib/docker/volumes/privatebin_privatebin_data/_data/DRILL-2026-08-21/árvíztűrő-tükörfúrógép.txt
+/var/lib/docker/volumes/privatebin_privatebin_data/_data/DRILL-r356-2026-08-22/SENTINEL.txt
+/var/lib/docker/volumes/privatebin_privatebin_data/_data/DRILL-r356-2026-08-22/binary-1mb.bin
+/var/lib/docker/volumes/privatebin_privatebin_data/_data/DRILL-r356-2026-08-22/nested/plain.txt
+/var/lib/docker/volumes/privatebin_privatebin_data/_data/DRILL-r356-2026-08-22/árvíztűrő-tükörfúrógép-r356.txt
+/var/lib/docker/volumes/privatebin_privatebin_data/_data/ff/a1/ffa12cc12cd4fa71.php
+== remaining files:
+/var/lib/docker/volumes/privatebin_privatebin_data/_data/.htaccess
+/var/lib/docker/volumes/privatebin_privatebin_data/_data/purge_limiter.php
+/var/lib/docker/volumes/privatebin_privatebin_data/_data/salt.php
+/var/lib/docker/volumes/privatebin_privatebin_data/_data/traffic_limiter.php

documentation/audits/DRILL-r356-hot-only-restore-2026-08-22/evidence/04-prepare-scratch.txt

@@ -0,0 +1,3 @@
+csrf_len=64
+HTTP/2 302 
+location: /backups/restore/app?name=privatebin&flash=A+t%C3%A1voli+vissza%C3%A1ll%C3%ADt%C3%A1s+elindult+%E2%80%94+az+%C3%A1llapot+itt+friss%C3%BCl.

documentation/audits/DRILL-r356-hot-only-restore-2026-08-22/evidence/05-scratch-contents.txt

@@ -0,0 +1,5 @@
+/mnt/felhom-drives/hdd_1/backups/offsite-restore/privatebin/mnt/sys_drive/felhom-data/backups/primary/privatebin/compose/.felhom.yml
+/mnt/felhom-drives/hdd_1/backups/offsite-restore/privatebin/mnt/sys_drive/felhom-data/backups/primary/privatebin/compose/app.yaml
+/mnt/felhom-drives/hdd_1/backups/offsite-restore/privatebin/mnt/sys_drive/felhom-data/backups/primary/privatebin/compose/docker-compose.yml
+/mnt/felhom-drives/hdd_1/backups/offsite-restore/privatebin/mnt/sys_drive/felhom-data/backups/primary/privatebin/manifest.json
+/mnt/felhom-drives/hdd_1/backups/offsite-restore/privatebin/mnt/sys_drive/felhom-data/backups/primary/privatebin/volume-dumps/privatebin_privatebin_data.tar

documentation/audits/DRILL-r356-hot-only-restore-2026-08-22/evidence/06-reconstitute-post.txt

@@ -0,0 +1,3 @@
+csrf_len=64
+HTTP/2 302 
+location: /backups/restore/app?name=privatebin&flash=A+teljes+vissza%C3%A1ll%C3%ADt%C3%A1s+elindult+%E2%80%94+az+%C3%A1llapot+itt+friss%C3%BCl.

documentation/audits/DRILL-r356-hot-only-restore-2026-08-22/evidence/07-reconstitute-log.txt

@@ -0,0 +1,30 @@
+2026/08/22 11:14:26 backup.go:750: [INFO] [backup] Volume dump: opengist/opengist_opengist_data → 177.0 KB
+2026/08/22 11:14:26 backup.go:859: [INFO] [backup] Restarting opengist after volume dump
+2026/08/22 11:14:27 backup.go:849: [INFO] [backup] Stopping paperless-ngx for safe volume dump
+2026/08/22 11:14:34 backup.go:725: [DEBUG] [backup] Dumping volume paperless-ngx_paperless_postgres_data for paperless-ngx
+2026/08/22 11:14:35 backup.go:750: [INFO] [backup] Volume dump: paperless-ngx/paperless-ngx_paperless_postgres_data → 68.9 MB
+2026/08/22 11:14:35 backup.go:725: [DEBUG] [backup] Dumping volume paperless-ngx_paperless_redis_data for paperless-ngx
+2026/08/22 11:14:35 backup.go:750: [INFO] [backup] Volume dump: paperless-ngx/paperless-ngx_paperless_redis_data → 383.0 KB
+2026/08/22 11:14:35 backup.go:725: [DEBUG] [backup] Dumping volume paperless-ngx_paperless_data for paperless-ngx
+2026/08/22 11:14:36 backup.go:750: [INFO] [backup] Volume dump: paperless-ngx/paperless-ngx_paperless_data → 488.5 KB
+2026/08/22 11:14:36 backup.go:859: [INFO] [backup] Restarting paperless-ngx after volume dump
+2026/08/22 11:14:47 backup.go:849: [INFO] [backup] Stopping privatebin for safe volume dump
+2026/08/22 11:14:47 backup.go:725: [DEBUG] [backup] Dumping volume privatebin_privatebin_data for privatebin
+2026/08/22 11:14:47 backup.go:750: [INFO] [backup] Volume dump: privatebin/privatebin_privatebin_data → 2.0 MB
+2026/08/22 11:14:47 backup.go:859: [INFO] [backup] Restarting privatebin after volume dump
+2026/08/22 11:14:48 backup.go:849: [INFO] [backup] Stopping romm for safe volume dump
+2026/08/22 11:14:52 backup.go:725: [DEBUG] [backup] Dumping volume romm_romm_config for romm
+2026/08/22 11:14:52 backup.go:750: [INFO] [backup] Volume dump: romm/romm_romm_config → 2.5 KB
+2026/08/22 11:14:52 backup.go:725: [DEBUG] [backup] Dumping volume romm_romm_db_data for romm
+2026/08/22 11:14:53 backup.go:750: [INFO] [backup] Volume dump: romm/romm_romm_db_data → 152.8 MB
+2026/08/22 11:14:53 backup.go:725: [DEBUG] [backup] Dumping volume romm_romm_redis_data for romm
+2026/08/22 11:14:53 backup.go:750: [INFO] [backup] Volume dump: romm/romm_romm_redis_data → 14.5 MB
+2026/08/22 11:14:53 backup.go:859: [INFO] [backup] Restarting romm after volume dump
+2026/08/22 11:15:05 backup.go:584: [INFO] [backup] App-data backup completed: 3 databases (422.0 KB total), 6 volume dump(s) (55.518s)
+2026/08/22 11:18:22 auth.go:134: [DEBUG] [web] auth: valid session for POST /backup/offbox/reconstitute
+2026/08/22 11:18:22 server.go:393: [DEBUG] [web] ServeHTTP: POST /backup/offbox/reconstitute from 172.18.0.4:60064
+2026/08/22 11:18:25 restore.go:140: [INFO] [backup] Restoring Docker volume privatebin_privatebin_data for privatebin
+2026/08/22 11:18:26 restore.go:169: [DEBUG] [backup] Volume privatebin_privatebin_data restored successfully
+2026/08/22 11:18:26 restore.go:174: [INFO] [backup] Restored 1 Docker volume(s) for privatebin
+2026/08/22 11:18:34 offbox_reconstitute.go:452: [INFO] [offbox] reconstituted privatebin from snapshot 306accff: 0 file(s) placed, 0 DB dump(s) replayed, safety dump=., skewed=false
+2026/08/22 11:18:34 offbox_handlers.go:455: [INFO] [web] off-box reconstitute privatebin completed (async): files=0 dbs=0 snapshot=306accff

documentation/audits/DRILL-r356-hot-only-restore-2026-08-22/evidence/08-restored-manifest.sha256

@@ -0,0 +1,15 @@
+fa28d421a28ed9b6f4b997d5119c2b8adbacff6e2dd119a179b71230f9664934  ./.htaccess
+0e9aeb5dd4334bfaacfd51fd5fa5905ea6856b63c375a842bedc1620ce7c16ab  ./38/8e/388e3bd405a76985.php
+0c23c8531214fe20cbc1ed177da22f51d65570c5aaedb39e380aa8d4d3e62991  ./DRILL-2026-08-21/SENTINEL.txt
+725763bbe679b22d5c231a14083ff13155f475c028c22ade4623955b50a2a84a  ./DRILL-2026-08-21/binary-1mb.bin
+a39ad6f623da67ac72f2e62a24245eef46c722c279ae89cd6b0c7c45afa6a4c4  ./DRILL-2026-08-21/nested/őszibarack.md
+07e91a985809fc96752f97cddcf6523b211bc607c81a5808befabe35573926f1  ./DRILL-2026-08-21/plain.txt
+0d6a22ec56acf61b8a80e73eb12cb223d607acaa3684acfcfa7c3e34ecdfabea  ./DRILL-2026-08-21/árvíztűrő-tükörfúrógép.txt
+eb8c7a45d319701dc42c13b3175babc17367ce0f6e945386b11d99a4f9ac1d54  ./DRILL-r356-2026-08-22/SENTINEL.txt
+4bcf89782290f2057c85ab875b912be47d5d46732f01f09e35221a25401d74f7  ./DRILL-r356-2026-08-22/binary-1mb.bin
+576ff7dcc11b2b980bff935b9c35132bca72fbd8155d8e7c1c716d77b2546740  ./DRILL-r356-2026-08-22/nested/plain.txt
+2bfa318e42a4514ebd00da28fac8e1ca5f44a674dd72c30e087c168688503835  ./DRILL-r356-2026-08-22/árvíztűrő-tükörfúrógép-r356.txt
+f450cdc502162f20c7444576dec29d22140d009845c43e824cda5c3d3b84e320  ./ff/a1/ffa12cc12cd4fa71.php
+985ef4b2fca0b01bbbe5bcf25125293926e4d58053a2737ec9c470b0e74c397d  ./purge_limiter.php
+b5106bc17cb0ea56e7dcf69cc5480def0668670204c24aa8c7ee6db3f8e569eb  ./salt.php
+9c94379f2de428183db3364c23af55f1dba88c43af95f737ad5926be26e49713  ./traffic_limiter.php

documentation/audits/DRILL-r356-hot-only-restore-2026-08-22/evidence/09-outcome-message-verbatim.txt

@@ -0,0 +1,7 @@
+=== CUSTOMER-FACING OUTCOME MESSAGE, VERBATIM ===
+A(z) privatebin: 0 fájl és 1 adatkötet visszaállítva (mentés: 2026-08-22 13:14) — az alkalmazás újraindult. Ennek az alkalmazásnak nincs adatbázisa.
+
+=== the same string as UTF-8 hex bytes ===
+41287a29207072697661746562696e3a20302066c3a16a6c20c3a973203120616461746bc3b674657420766973737a61c3a16c6cc3ad74766120286d656e74c3a9733a20323032362d30382d32322031333a31342920e2809420617a20616c6b616c6d617ac3a17320c3ba6a7261696e64756c742e20456e6e656b20617a20616c6b616c6d617ac3a1736e616b206e696e6373206164617462c3a17a6973612e
+
+=== byte length: 160

documentation/audits/DRILL-r356-hot-only-restore-2026-08-22/evidence/10-controller-log-phase1.txt

@@ -0,0 +1,400 @@
+2026/08/22 11:17:27 server.go:393: [DEBUG] [web] ServeHTTP: GET / from 172.18.0.4:60064
+2026/08/22 11:17:27 auth.go:134: [DEBUG] [web] auth: valid session for GET /launcher
+2026/08/22 11:17:27 server.go:393: [DEBUG] [web] ServeHTTP: GET /launcher from 172.18.0.4:60064
+2026/08/22 11:17:27 auth.go:134: [DEBUG] [web] auth: valid session for GET /backup/offbox/status
+2026/08/22 11:17:27 server.go:393: [DEBUG] [web] ServeHTTP: GET /backup/offbox/status from 172.18.0.4:60064
+2026/08/22 11:17:32 scheduler.go:67: [DEBUG] [scheduler] job status-refresh: execution starting
+2026/08/22 11:17:32 scheduler.go:67: [DEBUG] [scheduler] job health-probes: execution starting
+2026/08/22 11:17:32 manager.go:621: [INFO] [stacks] Status refresh: 14 containers across 56 stacks
+2026/08/22 11:17:32 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping calibre-web — last check 3m0s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:17:32 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping kimai — last check 2m30s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:17:32 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping opengist — last check 2m50s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:17:32 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping privatebin — last check 2m30s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:17:32 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping romm — last check 1m50s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:17:32 healthprobe.go:76: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (5 skipped not due, 1 skipped no container)
+2026/08/22 11:17:38 auth.go:146: [DEBUG] [web] login attempt from 172.18.0.4:60064 (X-Forwarded-For: 172.18.0.1)
+2026/08/22 11:17:38 auth.go:194: [DEBUG] [web] login successful from 172.18.0.1, creating session
+2026/08/22 11:17:38 auth.go:260: [DEBUG] [web] session created, expires=2026-08-29T11:17:38Z, active_sessions=24
+2026/08/22 11:17:38 auth.go:214: [INFO] [web] Login from 172.18.0.4:60064
+2026/08/22 11:17:38 auth.go:134: [DEBUG] [web] auth: valid session for GET /
+2026/08/22 11:17:38 server.go:393: [DEBUG] [web] ServeHTTP: GET / from 172.18.0.4:60064
+2026/08/22 11:17:38 auth.go:134: [DEBUG] [web] auth: valid session for GET /launcher
+2026/08/22 11:17:38 server.go:393: [DEBUG] [web] ServeHTTP: GET /launcher from 172.18.0.4:60064
+2026/08/22 11:17:38 auth.go:134: [DEBUG] [web] auth: valid session for GET /backup/offbox/status
+2026/08/22 11:17:38 server.go:393: [DEBUG] [web] ServeHTTP: GET /backup/offbox/status from 172.18.0.4:60064
+2026/08/22 11:17:38 auth.go:146: [DEBUG] [web] login attempt from 172.18.0.4:60064 (X-Forwarded-For: 172.18.0.1)
+2026/08/22 11:17:38 auth.go:194: [DEBUG] [web] login successful from 172.18.0.1, creating session
+2026/08/22 11:17:38 auth.go:260: [DEBUG] [web] session created, expires=2026-08-29T11:17:38Z, active_sessions=25
+2026/08/22 11:17:38 auth.go:214: [INFO] [web] Login from 172.18.0.4:60064
+2026/08/22 11:17:38 auth.go:134: [DEBUG] [web] auth: valid session for GET /
+2026/08/22 11:17:38 server.go:393: [DEBUG] [web] ServeHTTP: GET / from 172.18.0.4:60064
+2026/08/22 11:17:38 auth.go:134: [DEBUG] [web] auth: valid session for GET /launcher
+2026/08/22 11:17:38 server.go:393: [DEBUG] [web] ServeHTTP: GET /launcher from 172.18.0.4:60064
+2026/08/22 11:17:38 auth.go:134: [DEBUG] [web] auth: valid session for GET /backup/offbox/status
+2026/08/22 11:17:38 server.go:393: [DEBUG] [web] ServeHTTP: GET /backup/offbox/status from 172.18.0.4:60064
+2026/08/22 11:17:42 scheduler.go:67: [DEBUG] [scheduler] job health-probes: execution starting
+2026/08/22 11:17:42 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping calibre-web — last check 3m10s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:17:42 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping kimai — last check 2m40s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:17:42 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping opengist — last check 3m0s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:17:42 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping privatebin — last check 2m40s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:17:42 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping romm — last check 2m0s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:17:42 healthprobe.go:76: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (5 skipped not due, 1 skipped no container)
+2026/08/22 11:17:42 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
+2026/08/22 11:17:42 scheduler.go:67: [DEBUG] [scheduler] job ring-spill: execution starting
+2026/08/22 11:17:42 scheduler.go:67: [DEBUG] [scheduler] job status-refresh: execution starting
+2026/08/22 11:17:42 manager.go:621: [INFO] [stacks] Status refresh: 14 containers across 56 stacks
+2026/08/22 11:17:43 client.go:1142: [DEBUG] [agentapi] GET /disks -> 200 (602ms)
+2026/08/22 11:17:44 client.go:1142: [DEBUG] [agentapi] GET /disks -> 200 (623ms)
+2026/08/22 11:17:52 scheduler.go:67: [DEBUG] [scheduler] job health-probes: execution starting
+2026/08/22 11:17:52 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping calibre-web — last check 3m20s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:17:52 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping kimai — last check 2m50s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:17:52 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping opengist — last check 3m10s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:17:52 scheduler.go:67: [DEBUG] [scheduler] job status-refresh: execution starting
+2026/08/22 11:17:52 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping privatebin — last check 2m50s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:17:52 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping romm — last check 2m10s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:17:52 healthprobe.go:76: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (5 skipped not due, 1 skipped no container)
+2026/08/22 11:17:52 manager.go:621: [INFO] [stacks] Status refresh: 14 containers across 56 stacks
+2026/08/22 11:17:54 auth.go:146: [DEBUG] [web] login attempt from 172.18.0.4:60064 (X-Forwarded-For: 172.18.0.1)
+2026/08/22 11:17:54 auth.go:194: [DEBUG] [web] login successful from 172.18.0.1, creating session
+2026/08/22 11:17:54 auth.go:260: [DEBUG] [web] session created, expires=2026-08-29T11:17:54Z, active_sessions=26
+2026/08/22 11:17:54 auth.go:214: [INFO] [web] Login from 172.18.0.4:60064
+2026/08/22 11:17:54 auth.go:134: [DEBUG] [web] auth: valid session for GET /
+2026/08/22 11:17:54 server.go:393: [DEBUG] [web] ServeHTTP: GET / from 172.18.0.4:60064
+2026/08/22 11:17:54 auth.go:134: [DEBUG] [web] auth: valid session for GET /launcher
+2026/08/22 11:17:54 server.go:393: [DEBUG] [web] ServeHTTP: GET /launcher from 172.18.0.4:60064
+2026/08/22 11:17:54 auth.go:134: [DEBUG] [web] auth: valid session for POST /backup/offbox/restore
+2026/08/22 11:17:54 server.go:393: [DEBUG] [web] ServeHTTP: POST /backup/offbox/restore from 172.18.0.4:60064
+2026/08/22 11:17:59 auth.go:146: [DEBUG] [web] login attempt from 172.18.0.4:60064 (X-Forwarded-For: 172.18.0.1)
+2026/08/22 11:17:59 auth.go:194: [DEBUG] [web] login successful from 172.18.0.1, creating session
+2026/08/22 11:17:59 auth.go:260: [DEBUG] [web] session created, expires=2026-08-29T11:17:59Z, active_sessions=27
+2026/08/22 11:17:59 auth.go:214: [INFO] [web] Login from 172.18.0.4:60064
+2026/08/22 11:17:59 auth.go:134: [DEBUG] [web] auth: valid session for GET /
+2026/08/22 11:17:59 server.go:393: [DEBUG] [web] ServeHTTP: GET / from 172.18.0.4:60064
+2026/08/22 11:17:59 auth.go:134: [DEBUG] [web] auth: valid session for GET /launcher
+2026/08/22 11:17:59 server.go:393: [DEBUG] [web] ServeHTTP: GET /launcher from 172.18.0.4:60064
+2026/08/22 11:18:00 auth.go:134: [DEBUG] [web] auth: valid session for GET /backup/offbox/status
+2026/08/22 11:18:00 server.go:393: [DEBUG] [web] ServeHTTP: GET /backup/offbox/status from 172.18.0.4:60064
+2026/08/22 11:18:02 scheduler.go:67: [DEBUG] [scheduler] job status-refresh: execution starting
+2026/08/22 11:18:02 scheduler.go:67: [DEBUG] [scheduler] job health-probes: execution starting
+2026/08/22 11:18:02 manager.go:621: [INFO] [stacks] Status refresh: 14 containers across 56 stacks
+2026/08/22 11:18:02 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping calibre-web — last check 3m30s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:18:02 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping kimai — last check 3m0s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:18:02 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping opengist — last check 3m20s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:18:02 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping privatebin — last check 3m0s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:18:02 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping romm — last check 2m20s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:18:02 healthprobe.go:76: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (5 skipped not due, 1 skipped no container)
+2026/08/22 11:18:03 auth.go:146: [DEBUG] [web] login attempt from 172.18.0.4:60064 (X-Forwarded-For: 172.18.0.1)
+2026/08/22 11:18:03 auth.go:194: [DEBUG] [web] login successful from 172.18.0.1, creating session
+2026/08/22 11:18:03 auth.go:260: [DEBUG] [web] session created, expires=2026-08-29T11:18:03Z, active_sessions=28
+2026/08/22 11:18:03 auth.go:214: [INFO] [web] Login from 172.18.0.4:60064
+2026/08/22 11:18:03 auth.go:134: [DEBUG] [web] auth: valid session for GET /
+2026/08/22 11:18:03 server.go:393: [DEBUG] [web] ServeHTTP: GET / from 172.18.0.4:60064
+2026/08/22 11:18:03 auth.go:134: [DEBUG] [web] auth: valid session for GET /launcher
+2026/08/22 11:18:03 server.go:393: [DEBUG] [web] ServeHTTP: GET /launcher from 172.18.0.4:60064
+2026/08/22 11:18:03 auth.go:134: [DEBUG] [web] auth: valid session for GET /backup/restore/status
+2026/08/22 11:18:03 server.go:393: [DEBUG] [web] ServeHTTP: GET /backup/restore/status from 172.18.0.4:60064
+2026/08/22 11:18:03 server.go:636: [WARN] [web] 404 Not Found: GET /backup/restore/status
+2026/08/22 11:18:05 offbox_restore.go:270: [INFO] [offbox] restored privatebin (306accff, full=true) → /mnt/felhom-drives/hdd_1/backups/offsite-restore/privatebin
+2026/08/22 11:18:05 offbox_handlers.go:372: [INFO] [web] off-box restore privatebin completed (full=true, async)
+2026/08/22 11:18:11 info.go:13: [DEBUG] [system] GetInfo starting (hddPath="/mnt/felhom-drives/hdd_1", hasCPUCollector=true)
+2026/08/22 11:18:11 info.go:13: [DEBUG] [system] readMemInfo: guest cap=25898MB (host total was 30714460KB) → used≈3517MB avail≈22381MB
+2026/08/22 11:18:11 info.go:13: [DEBUG] [system] readMemInfo: totalKB=30714460 availKB=26542728 → total=25898MB avail=22381MB used=3517MB (13.6%)
+2026/08/22 11:18:11 info.go:13: [DEBUG] [system] readDiskUsage: path="/" bsize=4096 total=68.7GB used=8.0GB avail=57.2GB (11.6%)
+2026/08/22 11:18:11 info.go:13: [DEBUG] [system] readDiskUsage: path="/mnt/felhom-drives/hdd_1" bsize=4096 total=937.8GB used=5.5GB avail=884.7GB (0.6%)
+2026/08/22 11:18:11 info.go:13: [DEBUG] [system] readLoadAvg: raw="1.08 1.32 0.93 3/765 2831" → 1m=1.08 5m=1.32 15m=0.93
+2026/08/22 11:18:11 info.go:13: [DEBUG] [system] readThermalZones: /sys — found 1 zones
+2026/08/22 11:18:11 info.go:13: [DEBUG] [system] readTemperature: found via hwmon at /sys — 52.9°C (hwmon1)
+2026/08/22 11:18:11 info.go:13: [DEBUG] [system] GetInfo done in 59ms — mem=3517MB/25898MB (13.6%), rootDisk=8.0GB/68.7GB (11.6%), load=1.08/1.32/0.93, temp=52.9°C (hwmon1), cpu=10.2%
+2026/08/22 11:18:12 scheduler.go:346: [INFO] [scheduler] Running job: stack-scan
+2026/08/22 11:18:12 scheduler.go:67: [DEBUG] [scheduler] job stack-scan: execution starting
+2026/08/22 11:18:12 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
+2026/08/22 11:18:12 scheduler.go:67: [DEBUG] [scheduler] job ring-spill: execution starting
+2026/08/22 11:18:12 scheduler.go:67: [DEBUG] [scheduler] job health-probes: execution starting
+2026/08/22 11:18:12 scheduler.go:67: [DEBUG] [scheduler] job status-refresh: execution starting
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "actualbudget" deployed=false composePath=/opt/docker/stacks/actualbudget/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "adventurelog" deployed=false composePath=/opt/docker/stacks/adventurelog/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "audiobookshelf" deployed=false composePath=/opt/docker/stacks/audiobookshelf/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "bentopdf" deployed=false composePath=/opt/docker/stacks/bentopdf/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "bookstack" deployed=false composePath=/opt/docker/stacks/bookstack/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "calcom" deployed=false composePath=/opt/docker/stacks/calcom/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "calibre-web" deployed=true composePath=/opt/docker/stacks/calibre-web/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "claper" deployed=false composePath=/opt/docker/stacks/claper/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "cloudflared" deployed=false composePath=/opt/docker/stacks/cloudflared/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "code-server" deployed=false composePath=/opt/docker/stacks/code-server/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "crafty-controller" deployed=false composePath=/opt/docker/stacks/crafty-controller/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "docmost" deployed=false composePath=/opt/docker/stacks/docmost/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "emby" deployed=false composePath=/opt/docker/stacks/emby/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "filebrowser" deployed=false composePath=/opt/docker/stacks/filebrowser/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "ghost" deployed=false composePath=/opt/docker/stacks/ghost/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "gitea" deployed=false composePath=/opt/docker/stacks/gitea/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "glance" deployed=false composePath=/opt/docker/stacks/glance/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "gokapi" deployed=false composePath=/opt/docker/stacks/gokapi/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "grafana" deployed=false composePath=/opt/docker/stacks/grafana/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "gramps-web" deployed=false composePath=/opt/docker/stacks/gramps-web/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "home-assistant" deployed=false composePath=/opt/docker/stacks/home-assistant/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "homebox" deployed=false composePath=/opt/docker/stacks/homebox/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "homepage" deployed=false composePath=/opt/docker/stacks/homepage/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "immich" deployed=false composePath=/opt/docker/stacks/immich/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "jellyfin" deployed=false composePath=/opt/docker/stacks/jellyfin/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "kimai" deployed=true composePath=/opt/docker/stacks/kimai/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "komga" deployed=false composePath=/opt/docker/stacks/komga/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "mealie" deployed=false composePath=/opt/docker/stacks/mealie/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "n8n" deployed=false composePath=/opt/docker/stacks/n8n/docker-compose.yml
+2026/08/22 11:18:12 scheduler.go:346: [INFO] [scheduler] Running job: agent-channel-health
+2026/08/22 11:18:12 scheduler.go:67: [DEBUG] [scheduler] job agent-channel-health: execution starting
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "navidrome" deployed=false composePath=/opt/docker/stacks/navidrome/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "nextcloud" deployed=false composePath=/opt/docker/stacks/nextcloud/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "onlyoffice" deployed=false composePath=/opt/docker/stacks/onlyoffice/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "opengist" deployed=true composePath=/opt/docker/stacks/opengist/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "outline" deployed=false composePath=/opt/docker/stacks/outline/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "paperless-ngx" deployed=true composePath=/opt/docker/stacks/paperless-ngx/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "papra" deployed=false composePath=/opt/docker/stacks/papra/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "plant-it" deployed=false composePath=/opt/docker/stacks/plant-it/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "plex" deployed=false composePath=/opt/docker/stacks/plex/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "privatebin" deployed=true composePath=/opt/docker/stacks/privatebin/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "radarr" deployed=false composePath=/opt/docker/stacks/radarr/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "rallly" deployed=false composePath=/opt/docker/stacks/rallly/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "recipe-importer" deployed=false composePath=/opt/docker/stacks/recipe-importer/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "romm" deployed=true composePath=/opt/docker/stacks/romm/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "seerr" deployed=false composePath=/opt/docker/stacks/seerr/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "sonarr" deployed=false composePath=/opt/docker/stacks/sonarr/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "sparkyfitness" deployed=false composePath=/opt/docker/stacks/sparkyfitness/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "tandoor" deployed=false composePath=/opt/docker/stacks/tandoor/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "termix" deployed=false composePath=/opt/docker/stacks/termix/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "traefik" deployed=false composePath=/opt/docker/stacks/traefik/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "uptime-kuma" deployed=false composePath=/opt/docker/stacks/uptime-kuma/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "vaultwarden" deployed=false composePath=/opt/docker/stacks/vaultwarden/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "vikunja" deployed=false composePath=/opt/docker/stacks/vikunja/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "wanderer" deployed=false composePath=/opt/docker/stacks/wanderer/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "wger" deployed=false composePath=/opt/docker/stacks/wger/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "wishlist" deployed=false composePath=/opt/docker/stacks/wishlist/docker-compose.yml
+2026/08/22 11:18:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "zipline" deployed=false composePath=/opt/docker/stacks/zipline/docker-compose.yml
+2026/08/22 11:18:12 manager.go:1497: [DEBUG] [stacks] getCatalogTemplateSlugs: found 53 template slugs in /opt/docker/felhom-controller/data/catalog-cache/templates
+2026/08/22 11:18:12 manager.go:539: [DEBUG] [stacks] ScanStacks: catalog has 53 template slugs for orphan detection
+2026/08/22 11:18:12 manager.go:570: [INFO] [stacks] ScanStacks complete: 56 stacks found (6 deployed, 50 available)
+2026/08/22 11:18:12 manager.go:621: [INFO] [stacks] Status refresh: 14 containers across 56 stacks
+2026/08/22 11:18:12 scheduler.go:363: [INFO] [scheduler] Job stack-scan completed (took 47ms)
+2026/08/22 11:18:12 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping calibre-web — last check 3m40s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:18:12 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping kimai — last check 3m10s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:18:12 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping opengist — last check 3m30s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:18:12 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping privatebin — last check 3m10s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:18:12 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping romm — last check 2m30s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:18:12 healthprobe.go:76: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (5 skipped not due, 1 skipped no container)
+2026/08/22 11:18:12 manager.go:621: [INFO] [stacks] Status refresh: 14 containers across 56 stacks
+2026/08/22 11:18:12 client.go:1142: [DEBUG] [agentapi] GET /storage -> 200 (577ms)
+2026/08/22 11:18:12 scheduler.go:363: [INFO] [scheduler] Job agent-channel-health completed (took 577ms)
+2026/08/22 11:18:13 client.go:1142: [DEBUG] [agentapi] GET /disks -> 200 (607ms)
+2026/08/22 11:18:14 client.go:1142: [DEBUG] [agentapi] GET /disks -> 200 (676ms)
+2026/08/22 11:18:21 auth.go:146: [DEBUG] [web] login attempt from 172.18.0.4:60064 (X-Forwarded-For: 172.18.0.1)
+2026/08/22 11:18:21 auth.go:194: [DEBUG] [web] login successful from 172.18.0.1, creating session
+2026/08/22 11:18:21 auth.go:260: [DEBUG] [web] session created, expires=2026-08-29T11:18:21Z, active_sessions=29
+2026/08/22 11:18:21 auth.go:214: [INFO] [web] Login from 172.18.0.4:60064
+2026/08/22 11:18:21 auth.go:134: [DEBUG] [web] auth: valid session for GET /
+2026/08/22 11:18:21 server.go:393: [DEBUG] [web] ServeHTTP: GET / from 172.18.0.4:60064
+2026/08/22 11:18:21 auth.go:134: [DEBUG] [web] auth: valid session for GET /launcher
+2026/08/22 11:18:21 server.go:393: [DEBUG] [web] ServeHTTP: GET /launcher from 172.18.0.4:60064
+2026/08/22 11:18:21 auth.go:134: [DEBUG] [web] auth: valid session for GET /backup/offbox/status
+2026/08/22 11:18:21 server.go:393: [DEBUG] [web] ServeHTTP: GET /backup/offbox/status from 172.18.0.4:60064
+2026/08/22 11:18:22 scheduler.go:67: [DEBUG] [scheduler] job health-probes: execution starting
+2026/08/22 11:18:22 scheduler.go:67: [DEBUG] [scheduler] job status-refresh: execution starting
+2026/08/22 11:18:22 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping kimai — last check 3m20s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:18:22 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping opengist — last check 3m40s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:18:22 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping privatebin — last check 3m20s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:18:22 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping romm — last check 2m40s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:18:22 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping calibre-web — last check 3m50s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:18:22 healthprobe.go:76: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (5 skipped not due, 1 skipped no container)
+2026/08/22 11:18:22 manager.go:621: [INFO] [stacks] Status refresh: 14 containers across 56 stacks
+2026/08/22 11:18:22 auth.go:146: [DEBUG] [web] login attempt from 172.18.0.4:60064 (X-Forwarded-For: 172.18.0.1)
+2026/08/22 11:18:22 auth.go:194: [DEBUG] [web] login successful from 172.18.0.1, creating session
+2026/08/22 11:18:22 auth.go:260: [DEBUG] [web] session created, expires=2026-08-29T11:18:22Z, active_sessions=30
+2026/08/22 11:18:22 auth.go:214: [INFO] [web] Login from 172.18.0.4:60064
+2026/08/22 11:18:22 auth.go:134: [DEBUG] [web] auth: valid session for GET /
+2026/08/22 11:18:22 server.go:393: [DEBUG] [web] ServeHTTP: GET / from 172.18.0.4:60064
+2026/08/22 11:18:22 auth.go:134: [DEBUG] [web] auth: valid session for GET /launcher
+2026/08/22 11:18:22 server.go:393: [DEBUG] [web] ServeHTTP: GET /launcher from 172.18.0.4:60064
+2026/08/22 11:18:22 auth.go:134: [DEBUG] [web] auth: valid session for POST /backup/offbox/reconstitute
+2026/08/22 11:18:22 server.go:393: [DEBUG] [web] ServeHTTP: POST /backup/offbox/reconstitute from 172.18.0.4:60064
+2026/08/22 11:18:25 dbdump.go:92: [DEBUG] DiscoverDatabases: running docker ps to find database containers
+2026/08/22 11:18:25 dbdump.go:111: [DEBUG] DiscoverDatabases: docker ps output: 4c72492af5ca	romm	romm	rommapp/romm:5.0.0
+b261d717618d	romm-db	romm	mariadb:11.4
+9010a9d7927f	romm-redis	romm	redis:7-alpine
+83d6e8adc484	privatebin	privatebin	privatebin/pdo:2.0.5
+80fc6005741e	paperless-webserver	paperless-ngx	ghcr.io/paperless-ngx/paperless-ngx:2.20.15
+ae015380d719	paperless-postgres	paperless-ngx	postgres:16-alpine
+fddae44de81e	paperless-redis	paperless-ngx	redis:7-alpine
+3aae6ced0216	opengist	opengist	ghcr.io/thomiceli/opengist:1.13
+d0c84add6adf	kimai	kimai	kimai/kimai2:apac...
+2026/08/22 11:18:25 dbdump.go:133: [DEBUG] DiscoverDatabases: skipping container romm (image=rommapp/romm:5.0.0, not a database)
+2026/08/22 11:18:25 dbdump.go:140: [DEBUG] DiscoverDatabases: found mariadb container: romm-db (id=b261d717618d)
+2026/08/22 11:18:25 dbdump.go:160: [DEBUG] DiscoverDatabases: romm-db → stack=romm, dbUser=root, dbName=romm
+2026/08/22 11:18:25 dbdump.go:133: [DEBUG] DiscoverDatabases: skipping container romm-redis (image=redis:7-alpine, not a database)
+2026/08/22 11:18:25 dbdump.go:133: [DEBUG] DiscoverDatabases: skipping container privatebin (image=privatebin/pdo:2.0.5, not a database)
+2026/08/22 11:18:25 dbdump.go:133: [DEBUG] DiscoverDatabases: skipping container paperless-webserver (image=ghcr.io/paperless-ngx/paperless-ngx:2.20.15, not a database)
+2026/08/22 11:18:25 dbdump.go:140: [DEBUG] DiscoverDatabases: found postgres container: paperless-postgres (id=ae015380d719)
+2026/08/22 11:18:25 dbdump.go:805: [DEBUG] DiscoverDatabases: paperless-postgres → stack "paperless-ngx" from the compose project label (the container name would have given "paperless")
+2026/08/22 11:18:25 dbdump.go:160: [DEBUG] DiscoverDatabases: paperless-postgres → stack=paperless-ngx, dbUser=paperless, dbName=paperless
+2026/08/22 11:18:25 dbdump.go:133: [DEBUG] DiscoverDatabases: skipping container paperless-redis (image=redis:7-alpine, not a database)
+2026/08/22 11:18:25 dbdump.go:133: [DEBUG] DiscoverDatabases: skipping container opengist (image=ghcr.io/thomiceli/opengist:1.13, not a database)
+2026/08/22 11:18:25 dbdump.go:133: [DEBUG] DiscoverDatabases: skipping container kimai (image=kimai/kimai2:apache-2.57.0, not a database)
+2026/08/22 11:18:25 dbdump.go:140: [DEBUG] DiscoverDatabases: found mariadb container: kimai-db (id=fb9ebd95951c)
+2026/08/22 11:18:25 dbdump.go:160: [DEBUG] DiscoverDatabases: kimai-db → stack=kimai, dbUser=root, dbName=kimai
+2026/08/22 11:18:25 dbdump.go:133: [DEBUG] DiscoverDatabases: skipping container calibre-web (image=crocodilestick/calibre-web-automated:v4.0.6, not a database)
+2026/08/22 11:18:25 dbdump.go:133: [DEBUG] DiscoverDatabases: skipping container felhom-controller (image=gitea.dooplex.hu/admin/felhom-controller:0.219.0, not a database)
+2026/08/22 11:18:25 dbdump.go:133: [DEBUG] DiscoverDatabases: skipping container filebrowser (image=gtstef/filebrowser:1.3.3-stable, not a database)
+2026/08/22 11:18:25 dbdump.go:133: [DEBUG] DiscoverDatabases: skipping container cloudflared (image=cloudflare/cloudflared:2026.6.0, not a database)
+2026/08/22 11:18:25 dbdump.go:133: [DEBUG] DiscoverDatabases: skipping container traefik (image=traefik:v3.6.7, not a database)
+2026/08/22 11:18:25 dbdump.go:167: [DEBUG] DiscoverDatabases: found 3 database(s), skipped 12 non-DB container(s)
+2026/08/22 11:18:25 dbdump.go:170: [INFO] [backup] Discovered 3 databases
+2026/08/22 11:18:25 manager.go:1067: [DEBUG] [stacks] StopStack privatebin: current state=running deployed=true containers=1
+2026/08/22 11:18:25 manager.go:1070: [INFO] [stacks] Stopping stack: privatebin
+2026/08/22 11:18:25 manager.go:1261: [DEBUG] Env vars for compose: [PATH, HOSTNAME, FELHOM_BOOTSTRAP_PATH, HOME, DOMAIN, SUBDOMAIN, DOMAIN, IMPORT_PATH] (8 app + 0 system)
+2026/08/22 11:18:25 manager.go:1270: [DEBUG] Running: docker compose down (in /opt/docker/stacks/privatebin)
+2026/08/22 11:18:25 manager.go:1290: [DEBUG] Command completed: docker compose down (took 0.3s)
+2026/08/22 11:18:25 manager.go:1079: [INFO] [stacks] Stack privatebin stopped successfully (took 0.3s)
+2026/08/22 11:18:25 manager.go:621: [INFO] [stacks] Status refresh: 13 containers across 56 stacks
+2026/08/22 11:18:25 restore.go:140: [INFO] [backup] Restoring Docker volume privatebin_privatebin_data for privatebin
+2026/08/22 11:18:26 restore.go:169: [DEBUG] [backup] Volume privatebin_privatebin_data restored successfully
+2026/08/22 11:18:26 restore.go:174: [INFO] [backup] Restored 1 Docker volume(s) for privatebin
+2026/08/22 11:18:26 manager.go:986: [DEBUG] [stacks] StartStack privatebin: current state=stopped deployed=true
+2026/08/22 11:18:26 manager.go:989: [INFO] [stacks] Starting stack: privatebin
+2026/08/22 11:18:26 manager.go:996: [DEBUG] [stacks] StartStack privatebin: prepared 8 env vars for compose
+2026/08/22 11:18:26 manager.go:1261: [DEBUG] Env vars for compose: [PATH, HOSTNAME, FELHOM_BOOTSTRAP_PATH, HOME, DOMAIN, DOMAIN, SUBDOMAIN, IMPORT_PATH] (8 app + 0 system)
+2026/08/22 11:18:26 manager.go:1270: [DEBUG] Running: docker compose up -d (in /opt/docker/stacks/privatebin)
+2026/08/22 11:18:26 manager.go:1290: [DEBUG] Command completed: docker compose up -d (took 0.3s)
+2026/08/22 11:18:26 manager.go:1004: [INFO] [stacks] Stack privatebin started successfully (took 0.3s)
+2026/08/22 11:18:26 manager.go:621: [INFO] [stacks] Status refresh: 14 containers across 56 stacks
+2026/08/22 11:18:29 manager.go:1261: [DEBUG] Env vars for compose: [PATH, HOSTNAME, FELHOM_BOOTSTRAP_PATH, HOME, DOMAIN, DOMAIN, SUBDOMAIN, IMPORT_PATH] (8 app + 0 system)
+2026/08/22 11:18:29 manager.go:1270: [DEBUG] Running: docker compose ps -a --format table {{.Name}}	{{.Image}}	{{.State}}	{{.Status}} (in /opt/docker/stacks/privatebin)
+2026/08/22 11:18:29 manager.go:621: [INFO] [stacks] Status refresh: 14 containers across 56 stacks
+2026/08/22 11:18:29 restore.go:204: [DEBUG] [backup] Post-restore health check: privatebin not yet running, waiting...
+2026/08/22 11:18:29 manager.go:1290: [DEBUG] Command completed: docker compose ps -a --format table {{.Name}}	{{.Image}}	{{.State}}	{{.Status}} (took 0.1s)
+2026/08/22 11:18:29 manager.go:1350: [INFO] [stacks] Stack privatebin post-start status:
+2026/08/22 11:18:29 manager.go:1353: [INFO] [stacks]   privatebin   privatebin/pdo:2.0.5   running   Up 3 seconds (health: starting)
+2026/08/22 11:18:32 scheduler.go:67: [DEBUG] [scheduler] job health-probes: execution starting
+2026/08/22 11:18:32 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping kimai — last check 3m30s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:18:32 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping opengist — last check 3m50s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:18:32 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping romm — last check 2m50s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:18:32 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping calibre-web — last check 4m0s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:18:32 healthprobe.go:76: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (4 skipped not due, 1 skipped no container)
+2026/08/22 11:18:32 scheduler.go:67: [DEBUG] [scheduler] job status-refresh: execution starting
+2026/08/22 11:18:32 manager.go:621: [INFO] [stacks] Status refresh: 14 containers across 56 stacks
+2026/08/22 11:18:34 manager.go:621: [INFO] [stacks] Status refresh: 14 containers across 56 stacks
+2026/08/22 11:18:34 restore.go:199: [DEBUG] [backup] Post-restore health check: privatebin is running
+2026/08/22 11:18:34 offbox_reconstitute.go:452: [INFO] [offbox] reconstituted privatebin from snapshot 306accff: 0 file(s) placed, 0 DB dump(s) replayed, safety dump=., skewed=false
+2026/08/22 11:18:34 offbox_handlers.go:455: [INFO] [web] off-box reconstitute privatebin completed (async): files=0 dbs=0 snapshot=306accff
+2026/08/22 11:18:42 scheduler.go:67: [DEBUG] [scheduler] job ring-spill: execution starting
+2026/08/22 11:18:42 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
+2026/08/22 11:18:42 scheduler.go:67: [DEBUG] [scheduler] job health-probes: execution starting
+2026/08/22 11:18:42 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping opengist — last check 4m0s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:18:42 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping romm — last check 3m0s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:18:42 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping calibre-web — last check 4m10s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:18:42 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping kimai — last check 3m40s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:18:42 healthprobe.go:76: [DEBUG] [stacks] RunHealthProbes: collected 1 targets (4 skipped not due, 1 skipped no container)
+2026/08/22 11:18:42 scheduler.go:67: [DEBUG] [scheduler] job status-refresh: execution starting
+2026/08/22 11:18:42 healthprobe.go:153: [DEBUG] Health probe privatebin: HTTP GET :8080/ → 200 (5ms)
+2026/08/22 11:18:42 manager.go:621: [INFO] [stacks] Status refresh: 14 containers across 56 stacks
+2026/08/22 11:18:42 healthprobe.go:133: [INFO] Health probes: 1 ok (of 1 probed)
+2026/08/22 11:18:42 auth.go:146: [DEBUG] [web] login attempt from 172.18.0.4:60064 (X-Forwarded-For: 172.18.0.1)
+2026/08/22 11:18:42 auth.go:194: [DEBUG] [web] login successful from 172.18.0.1, creating session
+2026/08/22 11:18:42 auth.go:260: [DEBUG] [web] session created, expires=2026-08-29T11:18:42Z, active_sessions=31
+2026/08/22 11:18:42 auth.go:214: [INFO] [web] Login from 172.18.0.4:60064
+2026/08/22 11:18:42 auth.go:134: [DEBUG] [web] auth: valid session for GET /
+2026/08/22 11:18:42 server.go:393: [DEBUG] [web] ServeHTTP: GET / from 172.18.0.4:60064
+2026/08/22 11:18:42 auth.go:134: [DEBUG] [web] auth: valid session for GET /launcher
+2026/08/22 11:18:42 server.go:393: [DEBUG] [web] ServeHTTP: GET /launcher from 172.18.0.4:60064
+2026/08/22 11:18:42 auth.go:134: [DEBUG] [web] auth: valid session for GET /backup/offbox/status
+2026/08/22 11:18:42 server.go:393: [DEBUG] [web] ServeHTTP: GET /backup/offbox/status from 172.18.0.4:60064
+2026/08/22 11:18:43 client.go:1142: [DEBUG] [agentapi] GET /disks -> 200 (589ms)
+2026/08/22 11:18:44 client.go:1142: [DEBUG] [agentapi] GET /disks -> 200 (601ms)
+2026/08/22 11:18:52 scheduler.go:67: [DEBUG] [scheduler] job status-refresh: execution starting
+2026/08/22 11:18:52 scheduler.go:67: [DEBUG] [scheduler] job health-probes: execution starting
+2026/08/22 11:18:52 manager.go:621: [INFO] [stacks] Status refresh: 14 containers across 56 stacks
+2026/08/22 11:18:52 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping calibre-web — last check 4m20s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:18:52 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping kimai — last check 3m50s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:18:52 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping opengist — last check 4m10s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:18:52 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping privatebin — last check 10s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:18:52 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping romm — last check 3m10s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:18:52 healthprobe.go:76: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (5 skipped not due, 1 skipped no container)
+2026/08/22 11:19:02 scheduler.go:67: [DEBUG] [scheduler] job health-probes: execution starting
+2026/08/22 11:19:02 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping calibre-web — last check 4m30s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:19:02 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping kimai — last check 4m0s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:19:02 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping opengist — last check 4m20s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:19:02 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping privatebin — last check 20s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:19:02 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping romm — last check 3m20s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:19:02 healthprobe.go:76: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (5 skipped not due, 1 skipped no container)
+2026/08/22 11:19:02 scheduler.go:67: [DEBUG] [scheduler] job status-refresh: execution starting
+2026/08/22 11:19:02 manager.go:621: [INFO] [stacks] Status refresh: 14 containers across 56 stacks
+2026/08/22 11:19:04 auth.go:146: [DEBUG] [web] login attempt from 172.18.0.4:60064 (X-Forwarded-For: 172.18.0.1)
+2026/08/22 11:19:04 auth.go:194: [DEBUG] [web] login successful from 172.18.0.1, creating session
+2026/08/22 11:19:04 auth.go:260: [DEBUG] [web] session created, expires=2026-08-29T11:19:04Z, active_sessions=32
+2026/08/22 11:19:04 auth.go:214: [INFO] [web] Login from 172.18.0.4:60064
+2026/08/22 11:19:04 auth.go:134: [DEBUG] [web] auth: valid session for GET /
+2026/08/22 11:19:04 server.go:393: [DEBUG] [web] ServeHTTP: GET / from 172.18.0.4:60064
+2026/08/22 11:19:04 auth.go:134: [DEBUG] [web] auth: valid session for GET /launcher
+2026/08/22 11:19:04 server.go:393: [DEBUG] [web] ServeHTTP: GET /launcher from 172.18.0.4:60064
+2026/08/22 11:19:04 auth.go:134: [DEBUG] [web] auth: valid session for GET /backup/offbox/status
+2026/08/22 11:19:04 server.go:393: [DEBUG] [web] ServeHTTP: GET /backup/offbox/status from 172.18.0.4:60064
+2026/08/22 11:19:06 auth.go:146: [DEBUG] [web] login attempt from 172.18.0.4:60064 (X-Forwarded-For: 172.18.0.1)
+2026/08/22 11:19:06 auth.go:194: [DEBUG] [web] login successful from 172.18.0.1, creating session
+2026/08/22 11:19:06 auth.go:260: [DEBUG] [web] session created, expires=2026-08-29T11:19:06Z, active_sessions=33
+2026/08/22 11:19:06 auth.go:214: [INFO] [web] Login from 172.18.0.4:60064
+2026/08/22 11:19:06 auth.go:134: [DEBUG] [web] auth: valid session for GET /
+2026/08/22 11:19:06 server.go:393: [DEBUG] [web] ServeHTTP: GET / from 172.18.0.4:60064
+2026/08/22 11:19:06 auth.go:134: [DEBUG] [web] auth: valid session for GET /launcher
+2026/08/22 11:19:06 server.go:393: [DEBUG] [web] ServeHTTP: GET /launcher from 172.18.0.4:60064
+2026/08/22 11:19:06 auth.go:134: [DEBUG] [web] auth: valid session for GET /backups/restore/app
+2026/08/22 11:19:06 server.go:393: [DEBUG] [web] ServeHTTP: GET /backups/restore/app from 172.18.0.4:60064
+2026/08/22 11:19:11 info.go:13: [DEBUG] [system] GetInfo starting (hddPath="/mnt/felhom-drives/hdd_1", hasCPUCollector=true)
+2026/08/22 11:19:11 info.go:13: [DEBUG] [system] readMemInfo: guest cap=25898MB (host total was 30714460KB) → used≈3600MB avail≈22298MB
+2026/08/22 11:19:11 info.go:13: [DEBUG] [system] readMemInfo: totalKB=30714460 availKB=26444532 → total=25898MB avail=22298MB used=3600MB (13.9%)
+2026/08/22 11:19:11 info.go:13: [DEBUG] [system] readDiskUsage: path="/" bsize=4096 total=68.7GB used=8.0GB avail=57.2GB (11.6%)
+2026/08/22 11:19:11 info.go:13: [DEBUG] [system] readDiskUsage: path="/mnt/felhom-drives/hdd_1" bsize=4096 total=937.8GB used=5.5GB avail=884.7GB (0.6%)
+2026/08/22 11:19:11 info.go:13: [DEBUG] [system] readLoadAvg: raw="0.95 1.25 0.92 6/825 3250" → 1m=0.95 5m=1.25 15m=0.92
+2026/08/22 11:19:11 info.go:13: [DEBUG] [system] readThermalZones: /sys — found 1 zones
+2026/08/22 11:19:11 info.go:13: [DEBUG] [system] readTemperature: found via hwmon at /sys — 57.8°C (hwmon2)
+2026/08/22 11:19:11 info.go:13: [DEBUG] [system] GetInfo done in 59ms — mem=3600MB/25898MB (13.9%), rootDisk=8.0GB/68.7GB (11.6%), load=0.95/1.25/0.92, temp=57.8°C (hwmon2), cpu=8.4%
+2026/08/22 11:19:12 scheduler.go:67: [DEBUG] [scheduler] job ring-spill: execution starting
+2026/08/22 11:19:12 scheduler.go:67: [DEBUG] [scheduler] job status-refresh: execution starting
+2026/08/22 11:19:12 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
+2026/08/22 11:19:12 scheduler.go:67: [DEBUG] [scheduler] job health-probes: execution starting
+2026/08/22 11:19:12 scheduler.go:346: [INFO] [scheduler] Running job: agent-channel-health
+2026/08/22 11:19:12 scheduler.go:67: [DEBUG] [scheduler] job agent-channel-health: execution starting
+2026/08/22 11:19:12 manager.go:621: [INFO] [stacks] Status refresh: 14 containers across 56 stacks
+2026/08/22 11:19:12 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping calibre-web — last check 4m40s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:19:12 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping kimai — last check 4m10s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:19:12 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping opengist — last check 4m30s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:19:12 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping privatebin — last check 30s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:19:12 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping romm — last check 3m30s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:19:12 healthprobe.go:76: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (5 skipped not due, 1 skipped no container)
+2026/08/22 11:19:12 client.go:1142: [DEBUG] [agentapi] GET /storage -> 200 (561ms)
+2026/08/22 11:19:12 scheduler.go:363: [INFO] [scheduler] Job agent-channel-health completed (took 561ms)
+2026/08/22 11:19:13 client.go:1142: [DEBUG] [agentapi] GET /disks -> 200 (660ms)
+2026/08/22 11:19:14 client.go:1142: [DEBUG] [agentapi] GET /disks -> 200 (674ms)
+2026/08/22 11:19:22 scheduler.go:67: [DEBUG] [scheduler] job health-probes: execution starting
+2026/08/22 11:19:22 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping opengist — last check 4m40s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:19:22 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping privatebin — last check 40s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:19:22 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping romm — last check 3m40s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:19:22 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping calibre-web — last check 4m50s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:19:22 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping kimai — last check 4m20s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:19:22 healthprobe.go:76: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (5 skipped not due, 1 skipped no container)
+2026/08/22 11:19:22 scheduler.go:67: [DEBUG] [scheduler] job status-refresh: execution starting
+2026/08/22 11:19:22 manager.go:621: [INFO] [stacks] Status refresh: 14 containers across 56 stacks
+2026/08/22 11:19:25 auth.go:146: [DEBUG] [web] login attempt from 172.18.0.4:60064 (X-Forwarded-For: 172.18.0.1)
+2026/08/22 11:19:25 auth.go:194: [DEBUG] [web] login successful from 172.18.0.1, creating session
+2026/08/22 11:19:25 auth.go:260: [DEBUG] [web] session created, expires=2026-08-29T11:19:25Z, active_sessions=34
+2026/08/22 11:19:25 auth.go:214: [INFO] [web] Login from 172.18.0.4:60064
+2026/08/22 11:19:25 auth.go:134: [DEBUG] [web] auth: valid session for GET /
+2026/08/22 11:19:25 server.go:393: [DEBUG] [web] ServeHTTP: GET / from 172.18.0.4:60064
+2026/08/22 11:19:25 auth.go:134: [DEBUG] [web] auth: valid session for GET /launcher
+2026/08/22 11:19:25 server.go:393: [DEBUG] [web] ServeHTTP: GET /launcher from 172.18.0.4:60064
+2026/08/22 11:19:25 auth.go:134: [DEBUG] [web] auth: valid session for GET /backup/offbox/status
+2026/08/22 11:19:25 server.go:393: [DEBUG] [web] ServeHTTP: GET /backup/offbox/status from 172.18.0.4:60064
+2026/08/22 11:19:32 scheduler.go:67: [DEBUG] [scheduler] job health-probes: execution starting
+2026/08/22 11:19:32 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping calibre-web — last check 5m0s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:19:32 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping kimai — last check 4m30s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:19:32 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping opengist — last check 4m50s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:19:32 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping privatebin — last check 50s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:19:32 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping romm — last check 3m50s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:19:32 healthprobe.go:76: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (5 skipped not due, 1 skipped no container)
+2026/08/22 11:19:32 scheduler.go:67: [DEBUG] [scheduler] job status-refresh: execution starting
+2026/08/22 11:19:32 manager.go:621: [INFO] [stacks] Status refresh: 14 containers across 56 stacks

documentation/audits/DRILL-r356-hot-only-restore-2026-08-22/evidence/11-13class-planted.sha256

@@ -0,0 +1,3 @@
+bc6d35efa862246b543a1e66b2a2ab4fa889bbe28a827cc30018593ca026561a  /mnt/felhom-drives/hdd_1/userdata/media/books/DRILL-r356-13class-2026-08-22/book.bin
+93defa514998c95ec4e1483b7b4935166ab8f9c6d850b464c39114118360f82d  /mnt/felhom-drives/hdd_1/userdata/media/books/DRILL-r356-13class-2026-08-22/Örkény-egyperces-r356.txt
+85c29e22129c14d03815e107fc1fceab004bf682dc030d927f24fdd68fc6ceea  /mnt/felhom-drives/hdd_1/userdata/media/books/DRILL-r356-13class-2026-08-22/SENTINEL.txt

documentation/audits/DRILL-r356-hot-only-restore-2026-08-22/evidence/12-13class-destruction.txt

@@ -0,0 +1,13 @@
+/mnt/felhom-drives/hdd_1/userdata/media/books/DRILL-r356-13class-2026-08-22/SENTINEL.txt
+/mnt/felhom-drives/hdd_1/userdata/media/books/DRILL-r356-13class-2026-08-22/book.bin
+/mnt/felhom-drives/hdd_1/userdata/media/books/DRILL-r356-13class-2026-08-22/Örkény-egyperces-r356.txt
+-- after:
+total 456
+drwxrwsr-x  4 1000 1000   4096 Aug 22 11:22 .
+drwxrwsr-x 10 root 1000   4096 Jul 26 06:17 ..
+drwxrwxr-x  3 1000 1000   4096 Aug 21 20:09 DRILL-2026-08-21
+-rw-r--r--  1 1000 1000    181 Aug  4 12:53 DRILL-SENTINEL.txt
+-rw-r--r--  1 1000 1000 413696 Aug  4 12:52 metadata.db
+-rw-r--r--  1 1000 1000  32768 Aug 22 11:20 metadata.db-shm
+-rw-r--r--  1 1000 1000      0 Aug  9 02:15 metadata.db-wal
+drwxrwxr-x  3 1000 1000   4096 Aug  9 07:59 rehearsal-2026-08-09

documentation/audits/DRILL-r356-hot-only-restore-2026-08-22/evidence/13-13class-scratch.txt

@@ -0,0 +1,20 @@
+/mnt/felhom-drives/hdd_1/backups/offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/backups/primary/calibre-web/compose/.felhom.yml
+/mnt/felhom-drives/hdd_1/backups/offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/backups/primary/calibre-web/compose/app.yaml
+/mnt/felhom-drives/hdd_1/backups/offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/backups/primary/calibre-web/compose/docker-compose.yml
+/mnt/felhom-drives/hdd_1/backups/offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/backups/primary/calibre-web/manifest.json
+/mnt/felhom-drives/hdd_1/backups/offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/backups/primary/calibre-web/volume-dumps/calibre-web_calibre_web_config.tar
+/mnt/felhom-drives/hdd_1/backups/offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/userdata/media/books/DRILL-2026-08-21/POST-SNAPSHOT.txt
+/mnt/felhom-drives/hdd_1/backups/offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/userdata/media/books/DRILL-2026-08-21/SENTINEL.txt
+/mnt/felhom-drives/hdd_1/backups/offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/userdata/media/books/DRILL-2026-08-21/binary-1mb.bin
+/mnt/felhom-drives/hdd_1/backups/offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/userdata/media/books/DRILL-2026-08-21/nested/őszibarack.md
+/mnt/felhom-drives/hdd_1/backups/offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/userdata/media/books/DRILL-2026-08-21/plain.txt
+/mnt/felhom-drives/hdd_1/backups/offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/userdata/media/books/DRILL-2026-08-21/árvíztűrő-tükörfúrógép.txt
+/mnt/felhom-drives/hdd_1/backups/offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/userdata/media/books/DRILL-SENTINEL.txt
+/mnt/felhom-drives/hdd_1/backups/offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/userdata/media/books/DRILL-r356-13class-2026-08-22/SENTINEL.txt
+/mnt/felhom-drives/hdd_1/backups/offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/userdata/media/books/DRILL-r356-13class-2026-08-22/book.bin
+/mnt/felhom-drives/hdd_1/backups/offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/userdata/media/books/DRILL-r356-13class-2026-08-22/Örkény-egyperces-r356.txt
+/mnt/felhom-drives/hdd_1/backups/offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/userdata/media/books/metadata.db
+/mnt/felhom-drives/hdd_1/backups/offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/userdata/media/books/metadata.db-shm
+/mnt/felhom-drives/hdd_1/backups/offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/userdata/media/books/metadata.db-wal
+/mnt/felhom-drives/hdd_1/backups/offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/userdata/media/books/rehearsal-2026-08-09/binary-3mb.bin
+/mnt/felhom-drives/hdd_1/backups/offsite-restore/calibre-web/mnt/felhom-drives/hdd_1/userdata/media/books/rehearsal-2026-08-09/nested/őszibarack.md

documentation/audits/DRILL-r356-hot-only-restore-2026-08-22/evidence/14-13class-restored.sha256

@@ -0,0 +1,3 @@
+85c29e22129c14d03815e107fc1fceab004bf682dc030d927f24fdd68fc6ceea  ./SENTINEL.txt
+bc6d35efa862246b543a1e66b2a2ab4fa889bbe28a827cc30018593ca026561a  ./book.bin
+93defa514998c95ec4e1483b7b4935166ab8f9c6d850b464c39114118360f82d  ./Örkény-egyperces-r356.txt

documentation/audits/DRILL-r356-hot-only-restore-2026-08-22/evidence/15-13class-outcome-message.txt

@@ -0,0 +1,7 @@
+=== 13-CLASS OUTCOME MESSAGE, VERBATIM ===
+A(z) calibre-web: 3 fájl és 1 adatkötet visszaállítva (mentés: 2026-08-22 13:20) — az alkalmazás újraindult. Ennek az alkalmazásnak nincs adatbázisa.
+
+=== UTF-8 hex ===
+41287a292063616c696272652d7765623a20332066c3a16a6c20c3a973203120616461746bc3b674657420766973737a61c3a16c6cc3ad74766120286d656e74c3a9733a20323032362d30382d32322031333a32302920e2809420617a20616c6b616c6d617ac3a17320c3ba6a7261696e64756c742e20456e6e656b20617a20616c6b616c6d617ac3a1736e616b206e696e6373206164617462c3a17a6973612e
+
+=== byte length: 161

documentation/audits/DRILL-r356-hot-only-restore-2026-08-22/evidence/16-controller-log-phase2.txt

@@ -0,0 +1,500 @@
+2026/08/22 11:22:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "homepage" deployed=false composePath=/opt/docker/stacks/homepage/docker-compose.yml
+2026/08/22 11:22:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "immich" deployed=false composePath=/opt/docker/stacks/immich/docker-compose.yml
+2026/08/22 11:22:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "jellyfin" deployed=false composePath=/opt/docker/stacks/jellyfin/docker-compose.yml
+2026/08/22 11:22:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "kimai" deployed=true composePath=/opt/docker/stacks/kimai/docker-compose.yml
+2026/08/22 11:22:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "komga" deployed=false composePath=/opt/docker/stacks/komga/docker-compose.yml
+2026/08/22 11:22:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "mealie" deployed=false composePath=/opt/docker/stacks/mealie/docker-compose.yml
+2026/08/22 11:22:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "n8n" deployed=false composePath=/opt/docker/stacks/n8n/docker-compose.yml
+2026/08/22 11:22:12 scheduler.go:346: [INFO] [scheduler] Running job: agent-channel-health
+2026/08/22 11:22:12 scheduler.go:67: [DEBUG] [scheduler] job agent-channel-health: execution starting
+2026/08/22 11:22:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "navidrome" deployed=false composePath=/opt/docker/stacks/navidrome/docker-compose.yml
+2026/08/22 11:22:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "nextcloud" deployed=false composePath=/opt/docker/stacks/nextcloud/docker-compose.yml
+2026/08/22 11:22:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "onlyoffice" deployed=false composePath=/opt/docker/stacks/onlyoffice/docker-compose.yml
+2026/08/22 11:22:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "opengist" deployed=true composePath=/opt/docker/stacks/opengist/docker-compose.yml
+2026/08/22 11:22:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "outline" deployed=false composePath=/opt/docker/stacks/outline/docker-compose.yml
+2026/08/22 11:22:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "paperless-ngx" deployed=true composePath=/opt/docker/stacks/paperless-ngx/docker-compose.yml
+2026/08/22 11:22:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "papra" deployed=false composePath=/opt/docker/stacks/papra/docker-compose.yml
+2026/08/22 11:22:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "plant-it" deployed=false composePath=/opt/docker/stacks/plant-it/docker-compose.yml
+2026/08/22 11:22:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "plex" deployed=false composePath=/opt/docker/stacks/plex/docker-compose.yml
+2026/08/22 11:22:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "privatebin" deployed=true composePath=/opt/docker/stacks/privatebin/docker-compose.yml
+2026/08/22 11:22:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "radarr" deployed=false composePath=/opt/docker/stacks/radarr/docker-compose.yml
+2026/08/22 11:22:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "rallly" deployed=false composePath=/opt/docker/stacks/rallly/docker-compose.yml
+2026/08/22 11:22:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "recipe-importer" deployed=false composePath=/opt/docker/stacks/recipe-importer/docker-compose.yml
+2026/08/22 11:22:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "romm" deployed=true composePath=/opt/docker/stacks/romm/docker-compose.yml
+2026/08/22 11:22:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "seerr" deployed=false composePath=/opt/docker/stacks/seerr/docker-compose.yml
+2026/08/22 11:22:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "sonarr" deployed=false composePath=/opt/docker/stacks/sonarr/docker-compose.yml
+2026/08/22 11:22:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "sparkyfitness" deployed=false composePath=/opt/docker/stacks/sparkyfitness/docker-compose.yml
+2026/08/22 11:22:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "tandoor" deployed=false composePath=/opt/docker/stacks/tandoor/docker-compose.yml
+2026/08/22 11:22:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "termix" deployed=false composePath=/opt/docker/stacks/termix/docker-compose.yml
+2026/08/22 11:22:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "traefik" deployed=false composePath=/opt/docker/stacks/traefik/docker-compose.yml
+2026/08/22 11:22:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "uptime-kuma" deployed=false composePath=/opt/docker/stacks/uptime-kuma/docker-compose.yml
+2026/08/22 11:22:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "vaultwarden" deployed=false composePath=/opt/docker/stacks/vaultwarden/docker-compose.yml
+2026/08/22 11:22:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "vikunja" deployed=false composePath=/opt/docker/stacks/vikunja/docker-compose.yml
+2026/08/22 11:22:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "wanderer" deployed=false composePath=/opt/docker/stacks/wanderer/docker-compose.yml
+2026/08/22 11:22:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "wger" deployed=false composePath=/opt/docker/stacks/wger/docker-compose.yml
+2026/08/22 11:22:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "wishlist" deployed=false composePath=/opt/docker/stacks/wishlist/docker-compose.yml
+2026/08/22 11:22:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "zipline" deployed=false composePath=/opt/docker/stacks/zipline/docker-compose.yml
+2026/08/22 11:22:12 manager.go:1497: [DEBUG] [stacks] getCatalogTemplateSlugs: found 53 template slugs in /opt/docker/felhom-controller/data/catalog-cache/templates
+2026/08/22 11:22:12 manager.go:539: [DEBUG] [stacks] ScanStacks: catalog has 53 template slugs for orphan detection
+2026/08/22 11:22:12 manager.go:570: [INFO] [stacks] ScanStacks complete: 56 stacks found (6 deployed, 50 available)
+2026/08/22 11:22:12 manager.go:621: [INFO] [stacks] Status refresh: 14 containers across 56 stacks
+2026/08/22 11:22:12 scheduler.go:363: [INFO] [scheduler] Job stack-scan completed (took 44ms)
+2026/08/22 11:22:12 manager.go:621: [INFO] [stacks] Status refresh: 14 containers across 56 stacks
+2026/08/22 11:22:12 client.go:1142: [DEBUG] [agentapi] GET /storage -> 200 (557ms)
+2026/08/22 11:22:12 scheduler.go:363: [INFO] [scheduler] Job agent-channel-health completed (took 557ms)
+2026/08/22 11:22:13 settings.go:697: [DEBUG] [settings] saved to /opt/docker/felhom-controller/data/settings.json (5736 bytes)
+2026/08/22 11:22:13 settings.go:700: [INFO] [settings] Settings saved
+2026/08/22 11:22:13 settings.go:697: [DEBUG] [settings] saved to /opt/docker/felhom-controller/data/settings.json (5730 bytes)
+2026/08/22 11:22:13 settings.go:700: [INFO] [settings] Settings saved
+2026/08/22 11:22:13 offbox.go:1129: [INFO] [offbox] backup OK: 6 app(s) backed up, 27 snapshot(s), 2m8s
+2026/08/22 11:22:13 offbox_progress.go:342: [INFO] [offbox] manual run progress reporting ended after 2m14s
+2026/08/22 11:22:13 client.go:1142: [DEBUG] [agentapi] GET /disks -> 200 (679ms)
+2026/08/22 11:22:14 auth.go:146: [DEBUG] [web] login attempt from 172.18.0.4:36778 (X-Forwarded-For: 172.18.0.1)
+2026/08/22 11:22:14 auth.go:146: [DEBUG] [web] login attempt from 172.18.0.4:60064 (X-Forwarded-For: 172.18.0.1)
+2026/08/22 11:22:14 auth.go:194: [DEBUG] [web] login successful from 172.18.0.1, creating session
+2026/08/22 11:22:14 auth.go:260: [DEBUG] [web] session created, expires=2026-08-29T11:22:14Z, active_sessions=54
+2026/08/22 11:22:14 auth.go:214: [INFO] [web] Login from 172.18.0.4:36778
+2026/08/22 11:22:14 auth.go:134: [DEBUG] [web] auth: valid session for GET /
+2026/08/22 11:22:14 server.go:393: [DEBUG] [web] ServeHTTP: GET / from 172.18.0.4:36778
+2026/08/22 11:22:14 auth.go:134: [DEBUG] [web] auth: valid session for GET /launcher
+2026/08/22 11:22:14 server.go:393: [DEBUG] [web] ServeHTTP: GET /launcher from 172.18.0.4:36778
+2026/08/22 11:22:14 auth.go:194: [DEBUG] [web] login successful from 172.18.0.1, creating session
+2026/08/22 11:22:14 auth.go:260: [DEBUG] [web] session created, expires=2026-08-29T11:22:14Z, active_sessions=55
+2026/08/22 11:22:14 auth.go:214: [INFO] [web] Login from 172.18.0.4:60064
+2026/08/22 11:22:14 auth.go:134: [DEBUG] [web] auth: valid session for GET /backup/offbox/status
+2026/08/22 11:22:14 server.go:393: [DEBUG] [web] ServeHTTP: GET /backup/offbox/status from 172.18.0.4:60064
+2026/08/22 11:22:14 auth.go:134: [DEBUG] [web] auth: valid session for GET /
+2026/08/22 11:22:14 server.go:393: [DEBUG] [web] ServeHTTP: GET / from 172.18.0.4:60064
+2026/08/22 11:22:14 auth.go:134: [DEBUG] [web] auth: valid session for GET /launcher
+2026/08/22 11:22:14 server.go:393: [DEBUG] [web] ServeHTTP: GET /launcher from 172.18.0.4:60064
+2026/08/22 11:22:14 auth.go:134: [DEBUG] [web] auth: valid session for GET /backup/offbox/status
+2026/08/22 11:22:14 server.go:393: [DEBUG] [web] ServeHTTP: GET /backup/offbox/status from 172.18.0.4:60064
+2026/08/22 11:22:14 client.go:1142: [DEBUG] [agentapi] GET /disks -> 200 (681ms)
+2026/08/22 11:22:14 auth.go:146: [DEBUG] [web] login attempt from 172.18.0.4:60064 (X-Forwarded-For: 172.18.0.1)
+2026/08/22 11:22:15 auth.go:194: [DEBUG] [web] login successful from 172.18.0.1, creating session
+2026/08/22 11:22:15 auth.go:260: [DEBUG] [web] session created, expires=2026-08-29T11:22:15Z, active_sessions=56
+2026/08/22 11:22:15 auth.go:214: [INFO] [web] Login from 172.18.0.4:60064
+2026/08/22 11:22:15 auth.go:134: [DEBUG] [web] auth: valid session for GET /
+2026/08/22 11:22:15 server.go:393: [DEBUG] [web] ServeHTTP: GET / from 172.18.0.4:60064
+2026/08/22 11:22:15 auth.go:134: [DEBUG] [web] auth: valid session for GET /launcher
+2026/08/22 11:22:15 server.go:393: [DEBUG] [web] ServeHTTP: GET /launcher from 172.18.0.4:60064
+2026/08/22 11:22:15 auth.go:134: [DEBUG] [web] auth: valid session for GET /backup/offbox/status
+2026/08/22 11:22:15 server.go:393: [DEBUG] [web] ServeHTTP: GET /backup/offbox/status from 172.18.0.4:60064
+2026/08/22 11:22:15 auth.go:146: [DEBUG] [web] login attempt from 172.18.0.4:60064 (X-Forwarded-For: 172.18.0.1)
+2026/08/22 11:22:15 auth.go:194: [DEBUG] [web] login successful from 172.18.0.1, creating session
+2026/08/22 11:22:15 auth.go:260: [DEBUG] [web] session created, expires=2026-08-29T11:22:15Z, active_sessions=57
+2026/08/22 11:22:15 auth.go:214: [INFO] [web] Login from 172.18.0.4:60064
+2026/08/22 11:22:15 auth.go:134: [DEBUG] [web] auth: valid session for GET /
+2026/08/22 11:22:15 server.go:393: [DEBUG] [web] ServeHTTP: GET / from 172.18.0.4:60064
+2026/08/22 11:22:15 auth.go:134: [DEBUG] [web] auth: valid session for GET /launcher
+2026/08/22 11:22:15 server.go:393: [DEBUG] [web] ServeHTTP: GET /launcher from 172.18.0.4:60064
+2026/08/22 11:22:15 auth.go:134: [DEBUG] [web] auth: valid session for GET /backup/offbox/status
+2026/08/22 11:22:15 server.go:393: [DEBUG] [web] ServeHTTP: GET /backup/offbox/status from 172.18.0.4:60064
+2026/08/22 11:22:22 scheduler.go:67: [DEBUG] [scheduler] job health-probes: execution starting
+2026/08/22 11:22:22 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping kimai — last check 1m20s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:22:22 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping opengist — last check 1m50s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:22:22 scheduler.go:67: [DEBUG] [scheduler] job status-refresh: execution starting
+2026/08/22 11:22:22 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping privatebin — last check 1m20s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:22:22 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping romm — last check 1m0s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:22:22 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping calibre-web — last check 1m50s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:22:22 healthprobe.go:76: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (5 skipped not due, 1 skipped no container)
+2026/08/22 11:22:22 manager.go:621: [INFO] [stacks] Status refresh: 14 containers across 56 stacks
+2026/08/22 11:22:28 auth.go:146: [DEBUG] [web] login attempt from 172.18.0.4:60064 (X-Forwarded-For: 172.18.0.1)
+2026/08/22 11:22:28 auth.go:194: [DEBUG] [web] login successful from 172.18.0.1, creating session
+2026/08/22 11:22:28 auth.go:260: [DEBUG] [web] session created, expires=2026-08-29T11:22:28Z, active_sessions=58
+2026/08/22 11:22:28 auth.go:214: [INFO] [web] Login from 172.18.0.4:60064
+2026/08/22 11:22:28 auth.go:134: [DEBUG] [web] auth: valid session for GET /
+2026/08/22 11:22:28 server.go:393: [DEBUG] [web] ServeHTTP: GET / from 172.18.0.4:60064
+2026/08/22 11:22:28 auth.go:134: [DEBUG] [web] auth: valid session for GET /launcher
+2026/08/22 11:22:28 server.go:393: [DEBUG] [web] ServeHTTP: GET /launcher from 172.18.0.4:60064
+2026/08/22 11:22:28 auth.go:134: [DEBUG] [web] auth: valid session for POST /backup/offbox/restore
+2026/08/22 11:22:28 server.go:393: [DEBUG] [web] ServeHTTP: POST /backup/offbox/restore from 172.18.0.4:60064
+2026/08/22 11:22:32 scheduler.go:67: [DEBUG] [scheduler] job status-refresh: execution starting
+2026/08/22 11:22:32 scheduler.go:67: [DEBUG] [scheduler] job health-probes: execution starting
+2026/08/22 11:22:32 manager.go:621: [INFO] [stacks] Status refresh: 14 containers across 56 stacks
+2026/08/22 11:22:32 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping calibre-web — last check 2m0s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:22:32 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping kimai — last check 1m30s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:22:32 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping opengist — last check 2m0s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:22:32 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping privatebin — last check 1m30s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:22:32 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping romm — last check 1m10s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:22:32 healthprobe.go:76: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (5 skipped not due, 1 skipped no container)
+2026/08/22 11:22:35 auth.go:146: [DEBUG] [web] login attempt from 172.18.0.4:60064 (X-Forwarded-For: 172.18.0.1)
+2026/08/22 11:22:35 auth.go:194: [DEBUG] [web] login successful from 172.18.0.1, creating session
+2026/08/22 11:22:35 auth.go:260: [DEBUG] [web] session created, expires=2026-08-29T11:22:35Z, active_sessions=59
+2026/08/22 11:22:35 auth.go:214: [INFO] [web] Login from 172.18.0.4:60064
+2026/08/22 11:22:35 auth.go:134: [DEBUG] [web] auth: valid session for GET /
+2026/08/22 11:22:35 server.go:393: [DEBUG] [web] ServeHTTP: GET / from 172.18.0.4:60064
+2026/08/22 11:22:35 auth.go:134: [DEBUG] [web] auth: valid session for GET /launcher
+2026/08/22 11:22:35 server.go:393: [DEBUG] [web] ServeHTTP: GET /launcher from 172.18.0.4:60064
+2026/08/22 11:22:35 auth.go:134: [DEBUG] [web] auth: valid session for GET /backup/offbox/status
+2026/08/22 11:22:35 server.go:393: [DEBUG] [web] ServeHTTP: GET /backup/offbox/status from 172.18.0.4:60064
+2026/08/22 11:22:36 auth.go:146: [DEBUG] [web] login attempt from 172.18.0.4:60064 (X-Forwarded-For: 172.18.0.1)
+2026/08/22 11:22:36 auth.go:194: [DEBUG] [web] login successful from 172.18.0.1, creating session
+2026/08/22 11:22:36 auth.go:260: [DEBUG] [web] session created, expires=2026-08-29T11:22:36Z, active_sessions=60
+2026/08/22 11:22:36 auth.go:214: [INFO] [web] Login from 172.18.0.4:60064
+2026/08/22 11:22:36 auth.go:134: [DEBUG] [web] auth: valid session for GET /
+2026/08/22 11:22:36 server.go:393: [DEBUG] [web] ServeHTTP: GET / from 172.18.0.4:60064
+2026/08/22 11:22:36 auth.go:134: [DEBUG] [web] auth: valid session for GET /launcher
+2026/08/22 11:22:36 server.go:393: [DEBUG] [web] ServeHTTP: GET /launcher from 172.18.0.4:60064
+2026/08/22 11:22:36 auth.go:134: [DEBUG] [web] auth: valid session for GET /backup/offbox/status
+2026/08/22 11:22:36 server.go:393: [DEBUG] [web] ServeHTTP: GET /backup/offbox/status from 172.18.0.4:60064
+2026/08/22 11:22:39 offbox_restore.go:270: [INFO] [offbox] restored calibre-web (b921bae3, full=true) → /mnt/felhom-drives/hdd_1/backups/offsite-restore/calibre-web
+2026/08/22 11:22:39 offbox_handlers.go:372: [INFO] [web] off-box restore calibre-web completed (full=true, async)
+2026/08/22 11:22:42 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
+2026/08/22 11:22:42 scheduler.go:67: [DEBUG] [scheduler] job status-refresh: execution starting
+2026/08/22 11:22:42 scheduler.go:67: [DEBUG] [scheduler] job health-probes: execution starting
+2026/08/22 11:22:42 scheduler.go:67: [DEBUG] [scheduler] job ring-spill: execution starting
+2026/08/22 11:22:42 manager.go:621: [INFO] [stacks] Status refresh: 14 containers across 56 stacks
+2026/08/22 11:22:42 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping calibre-web — last check 2m10s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:22:42 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping kimai — last check 1m40s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:22:42 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping opengist — last check 2m10s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:22:42 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping privatebin — last check 1m40s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:22:42 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping romm — last check 1m20s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:22:42 healthprobe.go:76: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (5 skipped not due, 1 skipped no container)
+2026/08/22 11:22:43 client.go:1142: [DEBUG] [agentapi] GET /disks -> 200 (605ms)
+2026/08/22 11:22:44 client.go:1142: [DEBUG] [agentapi] GET /disks -> 200 (596ms)
+2026/08/22 11:22:52 scheduler.go:67: [DEBUG] [scheduler] job status-refresh: execution starting
+2026/08/22 11:22:52 scheduler.go:67: [DEBUG] [scheduler] job health-probes: execution starting
+2026/08/22 11:22:52 manager.go:621: [INFO] [stacks] Status refresh: 14 containers across 56 stacks
+2026/08/22 11:22:52 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping calibre-web — last check 2m20s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:22:52 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping kimai — last check 1m50s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:22:52 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping opengist — last check 2m20s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:22:52 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping privatebin — last check 1m50s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:22:52 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping romm — last check 1m30s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:22:52 healthprobe.go:76: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (5 skipped not due, 1 skipped no container)
+2026/08/22 11:22:56 auth.go:146: [DEBUG] [web] login attempt from 172.18.0.4:60064 (X-Forwarded-For: 172.18.0.1)
+2026/08/22 11:22:56 auth.go:194: [DEBUG] [web] login successful from 172.18.0.1, creating session
+2026/08/22 11:22:56 auth.go:260: [DEBUG] [web] session created, expires=2026-08-29T11:22:56Z, active_sessions=61
+2026/08/22 11:22:56 auth.go:214: [INFO] [web] Login from 172.18.0.4:60064
+2026/08/22 11:22:56 auth.go:134: [DEBUG] [web] auth: valid session for GET /
+2026/08/22 11:22:56 server.go:393: [DEBUG] [web] ServeHTTP: GET / from 172.18.0.4:60064
+2026/08/22 11:22:56 auth.go:134: [DEBUG] [web] auth: valid session for GET /launcher
+2026/08/22 11:22:56 server.go:393: [DEBUG] [web] ServeHTTP: GET /launcher from 172.18.0.4:60064
+2026/08/22 11:22:56 auth.go:134: [DEBUG] [web] auth: valid session for GET /backup/offbox/status
+2026/08/22 11:22:56 server.go:393: [DEBUG] [web] ServeHTTP: GET /backup/offbox/status from 172.18.0.4:60064
+2026/08/22 11:22:57 auth.go:146: [DEBUG] [web] login attempt from 172.18.0.4:60064 (X-Forwarded-For: 172.18.0.1)
+2026/08/22 11:22:57 auth.go:194: [DEBUG] [web] login successful from 172.18.0.1, creating session
+2026/08/22 11:22:57 auth.go:260: [DEBUG] [web] session created, expires=2026-08-29T11:22:57Z, active_sessions=62
+2026/08/22 11:22:57 auth.go:214: [INFO] [web] Login from 172.18.0.4:60064
+2026/08/22 11:22:57 auth.go:134: [DEBUG] [web] auth: valid session for GET /
+2026/08/22 11:22:57 server.go:393: [DEBUG] [web] ServeHTTP: GET / from 172.18.0.4:60064
+2026/08/22 11:22:57 auth.go:134: [DEBUG] [web] auth: valid session for GET /launcher
+2026/08/22 11:22:57 server.go:393: [DEBUG] [web] ServeHTTP: GET /launcher from 172.18.0.4:60064
+2026/08/22 11:22:57 auth.go:134: [DEBUG] [web] auth: valid session for GET /backup/offbox/status
+2026/08/22 11:22:57 server.go:393: [DEBUG] [web] ServeHTTP: GET /backup/offbox/status from 172.18.0.4:60064
+2026/08/22 11:23:02 scheduler.go:67: [DEBUG] [scheduler] job health-probes: execution starting
+2026/08/22 11:23:02 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping romm — last check 1m40s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:23:02 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping calibre-web — last check 2m30s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:23:02 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping kimai — last check 2m0s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:23:02 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping opengist — last check 2m30s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:23:02 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping privatebin — last check 2m0s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:23:02 healthprobe.go:76: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (5 skipped not due, 1 skipped no container)
+2026/08/22 11:23:02 scheduler.go:67: [DEBUG] [scheduler] job status-refresh: execution starting
+2026/08/22 11:23:02 manager.go:621: [INFO] [stacks] Status refresh: 14 containers across 56 stacks
+2026/08/22 11:23:02 auth.go:146: [DEBUG] [web] login attempt from 172.18.0.4:60064 (X-Forwarded-For: 172.18.0.1)
+2026/08/22 11:23:02 auth.go:194: [DEBUG] [web] login successful from 172.18.0.1, creating session
+2026/08/22 11:23:02 auth.go:260: [DEBUG] [web] session created, expires=2026-08-29T11:23:02Z, active_sessions=63
+2026/08/22 11:23:02 auth.go:214: [INFO] [web] Login from 172.18.0.4:60064
+2026/08/22 11:23:02 auth.go:134: [DEBUG] [web] auth: valid session for GET /
+2026/08/22 11:23:02 server.go:393: [DEBUG] [web] ServeHTTP: GET / from 172.18.0.4:60064
+2026/08/22 11:23:02 auth.go:134: [DEBUG] [web] auth: valid session for GET /launcher
+2026/08/22 11:23:02 server.go:393: [DEBUG] [web] ServeHTTP: GET /launcher from 172.18.0.4:60064
+2026/08/22 11:23:02 auth.go:134: [DEBUG] [web] auth: valid session for POST /backup/offbox/reconstitute
+2026/08/22 11:23:02 server.go:393: [DEBUG] [web] ServeHTTP: POST /backup/offbox/reconstitute from 172.18.0.4:60064
+2026/08/22 11:23:04 dbdump.go:92: [DEBUG] DiscoverDatabases: running docker ps to find database containers
+2026/08/22 11:23:04 dbdump.go:111: [DEBUG] DiscoverDatabases: docker ps output: 5afaefbd4e91	romm	romm	rommapp/romm:5.0.0
+79cae59f7986	romm-db	romm	mariadb:11.4
+c7c079ae23af	romm-redis	romm	redis:7-alpine
+adca811fb993	privatebin	privatebin	privatebin/pdo:2.0.5
+9fe7dc9f36ae	paperless-webserver	paperless-ngx	ghcr.io/paperless-ngx/paperless-ngx:2.20.15
+4a432b653108	paperless-postgres	paperless-ngx	postgres:16-alpine
+950113e71f40	paperless-redis	paperless-ngx	redis:7-alpine
+4d81ef1c6cde	opengist	opengist	ghcr.io/thomiceli/opengist:1.13
+4df3efdb8366	kimai	kimai	kimai/kimai2:apac...
+2026/08/22 11:23:04 dbdump.go:133: [DEBUG] DiscoverDatabases: skipping container romm (image=rommapp/romm:5.0.0, not a database)
+2026/08/22 11:23:04 dbdump.go:140: [DEBUG] DiscoverDatabases: found mariadb container: romm-db (id=79cae59f7986)
+2026/08/22 11:23:04 dbdump.go:160: [DEBUG] DiscoverDatabases: romm-db → stack=romm, dbUser=root, dbName=romm
+2026/08/22 11:23:04 dbdump.go:133: [DEBUG] DiscoverDatabases: skipping container romm-redis (image=redis:7-alpine, not a database)
+2026/08/22 11:23:04 dbdump.go:133: [DEBUG] DiscoverDatabases: skipping container privatebin (image=privatebin/pdo:2.0.5, not a database)
+2026/08/22 11:23:04 dbdump.go:133: [DEBUG] DiscoverDatabases: skipping container paperless-webserver (image=ghcr.io/paperless-ngx/paperless-ngx:2.20.15, not a database)
+2026/08/22 11:23:04 dbdump.go:140: [DEBUG] DiscoverDatabases: found postgres container: paperless-postgres (id=4a432b653108)
+2026/08/22 11:23:04 dbdump.go:805: [DEBUG] DiscoverDatabases: paperless-postgres → stack "paperless-ngx" from the compose project label (the container name would have given "paperless")
+2026/08/22 11:23:05 dbdump.go:160: [DEBUG] DiscoverDatabases: paperless-postgres → stack=paperless-ngx, dbUser=paperless, dbName=paperless
+2026/08/22 11:23:05 dbdump.go:133: [DEBUG] DiscoverDatabases: skipping container paperless-redis (image=redis:7-alpine, not a database)
+2026/08/22 11:23:05 dbdump.go:133: [DEBUG] DiscoverDatabases: skipping container opengist (image=ghcr.io/thomiceli/opengist:1.13, not a database)
+2026/08/22 11:23:05 dbdump.go:133: [DEBUG] DiscoverDatabases: skipping container kimai (image=kimai/kimai2:apache-2.57.0, not a database)
+2026/08/22 11:23:05 dbdump.go:140: [DEBUG] DiscoverDatabases: found mariadb container: kimai-db (id=d3e87cbf6539)
+2026/08/22 11:23:05 dbdump.go:160: [DEBUG] DiscoverDatabases: kimai-db → stack=kimai, dbUser=root, dbName=kimai
+2026/08/22 11:23:05 dbdump.go:133: [DEBUG] DiscoverDatabases: skipping container calibre-web (image=crocodilestick/calibre-web-automated:v4.0.6, not a database)
+2026/08/22 11:23:05 dbdump.go:133: [DEBUG] DiscoverDatabases: skipping container felhom-controller (image=gitea.dooplex.hu/admin/felhom-controller:0.219.0, not a database)
+2026/08/22 11:23:05 dbdump.go:133: [DEBUG] DiscoverDatabases: skipping container filebrowser (image=gtstef/filebrowser:1.3.3-stable, not a database)
+2026/08/22 11:23:05 dbdump.go:133: [DEBUG] DiscoverDatabases: skipping container cloudflared (image=cloudflare/cloudflared:2026.6.0, not a database)
+2026/08/22 11:23:05 dbdump.go:133: [DEBUG] DiscoverDatabases: skipping container traefik (image=traefik:v3.6.7, not a database)
+2026/08/22 11:23:05 dbdump.go:167: [DEBUG] DiscoverDatabases: found 3 database(s), skipped 12 non-DB container(s)
+2026/08/22 11:23:05 dbdump.go:170: [INFO] [backup] Discovered 3 databases
+2026/08/22 11:23:05 manager.go:1067: [DEBUG] [stacks] StopStack calibre-web: current state=running deployed=true containers=1
+2026/08/22 11:23:05 manager.go:1070: [INFO] [stacks] Stopping stack: calibre-web
+2026/08/22 11:23:05 manager.go:1261: [DEBUG] Env vars for compose: [PATH, HOSTNAME, FELHOM_BOOTSTRAP_PATH, HOME, DOMAIN, HDD_PATH, SUBDOMAIN, DOMAIN, USERDATA_PATH, IMPORT_PATH] (10 app + 0 system)
+2026/08/22 11:23:05 manager.go:1270: [DEBUG] Running: docker compose down (in /opt/docker/stacks/calibre-web)
+2026/08/22 11:23:09 manager.go:1290: [DEBUG] Command completed: docker compose down (took 4.0s)
+2026/08/22 11:23:09 manager.go:1079: [INFO] [stacks] Stack calibre-web stopped successfully (took 4.0s)
+2026/08/22 11:23:09 manager.go:621: [INFO] [stacks] Status refresh: 13 containers across 56 stacks
+2026/08/22 11:23:09 restore.go:140: [INFO] [backup] Restoring Docker volume calibre-web_calibre_web_config for calibre-web
+2026/08/22 11:23:09 restore.go:169: [DEBUG] [backup] Volume calibre-web_calibre_web_config restored successfully
+2026/08/22 11:23:09 restore.go:174: [INFO] [backup] Restored 1 Docker volume(s) for calibre-web
+2026/08/22 11:23:09 manager.go:986: [DEBUG] [stacks] StartStack calibre-web: current state=stopped deployed=true
+2026/08/22 11:23:09 manager.go:989: [INFO] [stacks] Starting stack: calibre-web
+2026/08/22 11:23:09 manager.go:996: [DEBUG] [stacks] StartStack calibre-web: prepared 10 env vars for compose
+2026/08/22 11:23:09 manager.go:1261: [DEBUG] Env vars for compose: [PATH, HOSTNAME, FELHOM_BOOTSTRAP_PATH, HOME, DOMAIN, DOMAIN, HDD_PATH, SUBDOMAIN, USERDATA_PATH, IMPORT_PATH] (10 app + 0 system)
+2026/08/22 11:23:09 manager.go:1270: [DEBUG] Running: docker compose up -d (in /opt/docker/stacks/calibre-web)
+2026/08/22 11:23:09 manager.go:1290: [DEBUG] Command completed: docker compose up -d (took 0.3s)
+2026/08/22 11:23:09 manager.go:1004: [INFO] [stacks] Stack calibre-web started successfully (took 0.3s)
+2026/08/22 11:23:09 manager.go:621: [INFO] [stacks] Status refresh: 14 containers across 56 stacks
+2026/08/22 11:23:11 info.go:13: [DEBUG] [system] GetInfo starting (hddPath="/mnt/felhom-drives/hdd_1", hasCPUCollector=true)
+2026/08/22 11:23:11 info.go:13: [DEBUG] [system] readMemInfo: guest cap=25898MB (host total was 30714460KB) → used≈3326MB avail≈22572MB
+2026/08/22 11:23:11 info.go:13: [DEBUG] [system] readMemInfo: totalKB=30714460 availKB=26768972 → total=25898MB avail=22572MB used=3326MB (12.8%)
+2026/08/22 11:23:11 info.go:13: [DEBUG] [system] readDiskUsage: path="/" bsize=4096 total=68.7GB used=8.0GB avail=57.3GB (11.6%)
+2026/08/22 11:23:11 info.go:13: [DEBUG] [system] readDiskUsage: path="/mnt/felhom-drives/hdd_1" bsize=4096 total=937.8GB used=5.5GB avail=884.7GB (0.6%)
+2026/08/22 11:23:11 info.go:13: [DEBUG] [system] readLoadAvg: raw="1.11 1.44 1.09 3/776 5238" → 1m=1.11 5m=1.44 15m=1.09
+2026/08/22 11:23:11 info.go:13: [DEBUG] [system] readThermalZones: /sys — found 1 zones
+2026/08/22 11:23:11 info.go:13: [DEBUG] [system] readTemperature: found via hwmon at /sys — 55.9°C (hwmon2)
+2026/08/22 11:23:11 info.go:13: [DEBUG] [system] GetInfo done in 62ms — mem=3326MB/25898MB (12.8%), rootDisk=8.0GB/68.7GB (11.6%), load=1.11/1.44/1.09, temp=55.9°C (hwmon2), cpu=5.8%
+2026/08/22 11:23:12 scheduler.go:67: [DEBUG] [scheduler] job ring-spill: execution starting
+2026/08/22 11:23:12 scheduler.go:67: [DEBUG] [scheduler] job health-probes: execution starting
+2026/08/22 11:23:12 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping romm — last check 1m50s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:23:12 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
+2026/08/22 11:23:12 scheduler.go:67: [DEBUG] [scheduler] job status-refresh: execution starting
+2026/08/22 11:23:12 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping kimai — last check 2m10s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:23:12 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping opengist — last check 2m40s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:23:12 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping privatebin — last check 2m10s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:23:12 healthprobe.go:76: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (4 skipped not due, 1 skipped no container)
+2026/08/22 11:23:12 scheduler.go:346: [INFO] [scheduler] Running job: agent-channel-health
+2026/08/22 11:23:12 scheduler.go:67: [DEBUG] [scheduler] job agent-channel-health: execution starting
+2026/08/22 11:23:12 manager.go:621: [INFO] [stacks] Status refresh: 14 containers across 56 stacks
+2026/08/22 11:23:12 client.go:1142: [DEBUG] [agentapi] GET /storage -> 200 (580ms)
+2026/08/22 11:23:12 scheduler.go:363: [INFO] [scheduler] Job agent-channel-health completed (took 580ms)
+2026/08/22 11:23:12 manager.go:1261: [DEBUG] Env vars for compose: [PATH, HOSTNAME, FELHOM_BOOTSTRAP_PATH, HOME, DOMAIN, DOMAIN, HDD_PATH, SUBDOMAIN, USERDATA_PATH, IMPORT_PATH] (10 app + 0 system)
+2026/08/22 11:23:12 manager.go:1270: [DEBUG] Running: docker compose ps -a --format table {{.Name}}	{{.Image}}	{{.State}}	{{.Status}} (in /opt/docker/stacks/calibre-web)
+2026/08/22 11:23:12 manager.go:621: [INFO] [stacks] Status refresh: 14 containers across 56 stacks
+2026/08/22 11:23:12 restore.go:204: [DEBUG] [backup] Post-restore health check: calibre-web not yet running, waiting...
+2026/08/22 11:23:12 manager.go:1290: [DEBUG] Command completed: docker compose ps -a --format table {{.Name}}	{{.Image}}	{{.State}}	{{.Status}} (took 0.1s)
+2026/08/22 11:23:12 manager.go:1350: [INFO] [stacks] Stack calibre-web post-start status:
+2026/08/22 11:23:12 manager.go:1353: [INFO] [stacks]   calibre-web   crocodilestick/calibre-web-automated:v4.0.6   running   Up 3 seconds (health: starting)
+2026/08/22 11:23:14 client.go:1142: [DEBUG] [agentapi] GET /disks -> 200 (713ms)
+2026/08/22 11:23:14 client.go:1142: [DEBUG] [agentapi] GET /disks -> 200 (637ms)
+2026/08/22 11:23:17 manager.go:621: [INFO] [stacks] Status refresh: 14 containers across 56 stacks
+2026/08/22 11:23:17 restore.go:204: [DEBUG] [backup] Post-restore health check: calibre-web not yet running, waiting...
+2026/08/22 11:23:18 auth.go:146: [DEBUG] [web] login attempt from 172.18.0.4:60064 (X-Forwarded-For: 172.18.0.1)
+2026/08/22 11:23:18 auth.go:194: [DEBUG] [web] login successful from 172.18.0.1, creating session
+2026/08/22 11:23:18 auth.go:260: [DEBUG] [web] session created, expires=2026-08-29T11:23:18Z, active_sessions=64
+2026/08/22 11:23:18 auth.go:214: [INFO] [web] Login from 172.18.0.4:60064
+2026/08/22 11:23:18 auth.go:134: [DEBUG] [web] auth: valid session for GET /
+2026/08/22 11:23:18 server.go:393: [DEBUG] [web] ServeHTTP: GET / from 172.18.0.4:60064
+2026/08/22 11:23:18 auth.go:134: [DEBUG] [web] auth: valid session for GET /launcher
+2026/08/22 11:23:18 server.go:393: [DEBUG] [web] ServeHTTP: GET /launcher from 172.18.0.4:60064
+2026/08/22 11:23:18 auth.go:134: [DEBUG] [web] auth: valid session for GET /backup/offbox/status
+2026/08/22 11:23:18 server.go:393: [DEBUG] [web] ServeHTTP: GET /backup/offbox/status from 172.18.0.4:60064
+2026/08/22 11:23:18 auth.go:146: [DEBUG] [web] login attempt from 172.18.0.4:60064 (X-Forwarded-For: 172.18.0.1)
+2026/08/22 11:23:18 auth.go:194: [DEBUG] [web] login successful from 172.18.0.1, creating session
+2026/08/22 11:23:18 auth.go:260: [DEBUG] [web] session created, expires=2026-08-29T11:23:18Z, active_sessions=65
+2026/08/22 11:23:18 auth.go:214: [INFO] [web] Login from 172.18.0.4:60064
+2026/08/22 11:23:19 auth.go:134: [DEBUG] [web] auth: valid session for GET /
+2026/08/22 11:23:19 server.go:393: [DEBUG] [web] ServeHTTP: GET / from 172.18.0.4:60064
+2026/08/22 11:23:19 auth.go:134: [DEBUG] [web] auth: valid session for GET /launcher
+2026/08/22 11:23:19 server.go:393: [DEBUG] [web] ServeHTTP: GET /launcher from 172.18.0.4:60064
+2026/08/22 11:23:19 auth.go:134: [DEBUG] [web] auth: valid session for GET /backup/offbox/status
+2026/08/22 11:23:19 server.go:393: [DEBUG] [web] ServeHTTP: GET /backup/offbox/status from 172.18.0.4:60064
+2026/08/22 11:23:22 scheduler.go:67: [DEBUG] [scheduler] job health-probes: execution starting
+2026/08/22 11:23:22 scheduler.go:67: [DEBUG] [scheduler] job status-refresh: execution starting
+2026/08/22 11:23:22 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping opengist — last check 2m50s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:23:22 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping privatebin — last check 2m20s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:23:22 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping romm — last check 2m0s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:23:22 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping kimai — last check 2m20s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:23:22 healthprobe.go:76: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (4 skipped not due, 1 skipped no container)
+2026/08/22 11:23:22 manager.go:621: [INFO] [stacks] Status refresh: 14 containers across 56 stacks
+2026/08/22 11:23:22 manager.go:621: [INFO] [stacks] Status refresh: 14 containers across 56 stacks
+2026/08/22 11:23:22 restore.go:204: [DEBUG] [backup] Post-restore health check: calibre-web not yet running, waiting...
+2026/08/22 11:23:27 manager.go:621: [INFO] [stacks] Status refresh: 14 containers across 56 stacks
+2026/08/22 11:23:27 restore.go:199: [DEBUG] [backup] Post-restore health check: calibre-web is running
+2026/08/22 11:23:27 offbox_reconstitute.go:452: [INFO] [offbox] reconstituted calibre-web from snapshot b921bae3: 3 file(s) placed, 0 DB dump(s) replayed, safety dump=., skewed=false
+2026/08/22 11:23:27 offbox_handlers.go:455: [INFO] [web] off-box reconstitute calibre-web completed (async): files=3 dbs=0 snapshot=b921bae3
+2026/08/22 11:23:32 scheduler.go:67: [DEBUG] [scheduler] job status-refresh: execution starting
+2026/08/22 11:23:32 scheduler.go:67: [DEBUG] [scheduler] job health-probes: execution starting
+2026/08/22 11:23:32 manager.go:621: [INFO] [stacks] Status refresh: 14 containers across 56 stacks
+2026/08/22 11:23:32 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping kimai — last check 2m30s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:23:32 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping opengist — last check 3m0s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:23:32 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping privatebin — last check 2m30s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:23:32 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping romm — last check 2m10s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:23:32 healthprobe.go:76: [DEBUG] [stacks] RunHealthProbes: collected 1 targets (4 skipped not due, 1 skipped no container)
+2026/08/22 11:23:32 healthprobe.go:153: [DEBUG] Health probe calibre-web: HTTP GET :8083/ → 302 (4ms)
+2026/08/22 11:23:32 healthprobe.go:133: [INFO] Health probes: 1 ok (of 1 probed)
+2026/08/22 11:23:39 auth.go:146: [DEBUG] [web] login attempt from 172.18.0.4:60064 (X-Forwarded-For: 172.18.0.1)
+2026/08/22 11:23:39 auth.go:194: [DEBUG] [web] login successful from 172.18.0.1, creating session
+2026/08/22 11:23:39 auth.go:260: [DEBUG] [web] session created, expires=2026-08-29T11:23:39Z, active_sessions=66
+2026/08/22 11:23:39 auth.go:214: [INFO] [web] Login from 172.18.0.4:60064
+2026/08/22 11:23:39 auth.go:134: [DEBUG] [web] auth: valid session for GET /
+2026/08/22 11:23:39 server.go:393: [DEBUG] [web] ServeHTTP: GET / from 172.18.0.4:60064
+2026/08/22 11:23:39 auth.go:134: [DEBUG] [web] auth: valid session for GET /launcher
+2026/08/22 11:23:39 server.go:393: [DEBUG] [web] ServeHTTP: GET /launcher from 172.18.0.4:60064
+2026/08/22 11:23:39 auth.go:134: [DEBUG] [web] auth: valid session for GET /backup/offbox/status
+2026/08/22 11:23:39 server.go:393: [DEBUG] [web] ServeHTTP: GET /backup/offbox/status from 172.18.0.4:60064
+2026/08/22 11:23:40 auth.go:146: [DEBUG] [web] login attempt from 172.18.0.4:60064 (X-Forwarded-For: 172.18.0.1)
+2026/08/22 11:23:40 auth.go:194: [DEBUG] [web] login successful from 172.18.0.1, creating session
+2026/08/22 11:23:40 auth.go:260: [DEBUG] [web] session created, expires=2026-08-29T11:23:40Z, active_sessions=67
+2026/08/22 11:23:40 auth.go:214: [INFO] [web] Login from 172.18.0.4:60064
+2026/08/22 11:23:40 auth.go:134: [DEBUG] [web] auth: valid session for GET /
+2026/08/22 11:23:40 server.go:393: [DEBUG] [web] ServeHTTP: GET / from 172.18.0.4:60064
+2026/08/22 11:23:40 auth.go:134: [DEBUG] [web] auth: valid session for GET /launcher
+2026/08/22 11:23:40 server.go:393: [DEBUG] [web] ServeHTTP: GET /launcher from 172.18.0.4:60064
+2026/08/22 11:23:40 auth.go:134: [DEBUG] [web] auth: valid session for GET /backup/offbox/status
+2026/08/22 11:23:40 server.go:393: [DEBUG] [web] ServeHTTP: GET /backup/offbox/status from 172.18.0.4:60064
+2026/08/22 11:23:42 scheduler.go:67: [DEBUG] [scheduler] job status-refresh: execution starting
+2026/08/22 11:23:42 scheduler.go:67: [DEBUG] [scheduler] job health-probes: execution starting
+2026/08/22 11:23:42 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
+2026/08/22 11:23:42 scheduler.go:67: [DEBUG] [scheduler] job ring-spill: execution starting
+2026/08/22 11:23:42 manager.go:621: [INFO] [stacks] Status refresh: 14 containers across 56 stacks
+2026/08/22 11:23:42 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping opengist — last check 3m10s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:23:42 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping privatebin — last check 2m40s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:23:42 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping romm — last check 2m20s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:23:42 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping calibre-web — last check 10s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:23:42 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping kimai — last check 2m40s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:23:42 healthprobe.go:76: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (5 skipped not due, 1 skipped no container)
+2026/08/22 11:23:43 client.go:1142: [DEBUG] [agentapi] GET /disks -> 200 (622ms)
+2026/08/22 11:23:44 client.go:1142: [DEBUG] [agentapi] GET /disks -> 200 (609ms)
+2026/08/22 11:23:52 scheduler.go:67: [DEBUG] [scheduler] job health-probes: execution starting
+2026/08/22 11:23:52 scheduler.go:67: [DEBUG] [scheduler] job status-refresh: execution starting
+2026/08/22 11:23:52 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping calibre-web — last check 20s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:23:52 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping kimai — last check 2m50s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:23:52 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping opengist — last check 3m20s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:23:52 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping privatebin — last check 2m50s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:23:52 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping romm — last check 2m30s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:23:52 healthprobe.go:76: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (5 skipped not due, 1 skipped no container)
+2026/08/22 11:23:52 manager.go:621: [INFO] [stacks] Status refresh: 14 containers across 56 stacks
+2026/08/22 11:24:00 auth.go:146: [DEBUG] [web] login attempt from 172.18.0.4:60064 (X-Forwarded-For: 172.18.0.1)
+2026/08/22 11:24:00 auth.go:194: [DEBUG] [web] login successful from 172.18.0.1, creating session
+2026/08/22 11:24:00 auth.go:260: [DEBUG] [web] session created, expires=2026-08-29T11:24:00Z, active_sessions=68
+2026/08/22 11:24:00 auth.go:214: [INFO] [web] Login from 172.18.0.4:60064
+2026/08/22 11:24:00 auth.go:134: [DEBUG] [web] auth: valid session for GET /
+2026/08/22 11:24:00 server.go:393: [DEBUG] [web] ServeHTTP: GET / from 172.18.0.4:60064
+2026/08/22 11:24:00 auth.go:134: [DEBUG] [web] auth: valid session for GET /launcher
+2026/08/22 11:24:00 server.go:393: [DEBUG] [web] ServeHTTP: GET /launcher from 172.18.0.4:60064
+2026/08/22 11:24:00 auth.go:134: [DEBUG] [web] auth: valid session for GET /backup/offbox/status
+2026/08/22 11:24:00 server.go:393: [DEBUG] [web] ServeHTTP: GET /backup/offbox/status from 172.18.0.4:60064
+2026/08/22 11:24:01 auth.go:146: [DEBUG] [web] login attempt from 172.18.0.4:60064 (X-Forwarded-For: 172.18.0.1)
+2026/08/22 11:24:01 auth.go:194: [DEBUG] [web] login successful from 172.18.0.1, creating session
+2026/08/22 11:24:01 auth.go:260: [DEBUG] [web] session created, expires=2026-08-29T11:24:01Z, active_sessions=69
+2026/08/22 11:24:01 auth.go:214: [INFO] [web] Login from 172.18.0.4:60064
+2026/08/22 11:24:01 auth.go:134: [DEBUG] [web] auth: valid session for GET /
+2026/08/22 11:24:01 server.go:393: [DEBUG] [web] ServeHTTP: GET / from 172.18.0.4:60064
+2026/08/22 11:24:01 auth.go:134: [DEBUG] [web] auth: valid session for GET /launcher
+2026/08/22 11:24:01 server.go:393: [DEBUG] [web] ServeHTTP: GET /launcher from 172.18.0.4:60064
+2026/08/22 11:24:01 auth.go:134: [DEBUG] [web] auth: valid session for GET /backup/offbox/status
+2026/08/22 11:24:01 server.go:393: [DEBUG] [web] ServeHTTP: GET /backup/offbox/status from 172.18.0.4:60064
+2026/08/22 11:24:02 scheduler.go:67: [DEBUG] [scheduler] job status-refresh: execution starting
+2026/08/22 11:24:02 scheduler.go:67: [DEBUG] [scheduler] job health-probes: execution starting
+2026/08/22 11:24:02 manager.go:621: [INFO] [stacks] Status refresh: 14 containers across 56 stacks
+2026/08/22 11:24:02 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping kimai — last check 3m0s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:24:02 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping opengist — last check 3m30s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:24:02 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping privatebin — last check 3m0s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:24:02 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping romm — last check 2m40s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:24:02 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping calibre-web — last check 30s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:24:02 healthprobe.go:76: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (5 skipped not due, 1 skipped no container)
+2026/08/22 11:24:09 auth.go:146: [DEBUG] [web] login attempt from 172.18.0.4:60064 (X-Forwarded-For: 172.18.0.1)
+2026/08/22 11:24:09 auth.go:194: [DEBUG] [web] login successful from 172.18.0.1, creating session
+2026/08/22 11:24:09 auth.go:260: [DEBUG] [web] session created, expires=2026-08-29T11:24:09Z, active_sessions=70
+2026/08/22 11:24:09 auth.go:214: [INFO] [web] Login from 172.18.0.4:60064
+2026/08/22 11:24:09 auth.go:134: [DEBUG] [web] auth: valid session for GET /
+2026/08/22 11:24:09 server.go:393: [DEBUG] [web] ServeHTTP: GET / from 172.18.0.4:60064
+2026/08/22 11:24:09 auth.go:134: [DEBUG] [web] auth: valid session for GET /launcher
+2026/08/22 11:24:09 server.go:393: [DEBUG] [web] ServeHTTP: GET /launcher from 172.18.0.4:60064
+2026/08/22 11:24:09 auth.go:134: [DEBUG] [web] auth: valid session for GET /backups/restore/app
+2026/08/22 11:24:09 server.go:393: [DEBUG] [web] ServeHTTP: GET /backups/restore/app from 172.18.0.4:60064
+2026/08/22 11:24:11 info.go:13: [DEBUG] [system] GetInfo starting (hddPath="/mnt/felhom-drives/hdd_1", hasCPUCollector=true)
+2026/08/22 11:24:11 info.go:13: [DEBUG] [system] readMemInfo: guest cap=25898MB (host total was 30714460KB) → used≈3555MB avail≈22343MB
+2026/08/22 11:24:11 info.go:13: [DEBUG] [system] readMemInfo: totalKB=30714460 availKB=26497444 → total=25898MB avail=22343MB used=3555MB (13.7%)
+2026/08/22 11:24:11 info.go:13: [DEBUG] [system] readDiskUsage: path="/" bsize=4096 total=68.7GB used=8.0GB avail=57.2GB (11.6%)
+2026/08/22 11:24:11 info.go:13: [DEBUG] [system] readDiskUsage: path="/mnt/felhom-drives/hdd_1" bsize=4096 total=937.8GB used=5.5GB avail=884.7GB (0.6%)
+2026/08/22 11:24:11 info.go:13: [DEBUG] [system] readLoadAvg: raw="0.99 1.36 1.08 3/810 5442" → 1m=0.99 5m=1.36 15m=1.08
+2026/08/22 11:24:11 info.go:13: [DEBUG] [system] readThermalZones: /sys — found 1 zones
+2026/08/22 11:24:11 info.go:13: [DEBUG] [system] readTemperature: found via hwmon at /sys — 53.9°C (hwmon2)
+2026/08/22 11:24:11 info.go:13: [DEBUG] [system] GetInfo done in 61ms — mem=3555MB/25898MB (13.7%), rootDisk=8.0GB/68.7GB (11.6%), load=0.99/1.36/1.08, temp=53.9°C (hwmon2), cpu=1.4%
+2026/08/22 11:24:12 scheduler.go:346: [INFO] [scheduler] Running job: stack-scan
+2026/08/22 11:24:12 scheduler.go:67: [DEBUG] [scheduler] job deadapp-check: execution starting
+2026/08/22 11:24:12 scheduler.go:67: [DEBUG] [scheduler] job stack-scan: execution starting
+2026/08/22 11:24:12 scheduler.go:67: [DEBUG] [scheduler] job health-probes: execution starting
+2026/08/22 11:24:12 scheduler.go:67: [DEBUG] [scheduler] job ring-spill: execution starting
+2026/08/22 11:24:12 scheduler.go:67: [DEBUG] [scheduler] job status-refresh: execution starting
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "actualbudget" deployed=false composePath=/opt/docker/stacks/actualbudget/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "adventurelog" deployed=false composePath=/opt/docker/stacks/adventurelog/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "audiobookshelf" deployed=false composePath=/opt/docker/stacks/audiobookshelf/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "bentopdf" deployed=false composePath=/opt/docker/stacks/bentopdf/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "bookstack" deployed=false composePath=/opt/docker/stacks/bookstack/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "calcom" deployed=false composePath=/opt/docker/stacks/calcom/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "calibre-web" deployed=true composePath=/opt/docker/stacks/calibre-web/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "claper" deployed=false composePath=/opt/docker/stacks/claper/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "cloudflared" deployed=false composePath=/opt/docker/stacks/cloudflared/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "code-server" deployed=false composePath=/opt/docker/stacks/code-server/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "crafty-controller" deployed=false composePath=/opt/docker/stacks/crafty-controller/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "docmost" deployed=false composePath=/opt/docker/stacks/docmost/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "emby" deployed=false composePath=/opt/docker/stacks/emby/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "filebrowser" deployed=false composePath=/opt/docker/stacks/filebrowser/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "ghost" deployed=false composePath=/opt/docker/stacks/ghost/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "gitea" deployed=false composePath=/opt/docker/stacks/gitea/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "glance" deployed=false composePath=/opt/docker/stacks/glance/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "gokapi" deployed=false composePath=/opt/docker/stacks/gokapi/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "grafana" deployed=false composePath=/opt/docker/stacks/grafana/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "gramps-web" deployed=false composePath=/opt/docker/stacks/gramps-web/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "home-assistant" deployed=false composePath=/opt/docker/stacks/home-assistant/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "homebox" deployed=false composePath=/opt/docker/stacks/homebox/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "homepage" deployed=false composePath=/opt/docker/stacks/homepage/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "immich" deployed=false composePath=/opt/docker/stacks/immich/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "jellyfin" deployed=false composePath=/opt/docker/stacks/jellyfin/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "kimai" deployed=true composePath=/opt/docker/stacks/kimai/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "komga" deployed=false composePath=/opt/docker/stacks/komga/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "mealie" deployed=false composePath=/opt/docker/stacks/mealie/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "n8n" deployed=false composePath=/opt/docker/stacks/n8n/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "navidrome" deployed=false composePath=/opt/docker/stacks/navidrome/docker-compose.yml
+2026/08/22 11:24:12 scheduler.go:346: [INFO] [scheduler] Running job: agent-channel-health
+2026/08/22 11:24:12 scheduler.go:67: [DEBUG] [scheduler] job agent-channel-health: execution starting
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "nextcloud" deployed=false composePath=/opt/docker/stacks/nextcloud/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "onlyoffice" deployed=false composePath=/opt/docker/stacks/onlyoffice/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "opengist" deployed=true composePath=/opt/docker/stacks/opengist/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "outline" deployed=false composePath=/opt/docker/stacks/outline/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "paperless-ngx" deployed=true composePath=/opt/docker/stacks/paperless-ngx/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "papra" deployed=false composePath=/opt/docker/stacks/papra/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "plant-it" deployed=false composePath=/opt/docker/stacks/plant-it/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "plex" deployed=false composePath=/opt/docker/stacks/plex/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "privatebin" deployed=true composePath=/opt/docker/stacks/privatebin/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "radarr" deployed=false composePath=/opt/docker/stacks/radarr/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "rallly" deployed=false composePath=/opt/docker/stacks/rallly/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "recipe-importer" deployed=false composePath=/opt/docker/stacks/recipe-importer/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "romm" deployed=true composePath=/opt/docker/stacks/romm/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "seerr" deployed=false composePath=/opt/docker/stacks/seerr/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "sonarr" deployed=false composePath=/opt/docker/stacks/sonarr/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "sparkyfitness" deployed=false composePath=/opt/docker/stacks/sparkyfitness/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "tandoor" deployed=false composePath=/opt/docker/stacks/tandoor/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "termix" deployed=false composePath=/opt/docker/stacks/termix/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "traefik" deployed=false composePath=/opt/docker/stacks/traefik/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "uptime-kuma" deployed=false composePath=/opt/docker/stacks/uptime-kuma/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "vaultwarden" deployed=false composePath=/opt/docker/stacks/vaultwarden/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "vikunja" deployed=false composePath=/opt/docker/stacks/vikunja/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "wanderer" deployed=false composePath=/opt/docker/stacks/wanderer/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "wger" deployed=false composePath=/opt/docker/stacks/wger/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "wishlist" deployed=false composePath=/opt/docker/stacks/wishlist/docker-compose.yml
+2026/08/22 11:24:12 manager.go:502: [DEBUG] [stacks] ScanStacks: found stack "zipline" deployed=false composePath=/opt/docker/stacks/zipline/docker-compose.yml
+2026/08/22 11:24:12 manager.go:1497: [DEBUG] [stacks] getCatalogTemplateSlugs: found 53 template slugs in /opt/docker/felhom-controller/data/catalog-cache/templates
+2026/08/22 11:24:12 manager.go:539: [DEBUG] [stacks] ScanStacks: catalog has 53 template slugs for orphan detection
+2026/08/22 11:24:12 manager.go:570: [INFO] [stacks] ScanStacks complete: 56 stacks found (6 deployed, 50 available)
+2026/08/22 11:24:12 manager.go:621: [INFO] [stacks] Status refresh: 14 containers across 56 stacks
+2026/08/22 11:24:12 scheduler.go:363: [INFO] [scheduler] Job stack-scan completed (took 47ms)
+2026/08/22 11:24:12 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping romm — last check 2m50s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:24:12 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping calibre-web — last check 40s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:24:12 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping kimai — last check 3m10s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:24:12 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping opengist — last check 3m40s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:24:12 healthprobe.go:53: [DEBUG] [stacks] RunHealthProbes: skipping privatebin — last check 3m10s ago, effective interval 5m0s, healthy=true
+2026/08/22 11:24:12 healthprobe.go:76: [DEBUG] [stacks] RunHealthProbes: collected 0 targets (5 skipped not due, 1 skipped no container)
+2026/08/22 11:24:12 manager.go:621: [INFO] [stacks] Status refresh: 14 containers across 56 stacks
+2026/08/22 11:24:12 client.go:1142: [DEBUG] [agentapi] GET /storage -> 200 (559ms)
+2026/08/22 11:24:12 scheduler.go:363: [INFO] [scheduler] Job agent-channel-health completed (took 559ms)
+2026/08/22 11:24:13 client.go:1142: [DEBUG] [agentapi] GET /disks -> 200 (660ms)
+2026/08/22 11:24:14 client.go:1142: [DEBUG] [agentapi] GET /disks -> 200 (679ms)