diff --git a/REPORT.md b/REPORT.md index 3a31e41..447d0fe 100644 --- a/REPORT.md +++ b/REPORT.md @@ -1,190 +1,85 @@ -# REPORT — D5: an app restore works from the drive alone (controller v0.188.0, 2026-07-30) +# REPORT — gate enforcement: the `-v` allowlist, one entry point, a pre-push hook (2026-08-02) -**Overwritten** per the standing rule. Controller **v0.187.0 → v0.188.0**, MinAgent 0.113.0 (unchanged). +**Overwritten** per the standing rule. The prior contents (D5, controller v0.188.0, 2026-07-30) have +their durable record in `CHANGELOG.md`; nothing was lost by this overwrite. -**Outcome: SHIPPED + PROVEN-LIVE.** Tier-1/Tier-2 no longer depend on the whole-guest tier. -Commit `4ed938c`; image `gitea.dooplex.hu/admin/felhom-controller:0.188.0` -(digest `sha256:f297fbf0…`). Full record: -`felhom.eu/documentation/audits/D5-drive-alone-restore-2026-07-30.md`. +**No version bump, no image build, no deploy.** Nothing compiled changed — this touches +`controller/scripts/` and `.githooks/` only, so no behaviour on any box moved. Stated explicitly so +the omission reads as a decision rather than a miss. Controller stays **v0.188.0**; guest 9201 was +not touched. -## 1. Baselines +## What changed -| Repo | State at start | +| Commit | What | |---|---| -| `felhom-controller` | clean, `HEAD == origin/main` at `2f27a36` (v0.187.0, R-108) | -| `app-catalog-felhom.eu` | `4252121`, 53 templates | -| Golden-baked controller | **0.186.0** — a release behind the fleet's 0.187.0 | -| R-120 vouch gate | **did NOT block.** It lives in `hub/internal/web/configs.go` `handleSetArtifacts` — the golden **vouch** form — and never runs on a controller image deploy. Re-baking is a **follow-on so fresh installs get D5**, not a prerequisite; it is its own ceremony, and bundling it here would have put two unrelated risky ceremonies in one session | +| `c432f70` | **the ALLOWLIST entry, alone in its own reviewed diff** | +| `8cb3d7a` | `controller/scripts/controller_gates.py` + `controller/scripts/test_controller_gates.py` + `.githooks/pre-push` + `CLAUDE.md` | +| `7c32c74` | `CHANGELOG.md` + `controller/README.md` | -## 2. The §1.1 / §1.2 rulings — the brief's recommendation was tested and rejected +Baseline on arrival: `4115e88f688b`, clean, `HEAD == origin/main` — matched the spec's anchor. -Both needed the operator; both were asked before any code was written. +### 1. `docker_run_volume_path_gate.py` — one allowlist entry (R-29 leg (a), now CLOSED) -**§1.1 — what travels.** The brief proposed `data_key`-only. Two findings killed it: +The gate was **RED**, flagging `internal/appexport/estimate.go:179`. The finding is benign and was +reviewed rather than suppressed: `realVolumeSize` mounts a **named Docker volume** read-only into a +throwaway `alpine` to `du` it from a container view. No host path is involved — the daemon resolves +the volume name daemon-side — which makes it structurally identical to the already-allowlisted +`internal/backup/backup.go` entry `'"-v", volName+":/vol:ro"'`. The entry carries its WHY in the +voice of its neighbours. -1. **The flag is unreliable → R-127.** Only 5 fields across 4 apps set `data_key: true`, yet - `n8n/N8N_ENCRYPTION_KEY` („Titkosítási kulcs"), `wanderer/POCKETBASE_ENCRYPTION_KEY` („Adatbázis - titkosítási kulcs"), `calcom/CALENDSO_ENCRYPTION_KEY` and `bookstack/APP_KEY` are unflagged — the - catalog's own labels contradict the flag. Data-keys-only would omit real data keys, and the - fail-closed gate would not fire for them. -2. **A DB password is not resettable in practice — proven on `postgres:16-alpine`.** With PGDATA - restored from the volume tar, `POSTGRES_PASSWORD` is **ignored** (initdb skipped): the replay - **succeeds** via the container's local `trust` socket, the app **fails** over the compose network - (`FATAL: password authentication failed`), and the **old** password still works. Success reported, - data unreachable. 18 DB/root-password fields; MariaDB fails louder (Access denied). *(A first - attempt tested `-h 127.0.0.1` and got a false pass — the image trusts localhost. Recorded because it - is the same class of false observable this task exists to catch.)* +`realVolumeSize` was **not** rewritten: the code is correct, the allowlist was incomplete. The diff +is alone in its commit because burying an allowlist widening inside a feature commit is how an +allowlist stops meaning anything. -**Ruling (operator):** `type: secret` travels (45 fields); `type: password` never (7) plus the -`nonPortableSecrets` code register (`vaultwarden/ADMIN_TOKEN`). +**Red-proof:** renaming the new entry's suffix to a non-existent file returns the gate to **exit 1**, +naming `estimate.go:179`; restored → **exit 0**. -**§1.2 — plaintext**, as the data already is. Defensible on D2's own logic *because* the -internet-reachable class is withheld: every travelling secret decrypts data on the same drive or -authenticates to a container on an internal compose network with no external listener. **The exclusion -licenses the plaintext — the two are coupled and must not be relaxed independently.** +### 2. `controller_gates.py` — THE entry point -## 3. What a customer must possess to complete a Tier-1/2 restore +Runs all seven local gates plus `reuse_refs_check` on the repo root, streams each gate's own output, +exits worst-wins non-zero, and reports exit 2 distinctly as INCONCLUSIVE. `--fast` selects the gates +that touch no network and no container runtime — today all eight. **A missing gate script is a +FAILURE with the path printed, never a skip.** -**The drive.** Nothing else. +Why a runner at all: the 2026-08-02 census across the four repos found that **every gate a +`CLAUDE.md` names was green, and the ones nobody names were where the red was.** This repo had seven +gates and `CLAUDE.md` named two; four more lived behind a line in `REUSE.md`, and the docker-`-v` +gate behind one line in `REUSE.md` — while red. The canonical shape is +`app-catalog-felhom.eu/scripts/catalog_gates.py` (R-161). -## 4. Files modified — commit `4ed938c`, version 0.188.0 +**The shared checker is never copied here.** `reuse_refs_check.py` stays in `felhom.eu/scripts/` and +is invoked at `/../felhom.eu/scripts/`; a copy would recreate exactly the drift it detects. +It now resolves this repo's REUSE map cleanly: **133 cited paths — 126 exact, 6 by suffix, 1 +cross-repo** (`wgsync/reconciler.go`, which lives in the hub), 0 failures. -| File | Change | -|---|---| -| `internal/stacks/deploy.go` | **`PortableSecretEnvVars`** + the `nonPortableSecrets` register — the single boundary, in code not catalog (R-97a) | -| `cmd/controller/main.go` | `GetStackRecoveryInfo` decrypts the portable class via the **same** `LoadAppConfigDecrypted` the restore side uses | -| `internal/appbackup/appdata.go` | `RecoveryInfo` gains `PortableSecretEnvVars` + `PortableSecrets`; stale "SECRET-FREE" comment corrected | -| `internal/backup/recovery_unit.go` | `buildStrippedAppYaml` → **`buildUnitAppYaml`** (0600, names the withheld class in its header); manifest **schema 2** + `portable_secret_env_vars` | -| `internal/backup/restore_unit.go` | `readStrippedEnv` → **`readUnitEnv`** (manifest-driven split); `reconcileRestoreSecrets` takes `unitSecrets` as an **argument** and stays pure; **precedence documented** | -| `internal/appbackup/paths.go` | stale "the unit is secret-free" corrected | -| `CHANGELOG.md`, `REUSE.md`, `controller/README.md` | updated same-commit | +### 3. `.githooks/pre-push` -**Precedence: the UNIT WINS** — its secrets were captured in the same run as the dumps beside them, so -they match *the data being restored*; the guest's is merely newest. **The fail-closed gate is -unchanged.** Schema-1 units still restore from the guest. +Runs `controller_gates.py --fast` and refuses the push. It prints a line before and after, because an +absent log line is not evidence a hook ran. Its limits are written into the hook itself: **per-clone** +(`core.hooksPath` is local config; this clone is armed — `git config --get core.hooksPath` → +`.githooks`) and **`git push --no-verify` bypasses it on purpose**, which must be stated in a session +report when used. It was **not** used against this repo; both real pushes ran the hook and passed. -Three comments asserting invariants D5 makes false were corrected rather than left reading as settled, -and the O4 WARN that claimed *"stored data is unaffected"* for every non-data-key secret was scoped to -what §2 shows is true. +### 4. `test_controller_gates.py` — a seam test, not a smoke test -## 5. Tests — suite green, `rc=0` (read on its own; never combined with a commit) +Asserts each member gate's **own distinctive stdout**, never the runner's summary line, which an +inert runner prints while calling nothing. **Red-proof:** replacing `run_gate`'s body with `return 0` +still prints `all controller gates OK` and exits 0 — and turns the seam test red. It also pins that +`reuse_refs_check.py` has not been copied into this repo. -| Test | Asserts | -|---|---| -| `TestRestoreFromRecoveryUnitWithGuestAbsent` | **the consequence** — restore SUCCEEDS with the guest absent | -| `TestRestoreFromRecoveryUnitGuestAbsentStillFailsClosed` | data key in neither source ⇒ refuse; nothing mutated, app not even stopped | -| `TestReconcileRestoreSecretsPrecedence` | unit-wins; guest fills in when unit is silent/empty; no shadowing by stray plain config | -| `TestCaptureRecoveryUnitCarriesPortableSecretsOnly` | portable present, 0600, schema 2, **and the withheld value appears NOWHERE** | -| `TestReadUnitEnvSplitsByManifest` | manifest-driven split; schema-1 degrades losslessly | -| `TestPortableSecretEnvVars` | the boundary both directions; register is slug-scoped | -| `…Orchestration/schema-1 unit still restores` | no regression for pre-D5 units | +## Verification -**Seam (R-125): `Manager.stackProvider` only** — docker/compose ops + the guest app.yaml decrypt. -`RecoverStackSecrets` returning nil **is** the guest being gone (what the real adapter returns when the -stack or its app.yaml can't be read). Fixtures come from a unit written by the **real** -`CaptureRecoveryUnit`, so capture and restore meet at real bytes rather than a test agreeing with itself. +``` + template-id OK (exit 0) app-row-dedup OK (exit 0) + emoji OK (exit 0) mojibake OK (exit 0) + native-confirm OK (exit 0) docker-v OK (exit 0) + offbox-rename OK (exit 0) reuse-refs OK (exit 0) -**Four red-proofs, each mutation verified present in the working file before the run, then reverted:** +all controller gates OK +``` -| # | Mutation | Result | -|---|---|---| -| 1 | drop the `PortableSecrets` merge | consequence + capture tests FAIL | -| 2 | `if false &&` the fail-closed gate | **4** failures | -| 3 | flip precedence to guest-first | unit-wins test FAILS (`got "guest-key"`) | -| 4 | widen the class to `type: password` | boundary test FAILS | +`python3 scripts/test_controller_gates.py` → **4 tests, OK** (0 before). No Go test count changed; no +Go file was touched. -## 6. The live restore — with the guest's `app.yaml` unavailable - -**Venue: a scratch drill LXC (9301) on felhom-pve**, destroyed after. Correct class per -`target-selection.md`: D5's claim is about **restore code**, not the install path/golden/first-boot, so -**fixture, not fresh day-0**. demo-hp could not host it — its `local` storage has no `rootdir` content -type and `local-lvm` was fenced by the task. Guest 9201 (both hosts) and `drill-r50` untouched. - -Real endpoints only: `POST /api/stacks/{app}/deploy` → `POST /api/backup/run` → `POST /backup/restore`. - -**Portable half — AdventureLog (`SECRET_KEY` data_key + `DB_PASSWORD`):** - - Recovery unit captured … (data_keys=1, portable-carried=2/2, withheld=0) - Restoring adventurelog from recovery unit: secrets recovered=2/2, data_keys=1 - Restore-from-unit completed: adventurelog (27.6 s) - -`2/2` with the guest's `app.yaml` moved aside. Pre-D5 that was `0/2` and the gate refused. - -**The data is readable — the observable that matters:** - - APP-READ-OK rows= ['D5-CUSTOMER-DATA-2026-07-30'] - connected_as= adventurelog over_TCP= True row_count= 1 - -Read through **Django's own connection, as the app, over TCP** — not the localhost trust socket that -produced §2's false pass. Plus `manage.py check --database default` clean and 51 `public` tables intact. - -**The discriminator held:** the row seeded *before* the backup returned; a row inserted *after* it was -**gone** — so the tar was genuinely restored, not left alone. The unit held **no `.sql` dump**, so the DB -came back from the **volume tar** — exactly the case a regenerated password breaks silently. - -**Withheld half — Grafana (`GF_SECURITY_ADMIN_PASSWORD`, `type: password`):** `withheld=1, -portable-carried=0/0`. A sentinel value was live in the container and `ENC:` in the guest — so the -absence is not vacuous — and appeared in **0 files** in the unit and **0 files** anywhere under the -backup namespace. The unit's app.yaml header names it as withheld. - -**Not exercised live:** O4 regeneration of a withheld secret on restore, and Tier-2's cross-drive copy of -a secret-bearing unit (both unit-tested only). - -## 7. Does Tier-1/2 still depend on the whole-guest tier? - -**No — for the app-rebuild path.** One leg remains, named rather than glossed: **Tier-2's additive -file-merge** still reads its destination from the guest's `settings.json` -(`tier2_restore.go:114-116`), and Tier-3's reconstitution still requires the app deployed. **D5 removed -the secrets leg, not the living-app leg.** - -## 8. Four-artifact update — confirmed (felhom.eu `0a9bd38`) - -`architecture/07-backup-architecture.md` (owning doc: new **§7.4**, §7.1 leg 1 superseded, §7.3 → history -with the rejected-premise correction, §3, matrix rows **3 / 3c (new) / 13**, §10.1) · -`00-capability-map.md` (new PROVEN-LIVE row) · `ROADMAP.md` (D5 collapsed, R-127 filed) · -`OPEN-ITEMS.md` (D5 collapsed, R-127 filed) · plus the audit. - -## 9. Arc status — the definition of done IS met - -**R-106**, **R-109** (agent v0.118.1 / hub v0.83.0), **R-108** (v0.187.0) and **D5** (v0.188.0) are all -SHIPPED + PROVEN-LIVE. - -Open, none part of that definition: **R-105** (three hub-held DR records empty fleet-wide), **R-121** -(installed-vs-vouched agent drift), **R-123** (PARTLY CLOSED), **R-124** (PBS root namespace `"root"` vs -`""`), **R-126** (`.fab` export onto NAS), **R-117g** (no guided recovery for an aborted fs), -**R-117h** (`StablePathForRaw` hardcodes the live parent), **R-127** (new). -**R-119 does not exist** in either backlog file — reported absent rather than given an invented status. - -## 10. Teardown — all three layers - -| Layer | Disposition | -|---|---| -| Guest | LXC **9301 destroyed** (`--purge`), `vm-9301-disk-0` removed; `pct list` shows only 9201 | -| Host | no leftover `/tmp` files; thin pool back to **27.88 %**, identical to pre-drill | -| Hub | **nothing to dispose** — drill ran `hub.enabled: false`, never registered. Verified positively: `/hosts` + `/configs` loaded (9880 / 10160 bytes) with **0** `d5drill` occurrences | - -Secret-bearing files **shredded** (`shred -u -z`) in the guest before the volume was destroyed: both -units' `compose/app.yaml`, both stack `app.yaml`s, `encryption.key`, the hidden app.yaml, every -transported helper — plus host tarballs and local scratchpad scripts. - -## 11. Findings - -- **R-127 (NEW, READY)** — filed, not fixed: (a) `data_key` unreliable; (b) O4 can regenerate a DB - password that no longer matches the restored data directory. Both sit outside D5's ruling. -- D5's own defect surface — the false *"stored data is unaffected"* WARN and three stale - invariant-asserting comments — was corrected in place, being the defect this task was inside. - -## 12. What did not happen, and why - -- **Golden re-bake** — not done. R-120's gate does not apply (§1); it is a follow-on for fresh installs, - and its own ceremony. -- **A fresh day-0 box** — not used. `target-selection.md` makes fixture-class correct for a restore-code - claim; a fresh box would also have installed the pre-D5 golden. -- **R-127 not fixed** — a catalog change plus a separate guard decision, outside the ruling. -- **Escrow / whole-guest / offsite / hub role** — untouched, as fenced. The offsite copy carries the - secrets inside the unit it already pushed (restic-encrypted under the customer-owned password); that is - a recorded consequence, not a change. -- **One self-inflicted slip:** an inspection script's redaction regex assumed 2-space YAML indentation - while the unit's app.yaml uses 4, so the drill's two secret values printed to the session transcript - once. Throwaway values, in a guest that no longer exists, in **no committed file** — but the guard was - mine to get right. A redaction pattern is itself a claim and needs checking against the real format. +Full cross-repo detail, every red-proof, the `core.hooksPath` probe measurements and the hub +deployment: `felhom.eu/REPORT.md`.