359 lines
24 KiB
Markdown
359 lines
24 KiB
Markdown
# REPORT — R-178: both demo boxes reinstalled from the merged golden and proven (2026-08-03)
|
||
|
||
**Overwritten** per the standing rule. The prior contents (R-165, the bake, 2026-08-03) have their
|
||
durable record in `documentation/backlog/OPEN-ITEMS.md` R-165 and the per-repo CHANGELOGs.
|
||
|
||
**Runbook, not a task.** No repo got a version bump and nothing was built. One artifact was
|
||
**published** (agent 0.120.0) on an explicit operator ruling — see §2. All code findings are filed as
|
||
register rows, not commits, per the runbook's §7.
|
||
|
||
---
|
||
|
||
## 1. Preconditions P1–P6, each as measured
|
||
|
||
| # | Precondition | Measurement |
|
||
|---|---|---|
|
||
| **P1** | Golden sha256 matches R-178's record | **PASS** — computed from the artifact itself on demo-hp: `sha256sum` → `54e2a4c431daf580d2807b82d810be36a7c6a9f697b27094dc2912d26a43b3e0`, 649,547,835 B. Matches R-178's recorded prefix and the hub manifest's stored value exactly. Re-verified after copying to `local`: identical |
|
||
| **P2** | A rollback golden still exists | **PASS, three depths.** Split-layout golden **0.188.0** fetchable from Gitea (`HTTP 200`, 649,310,288 B). A **local** split-layout golden sat on each box — `local:backup/vzdump-lxc-9100-2026_07_21-18_24_49.tar.zst` (demo-hp), `…2026_07_20-17_50_57…` (demo-felhom). Best: **full guest vzdumps**, three per box, newest `vzdump-lxc-9201-2026_08_03-04_54_28.tar.zst` (2,351,557,870 B, demo-hp) and `…2026_08_03-04_44_50…` (6,323,506,165 B, demo-felhom), both ~3 h old — so each box could be put back exactly as it was |
|
||
| **P3** | Neither demo box holds anything wanted | **PASS, stated explicitly.** demo-hp: paperless-ngx (webserver/postgres/redis) + filebrowser + samba shares. demo-felhom: immich (×4), docmost (×3), calibre-web, bookstack (×2) + filebrowser + samba. Both are demo customers (`Demo HP`, `Demo Ügyfél`); no real customer data. **All of it was destroyed by the wipes and none of it was restored** — that was the point, and the operator confirmed each wipe |
|
||
| **P4** | The colleague's box untouched | **PASS** — `peti-felhom` is Tier 2 and out of scope. No command was sent to it; it appears in the hub customer list as `DOWN`, exactly as before |
|
||
| **P5** | Both hosts' agent is v0.120.0 | **PASS** — `felhom-agent 0.120.0` on both, before and after |
|
||
| **P6** | The hub's host register, BEFORE | **Captured**: `demo-felhom-8363b5 / Demo Ügyfél / 0.120.0 / ONLINE / 1-of-2 guests`; `demo-hp-bb76ea / Demo HP / 0.120.0 / ONLINE / 1-of-2`; `drill-r50-0a4f9a / drill-r50 / 0.113.0 / DOWN` (the fixture, untouched throughout) |
|
||
|
||
---
|
||
|
||
## 2. Two findings that contradicted the runbook's premise, both surfaced before any wipe
|
||
|
||
**(a) The golden was ALREADY vouched.** Hub log, a positive observable:
|
||
`2026/08/03 07:23:26 [INFO] Artifact manifest set: agent=0.119.0 golden=0.192.0 min_agent="0.113.0"
|
||
wrapper_sha=true` — roughly ten minutes before this session's first hub read, and this session had
|
||
POSTed nothing. The configuration page confirmed `Artifacts.GoldenVersion=0.192.0`,
|
||
`GoldenSHA256=54e2a4c4…3b3e0`. So §7's prove-then-vouch order was already spent, and Phase A's stated
|
||
safety ("nothing is official yet, so a failure reaches nobody") was void. **Operator ruling: accept it
|
||
and proceed**, keeping the two different supply paths. Recorded on `CONTEXT.md` S-14, now marked SPENT
|
||
with the reason — the rule lived only in prose and nothing enforced it.
|
||
|
||
**(b) Agent v0.120.0 had never been published — the vouched agent was 0.119.0.**
|
||
`GET …/generic/felhom-agent/0.120.0/felhom-agent` → **HTTP 404** (0.119.0 → 200). Installer step 5's
|
||
idempotent skip requires `installed == vouched` exactly, so a documented-path reinstall would have
|
||
**downgraded both boxes** to the pre-merge 0.119.0 — and would have *succeeded* while doing it, since
|
||
the current `step_grows` sets `SYSDATA_GROW=0` and 0.119.0's `mp1` resize (`bringup.go` 4c, fatal on
|
||
error) therefore never fires. The session would have proven a stack nobody ships. **Operator ruling:
|
||
publish and vouch first.** `scripts/publish-agent.sh 0.120.0` from a clean tree at `4bb84fc3`
|
||
(`git status --porcelain` empty, `HEAD == origin/main`): upload HTTP 201, **round-trip GET verified**,
|
||
`AGENT_SHA256=a7763d31b55b5ce75457b4dba7b06aa300325811834b0be78af4587b47110b9d`. Vouched via
|
||
`POST /configuration/artifacts` → `303 …flash=artifacts_set`, hub log
|
||
`Artifact manifest set: agent=0.120.0 golden=0.192.0`; the hub resolved the sha authoritatively from
|
||
Gitea rather than trusting the submitted value. Filed as the **third instance of R-115**, not a new
|
||
ID — R-115 is that row, and it has been WAITING-ON-OPERATOR since 2026-07-29.
|
||
|
||
---
|
||
|
||
## 3. demo-hp — the layout proof
|
||
|
||
**Install path.** Uninstall: `./felhom-host-install.sh --uninstall --vmid 9201` (typed-vmid
|
||
confirmation supplied over a pty). Install:
|
||
|
||
```
|
||
./felhom-host-install.sh --customer-id demo-hp --mode appliance --vmid 9201 \
|
||
--cores 7 --memory 26906 \
|
||
--golden local:backup/vzdump-lxc-9100-2026_08_03-07_33_00.tar.zst \
|
||
--passphrase-file /root/.pp-demo-hp
|
||
```
|
||
|
||
Script fetched from `https://felhom.eu/scripts/felhom-host-install.sh`, **v1.22.0**, sha256
|
||
`ed02acb2da46c8d2b5c486ce99d5b9a2747e8786c6eb03652cf755ed1abdd9f4` — byte-identical to the repo copy
|
||
that was read. The passphrase went file→file into a 0600 file and never onto a command line. A
|
||
`--dry-run` preceded the real run and resolved `manifest: agent v0.120.0 (sha a7763d31…), golden
|
||
v0.192.0` with `grows: rootfs +0G (->32G), data +46G (->70G, ONE volume)`.
|
||
|
||
**One deviation, mine, and it cost a restart.** The first attempt staged the golden on
|
||
`felhom-backup` with `--archive-storage felhom-backup`. Pre-flight passed; **step 8/8** failed:
|
||
`HTTP 403: permission denied at /storage/felhom-backup (missing privilege Datastore.AllocateSpace)` —
|
||
the default pool-scoped ACL grants `local local-lvm felhom-pbs` only. Fixed by copying the golden to
|
||
`local` (sha re-verified after the copy) and `--resume`. Filed as **R-180**: the condition is
|
||
statically checkable in pre-flight, and the failure lands *after* step 4b has rotated and vaulted
|
||
root@pam.
|
||
|
||
**Layout evidence.**
|
||
|
||
```
|
||
mp0: local-lvm:vm-9201-disk-1,mp=/var/lib/felhom,backup=1,size=70G # and NO mp1 line
|
||
rootfs: local-lvm:vm-9201-disk-0,size=32G
|
||
|
||
/var/lib/felhom /dev/mapper/pve-vm--9201--disk--1 ext4 rw,relatime,stripe=16
|
||
/var/lib/docker /dev/mapper/pve-vm--9201--disk--1[/docker] ext4
|
||
/mnt/sys_drive /dev/mapper/pve-vm--9201--disk--1[/sys_drive] ext4
|
||
|
||
Filesystem Size Used Avail Use% Mounted on
|
||
/dev/mapper/pve-vm--9201--disk--1 69G 977M 65G 2% /var/lib/felhom
|
||
/dev/mapper/pve-vm--9201--disk--1 69G 977M 65G 2% /var/lib/docker
|
||
/dev/mapper/pve-vm--9201--disk--1 69G 977M 65G 2% /mnt/sys_drive
|
||
|
||
stat -c %d → 64519 for all three paths
|
||
/etc/fstab: /var/lib/felhom/docker → /var/lib/docker ; /var/lib/felhom/sys_drive → /mnt/sys_drive
|
||
both binds writable (touch succeeded on each)
|
||
```
|
||
|
||
`/mnt/sys_drive` shows a second `findmnt` row — it is the controller container's
|
||
`-v /mnt:/mnt:rslave` propagation (peer group 383 vs the fstab bind's 333), the same shape the split
|
||
layout had, not a stacked bind.
|
||
|
||
**Reboots — each individually, minimum three:**
|
||
|
||
| # | started | controller healthy | outcome |
|
||
|---|---|---|---|
|
||
| 1 | 08:13:33 | 08:13:55, `Up 12 seconds (healthy)` | one filesystem, 69G/65G on both paths |
|
||
| 2 | 08:13:59 | 08:14:15, `Up 7 seconds (healthy)` | same |
|
||
| 3 | 08:14:19 | 08:14:35, `Up 7 seconds (healthy)` | same |
|
||
|
||
After all three, `mountpoint -q` returns true for **all three paths** and both binds still resolve to
|
||
the single volume's subdirectories — which is what the reboots exist to test. `uptime -s` =
|
||
`2026-08-03 06:14:24 UTC`, matching reboot 3, so these were real reboots.
|
||
|
||
**The journey — method stated: endpoint-level, not a browser.** `claude-in-chrome` does not exist on
|
||
DooPlex; every step below invoked the exact endpoint the dashboard's own JavaScript calls.
|
||
|
||
| Leg | Endpoint | Observable |
|
||
|---|---|---|
|
||
| Claim | `POST /claim` with the pre-auth HMAC CSRF token (64 chars) **and** its `felhom_claim_csrf` cookie | `302 → /`; gate discriminator flipped `{"error":"dashboard not yet claimed"}` → `{"error":"authentication required"}`. **The code is emailed-only (R-119) — the operator supplied it**, after an operator resend rotated the generation (the previous code had been consumed 12 d earlier) |
|
||
| Deploy | `POST /api/stacks/<n>/deploy` with `{"values":{…}}`, field values scraped from the deploy form exactly as the form's own `fetch` does | `{"ok":true}`; `privatebin Up (healthy)`, then `opengist Up (healthy)` |
|
||
| Back up | `POST /api/debug/backup/dbdump` — this runs the **production** `RunDBDumps` path (DB leg → `runVolumeDumps` → `captureAllRecoveryUnits`); the debug route only starts it instead of waiting for 03:30 | `Volume dump: opengist/… → 178.0 KB`, `privatebin/… → 2.5 KB`, `App-data backup completed … 2 volume dump(s) (3.962s)`, `Recovery unit captured for …` ×2 |
|
||
| **Restore** | `POST /backup/restore` `stack_name=privatebin snapshot_id=primary` | A marker planted in the live volume was **deleted**, then restored: `{"ok":true,"message":"privatebin visszaállítva (primary)."}` in **9.2 s**, and the marker returned with an **identical sha256 `ac1faae6134871d640d5cb1bbc6b5092d1e96b7e46c6c3bcb6e8236cc33ae861`**. App `Up (healthy)` afterwards |
|
||
|
||
**Recovery unit path, and it lands on the single volume:**
|
||
`/mnt/sys_drive/felhom-data/backups/primary/<app>/{manifest.json,compose/,volume-dumps/}`, whose `df`
|
||
is `/dev/mapper/pve-vm--9201--disk--1` — the merged volume.
|
||
|
||
**Note on the first app chosen.** `privatebin`'s catalog entry declares no `backup:` section, so its
|
||
first per-stack capture produced `"volume_dumps": null` — correct for that declaration, not a defect,
|
||
but it means a per-stack `POST /stacks/<n>/backup` writes compose+config only; the volume leg lives in
|
||
the full pass. A second app (`opengist`) was deployed so the run had both a capture and, later, a
|
||
refusal.
|
||
|
||
**The ceiling is gone, measured:** a recovery unit can use **65 GiB** — the whole volume — against the
|
||
**19 GiB** the pre-wipe `mp1` slice offered (`/dev/mapper/pve-vm--9201--disk--2 20G 95M 19G 1%`).
|
||
|
||
---
|
||
|
||
## 4. The floor's first live firing — and it does not do what it says
|
||
|
||
**Instrument, proven before use.** demo-hp's thin pool is **53.93 GiB** and the auto-sized volume is
|
||
70 G, so a real fill to 97 % would have exhausted the pool and corrupted every guest on the box,
|
||
including the protected `drill-r50` fixture. A 5 GiB `fallocate` probe moved guest `df` from
|
||
`977M used / 65G avail` to `6.0G / 60G` while thin-pool `data_percent` stayed **29.03 → 29.03** —
|
||
zero blocks allocated — and cleanup returned both to baseline. The floor reads `statfs`, which is
|
||
exactly what `fallocate` moves, so the condition it guards is genuinely present.
|
||
|
||
**Setup.** A **real** 2 GiB file in opengist's data volume (so its capture writes 2 GiB), then
|
||
`fallocate` to leave `69G / 63G used / 3.0G avail / 96%` — **both floor terms deliberately still
|
||
clear**, so the run would start. Capture order was established empirically from the previous run's log
|
||
(opengist first, privatebin second), not assumed.
|
||
|
||
**What happened, 06:40:03:**
|
||
|
||
```
|
||
Volume dump: opengist/opengist_opengist_data → 2.0 GB # unguarded
|
||
Volume dump: privatebin/privatebin_privatebin_data → 2.5 KB
|
||
App-data backup completed … 2 volume dump(s) (20.737s)
|
||
[WARN] Recovery unit capture REFUSED for opengist — … 1.0 GB free; the previous unit is untouched and NOTHING was deleted
|
||
[WARN] Recovery unit capture REFUSED for privatebin — … 1.0 GB free; the previous unit is untouched and NOTHING was deleted
|
||
[INFO] Event pushed: recovery_unit_capture_failed (error) — … ×2 (HTTP 200)
|
||
```
|
||
|
||
**What holds:** it refuses **per app** rather than aborting the run; **nothing was deleted** (both
|
||
units present afterwards); and the operator alert reached the hub — `recovery_unit_capture_failed`,
|
||
severity `error`, accepted `HTTP 200`, twice.
|
||
|
||
**What does not hold — measured, not inferred.** The refusal message claims *"the previous unit is
|
||
untouched"*:
|
||
|
||
| unit file | before | after |
|
||
|---|---|---|
|
||
| `privatebin/volume-dumps/privatebin_privatebin_data.tar` | `26c546c2…` | **`b538ab89…`** |
|
||
| `opengist/volume-dumps/opengist_opengist_data.tar` | 182,272 B | **2,147,666,432 B** |
|
||
|
||
Both were rewritten by the earlier leg, while each `manifest.json` kept
|
||
`created_at: 2026-08-03T06:34:26Z` and its `checksums` block covers only the three compose files — so
|
||
a unit's payload can be swapped under a stale descriptor and nothing inside the unit can detect it.
|
||
|
||
**Cause, in the code and not the log.** The floor is consulted in exactly one place —
|
||
`m.unitFloorBlocked(stack.Name)` at `recovery_unit.go:328`, inside `captureAllRecoveryUnits`, which
|
||
writes a manifest and a compose copy: a few KB. `runVolumeDumps` (`backup.go:535`) — the leg that
|
||
writes the bulk, and the leg that consumed the reserve — has **no floor check at all**; its gates are
|
||
protected-stack, volume-less, disconnected, decommissioned. And it runs first *by design*
|
||
(`backup.go:483`). **The floor guards the cheap leg and not the leg that fills the volume.**
|
||
|
||
This is why R-165 is IMPLEMENTED and not PROVEN-LIVE: that row records B2 as the deliberate
|
||
replacement for the bulkhead the `mp1` partition provided, and pre-merge the unguarded leg could only
|
||
fill a dedicated 20 G partition — post-merge it can reach Docker's data-root. Filed as **R-181**.
|
||
**No code was written**, per the runbook's §7.
|
||
|
||
**Cleanup:** fill file and payload removed; `df` back to `1.2G used / 65G avail`; a clean re-run left
|
||
both units valid (`Volume dump … 178.0 KB` / `2.5 KB`, `App-data backup completed … (2.399s)`).
|
||
|
||
---
|
||
|
||
## 5. demo-felhom — the pipeline proof
|
||
|
||
**Install path — deliberately different, and this is the reason the second box exists.**
|
||
|
||
```
|
||
./felhom-host-install.sh --customer-id demo-felhom --mode appliance --vmid 9201 \
|
||
--cores 3 --memory 12288 \
|
||
--force-gitea-golden \
|
||
--passphrase-file /root/.pp-demo-felhom
|
||
```
|
||
|
||
A copy of golden 0.192.0 already sat on this box's `local` storage — it is where the golden was
|
||
**baked** at 06:58 (a `.log` beside it), sha `54e2a4c4…`. `--force-gitea-golden` is the documented
|
||
C.3 customer path and overrides local discovery in **both** pre-flight and step 7, which pre-flight
|
||
confirmed: `golden: none local — will fetch + verify from Gitea in step 7/8`. The bake artifact was
|
||
left untouched.
|
||
|
||
**`fetch_verify` succeeding against the vouched sha — the observable this box exists to produce:**
|
||
|
||
```
|
||
5/8 fetching agent binary v0.120.0 from Gitea …
|
||
verified sha256 a7763d31b55b5ce7… matches the hub manifest
|
||
7/8 fetching golden v0.192.0 from Gitea → /var/lib/vz/dump/vzdump-lxc-9100-2026_08_03-09_15_34.tar.zst
|
||
verified sha256 54e2a4c431daf580… matches the hub manifest
|
||
golden imported + verified: local:backup/vzdump-lxc-9100-2026_08_03-09_15_34.tar.zst
|
||
Day-0 provision SUCCESS — vmid=9201 host_id=demo-felhom-8363b5 customer=demo-felhom
|
||
```
|
||
|
||
Both artifacts — agent and golden — were fetched anonymously (the normal customer shape) and
|
||
sha-verified against the manifest. Controller `0.192.0` healthy.
|
||
|
||
**Layout:** `mp0: local-lvm:vm-9201-disk-1,mp=/var/lib/felhom,backup=1,size=250G`;
|
||
`pct config 9201 | grep -c '^mp1:'` → **0**. Both binds real mounts
|
||
(`…disk--1[/docker]`, `…disk--1[/sys_drive]`), `df` one figure — `246G 977M 233G 1%` on all three
|
||
paths — `stat -c %d` = `64519` on all three, fstab carries both binds.
|
||
|
||
**Reboots:** 1 — 09:18:55 → healthy 09:19:09; 2 — 09:19:12 → 09:19:26; 3 — 09:19:30 → 09:19:45. All
|
||
three paths still mountpoints after each; `uptime -s` = `2026-08-03 07:19:34 UTC`, matching reboot 3.
|
||
|
||
**Journey:** claim (`302 → /`, discriminator flipped to `authentication required`; code supplied by
|
||
the operator after a resend) → deploy `opengist` (`{"ok":true}`, `Up (healthy)`) → capture
|
||
(`Volume dump: opengist/… → 178.0 KB`, `Recovery unit captured`, unit on
|
||
`/dev/mapper/pve-vm--9201--disk--1`, marker present inside the tar) → **restore**
|
||
(`{"ok":true,"message":"opengist visszaállítva (primary)."}` in **9.4 s**, marker back with identical
|
||
sha256 `bc5507987f3f56dc19a9c24785826c304f1a986998d4aeb0b06d1a810b59e939`, app healthy).
|
||
|
||
**Ceiling:** 233 GiB available to a recovery unit, against the 45 GiB the pre-wipe `mp1` offered.
|
||
|
||
**Step 8 was NOT repeated here, deliberately** — stating it rather than leaving it ambiguous. The
|
||
floor fired on demo-hp and R-181 characterises it fully; re-firing would add no information and would
|
||
mean filling a 246 G volume.
|
||
|
||
---
|
||
|
||
## 6. Vouching
|
||
|
||
Already done before the session (§2a), at **07:23:26 CEST on 2026-08-03**, by an operator action in
|
||
the hub UI. The session's own manifest write was the **agent** half, at **07:44:31**:
|
||
`Artifact manifest set: agent=0.120.0 golden=0.192.0 min_agent="0.113.0" wrapper_sha=true`. The
|
||
manifest afterwards, read back: `agent_sha256=a7763d31b55b5ce7…10b9d`,
|
||
`golden_sha256=54e2a4c431daf580…43b3e0`, `min_agent=0.113.0`, `wrapper_sha256=104db0a4…` (preserved
|
||
verbatim). The R-120 gate did not block: golden 0.192.0 equals the newest controller the fleet
|
||
reports.
|
||
|
||
---
|
||
|
||
## 7. Teardown — all three layers
|
||
|
||
1. **The machine.** No throwaway guest was created this session, so there is none to delete. VM 300
|
||
`drill-r50` on demo-hp — the protected drift fixture — was never touched and is still `stopped`;
|
||
it is not in the `felhom` pool (`pvesh get /pools/felhom` listed only `lxc/9201`), so the
|
||
uninstall's shared-box logic never reached it.
|
||
2. **The host.** `local-lvm`, before → after: **demo-felhom 29.31 % → 1.37 %** (the old 200 G + 50 G
|
||
volumes returned; the new 250 G volume is thin and barely allocated). **demo-hp 39.13 % →
|
||
36.75 %** — *higher than a clean reinstall would leave it*, because the floor test's 2 GB tar
|
||
blocks cannot be reclaimed: `fstrim` inside an unprivileged LXC returns
|
||
`FITRIM ioctl failed: Operation not permitted`. No operational impact (the guest shows 65 G free of
|
||
69 G, the host 35.7 GiB free of 53.93), but it is real residue and is stated rather than rounded
|
||
away. Each box's data volume is now the only data volume; no old guest volumes remain.
|
||
**Residue found and cleared by hand on demo-hp:** `--uninstall` left the NAS network-storage units
|
||
`mnt-felhom\x2ddrives-Felhom\x2dShare.{mount,automount}` behind (automount `failed`, parent bind
|
||
still mounted) → **R-179**. demo-felhom left none, because it had no network share configured.
|
||
3. **The hub.** **No old records exist to dispose of, and this is the honest finding, not an
|
||
omission:** both enrollments were **idempotent** — `host REUSED (idempotent — existing
|
||
credential)`, `host_id: demo-hp-bb76ea` and `demo-felhom-8363b5`, the same ids as before. The
|
||
reinstalls therefore produced **no new host records**, so nothing was orphaned and nothing needed
|
||
deleting. Final register: `demo-felhom-8363b5 ONLINE 0.120.0`, `demo-hp-bb76ea ONLINE 0.120.0`,
|
||
`drill-r50-0a4f9a DOWN 0.113.0` — the same three rows as at P6. No scratch customers were created.
|
||
`/appliances` returns 404 on hub 0.89.0 — there is no appliance-record surface to clean.
|
||
|
||
**Secrets:** both retrieval passphrases were moved file→file into 0600 files, used via
|
||
`--passphrase-file`, and `shred -u`'d afterwards on both hosts along with the session cookie files and
|
||
helper scripts; the local scratch copies were deleted. Nothing was written to a committed file and
|
||
`curl -w '%{redirect_url}'` was never used (R-132).
|
||
|
||
---
|
||
|
||
## 8. Registers changed
|
||
|
||
| Row | Change |
|
||
|---|---|
|
||
| **R-178** | **CLOSED** — both boxes reinstalled and proven, by two different supply paths |
|
||
| **R-165** | **IMPLEMENTED**, not PROVEN-LIVE — operator ruling; the layout half is proven, the B2 half is not (→ R-181) |
|
||
| **R-115** | **Third instance recorded** — agent 0.120.0 built, deployed to both hosts, never published |
|
||
| **R-181** *(new)* | The capture floor guards the recovery-unit leg and not `runVolumeDumps`; its "previous unit is untouched" claim measured false |
|
||
| **R-180** *(new)* | `--archive-storage` is not cross-checked against the ACL grant; the 403 lands at step 8/8, after root@pam has been rotated |
|
||
| **R-179** *(new)* | `--uninstall` leaves NAS network-storage systemd units behind when a share was configured |
|
||
|
||
**IDs established free before minting:** `grep -ro "R-179\b\|R-180\b\|R-181\b\|R-182\b"` over
|
||
`documentation/` and `*.md` → **0 hits**, and over all four repo roots (`felhom-agent`,
|
||
`felhom-controller`, `felhom.eu`, `app-catalog-felhom.eu`) → **0 hits**. `R-182` was checked and left
|
||
unused.
|
||
|
||
---
|
||
|
||
## 9. CI — checked by run id, not assumed
|
||
|
||
Two commits, both docs-only.
|
||
|
||
| repo | commit | CI run | result |
|
||
|---|---|---|---|
|
||
| `felhom-agent` | `9dfd89c` — *docs: agent 0.120.0 published + vouched…* | **run 46**, `head_sha 9dfd89cb` | **success** (`gates`) |
|
||
| `felhom.eu` | `aa62449` — *R-178 CLOSED: both demo boxes reinstalled…* | **run 47**, `head_sha aa624496` | **success** (`gates`) |
|
||
|
||
Queried with
|
||
`curl -s "https://gitea.dooplex.hu/api/v1/repos/admin/<repo>/actions/tasks?limit=3"` and matched on
|
||
`head_sha`, per `CLAUDE.md`'s pull-check rule — CI mails on failure, which is a push signal; this is
|
||
the pull check that catches a lost or unread mail.
|
||
|
||
**`--no-verify` was NOT used**, anywhere. This clone is armed (`core.hooksPath = .githooks`). Both
|
||
gate entry points were also run by hand before committing:
|
||
`felhom.eu/scripts/repo_gates.py` → **all five OK** (site, hostinstall, hub-confirm,
|
||
manifest-bearer, reuse-refs), and `felhom-agent/scripts/agent_gates.py` → **OK** (reuse-refs).
|
||
No test suite was run because no code changed in either repo.
|
||
|
||
---
|
||
|
||
## 10. Observations — noticed and NOT acted on
|
||
|
||
- **The runbook's central claim was wrong in a way that mattered.** R-178 said *"a reinstall is now a
|
||
self-contained piece of work with no code left to write."* True about code; false about the artifact
|
||
channel — the agent half of the merge was unpublished, and following the documented path without
|
||
checking would have downgraded both boxes and produced a green, meaningless result. **No code change
|
||
was needed to make any step pass** — §10 asks this loudly, and the answer is no. What was needed was
|
||
a publish.
|
||
- **Prove-then-vouch was already spent when the session opened.** Not a defect in anything, but the
|
||
rule protected nothing because nothing enforced it. R-120's gate is the shape that would.
|
||
- **A per-stack `POST /stacks/<n>/backup` does not produce volume dumps** — the volume leg lives in the
|
||
full app-data pass. Not wrong, but the endpoint's name suggests otherwise and it cost time here.
|
||
- **privatebin's recovery unit carries no user data** (its catalog entry declares no `backup:`
|
||
section), so restoring it loses every paste. That may be intentional for an expiring, E2E-encrypted
|
||
paste bin — but nothing in the app's description tells the customer so. Catalog question, not filed.
|
||
- **V-c doubles systemd's mount-unit count** — every docker overlay appears twice,
|
||
`var-lib-docker-…-merged.mount` and `var-lib-felhom-docker-…-merged.mount`, because `/var/lib/docker`
|
||
is a bind of `/var/lib/felhom/docker`. Cosmetic, inherent to the chosen variant, no action.
|
||
- **`fstrim` cannot run inside the guest** (`EPERM`, unprivileged LXC), so space freed inside the guest
|
||
is not returned to the thin pool. It did not matter here; on a box that fills and empties repeatedly
|
||
it would.
|
||
- **The floor's own message mixes units** — it reports `64.2/68.7 GB used (93%)` against a threshold
|
||
stated as `97% used or 1.0 GiB free`, while `df` showed 96 %. GB-vs-GiB, so the percentage term
|
||
fires later than an operator reading `df` would expect. Minor; noted on R-181's fix shape rather
|
||
than filed separately.
|