admin/felhom.eu
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: retract the expired NVMe fence, drop component versions from the inventory, fence acts in the template

Browse Source 
Follow-up acting on the observations filed with runbooks/target-selection.md.

operations/nodes.md
- The demo-hp NVMe was documented "PRESENT AND UNENROLLED -- do not touch" and
  listed under "What is NOT enrolled here (deliberately)". Both are FALSE and had
  been for eight days: it was enrolled 2026-07-22 through the normal Tarhely flow
  and is now /mnt/nvme-1tb -- the enrolled user-data drive AND the felhom-backup
  target (verified live 2026-07-30: nvme0n1 -> /mnt/nvme-1tb, and
  dir: felhom-backup / path /mnt/nvme-1tb / is_mountpoint 1). The fence's own
  condition (join via Tarhely, not the installer, not by hand) was SATISFIED, so
  the prohibition expired with it -- while still contradicting the task specs that
  correctly sent drill-VM disks there. Retracted with its reason recorded, and the
  caution that IS still live kept (dir storage at the mountpoint ROOT, else
  exactMount fails and the storage reads disconnected forever).
- Component versions REMOVED and a note explains why: agent/controller/hub versions
  change several times a day, so a number written in an inventory is wrong within
  hours and then read as fact -- and the fleet is not uniform (on 2026-07-30 the two
  boxes ran different agent AND different controller versions). Points at the
  authorities instead: hub /hosts + /configs, felhom-agent --version, docker ps.
- Site addresses now say re-check rather than asserting one (the N100 read .162, not
  the recorded .147); records that LAN literals are unreachable from DooPlex while
  the boxes are away. Adds the target-selection pointer: this page is what the
  hardware IS, that page is what may be done to it.

PROMPT-TEMPLATE.md -- the upstream generator of the defect
- Section 12's "Do NOT touch [the untouchable]" asked the spec author to name a
  THING. Now asks for the forbidden ACT plus its REASON, with the demo-hp case as
  the worked example of how a bare object-fence over-reads.
- Section 13 gains the positive counterpart, which was the actual gap: if a task
  needs a machine to break, NAME IT. Listing only what is off-limits leaves the
  most valuable unfenced machine as the residual choice.

runbooks/workspace-CLAUDE.md (+ the untracked root copy re-synced, verified identical)
- Host table gains a Blast radius column and the missing demo-hp row, notes felhotest
  as Connection refused, and points at target-selection.md. This is the file that
  loads FIRST every session, so leaving it with the old table would have undercut
  the whole fix.

No code, no build, no deploy, no host reconfigured or renamed.
This commit is contained in:
authentik Default Admin
2026-07-30 08:25:14 +02:00
parent 699790b12d
commit e6b5fa1e63
3 changed files with 69 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files
documentation/PROMPT-TEMPLATE.md
+12 -1
View File
@@ -377,7 +377,12 @@ an owner — a row nobody owns is how items got lost in the first place.
<!-- Concrete temptations for THIS task, not "don't add features". -->
- Do NOT [specific out-of-scope thing CC might build].
- Do NOT touch [the untouchable: e.g. the operator-signed `DecommissionExecutor` / `classify.go`].
- Do NOT [**the forbidden ACT**, and **its reason**: e.g. "modify the operator-signed
`DecommissionExecutor` — it is the signature boundary"]. **Fence the act, not the object.** A bare
"do not touch X" is read as covering every act on X, including ones nobody meant to restrict — that
is how "do not re-target demo-hp's backup target" became "do not use demo-hp at all", which pushed a
destructive drill onto the one machine holding the recovery chain. State the reason too: a rule whose
reason is recorded can be correctly narrowed later, and one without it becomes permanent by default.
- Do NOT reuse [the dangerous lookalike from §5].
- Do NOT refactor nearby code, change passing tests, or create a branch.
- Do NOT add "Co-Authored by..." anywhere
@@ -391,6 +396,12 @@ an owner — a row nobody owns is how items got lost in the first place.
**Clean-tree gate first:** `git status --porcelain` empty AND `HEAD` == `origin/main` in the repo
being built. An unpushed change does not exist.
**If the task needs a machine to break — NAME IT.** A drill, destructive test or throwaway VM gets an
explicit target: *"run this on demo-hp"*. Do not leave it to be inferred from the prohibition list;
listing only what is off-limits leaves the most valuable unfenced machine as the residual choice, which
is exactly how a drill landed on DooPlex. Tiers and per-machine permitted/care/forbidden:
[`runbooks/target-selection.md`](runbooks/target-selection.md).
**Controller** (local build on DooPlex → guest via golden/bootstrap):
```bash
FELHOM_ROOT=/mnt/5_hdd/felhom.eu
documentation/operations/nodes.md
+41 -15
View File
@@ -13,15 +13,30 @@
| Firmware | AMI AN3PLUS-class | **AMI M42 v01.10 (11/11/2020)** |
| PVE node name | `demo-felhom` | `felhom-host` |
| Customer | `demo-felhom` | `demo-hp` |
| Agent | 0.96.0 | 0.96.0 |
| Agent / controller | *not recorded here* — see the note below | *not recorded here* |
| Control plane | **island** `169.254.253.1:8443` on `vmbr9` (R-50, migrated 2026-07-25) | **island** `169.254.253.1:8443` on `vmbr9` (R-50, migrated 2026-07-25) |
| SSH alias | `felhom-pve` | **`demo-hp`** |
| Tailnet | `100.70.170.35` | **`100.76.96.79`** |
| Loader used to install | `mkimage` (unsigned, SB **off** — firmware workaround) | **`shim`, Secure Boot ENABLED** |
> **No component versions are recorded on this page — deliberately.** Agent, controller, hub and
> host-install versions change several times a day, so any number written here is wrong within hours and
> is then read as fact. **Ask the fleet instead:** the hub host list (`/hosts`) and customer list
> (`/configs`) carry the live agent and controller versions per box; `felhom-agent --version` on the
> node and `pct exec <vmid> -- docker ps` in the guest are the authorities. Versions that must be pinned
> in writing belong in the per-repo `CHANGELOG.md` and the hub's Day-0 artifact manifest — not in an
> inventory. **The fleet is not uniform**: on 2026-07-30 the two boxes ran different agent *and*
> different controller versions, so a single number for "the fleet" would have been wrong regardless.
**Both are at the VACATION site** and travel home with the rest of the kit **~2026-08-02**. While
away, their LAN addresses are on that site's `192.168.0.0/24`: N100 `.147`/`.162`, HP `.87`. The
tailnet addresses are the stable ones — use those.
away, their LAN addresses are on that site's `192.168.0.0/24`: N100 `.147`/`.162`, HP `.87` — **re-check
rather than trusting these** (`ip -br addr show vmbr0`; the N100 read `.162` on 2026-07-30). The
tailnet addresses are the stable ones — use those. Direct LAN literals are **not** reachable from
DooPlex while the boxes are away (`felhom-pve-lan` → `No route to host`, 2026-07-30).
**Which box is safe to break, and what may be done to each:
[`../runbooks/target-selection.md`](../runbooks/target-selection.md).** This page is *what the hardware
is*; that page is *what you may do to it*.
## demo-hp — the HP t740, in detail
@@ -30,13 +45,23 @@ tailnet addresses are the stable ones — use those.
| device | model | serial | role |
|---|---|---|---|
| `sda` | SanDisk X600 M.2 2280 SATA 128GB | `182195804614` | **system disk** — PVE, LVM (`pve-root` 39.6G, `pve-data` thin pool, guest 9201's three volumes) |
| `nvme0n1` | KXG50PNV1T02 NVMe TOSHIBA 1024GB | `58BS11AFT8MQ` | **PRESENT AND UNENROLLED — do not touch** |
| `nvme0n1` | KXG50PNV1T02 NVMe TOSHIBA 1024GB | `58BS11AFT8MQ` | **ENROLLED** — mounted `/mnt/nvme-1tb`, the enrolled user-data drive **and** the `felhom-backup` whole-guest backup target |
> **The NVMe is a future Tárhely candidate and is deliberately outside everything.** It still holds
> its previous **NTFS** partition, is not mounted, and appears in no LVM PV and no ZFS pool. The
> install ISO's exact-serial filter pinned `sda` only, which is why it survived. **It joins the
> product later through the normal Tárhely (storage) flow — never through the installer, and never
> by hand.** Anything that formats it before that decision is a mistake.
> **The NVMe joined the product on 2026-07-22, through the normal Tárhely flow, as intended.** Enrolled
> to guest 9201 (PUBLISH TRAIN, `pilot/RUNBOOK-publish-agent-0.93-2026-07-22.md`); the vzdump target was
> moved onto it by E-2a. Verified live 2026-07-30: `nvme0n1` → `/mnt/nvme-1tb`, and
> `dir: felhom-backup / path /mnt/nvme-1tb / is_mountpoint 1` in `storage.cfg`.
>
> **The "PRESENT AND UNENROLLED — do not touch" fence that stood here is RETRACTED**, and its reason is
> recorded so it is not mistaken for a live rule: the NVMe was outside everything because the install
> ISO's exact-serial filter pinned `sda` only, and it was to join *through the Tárhely flow rather than
> the installer or by hand*. **That condition was satisfied; the prohibition expired with it.** It stood
> for eight days after enrolment and contradicted the task specs that (correctly) sent drill-VM disks to
> `/mnt/nvme-1tb`.
>
> Still true, and now the operative caution: a dir storage there **must sit at the mountpoint root** (a
> subdirectory fails the agent's `exactMount` check → the storage reads `disconnected` forever), and it
> shares the device with the box's own backups — so remove scratch storages when done.
### NIC map — and the trap
@@ -58,8 +83,8 @@ that looked installed and could never call home. Repaired on the console by brid
Current, post-repair: `vmbr0` static `192.168.0.87/24`, gw `192.168.0.1`, bridge-port `enp2s0f0`.
No trace of `192.168.100.2` remains. `wg-felhom` `10.77.0.3/32` up to the hub. Guest **9201
`demo-hp`** running. Agent **0.96.0** (R-50 island: `local_api` on `169.254.253.1:8443`/`vmbr9`,
guest `eth1 169.254.253.2/30`, `lan_resolver.host_ip` pinned to `192.168.0.87`; `.bak-0.95.0` retained; caps 68/68 ok).
`demo-hp`** running. Agent config shape (R-50 island): `local_api` on `169.254.253.1:8443`/`vmbr9`,
guest `eth1 169.254.253.2/30`, `lan_resolver.host_ip` pinned to `192.168.0.87`.
### Designated drill + build VM host (operator ruling, 2026-07-25)
@@ -71,8 +96,8 @@ relocation to the t740 is NOT ruled or implemented here.**
drill appliance. **QEMU VM `300` = `drill-r50`**, a nested PVE-in-a-VM (8 GiB RAM, 4 vCPU cpu=host, 32 GiB
local-lvm disk, OVMF/SB-off, one NIC on `vmbr0` DHCP). Installed from
`felhom-pve-9.2-1-v1.25.0-nested-vm-generic-mkimage.iso` through the **real day-0** (self-register →
operator bind to scratch customer `drill-r50` → deliver → nested guest **9201** provisioned, controller
0.161.0 / agent 0.93.0 healthy). Its own break-glass root is vaulted in the hub
operator bind to scratch customer `drill-r50` → deliver → nested guest **9201** provisioned and healthy
on the then-current controller + agent). Its own break-glass root is vaulted in the hub
`host_recovery/drill-r50-0a4f9a`; reach it as `root@192.168.0.176` **through `demo-hp`** (it has no key
and no tailnet — it is a peer on demo-hp's LAN). It has snapshot **`r50pre`** (clean LAN-literal day-0).
This VM was provisioned to unblock and run the **R-50 island-bridge empirical spike**
@@ -152,6 +177,7 @@ Details, and the two hard rules that apply to any host node, in `operations/tail
## What is NOT enrolled here (deliberately)
- The **1TB NVMe** on demo-hp — see above.
- **No PBS datastore, no offsite target** on demo-hp yet; the DR tier is the N100's.
- **No PBS datastore, no offsite target** on demo-hp; the DR tier is the N100's.
- **No second customer guest** beyond 9201.
*(The 1TB NVMe was on this list until 2026-07-22. It is enrolled — see the disks table above.)*
documentation/runbooks/workspace-CLAUDE.md
+16 -6
View File
@@ -122,12 +122,22 @@ Local (this host): repos `/mnt/5_hdd/felhom.eu/git/<repo>`, build dirs
`/mnt/5_hdd/felhom.eu/build/felhom-{controller,hub,agent}`, `sudo kubectl`, Go toolchain, Docker
build+push to `gitea.dooplex.hu/admin/`.
| Host | Access | Use |
|---|---|---|
| **DooPlex (this host)** | local — Debian 13, `kisfenyo`, `/mnt/5_hdd/felhom.eu/` | build/push images, `sudo kubectl`, build+run the agent for tests |
| Demo Proxmox host `demo-felhom` | `ssh felhom-pve` (root@192.168.0.162, no sudo) | pveum/pct + live Proxmox validation |
| Demo guest 9201 | `ssh felhom-pve "pct exec 9201 -- ..."` | the live demo controller |
| felhotest (legacy) | `ssh -p 33022 kisfenyo@router.abonet.hu` | OLD /opt/docker compose mechanism |
| Host | Access | Use | Blast radius |
|---|---|---|---|
| **DooPlex (this host)** | local — Debian 13, `kisfenyo`, `/mnt/5_hdd/felhom.eu/` | build/push images, `sudo kubectl`, build+run the agent for tests | **Tier 2 — precious.** It *is* the recovery chain (hub, Gitea, registry, PBS, k3s+Longhorn). **Never a drill target** |
| Demo Proxmox host `demo-hp` (HP t740) | `ssh demo-hp` (tailnet `100.76.96.79`; **no baked key** — G1 break-glass password vaulted in the hub) | **the designated drill + build VM host** (operator ruling 2026-07-25) | **Tier 0 — disposable. Reach here first** |
| Demo Proxmox host `demo-felhom` (N100) | `ssh felhom-pve` (root, no sudo; tailnet `100.70.170.35`) | pveum/pct + live Proxmox validation | **Tier 0 — disposable** |
| Demo guest 9201 | `ssh felhom-pve "pct exec 9201 -- ..."` | the live demo controller | Tier 0 (rides its host) |
| felhotest (legacy) | `ssh -p 33022 kisfenyo@router.abonet.hu` — **`Connection refused` 2026-07-30** | OLD /opt/docker compose mechanism | untiered — assume nothing |
**Which box do I break?** → **`felhom.eu/documentation/runbooks/target-selection.md`** — the tiers, and
per machine what is freely permitted / needs care / forbidden, each with its reason. Read it before
picking a machine for a drill, a destructive test or a throwaway VM. **A task that needs a victim names
one; an absent fence is not permission.**
**Component versions are not recorded in any inventory doc** — agent/controller/hub versions change
several times a day and the fleet is not uniform. Ask the hub's `/hosts` + `/configs`, or
`felhom-agent --version` / `pct exec <vmid> -- docker ps` on the box.
The demo Proxmox host key changes on reprovision (N100) → refresh with
`ssh-keygen -R 192.168.0.162` then connect with `-o StrictHostKeyChecking=accept-new`