Files
felhom.eu/documentation/audits/evidence-rehearsal-2026-08-09/P1-demo-hp-after.txt
T
admin b1afbb8a4d
gates / gates (push) Successful in 24s
Rehearsal 2026-08-09: the walk stops at P3 — R-273 blocks every install fleet-wide
P1 uninstall, P2 preflight, P3 install. The install FAILED at step 5/8 in 44s,
and the two rank-1 findings are both on the setting-up path a tester's visit is
made of. Eleven register rows minted (R-269..R-279); ceiling moves 268 -> 279.

R-273 (RANK 1) — the hub vouches agent 0.128.0; that version was published as a
Gitea PACKAGE but never git-tagged. Since R-183 the installer correctly pins its
config fetches to raw/tag/v<vouched>, so every fresh install and every reinstall
now 404s as root, mid-install. Measured: main 200, v0.127.0 200, v0.128.0 404.
This is R-184 arriving; release-agent.sh:23 already documents the exact hazard.
Existing boxes are fine (self-update takes the binary from the registry).
NOT fixed here — publishing a release tag is outward-facing and the runbook says
stop and report. One command unblocks it; it is in STATUS.md.

R-272 (RANK 1) — Felhom's own uninstall leaves the condition that makes Felhom's
own reinstall refuse. It installs dnsmasq at day-0, then on teardown removes the
snippet and RESTARTS the daemon unconstrained (process start time lands inside
the uninstall window), which grabs 0.0.0.0:53; the next preflight then refuses,
and the message reads as though the owner's LAN DNS is at fault.

R-274 — a local golden is adopted with no version and no sha check; the manifest
vouch is consulted only on the fetch path. demo-hp's local copy is controller
0.192.0 against a vouched 0.210.0, and below the 0.200.0 where the recovery
screen shipped. Not yet observed end-to-end (R-273 killed step 5 first).

Also: R-275 orphaned credential backups + uid reuse, R-276 the wg tunnel outlives
the uninstall, R-269/270/271 from the token rotation, R-277 three hub surfaces
misreport a healthy off-site tier, R-278 demo-felhom six days unprotected,
R-279 no operator-triggerable off-site run.

Two hypotheses of mine were tested and REFUTED rather than shipped as findings:
the leftover sudoers file is inert (sudo skips dotted filenames), and demo-hp's
off-site tier was healthy all along - I had misread the hub and said so.

STATUS.md records the three rulings §8.3 asked for, with the floor CORRECTED to
its live value 0.200.0 and the count corrected to twenty.
2026-08-09 10:59:44 +02:00

168 lines
8.1 KiB
Plaintext

=== captured 2026-08-09T08:38:48Z UTC ===
### agent binary
ABSENT
### agent unit
felhom-bootstrap.service disabled enabled
### service user
id: 'felhom-agent': no such user
ABSENT
### felhom-op user (OOB)
id: 'felhom-op': no such user
ABSENT
### sudoers
total 40
drwxr-xr-x 2 root root 4096 Aug 9 10:37 .
drwxr-xr-x 101 root root 4096 Aug 9 10:37 ..
-r--r----- 1 root root 1068 Apr 11 14:21 README
-r--r----- 1 root root 21833 Jul 28 11:29 felhom-agent.bak-pre-e2a
-r--r----- 1 root root 666 May 20 2023 zfs
/etc/sudoers.d/felhom-agent.bak-pre-e2a
### pct list
### qm list
VMID NAME STATUS MEM(MB) BOOTDISK(GB) PID
300 drill-r50 stopped 8192 32.00 0
### pools
[]
### pveum users/roles/acl
+----------+---------+-------------------+--------+--------+-----------+--------+------+----------+------------+------------------+--------+-------------+
| userid | comment | email | enable | expire | firstname | groups | keys | lastname | realm-type | tfa-locked-until | tokens | totp-locked |
+==========+=========+===================+========+========+===========+========+======+==========+============+==================+========+=============+
| root@pam | | noreply@felhom.eu | 1 | 0 | | | | | pam | | | |
+----------+---------+-------------------+--------+--------+-----------+--------+------+----------+------------+------------------+--------+-------------+
--roles--
no felhom roles
--acl--
### pvesm status
Name Type Status Total (KiB) Used (KiB) Available (KiB) %
c11-scratch dir active 983379700 6940296 926412792 0.71%
felhom-backup dir active 983379700 6940296 926412792 0.71%
felhom-pbs pbs active 0 0 0 0.00%
local dir active 40516856 19063736 19362728 47.05%
local-lvm lvmthin active 56545280 12355143 44190136 21.85%
### storage.cfg
dir: local
path /var/lib/vz
content iso,import,vztmpl,backup
lvmthin: local-lvm
thinpool data
vgname pve
content rootdir,images
pbs: felhom-pbs
datastore felhom-offsite
server 10.77.0.1
content backup
encryption-key 3f:4f:65:c0:d8:f3:9f:3c:43:29:8c:c8:61:3c:89:40:e6:7a:c8:ae:b1:44:70:09:0e:f1:f7:1b:60:a0:c4:99
fingerprint c6:07:28:3f:5b:7b:5a:41:90:28:d7:ca:4f:37:14:70:56:39:2e:2f:0b:71:e8:06:ca:60:4a:d5:56:5f:3c:fd
namespace demo-hp
username felhom@pbs!demo-hp
dir: felhom-backup
path /mnt/nvme-1tb
content backup
is_mountpoint 1
dir: c11-scratch
path /mnt/nvme-1tb
content images
### df
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/pve-root 39G 19G 19G 50% /
/dev/nvme0n1 938G 6.7G 884G 1% /mnt/nvme-1tb
### mounts under /mnt
/mnt/nvme-1tb /dev/nvme0n1
### felhom-drives tree
/mnt/felhom-drives/:
total 12
drwxr-xr-x 3 root root 4096 Aug 3 08:05 .
drwxr-xr-x 6 root root 4096 Aug 6 14:39 ..
drwxr-xr-x 2 root root 4096 Jul 22 13:53 Felhom-Share
/mnt/felhom-drives/Felhom-Share:
total 8
drwxr-xr-x 2 root root 4096 Jul 22 13:53 .
drwxr-xr-x 3 root root 4096 Aug 3 08:05 ..
### leftover felhom paths
/etc/felhom
/etc/felhom-agent
/usr/local/sbin/felhom-bootstrap.sh
none of the usual paths remain
### systemd leftovers on disk
-rw-r--r-- 1 root root 1662 Jul 21 18:20 felhom-bootstrap.service
### guest hook snippets
total 8
drwxr-xr-x 2 root root 4096 Aug 9 10:37 .
drwxr-xr-x 7 root root 4096 Jul 30 12:36 ..
### dnsmasq snippets
no felhom dnsmasq snippets
### wg tunnel
interface: wg-felhom
public key: KEYSzMyTKEZVDqDbikDXnadyeX9yJUjy8T58q0f4/1U=
private key: (hidden)
listening port: 59688
peer: f3d1ZI7/2/r+FWz8eo8Wl2HVygTZK9wBIN2M3dNHsgA=
endpoint: 167.233.158.164:443
allowed ips: 10.77.0.1/32, 10.77.0.250/32
latest handshake: 52 seconds ago
transfer: 5.86 GiB received, 2.48 GiB sent
vmbr9 DOWN 00:00:00:00:00:00 <NO-CARRIER,BROADCAST,MULTICAST,UP>
wg-felhom UNKNOWN <POINTOPOINT,NOARP,UP,LOWER_UP>
### network bridges
auto vmbr9
iface vmbr9 inet static
address 169.254.253.1/30
bridge-ports none
bridge-stp off
bridge-fd 0
### listening ports
State Recv-Q Send-Q Local Address:Port Peer Address:PortProcess
LISTEN 0 4096 100.76.96.79:39440 0.0.0.0:* users:(("tailscaled",pid=1152166,fd=24))
LISTEN 0 128 0.0.0.0:22 0.0.0.0:* users:(("sshd",pid=1052,fd=6))
LISTEN 0 32 0.0.0.0:53 0.0.0.0:* users:(("dnsmasq",pid=1291306,fd=5))
LISTEN 0 4096 0.0.0.0:111 0.0.0.0:* users:(("rpcbind",pid=701,fd=4),("systemd",pid=1,fd=97))
LISTEN 0 100 127.0.0.1:25 0.0.0.0:* users:(("master",pid=1230,fd=13))
LISTEN 0 4096 127.0.0.1:85 0.0.0.0:* users:(("pvedaemon worke",pid=1179410,fd=6),("pvedaemon worke",pid=1167090,fd=6),("pvedaemon worke",pid=1015859,fd=6),("pvedaemon",pid=1283,fd=6))
LISTEN 0 100 [::1]:25 [::]:* users:(("master",pid=1230,fd=14))
LISTEN 0 4096 *:3128 *:* users:(("spiceproxy work",pid=681975,fd=6),("spiceproxy",pid=1370,fd=6))
LISTEN 0 128 [::]:22 [::]:* users:(("sshd",pid=1052,fd=7))
LISTEN 0 32 [::]:53 [::]:* users:(("dnsmasq",pid=1291306,fd=7))
LISTEN 0 4096 [::]:111 [::]:* users:(("rpcbind",pid=701,fd=6),("systemd",pid=1,fd=99))
LISTEN 0 4096 [fd7a:115c:a1e0::ce36:6051]:51957 [::]:* users:(("tailscaled",pid=1152166,fd=25))
LISTEN 0 4096 *:8006 *:* users:(("pveproxy worker",pid=1245509,fd=6),("pveproxy worker",pid=1244391,fd=6),("pveproxy worker",pid=1241021,fd=6),("pveproxy",pid=1295,fd=6))
### PBS snapshots (must be KEPT)
Volid Format Type Size VMID
felhom-pbs:backup/ct/9201/2026-07-28T19:19:45Z pbs-ct backup 6264034049 9201
felhom-pbs:backup/ct/9201/2026-08-04T19:24:16Z pbs-ct backup 4637840513 9201
### local vzdump (must be KEPT)
Volid Format Type Size VMID
felhom-backup:backup/vzdump-lxc-9100-2026_08_03-07_33_00.tar.zst tar.zst backup 649547835 9100
felhom-backup:backup/vzdump-lxc-9201-2026_08_07-04_34_58.tar.zst tar.zst backup 2073915792 9201
felhom-backup:backup/vzdump-lxc-9201-2026_08_08-04_37_15.tar.zst tar.zst backup 2142232544 9201
felhom-backup:backup/vzdump-lxc-9201-2026_08_09-04_42_15.tar.zst tar.zst backup 2143026831 9201