# RUNBOOK — pull the failed disk Z6A07P2G out of DooPlex

> Written 2026-10-12 by CC from read-only checks (`audits/answers-2026-10-12/disk-Z6A07P2G.txt`). Operator's job: the
> power-off and the hands. Register row R-934.

## The disk — by SERIAL, never by `sdX`

| | |
|---|---|
| Model | Seagate Skyhawk **ST3000VX010-2E3166**, 3 TB |
| Serial | **Z6A07P2G** |
| Port (2026-10-12) | the onboard SATA controller `0000:00:1f.2`, port **ata-6** (`/dev/disk/by-path/pci-0000:00:1f.2-ata-6`); kernel `ata10` |
| Fault | 352 unreadable (pending) + 352 offline-uncorrectable sectors, flat since 2026-10-09 |

**⚠ THE TWIN.** DooPlex has a SECOND disk of the same model: **ST3000VX010, serial Z6A070XZ — IN USE as `/mnt/3_hdd`, keep it.**
`Z6A07P2G` (pull) and `Z6A070XZ` (keep) differ only in the last four characters. **Read the serial on the disk's
label before you unplug anything.** If you cannot read it, follow the cable to SATA port 6 on the board. Do not
make the disk busy to find it by its LED: no reads on a failing disk.

## Before power-off (already true on 2026-10-12 — re-check in one minute)

```bash
lsblk -o NAME,SERIAL,MOUNTPOINTS | grep Z6A07P2G      # no mountpoint
grep -n 9cf20140 /etc/fstab                          # only a '# DISABLED 2026-08-14' line
grep RequiresMountsFor /etc/systemd/system/k3s.service.d/wait-for-mounts.conf   # no /mnt/4_hdd
sudo kubectl -n longhorn-system get nodes.longhorn.io dooplex -o jsonpath='{.spec.disks}' | grep -c 4_hdd   # 0 (HDD4 entry removed 2026-10-12)
```

Nothing else reads the disk: no backup script, PV, hostPath or running app names `/mnt/4_hdd` (2026-10-12).

## Pull it

1. Plan a window: DooPlex runs Gitea, the registry, the hub and Longhorn. A box keeps running while the hub is off.
2. `sudo poweroff`. Unplug power.
3. Find the disk with serial **Z6A07P2G** (label), on SATA port 6. Unplug its data and power cable. Remove it.
4. Power on.

## After the first boot

1. All mounts back: `findmnt /mnt/1_hdd /mnt/2_hdd /mnt/3_hdd /mnt/5_hdd /mnt/ssd_2` — five lines.
2. k3s up and Longhorn healthy: `sudo kubectl -n longhorn-system get nodes.longhorn.io` READY True; all volumes
   `healthy` (`sudo kubectl -n longhorn-system get volumes.longhorn.io`).
3. The hub answers (`https://hub.felhom.eu`), Gitea answers.
4. The SMART exporter: delete the line `--smartctl.device=/dev/disk/by-id/ata-ST3000VX010-2E3166_Z6A07P2G` in
   `homelab-manifests/admin-system/smartctl-exporter.yaml`, commit, sync — otherwise it keeps asking for a disk that
   is gone.
5. The next DooPlex backup (03:10) is green: `systemctl show dooplex-backup.service -p Result -p ExecMainStatus`.
6. Delete the Alertmanager silence for `device=~".*_Z6A07P2G"` (it ends by itself on **2026-11-11 14:32 UTC**).
   If the disk is still in the machine on that date, the two alarms come back **on purpose**.
7. Tell CC: the row closes.