felhom.eu/documentation/runbooks/provider-questions-2026-09-01.md

Two questions for Hetzner — drafted, ready to send (2026-09-01)

These are the only thing standing between us and finishing R-95 properly. They cost nothing and they are answerable by a support agent without escalation.

Send them yourself. CC drafted them and did not send them, and did not call the provider API — §11-D is still the operator's fence. No credential, password or token appears below, and none should be added. The account id and the product name are all either question needs.

What to fill in: the ticket needs the Storage Box account. Ours is u629488 (the box the register calls storage-box-pool-1, plan BX11). Nothing else.

Why two separate tickets: they go to different parts of the answer — one is about the snapshot product, one is about the SSH endpoint's configuration — and a single ticket asking both tends to get one answered and the other dropped.

---

⚠ READ THIS BEFORE YOU SEND, AND BEFORE YOU BELIEVE AN ANSWER: two Hetzner products, opposite answers

Hetzner has two similarly-named storage products and their documentation says OPPOSITE things about restoring a single file. We are a Storage BOX. The answer that says "no" belongs to the other one.

	Storage Box — what we have	Storage Share — not us
what it is	plain SFTP/SSH storage, sub-accounts, u629488	a managed Nextcloud instance
single file out of a snapshot	"You can download individual files or entire directories as usual" — docs.hetzner.com/storage/storage-box/snapshots/	"Currently, we only support restores for the full backup ZFS snapshot to a specific point in time" — docs.hetzner.com/storage/storage-share/faq/backup-snapshot/

How to tell the Storage Share page apart at a glance: it talks about Nextcloud's data cache, a database dump, and the konsoleH web interface, and it never mentions Storage Box.

Why this is load-bearing rather than trivia. A search for "Hetzner restore individual files from snapshot" surfaces the Storage Share FAQ, and it reads like a definitive "no". A support agent answering Question 1 could reasonably reply from that page and give us a wrong answer — one that would send R-95 to the top of the register for no reason. Question 1 below therefore names the product, quotes the Storage Box line, and asks about the main account specifically. If an answer comes back citing full-snapshot-only, check which product it is about before acting on it. Caught 2026-09-01 by the operator noticing the URL said storage-share.

---

Question 1 — can the MAIN account retrieve individual files from a snapshot?

Why it matters, in one line: if it cannot, the only route back is a whole-box rollback that hits every customer on the box and destroys every newer snapshot — which would mean the snapshots protect almost nobody in practice. This is the question that decides how urgent R-95 is.

Subject: Storage Box u629488 — retrieving individual files from a snapshot

Hello,

We use Storage Box u629488 with sub-accounts, and daily automatic snapshots are enabled.

We can reach /.zfs/snapshot from a sub-account, but it lists as empty, and no snapshot name we try can be entered. We understand sub-accounts may be restricted here.

Your Storage Box documentation says, under Snapshots: "You can download individual files or entire directories as usual."

Our question is about the main account: from the main account, over SSH or SFTP on port 23, can we read or download individual files and directories out of a specific snapshot — for example a single directory under one sub-account's home — without performing a snapshot restore of the whole Storage Box?

To be clear, this question is about a Storage Box, not about Storage Share. We are aware the Storage Share FAQ says only full-snapshot restores are supported; we are asking whether that also applies to Storage Box, because the Storage Box snapshot documentation appears to say the opposite.

If yes, please tell us the exact path we should use and how the snapshot directory is named.

If no, please confirm that the only way to get data out of a snapshot is the full "restore snapshot" action on the whole Storage Box.

Thank you.

How to read the answer.

• "Yes, from the main account" → per-file recovery exists, but it is an operator act in a browser or over the main account's own SSH, and it can never be something the product does for the customer. R-433 closes at that. R-95 stays where it is.
• "No, only a full restore" → the snapshots do not bound a single customer's exposure at all, because using them costs every other customer on the box their newer snapshots. R-95 becomes urgent and the transport change stops being optional.

---

Question 2 — is --append-only enforced on the rclone serve restic endpoint?

Why it matters, in one line: if it is enforced server-side, a compromised box cannot delete its own backups, with no new machine and no data migration. This is the question that could make R-95 disappear.

Subject: Storage Box u629488 — rclone serve restic endpoint and --append-only

Hello,

Your Storage Box documentation states: "Restic is natively supported with the SFTP backend. As another option, we support the restic backend, which is provided by Rclone over SSH." The restricted SSH shell on Storage Box u629488 (port 23) also lists rclone serve restic --stdio among the available server-side backends.

Our question is about how that command is run on your side: is --append-only enforced by you, or is the command line taken from what the client sends?

In other words, if a client connects and asks for rclone serve restic --stdio without --append-only, does it get a server that permits deletions?

If the flag can be enforced, is there any way for us to request that for this account or for individual sub-accounts?

Thank you.

How to read the answer.

• "Enforced server-side" or "can be enabled per account" → this is the cheap prevention the 2026-09-01 spike priced at a new always-on service plus either a mount in the hot path or migrating every customer's history. It needs none of that — the server already runs at the provider, and restic 0.14.0 already speaks the rclone: backend (measured, with a control: banana: → invalid backend, rclone: → the helper was executed). The remaining work is putting rclone in the controller image and switching the repository URL. R-95's root cause goes away.
• "The client supplies the command line" → the lead is worth nothing and should be recorded as dead, not left looking promising. Prevention then still needs a machine in front of the store, and the decision reverts to the spike's option 3 at its original price.

---

Where these came from

• R-433 — no snapshot is reachable from a sub-account by any name. 777,600 exact names in the vendor's YYYY-MM-DDTHH-MM-SS format over nine days, zero hits, with a passing control; /home and /.zfs are different filesystems and /home/.zfs does not exist. Question 1 exists because that measurement can only speak for a sub-account.
• R-436 — the rclone serve restic --stdio backend and restic's rclone: support, both measured, and since confirmed as officially supported by the vendor's own Storage Box access page ("we support the restic backend, which is provided by Rclone over SSH"). That page says nothing at all about append-only, in either its Rclone or its Restic section — so Question 2 is genuinely unanswered by the documentation and is not a question the docs could have saved us.
• The two-product trap above, caught 2026-09-01. The Storage Box command table on that same access page also matches, item for item, the help output measured live on our own sub-account — independent corroboration that we were reading the right product's surface.
• Evidence for both: documentation/audits/evidence-drill-r95-recovery-2026-09-01/.