admin
5f35aa0346
The S figures (66 MB -> 42.0s, two passes agreeing to 0.6s) had a spread tight enough to prove fixed work dominates, which is exactly why they said nothing about M. Second point taken 327x larger, same app, same method: clock from restore request to the app serving the correct discriminator. rallly's postgres volume grown 66 MB -> 21.1 GB (200k rows, STORAGE EXTERNAL so TOAST cannot compress it into a fake number). Two reps: rep 1 backup 406.4s unit 41149 MB RTO 624.5s discriminator correct rep 2 backup 387.2s unit 41133 MB RTO 591.8s discriminator correct 327x the data cost 14.5x the time - strongly sub-linear: RTO ~= 40s + 26.9 s/GB backup ~= 29s + 17.4 s/GB 10 GB -> 5.2 min 20 GB -> 9.6 min (measured 10.1) 100 GB -> 46 min The fixed ~40s dominates below ~1.5 GB, which IS the S band and explains its tight clustering. The more consequential result is capacity. A DB-backed app's recovery unit is 1.90x its data (volume tar PLUS SQL dump): 21.1 GB produced a 40.2 GB unit. The default appliance ships /mnt/sys_drive at 20 GB, so the largest app that can hold a local Tier-1/2 recovery unit on a default box is about 10 GB - and that fills the volume. The M band does not fit on a default box at all; this test only reached 21 GB because sys_drive was first grown 20G -> 70G with the same operation the product performs via SysDataGrowGB. A tier-sizing decision, not a defect, but it is invisible until an app crosses it. Caveats stated in the doc: two points define a line but do not test linearity; the 1.90x is DB-app-specific and a file-only app should be nearer 1.0x (inferred, not measured); synthetic incompressible data; one app, one box.
3.8 KiB
RTO — the M band, measured (2026-08-02)
The S-band figure (66 MB → 42.0 s, two independent passes agreeing to 0.6 s) had a spread tight
enough to prove fixed work dominates, which is exactly why it said nothing about M. This adds a
second point 327× larger, measured on the same app, the same way: clock from the restore request
to the app serving the correct discriminator, read over the path DATABASE_URL actually names.
Method
rallly's postgres volume grown 66 MB → 21.1 GB by inserting 200 000 rows into a cc_bulk table.
ALTER TABLE … SET STORAGE EXTERNAL is load-bearing: without it TOAST compresses repeat(...) to
almost nothing and the volume never really grows, which would have produced a fake M-band number.
The cc_proof canary row is untouched, so the discriminator is unchanged.
/mnt/sys_drive was first grown 20 G → 70 G with pct resize — the same operation the product
performs via SysDataGrowGB (felhom-agent/internal/reconcile/bringup.go:426). It had to be:
see the capacity ceiling below.
Results — 2 reps, both returning the correct discriminator
| rep | volume | backup | recovery unit | RTO to correct data | discriminator |
|---|---|---|---|---|---|
| 1 | 21 616 MB | 406.4 s | 41 149 MB | 624.5 s | correct |
| 2 | 21 615 MB | 387.2 s | 41 133 MB | 591.8 s | correct |
| mean | 21.1 GB | 396.8 s | 40.2 GB (1.90×) | 608.1 s (10.1 min) | 2/2 |
The two-point model
| 66 MB (S) | 21.1 GB (M) | ratio | |
|---|---|---|---|
| data | ×1 | ×327 | |
| RTO | 42.0 s | 608.1 s | ×14.5 |
Time grew 14.5× for 327× the data — strongly sub-linear, i.e. a large fixed cost plus a byte-proportional term:
RTO ≈ 40 s + 26.9 s/GB · backup ≈ 29 s + 17.4 s/GB
| size | projected RTO |
|---|---|
| 1 GB | 67 s |
| 5 GB | 2.9 min |
| 10 GB | 5.2 min |
| 20 GB | 9.6 min (measured 10.1 min — the model is fit through the mean) |
| 50 GB | 23.1 min |
| 100 GB | 45.5 min |
Where the bands actually break. The fixed ~40 s dominates below roughly 1.5 GB — that is the S band, and it is why the S numbers were so tightly clustered. Above it the per-GB term takes over and RTO becomes essentially linear in data.
The capacity ceiling — the more consequential result
The recovery unit is 1.90× the app's data, because for a DB-backed app it carries both the volume tar and the SQL dump: 21.1 GB of data produced a 40.2 GB unit.
The default appliance ships /mnt/sys_drive at 20 GB (mp1 … size=20G, the golden's baked size;
SysDataGrowGB is a per-customer grow that was 0 for this customer). So on a default box:
the largest app that can hold a local Tier-1/Tier-2 recovery unit is ≈ 10 GB — and that fills the volume completely. A realistic ceiling is nearer 8 GB.
The M band does not fit on a default box at all. This test only reached 21 GB because
/mnt/sys_drive was grown to 70 G first. That is a sizing decision for the S/M/L tiers, not a defect
— but it is the constraint that actually bites, and it is invisible until an app crosses it.
Caveats — stated, not buried
- Two points define a line; they do not test linearity. A third point (~5 GB) would confirm the model's shape. Not taken.
- The 1.90× ratio is DB-app-specific. It is volume-tar + SQL-dump. A file-only app (sqlite, no DB container) should be nearer 1.0×, but that was not measured and is inference.
- Synthetic data.
repeat(md5(...))stored uncompressed is deliberately incompressible; real app data that compresses well would move both the unit size and the per-GB rate. - One app, one shape, one box. The fixed term in particular is a property of this stack's stop/start and health-wait, which differs per app.