Nine rows into OPEN-ITEMS.md and ROADMAP.md, matching each file's column shape.
R-156 and R-157 had lived only in audit documents - the identical "minted in a
spike doc and never carried across" failure the register already records for
R-153/R-154/R-155, caught by the catalog sweep's own section 8.0 while it was
happening. R-158 was minted by a second session the same day for an unrelated
finding, which is why the sweep's proposals were renumbered R-159..R-162 at filing
time. All nine IDs verified free in BOTH backlog files before use.
Part 0 settled the question the sizing item depended on, by reading:
mp1 is RETENTION, not staging, and neither of the two framings was right. A unit
is the KEPT copy on the app's OWN drive (backup.go:245-255); for an app with no
HDD_PATH the namespace falls back to the system SSD - "the SSD-only system-data
fallback" (appbackup/paths.go:26-27). There is no post-copy deletion: the only
prune is F5 residue-on-old-drives when an app MOVES (backup.go:1053-1112). So mp1
retains the units of driveless apps only - not every app, but not transient
either. Confirmed against the spike: sys_drive held exactly the four driveless
apps and not calibre-web, which had a drive and was still backed up.
A unit is volume tars + DB dumps only, never mp8 userdata
(recovery_unit.go:20-25), so a 1 TB photo library can never overflow one. And mp1
gates the WHOLE chain, not just Tier 1: Tier-2 mirrors the unit "(always)" from
RecoveryUnitPath (tier2.go:302,368) and Tier-3 carries it, so a unit that cannot
be written leaves both with nothing to copy.
Part 2 fired on both triggers - retention, and the fallback undocumented - so
07-backup-architecture.md gains section 7.5. Section 6.1 said a unit lives "on the
app's own drive", which is true and was the whole story only for drive-resident
apps; the no-drive case was undocumented, as was the sizing constraint. 7.5
records the mp0-50G-vs-mp1-20G mismatch, the measured ratios (DB app up to ~2x,
21.1GB -> 40.2GB; file-only 1.00x), and the bound this puts on D5's Lane-1
independence: restorable from the drive alone only while the unit still fits -
about 19 GB file-only, about 10 GB DB-backed. No number proposed; the ratio is the
operator's ruling (R-163).
R-159/R-160 marked SHIPPED only after verifying the template changes are in
app-catalog origin/main, and R-156's gate likewise (check-volume-persistence.py
present). papra is NOT fixed - referred - so R-156 stays open on that one app.
Ranked, with one line of reasoning each: R-157 first (an app can stay down
indefinitely with mechanism B silent on every channel), then R-161 (the gate
exists and nothing runs it, which is why R-156's class recurs - R-29's record is
three orphaned gates and one enforced), R-156, R-163, R-158, R-164, R-162.
Records controller v0.188.0 across the four coupled artifacts.
07-backup-architecture.md is the owning doc:
- new 7.4 = the recovery chain AFTER D5 (7.1 leg 1 superseded; leg 2,
the living-app dependency, explicitly unchanged so this is not read
as more than it is)
- 7.3 collapsed to history, with the correction that the target as
written (data_key-only) was tested in Part 0 and rejected
- 3 records that the two-lane split is now real, not just intended
- matrix rows 3 / 3c (new) / 13; 10.1 D5 itself shipped
Also: new capability-map row, D5 collapsed in ROADMAP + OPEN-ITEMS,
and R-127 filed in both (data_key flag unreliable; O4 can regenerate a
DB password that no longer matches the restored data directory).
The audit is named D5-drive-alone-restore rather than "...secrets..."
because .gitignore blocks *secret* -- a guard worth respecting, not
forcing past.
Four-artifact update per the coupling rule, plus the audit.
07-backup-architecture.md: §10.1 retitled CLOSED with the ruling and the D5
sentence; the FileBrowser network-share row flipped YES->NO, closed at the
PLACEMENT rather than at the bind; the exposure chain annotated with the fifth
surface (decommission-with-migrate guarded only its source) and the correction
that the boundary is the deploy POST, not the dropdown; §7.3 retitled UNBLOCKED;
register row collapsed; open question F answered.
00-capability-map.md: new §D row PROVEN-LIVE, with the un-exercised legs named —
the deploy-POST and decommission refusals are unit-tested, not live-fired.
OPEN-ITEMS.md: R-108 dispositioned; D5 given its OWN row as READY/UNBLOCKED (it
had existed only inside other rows' prose — the R-123 thread-loss pattern);
R-126 registered.
ROADMAP.md: R-108 collapsed to a shipped one-liner; R-126 added.
R-126 filed not fixed: a .fab bundle (plaintext secrets, optional password) can
be exported ONTO a NAS. Split out of R-108 rather than folded in — it is an
explicit customer-chosen export destination, not a browsing surface reaching a
backup tree, so it was never part of D5's precondition.
Live evidence: same-box before/after on demo-felhom through the real authenticated
endpoint, the network-specific refusal on demo-hp, non-effect verified in the
registry, and R-67's share-root bind diffed byte-identical across the deploy.
Replaces the 2026-07-14 DRAFT entirely - it was verified against controller v0.132.0,
51 versions stale, while being cited as authoritative. Now verified against controller
0.183.0 / agent 0.110.0 / hub 0.80.0 / catalog 4252121. NOT ratified.
Every statement is labelled [DESIGN] (a decision from the architecture discussion) or
[FACT] (file:line, live output, or a citation to _recovery-inventory-2026-07-28.md).
Phase 0 verified D5's premise and it FAILED. Ten read surfaces were checked; nine are
clean (SMB under both namespace shapes, FileBrowser for local drives, .fab import and
download, /api/debug/*, all three ServeFile sites, registering the backup dir as a
drive). One is not: FileBrowser binds a NETWORK share at its ROOT while local drives
are userdata-scoped, and no IsNetwork() filter guards the deploy dropdown, the per-app
migrate targets, or handleStorageMigrateApp. So an app namespace - and under D5 its
plaintext secrets - can land somewhere a browsing surface serves with download:true.
Not a leak today (the unit's app.yaml is secret-stripped); it becomes one under D5.
D5 is therefore written as a BLOCKED target, not as settled. -> R-108
Core artifact is the failure -> recovery matrix (15 rows). RTO cells carry only
measured durations; 12 are blank and each blank is listed explicitly as a finding.
RPO cells carry the configured cadence that bounds RPO, labelled as such - no RPO has
ever been measured from an incident and none is invented.
The 7/53 vs 9/43/1 Tier-2 coverage counts are both recorded, unresolved; neither adopted.
Coupling (S-1): capability map section C now cites the matrix instead of restating
routes; ROADMAP gains R-102..R-108; OPEN-ITEMS gains only the in-flight R-108 plus
C9-F1b/F3/F4 -> R-n cross-references; CONTEXT records D1-D6 as standing ruling S-3.
No code, no version bump, no deploy, no box touched.
The last open item from the datastore relocation. Hub operator UI
(Offsite -> PBS DR) reports felhom-offsite (ep0) at 97.9 GB capacity,
12.6 GB used, 13% full - agreeing with the on-box df (98 G / 13 G / 13%).
The gauge follows the datastore's CONFIGURED PATH, so the relocation
required no hub-side change. RUNBOOK section 10.3 warned that a stale
37.2 GB reading would mean the gauge reads the wrong filesystem and
would be a real bug worth a roadmap item - it does not, and there is
no bug.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018ARoadHBf8rHoscfiqeVZn
Supervised runbook execution. No code change, no version bump.
felhom-offsite moved from ep0's 40 GB root disk (/srv/pbs-felhom) to a
dedicated 100 GB Hetzner Cloud Volume (/mnt/pbs-datastore, ext4 -m 0,
by-id fstab, relatime). Datastore NAME unchanged, so the PBS-DR
descriptors, per-box storage ids, ACLs and namespaces are untouched.
Capacity: 37.2 GB -> 98 GB total, 28.9% -> 13% used, headroom to the 80%
warn 19 GB -> ~65 GB. This CLEARS the R-82 Phase 0 P0.3 STOP. Per-tenant
encryption still precludes cross-customer dedup, so the slope is
unchanged - the volume buys runway, not a better cost model.
Verified: byte totals and chunk counts identical (9748), 7/7 snapshots
across all three namespaces, backup:backup ownership, clean itemised
dry-run, full verify job TASK OK with 0 errors, and a restore round-trip
(source_tier pbs, pass true, mount_parity ok, clean teardown).
Nothing deleted - the original 13 GB stays at /srv/pbs-felhom as the
rollback until a new weekly backup lands. GC deliberately not run.
Three findings recorded:
- the `scratch` datastore points at a non-existent path (pre-existing;
now logs ENOENT every start) - operator decision
- the runbook's S6 guard test proves the wrong proposition:
RequiresMountsFor re-mounts rather than refusing, so the test only
bites when the device is genuinely unavailable (re-run that way, and
the refusal was observed) - amendment recommended
- S11: storage box u629193 has no live backup path, BUT ep0 carries an
enabled sshfs mount unit against it that must be removed before the
box is deleted
Deviations: the volume arrived pre-formatted and mounted; S8 ran on
demo-felhom rather than demo-hp (no SSH key for demo-hp); the window was
contended by a stale in-memory 10-minute restore-test cadence whose
config had already been reverted on disk.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018ARoadHBf8rHoscfiqeVZn
- ROADMAP: R-85 row. Code SHIPPED; rotation NOT YET OBSERVED LIVE, stated as
such rather than written as done.
- Capability map: a new row for UNATTENDED restore-proof, IMPLEMENTED not
PROVEN-LIVE, kept distinct from the R-82 row that a MANUAL selftest earned.
That distinction is the same one the activation-vs-arrival split made.
- 03-host-agent §8: the scheduler covers every tier, oldest-proven first; the
spec is per-run; a restore-test joins the one-heavy-op gate. The safety
properties that must not be re-derived are listed.
- 07: restore-proof recorded as a per-tier property. Doc still NOT ratified.
- 06: corrects S4.1's 'the offsite restore-test now runs unattended' — it
silently stopped being true when local_backup_target was retargeted to 'local',
the SECOND time in that doc that a correct mechanism was broken by its input
changing underneath it.
- CONTEXT + REUSE.
Hub gate green (17 packages, rc=0).
Slice D.1 — host-install 1.20.0: a FRESH box defaults to local-daily +
offsite-weekly (felhom-pbs, 604800s, keep_last=2). setdefault semantics proven
both ways: fresh gets the tier, an UPGRADE preserves the existing backup block
verbatim — so an in-place upgrade can never silently start writing to an
offsite datastore. Existing boxes are migrated explicitly.
Slice E:
- 07-backup-architecture.md: honest status header per CONTEXT ruling S-2, with
an explicit STALE-outside-the-PBS-tier verdict (the controller tiers were last
verified 41 controller versions ago). The PBS row claimed 'PBS on DooPlex'
(the retired spike store) with no cadence; it now names felhom-pbs ->
felhom-offsite on ep0 over wg-felhom, weekly, keep_last=2. NOT marked
ratified — that is Viktor's review of the section 10 list. Discharges R-83.
- 06-offsite-connectivity.md: the target-split remaining-work note collapsed
(shipped), and records HOW S4.1's tier-aware timeout silently regressed — the
mechanism was never removed, its INPUT changed when local_backup_target was
retargeted to 'local'. Also notes S4.1 already diagnosed the teardown 403 as a
phantom (a timeout consequence, not an ACL gap).
- capability map: new row for recurring offsite backups actually LANDING, as
distinct from the existing row proving ACTIVATION. IMPLEMENTED, not
PROVEN-LIVE — the restore round-trip has not completed under the fixed code.
- ROADMAP: R-82 SHIPPED with its remaining gate named, R-83 DISCHARGED, R-84
left open.
- CONTEXT + REPORT: the arc, including the mid-arc correction I had to make.
Per the operator's 2026-07-26 ruling: datastore growth deferred, R-82 proceeds.
Records the measurements so the constraint is written down rather than carried
in a session: 37.2 GB total / 10.8 GB used, no cross-customer dedup (per-tenant
encryption keys), 80% warn reached at roughly the second additional customer,
and the pvesm 0/0/0 reporting artifact that means operators must read fill from
the hub gauge. Also records what the tier does and does not carry, and the
conditional on the P0.1 weekly verdict (Tier-3 offsite must be healthy).
Doc NOT marked ratified — that stays Viktor's review (R-83).
admin
Claude Code