admin/felhom.eu
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

hub v0.12.0: retire Infra Backup, purge its plaintext secrets, fix backup-deadline email

Browse Source 
Phase-1 of SPIKE-infra-backup-2026-06-15. The infra-backup mechanism was dead
since slice 8C yet stored plaintext customer secrets at rest (app-secret key,
restic password, Cloudflare tokens) — a zero-knowledge violation — and its
absence made the daily expected_backup_missed email fire for healthy customers.

- Repoint monitor.CheckBackupDeadlines backup half to the agent host-report's
  PBS snapshots (+vzdump): alarm only on no-backup / >26h stale / verify failed.
  Keep the db_dump half. No host-report → no backup alarm (liveness owns that).
  New store.GetLatestHostReportJSON. Tests incl. a companion that fails pre-fix.
- Remove the infra-backup endpoints, store methods/types, and operator panel;
  /recovery now returns config_yaml only.
- migrate(): DROP infra_backup_versions/infra_backups + VACUUM (+wal_checkpoint)
  to physically reclaim the plaintext pages, gated on table existence.

Flagged out-of-scope: exposed creds need operator rotation; legacy reports table
holds historical plaintext restic_password rows (separate leak, not purged here).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
authentik Default Admin
2026-06-16 11:08:06 +02:00
parent 2f7acb7d07
commit 0635640848
10 changed files with 466 additions and 595 deletions
Download Patch File Download Diff File Expand all files Collapse all files
hub/README.md
+16 -27
View File
@@ -18,7 +18,7 @@ A lightweight Go service that receives periodic reports and structured events fr
│ │ │ ┌─────────────────┐ │
│ POST /api/v1/ │ │ │ API Handler │ │
│ report │ │ │ (ingest reports, │ │
│ infra-backup │◀── config push ────│ │ infra backups, │ │
│ host-report │◀── config push ────│ │ host reports, │ │
│ notify │ (YAML body) │ │ config push, │ │
│ │ │ │ asset serving) │ │
│ GET /api/v1/ │ │ └────────┬────────┘ │
@@ -27,7 +27,7 @@ A lightweight Go service that receives periodic reports and structured events fr
│ │ SQLite Store │ │
Operator browser │ │ (reports, │ │
┌─────────────────┐ │ │ assets, │ │
│ Web Dashboard │◀── HTML pages ──────│ │ infra_backups, │ │
│ Web Dashboard │◀── HTML pages ──────│ │ host_reports, │ │
│ (hub.felhom.eu) │ (bcrypt auth) │ │ configs, │ │
└─────────────────┘ │ │ notifications) │ │
│ └─────────────────┘ │
@@ -62,44 +62,34 @@ All API endpoints require `Authorization: Bearer <api_key>` (except `/healthz` a
The `POST /api/v1/report` handler (v0.4.0+) automatically parses the optional `app_telemetry` JSON array from the request body and stores it in `app_telemetry` / `app_log_issues` tables. Old controllers (no `app_telemetry` key) continue to work unchanged.
### Infrastructure Backup (Disaster Recovery)
### Infrastructure Backup — RETIRED (Phase-1, 2026-06-16, hub v0.12.0)
| Method | Path | Description |
|--------|------|-------------|
| `POST` | `/api/v1/infra-backup` | Controller pushes infrastructure snapshot |
| `GET` | `/api/v1/infra-backup/{customer_id}` | Fresh controller pulls backup for restore |
The infra-backup payload contains everything needed to restore a customer deployment:
- `controller.yaml` (base64, full config including secrets)
- `settings.json` (base64, backup preferences, storage paths)
- Disk layout (UUIDs, labels, mount points, fstab options, bind-mount topology)
- Deployed stacks manifest (app names, HDD paths, display names)
- Restic passwords (primary + cross-drive, for encrypted backup access)
**Disaster recovery flow:**
1. Customer's system drive fails → replaced with fresh Debian install
2. `docker-setup.sh` deploys controller with minimal config (domain only)
3. Controller enters setup wizard → user chooses restore from local drive or Hub
4. For Hub restore: calls `GET /api/v1/recovery/{customer_id}` (gets config + infra backup)
5. Controller uses disk UUIDs to auto-mount surviving drives
6. Controller restores apps from local backups on those drives
The Infra Backup mechanism (`POST/GET /api/v1/infra-backup`, the operator panel, the
`infra_backup_versions` / `infra_backups` tables) has been **removed**. It had been dead since
slice 8C (the disk-tier backup moved to the host agent), and it stored each version as a **plaintext
JSON blob** holding the customer's app-secret encryption key, restic password, and Cloudflare tokens —
a zero-knowledge violation. The retirement migration `DROP`s both tables and `VACUUM`s the DB to
physically reclaim the plaintext pages. Disaster recovery now rests on the agent's **PBS whole-CT
snapshot** (the data bytes) plus the generated `controller.yaml` from the recovery endpoint (the
config); a secret-free **DR recipe** is the later DR slice's job. See
`documentation/audits/SPIKE-infra-backup-2026-06-15.md`.
### Recovery (Disaster Recovery)
| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/api/v1/recovery/{customer_id}` | Combined recovery: returns generated controller.yaml + infra backup in one response |
| `GET` | `/api/v1/recovery/{customer_id}` | Returns the generated controller.yaml for a customer |
Auth: `X-Retrieval-Password` header (same per-customer password as config retrieval). Response:
```json
{
"customer_id": "example",
"config_yaml": "customer:\n id: example\n ...",
"infra_backup": { ... },
"has_infra_backup": true
"has_infra_backup": false
}
```
If no infra backup exists yet, `infra_backup` is null and `has_infra_backup` is false.
The `has_infra_backup` field is retained as `false` so any old client degrades gracefully to the
config-only path.
### Report Response
@@ -228,7 +218,6 @@ SQLite with WAL mode. Tables:
|-------|---------|
| `reports` | Full JSON reports with denormalized fields for dashboard queries |
| `events` | Structured events from controllers and Hub (type, severity, message, details, source) |
| `infra_backups` | Per-customer infrastructure snapshots for disaster recovery |
| `customer_notifications` | Email, enabled event types, cooldown hours per customer |
| `notification_log` | Send/skip/fail history for notifications with channel (operator/customer) |
| `customer_configs` | Pre-configured customer settings, retrieval passwords, per-customer API keys, status (active/blocked) |