Commit Graph

102 Commits

Author SHA1 Message Date
admin 9b3381be0a feat(hub): v0.70.0 — a deleted customer actually disappears (residue leg + ghost cleanup)
Found validating v0.69.0 against the live hub. demo-vm-felhom was deleted
on 07-18 and was still on the Customers list AND still raising offsite_stale
(10 events, latest 07-21 17:34, operator email at 19:34) — because
GetCustomers() is report-derived and no lifecycle tier ever deleted a report.

New leg 3 (residue), before the record purge: reports, app_telemetry,
app_log_tails, log_tail_requests, customer_notifications, plus the
credential-bearing appliance_registrations and selfbind_tokens. Audit
(events, notification_log) and F-14 provenance still survive.

Ghost customers are now deletable: 404 means "nothing here", not "no config
row". With no config row the offsite descriptor is unknowable, so the Hetzner
and descriptor legs record skipped_no_config rather than a bare "skipped".

Two more red-proofs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J55BQE1gE2V4ffud5jweGS
2026-07-21 20:28:06 +02:00
admin 61dbd870c3 feat(hub): v0.69.0 — customer DELETE is the guided full-teardown cascade (R-25b)
POST /configs/{id}/delete now runs hosts -> RESET -> purge behind three
acknowledgements, a typed customer-id, a stale-preview check and the
ONLINE-host refusal (every gate before any write, so a refusal has zero
side effects). The shallow handleConfigDelete is gone.

Two invariants are asserted, not just commented: ruling 3 is preserved by
construction (leg 2 never sees a host row) and retained escrow custody is
purged exactly once, in leg 3 (leg 2 runs with purgeEscrow=false).

handleCustomerReset's committed half was extracted as commitCustomerReset;
the standalone RESET path is byte-identical to v0.68.1 and its suite is
untouched. Five red-proofs run.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J55BQE1gE2V4ffud5jweGS
2026-07-21 19:31:48 +02:00
admin 4a4233059d hub v0.68.1 — fix the Configuration layout broken by the wrapper-sha field
The v0.68.0 row wrapped itself in a <div>, but the artifacts <form> IS the CSS grid
(display:grid, no inner container). The stray </div> closed the surrounding card from
inside the form and the new <div> was never closed — it swallowed the submit button and
ran to </form>, so the row rendered outside the card and Save landed inline. Reported by
the operator on first use.

The field still submitted (it stayed inside the form), so this was layout damage rather
than data loss, but the unbalanced markup put every section below it in the wrong
container.

Fixed as plain grid cells (grid-column: 2/4), no nested elements.

There was no render assertion on this form at all, which is why a hand-edit broke it
silently. The new test asserts the field is inside the form, the button has not escaped,
the form contains ZERO divs, whole-page div balance holds, and the sections after it
survive. Red-proofed against the broken shape.
2026-07-21 10:34:34 +02:00
admin 107f74ea3c hub v0.68.0 — auth_failed self-heal, consumed_at honesty gauge, wrapper drift (R-39 + R-50b(a))
Completes the hub half of R-39's fleet fix on top of the generation core (c484aa2).

pbsdrheal gains an auth_failed TRIGGER — a new trigger in the existing machine, not a
new machine. A box whose credential PBS rejects escalates to a fresh mint, never a
re-stage (which would re-feed the secret PBS just rejected), through the EXISTING damper:
a 401 flap must not become a secret-minting chain. With the generation stamp this closes
the loop end to end — agent proves the 401, hub re-keys, generation advances, descriptor
hash moves, agent re-consumes.

consumed_at honesty gauge: a staged secret still unconsumed past a 15-minute grace while
the box reports `applied` is surfaced with its own event. That is the exact 2026-07-18
fingerprint and a disagreement no single tier can see alone. Deliberately a SURFACE, not
a heal — auto-re-issuing on it would mint a second secret on top of an unconsumed one,
which is the mint/consume race R-39(a) already recorded. One event per distinct report,
and an honestly-stuck box does not double-report (its unconsumed secret is the symptom
being healed, not a contradiction).

R-50b(a): ArtifactManifest.WrapperSHA256 + operator field + host-page drift surface. The
PBS wrapper is root-owned 0755 and the pinned sudoers vector, yet installed unversioned
from raw/branch/main and absent from every manifest. Agents >=0.91.0 report the installed
hash; a mismatch is surfaced. An unknown on EITHER side reads as quiet, never as drift —
lighting every host amber on rollout day is how a warning becomes background noise. The
delivery channel itself stays R-50b(b)/(c).

Compatibility unchanged: safe for 0.90.0 agents (unknown JSON key dropped); the re-arm
and auth-honesty guarantees need agent >=0.91.0, so MinAgent moves only after the fleet
has self-updated.

Tests: auth_failed escalate/debounce/recovery-forgets-streak; honesty gauge incl. grace
window, the restage edge (consumed_at deliberately NULLed), consumed-never-alarms, and
honest-stuck-no-double-report; wrapper drift incl. both unknown directions. Red-proof run
at the assertion level: removing the auth_failed arm fails the escalation tests with
reissues=0.
2026-07-21 10:01:35 +02:00
admin b6d537d86c hub v0.67.0 — auto-minted self-bind link, post-RESET staleness, unprovisioned-offsite warning
Four small items, each a case where the hub already knew something and said
nothing. Green: build, vet, tests all pass.

(a) Self-bind link is minted automatically at customer creation AND at RESET
    completion (R-36 sub-item). The console banner tells the customer to open
    "az e-mailben kapott link"; until now that email existed only once the
    operator remembered the button, so the banner could point at something that
    did not exist — during the 2026-07-18 rehearsal the box waited ~11.7 min on
    exactly that. handleSelfBindLinkSend's body was extracted into a shared
    mintAndSendSelfBindLink core so the button and the auto-mint callers cannot
    drift apart on the honesty rules: F1 (no address -> mint nothing) and F2
    (send failed -> delete the token, never leave it live). The wrapper NEVER
    fails the operation it rides on — a create that provisioned Cloudflare,
    offsite and PBS must not 500 over a courtesy email.

    Gap found and closed while wiring it: PurgeCustomerResetDBState does NOT
    clear selfbind_tokens, so a link minted BEFORE a reset would have stayed
    live across it. A successful mint already replaces it (delete-then-insert,
    single-active); the skip paths would not have, so they now clear stale
    tokens too. Invariant: after auto-mint runs the only live link is one it
    just issued, or none.

(b) Post-RESET staleness banner (R-37). When a RESET COMPLETED after the newest
    report, every health figure on the page describes a lifecycle that no longer
    exists, and the page kept showing pre-RESET warnings as current. Narrow on
    purpose: an in-flight reset does not trigger it, and it clears itself when a
    report arrives. Ties resolve to STALE — SQLite timestamps are second-
    resolution and a same-second report almost certainly predates the reset;
    erring the other way would hide the banner exactly when it matters.

(c) Unprovisioned-offsite warning (R-36 interim). enabled==true with type=="" is
    a real, stable, silent state: provisioning is Save-triggered and the
    re-enroll auto-re-issue deliberately skips an unprovisioned target, so
    nothing self-heals it. Reuses the exact predicate the offsite re-issue
    handler already refuses on.

(d) pbsdr_reissued rendered an EMPTY flash box — the key had no template branch,
    so re-issuing PBS credentials showed a success box with no words (observed
    live 2026-07-18). Now describes what was staged plus the R-39 caveat:
    confirm `pvesm status` shows the entry active, because a converged agent can
    report `applied` while the storage still 401s.

New .flash-warn (amber, --warn tokens) for the deviation tier between success
and error — exception-color principle: only on deviation, never on a healthy
page.

Tests assert each banner is ABSENT in the nominal cases as well as present in
the deviating one — a banner that always renders is worse than none. Both
red-proofed: deleting the pbsdr_reissued branch reproduces the original empty
box; neutering the staleness predicate fails the banner assertion. New
read-only store accessor CountSelfBindTokens makes the single-active invariant
assertable.

NOT in this train: the R-39 hub-side generation-bump fix the pre-travel task
made conditional. Its condition was REFUTED (SetHostDesired bumps
unconditionally; applyPBSDR is idempotent as documented) — the real mechanism is
the agent's descriptor-hash convergence and needs its own spec.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nn3VgQk9iwEGgyx6QJ2NvE
2026-07-18 21:45:11 +02:00
admin 592818492c hub v0.66.0 + ISO v1.20.0: customer self-bind (R-27 slice 1)
Let a customer bind their own freshly-installed appliance without the
operator: operator "Send self-bind link" mints a 7-day tokenized
capability link, emailed (Hungarian, sibling sender) to the customer, who
opens a public /bind/<token> page and proves two factors — the console
pairing code shown on the box screen + their retrieval passphrase — and
the hub stages the bind via the same BindAppliance (provenance
customer_selfbind). The box's ~30s appliance poll delivers.

Viktor's three rulings verbatim: console pairing code (no appliance list
ever rendered), operator-sent tokenized link, 5-attempt lockout ->
"call support". Wrong code == wrong passphrase (one generic failure, no
oracle, both factors compared unconditionally); expiry falls back to
operator-bind unchanged.

THE TRAP: one public prefix /bind/, exempt from auth+CSRF at both /login
gate sites via a single isPublicBindPath predicate (tight trailing-slash
match; ServeMux ..-cleans; handler rejects '/' in token). 9 tests
(Scenarios A-F + F1/F2); 4 red-proofs verified red-then-green (lockout,
oracle, widened-prefix, single-active). GC verdict: no appliance GC ->
the 7-day TTL stands alone. Controller/agent untouched; R-27b deferred.

Green: full hub build/vet/test (17 ok) + bash -n + hub confirm gate.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017qDiBqKKQ5vPB5fXBqu7Kp
2026-07-17 23:56:53 +02:00
admin 7f11cfb36c hub v0.65.0 — PBS DR storage visibility (ep0 usage op) + Offsite tab split + dual dashboard gauges (R-5)
Makes PBS DR storage visible like the restic pool box (v0.64.0), differentiated. Scoping
correction: restic = subaccounts on the shared Hetzner Storage Box (Hetzner API); PBS DR =
the felhom-offsite PBS datastore on the ep0 endpoint VM (NO Hetzner API). Option A
(Viktor-ruled): a read-only `usage` op on the felhom-tenantsync ep0 forced command (twin of
fingerprint), polled by a new hub checker on the 15-min throttle. READ-ONLY throughout.

Phase-0 (gate PASSED): on ep0 (PBS 4.2.3), df -B1 --output=size,used,avail <datastore path>
yields bytes (39990112256/7627939840/... ~19%), read-only, existing sudo context, no admin token.

- scripts/felhom-tenantsync.sh -> v1.2.0: read-only `usage` short-circuit (df on the datastore
  path), no customer_id, no admin token, NO mutation. + a bash harness proving zero mutation.
- tenantsync.Client.Usage() + BoxUsage; unknown-op -> typed ErrUsageUnsupported (graceful).
- monitor.PBSDRBoxChecker: OffsiteBoxChecker clone over a usageReader seam; 15-min throttle,
  cached PBSBoxSnapshot, escalation-only pbsdr_box_fill on the "pbsdr-box" scope (operator only,
  no SaveEvent), recovery re-arm. Fill only. THREE states: ok / unavailable (ep0 <=v1.1.0,
  neutral no-alert) / degraded (exec failed, keep last).
- config: Alerting.PBSDRBoxFill{Warn,Crit}Percent (80/90); built with the tenantsync client,
  60s sweep, SetPBSDRBox. Hub deploy INDEPENDENT of the ep0 update (graceful degradation).
- web: /offsite splits into Restic + PBS DR hash tabs (endpoint cards under PBS DR); PBS panel;
  the single dashboard tile becomes two gauges (RESTIC pct.ratio, PBS DR pct / n/a).
- runbook offsite-endpoint.md 10: v1.2.0 update steps (no sudoers/authorized_keys change).

Tests: 10 Go + the harness; 3 red-proofs (usage mutation, escalation-only, unavailable-drives-band)
confirmed red then restored. go build/vet/test + bash -n + hub confirm gate all pass.
2026-07-17 21:13:30 +02:00
admin 4bb2df0dc4 hub v0.64.0 — offsite pool-box aggregate: fill, oversubscription, per-customer bars, operator alert (R-5)
The operator sees the shared pool box's real state on the hub: total box fill vs
capacity, Σ(shared soft quotas) vs capacity (the oversubscription ratio), per-customer
usage/quota bars, and a box-level operator alert (fill % + oversub ratio) on the existing
dispatcher's operator channel. Per-customer fill alerts already existed; the box-level
aggregate was the gap. READ-ONLY against Hetzner (GET only).

Phase-0 probe (gate PASSED): the live pool box 611714 returns capacity via
storage_box_type.size (1 TiB / bx11) and usage via a stats object (size/size_data/
size_snapshots), all bytes; our token reads it (200).

- hetznerapi: additive StorageBoxType + StorageBoxStats on StorageBox (no existing field/
  method changed); fake carries them + a GetBoxCalls counter; golden decode test.
- monitor.OffsiteBoxChecker: OffsiteChecker-sibling for the box; fetch-throttled (1 GET/
  15min), cached BoxSnapshot, escalation-only + recovery re-arm. FILL (used/capacity 80/90)
  + OVERSUB (Σ shared+enabled quotas / capacity, 2.0x) — independent. Σ from the ConfigJSON
  Descriptor (offsite.ReadDescriptor, new), never the report echo; dedicated+disabled
  excluded. Scope "pool-box" -> operator channel only, no SaveEvent. Failed fetch keeps the
  last snapshot degraded; missing data never becomes 0% and never transitions a band.
- config: Alerting.OffsiteBoxFill{Warn,Crit}Percent + OffsiteOversubWarnRatio (80/90/2.0
  defaults; thresholds pending Viktor's ruling). Constructed in the HETZNER_TOKEN branch,
  60s sweep, snapshot handed to the web server.
- web: Offsite-tab panel (fill bar, Σ+ratio, per-customer usage/quota rows) + a compact
  dashboard tile; reads the cached snapshot only, never fetches; nil -> "not configured".

Tests: 10 new + 4 red-proofs (throttle, Σ filter, escalation-only, failed-fetch honesty),
all confirmed red then restored. go build/vet/test all pass; hub confirm gate OK.
2026-07-17 20:15:34 +02:00
admin 30972d8f54 hub v0.63.0 — system-initiated immediacy: wire poke/bump at every mutation site that lacked one
The immediate-sync arc covered only operator-initiated desired-state changes;
system-initiated mutations bumped the generation silently, so a freshly onboarded
box waited a full agent tick for state the hub had already minted (observed live at
slice-C onboarding). Wire the existing, live-proven notifiers into every system site
on the correct plane — call-site wiring only, no new mechanism.

Agent plane (poke.Notifier):
- web/pbsdr.go: PBSDRAutoProvision (the observed lag), ReissuePBSDR (also lifts the
  pbsdrheal reconciler escalation, zero reconciler changes), handlePBSDRReissue —
  each pokes AFTER the successful SetHostDesired, never on a blocked/error path.
- api: new nil-safe Poker seam (PokeHost/PokeAllHosts + SetPoker); handleAdminSetDesiredState
  pokes the target host; handleAdminSetOperatorPeer fires PokeAllHosts only when the
  fleet generation bump succeeded (fire-after-commit).
- main.go: one poke.Notifier now feeds both planes (SetPoke + SetPoker).

Controller plane (intent.Hub.Bump):
- api/reissueOnReenroll: one nil-guarded bump so a long-polling controller wakes in
  seconds instead of on the 15-min cycle.

Deliberate non-sites (unchanged): WG register (undeliverable pre-tunnel — the agent
fast-tick SECONDARY owns it), WG delete (transport removed), pbsdrheal Restage (no
generation bump → the 60s ticker is the pickup path). internal/pbsdrheal byte-unchanged.

Tests: 10 non-hollow tests (web async channel-synchronized fake sender; api synchronous
fake Poker) with explicit zero-count negatives; representative red-proofs per group
(A/B/C/D) run-fail-restored. Green: go build/vet/test all pass.
2026-07-17 17:30:41 +02:00
admin 36c5cd5fdf hub v0.62.0 + scripts v1.19.0 — R-21 slice C: the universal secret-free ISO
A generic ISO carries NO customer secret. The box registers itself at the hub
as an unclaimed appliance; the operator binds it to a customer; the hub delivers
the customer-id + retrieval passphrase ONCE; day-0 completes via the slice-A path.

Hub (v0.62.0):
- store/appliance.go: appliance_registrations keyed by (uuid, mac_set) — MAC set
  is the tiebreaker (duplicate SMBIOS UUIDs); token stored as sha256 only.
  Idempotent register (sticky-discard), atomic one-shot delivery, bind/discard.
- api/appliance.go: POST /appliance/register (the one unauth endpoint, per-IP
  rate-limited, 256-bit token); GET /appliance/poll (404 no-oracle / 204 unbound
  / 200 deliver-once / 410 delivered). Passphrase read live, never logged.
- web/appliances.go: Hosts-page "Unclaimed appliances" section + BIND (customer
  picker, host count display-only) + DISCARD; SSH host-key fingerprints; events.
- Red-proofs: one-shot delivery + register idempotency (both proven red);
  404-no-oracle, sticky-discard, bind staging, render. Green + confirm gate.

Scripts (v1.19.0):
- felhom-bootstrap.sh: ONE unit, TWO modes. Direct (env has customer/passphrase)
  = slice-A path, byte-identical, only branched around. Pairing (generic) =
  register + poll (RestartSec=30 is the poll timer); on delivery write the env
  0600 and fall through to direct. Secrets + token shredded on success.
- build-felhom-iso.sh --pairing: generic secret-free ISO, -generic filename,
  manifest mode=pairing. profiles/generic.profile (new).
- test/bootstrap-modes.sh: Scenario D (direct = zero appliance calls) + pairing
  register/poll + delivery handoff — all green in a debian container.
2026-07-17 15:07:31 +02:00
admin 4009401f46 hub v0.61.0 + felhom-tenantsync v1.1.0: Customer RESET (middle lifecycle tier)
One operator action returns a customer to pre-first-install: all operational
state dies (offsite repo, PBS namespace+backups, DR recipe, one-time secret,
claim state, retained escrow custody); identity + basic config + provenance +
events survive. Sits between host delete and customer Delete.

- store/customer_reset.go: customer_resets journal, live inventory, ack-gated
  purge (never touches identity/provenance/events), DeleteClaim.
- claim.ResetToUnclaimed: delete claim row -> fresh code next onboarding.
- offsite.Deprovision (idempotent) + OffsiteIdentifier + ClearProvisionedDescriptor.
- tenantsync.Deprovision + felhom-tenantsync.sh deprovision op (destroys ns +
  backup groups + token; shared user untouched; idempotent).
- web/customer_reset.go: GET reset -> inventory JSON; POST -> orchestration
  (external teardown FIRST, DB purge LAST; refuse-while-hosts; typed-id +
  separate escrow ack). Amber RESET card distinct from red Danger-zone Delete.
- Red-proofs: ack-gate + partial-failure resumability (both proven red);
  store ack-gating + journal round-trip; offsite idempotency + descriptor clear;
  RESET-card render. Green: build + vet + test.
2026-07-17 13:09:04 +02:00
admin 2752e12acc hub v0.60.1: host deletion demotes escrow custody (never destroys) + customer-delete purge point + S6b obsolete
- DeleteHost(deleteEscrow) demotes current host_escrow into host_escrow_superseded (copy-before-delete, same tx), spares existing; one shared demoteCurrentEscrowTx (reused by SaveHostEscrow). F-14 provenance/gate unchanged.
- DeleteCustomerConfig now purges both escrow tables for all the customer's hosts incl. already-deleted (F-14 provenance UNION) — the one true purge point.
- Wording: checkbox/refusal/Danger-zone → demotion. S6b OBSOLETE. Red-proofs TestDeleteHost_Demotes + TestDeleteCustomer_Purges + wording guard.
2026-07-17 11:25:38 +02:00
admin e247dbc1be hub v0.60.0: offsite continuity Part B — superseded-escrow retention (data-first)
- host_escrow_superseded table + SaveHostEscrow retains a different-sha old blob before overwrite (tx); same-sha idempotent (no supersede row); returns superseded bool. ACK/restore read the current row unchanged. CountSuperseded/ListSuperseded; DeleteHost drops retained rows.
- escrow_superseded audit event + operator retained-count on host detail; register offbox_repo_orphaned/reset. Red-proof TestSaveHostEscrow_RetainsSuperseded.
2026-07-17 10:47:50 +02:00
admin eb227486d0 hub v0.59.0: Direction-2a agent-plane immediate-sync poke sender + ep0 felhom-poke surface
- internal/poke: pinned-host-key SSH poke sender (wgsync sibling) + fire-and-forget Notifier (PokeHost/PokeAllHosts). Poke refuses non-WG targets pre-dial; contentless via ep0 forced command to the box WG /32:51822.
- wiring: Server.SetPoke; applyPBSDR pokes the host after each descriptor gen-bump; handleSetArtifacts (MinAgent floor) pokes all hosts. main.go env POKE_SSH_KEY_FILE (reuses peersync endpoint/hostkey).
- scripts/felhom-poke.sh (non-root forced command) + offsite-endpoint.md §11; manifests/hub.yaml Secret/agent-poke + POKE_SSH_KEY_FILE (image tag bump follows the build).
2026-07-16 22:48:15 +02:00
admin 60244727ad feat(hub): Direction-2 immediate-sync wait channel (v0.58.0)
GET /api/v1/wait long-poll: the box holds an authed hanging GET; the hub
completes it the instant any operator intent bumps that customer's in-memory
generation, then the box fires its ordinary report and the ACK delivers
everything through the unchanged machinery. 240s hold with a 25s heartbeat
newline defeats the nginx 60s proxy_read_timeout with no ingress annotation;
WriteTimeout lifted per-connection via ResponseController.

- internal/intent: per-customer generation counter + waiter registry
  (Bump/Wait/Close), coalescing to latest, race-closer, in-memory by design.
  Red-proofs: counter-vs-queue + race-closer (run-fail-reverted).
- api/wait.go: the endpoint (per-customer only; global key 400; A cannot see B).
- web bumps after every intent write (fire-after-commit): config CRUD, claim
  resend, offsite re-issue/freeze, password regen, block/unblock, floors
  (global bumps all config-managed), controller log-tail + log-bundle.
- main.go: one intent hub shared by web+api; Close() before server.Shutdown.

Pairs with controller v0.140.0 (the long-poll client). Grounding:
documentation/audits/SPIKE-immediate-sync-transport-2026-07-16.md.
2026-07-16 20:44:22 +02:00
admin 7747a16ff1 feat(hub): v0.57.0 reinstall-of-existing-customer arc — claim/offsite/escrow continuity
F2 claim re-issue on clean-slate re-enroll (ReissueForReenroll, host-enroll mint path,
single-bump, reset code; hub never stores the password so fork B). F3 offsite re-issue on
re-enroll (ReissueOffsiteForCustomer, same machinery as the manual button). 2.3 escrow honesty
(red-proofed): re-issuing offsite marks the escrow stale (MarkEscrowStale), withholds the
mismatched restic hash from auto-confirm, DR checklist shows stale not done. Events:
claim_reissued_reenroll / offsite_reissued / escrow_stale.

Controller + scripts unchanged (source contradicted both premises): the controller reads escrow
prereqs live from the agent; the installer can't know the descriptor-provisioned storage id. F4
root fix is agent-side -> ROADMAP R-22; demo unblocked live (Part 0 ACL grant). VALIDATION doc
F2 erratum + F3/F4 dispositions. Green gate + Scenario-C red-proof pass.
2026-07-16 18:00:13 +02:00
admin 65544ae689 docs: record PBS-DR self-heal drill live-leg result (auto-healed hands-free)
Reconciler observed waiting_secret across two reports (16:52+17:07 UTC), re-staged
the stored secret at 17:10:00 (no ep0 token, no gen bump); agent re-consumed
17:10:26 and converged (applied) 17:10:28 — no operator click. Demo host untouched
(scoped). Fleet widening remains a deliberate follow-up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HEPuEwyyGDJdcsXLFsTWJn
2026-07-15 19:12:03 +02:00
admin 6218e7919d hub v0.56.0: PBS-DR self-heal reconciler (re-stage a consumable secret)
Implements SPIKE-pbsdr-selfheal-2026-07-15 (e8f8c44). A box re-installed/rolled
back onto its stable host_id loses its agent-side converged marker; the hub
keeps the enabled descriptor + a CONSUMED one-time secret, the WG peer persists
(changed==false, cascade can't re-fire), so the agent sits in waiting_secret
forever. The missing piece is a consumable secret, not the descriptor.

New internal/pbsdrheal reconciler (5m, wgsync shape): for enabled+provisioned
hosts whose latest report pbs_dr.state is a stuck state past a >=2-distinct-report
debounce, re-stage the stored secret (store.RestageHostPBSSecret: clear
consumed_at, no ep0 call, NO generation bump); escalate to Re-issue (web
ReissuePBSDR) only when no secret is stored or the agent reports consumed_failed.
Converged/disabled/verify_failed/DR-OFF = no-op. PBSDRHEAL_ONLY_HOST scopes a
supervised rollout. Scenarios A-F + all six red-proofs verified. No agent change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HEPuEwyyGDJdcsXLFsTWJn
2026-07-15 18:27:39 +02:00
admin 08fef4872b hub v0.55.0: accept offbox_enlarge_blocked event (Task 3a-fix delivery chain)
allowedEventTypes gains offbox_enlarge_blocked (was 400 at ingestion, dropping the customer email).
Deliberate NON-change: no customerMessages entry — the static map would discard the controller's
dynamic two-number Hungarian message (templates.go:129 fallback is correct). event_test acceptance +
400 red-proof; templates_offbox_test locks the raw-message fallback. manifest bumped to :0.55.0.
2026-07-15 07:50:29 +02:00
admin a1d045079f @
hub v0.54.0: change operator login password from the Configuration UI

Adds a "Login password" card on /configuration. The password was previously
settable only via the hub-config ConfigMap (auth.password_hash) + redeploy.

- store: hub_settings key operator_password_hash + Get/SetOperatorPasswordHash
- server: passwordHash field -> configPasswordHash (seed); new
  effectivePasswordHash() (DB override wins, else seed) is now the single
  source for the CSRF gate, RequireAuth, and handleLogin
- POST /configuration/password (handleChangePassword): requires current
  password, 8-72 byte new + confirm, bcrypt cost 10, persists DB override;
  existing sessions kept valid; ConfigMap stays the break-glass reset path
- UI: current/new/confirm form + inline mismatch pre-check + 6 flashes
- tests + red-proofs: override precedence, happy-path via handleLogin,
  wrong-current rejection, mismatch/too-short/no-op, template render
- docs: CHANGELOG, README (auth+config), REUSE, REPORT

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LbMm4T7Ayzs1unB9pN6Uqd
@
2026-07-13 22:46:49 +02:00
admin 375cb082e2 host-install v1.16.0 (FELHOM_ESCROW via canonical sudoers fetch; hub hostInstallVersion synced, no deploy) + RUNBOOK-escrow-ceremony rewrite: controller wizard PRIMARY, CLI operator fallback, F1 threat-model paragraph (R over CF tunnel accepted 2026-07-13), stale-blob/supersede/void semantics 2026-07-13 19:01:46 +02:00
admin 3efb1b1af1 docs: hub v0.53.0 closing bundle - CHANGELOG + CONTEXT (both operator rulings verbatim) + take-two F-14 disposition (SHIPPED, gated on escrow-acked deletion provenance) 2026-07-13 15:02:44 +02:00
admin a92e18b21f docs: hub v0.52.0 polish batch — CHANGELOG + REPORT + CONTEXT + REUSE (inline_confirm partial); take-two §4 dispositions: F-15/F-16/zero-toggle SHIPPED
Claude-Session: https://claude.ai/code/session_01GzammAMzsJTgpQHqxwM2bC
2026-07-13 08:48:17 +02:00
admin 448a68237a hub v0.51.0: DR-tier-by-default — per-customer dr_tier flag (default ON, legacy backfill from reality), cascade stages, WG-registration auto-provision hook, offsite-requires-DR guard (F-6 policy), host-page capability chips (inactive=neutral)
Claude-Session: https://claude.ai/code/session_01NptTCFtu7dz2Ru89qHRagN
2026-07-12 20:37:00 +02:00
admin 3ded6cf7c1 hub v0.50.0: customer-claim arc docs (CHANGELOG, default template claim_code_* fields, REUSE rows)
Claude-Session: https://claude.ai/code/session_01NptTCFtu7dz2Ru89qHRagN
2026-07-12 18:45:43 +02:00
admin b1ab0ea0e5 docs+manifest: hub v0.49.0 — Edit tab merge train (CHANGELOG, REUSE rows, CONTEXT, GitOps bump)
Version note: the task spec targeted 0.48.0 but that tag had already shipped
(app_start_failed); published tags are never re-pointed → this train is 0.49.0.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TZc5w5jDhFLv6qDC32KN5v
2026-07-12 17:39:09 +02:00
admin d02a216d0f hub v0.48.0 CHANGELOG + logging-conventions: TRACE level + ring sizing/spill (controller fix-3/fix-6, CAMPAIGN-3 closed)
🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_017CDMFpFx84pfviCTVuGGhf
2026-07-12 10:31:26 +02:00
admin 95d71fef2e hub v0.47.0: CHANGELOG + README tab/host-lifecycle/offsite docs + REUSE map + manifest bump to 0.47.0
CONTEXT.md: v0.47.0 arc entry (resolves the 146d165 swept-WIP flag) + the
multi-endpoint-allocation deferral + pre-GitOps stale-template note.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Vvz1NCu22p8dGkRCpeX9re
2026-07-11 21:39:44 +02:00
admin e35b1ae0e6 hub v0.46.0: observability pass — per-box log pulls, bundle custody, 72h TTL + secret gate
log_bundle_requests + log_bundles store (gzip, newest-3, 72h TTL purged on the
60s sweep); SaveLogBundle secret gate fail-closed (blocked flag row, no payload;
REDACTED/checksums pass). Report ACK gains controller_log_requested + ingests
controller_log_tail; heartbeat envelope gains log_tail_requested + ingests
log_tail (consume-once on arrival; pre-0.83 agents stay visibly pending). Host
detail Diagnostics section: request buttons (controller/agent), state rows with
honest latency hints, View/Download endpoint. Red-proofs: gate disabled and
clear-on-arrival removed both FAIL their tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
2026-07-11 16:57:47 +02:00
admin bbecf0592e hub v0.45.0: floor-UI separation + effective-floor source + per-box MinAgent conditional floor
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
2026-07-11 15:33:14 +02:00
admin ce6a56691e hub v0.44.0: PBS DR tier SLICE 1 — felhom-tenantsync surface (script+client) + hub provisioning flow (consume-once host secret, pbs_dr desired-state descriptor, fail-closed + idempotent, re-issue)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
2026-07-10 20:49:48 +02:00
admin 3e0c327f16 hub v0.43.1: Git Sync form hint — credentials optional, private catalog only (pairs with controller v0.112.0)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
2026-07-10 17:53:52 +02:00
admin 865a4a6afb docs: hub v0.43.0 CHANGELOG + REUSE entries (log-tail pull pattern, upsert guards)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
2026-07-10 16:05:49 +02:00
admin c918baa656 hub v0.42.0: remote "Debug mód" toggle on the customer config editor
Adds a form-level debug-mode checkbox to the customer config editor so an
operator can flip the controller's Logging.Level=debug (/debug menu + verbose
log) remotely, without SSH. Form field (not raw-JSON injection) because
handleConfigUpdate rebuilds ConfigJSON from the form on every save; the
config-version bump makes the controller re-pull + self-restart next cycle.

- buildConfigJSON: debug_mode checked -> "logging":{"level":"debug"};
  unchecked -> logging key omitted.
- config_form.html: "Hibakeresési mód (fejlesztői)" section + render state.
- configs_debug_test.go: form->JSON both ways; full-path survival test
  (debug lands, offsite descriptor unchanged, foreign-key red-proof); render
  state; red-proof exercised.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
2026-07-10 15:14:43 +02:00
admin fad5573dd3 hub v0.41.0: OffsiteChecker (fill 90/95 + staleness >48h) + operator freeze lever (SLICE 4)
Sibling checker over the controller report's offsite object: quota-fill
warn/crit + the silently-stuck staleness detector (escrowed-only,
red-proofed; nil-safe on pre-v0.109 reports; same-second tie-guard).
SetOffsiteFrozen flips ONLY readonly on the exactly-1 labelled sub-account
(SSH preserved); Freeze/Unfreeze buttons — manual only, never automatic.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
2026-07-09 23:57:12 +02:00
admin 49d1233391 hub v0.40.0: store escrow restic_pw_sha256 + serve escrow status in the report ACK (SLICE 3)
Additive host_escrow migration; SaveHostEscrow/HostEscrow gain the hash
(NULL-safe for legacy rows); GetEscrowStatusForCustomer joins hosts;
the report ACK gains escrow:{identity_blob_present,restic_pw_sha256,
created_at} (omitted without a row). Contract test mirrors the agent's
v0.79.0 emit struct.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
2026-07-09 23:10:29 +02:00
admin 17cc67f7cd hub v0.39.0: offsite hardening — F4 credential re-issue, F2 scan retry, F5 save UX
F4: ReissueCredentials — explicit operator recovery for consumed-password
dead-ends; resets the labelled resource's password (exactly-1 guard,
red-proofed), stores a fresh one-time secret, bumps ConfigVersion.
New hetznerapi.ResetBoxPassword for the dedicated path.
F2: host-key scan retry-with-backoff (~60s ladder, red-proofed) — first
save survives fresh-subaccount DNS lag.
F5: config form disables submits + shows an in-flight notice (the re-click
bait that caused live F1).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
2026-07-09 22:39:08 +02:00
admin 7c70c545d9 hub v0.38.1: offsite provisioning must survive a client disconnect (live F1)
First supervised live run: the ~25s spinner-less offsite save invited a
re-click; the abandoned first request's r.Context() was canceled between
CreateSubaccount and SaveOneTimeSecret, stranding sub-account 268985 with a
password lost forever (consume 404s permanently).

applyOffsite now provisions on context.WithoutCancel + 3-minute absolute
timeout: once the create starts, create->wait->store runs to completion.
Regression test with a ctx-honoring fake that cancels the request context
mid-create; red-proofed against the raw-ctx pre-fix shape (reproduces the
exact live error).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
2026-07-09 21:00:27 +02:00
admin 229650b4ee hub v0.38.0: offsite SLICE 2 — capture the box host-key fingerprint
Descriptor.HostFingerprint (SHA256, non-secret), captured at provision via an
x/crypto/ssh keyscan (SSHHostKeyScanner — dials :23, grabs the host key from the
handshake, no ssh binary). Fail-closed: nil scanner or scan failure → error (don't
serve a descriptor the controller can't verify). Pairs with controller v0.106.0
which re-scans + refuses on mismatch (no blind TOFU).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
2026-07-09 19:14:36 +02:00
admin 44ec06b50f hub v0.37.0: offsite provisioning SLICE 1 — Cloud-API client + provisioning core
Hetzner storage-box provisioning against api.hetzner.com/v1 (NOT .cloud).
internal/hetznerapi (typed client + CloudAPI interface + Fake + WaitAction);
internal/offsite (Provisioner.ProvisionOffsite — idempotent by label, shared
sub-account/dedicated box, transient password, non-secret Descriptor,
fail-closed); one_time_secrets store (single-use Save/Consume); POST
/offsite/consume-password/{id} (customer-key auth, once); config-form Offsite
section → applyOffsite (502+no-save on error) → descriptor in ConfigJSON →
version bump. Token/passwords never logged/committed/in ConfigJSON. Tested vs a
faked Cloud API + fail-closed red-proof. NOT yet live-provisioned (needs the
dedicated-project scoped token; current token can delete ep0).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
2026-07-09 18:38:24 +02:00
admin 0efdc78d75 docs: GL-7 closeout — tester agreement + hub 0.36.0 CHANGELOG/REPORT/CONTEXT + GO-LIVE G7 done
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
2026-07-09 08:44:21 +02:00
admin b70f2d0763 deploy(hub): v0.35.0 — OOB operator access (TASK H1)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
2026-07-05 22:35:00 +02:00
admin 012e5f3ecc fix(hub): mgmt_plane_healed alerts on the FIRST auto-heal (TASK G1) — v0.34.1
A heal marker is an event, not a baseline: construction seeds pre-existing markers
(startup false-alarm guard) but a newly-observed marker now raises the warning, so
the first auto-heal surfaces (matches the live drill). Added tests for both halves.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
2026-07-05 19:12:50 +02:00
admin 05d81810d4 feat(hub,install): break-glass recovery vault + mgmt_plane surfacing (TASK G1)
Hub half of the management-plane break-glass (prereq for felhom-sshd/H1; agent
half = felhom-agent v0.71.0). Closes SPIKE-felhom-sshd §8/#9.

- store.host_recovery + methods: per-host root@pam console password, at-rest,
  operator-retrievable (the PVE-web-console fallback when sshd + auto-heal both fail).
- API: PUT /hosts/{id}/recovery-credential (self-scoped, day-0 vaults) + GET
  /admin/hosts/{id}/recovery-credential (global key only). Secret never logged
  (red-proofed).
- monitor/host_mgmtplane: parses the agent mgmt_plane stanza, raises
  mgmt_plane_healed WARNING on a new privsep_healed_at (recurring clobber surfaces
  before lockout; complements host_staleness).
- host-install: step_break_glass generates a strong root@pam password (openssl
  rand, never logged/filed — stdin to chpasswd + curl), vaults via host key;
  idempotent unless --rotate-recovery. Installs the G1 host artifacts (tmpfiles +
  agent-independent watchdog timer), RuntimeDirectory-guarded; uninstall removes them.

Hub v0.34.0. Non-hollow tests + red-proofs; full suite green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
2026-07-05 19:03:18 +02:00
admin 265f46c0e7 docs: S2 shipped — hub v0.33.0 CHANGELOG, peersync v1.0.1, 06 §3.2 wire shape + S2 status, CONTEXT, REPORT
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
2026-07-04 01:20:10 +02:00
admin 740cc42734 docs: S1 shipped — hub v0.32.0/0.32.1 CHANGELOG, peersync v1.0.0, 06 S1 status, CONTEXT, REPORT
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
2026-07-04 00:14:35 +02:00
admin 9282d60f96 docs+skills: felhom-{build-deploy,ui-design,testing} skills + install_skills.py (junction); CLAUDE.md refresh (version-free); consolidated REPORT
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
2026-07-03 11:27:48 +02:00
admin b5f00509ee hub v0.31.0: accept 'critical' severity at event ingest + UI badges/CSS; event_test.go (red-proofed); REUSE.md §1/§3 updated
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
2026-07-03 11:08:44 +02:00
admin d331eb26d1 docs: REUSE.md introduced — reuse map (hub+website+scripts+manifests) + reuse_refs_check.py gate + consolidated cross-repo REPORT
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSK5g6qYLknKj8u3QAFEr6
2026-07-03 09:40:39 +02:00
admin d3fdbbc5ad hub v0.30.1: status badges no-wrap (PENDING/NO REPORT wrapped in narrow columns) 2026-07-02 22:52:31 +02:00