The setup code and the owner passphrase now follow the household's language,
one word longer in English so the entropy never drops (setup 3 hu / 4 en,
passphrase 5 hu / 6 en). List and count are chosen together so a caller cannot
pair an English list with a Hungarian count. Hungarian is byte-unchanged.
Three claims in the row were wrong and are recorded as such:
- the RECOVERY CODE is minted by felhom-agent from the EFF list and has
always been English; the hub does not own it and no row was added.
- no claim mail states a word count; the only count wording was the bind
page's passphrase hint, whose English half is now count-free.
- the proposed phone-safe filter removes 68% of the list (5270 of 7772
words) and was measured, then declined, with the reason in source.
Also: guide_quote_gate binds the English volunteer guide's three quoted
messages to the controller's English bundle — nothing did, so the guide would
have gone on quoting Hungarian after the fix. Seven decoys, all convicting,
including the name-for-fact one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
The hub has written every customer e-mail in Hungarian whatever the box was set
to. The box has published its language since controller v0.247.0; nothing read
it. Now it does.
Nothing an operator reads changes. The Hungarian mails are byte-identical, and
that is a diff rather than a reading: 56 goldens per language captured from
v0.117.0 BEFORE any string moved, and all 56 Hungarian ones pass unchanged after
every sentence was routed through the new bundle.
- internal/i18n: flat bundle, 79 keys, hu authoritative + hu fallback, ceiling 0.
- customerMessages/severityLabels are DERIVED from the bundle, so a sentence is
written in one place and all 40+ tests that read those maps still work.
- Language order: last reported -> created-with -> hu. reports.language defaults
to EMPTY, never hu: "never told us" is not "chose Hungarian".
- message_customer on POST /api/v1/event, additive and optional forever, for the
sentences the box composes and the hub cannot translate.
- The bind page is per-language, and its `expired` state stays Hungarian: it is
the state an unknown token lands in, so rendering a real English customer's
token in English would make the LANGUAGE answer what the TEXT refuses to.
Two defects found inside the release:
- R-581: the newest report was picked by received_at, which has SECOND
granularity, so same-second reports tied and the winner was arbitrary. Ordered
by the autoincrement id now. GetCustomers() still has the shape - row open.
- R-582: the English copy-guard stems, ported word for word from Hungarian,
convicted 141 honest sentences. The English claim is a phrase with a modal.
R-555 closed: the language allowlist entry is out of wire_contract_gate.py.
hub_copy_gate.py follows the sentences into the bundle - without that it would
have scanned four files that no longer hold any customer text and reported
success. Three new decoys incl. an innocent control.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
R-549 (operator ruling A): controllerStatus hardcoded 30m/1h while both
staleness checkers and hostStatus read alerting.stale_threshold. Moving the
threshold to 45m would have painted a customer amber 15 minutes before the
alarm could fire - the second definition rollup.go's header forbids. It now
reads the same value, down at 2x. Both 'checker initialized' log lines print
the threshold, which no line did before.
R-539 (ruling 3 of 2026-09-16): controller_slow_crashloop (warning,
operator-only), minted when the agent's slow_crashloop_since moves, with the
fast sibling's first-sight rule.
R-550: restore_interrupted (warning, for the household) allowlisted with a
Hungarian customer message.
Red-proofs, each seen failing then passing: the status test with the old
hardcoded numbers; the checker test with the movement branch removed; the
operator-only test with the registration removed; the household-message test
with the Hungarian entry removed (asserted on the SUBJECT - the body
legitimately repeats the raw message, which my first version of the test
mistook for a fallback).
go build/vet/test ./... green, 18 packages.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
Off-site is ON by default for a new customer — shared, 100 GB soft quota prefilled,
the checkbox kept so an operator can opt a customer out. The reason is this repo's
own [FACT]: the whole-guest tiers do not carry the data drive and a Tier-1 unit has
no file leg, so with this unticked a one-drive box keeps NO copy of the household's
own files. Measured on a fresh box the same day.
The quota is prefilled because the fill warning only fires when quota_gb > 0.
Also registers controller v0.244.0's app_deploy_started / app_deploy_failed in both
allowedEventTypes and customerMessages, per the rule that the two move together.
Red-proofed: dropping the default fails the new render test.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
Phase 0: the public ISO never auto-installs by construction (no answer.toml,
G1); the operator re-affirmed the interactive installer 2026-09-14.
- felhom-bootstrap.sh: mask pvebanner.service, write a Hungarian /etc/issue
(no :8006 admin URL); pairing banner names the Tulajdonosi jelmondat and
paints through the CONSOLE_DEV seam (R-496). Harness: 8 checks, red first;
fake hub now sends a pairing code (the banner was never tested, R-502).
- hub: created flash + Credentials block tell the operator to hand the phrase
over; the self-bind mail names the operator (R-497). Tests red first.
- iso-release-gate G14-G16; domain ruling in 01-topology + CONTEXT; R-494
narrowed to P3; R-502..R-504 filed; volunteer guide and day-0 A.2 aligned.
ISO_VERSION 1.27.0 (not built, not published).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
Operator ruling 2026-09-13. Above the vouched golden, a floor saved with a
declared MinAgent is served under the same agent comparison; an undeclared
one is still held beyond the golden. The declaration is stored beside each
floor as FLOOR=MINAGENT so it never carries to a later floor. Both floor
forms require min_agent above the golden (flash floor_needs_min_agent,
nothing stored). The Hosts page and the API log name the source.
Vouch path and R-120 gate untouched. Scenarios A-E tested; red-proofs A
and C in documentation/audits/rulings-r472-r475-2026-09-13/.
The customer page's Backup card read `Snapshots 0 / Repo Size 0 MB / Integrity
Unknown` for EVERY customer, indefinitely. Measured on demo-hp 2026-08-30 while
that night's controller log said `[offbox] backup OK: 8 app(s) backed up, 67
snapshot(s), 2m14s` and the box held snapshot_count:67, repo_size_bytes:
140829678, stats_known:true.
A card reading "no backups" over a working backup is worse than no card -- the
R-88 direction of failure (degrade to NO BACKUP rather than to UNKNOWN) on the
one screen that answers "is this customer protected?".
The data was never missing. The card rendered the report's `backup` object,
whose snapshot/size/integrity fields have had no producer since slice 8C. The
live numbers are in the `offsite` object, which THIS PACKAGE already reads for
the Offsite page and which monitor.OffsiteChecker already alarms from. Proof the
bytes were arriving: the Offsite page rendered demo-hp's usage as 0.1 GB from
that very object while the Backup card said 0 MB. So this is a render fix over
an existing feed, not a new pipeline.
Not a one-line swap, because snapshot_count:0 means two opposite things --
"holds nothing" and "never measured". R-225 measured that confusion one layer
down. backup_card.go resolves a three-way ruling in Go (a {{if}} chain over
map[string]interface{} float64s cannot keep the absent/zero distinction the card
is entirely about):
no offsite object -> "No off-site data reported", and says explicitly that
this is NOT the same as "no backups"
disabled + state -> names the blocker (needs_credential)
stats_known:false -> em-dash + "never been measured". NEVER 0
stats_known:true -> the real numbers, INCLUDING a real 0
A pre-v0.225.0 controller sends no stats_known -> false -> "unknown". That
direction is pinned: upgrading the hub ahead of the fleet must not report every
un-upgraded customer as having zero backups.
The Integrity row is DELETED, not re-sourced: nothing produces it, the
controller runs no integrity check, and NotifyIntegrityOK/Failed are called from
nowhere.
RED-PROOF: restore the pre-fix card markup -> all four tests fail, reporting 67
and 134.3 MB absent from the rendered page and the Integrity row present. The
tests drive handleCustomerUnified and grep the HTML on purpose: the defect was
the template's choice of source object, so a test one layer below it would have
been green against the shipped bug.
Green gate clean: 18 packages, rc 0.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LB8FmJaGd2cyjvy6dbEjpM
Hub only. No controller change, no agent change, no wire change — nothing to bake.
demo-hp untouched: the operator is re-deploying it this evening.
R-323 — the five-word phrase is „Tulajdonosi jelmondat". It was „Visszaállító
jelszó": one word from the name retired last week, and false besides — it restores
nothing, it proves the account owns the box being bound. Five sites, all in the hub;
felhom-controller and felhom-agent carry the name nowhere, so no halt and no bake.
Both suggested names were rejected with reasons: „Fiókjelszó" would collide with the
dashboard login (a DIFFERENT real secret), and „Összekötési jelszó" would leave the
two factors on this page separated only by kód-versus-jelszó — the exact shape being
removed, since the other factor is the „Párosító kód". The chosen name differs on
both axes, stem and noun. Naming only; the acceptance pin drives the real handler.
R-324 — the hub's customer copy is under a guard for the first time. Retired names
banned across all 95 hub files; retrieval stems registered in four declared customer
surfaces. The selftest found a defect in its own instrument on the first run. One
shared vocabulary in scripts/, drift-checked into the controller gate rather than
copied (R-325 removes the scaffold).
R-321 — a machine we told to be quiet is no longer reported as dead, and it was two
doors, not one: because the state is RECORDED rather than deleted, the morning
deadline check can skip it too. A deleted state returns "", which is not "down" —
R-195's shape returning through a second door. The clock runs from the report the hub
can see, so re-enabling starts it there and emits no recovery for an outage that never
happened. Three red-proofs; the one that matters showed a genuinely dead machine
sitting at "disabled" when the suppression was made unconditional.
R-326 — "which claims are unproven" is answerable by a command now. The nine I have
been repeating was the count of claims the 9 August pass DOWNGRADED, not the count of
unproven ones. The real figures: 55 claims, 23 walked, 32 not — and only 6 of those 32
cite evidence. Its first run found a stale claim (R-327).
Four paper debts and one fact given a reader. Hub-only — nothing to bake.
A4 — the entry about "the tester's machine" named a risk correctly and labelled it
in a way that invited deleting it. Established from the hub's own store: `peti-felhom`
is a REAL machine (482 reports, 2026-02-27 → 2026-07-15, a named person's own box) and
the 3.6 GB with no key and no backup is real. `david` → `tester-1` is a DIFFERENT record
with no host, no escrow and no report, ever — deleted 07:55:49 and re-created 07:56:47
this morning. The prompt's premise conflated the two; the register now says which is which.
A1 — R-312/R-313/R-303 recorded as DECIDED with their re-open triggers, and moved out
of STATUS's "Waiting on you", which is now empty.
A3 — day0-install §C.1 said pushing the installer publishes it. It has not since
R-110. Corrected, with the two manifest pins named and an outside-verification command;
the one copy that repeated it (a dated audit, true when written) carries a superseded note.
A5 — standing rule 5: evidence comes off the machine at the end of the phase that
produced it, before any revert. Earned twice in three days on the same box at the same
point (R-320). Four homes, plus what to do when it is already gone.
R-295 hub half — „Beállító kód" everywhere; „Visszaállító kód" retired. New `reenroll`
mail kind so the mail names the page a REBUILT box actually shows („A szerver
beállítása"), not the „Elfelejtett jelszó" page it has no login screen to reach.
Naming only; the acceptance pin proves the secret is untouched.
R-319 — the hub models `guest_net` after 23 days of receiving and discarding it. The
signal is `heals_last_hour`, not `state`: a guest the watchdog keeps repairing reads
healthy between repairs. `heal_succeeded` decoded too (R-260's lesson). Unknown is never
drawn as healthy — three absences, three sentences. No alarm, deliberately.
Three red-proofs, mutations asserted applied. Wire-gate checked tags 182 → 190.
B1 — the operator's 2026-08-12 dispositions were NOT in the register; they are now.
Third allowlist kind for the five ruled "no reader wanted"; `reporting_disabled`
reclassified redundant. 8 read · 5 deliberately unread · 1 redundant · 6 still owed.
Also filed: R-321 (a deliberately-silent box still alarms stale/down — the checker is
age-only, and decoding the flag would not have fixed it), R-322 (the claim guard has
never scanned the hub; a hand scan returns zero, so it is a scope gap, not a defect).
The guard owed since Friday morning. Agent v0.128.0 was published as a package
and never git-tagged; it was vouched here; and because felhom-host-install.sh
fetches an agent's configs from raw/tag/v<version>/configs/, every fresh install
and reinstall died at step 5 of 8, as root, on a virgin machine, for most of a
day. handleSetArtifacts is the sole UI path to SetArtifactManifest, so the check
belongs here and nowhere else.
TWO LEGS, because both failed inside two days: the TAG (missing, R-273) and the
PACKAGE (pruned from under a still-tagged version, R-287). Either alone catches
one of them.
It asserts configs/felhom-mkfs-guarded.sh -- the FIRST of the installer's sixteen
fetch_raw calls and literally the file whose 404 broke Friday. A test pins the
constant, because probing a path that merely exists is how it stayed invisible.
The golden gets the package leg only: it has no config tree, so a tag probe would
assert something the installer never does.
"Could not verify" refuses too, with its own message. No override -- the registry
is the operator's own server, so if it is unreachable the vouch can wait.
ORDERING IS LOAD-BEARING AND A FAILING TEST FOUND IT. The probes run before
resolveArtifactSHA, whose flash conflates "missing", "unreachable" and "bad sha".
Probing first means an unreachable registry is reported as unreachable.
Five scenarios each naming the wrong outcome; three red-proofs, mutations asserted
applied and reverted. With the tag check removed, scenario A reports artifacts_set
-- Friday's exact defect returns.
v0.100.x removed the serialisation: 26.2s -> ~9.85s mean. What remained was one Gitea package SEARCH
per dropdown at 0.20-3.8s depending on load, which concurrency cannot help.
Memoised for 60s IN MEMORY. The TTL was ruled by the operator against the workflow that cares: a
bake-and-vouch session publishes an artifact and comes straight here to select it, so a minute is
short enough not to be noticed and long enough that every reload in that session is instant.
NOT persisted. Gitea IS the store for both the version list and the sha; a copy in hub_settings would
be a second source of truth that can drift from the registry it describes, and the operator reads the
sha here to confirm what they are about to vouch. An in-memory cache dies with the process and can
never be mistaken for a record.
A failed resolve is NOT cached — a blip must not pin an empty dropdown for a minute. But an empty
list from a package that genuinely has no versions IS cached, because 'we found nothing' and 'we
could not look' are different answers (CONTEXT S-39, applied to a list instead of a figure).
THE FIRST VERSION OF THIS GOT THAT WRONG: the comment said only successful resolves were cached and
the code cached the empty list anyway. TestArtifactChoices_FailureIsNotCached caught it before it
shipped — which is the argument for writing the test that asserts the comment, and the same class
this session spent the day closing.
go build/vet/test green, go test -race clean, run separately from this commit.
Follow-up to the fan-out, and the reason for it is worth recording: THE FIRST FIX DID LESS THAN THE
ARITHMETIC PREDICTED. Concurrency took the page from 26.2s to ~11-18s, not the ~2s expected, so the
gap was chased instead of declared closed.
What it found: the slowest single call on the page is not a per-version sha lookup at all, it is the
PACKAGE SEARCH (/api/v1/packages/admin?type=generic&q=...), measured in-cluster at 1.1-2.2s each
against ~0.24s for a file's metadata. Per-version fan-out cannot touch it — there is one search per
package and they ran in series.
The two dropdowns are independent, so they now resolve side by side, overlapping both searches and
both fan-outs. go test -race clean on the new concurrent paths.
Gitea's latency on this box is load-dependent and varies 2-4x between samples, so the CHANGELOG
quotes a range rather than a single pair of numbers.
MEASURED, NOT GUESSED: GET /configuration -> HTTP 200 in 26.2s.
The reasonable guess was that it hashes the artifacts on page load. It does not, and the code already
said so: Gitea stores each package file's sha256 and gitea.FileSHA256 reads it as metadata — "a cheap
metadata call, the artifact bytes are never downloaded". The cost was never CPU.
IT WAS LATENCY x COUNT. artifactChoices made ONE SERIAL round-trip per version, for two packages,
capped at 20 each: 2 x (1 version list + 20 sha lookups) = 42 sequential requests at ~0.6s each out
through the public ingress. 42 x 0.6 = 26s, which is what the clock said.
1. The sha lookups now run CONCURRENTLY, bounded at 8 in flight. Order preserved by writing into a
slot rather than appending — the dropdown is newest-first, and a scrambled sha would show the
operator a hash belonging to a DIFFERENT artifact. A failed lookup still drops that version only.
2. The client talks to Gitea IN-CLUSTER (http://gitea.gitea-system.svc.cluster.local:3000,
overridable via GITEA_API_URL). Measured from the hub pod: 0.11s against 0.26-1.16s, because the
public path adds DNS, the ingress hop and a TLS handshake to each of the 42. Plain HTTP is safe
ONLY because it never leaves the cluster network — the registry token rides the Authorization
header, so this must not point at a public host without TLS. Unreachable -> the existing graceful
degradation to manual text entry, unchanged.
DELIBERATELY NOT DONE: caching the sha in the hub's own database. That was the other half of the
proposal and it is the wrong shape. Gitea already IS the store; a copy in hub_settings would be a
second source of truth that can drift from the registry it describes — and the operator reads exactly
this value to confirm what they are about to vouch, so a stale one would be a confident wrong answer.
The same reasoning golden_currency_gate.py already records for the vouched version. With the fan-out,
a cold load needs no cache to be fast.
The cap stays at 20 and now bounds the FAN-OUT too, not just the rendered list.
Tests pin order (and that each sha belongs to its own version), per-version failure isolation, and
THE CONCURRENCY ITSELF — a wall-clock assertion plus an in-flight counter, so a fast run cannot be
luck, and an upper bound so a large package list cannot stampede Gitea. Red-proof: reverting to the
serial loop takes 861ms where the concurrent one takes 150ms, and the test fails naming the
26-second page.
go build / go vet / go test ./... green (18 packages), run separately from this commit.
v0.97.0 introduced a second hold reason and left both surfaces printing the first. The
freshly deployed hub logged, for the campaign box:
managed floor HELD for c11: agent "0.125.0" < MinAgent 0.113.0
which is FALSE — 0.125.0 is ABOVE 0.113.0. That box is held because its floor sits above
the vouched golden, not because of its agent. CLAUDE.md's corollary exactly: when a verdict
changes which field it counts from, the alarm text has to change with it, or a true alarm
reads as one to dismiss.
Both the ACK log line and the Hosts-dashboard HeldReason now come from one
ManagedFloorDecision.HoldReason(), and TestResolveManagedFloor_HoldReasonMatchesTheCause
pins each reason to its cause.
R-198 — host_escrow_superseded shipped with `blob` (the K-escrow / PBS datastore key) and
identity_blob was added to host_escrow LATER, never here. The offsite restic REPOSITORY
password lives in identity_blob. So demoteCurrentEscrowTx -- whose own comment calls it "THE
ONE escrow row-copy routine" -- retained the whole-guest key and silently dropped the off-site
data key, which is the secret the retention was built to preserve. And because the copy happens
as the new blob overwrites the old, the destroying act was the ESCROW CEREMONY: the exact thing
a rebuilt box tells its customer to run, on a card promising in Hungarian that the old backups
stay recoverable. Both demo boxes crossed that line on 2026-08-04.
- identity_blob added to the table (CREATE + additive ALTER) and carried in the shared copy
routine, so BOTH callers are fixed at once: re-escrow and host-delete demotion.
- ListSupersededEscrow reads it back; store.HostEscrow gains IdentityBlob.
- CountCurrentEscrowWithIdentity is the census of who the fix protects.
- Nothing is backfillable: pre-v0.93.0 retained rows have no blob and their sources are gone.
- Tests assert the CONSEQUENCE (a retained row can still yield a repo password), which is why
the pre-existing retention test stayed green for two months asserting the mechanism.
R-197 — SaveHostEscrow returns the hash it replaced; the escrow PUT raises
offsite_repo_key_changed (warning, operator-only, edge-triggered) when both hashes are known and
differ. No hash value travels. Severity chosen for the world v0.93.0 creates: with the identity
blob retained, a changed key is "this history now depends on an older recovery code", not a loss.
R-192 (half) — the stuck alert now reports the two shapes it actually covers, burned and
regressed, each stating its own measurement; the regressed text withdraws the Re-issue
recommendation. Every self-heal refusal leaves a notification_log row with its reason. The
guard's logic is unchanged; its 500-oldest-reports scoping stays OPEN and the window is named in
the alert text so the limitation travels with the number. offsite_delivery_stuck and
offsite_credential_restaged are added to operatorOnlyEvents -- neither was registered and neither
has a customerMessages entry, which is not a block.
R-196 — five comments (not the three the spec expected) claimed ReissueCredentials rotates the
restic repo password. It resets the PROVIDER password and cannot touch the repo password, which
is generated on the box. All five corrected; the staleness mark documented as precautionary. The
BEHAVIOUR stays open.
Not in this release: R-199, R-200, R-201 remain open -- the chain that hands the key back is
still unassembled. Part 5 hit its gate; the orphan card is untouched (R-202).
The deliberate hostInstallVersion const is removed. It existed only to produce a real red
run (#3-#6) and the demonstrated alarm; R-94's deletion stands.
The Setup tab said 'host-install 1.19.0' while the served script was 1.22.0, and had
been wrong since 2026-07-14. Deriving the number honestly is not possible: the Option-1
command downloads felhom-host-install.sh from the website at RUN TIME and the website
git-syncs main every 30s (R-110), so no build-time value in the hub can be true. R-94(a)
offered derive-or-delete; deleted, which removes the drift class instead of automating it.
- configs.go: hostInstallVersion const, pageData.ScriptVersion field and its assignment
all removed; a NOTE in their place records why there is no constant here.
- customer_unified.html: the sentence now says the command always fetches the current
installer, and renders no version.
- hostinstall_gates.py gate 1: the third assertion INVERTS — it used to require the hub
const to equal SCRIPT_VERSION, it now asserts the hub carries no host-install version
literal at all, matched in six code shapes across every .go/.html under hub/ (comments
are deliberately not stripped: a // inside a URL literal would blind the scan).
- render_test.go: the assertion 'html contains hostInstallVersion' compared the constant
to itself and passed at ANY value — demonstrated green with the const at 9.9.9 while the
script was 1.22.0. Deleted, not replaced: there is no longer a version to assert.
- felhom-host-install.sh: COMMENT ONLY (SCRIPT_VERSION untouched) — it claimed the gate
keeps the hub copy equal, an invariant that no longer exists.
Red-proofs: restoring the const fails the rewritten gate 1 (3 shapes hit); the old
render_test assertion passes at 9.9.9.
Found by the operator, in the way that matters: it cost a real login.
The v0.84.0 Console access card shipped its Copy button DISABLED until a Reveal.
Clicking it did nothing, silently, so the clipboard kept whatever was already in
it — another host's console password from an earlier reveal. That got pasted into
demo-hp's PVE login, which failed with no explanation: the box logged a plain
`password check failed for user (root)`, the credential was never at fault, and
nothing on screen said the copy had not happened.
A copy button that silently no-ops is worse than no copy button. The operator
cannot tell "copied" from "did nothing", and the stale value left behind is a
VALID secret for a DIFFERENT machine — so the failure looks like a stale
credential and sends you diagnosing the wrong thing.
Copy now works without revealing, and that is the safer default rather than a
concession: the secret goes straight to the clipboard and never renders on
screen, so it cannot be shoulder-surfed or caught in a screenshot. Reveal remains
for when it must be read.
Three silent-failure branches closed, all in the same eight-line function:
- not yet revealed -> was a disabled no-op; now fetches and copies
- navigator.clipboard absent -> was silently skipped; now shows it and says why
- writeText() REJECTED -> promise was ignored, so the operator believed it
copied; now shows it and reports the refusal
The success path names the host ("Copied demo-hp-bb76ea's root@pam password"),
because the clipboard is fleet-wide and every box has a different console
password — "copied" alone cannot say for WHICH box, which is the confusion that
produced the incident.
One retrieval path, shared: the endpoint is defined once (data-reveal-url) and
read back with getAttribute, so Copy cannot drift onto a different, unaudited URL
than Reveal. Server-side is unchanged — both buttons hit the same CSRF-gated
endpoint and both write the same recovery_credential_revealed event, which is
correct: the register records accesses, and a copy is an access.
Tests 566 -> 568. Red-proof: re-adding `disabled` reproduces the shipped bug.
Pairs with agent v0.119.0 and is useless without it.
A managed box's LAN IP was not shown anywhere in the hub, because nothing
reported it — the host report carried no address of any kind. The only IP
reachable from the UI at all was the WireGuard one, on /offsite's peer table
keyed by pubkey, so an operator could go peer->host and never host->peer, which
is the direction anyone actually asks in.
The host page grows a Network card: every routable address the box holds, one row
per (interface, address), plus a WireGuard row. On demo-felhom that is vmbr0
192.168.0.162/24 and tailscale0 100.70.170.35/32 — with the PVE web console at
https://<the LAN address>:8006, the thing the operator wanted and could not get.
WireGuard is rendered as TWO facts, deliberately. WGAssignedIP is the hub's own
allocation (wg_peers, authoritative desired state); WGConfirmed is whether the box
reports actually holding it. Showing the allocation alone would make a peer that
was never applied look healthy — the same shape as reading a timestamp that
records an attempt as if it recorded a result.
The split is keyed on the ALLOCATION, not the interface name: wg-felhom is the
agent's current unit name, and a UI keyed on that string would silently
mis-render the day it changes.
An old agent renders UNKNOWN, never "no addresses". Below agent 0.119.0 the field
is absent from the wire, and an absent signal is not a negative result — the page
says so and names the version needed. Rendering an empty list there would have
stated something false about the host.
No new store table and no new ingest path: the report is already stored opaquely
and GetWGPeerForHost already existed with no UI consumer. This is parse + render.
The report fixture in the tests is the REAL wire — the addresses block copied out
of `felhom-agent --selftest=hub` on demo-felhom running 0.119.0.
Tests 559 -> 566; four red-proofs (inert view-model, unconditional confirmation,
the old-agent branch, and the drift case) each run, observed failing, reverted.
The credential existed and was not reachable when it was wanted. Every box has
had a strong random root@pam password since TASK G1, vaulted in the hub at day 0
and used for real during the sshd incident — but the only way to read it back was
a hand-written curl carrying the global operator key, a secret kept out-of-band.
In practice the PVE web console on a demo box felt locked.
The host page grows a Console access card: presence + username + set_at by
default, Reveal fetches the plaintext on demand for 60 s with a Copy button.
Masking clears the JS variable, and also fires on a second click and on
visibilitychange. A host with nothing vaulted says so, and says why.
The secret is NEVER rendered into the page, and that constraint shapes the
change. The render path uses a new store.GetHostRecoveryMeta whose struct and
SELECT both omit the secret column, so it is structurally incapable of carrying
one. The plaintext crosses the wire only in the response to POST
/hosts/{id}/reveal-recovery-credential (Cache-Control: no-store, CSRF-gated at
the ServeHTTP level; POST precisely so that gate applies and so no secret is
retrievable by URL alone). Deliberately NOT the customer page's data-secret
widget, which embeds the plaintext on every load.
A delivered reveal writes one recovery_credential_revealed event on the host's
customer timeline (info, source hub, Hungarian) via SaveEvent alone — no
dispatcher, nobody emailed, the log_tail_requested shape. Two reveals write two
events: the register records accesses, not states. A 404 is not an access. An
unbound host reveals fine and writes no event; the [INFO] hub line, carrying the
username and a length only, is then the record.
The global-key API path is untouched by design — it is the route for when the
hub UI itself is broken, and coupling it to the session layer would delete the
independence that makes it a fallback.
Recorded as a real trade: the hub session password alone now unlocks console root
fleet-wide, where retrieval previously also needed the global key. Accepted for a
single-operator, HU-geo-fenced hub that already stores these passwords in
plaintext at rest (CONTEXT.md ruling S-4). The plaintext-at-rest half is filed as
R-133 — every hub DB backup is a fleet-wide console-credential dump.
Tests 550 -> 559; four red-proofs (page leak, audit event, CSRF gate, route
order) each run, observed failing, and reverted. The route-order proof is a seam
test driving ServeHTTP: a handler-level test cannot see that defect, because the
handler is correct and simply never runs.
The golden's version IS the controller it bakes (build-golden.sh:345 defaults
GOLDEN_VERSION to the controller tag), so a golden behind the newest deployed
controller means every FRESH install lands on stale application code. On the R-120
occurrence that stale code shipped a customer-facing falsehood: a box from the
0.185.1 golden told a customer whose backup drive had fallen out that the backup was
on the same disk as the system -- false, the drive was gone -- and offered a
different drive as the remedy.
WHY A GATE, NOT A REMINDER. The gap has opened three times: R-111 (golden's agent 17
releases behind), R-115 (agent built and deployed, never published), R-120 (this).
The first two were closed by re-baking and remembering; remembering then failed
again. R-29 is the standing proof that a check nobody runs is worse than none because
it reads as coverage -- hostinstall_gates.py sat RED and uninvoked across three
version bumps and hub_confirm_gate.py has never run at all. So the property that
matters is not whether a check exists but whether it BLOCKS.
- Wired into handleSetArtifacts (internal/web/configs.go), immediately before the
only write, on the sole UI path to SetArtifactManifest -- it runs on every vouch
without anyone choosing to. A script in scripts/ would have been a fourth orphan.
- It REFUSES (operator ruling, 2026-07-30), with a flash naming the remedy.
- Signal: store.NewestReportedControllerVersion() over reports.controller_version,
SEMVER-compared in Go -- MAX() in SQL ranks 0.99.0 above 0.186.0, a pair this
fleet has shipped. No outbound call, no new credential.
- Fail-open in exactly two deliberate cases: an empty golden field (clearing the
manifest is legitimate) and an unknown fleet version (a new hub must vouch its
first golden).
NEAR-MISS RECORDED: the first draft read guests.controller_version, a column that
exists in the schema and that NOTHING writes -- it would always have seen "" and
failed open, i.e. inert, this gate's own failure shape. Caught by grepping for a
writer before trusting the column.
Blind spot stated rather than papered over: a controller no box has ever run is
invisible to this signal. Not the failure that has bitten -- all three instances were
deployed-newer-than-baked.
4 tests through the PRODUCTION handler over httptest, never an injected seam. The
refusal asserts both the flash and that the manifest was NOT written, because a gate
that redirects and saves anyway reads as enforcement while providing none. Red-proof:
deleting the block makes the stale golden vouchable and both assertions fail.
ROADMAP R-29's audit list now records this as the FIRST enforced gate, so the
contrast with its three orphans is kept rather than lost. The orphans are unchanged.
Suite rc=0 read separately from this commit.
Found validating v0.69.0 against the live hub. demo-vm-felhom was deleted
on 07-18 and was still on the Customers list AND still raising offsite_stale
(10 events, latest 07-21 17:34, operator email at 19:34) — because
GetCustomers() is report-derived and no lifecycle tier ever deleted a report.
New leg 3 (residue), before the record purge: reports, app_telemetry,
app_log_tails, log_tail_requests, customer_notifications, plus the
credential-bearing appliance_registrations and selfbind_tokens. Audit
(events, notification_log) and F-14 provenance still survive.
Ghost customers are now deletable: 404 means "nothing here", not "no config
row". With no config row the offsite descriptor is unknowable, so the Hetzner
and descriptor legs record skipped_no_config rather than a bare "skipped".
Two more red-proofs.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J55BQE1gE2V4ffud5jweGS
POST /configs/{id}/delete now runs hosts -> RESET -> purge behind three
acknowledgements, a typed customer-id, a stale-preview check and the
ONLINE-host refusal (every gate before any write, so a refusal has zero
side effects). The shallow handleConfigDelete is gone.
Two invariants are asserted, not just commented: ruling 3 is preserved by
construction (leg 2 never sees a host row) and retained escrow custody is
purged exactly once, in leg 3 (leg 2 runs with purgeEscrow=false).
handleCustomerReset's committed half was extracted as commitCustomerReset;
the standalone RESET path is byte-identical to v0.68.1 and its suite is
untouched. Five red-proofs run.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J55BQE1gE2V4ffud5jweGS
The v0.68.0 row wrapped itself in a <div>, but the artifacts <form> IS the CSS grid
(display:grid, no inner container). The stray </div> closed the surrounding card from
inside the form and the new <div> was never closed — it swallowed the submit button and
ran to </form>, so the row rendered outside the card and Save landed inline. Reported by
the operator on first use.
The field still submitted (it stayed inside the form), so this was layout damage rather
than data loss, but the unbalanced markup put every section below it in the wrong
container.
Fixed as plain grid cells (grid-column: 2/4), no nested elements.
There was no render assertion on this form at all, which is why a hand-edit broke it
silently. The new test asserts the field is inside the form, the button has not escaped,
the form contains ZERO divs, whole-page div balance holds, and the sections after it
survive. Red-proofed against the broken shape.
Completes the hub half of R-39's fleet fix on top of the generation core (c484aa2).
pbsdrheal gains an auth_failed TRIGGER — a new trigger in the existing machine, not a
new machine. A box whose credential PBS rejects escalates to a fresh mint, never a
re-stage (which would re-feed the secret PBS just rejected), through the EXISTING damper:
a 401 flap must not become a secret-minting chain. With the generation stamp this closes
the loop end to end — agent proves the 401, hub re-keys, generation advances, descriptor
hash moves, agent re-consumes.
consumed_at honesty gauge: a staged secret still unconsumed past a 15-minute grace while
the box reports `applied` is surfaced with its own event. That is the exact 2026-07-18
fingerprint and a disagreement no single tier can see alone. Deliberately a SURFACE, not
a heal — auto-re-issuing on it would mint a second secret on top of an unconsumed one,
which is the mint/consume race R-39(a) already recorded. One event per distinct report,
and an honestly-stuck box does not double-report (its unconsumed secret is the symptom
being healed, not a contradiction).
R-50b(a): ArtifactManifest.WrapperSHA256 + operator field + host-page drift surface. The
PBS wrapper is root-owned 0755 and the pinned sudoers vector, yet installed unversioned
from raw/branch/main and absent from every manifest. Agents >=0.91.0 report the installed
hash; a mismatch is surfaced. An unknown on EITHER side reads as quiet, never as drift —
lighting every host amber on rollout day is how a warning becomes background noise. The
delivery channel itself stays R-50b(b)/(c).
Compatibility unchanged: safe for 0.90.0 agents (unknown JSON key dropped); the re-arm
and auth-honesty guarantees need agent >=0.91.0, so MinAgent moves only after the fleet
has self-updated.
Tests: auth_failed escalate/debounce/recovery-forgets-streak; honesty gauge incl. grace
window, the restage edge (consumed_at deliberately NULLed), consumed-never-alarms, and
honest-stuck-no-double-report; wrapper drift incl. both unknown directions. Red-proof run
at the assertion level: removing the auth_failed arm fails the escalation tests with
reissues=0.
The fleet half of R-39. An ep0 credential re-issue re-keys the SECRET of an existing
token, so token_id, fingerprint, datastore and namespace all come back byte-identical.
The agent re-applies on the descriptor's CONTENT HASH, so a re-issue was invisible to a
converged box: it short-circuited, never consumed the fresh secret, and served a revoked
credential while reporting `applied` — the N100 failure of 2026-07-18.
host_pbs_secrets gains a monotonic per-host `generation`, advanced by every fresh MINT and
by nothing else, stamped into the descriptor as `secret_generation`. That is now the only
field a re-key moves, and it is what re-arms the agent.
DEVIATION FROM SPEC, deliberate: the brief said to return "the new row's id (int64) …
no schema change". There is no row id — host_pbs_secrets is keyed by host_id and UPSERTed
last-write-wins, so a new row never exists, and created_at collides for two mints in the
same second. An additive counter column is the only monotonic source; it uses the repo's
existing idempotent ALTER-TABLE idiom.
RestageHostPBSSecret deliberately does NOT advance it: a re-stage re-arms the SAME secret,
the descriptor content genuinely has not changed, and a bump would cause a pointless agent
refetch loop (that method's own contract says so).
Also corrects a comment that asserted the re-issue refreshes the descriptor "with the NEW
token_id/fingerprint". That is false for a re-key, and believing it is why the descriptor
was never expected to be identical in the first place.
omitempty is load-bearing: a zero generation must not start emitting a new key into every
pre-existing descriptor, which would itself be a fleet-wide spurious re-apply.
Compatibility: agents below 0.91.0 drop the unknown JSON key and behave exactly as today —
inert, not breaking (Scenario C).
Tests: store-level monotonicity + per-host isolation + restage-leaves-it-alone; descriptor
byte-change, omitempty, and sibling-key round-trip; and a FLOW-level test driving
ReissuePBSDR against a fake that models a real re-key. Red-proof run at the assertion
level (not the compiler): commenting out the stamp makes the flow test fail with both
byte-identical blocks printed.
Four small items, each a case where the hub already knew something and said
nothing. Green: build, vet, tests all pass.
(a) Self-bind link is minted automatically at customer creation AND at RESET
completion (R-36 sub-item). The console banner tells the customer to open
"az e-mailben kapott link"; until now that email existed only once the
operator remembered the button, so the banner could point at something that
did not exist — during the 2026-07-18 rehearsal the box waited ~11.7 min on
exactly that. handleSelfBindLinkSend's body was extracted into a shared
mintAndSendSelfBindLink core so the button and the auto-mint callers cannot
drift apart on the honesty rules: F1 (no address -> mint nothing) and F2
(send failed -> delete the token, never leave it live). The wrapper NEVER
fails the operation it rides on — a create that provisioned Cloudflare,
offsite and PBS must not 500 over a courtesy email.
Gap found and closed while wiring it: PurgeCustomerResetDBState does NOT
clear selfbind_tokens, so a link minted BEFORE a reset would have stayed
live across it. A successful mint already replaces it (delete-then-insert,
single-active); the skip paths would not have, so they now clear stale
tokens too. Invariant: after auto-mint runs the only live link is one it
just issued, or none.
(b) Post-RESET staleness banner (R-37). When a RESET COMPLETED after the newest
report, every health figure on the page describes a lifecycle that no longer
exists, and the page kept showing pre-RESET warnings as current. Narrow on
purpose: an in-flight reset does not trigger it, and it clears itself when a
report arrives. Ties resolve to STALE — SQLite timestamps are second-
resolution and a same-second report almost certainly predates the reset;
erring the other way would hide the banner exactly when it matters.
(c) Unprovisioned-offsite warning (R-36 interim). enabled==true with type=="" is
a real, stable, silent state: provisioning is Save-triggered and the
re-enroll auto-re-issue deliberately skips an unprovisioned target, so
nothing self-heals it. Reuses the exact predicate the offsite re-issue
handler already refuses on.
(d) pbsdr_reissued rendered an EMPTY flash box — the key had no template branch,
so re-issuing PBS credentials showed a success box with no words (observed
live 2026-07-18). Now describes what was staged plus the R-39 caveat:
confirm `pvesm status` shows the entry active, because a converged agent can
report `applied` while the storage still 401s.
New .flash-warn (amber, --warn tokens) for the deviation tier between success
and error — exception-color principle: only on deviation, never on a healthy
page.
Tests assert each banner is ABSENT in the nominal cases as well as present in
the deviating one — a banner that always renders is worse than none. Both
red-proofed: deleting the pbsdr_reissued branch reproduces the original empty
box; neutering the staleness predicate fails the banner assertion. New
read-only store accessor CountSelfBindTokens makes the single-active invariant
assertable.
NOT in this train: the R-39 hub-side generation-bump fix the pre-travel task
made conditional. Its condition was REFUTED (SetHostDesired bumps
unconditionally; applyPBSDR is idempotent as documented) — the real mechanism is
the agent's descriptor-hash convergence and needs its own spec.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nn3VgQk9iwEGgyx6QJ2NvE
Let a customer bind their own freshly-installed appliance without the
operator: operator "Send self-bind link" mints a 7-day tokenized
capability link, emailed (Hungarian, sibling sender) to the customer, who
opens a public /bind/<token> page and proves two factors — the console
pairing code shown on the box screen + their retrieval passphrase — and
the hub stages the bind via the same BindAppliance (provenance
customer_selfbind). The box's ~30s appliance poll delivers.
Viktor's three rulings verbatim: console pairing code (no appliance list
ever rendered), operator-sent tokenized link, 5-attempt lockout ->
"call support". Wrong code == wrong passphrase (one generic failure, no
oracle, both factors compared unconditionally); expiry falls back to
operator-bind unchanged.
THE TRAP: one public prefix /bind/, exempt from auth+CSRF at both /login
gate sites via a single isPublicBindPath predicate (tight trailing-slash
match; ServeMux ..-cleans; handler rejects '/' in token). 9 tests
(Scenarios A-F + F1/F2); 4 red-proofs verified red-then-green (lockout,
oracle, widened-prefix, single-active). GC verdict: no appliance GC ->
the 7-day TTL stands alone. Controller/agent untouched; R-27b deferred.
Green: full hub build/vet/test (17 ok) + bash -n + hub confirm gate.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017qDiBqKKQ5vPB5fXBqu7Kp
Makes PBS DR storage visible like the restic pool box (v0.64.0), differentiated. Scoping
correction: restic = subaccounts on the shared Hetzner Storage Box (Hetzner API); PBS DR =
the felhom-offsite PBS datastore on the ep0 endpoint VM (NO Hetzner API). Option A
(Viktor-ruled): a read-only `usage` op on the felhom-tenantsync ep0 forced command (twin of
fingerprint), polled by a new hub checker on the 15-min throttle. READ-ONLY throughout.
Phase-0 (gate PASSED): on ep0 (PBS 4.2.3), df -B1 --output=size,used,avail <datastore path>
yields bytes (39990112256/7627939840/... ~19%), read-only, existing sudo context, no admin token.
- scripts/felhom-tenantsync.sh -> v1.2.0: read-only `usage` short-circuit (df on the datastore
path), no customer_id, no admin token, NO mutation. + a bash harness proving zero mutation.
- tenantsync.Client.Usage() + BoxUsage; unknown-op -> typed ErrUsageUnsupported (graceful).
- monitor.PBSDRBoxChecker: OffsiteBoxChecker clone over a usageReader seam; 15-min throttle,
cached PBSBoxSnapshot, escalation-only pbsdr_box_fill on the "pbsdr-box" scope (operator only,
no SaveEvent), recovery re-arm. Fill only. THREE states: ok / unavailable (ep0 <=v1.1.0,
neutral no-alert) / degraded (exec failed, keep last).
- config: Alerting.PBSDRBoxFill{Warn,Crit}Percent (80/90); built with the tenantsync client,
60s sweep, SetPBSDRBox. Hub deploy INDEPENDENT of the ep0 update (graceful degradation).
- web: /offsite splits into Restic + PBS DR hash tabs (endpoint cards under PBS DR); PBS panel;
the single dashboard tile becomes two gauges (RESTIC pct.ratio, PBS DR pct / n/a).
- runbook offsite-endpoint.md 10: v1.2.0 update steps (no sudoers/authorized_keys change).
Tests: 10 Go + the harness; 3 red-proofs (usage mutation, escalation-only, unavailable-drives-band)
confirmed red then restored. go build/vet/test + bash -n + hub confirm gate all pass.
The operator sees the shared pool box's real state on the hub: total box fill vs
capacity, Σ(shared soft quotas) vs capacity (the oversubscription ratio), per-customer
usage/quota bars, and a box-level operator alert (fill % + oversub ratio) on the existing
dispatcher's operator channel. Per-customer fill alerts already existed; the box-level
aggregate was the gap. READ-ONLY against Hetzner (GET only).
Phase-0 probe (gate PASSED): the live pool box 611714 returns capacity via
storage_box_type.size (1 TiB / bx11) and usage via a stats object (size/size_data/
size_snapshots), all bytes; our token reads it (200).
- hetznerapi: additive StorageBoxType + StorageBoxStats on StorageBox (no existing field/
method changed); fake carries them + a GetBoxCalls counter; golden decode test.
- monitor.OffsiteBoxChecker: OffsiteChecker-sibling for the box; fetch-throttled (1 GET/
15min), cached BoxSnapshot, escalation-only + recovery re-arm. FILL (used/capacity 80/90)
+ OVERSUB (Σ shared+enabled quotas / capacity, 2.0x) — independent. Σ from the ConfigJSON
Descriptor (offsite.ReadDescriptor, new), never the report echo; dedicated+disabled
excluded. Scope "pool-box" -> operator channel only, no SaveEvent. Failed fetch keeps the
last snapshot degraded; missing data never becomes 0% and never transitions a band.
- config: Alerting.OffsiteBoxFill{Warn,Crit}Percent + OffsiteOversubWarnRatio (80/90/2.0
defaults; thresholds pending Viktor's ruling). Constructed in the HETZNER_TOKEN branch,
60s sweep, snapshot handed to the web server.
- web: Offsite-tab panel (fill bar, Σ+ratio, per-customer usage/quota rows) + a compact
dashboard tile; reads the cached snapshot only, never fetches; nil -> "not configured".
Tests: 10 new + 4 red-proofs (throttle, Σ filter, escalation-only, failed-fetch honesty),
all confirmed red then restored. go build/vet/test all pass; hub confirm gate OK.
The immediate-sync arc covered only operator-initiated desired-state changes;
system-initiated mutations bumped the generation silently, so a freshly onboarded
box waited a full agent tick for state the hub had already minted (observed live at
slice-C onboarding). Wire the existing, live-proven notifiers into every system site
on the correct plane — call-site wiring only, no new mechanism.
Agent plane (poke.Notifier):
- web/pbsdr.go: PBSDRAutoProvision (the observed lag), ReissuePBSDR (also lifts the
pbsdrheal reconciler escalation, zero reconciler changes), handlePBSDRReissue —
each pokes AFTER the successful SetHostDesired, never on a blocked/error path.
- api: new nil-safe Poker seam (PokeHost/PokeAllHosts + SetPoker); handleAdminSetDesiredState
pokes the target host; handleAdminSetOperatorPeer fires PokeAllHosts only when the
fleet generation bump succeeded (fire-after-commit).
- main.go: one poke.Notifier now feeds both planes (SetPoke + SetPoker).
Controller plane (intent.Hub.Bump):
- api/reissueOnReenroll: one nil-guarded bump so a long-polling controller wakes in
seconds instead of on the 15-min cycle.
Deliberate non-sites (unchanged): WG register (undeliverable pre-tunnel — the agent
fast-tick SECONDARY owns it), WG delete (transport removed), pbsdrheal Restage (no
generation bump → the 60s ticker is the pickup path). internal/pbsdrheal byte-unchanged.
Tests: 10 non-hollow tests (web async channel-synchronized fake sender; api synchronous
fake Poker) with explicit zero-count negatives; representative red-proofs per group
(A/B/C/D) run-fail-restored. Green: go build/vet/test all pass.
A generic ISO carries NO customer secret. The box registers itself at the hub
as an unclaimed appliance; the operator binds it to a customer; the hub delivers
the customer-id + retrieval passphrase ONCE; day-0 completes via the slice-A path.
Hub (v0.62.0):
- store/appliance.go: appliance_registrations keyed by (uuid, mac_set) — MAC set
is the tiebreaker (duplicate SMBIOS UUIDs); token stored as sha256 only.
Idempotent register (sticky-discard), atomic one-shot delivery, bind/discard.
- api/appliance.go: POST /appliance/register (the one unauth endpoint, per-IP
rate-limited, 256-bit token); GET /appliance/poll (404 no-oracle / 204 unbound
/ 200 deliver-once / 410 delivered). Passphrase read live, never logged.
- web/appliances.go: Hosts-page "Unclaimed appliances" section + BIND (customer
picker, host count display-only) + DISCARD; SSH host-key fingerprints; events.
- Red-proofs: one-shot delivery + register idempotency (both proven red);
404-no-oracle, sticky-discard, bind staging, render. Green + confirm gate.
Scripts (v1.19.0):
- felhom-bootstrap.sh: ONE unit, TWO modes. Direct (env has customer/passphrase)
= slice-A path, byte-identical, only branched around. Pairing (generic) =
register + poll (RestartSec=30 is the poll timer); on delivery write the env
0600 and fall through to direct. Secrets + token shredded on success.
- build-felhom-iso.sh --pairing: generic secret-free ISO, -generic filename,
manifest mode=pairing. profiles/generic.profile (new).
- test/bootstrap-modes.sh: Scenario D (direct = zero appliance calls) + pairing
register/poll + delivery handoff — all green in a debian container.
- DeleteHost(deleteEscrow) demotes current host_escrow into host_escrow_superseded (copy-before-delete, same tx), spares existing; one shared demoteCurrentEscrowTx (reused by SaveHostEscrow). F-14 provenance/gate unchanged.
- DeleteCustomerConfig now purges both escrow tables for all the customer's hosts incl. already-deleted (F-14 provenance UNION) — the one true purge point.
- Wording: checkbox/refusal/Danger-zone → demotion. S6b OBSOLETE. Red-proofs TestDeleteHost_Demotes + TestDeleteCustomer_Purges + wording guard.
GET /api/v1/wait long-poll: the box holds an authed hanging GET; the hub
completes it the instant any operator intent bumps that customer's in-memory
generation, then the box fires its ordinary report and the ACK delivers
everything through the unchanged machinery. 240s hold with a 25s heartbeat
newline defeats the nginx 60s proxy_read_timeout with no ingress annotation;
WriteTimeout lifted per-connection via ResponseController.
- internal/intent: per-customer generation counter + waiter registry
(Bump/Wait/Close), coalescing to latest, race-closer, in-memory by design.
Red-proofs: counter-vs-queue + race-closer (run-fail-reverted).
- api/wait.go: the endpoint (per-customer only; global key 400; A cannot see B).
- web bumps after every intent write (fire-after-commit): config CRUD, claim
resend, offsite re-issue/freeze, password regen, block/unblock, floors
(global bumps all config-managed), controller log-tail + log-bundle.
- main.go: one intent hub shared by web+api; Close() before server.Shutdown.
Pairs with controller v0.140.0 (the long-poll client). Grounding:
documentation/audits/SPIKE-immediate-sync-transport-2026-07-16.md.
F2 claim re-issue on clean-slate re-enroll (ReissueForReenroll, host-enroll mint path,
single-bump, reset code; hub never stores the password so fork B). F3 offsite re-issue on
re-enroll (ReissueOffsiteForCustomer, same machinery as the manual button). 2.3 escrow honesty
(red-proofed): re-issuing offsite marks the escrow stale (MarkEscrowStale), withholds the
mismatched restic hash from auto-confirm, DR checklist shows stale not done. Events:
claim_reissued_reenroll / offsite_reissued / escrow_stale.
Controller + scripts unchanged (source contradicted both premises): the controller reads escrow
prereqs live from the agent; the installer can't know the descriptor-provisioned storage id. F4
root fix is agent-side -> ROADMAP R-22; demo unblocked live (Part 0 ACL grant). VALIDATION doc
F2 erratum + F3/F4 dispositions. Green gate + Scenario-C red-proof pass.
Implements SPIKE-pbsdr-selfheal-2026-07-15 (e8f8c44). A box re-installed/rolled
back onto its stable host_id loses its agent-side converged marker; the hub
keeps the enabled descriptor + a CONSUMED one-time secret, the WG peer persists
(changed==false, cascade can't re-fire), so the agent sits in waiting_secret
forever. The missing piece is a consumable secret, not the descriptor.
New internal/pbsdrheal reconciler (5m, wgsync shape): for enabled+provisioned
hosts whose latest report pbs_dr.state is a stuck state past a >=2-distinct-report
debounce, re-stage the stored secret (store.RestageHostPBSSecret: clear
consumed_at, no ep0 call, NO generation bump); escalate to Re-issue (web
ReissuePBSDR) only when no secret is stored or the agent reports consumed_failed.
Converged/disabled/verify_failed/DR-OFF = no-op. PBSDRHEAL_ONLY_HOST scopes a
supervised rollout. Scenarios A-F + all six red-proofs verified. No agent change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HEPuEwyyGDJdcsXLFsTWJn
hub v0.54.0: change operator login password from the Configuration UI
Adds a "Login password" card on /configuration. The password was previously
settable only via the hub-config ConfigMap (auth.password_hash) + redeploy.
- store: hub_settings key operator_password_hash + Get/SetOperatorPasswordHash
- server: passwordHash field -> configPasswordHash (seed); new
effectivePasswordHash() (DB override wins, else seed) is now the single
source for the CSRF gate, RequireAuth, and handleLogin
- POST /configuration/password (handleChangePassword): requires current
password, 8-72 byte new + confirm, bcrypt cost 10, persists DB override;
existing sessions kept valid; ConfigMap stays the break-glass reset path
- UI: current/new/confirm form + inline mismatch pre-check + 6 flashes
- tests + red-proofs: override precedence, happy-path via handleLogin,
wrong-current rejection, mismatch/too-short/no-op, template render
- docs: CHANGELOG, README (auth+config), REUSE, REPORT
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LbMm4T7Ayzs1unB9pN6Uqd
@
Customer status (dashboard row, /configs list, detail header + strip) is
now worst(controllerDerived, hostStatusOf(each expected host)) via the ONE
staleness definition (Server.hostStatus, hosts.go - shared with the
HostStalenessChecker; no second threshold). Any host down/stale caps the
customer at WARN with a cause chip naming the host ("host down: <id>");
pending (never-reported) hosts worsen only once the customer has reported
(onboarding exclusion). The three previously-inlined controller-status
chains collapse into controllerStatus() (rollup.go). Display + derivation
only - checker alerting untouched.
Live shape pinned (drill-1 / Peti cluster): host down 23h + controller
report minutes old rendered a GREEN row - TestRollup_DeadHostMasking now
fails that exact outcome. Red-proof: short-circuiting foldHostStatus to
controller-only flips C + two D subtests red ("dashboard row is GREEN
over a 23h-dead host").
pbsdrProvisionAtom, on tenantsync token_exists: consult the customer's
MOST RECENT host-deletion record. escrow_acked=true -> invoke the
EXISTING tenantsync Reissue op, store the audit event (pbsdr_auto_reissue,
hub-source, "Previous key destroyed (acknowledged deletion) - credentials
re-issued automatically."), proceed to secret+descriptor as a normal
provision. No record / un-acked -> the pre-existing refusal, byte-unchanged
(never-silently-re-key law; operator ruling 2026-07-13).
Scenario A test drives the REAL handler path over a REAL escrow-ack delete;
scenario B pins the exact non-effect (zero Reissue calls, no state).
Red-proof: bypassing the gate flips both B subtests to 303 (silent re-key).
admin