Compare commits
5 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| f5c3d77a61 | |||
| 5e05627c4e | |||
| 1e087bb158 | |||
| 29bcf4403e | |||
| 08ca67861f |
@@ -50,6 +50,12 @@ it **first** in the morning note. A decision you cannot write in that shape is o
|
|||||||
on Tester 1, demo-hp and demo-felhom through the normal update path and the box walk's checks are green
|
on Tester 1, demo-hp and demo-felhom through the normal update path and the box walk's checks are green
|
||||||
on all three — not at night unless the night brief says so. **Agent releases still stop for the
|
on all three — not at night unless the night brief says so. **Agent releases still stop for the
|
||||||
operator's vouch.** The golden is baked weekly and vouched by the operator; CC never vouches. **No `--no-verify`.**
|
operator's vouch.** The golden is baked weekly and vouched by the operator; CC never vouches. **No `--no-verify`.**
|
||||||
|
**A version number is pushed once** — `felhom-controller/controller/build.sh` now refuses a second
|
||||||
|
push of a tag that exists, and a registry it cannot reach is a refusal too. This is a FACT about
|
||||||
|
what the build does, not a new rule: a fix for something already pushed is the next patch version,
|
||||||
|
and a box on a bad image is moved by the normal update path, never by re-tagging. Earned on
|
||||||
|
2026-10-15, when 0.315.0 was pushed three times from three different commits and a cache kept
|
||||||
|
serving the first one (`09-update-architecture.md` §0).
|
||||||
7. **An enumerated gap becomes a row in the same session — or, if it is small, is fixed in it** (the size rule).
|
7. **An enumerated gap becomes a row in the same session — or, if it is small, is fixed in it** (the size rule).
|
||||||
Prose is not a record.
|
Prose is not a record.
|
||||||
8. **Hungarian text is searched with ASCII fragments**, with a positive and a negative control.
|
8. **Hungarian text is searched with ASCII fragments**, with a positive and a negative control.
|
||||||
|
|||||||
+37
-1
@@ -1,4 +1,40 @@
|
|||||||
## Unreleased (next: v0.164.0; 2026-10-24 attended release run) — the weekly upload line test (R-1247)
|
## 2026-10-24 (review round 2, unattended) — `.claude/rules/unprompted-work.md` aligned with its four siblings
|
||||||
|
|
||||||
|
- The copy lacked item 6's paragraph „A version number is pushed once …" that the felhom.eu and felhom-controller copies
|
||||||
|
have carried since 2026-10-15; the file's header says „change all five or none". The paragraph states a fact
|
||||||
|
(`felhom-controller/controller/build.sh` refuses an existing tag and an unreachable registry). Now byte-identical to
|
||||||
|
the other four (md5 b1891bd4). No code change. Record: felhom.eu `documentation/audits/review-round-2-2026-10-24/EDITS.md`.
|
||||||
|
|
||||||
|
## Unreleased (next: v0.165.0) — the line test for a box that seeds all day (R-1250) (2026-10-24) — committed, not released
|
||||||
|
|
||||||
|
Attended brief 2026-10-24, Part 4 (commit only; the release rides the next normal train). The weekly test (v0.164.0)
|
||||||
|
starts only after 10 quiet minutes; a torrent app that seeds all day never gives them, so that box never got a tested
|
||||||
|
line and qBittorrent never got its 50 % limit. Now:
|
||||||
|
|
||||||
|
- **`linetest.Tester.WaitingForQuiet`** — due, inside the window, tries left, tunnel up, and only the quiet missing.
|
||||||
|
`GET /host/line-test` carries it as `waiting_for_quiet`; a controller with a seeding torrent app acts only on it.
|
||||||
|
It needs the full watched window (11 minute samples): a freshly started agent is not known to be busy, so it never
|
||||||
|
invites a hold (found in the live proof, 2026-10-24: the first build said `true` before one minute was watched;
|
||||||
|
`TestFreshTesterIsNotWaiting`).
|
||||||
|
- **`linetest.Tester.RunNow`** and **`POST /host/line-test/run`** (synchronous, about 55 s) — a test on request while the
|
||||||
|
controller holds its app's upload low: every weekly rule holds (due, window, `MaxTriesPerDay`, tunnel, the one heavy-op
|
||||||
|
gate) except the 10 quiet minutes, which become a 5 s settle and a 10 s quiet check (below 2 Mbit/s, so a request
|
||||||
|
never measures the family's own upload); the three runs are 5 s apart instead of 30. A refusal answers its code
|
||||||
|
(`not-due`, `outside-window`, `tries-used`, `no-tunnel`, `heavy-op`, `not-quiet`, `server-busy`); a failed request
|
||||||
|
spends one of the day's tries. `Tick` and `RunNow` now share one `ready()` check.
|
||||||
|
- Tests (each seen failing): `TestSeedingBoxIsWaitingForQuietAndRunNowMeasuresIt` (the sleeps, the 60 s bound),
|
||||||
|
`TestRunNowRefusesAnUplinkThatIsStillBusy`, `TestRunNowKeepsTheWeeklyRules`, `TestRunNowFailuresSpendTheDaysTries`,
|
||||||
|
`TestQuietBoxIsNotWaiting`, `TestLineTestRunRoute`; a compile-time assertion that the Tester serves the request half.
|
||||||
|
|
||||||
|
**No sudoers or config-bundle change. MinController: none** — an older controller never calls the route.
|
||||||
|
|
||||||
|
## v0.164.0 — the weekly upload line test (R-1247) (2026-10-24) — RELEASED on the operator's ruling (attended release run); vouched by the operator 09:44:58 CEST
|
||||||
|
|
||||||
|
Binary sha256 `1b803d8b70f8afbe1546f6989f35b6b0e29c9ed63f6acb3843d3c04ef0d24b2b`; config bundle sha256
|
||||||
|
`1d7b54d39d0c1aff682ba0c0a7f91aea0f9407a10d2f5a8bbfdc858d32c13341` (configs unchanged — no bundle job sent). Built from
|
||||||
|
`f7e1dcb` (CI 2277 green) in a fresh clone by `scripts/release-agent.sh`. Delivered by one signed `agent_update` per box
|
||||||
|
(07:25Z): Tester 1, demo-hp and demo-felhom — COMPLETED, self-check 76/76, committed after the 1-minute dwell, the line
|
||||||
|
test started (felhom.eu `audits/line-test-2026-10-24/agent-0164-boxes.txt`).
|
||||||
|
|
||||||
Operator ruling 2026-10-24 07:26: once a week the box's HOST measures its upload line with a 10-second iperf3 upload to an
|
Operator ruling 2026-10-24 07:26: once a week the box's HOST measures its upload line with a 10-second iperf3 upload to an
|
||||||
iperf3 server on ep0 that answers **only inside the WireGuard tunnel** (felhom.eu `scripts/ep0-iperf3/`; installed the same
|
iperf3 server on ep0 that answers **only inside the WireGuard tunnel** (felhom.eu `scripts/ep0-iperf3/`; installed the same
|
||||||
|
|||||||
@@ -1,36 +1,23 @@
|
|||||||
# REPORT — day 2026-10-23: agent 0.163.0 released, delivered to the three test boxes, vouched by the operator
|
# REPORT — a test on request for a box that seeds all day (R-1250), 2026-10-24 (attended)
|
||||||
|
|
||||||
**What:** the day brief's Part 2. Design read: `03-host-agent.md` (the tunnel, the bundle), `11` §5.4.2 (the step
|
Brief: „two DooPlex backup alarms …", Part 4 (commit only; the release rides the next normal train and needs the
|
||||||
bundle, R-880).
|
operator's vouch). Commits `29bcf44` (CI 2286 success) and `1e087bb` (CI 2288 success). Not released.
|
||||||
|
|
||||||
## Release
|
- **What:** `GET /host/line-test` reports `waiting_for_quiet` (due, in the window, tries left, tunnel up, the full
|
||||||
|
10-minute window watched and busy). `POST /host/line-test/run` runs one measurement on request while the controller
|
||||||
|
holds its torrent app: every weekly rule holds except the 10 quiet minutes, which become a 5 s settle and a 10 s quiet
|
||||||
|
check (< 2 Mbit/s); runs 5 s apart; about 55 s. Refusals answer a code; a failed request spends one of the day's tries.
|
||||||
|
- **Found live and fixed (`1e087bb`):** a freshly started tester said `waiting_for_quiet` before it had watched a
|
||||||
|
minute — a quiet box's app would have been held for nothing. Now it needs the full window.
|
||||||
|
- **Tests:** 7 new, each seen failing (CHANGELOG lists them); a compile-time assertion that the Tester serves the route.
|
||||||
|
- **Live (demo-hp host, a probe built from `1e087bb` running the real `Tester` with its minute watcher, plain HTTP):**
|
||||||
|
a real 10 Mbit/s torrent seed → `waiting_for_quiet` after 11 samples (worst minute 10.8); the control request refused
|
||||||
|
`not-quiet` (11.1 Mbit/s); with the controller's hold: runs 93.6 / 89.6 / 94.2 Mbit/s, the request 56.1 s. Evidence:
|
||||||
|
felhom.eu `audits/dooplex-alarms-2026-10-24/part4-r1250-live-proof.txt`.
|
||||||
|
- **Teardown:** the probe stopped and deleted from demo-hp; no iperf3 left running; the installed agent (0.164.0) untouched.
|
||||||
|
|
||||||
- Code: `1109092` (D4 v2: the sshd drop-in `/etc/ssh/sshd_config.d/10-felhom-root-key-only.conf`,
|
---
|
||||||
`PermitRootLogin prohibit-password`, `try-reload-or-restart`, an `sshd -T` read-back; RootKeyOnly 6 tests). CI 2246
|
|
||||||
success. CHANGELOG `b4721ee` (v0.163.0 header, both sha256), CI 2249 success.
|
|
||||||
- `scripts/release-agent.sh 0.163.0`: binary sha256 `561b5316cd511de9fa9d8b8be4553da7add684f715d3bea533ca330ff7f3763c`,
|
|
||||||
config bundle sha256 `3afb88d9fbbc4adc16083fa52a98cd4974583c6378a77a1a6e0ed8ae6d1b80c8`; tag `v0.163.0`; verified by
|
|
||||||
an independent download.
|
|
||||||
- **Vouched by the operator** 07:58:46 UTC — read back from the hub DB (with its WAL) and from the running hub's
|
|
||||||
`/configuration` page: agent 0.163.0, both sha256 as above.
|
|
||||||
|
|
||||||
## Delivery — three signed jobs per box (felhom-opsign, key felhom-op-1)
|
**Also on 2026-10-24 (review round 2, a separate unattended session):** `.claude/rules/unprompted-work.md` aligned with
|
||||||
|
its four sibling copies (the missing „A version number is pushed once" paragraph); no code change. The report above is
|
||||||
The D4 drop-in is a NEW bundle path, and the boxes ran bundle 0.160.0 (sha256 `6b0609…`), so: `agent_update 0.163.0`
|
the attended session's and is kept as it was.
|
||||||
→ `agent_config_update 0.163.0-step1` (built with `scripts/build-step-bundle.py` from 0.160.0: only
|
|
||||||
`felhom-os-apply` replaced; sha256 `6cbc0669…`, uploaded and read back) → `agent_config_update 0.163.0`.
|
|
||||||
|
|
||||||
| Box | agent | bundle | self-check | swap = RAM (pool guest 9201) | drop-in, `sshd -T` | sshd reloaded | key login | felhom-sshd |
|
|
||||||
|---|---|---|---|---|---|---|---|---|
|
|
||||||
| Tester 1 | 0.163.0 | 0.163.0 | ok | 4096 / 4096 | in place, `without-password` | 09:27:38 SIGHUP | ok | active, listening, `sshd -t` ok |
|
|
||||||
| demo-hp | 0.163.0 | 0.163.0 | ok | 25898 / 25898 | in place, `without-password` | 09:19:44 SIGHUP | ok | active, listening, `sshd -t` ok |
|
|
||||||
| demo-felhom | 0.163.0 | 0.163.0 | ok | 12288 / 12288 | in place, `without-password` | 09:24:42 SIGHUP | ok | active, listening, `sshd -t` ok |
|
|
||||||
|
|
||||||
**Not proven, said plainly:** the real root password over SSH was not tried (CC does not hold it); a dummy password is
|
|
||||||
refused, but OpenSSH logs that only as „Failed password", so the policy proof is `sshd -T` + the reload. felhom-sshd
|
|
||||||
(:8822) admits only the operator peer over the tunnel, so a connection from DooPlex times out by design. The console
|
|
||||||
login is not touched by the drop-in (`passwd -S root` = P on all three); not tried at a physical console.
|
|
||||||
|
|
||||||
Installer 1.36.0 (the uninstall removes the drop-in) was published the same day (felhom.eu). R-982 and R-1204 closed.
|
|
||||||
Evidence: felhom.eu `documentation/audits/day-2026-10-23/07-agent-0163-boxes.txt`, `10-vouch-readback.txt`,
|
|
||||||
`11-installer-1360.txt`.
|
|
||||||
|
|||||||
@@ -5,8 +5,16 @@ import (
|
|||||||
"go/parser"
|
"go/parser"
|
||||||
"go/token"
|
"go/token"
|
||||||
"testing"
|
"testing"
|
||||||
|
|
||||||
|
"gitea.dooplex.hu/admin/felhom-agent/internal/linetest"
|
||||||
|
"gitea.dooplex.hu/admin/felhom-agent/internal/localapi"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
// R-1250: SetLineTest takes a LineTestReader and finds the request half by a type assertion, so a Tester that lost
|
||||||
|
// WaitingForQuiet or RunNow would compile and silently answer 503 to every seeding box. This line makes it a build
|
||||||
|
// failure instead.
|
||||||
|
var _ localapi.LineTestRequester = (*linetest.Tester)(nil)
|
||||||
|
|
||||||
// R-1247: the weekly line test has the fstrim shape (component + reporter seam + goroutine) plus a local-API reader,
|
// R-1247: the weekly line test has the fstrim shape (component + reporter seam + goroutine) plus a local-API reader,
|
||||||
// so its wiring is asserted from the AST. It must share the ONE heavy-op gate (heavyOps) — or it could run beside a
|
// so its wiring is asserted from the AST. It must share the ONE heavy-op gate (heavyOps) — or it could run beside a
|
||||||
// backup — and take its server from the tunnel manager, never a literal address.
|
// backup — and take its server from the tunnel manager, never a literal address.
|
||||||
|
|||||||
+113
-11
@@ -19,6 +19,10 @@
|
|||||||
// - No tunnel (no operational wg-felhom with a server address) → no test and no value; the controller then keeps its
|
// - No tunnel (no operational wg-felhom with a server address) → no test and no value; the controller then keeps its
|
||||||
// passive floor and sets no qBittorrent limit.
|
// passive floor and sets no qBittorrent limit.
|
||||||
// - The last result is persisted, so a restart neither loses it nor re-tests.
|
// - The last result is persisted, so a restart neither loses it nor re-tests.
|
||||||
|
// - A box whose torrent app seeds all day is never quiet for 10 minutes (R-1250). For it the controller holds the
|
||||||
|
// app's upload limit very low and asks for a test on request (RunNow): every rule above except the 10 quiet
|
||||||
|
// minutes still holds, the quiet is checked over RequestQuietWindow instead, and the runs are RequestRunGap
|
||||||
|
// apart, so the whole request ends in about 55 s.
|
||||||
package linetest
|
package linetest
|
||||||
|
|
||||||
import (
|
import (
|
||||||
@@ -56,6 +60,11 @@ const (
|
|||||||
TickInterval = time.Minute
|
TickInterval = time.Minute
|
||||||
FirstTickDelay = 5 * time.Minute
|
FirstTickDelay = 5 * time.Minute
|
||||||
GateName = "line-test"
|
GateName = "line-test"
|
||||||
|
|
||||||
|
// A test on request (R-1250): the controller holds the app's limit while it waits, so it is short.
|
||||||
|
RequestSettle = 5 * time.Second // the held app's last packets leave before the quiet is read
|
||||||
|
RequestQuietWindow = 10 * time.Second // the uplink must send below QuietMbps over this window
|
||||||
|
RequestRunGap = 5 * time.Second
|
||||||
)
|
)
|
||||||
|
|
||||||
// ScheduleText is carried on the host report.
|
// ScheduleText is carried on the host report.
|
||||||
@@ -258,30 +267,52 @@ func (t *Tester) Run(ctx context.Context) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Tick is one look.
|
// Refusal is why a test on request did not run (RunNow). Code is stable for the controller.
|
||||||
func (t *Tester) Tick(ctx context.Context) {
|
type Refusal struct {
|
||||||
|
Code string // not-due | outside-window | tries-used | no-tunnel | heavy-op | not-quiet
|
||||||
|
Msg string
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *Refusal) Error() string { return "linetest: " + r.Code + ": " + r.Msg }
|
||||||
|
|
||||||
|
// ready applies every rule except the quiet one: due this week, inside the window (the slot on the due day), tries
|
||||||
|
// left today, the tunnel up. It answers the server address and the local day.
|
||||||
|
func (t *Tester) ready() (addr, day string, refusal *Refusal) {
|
||||||
lt := t.now().In(t.loc)
|
lt := t.now().In(t.loc)
|
||||||
t.mu.Lock()
|
t.mu.Lock()
|
||||||
st := t.st
|
st := t.st
|
||||||
samples := append([]txSample(nil), t.samples...)
|
|
||||||
t.mu.Unlock()
|
t.mu.Unlock()
|
||||||
if !Due(st, lt) {
|
if !Due(st, lt) {
|
||||||
return
|
return "", "", &Refusal{"not-due", "measured this week"}
|
||||||
}
|
}
|
||||||
anchor := weekAnchor(lt)
|
anchor := weekAnchor(lt)
|
||||||
anchorDay := lt.YearDay() == anchor.YearDay() && lt.Year() == anchor.Year()
|
anchorDay := lt.YearDay() == anchor.YearDay() && lt.Year() == anchor.Year()
|
||||||
if !EligibleAt(lt, t.slot, anchorDay) {
|
if !EligibleAt(lt, t.slot, anchorDay) {
|
||||||
return
|
return "", "", &Refusal{"outside-window", "a test starts 10:00-15:59 local, from this host's slot on the due day"}
|
||||||
}
|
}
|
||||||
day := lt.Format("2006-01-02")
|
day = lt.Format("2006-01-02")
|
||||||
if st.TriesDay == day && st.TriesToday >= MaxTriesPerDay {
|
if st.TriesDay == day && st.TriesToday >= MaxTriesPerDay {
|
||||||
return
|
return "", "", &Refusal{"tries-used", fmt.Sprintf("%d tries today", st.TriesToday)}
|
||||||
}
|
}
|
||||||
ip, ok := t.target()
|
ip, ok := t.target()
|
||||||
if !ok {
|
if !ok {
|
||||||
t.logger.Debug("linetest: no operational tunnel — no test")
|
return "", "", &Refusal{"no-tunnel", "no operational tunnel"}
|
||||||
|
}
|
||||||
|
return netip.AddrPortFrom(ip, ServerPort).String(), day, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Tick is one look.
|
||||||
|
func (t *Tester) Tick(ctx context.Context) {
|
||||||
|
addr, day, refusal := t.ready()
|
||||||
|
if refusal != nil {
|
||||||
|
if refusal.Code == "no-tunnel" {
|
||||||
|
t.logger.Debug("linetest: no operational tunnel — no test")
|
||||||
|
}
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
t.mu.Lock()
|
||||||
|
samples := append([]txSample(nil), t.samples...)
|
||||||
|
t.mu.Unlock()
|
||||||
if q, worst := quiet(samples); !q {
|
if q, worst := quiet(samples); !q {
|
||||||
t.logger.Debug("linetest: uplink not quiet — waiting", "worst_minute_mbps", fmt.Sprintf("%.1f", worst), "samples", len(samples))
|
t.logger.Debug("linetest: uplink not quiet — waiting", "worst_minute_mbps", fmt.Sprintf("%.1f", worst), "samples", len(samples))
|
||||||
return
|
return
|
||||||
@@ -292,16 +323,80 @@ func (t *Tester) Tick(ctx context.Context) {
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
defer release()
|
defer release()
|
||||||
t.measure(ctx, netip.AddrPortFrom(ip, ServerPort).String(), day)
|
t.measure(ctx, addr, day, RunGap)
|
||||||
}
|
}
|
||||||
|
|
||||||
func (t *Tester) measure(ctx context.Context, addr, day string) {
|
// WaitingForQuiet reports that a measurement is due and could start now but the uplink has not been quiet for
|
||||||
|
// QuietMinutes — the case a seeding torrent app makes permanent (R-1250). The controller reads it from
|
||||||
|
// GET /host/line-test and only then holds its app's limit and calls RunNow.
|
||||||
|
func (t *Tester) WaitingForQuiet() bool {
|
||||||
|
if _, _, refusal := t.ready(); refusal != nil {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
t.mu.Lock()
|
||||||
|
samples := append([]txSample(nil), t.samples...)
|
||||||
|
t.mu.Unlock()
|
||||||
|
if len(samples) < QuietMinutes+1 {
|
||||||
|
return false // not yet watched for 10 minutes (a fresh start): not known to be busy, so no hold
|
||||||
|
}
|
||||||
|
q, _ := quiet(samples)
|
||||||
|
return !q
|
||||||
|
}
|
||||||
|
|
||||||
|
// RunNow is a test on request (POST /host/line-test/run, R-1250): every rule of the weekly test except the 10 quiet
|
||||||
|
// minutes, which the caller replaces by holding its own upload down. The uplink must still be quiet over
|
||||||
|
// RequestQuietWindow (after RequestSettle), so a request never measures the family's upload against itself. A
|
||||||
|
// *Refusal says why nothing ran; any other error is a failed measurement (the same as a failed weekly attempt).
|
||||||
|
func (t *Tester) RunNow(ctx context.Context) (mbps float64, runs []float64, err error) {
|
||||||
|
addr, day, refusal := t.ready()
|
||||||
|
if refusal != nil {
|
||||||
|
return 0, nil, refusal
|
||||||
|
}
|
||||||
|
release, busy, ok := t.gate.TryAcquire(GateName)
|
||||||
|
if !ok {
|
||||||
|
return 0, nil, &Refusal{"heavy-op", busy}
|
||||||
|
}
|
||||||
|
defer release()
|
||||||
|
if t.uplink == nil {
|
||||||
|
return 0, nil, &Refusal{"not-quiet", "no uplink counter"}
|
||||||
|
}
|
||||||
|
if !t.sleep(ctx, RequestSettle) {
|
||||||
|
return 0, nil, ctx.Err()
|
||||||
|
}
|
||||||
|
tx0, err0 := t.uplink()
|
||||||
|
at0 := t.now()
|
||||||
|
if !t.sleep(ctx, RequestQuietWindow) {
|
||||||
|
return 0, nil, ctx.Err()
|
||||||
|
}
|
||||||
|
tx1, err1 := t.uplink()
|
||||||
|
if err0 != nil || err1 != nil || tx1 < tx0 {
|
||||||
|
return 0, nil, &Refusal{"not-quiet", "uplink counter unreadable or reset"}
|
||||||
|
}
|
||||||
|
secs := t.now().Sub(at0).Seconds()
|
||||||
|
if secs <= 0 {
|
||||||
|
secs = RequestQuietWindow.Seconds()
|
||||||
|
}
|
||||||
|
if m := float64(tx1-tx0) * 8 / secs / 1e6; m >= QuietMbps {
|
||||||
|
t.logger.Info("linetest: test on request refused — the uplink is not quiet", "mbps", fmt.Sprintf("%.1f", m))
|
||||||
|
return 0, nil, &Refusal{"not-quiet", fmt.Sprintf("the uplink sent %.1f Mbit/s while held", m)}
|
||||||
|
}
|
||||||
|
t.logger.Info("linetest: test on request (a held torrent app, R-1250)", "server", addr)
|
||||||
|
if err := t.measure(ctx, addr, day, RequestRunGap); err != nil {
|
||||||
|
return 0, nil, err
|
||||||
|
}
|
||||||
|
t.mu.Lock()
|
||||||
|
defer t.mu.Unlock()
|
||||||
|
return t.st.Mbps, append([]float64(nil), t.st.RunsMbps...), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// measure runs the runs, gap apart, and stores the result. It returns nil only when a value was stored.
|
||||||
|
func (t *Tester) measure(ctx context.Context, addr, day string, gap time.Duration) error {
|
||||||
start := t.now()
|
start := t.now()
|
||||||
var runs []float64
|
var runs []float64
|
||||||
var lastErr error
|
var lastErr error
|
||||||
busy := false
|
busy := false
|
||||||
for i := 0; i < RunsPerMeasurement; i++ {
|
for i := 0; i < RunsPerMeasurement; i++ {
|
||||||
if i > 0 && !t.sleep(ctx, RunGap) {
|
if i > 0 && !t.sleep(ctx, gap) {
|
||||||
break
|
break
|
||||||
}
|
}
|
||||||
rctx, cancel := context.WithTimeout(ctx, RunTimeout)
|
rctx, cancel := context.WithTimeout(ctx, RunTimeout)
|
||||||
@@ -356,6 +451,13 @@ func (t *Tester) measure(ctx context.Context, addr, day string) {
|
|||||||
if err := t.saveLocked(); err != nil {
|
if err := t.saveLocked(); err != nil {
|
||||||
t.logger.Warn("linetest: state save failed — the result will not survive a restart", "path", t.statePath, "err", err)
|
t.logger.Warn("linetest: state save failed — the result will not survive a restart", "path", t.statePath, "err", err)
|
||||||
}
|
}
|
||||||
|
if len(runs) > 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if lastErr == nil {
|
||||||
|
lastErr = errors.New("linetest: no run succeeded")
|
||||||
|
}
|
||||||
|
return lastErr
|
||||||
}
|
}
|
||||||
|
|
||||||
// Value is the measured line (ok=false: never measured).
|
// Value is the measured line (ok=false: never measured).
|
||||||
|
|||||||
@@ -431,3 +431,144 @@ func TestQuietNeedsTenFullMinutes(t *testing.T) {
|
|||||||
t.Fatal("a gap in the samples must not read quiet")
|
t.Fatal("a gap in the samples must not read quiet")
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ── a test on request (R-1250): a torrent app that seeds all day ───────────────────────────────────────────────────
|
||||||
|
|
||||||
|
// seedingRig: due and eligible, but every minute of the last ten sent at the line's speed — the weekly Tick never runs.
|
||||||
|
func seedingRig(t *testing.T, outs ...runOut) *rig {
|
||||||
|
r := newRig(t, outs...)
|
||||||
|
for i := 0; i < QuietMinutes+1; i++ {
|
||||||
|
r.minute(600_000_000) // 80 Mbit/s: a seed at the line's speed
|
||||||
|
}
|
||||||
|
return r
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSeedingBoxIsWaitingForQuietAndRunNowMeasuresIt(t *testing.T) {
|
||||||
|
r := seedingRig(t, runOut{mbps: 88}, runOut{mbps: 92}, runOut{mbps: 90})
|
||||||
|
r.t.Tick(context.Background())
|
||||||
|
if r.run.calls != 0 {
|
||||||
|
t.Fatal("the weekly Tick tested a seeding box")
|
||||||
|
}
|
||||||
|
if !r.t.WaitingForQuiet() {
|
||||||
|
t.Fatal("a due, eligible, seeding box must report waiting_for_quiet")
|
||||||
|
}
|
||||||
|
var slept []time.Duration
|
||||||
|
r.t.sleep = func(_ context.Context, d time.Duration) bool { slept = append(slept, d); return true }
|
||||||
|
mbps, runs, err := r.t.RunNow(context.Background()) // the controller holds the app: the counter stands still
|
||||||
|
if err != nil || mbps != 92 || len(runs) != RunsPerMeasurement || r.run.calls != RunsPerMeasurement {
|
||||||
|
t.Fatalf("RunNow = %.1f %v %v, calls %d; want the best of 3 = 92", mbps, runs, err, r.run.calls)
|
||||||
|
}
|
||||||
|
want := []time.Duration{RequestSettle, RequestQuietWindow, RequestRunGap, RequestRunGap}
|
||||||
|
if len(slept) != len(want) {
|
||||||
|
t.Fatalf("slept %v, want %v", slept, want)
|
||||||
|
}
|
||||||
|
var total time.Duration
|
||||||
|
for i := range want {
|
||||||
|
if slept[i] != want[i] {
|
||||||
|
t.Fatalf("slept %v, want %v", slept, want)
|
||||||
|
}
|
||||||
|
total += slept[i]
|
||||||
|
}
|
||||||
|
if total+RunsPerMeasurement*TestTime > 60*time.Second {
|
||||||
|
t.Fatalf("a request holds the app %v + runs; it must end inside 60 s", total)
|
||||||
|
}
|
||||||
|
if v, _, ok := r.t.Value(); !ok || v != 92 {
|
||||||
|
t.Fatalf("the requested measurement was not stored: %.1f %v", v, ok)
|
||||||
|
}
|
||||||
|
if r.t.WaitingForQuiet() {
|
||||||
|
t.Fatal("measured this week: no longer waiting")
|
||||||
|
}
|
||||||
|
if _, _, err := r.t.RunNow(context.Background()); refusalCode(err) != "not-due" {
|
||||||
|
t.Fatalf("a second request this week: %v, want not-due", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func refusalCode(err error) string {
|
||||||
|
var ref *Refusal
|
||||||
|
if errors.As(err, &ref) {
|
||||||
|
return ref.Code
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
|
// The held app is not the only uploader: a request still refuses to measure a busy uplink.
|
||||||
|
func TestRunNowRefusesAnUplinkThatIsStillBusy(t *testing.T) {
|
||||||
|
r := seedingRig(t, runOut{mbps: 90}, runOut{mbps: 90}, runOut{mbps: 90})
|
||||||
|
r.t.uplink = func() (uint64, error) { r.tx += 20_000_000; return r.tx, nil } // 20 MB per 10 s = 16 Mbit/s
|
||||||
|
if _, _, err := r.t.RunNow(context.Background()); refusalCode(err) != "not-quiet" || r.run.calls != 0 {
|
||||||
|
t.Fatalf("RunNow on a busy uplink: %v, calls %d; want not-quiet and no run", err, r.run.calls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Every other rule of the weekly test holds for a request.
|
||||||
|
func TestRunNowKeepsTheWeeklyRules(t *testing.T) {
|
||||||
|
r := seedingRig(t)
|
||||||
|
r.gate.busy = "backup:felhom-pbs"
|
||||||
|
if _, _, err := r.t.RunNow(context.Background()); refusalCode(err) != "heavy-op" {
|
||||||
|
t.Fatalf("beside a backup: %v, want heavy-op", err)
|
||||||
|
}
|
||||||
|
r.gate.busy = ""
|
||||||
|
r.tunnel = false
|
||||||
|
if _, _, err := r.t.RunNow(context.Background()); refusalCode(err) != "no-tunnel" {
|
||||||
|
t.Fatalf("no tunnel: %v, want no-tunnel", err)
|
||||||
|
}
|
||||||
|
if r.t.WaitingForQuiet() {
|
||||||
|
t.Fatal("no tunnel: a request could not run, so it must not be invited")
|
||||||
|
}
|
||||||
|
r.tunnel = true
|
||||||
|
r.now = time.Date(2026, 10, 20, 3, 0, 0, 0, time.UTC) // Tuesday 03:00 — the night chain
|
||||||
|
if _, _, err := r.t.RunNow(context.Background()); refusalCode(err) != "outside-window" {
|
||||||
|
t.Fatalf("at night: %v, want outside-window", err)
|
||||||
|
}
|
||||||
|
if r.run.calls != 0 {
|
||||||
|
t.Fatalf("%d runs on refused requests", r.run.calls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A failed request counts as a try, so a controller cannot request more than MaxTriesPerDay tests a day.
|
||||||
|
func TestRunNowFailuresSpendTheDaysTries(t *testing.T) {
|
||||||
|
var outs []runOut
|
||||||
|
for i := 0; i < MaxTriesPerDay*RunsPerMeasurement; i++ {
|
||||||
|
outs = append(outs, runOut{err: errors.New("connection refused")})
|
||||||
|
}
|
||||||
|
r := seedingRig(t, outs...)
|
||||||
|
for i := 0; i < MaxTriesPerDay; i++ {
|
||||||
|
if _, _, err := r.t.RunNow(context.Background()); err == nil || refusalCode(err) != "" {
|
||||||
|
t.Fatalf("try %d: %v, want a failed measurement", i+1, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if _, _, err := r.t.RunNow(context.Background()); refusalCode(err) != "tries-used" {
|
||||||
|
t.Fatalf("after %d failures: %v, want tries-used", MaxTriesPerDay, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A quiet box tests itself on the weekly Tick: it must not invite the controller to hold its app.
|
||||||
|
func TestQuietBoxIsNotWaiting(t *testing.T) {
|
||||||
|
r := newRig(t)
|
||||||
|
if r.t.WaitingForQuiet() {
|
||||||
|
t.Fatal("ten quiet minutes: the weekly Tick runs; waiting_for_quiet must be false")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Live proof 2026-10-24 (demo-hp): a freshly started tester answered waiting_for_quiet before it had watched a single
|
||||||
|
// minute, so a quiet box's torrent app would be held for nothing. Waiting needs the full window, seen busy.
|
||||||
|
func TestFreshTesterIsNotWaiting(t *testing.T) {
|
||||||
|
r := &rig{run: &fakeRunner{}, gate: &fakeGate{}, now: time.Date(2026, 10, 20, 12, 0, 0, 0, time.UTC), tunnel: true}
|
||||||
|
r.t = New(r.run, func() (netip.Addr, bool) { return netip.MustParseAddr("10.77.0.1"), r.tunnel }, r.gate,
|
||||||
|
func() (uint64, error) { return r.tx, nil }, "host-x", filepath.Join(t.TempDir(), "line-test.json"), nil)
|
||||||
|
r.t.loc = time.UTC
|
||||||
|
r.t.now = func() time.Time { return r.now }
|
||||||
|
if r.t.WaitingForQuiet() {
|
||||||
|
t.Fatal("no minute watched yet: waiting_for_quiet must be false")
|
||||||
|
}
|
||||||
|
for i := 0; i < QuietMinutes; i++ {
|
||||||
|
r.minute(600_000_000)
|
||||||
|
}
|
||||||
|
if r.t.WaitingForQuiet() {
|
||||||
|
t.Fatal("10 samples are 9 minutes: still not a full window")
|
||||||
|
}
|
||||||
|
r.minute(600_000_000)
|
||||||
|
if !r.t.WaitingForQuiet() {
|
||||||
|
t.Fatal("a full busy window: waiting_for_quiet must be true")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -1,8 +1,12 @@
|
|||||||
package localapi
|
package localapi
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
"net/http"
|
"net/http"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
"gitea.dooplex.hu/admin/felhom-agent/internal/linetest"
|
||||||
)
|
)
|
||||||
|
|
||||||
// LineTestReader is the R-1247 weekly upload line test (internal/linetest.Tester) as the local API reads it.
|
// LineTestReader is the R-1247 weekly upload line test (internal/linetest.Tester) as the local API reads it.
|
||||||
@@ -10,6 +14,13 @@ type LineTestReader interface {
|
|||||||
Value() (mbps float64, at time.Time, ok bool)
|
Value() (mbps float64, at time.Time, ok bool)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// LineTestRequester is the R-1250 half: a due test the uplink never lets start, and a test on request. The Tester
|
||||||
|
// satisfies it; a reader without it answers waiting_for_quiet:false and 503 on the POST.
|
||||||
|
type LineTestRequester interface {
|
||||||
|
WaitingForQuiet() bool
|
||||||
|
RunNow(ctx context.Context) (mbps float64, runs []float64, err error)
|
||||||
|
}
|
||||||
|
|
||||||
// LineTestResponse is GET /host/line-test. Present=false: never measured, or the job is not wired (no tunnel, disabled,
|
// LineTestResponse is GET /host/line-test. Present=false: never measured, or the job is not wired (no tunnel, disabled,
|
||||||
// or an agent before the route answers 404 — the controller reads all three as „unknown").
|
// or an agent before the route answers 404 — the controller reads all three as „unknown").
|
||||||
type LineTestResponse struct {
|
type LineTestResponse struct {
|
||||||
@@ -17,6 +28,19 @@ type LineTestResponse struct {
|
|||||||
Mbps float64 `json:"mbps,omitempty"`
|
Mbps float64 `json:"mbps,omitempty"`
|
||||||
MeasuredAt string `json:"measured_at,omitempty"` // RFC3339 UTC
|
MeasuredAt string `json:"measured_at,omitempty"` // RFC3339 UTC
|
||||||
Method string `json:"method,omitempty"` // "iperf3-tunnel": a 10-second upload to ep0 inside the tunnel
|
Method string `json:"method,omitempty"` // "iperf3-tunnel": a 10-second upload to ep0 inside the tunnel
|
||||||
|
// WaitingForQuiet: a measurement is due and could start now, but the uplink has not been quiet for 10 minutes
|
||||||
|
// (R-1250). The controller with a seeding torrent app then holds its limit and POSTs /host/line-test/run.
|
||||||
|
WaitingForQuiet bool `json:"waiting_for_quiet,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// LineTestRunResponse is POST /host/line-test/run. OK=false with Refused: nothing ran (Refused is linetest.Refusal's
|
||||||
|
// code, or "server-busy"); OK=false without it: the runs failed.
|
||||||
|
type LineTestRunResponse struct {
|
||||||
|
OK bool `json:"ok"`
|
||||||
|
Mbps float64 `json:"mbps,omitempty"`
|
||||||
|
RunsMbps []float64 `json:"runs_mbps,omitempty"`
|
||||||
|
Refused string `json:"refused,omitempty"`
|
||||||
|
Error string `json:"error,omitempty"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// SetLineTest wires the weekly line test (main.go builds it after the local API server).
|
// SetLineTest wires the weekly line test (main.go builds it after the local API server).
|
||||||
@@ -27,11 +51,39 @@ func (s *Server) handleLineTest(w http.ResponseWriter, r *http.Request, vmid int
|
|||||||
writeOK(w, LineTestResponse{Present: false})
|
writeOK(w, LineTestResponse{Present: false})
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
var waiting bool
|
||||||
|
if q, ok := s.lineTest.(LineTestRequester); ok {
|
||||||
|
waiting = q.WaitingForQuiet()
|
||||||
|
}
|
||||||
mbps, at, ok := s.lineTest.Value()
|
mbps, at, ok := s.lineTest.Value()
|
||||||
if !ok {
|
if !ok {
|
||||||
writeOK(w, LineTestResponse{Present: false})
|
writeOK(w, LineTestResponse{Present: false, WaitingForQuiet: waiting})
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
s.logger.Debug("local-api: /host/line-test served", "vmid", vmid, "mbps", mbps, "measured_at", at.UTC().Format(time.RFC3339))
|
s.logger.Debug("local-api: /host/line-test served", "vmid", vmid, "mbps", mbps, "measured_at", at.UTC().Format(time.RFC3339), "waiting_for_quiet", waiting)
|
||||||
writeOK(w, LineTestResponse{Present: true, Mbps: mbps, MeasuredAt: at.UTC().Format(time.RFC3339), Method: "iperf3-tunnel"})
|
writeOK(w, LineTestResponse{Present: true, Mbps: mbps, MeasuredAt: at.UTC().Format(time.RFC3339), Method: "iperf3-tunnel", WaitingForQuiet: waiting})
|
||||||
|
}
|
||||||
|
|
||||||
|
// handleLineTestRun is POST /host/line-test/run (R-1250): a test on request, synchronous (about 55 s). The answer is
|
||||||
|
// always 200 with the outcome in the body, except 503 when no line test is wired.
|
||||||
|
func (s *Server) handleLineTestRun(w http.ResponseWriter, r *http.Request, vmid int) {
|
||||||
|
q, ok := s.lineTest.(LineTestRequester)
|
||||||
|
if s.lineTest == nil || !ok {
|
||||||
|
writeErr(w, http.StatusServiceUnavailable, "line test not configured")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
s.logger.Info("local-api: line test requested", "vmid", vmid)
|
||||||
|
mbps, runs, err := q.RunNow(r.Context())
|
||||||
|
var ref *linetest.Refusal
|
||||||
|
switch {
|
||||||
|
case err == nil:
|
||||||
|
writeOK(w, LineTestRunResponse{OK: true, Mbps: mbps, RunsMbps: runs})
|
||||||
|
case errors.As(err, &ref):
|
||||||
|
s.logger.Info("local-api: line test request refused", "vmid", vmid, "reason", ref.Code, "detail", ref.Msg)
|
||||||
|
writeOK(w, LineTestRunResponse{Refused: ref.Code, Error: ref.Msg})
|
||||||
|
case errors.Is(err, linetest.ErrBusy):
|
||||||
|
writeOK(w, LineTestRunResponse{Refused: "server-busy", Error: err.Error()})
|
||||||
|
default:
|
||||||
|
writeOK(w, LineTestRunResponse{Error: err.Error()})
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,12 +1,16 @@
|
|||||||
package localapi
|
package localapi
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"context"
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
"log/slog"
|
"log/slog"
|
||||||
"net/http/httptest"
|
"net/http/httptest"
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
"gitea.dooplex.hu/admin/felhom-agent/internal/linetest"
|
||||||
)
|
)
|
||||||
|
|
||||||
type fixedLine struct {
|
type fixedLine struct {
|
||||||
@@ -49,3 +53,64 @@ func TestLineTestRoute(t *testing.T) {
|
|||||||
t.Fatalf("measured: %+v", got)
|
t.Fatalf("measured: %+v", got)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
type requestableLine struct {
|
||||||
|
fixedLine
|
||||||
|
waiting bool
|
||||||
|
mbps float64
|
||||||
|
err error
|
||||||
|
calls int
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *requestableLine) WaitingForQuiet() bool { return r.waiting }
|
||||||
|
func (r *requestableLine) RunNow(context.Context) (float64, []float64, error) {
|
||||||
|
r.calls++
|
||||||
|
if r.err != nil {
|
||||||
|
return 0, nil, r.err
|
||||||
|
}
|
||||||
|
return r.mbps, []float64{r.mbps - 1, r.mbps}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func lineTestRun(t *testing.T, s *Server) (int, LineTestRunResponse) {
|
||||||
|
t.Helper()
|
||||||
|
rec := httptest.NewRecorder()
|
||||||
|
s.handleLineTestRun(rec, httptest.NewRequest("POST", "/host/line-test/run", nil), 9201)
|
||||||
|
var env struct {
|
||||||
|
OK bool `json:"ok"`
|
||||||
|
Data LineTestRunResponse `json:"data"`
|
||||||
|
}
|
||||||
|
_ = json.Unmarshal(rec.Body.Bytes(), &env)
|
||||||
|
return rec.Code, env.Data
|
||||||
|
}
|
||||||
|
|
||||||
|
// R-1250: the controller acts only on waiting_for_quiet, and reads a refusal by its code — never as a measurement.
|
||||||
|
func TestLineTestRunRoute(t *testing.T) {
|
||||||
|
s := &Server{logger: slog.Default()}
|
||||||
|
if code, _ := lineTestRun(t, s); code != 503 {
|
||||||
|
t.Fatalf("not wired: HTTP %d, want 503", code)
|
||||||
|
}
|
||||||
|
s.SetLineTest(fixedLine{ok: true, mbps: 90})
|
||||||
|
if code, _ := lineTestRun(t, s); code != 503 {
|
||||||
|
t.Fatalf("a reader without the request half: HTTP %d, want 503", code)
|
||||||
|
}
|
||||||
|
l := &requestableLine{fixedLine: fixedLine{ok: false}, waiting: true, mbps: 93.5}
|
||||||
|
s.SetLineTest(l)
|
||||||
|
if got := lineTestBody(t, s); got.Present || !got.WaitingForQuiet {
|
||||||
|
t.Fatalf("GET while waiting: %+v, want present:false waiting_for_quiet:true", got)
|
||||||
|
}
|
||||||
|
if code, got := lineTestRun(t, s); code != 200 || !got.OK || got.Mbps != 93.5 || len(got.RunsMbps) != 2 {
|
||||||
|
t.Fatalf("a measured request: HTTP %d %+v", code, got)
|
||||||
|
}
|
||||||
|
l.err = &linetest.Refusal{Code: "not-quiet", Msg: "the uplink sent 16.0 Mbit/s while held"}
|
||||||
|
if _, got := lineTestRun(t, s); got.OK || got.Refused != "not-quiet" || got.Mbps != 0 {
|
||||||
|
t.Fatalf("a refusal: %+v, want ok:false refused:not-quiet", got)
|
||||||
|
}
|
||||||
|
l.err = linetest.ErrBusy
|
||||||
|
if _, got := lineTestRun(t, s); got.OK || got.Refused != "server-busy" {
|
||||||
|
t.Fatalf("server busy: %+v, want refused:server-busy", got)
|
||||||
|
}
|
||||||
|
l.err = errors.New("dial tcp 10.77.0.1:5201: i/o timeout")
|
||||||
|
if _, got := lineTestRun(t, s); got.OK || got.Refused != "" || got.Error == "" {
|
||||||
|
t.Fatalf("a failed run: %+v, want ok:false, no refusal, the error", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -577,6 +577,7 @@ func (s *Server) Handler() http.Handler {
|
|||||||
// R-1247: the weekly upload line test's last measurement — the controller's qBittorrent limit and upload watch
|
// R-1247: the weekly upload line test's last measurement — the controller's qBittorrent limit and upload watch
|
||||||
// read it. Read-only; never measured (or the job not wired) = present:false.
|
// read it. Read-only; never measured (or the job not wired) = present:false.
|
||||||
mux.HandleFunc("GET /host/line-test", s.withGuest(s.handleLineTest))
|
mux.HandleFunc("GET /host/line-test", s.withGuest(s.handleLineTest))
|
||||||
|
mux.HandleFunc("POST /host/line-test/run", s.withGuest(s.handleLineTestRun)) // R-1250: a test on request
|
||||||
// Disk management (slice 8C) — self-scoped; format routes through the data-bearing classifier+gate.
|
// Disk management (slice 8C) — self-scoped; format routes through the data-bearing classifier+gate.
|
||||||
mux.HandleFunc("GET /disks", s.withGuest(s.handleDisks))
|
mux.HandleFunc("GET /disks", s.withGuest(s.handleDisks))
|
||||||
mux.HandleFunc("GET /disks/candidates", s.withGuest(s.handleDiskCandidates))
|
mux.HandleFunc("GET /disks/candidates", s.withGuest(s.handleDiskCandidates))
|
||||||
|
|||||||
Reference in New Issue
Block a user