admin
52b6599d97
gates / gates (push) Successful in 6m36s
Read-only. fetch.py (polite, 5 s apart, 34 pages, all 200), parse.py (own text only; four parser defects found by the controls and fixed), build.py + spotcheck.py (stdlib). tags.csv holds post numbers and tags only - no username, author id, salt or post text in git. FINDINGS.md (2,908 posts, 195 people, 82 beginner threads in 7 clusters), FELHOM.md (catalog gap, hardware fit, 15 website questions, objections, ten ranked recommendations), METHOD.md (controls, spot-check: role and hardware below 85 % after two rounds, the table-driving splits 92.5 % and 97.3 %), REPORT.md. R-990: operator rules on the recommendations. R-991: no buyer-readable hardware floor. Claude-Session: https://claude.ai/code/session_012qRErfCoiTkvDK9N5XHbzb
86 lines
6.1 KiB
Markdown
86 lines
6.1 KiB
Markdown
# METHOD — forum-research-2026-10-12
|
||
|
||
| Item | Value |
|
||
|---|---|
|
||
| Source | `https://prohardver.hu/tema/hazi_szerver_epitese/`, public, no login, no account |
|
||
| Window | posts dated 2023-10-12 00:00 or later: **#7943 (2023-10-12 17:01) → #11092 (2026-10-12 17:56)** |
|
||
| Fetch | 2026-10-12 21:53–21:56 CEST; `fetch.py`; one request at a time, 5 s apart (robots.txt asks 1 s); browser UA; cookie jar; 34 pages (hsz_7801-7900 … hsz_11101-11200); every answer 200; no block, captcha or login wall |
|
||
| robots.txt | read 2026-10-12 before fetching: `User-agent: *`, `crawl-delay: 1`, disallows only `/*/keres.php?*offset=` — unchanged from the spec |
|
||
| Cache | `~/research-cache/prohardver-hazi-szerver/` on DooPlex, outside every repo: raw pages, `posts.jsonl`, salt, tags, spot files. **Not in git.** |
|
||
| Posts | 3,150 numbers in the window; 2,909 present; 241 missing (#9874–#10108 the 2025 outage, #10118–10120, #8571, #9230, #11073); #11091 (Felhom's own announcement) excluded → **2,908 analysed** |
|
||
| People | 195 distinct writers (`author_id` = first 12 hex of sha256(salt + username); salt local only) |
|
||
|
||
## Parser controls (`parse.py`, all PASS on the final run)
|
||
|
||
1. **Counts:** every full page yields 100 posts, numbers contiguous; hsz_7801 99, hsz_8501 99, hsz_9201
|
||
99 (one post each removed by the site), hsz_9801 73 and hsz_10101 89 (the outage edges), two pages 0,
|
||
hsz_11001 91 (newest). The last request (11101-11200) redirected to 11001-11100; duplicates kept once.
|
||
2. **Five random posts by eye** (seed 20261012): #9610, #7975, #8149, #7948, #8119 — own text whole,
|
||
quoted parent absent in all five, checked against the raw page.
|
||
3. **#8099** yields its own Proxmox 7.x complaint; the #8096 Raspberry Pi 4 → i5-6500T text is absent;
|
||
and #8096 itself carries that text (the control can fail).
|
||
|
||
**Four parser defects found by these controls and fixed before tagging:** posts with no parent
|
||
(`data-rplid=""`) were dropped (85 → 100 per page); edited posts (pencil icon inside `<time>`) lost
|
||
their date; this year's dates have no year („aug. 9.”, „ma”, „tegnap”) and were normalised against the
|
||
fetch day; **hand-pasted quotes** — users copy the parent's words into their own text, often in
|
||
`<small><b><i>` — were removed line by line and sentence by sentence when ≥ 20/25 characters match the
|
||
parent (142 removed; quote-leak sweep 17 → 0). A pasted fragment shorter than that, or with no space
|
||
after its full stop, can survive (seen once, #8173).
|
||
|
||
## Tagging
|
||
|
||
Schema: `TAGGING-BRIEF.md` (v1 for all columns; v2 for `role`, `hardware`, `hardware_status`).
|
||
Nine helper agents, one block of whole pages each, no overlap, the same brief and the full fence list.
|
||
Columns added beyond the spec: `platform_status` (needed for table 5), `turnkey` (table 5e), and
|
||
`thread` in `tags.csv`. A `gist` (≤ 12 own English words) was used for clustering and stays in the cache.
|
||
|
||
**Beginner** = a person with at least one post whose role is `beginner-asks`: says „kezdő”, no Linux,
|
||
lost in commands, a first server, a basic question, or is told so (`beginner_evidence`).
|
||
**Threads:** walk up `reply_to` to the first ask; follow-ups by the same asker join; a lost-link
|
||
follow-up with no question type joins the asker's previous thread within 30 days. 308 threads, 82 beginner.
|
||
**Clusters:** the 82 beginner threads were grouped by reading their summaries (`clusters.json`, cache).
|
||
|
||
**Pain tags used (47):** power-bill · noise · cost-upfront · too-complex · space · hardware-failure · heat ·
|
||
time-cost · security-fear · data-loss-fear · family-acceptance · update-broke-it · vendor-lock-in ·
|
||
remote-access-confusion · bad-docs · disk-failure · gave-up · fears-terminal · cloud-distrust · cloud-cost ·
|
||
install-error · data-loss-happened · fears-docker · usb-enclosure-distrust · subscription-cost · dust ·
|
||
isp-cgnat · update-fear · distrusts-chinese-hardware · stale-plugins · device-sprawl · choice-overload ·
|
||
vpn-hassle · wants-gui · avoids-containers · lost-in-commands · dynamic-ip-outage · dislikes-linux ·
|
||
weak-hardware · bad-firmware · underpowered-hardware · raid-capacity-loss · install-failed ·
|
||
wants-hungarian-guide · update-burden · end-of-support · power-outage.
|
||
|
||
## Spot-check (spec 9.6) — `spotcheck.py`
|
||
|
||
The session re-tagged a random 146 posts (5.0 %, seed 77) itself, blind to the helper tags.
|
||
|
||
| Column | Round 1 | Round 2 (v2 rules) | Note |
|
||
|---|---|---|---|
|
||
| role | 83.6 % | 80.8 % | **below 85 % twice** |
|
||
| ↳ asks vs not asks | — | 92.5 % | the split every thread and cluster rests on |
|
||
| ↳ beginner-asks vs not | — | 97.3 % | the split every „beginner” table rests on |
|
||
| ↳ substantive vs other | — | 87.7 % | |
|
||
| beginner (y/n) | 99.3 % | — | |
|
||
| question_type | 95.2 % | — | |
|
||
| hardware | 79.5 % | 81.5 % (class set 82.9 %; ≥ 1 class shared when both tagged: 87.1 %) | **below 85 % twice** |
|
||
| hardware_status | 87.7 % | 83.6 % | |
|
||
| apps · apps_status | 92.5 % · 96.6 % | — | |
|
||
| platform · platform_status | 89.0 % · 90.4 % | — | |
|
||
| pain | 85.6 % | — | 30 % where either side tagged one — pains are leads |
|
||
| budget_huf · resolved | 97.3 % · 97.9 % | — | |
|
||
|
||
Round 1 also caught four of the session's own spot rows with a shifted field (fixed before scoring).
|
||
**Not followed: a third re-tag round.** Round 2's rules moved the disagreement rather than removed it —
|
||
it sits on „other vs answers” for chat posts and on whether a machine mentioned in passing counts.
|
||
Neither boundary changes a table: the decisions the tables use pass (92.5 %, 97.3 %), and hardware
|
||
counts carry an „approximate” banner in `FINDINGS.md`.
|
||
|
||
## Limits
|
||
|
||
- A **hobby forum**: self-selected and technical; a household buyer does not post here.
|
||
- **Hungarian only**; one topic; 195 people. Counts under ~5 people are anecdotes.
|
||
- 2023 covers Q4 only; 2026 runs to 12 October. The 2025 outage removed 235 posts (between 2025-04-28 and 2025-07-12).
|
||
- `resolved` is mostly unknown — askers seldom report back.
|
||
- Tags are an AI reading of short posts; every claim cites post numbers so a person can check it.
|
||
- Privacy: no username, `author_id`, salt or post text is in git; the cache stays on DooPlex.
|