admin/misc-scripts
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

gitea-image-prune.sh: keep the newest 20 and every version in use; refuse when the in-use list is unreadable (Felhom 09 decision 62, R-750)

Browse Source 
Protected whatever --keep says: the controller floor, the vouched golden, the vouched agent and min_agent (the hub's
Configuration page), every image of ours the vouched golden baked (its bake.log), the hub manifest's image. The dry-run
prints each kept version and why. tests/test-prune-plan.sh pins it without network (red-proofed). No --apply was run.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0159rPz1ZhFKsS53msqPYxtS
This commit is contained in:
authentik Default Admin
2026-10-01 13:05:31 +02:00
parent 7739c830cb
commit c9d5ed575c
5 changed files with 209 additions and 133 deletions
Download Patch File Download Diff File Expand all files Collapse all files
REPORT.md
+10 -106
View File
@@ -1,108 +1,12 @@
# REPORT — `gitea-image-prune.sh` (2026-06-17)
# REPORT — gitea-image-prune.sh gets the retention rule (2026-10-01)
New operator CLI to inspect/prune old container images in the Gitea registry and
reclaim disk, with a load-bearing live spike to **prove** the reclaim mechanism.
Felhom `09` §3 decision 62 (operator ruling, R-750). Full report: `felhom.eu/REPORT-calibre-name-and-prune-2026-10-01.md`.
## Confirmed baselines (live)
| Thing | Value |
|---|---|
| Gitea version | **1.26.2** (`GET /api/v1/version`) |
| Owner namespace | `admin` (standard per-owner packages API; "admin" in the path is the owner) |
| Container packages | `felhom-controller` **96 tags / 247 digests**, `felhom-hub` **42 tags / 96 digests**, plus `recipe-importer` (42), `revfulop-calendar` (12), `jarr` (2), `wan-probe` (1) |
| Packages cron | `cleanup_packages`, default `@midnight`, `OLDER_THAN = 24h` |
| Packages dir | `/data/gitea/packages` (gitea-system pod, container `gitea`), **baseline 5,122,143,723 B ≈ 4.77 GiB** |
| Tooling | `jq` 1.7 present on build server; `shellcheck` **absent** (not run — script written carefully, `bash -n` clean) |
## Minimal token scope set (empirically confirmed via 403 bodies)
| Operation | Required scope |
|---|---|
| list packages / versions / files | `read:package` |
| delete a tag / manifest version | `write:package` |
| list cron tasks | `read:admin` |
| trigger `cleanup_packages` cron | **`write:admin`** |
The build server token (`~/.gitea-token`) has `read:admin` + `write:package` but
**not** `write:admin` (its 403 body named exactly `required=[write:admin]`), so it
can list/prune/delete-orphans but cannot trigger the GC cron on demand. Token must
belong to a site-admin user. (Scopes were *not* minimized by minting reduced
tokens — that needs `write:user`, which this token also lacks — but each required
scope was confirmed by a successful call and the cron requirement by its 403.)
## §3 spike — the reclaim mechanism (PROVEN, not assumed)
Gitea stores a tag as a tiny OCI **index** pointer; the real bytes are in untagged
`sha256:` **manifest versions** (config + layer blobs), whose layers are shared
across tags. The mechanism turned out to be **three steps**, not two:
| Step (single-version spike, `felhom-hub`) | `du` (bytes) | freed |
|---|---|---|
| baseline | 5,122,143,723 | — |
| DELETE tag `0.1.1` (via the script, HTTP 204) | 5,122,143,723 | **0** |
| run `cleanup_packages` (tag-only) | 5,122,138,771 | ~5 KB (index pointer only) |
| DELETE tag `0.1.2` + its **2 orphaned manifests** (204×3) | 5,122,138,771 | **0** |
| run `cleanup_packages` GC | 5,116,799,814 | **5,338,957 B ≈ 5.1 MiB** |
**Conclusions:**
1. Deleting a tag frees ~nothing (only the index pointer).
2. **Default `cleanup_packages` does NOT remove untagged manifest versions** —
only unreferenced *blobs*. So the orphaned `sha256:` manifests must be deleted
explicitly (the script's reclaim does this); otherwise their blobs stay
referenced forever. (Confirmed: a tag-only delete + cron left `felhom-hub`
digests at 96.)
3. Once the orphaned manifests are deleted, `cleanup_packages` GCs their *unique*
blobs (created > `OLDER_THAN`); shared base layers stay. 5.1 MiB freed for one
9.4 MB-apparent image — the difference is shared layers, correctly retained.
Because the token lacks `write:admin`, the GC cron was triggered by adding
`[cron.cleanup_packages] RUN_AT_START = true` to `app.ini` (on the data PVC,
backup `app.ini.bak.prune-spike`) and rolling-restarting Gitea — left in place per
operator request (the daily `@midnight` run also performs the GC).
## §9 verification results
1. **List (read-only):** `--repo felhom-hub list` → 42 tags, newest-first, per-tag
sizes resolved via OCI (24 MB recent, ~9 MB older), `latest` flagged PROTECTED,
shared-layer caveat printed. `--all` lists all 6 packages; `--no-sizes` fast path
works. No mutation.
2. **Dry-run prune:** `--repo felhom-hub --keep 5 --dry-run` → would delete 36,
keep 5 + 1 protected, oldest first, totals shown, **nothing changed**.
3. **One-version live proof (spike):** see table above — delete-alone = 0 bytes;
delete + orphan-manifest delete + GC = 5.1 MiB.
4. **Full reclaim path validated** on `felhom-hub` only: `reclaim --apply` deleted
the **16 accumulated orphan manifests** (untagged, referenced by no tag — dead
weight from re-pointed `latest` + buildx attestations), then GC freed
**5,116,799,814 → 5,026,431,222 = 90,368,592 B ≈ 86 MiB**.
5. **Safety:** after reclaim, surviving tags still resolve and **`docker pull`
cleanly** (`latest`, `0.1.3` — the immediate neighbor of the deleted tags).
Orphan detection is fail-closed (skips a package if any surviving tag won't
resolve).
6. **Audit log** captured every RUN / DRY-RUN / APPLIED / RECLAIM line; **token
scan of the log = 0 hits**. Edge cases: `--keep`+`--older-than` → error;
`--keep 999` → "Nothing to prune (40 tags: 39 kept, 1 protected)".
## State left on the registry
- `felhom-hub`: tags `0.1.1` and `0.1.2` deleted (spike); 16 orphan manifests
cleaned; now **40 tags / 78 digests**; ~91 MiB reclaimed total. All remaining
tags pull cleanly.
- **`felhom-controller` (96 tags) and all other packages: UNTOUCHED.**
- `app.ini`: `RUN_AT_START = true` added for `cleanup_packages` (kept).
## NOT yet run
**The real bulk cleanup — left to the operator (interactive).** This run proved
the mechanism on one disposable version and validated the full reclaim path on
`felhom-hub`'s dead orphans only. Pruning the ~90 `felhom-controller` tags (and
the bulk of `felhom-hub`/`recipe-importer` history) is the operator's call via
`gitea-image-prune.sh --repo … --keep N --apply --reclaim`.
## Backlog / notes
- A `write:admin` token (or the native cleanup rule in the UI) would let `reclaim`
trigger the GC immediately instead of relying on the `@midnight`/restart run.
- Per-tag "apparent" sizes overlap (shared base layers counted once per tag);
`--measure` (`du`) is the honest real-reclaim signal. Documented in the tool.
- `shellcheck` was unavailable on the build server, so the script was not
statically linted (only `bash -n` syntax-checked + extensively run live).
- **Rule:** keep the newest 20 + every version in use (floor, vouched golden, vouched agent, `min_agent`, the golden's
baked images, the running hub); refuse (exit 3) when the in-use list cannot be read; `--apply` only by a person.
- **Test:** `tests/test-prune-plan.sh` — 7 checks pass; red-proof: the in-use check removed from `is_protected` → 3 fail
(the plan deletes 5 and lists 0.262.0).
- **Live dry-run, 2026-10-01 (nothing deleted):** in use — controller 0.285.0, golden 0.285.0, agent 0.138.0 and 0.131.0,
hub 0.126.0, felhom-samba 1.1.0. Would delete: felhom-controller 70 tags, felhom-hub 8; every other package nothing.
Evidence: `felhom.eu/documentation/audits/calibre-name-and-prune-2026-10-01/B/`.
- **No `--apply` was run.**