Add gitea-image-prune.sh: inspect/prune Gitea container images + reclaim disk

New operator CLI (curl+jq, dry-run default) to list, prune (keep-N or
older-than), and reclaim old container images in the self-hosted Gitea
registry. Reclaim implements the three-step mechanism proven live on
Gitea 1.26.2: delete tag (frees only the index pointer) -> delete the
orphaned sha256 manifest versions (default cleanup_packages does NOT
remove untagged manifests) -> cleanup_packages cron GCs the now
unreferenced blobs. Orders by upload date, protects ^latest$, fail-closed
orphan detection, audit log, never logs the token.

Live-verified: single-version spike freed 5.1 MiB; cleaning felhom-hub's
16 orphan manifests freed 86 MiB; surviving tags still docker-pull.
felhom-controller and other packages left untouched for the operator.

Adds README section (usage, minimal token scopes, reclaim caveat, native
cleanup-rule recommendation), CHANGELOG, REPORT, and .gitattributes (LF).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-17 09:16:07 +02:00
parent 3db14bf5f9
commit 761dc3856e
5 changed files with 974 additions and 0 deletions
+134
View File
@@ -0,0 +1,134 @@
# misc-scripts
Operator helper scripts for the Felhom / DooPlex infrastructure. These are
stand-alone CLI utilities (English output — operator-facing, not customer UI).
| Script | Purpose |
|---|---|
| `build-felhom-hub.sh` | Build & push the `felhom-hub` Docker image to the Gitea registry. |
| `build-felhom-controller.sh` | Build & push the `felhom-controller` image. |
| `collect-repos.sh` | Concatenate repo sources into text files (for review/archival). |
| `gitea-image-prune.sh` | Inspect & prune old container images in the Gitea registry, then reclaim disk. |
---
## `gitea-image-prune.sh`
`build-felhom-{hub,controller}.sh` push `:<version>` **and** `:latest` on every
build, so the container packages accumulate one image per build and the Gitea
Longhorn PVC fills up. This tool lists, prunes, and reclaims that space, with a
safe **dry-run default**.
Best run on the **build server (192.168.0.180)** — it has `kubectl` for the
optional disk measurement and network to Gitea. The core (list / prune /
reclaim) is pure `curl` + `jq` and runs from any host with a token.
### How Gitea stores container images (why reclaim takes three steps)
A pushed **tag** is an OCI image *index* — a ~850 B pointer. The real bytes live
in untagged **`sha256:` manifest versions** (config + layer blobs, ~925 MB
each), whose layer blobs are content-addressed and **shared across tags**.
Confirmed live on Gitea 1.26.2 (2026-06-17), the reclaim path is **three steps**:
1. **Delete the tag** → frees ~nothing (only the tiny index pointer).
2. **Delete the now-orphaned `sha256:` manifest versions** (referenced by no
surviving tag). Default `cleanup_packages` does **not** remove untagged
manifests — only a configured *cleanup rule* would — so the tool deletes them
itself. This makes their *unique* blobs unreferenced.
3. **`cleanup_packages` cron runs** → garbage-collects unreferenced blobs created
more than `OLDER_THAN` (24 h default) ago → **this is what frees disk**. Layer
blobs still shared with surviving tags are correctly retained.
`prune` does step 1; `reclaim` does steps 2 + 3.
> **Live proof:** deleting one 9.4 MB-apparent tag + its 2 manifests + GC freed
> 5.1 MiB (the rest was shared base layers, correctly kept). Cleaning 16
> accumulated orphan manifests on `felhom-hub` then freed **86 MiB**. After
> reclaim, surviving tags (`latest`, `0.1.3`, …) still `docker pull` cleanly.
### Token & required scopes
Pass an admin-user token via `GITEA_TOKEN` (env) or `--token-file <path>`. The
token must belong to a Gitea **site-admin** user. Minimal fine-grained scopes
(empirically confirmed against 1.26.2 via the API's 403 bodies):
| Operation | Scope |
|---|---|
| list packages / versions / files | `read:package` |
| delete a tag / manifest version | `write:package` |
| list cron tasks | `read:admin` |
| **trigger** the `cleanup_packages` GC cron | `write:admin` |
The token is **never** echoed, logged, or committed. If the token lacks
`write:admin`, the tool still deletes orphaned manifests and reports that their
blobs will be freed by the daily `@midnight` `cleanup_packages` run (or on the
next Gitea restart — see "native retention" below).
> The project's read-only token (and the build server's `~/.gitea-token`, which
> has `read:admin` + `write:package` but **not** `write:admin`) can list, prune,
> and delete orphans, but cannot trigger the GC cron on demand.
### Usage
```bash
# Interactive menu (pick packages, then an action):
GITEA_TOKEN=… ./gitea-image-prune.sh
# List one / all packages (with apparent per-tag sizes):
./gitea-image-prune.sh --repo felhom-hub list
./gitea-image-prune.sh --all list
./gitea-image-prune.sh --all --no-sizes list # fast, skip size resolution
# Dry-run prune (DEFAULT — shows what would go, mutates nothing):
./gitea-image-prune.sh --repo felhom-hub --keep 10
./gitea-image-prune.sh --repo felhom-controller --older-than 90
# Apply for real (typed confirmation unless --yes):
./gitea-image-prune.sh --repo felhom-hub --keep 10 --apply
./gitea-image-prune.sh --repo felhom-hub --keep 10 --apply --reclaim --measure
# Reclaim only (delete orphaned manifests + trigger/await GC):
./gitea-image-prune.sh --repo felhom-hub reclaim --apply
# Cron-friendly, non-interactive, all packages:
./gitea-image-prune.sh --all --keep 15 --apply --yes --reclaim
```
### Flags
| Flag | Meaning |
|---|---|
| `--repo NAME` (repeatable) / `--all` | Which packages to act on. |
| `list` / `prune` / `reclaim` | Action (positional; inferred as `prune` if `--keep`/`--older-than` given, else `list`). |
| `--keep N` | Keep the N most-recent tags; delete older. |
| `--older-than DAYS` | Keep tags newer than DAYS; delete older. (Mutually exclusive with `--keep`.) |
| `--dry-run` (default) / `--apply` | Mutate only with `--apply`. |
| `--yes` | Skip confirmation prompts (for cron). |
| `--reclaim` | After an `--apply` prune, also run the reclaim stage. |
| `--measure` | Best-effort before/after `du` of the packages dir (needs `kubectl`; skipped cleanly otherwise). |
| `--protect REGEX` (repeatable) | Never delete matching tags. `^latest$` is always protected. |
| `--no-sizes` | Skip per-tag size resolution (much faster lists). |
| `--token-file PATH` | Read the token from a file instead of `GITEA_TOKEN`. |
| `--owner NAME` | Package owner (default `admin`). |
| `--log PATH` | Audit log (default `misc-scripts/logs/gitea-prune-<date>.log`). |
Safety: dry-run is the default; `latest` (and `--protect` matches) are never
deleted even if "old"; tags are ordered by **upload date**, never by parsing the
tag (the registry mixes `v`-prefixed and bare tags); every HTTP status is checked
(no silent failures); orphan detection is **fail-closed** (if any surviving tag
can't be resolved, that package's orphan cleanup is skipped rather than risk
deleting a referenced manifest).
### Native retention (set-and-forget complement)
This script is the **on-demand** tool. The **standing** complement is a Gitea
per-owner **cleanup rule** (web UI: *Packages → owner → Settings → Cleanup
Rules*): set *Remove all versions except the most recent N*, and add a tag-match
exclusion for `latest`. The daily `cleanup_packages` cron then enforces it
automatically. This tool does **not** auto-create rules (deliberately) — configure
them once in the UI if you want hands-off retention.
> Note: on this instance, `[cron.cleanup_packages] RUN_AT_START = true` was added
> to `app.ini` (2026-06-17) so the GC also runs on every Gitea restart — useful
> when reclaiming without a `write:admin` token.