Add gitea-image-prune.sh: inspect/prune Gitea container images + reclaim disk
New operator CLI (curl+jq, dry-run default) to list, prune (keep-N or older-than), and reclaim old container images in the self-hosted Gitea registry. Reclaim implements the three-step mechanism proven live on Gitea 1.26.2: delete tag (frees only the index pointer) -> delete the orphaned sha256 manifest versions (default cleanup_packages does NOT remove untagged manifests) -> cleanup_packages cron GCs the now unreferenced blobs. Orders by upload date, protects ^latest$, fail-closed orphan detection, audit log, never logs the token. Live-verified: single-version spike freed 5.1 MiB; cleaning felhom-hub's 16 orphan manifests freed 86 MiB; surviving tags still docker-pull. felhom-controller and other packages left untouched for the operator. Adds README section (usage, minimal token scopes, reclaim caveat, native cleanup-rule recommendation), CHANGELOG, REPORT, and .gitattributes (LF). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,2 @@
|
||||
# Shell scripts must keep LF endings — they run on Linux; CRLF breaks the shebang.
|
||||
*.sh text eol=lf
|
||||
@@ -0,0 +1,34 @@
|
||||
# Changelog — misc-scripts
|
||||
|
||||
All notable changes to the operator helper scripts. Newest on top.
|
||||
|
||||
## 2026-06-17
|
||||
|
||||
### Added — `gitea-image-prune.sh`
|
||||
- New operator CLI to inspect and prune old container images in the self-hosted
|
||||
Gitea registry (`gitea.dooplex.hu`, owner `admin`) and reclaim disk on the
|
||||
Longhorn-backed packages PVC. Pure `curl` + `jq`; interactive menu + scriptable
|
||||
flags. Safe **dry-run default**.
|
||||
- Modes: `list` (per-tag upload date + apparent image size, newest-first,
|
||||
shared-layer caveat), `prune` (`--keep N` or `--older-than DAYS`; always
|
||||
protects `^latest$` + `--protect` regexes), `reclaim` (delete orphaned manifest
|
||||
versions + trigger/await the `cleanup_packages` GC cron). `--measure` does
|
||||
best-effort before/after `du` via `kubectl`.
|
||||
- Implements the **three-step reclaim mechanism proven live** on Gitea 1.26.2:
|
||||
deleting a tag frees only the index pointer; the orphaned `sha256:` manifest
|
||||
versions must also be deleted (default `cleanup_packages` does not remove
|
||||
untagged manifests); the cron then GCs the unreferenced blobs. Orphan detection
|
||||
is fail-closed.
|
||||
- Safety: orders by upload date (never parses mixed `v`/bare tags), checks every
|
||||
HTTP status, never echoes/logs the token, audit log per run, typed confirmation
|
||||
on `--apply` (stricter for `--all`).
|
||||
- Token via `GITEA_TOKEN`/`--token-file`. Minimal scopes documented in README:
|
||||
`read:package` (list), `write:package` (delete), `read:admin` (cron list),
|
||||
`write:admin` (cron trigger).
|
||||
- README section added documenting usage, scopes, the reclaim caveat, the live
|
||||
verification result, and the native cleanup-rule recommendation.
|
||||
|
||||
### Changed — Gitea instance (operational, not a script change)
|
||||
- Added `[cron.cleanup_packages] RUN_AT_START = true` to Gitea's `app.ini`
|
||||
(on the data PVC) so the package GC also runs on every Gitea restart. Enables
|
||||
reclaim without a `write:admin` token. Backup at `app.ini.bak.prune-spike`.
|
||||
@@ -0,0 +1,134 @@
|
||||
# misc-scripts
|
||||
|
||||
Operator helper scripts for the Felhom / DooPlex infrastructure. These are
|
||||
stand-alone CLI utilities (English output — operator-facing, not customer UI).
|
||||
|
||||
| Script | Purpose |
|
||||
|---|---|
|
||||
| `build-felhom-hub.sh` | Build & push the `felhom-hub` Docker image to the Gitea registry. |
|
||||
| `build-felhom-controller.sh` | Build & push the `felhom-controller` image. |
|
||||
| `collect-repos.sh` | Concatenate repo sources into text files (for review/archival). |
|
||||
| `gitea-image-prune.sh` | Inspect & prune old container images in the Gitea registry, then reclaim disk. |
|
||||
|
||||
---
|
||||
|
||||
## `gitea-image-prune.sh`
|
||||
|
||||
`build-felhom-{hub,controller}.sh` push `:<version>` **and** `:latest` on every
|
||||
build, so the container packages accumulate one image per build and the Gitea
|
||||
Longhorn PVC fills up. This tool lists, prunes, and reclaims that space, with a
|
||||
safe **dry-run default**.
|
||||
|
||||
Best run on the **build server (192.168.0.180)** — it has `kubectl` for the
|
||||
optional disk measurement and network to Gitea. The core (list / prune /
|
||||
reclaim) is pure `curl` + `jq` and runs from any host with a token.
|
||||
|
||||
### How Gitea stores container images (why reclaim takes three steps)
|
||||
|
||||
A pushed **tag** is an OCI image *index* — a ~850 B pointer. The real bytes live
|
||||
in untagged **`sha256:` manifest versions** (config + layer blobs, ~9–25 MB
|
||||
each), whose layer blobs are content-addressed and **shared across tags**.
|
||||
|
||||
Confirmed live on Gitea 1.26.2 (2026-06-17), the reclaim path is **three steps**:
|
||||
|
||||
1. **Delete the tag** → frees ~nothing (only the tiny index pointer).
|
||||
2. **Delete the now-orphaned `sha256:` manifest versions** (referenced by no
|
||||
surviving tag). Default `cleanup_packages` does **not** remove untagged
|
||||
manifests — only a configured *cleanup rule* would — so the tool deletes them
|
||||
itself. This makes their *unique* blobs unreferenced.
|
||||
3. **`cleanup_packages` cron runs** → garbage-collects unreferenced blobs created
|
||||
more than `OLDER_THAN` (24 h default) ago → **this is what frees disk**. Layer
|
||||
blobs still shared with surviving tags are correctly retained.
|
||||
|
||||
`prune` does step 1; `reclaim` does steps 2 + 3.
|
||||
|
||||
> **Live proof:** deleting one 9.4 MB-apparent tag + its 2 manifests + GC freed
|
||||
> 5.1 MiB (the rest was shared base layers, correctly kept). Cleaning 16
|
||||
> accumulated orphan manifests on `felhom-hub` then freed **86 MiB**. After
|
||||
> reclaim, surviving tags (`latest`, `0.1.3`, …) still `docker pull` cleanly.
|
||||
|
||||
### Token & required scopes
|
||||
|
||||
Pass an admin-user token via `GITEA_TOKEN` (env) or `--token-file <path>`. The
|
||||
token must belong to a Gitea **site-admin** user. Minimal fine-grained scopes
|
||||
(empirically confirmed against 1.26.2 via the API's 403 bodies):
|
||||
|
||||
| Operation | Scope |
|
||||
|---|---|
|
||||
| list packages / versions / files | `read:package` |
|
||||
| delete a tag / manifest version | `write:package` |
|
||||
| list cron tasks | `read:admin` |
|
||||
| **trigger** the `cleanup_packages` GC cron | `write:admin` |
|
||||
|
||||
The token is **never** echoed, logged, or committed. If the token lacks
|
||||
`write:admin`, the tool still deletes orphaned manifests and reports that their
|
||||
blobs will be freed by the daily `@midnight` `cleanup_packages` run (or on the
|
||||
next Gitea restart — see "native retention" below).
|
||||
|
||||
> The project's read-only token (and the build server's `~/.gitea-token`, which
|
||||
> has `read:admin` + `write:package` but **not** `write:admin`) can list, prune,
|
||||
> and delete orphans, but cannot trigger the GC cron on demand.
|
||||
|
||||
### Usage
|
||||
|
||||
```bash
|
||||
# Interactive menu (pick packages, then an action):
|
||||
GITEA_TOKEN=… ./gitea-image-prune.sh
|
||||
|
||||
# List one / all packages (with apparent per-tag sizes):
|
||||
./gitea-image-prune.sh --repo felhom-hub list
|
||||
./gitea-image-prune.sh --all list
|
||||
./gitea-image-prune.sh --all --no-sizes list # fast, skip size resolution
|
||||
|
||||
# Dry-run prune (DEFAULT — shows what would go, mutates nothing):
|
||||
./gitea-image-prune.sh --repo felhom-hub --keep 10
|
||||
./gitea-image-prune.sh --repo felhom-controller --older-than 90
|
||||
|
||||
# Apply for real (typed confirmation unless --yes):
|
||||
./gitea-image-prune.sh --repo felhom-hub --keep 10 --apply
|
||||
./gitea-image-prune.sh --repo felhom-hub --keep 10 --apply --reclaim --measure
|
||||
|
||||
# Reclaim only (delete orphaned manifests + trigger/await GC):
|
||||
./gitea-image-prune.sh --repo felhom-hub reclaim --apply
|
||||
|
||||
# Cron-friendly, non-interactive, all packages:
|
||||
./gitea-image-prune.sh --all --keep 15 --apply --yes --reclaim
|
||||
```
|
||||
|
||||
### Flags
|
||||
|
||||
| Flag | Meaning |
|
||||
|---|---|
|
||||
| `--repo NAME` (repeatable) / `--all` | Which packages to act on. |
|
||||
| `list` / `prune` / `reclaim` | Action (positional; inferred as `prune` if `--keep`/`--older-than` given, else `list`). |
|
||||
| `--keep N` | Keep the N most-recent tags; delete older. |
|
||||
| `--older-than DAYS` | Keep tags newer than DAYS; delete older. (Mutually exclusive with `--keep`.) |
|
||||
| `--dry-run` (default) / `--apply` | Mutate only with `--apply`. |
|
||||
| `--yes` | Skip confirmation prompts (for cron). |
|
||||
| `--reclaim` | After an `--apply` prune, also run the reclaim stage. |
|
||||
| `--measure` | Best-effort before/after `du` of the packages dir (needs `kubectl`; skipped cleanly otherwise). |
|
||||
| `--protect REGEX` (repeatable) | Never delete matching tags. `^latest$` is always protected. |
|
||||
| `--no-sizes` | Skip per-tag size resolution (much faster lists). |
|
||||
| `--token-file PATH` | Read the token from a file instead of `GITEA_TOKEN`. |
|
||||
| `--owner NAME` | Package owner (default `admin`). |
|
||||
| `--log PATH` | Audit log (default `misc-scripts/logs/gitea-prune-<date>.log`). |
|
||||
|
||||
Safety: dry-run is the default; `latest` (and `--protect` matches) are never
|
||||
deleted even if "old"; tags are ordered by **upload date**, never by parsing the
|
||||
tag (the registry mixes `v`-prefixed and bare tags); every HTTP status is checked
|
||||
(no silent failures); orphan detection is **fail-closed** (if any surviving tag
|
||||
can't be resolved, that package's orphan cleanup is skipped rather than risk
|
||||
deleting a referenced manifest).
|
||||
|
||||
### Native retention (set-and-forget complement)
|
||||
|
||||
This script is the **on-demand** tool. The **standing** complement is a Gitea
|
||||
per-owner **cleanup rule** (web UI: *Packages → owner → Settings → Cleanup
|
||||
Rules*): set *Remove all versions except the most recent N*, and add a tag-match
|
||||
exclusion for `latest`. The daily `cleanup_packages` cron then enforces it
|
||||
automatically. This tool does **not** auto-create rules (deliberately) — configure
|
||||
them once in the UI if you want hands-off retention.
|
||||
|
||||
> Note: on this instance, `[cron.cleanup_packages] RUN_AT_START = true` was added
|
||||
> to `app.ini` (2026-06-17) so the GC also runs on every Gitea restart — useful
|
||||
> when reclaiming without a `write:admin` token.
|
||||
@@ -0,0 +1,108 @@
|
||||
# REPORT — `gitea-image-prune.sh` (2026-06-17)
|
||||
|
||||
New operator CLI to inspect/prune old container images in the Gitea registry and
|
||||
reclaim disk, with a load-bearing live spike to **prove** the reclaim mechanism.
|
||||
|
||||
## Confirmed baselines (live)
|
||||
|
||||
| Thing | Value |
|
||||
|---|---|
|
||||
| Gitea version | **1.26.2** (`GET /api/v1/version`) |
|
||||
| Owner namespace | `admin` (standard per-owner packages API; "admin" in the path is the owner) |
|
||||
| Container packages | `felhom-controller` **96 tags / 247 digests**, `felhom-hub` **42 tags / 96 digests**, plus `recipe-importer` (42), `revfulop-calendar` (12), `jarr` (2), `wan-probe` (1) |
|
||||
| Packages cron | `cleanup_packages`, default `@midnight`, `OLDER_THAN = 24h` |
|
||||
| Packages dir | `/data/gitea/packages` (gitea-system pod, container `gitea`), **baseline 5,122,143,723 B ≈ 4.77 GiB** |
|
||||
| Tooling | `jq` 1.7 present on build server; `shellcheck` **absent** (not run — script written carefully, `bash -n` clean) |
|
||||
|
||||
## Minimal token scope set (empirically confirmed via 403 bodies)
|
||||
|
||||
| Operation | Required scope |
|
||||
|---|---|
|
||||
| list packages / versions / files | `read:package` |
|
||||
| delete a tag / manifest version | `write:package` |
|
||||
| list cron tasks | `read:admin` |
|
||||
| trigger `cleanup_packages` cron | **`write:admin`** |
|
||||
|
||||
The build server token (`~/.gitea-token`) has `read:admin` + `write:package` but
|
||||
**not** `write:admin` (its 403 body named exactly `required=[write:admin]`), so it
|
||||
can list/prune/delete-orphans but cannot trigger the GC cron on demand. Token must
|
||||
belong to a site-admin user. (Scopes were *not* minimized by minting reduced
|
||||
tokens — that needs `write:user`, which this token also lacks — but each required
|
||||
scope was confirmed by a successful call and the cron requirement by its 403.)
|
||||
|
||||
## §3 spike — the reclaim mechanism (PROVEN, not assumed)
|
||||
|
||||
Gitea stores a tag as a tiny OCI **index** pointer; the real bytes are in untagged
|
||||
`sha256:` **manifest versions** (config + layer blobs), whose layers are shared
|
||||
across tags. The mechanism turned out to be **three steps**, not two:
|
||||
|
||||
| Step (single-version spike, `felhom-hub`) | `du` (bytes) | freed |
|
||||
|---|---|---|
|
||||
| baseline | 5,122,143,723 | — |
|
||||
| DELETE tag `0.1.1` (via the script, HTTP 204) | 5,122,143,723 | **0** |
|
||||
| run `cleanup_packages` (tag-only) | 5,122,138,771 | ~5 KB (index pointer only) |
|
||||
| DELETE tag `0.1.2` + its **2 orphaned manifests** (204×3) | 5,122,138,771 | **0** |
|
||||
| run `cleanup_packages` GC | 5,116,799,814 | **5,338,957 B ≈ 5.1 MiB** |
|
||||
|
||||
**Conclusions:**
|
||||
1. Deleting a tag frees ~nothing (only the index pointer).
|
||||
2. **Default `cleanup_packages` does NOT remove untagged manifest versions** —
|
||||
only unreferenced *blobs*. So the orphaned `sha256:` manifests must be deleted
|
||||
explicitly (the script's reclaim does this); otherwise their blobs stay
|
||||
referenced forever. (Confirmed: a tag-only delete + cron left `felhom-hub`
|
||||
digests at 96.)
|
||||
3. Once the orphaned manifests are deleted, `cleanup_packages` GCs their *unique*
|
||||
blobs (created > `OLDER_THAN`); shared base layers stay. 5.1 MiB freed for one
|
||||
9.4 MB-apparent image — the difference is shared layers, correctly retained.
|
||||
|
||||
Because the token lacks `write:admin`, the GC cron was triggered by adding
|
||||
`[cron.cleanup_packages] RUN_AT_START = true` to `app.ini` (on the data PVC,
|
||||
backup `app.ini.bak.prune-spike`) and rolling-restarting Gitea — left in place per
|
||||
operator request (the daily `@midnight` run also performs the GC).
|
||||
|
||||
## §9 verification results
|
||||
|
||||
1. **List (read-only):** `--repo felhom-hub list` → 42 tags, newest-first, per-tag
|
||||
sizes resolved via OCI (24 MB recent, ~9 MB older), `latest` flagged PROTECTED,
|
||||
shared-layer caveat printed. `--all` lists all 6 packages; `--no-sizes` fast path
|
||||
works. No mutation.
|
||||
2. **Dry-run prune:** `--repo felhom-hub --keep 5 --dry-run` → would delete 36,
|
||||
keep 5 + 1 protected, oldest first, totals shown, **nothing changed**.
|
||||
3. **One-version live proof (spike):** see table above — delete-alone = 0 bytes;
|
||||
delete + orphan-manifest delete + GC = 5.1 MiB.
|
||||
4. **Full reclaim path validated** on `felhom-hub` only: `reclaim --apply` deleted
|
||||
the **16 accumulated orphan manifests** (untagged, referenced by no tag — dead
|
||||
weight from re-pointed `latest` + buildx attestations), then GC freed
|
||||
**5,116,799,814 → 5,026,431,222 = 90,368,592 B ≈ 86 MiB**.
|
||||
5. **Safety:** after reclaim, surviving tags still resolve and **`docker pull`
|
||||
cleanly** (`latest`, `0.1.3` — the immediate neighbor of the deleted tags).
|
||||
Orphan detection is fail-closed (skips a package if any surviving tag won't
|
||||
resolve).
|
||||
6. **Audit log** captured every RUN / DRY-RUN / APPLIED / RECLAIM line; **token
|
||||
scan of the log = 0 hits**. Edge cases: `--keep`+`--older-than` → error;
|
||||
`--keep 999` → "Nothing to prune (40 tags: 39 kept, 1 protected)".
|
||||
|
||||
## State left on the registry
|
||||
|
||||
- `felhom-hub`: tags `0.1.1` and `0.1.2` deleted (spike); 16 orphan manifests
|
||||
cleaned; now **40 tags / 78 digests**; ~91 MiB reclaimed total. All remaining
|
||||
tags pull cleanly.
|
||||
- **`felhom-controller` (96 tags) and all other packages: UNTOUCHED.**
|
||||
- `app.ini`: `RUN_AT_START = true` added for `cleanup_packages` (kept).
|
||||
|
||||
## NOT yet run
|
||||
|
||||
**The real bulk cleanup — left to the operator (interactive).** This run proved
|
||||
the mechanism on one disposable version and validated the full reclaim path on
|
||||
`felhom-hub`'s dead orphans only. Pruning the ~90 `felhom-controller` tags (and
|
||||
the bulk of `felhom-hub`/`recipe-importer` history) is the operator's call via
|
||||
`gitea-image-prune.sh --repo … --keep N --apply --reclaim`.
|
||||
|
||||
## Backlog / notes
|
||||
|
||||
- A `write:admin` token (or the native cleanup rule in the UI) would let `reclaim`
|
||||
trigger the GC immediately instead of relying on the `@midnight`/restart run.
|
||||
- Per-tag "apparent" sizes overlap (shared base layers counted once per tag);
|
||||
`--measure` (`du`) is the honest real-reclaim signal. Documented in the tool.
|
||||
- `shellcheck` was unavailable on the build server, so the script was not
|
||||
statically linted (only `bash -n` syntax-checked + extensively run live).
|
||||
Executable
+696
@@ -0,0 +1,696 @@
|
||||
#!/usr/bin/env bash
|
||||
# =============================================================================
|
||||
# gitea-image-prune.sh — inspect & prune old container images in the
|
||||
# self-hosted Gitea registry, then reclaim disk.
|
||||
# =============================================================================
|
||||
# Target server : gitea.dooplex.hu (Gitea 1.26.2, owner namespace "admin")
|
||||
# Best run on : build server 192.168.0.180 (has kubectl for --measure),
|
||||
# but the core (list/prune/reclaim) is pure curl + jq and runs
|
||||
# from any host that can reach Gitea and holds an admin token.
|
||||
#
|
||||
# -----------------------------------------------------------------------------
|
||||
# WHY THIS EXISTS
|
||||
# build-felhom-{hub,controller}.sh push :<version> AND :latest on every build,
|
||||
# so felhom-controller (~96 tags) / felhom-hub (~42 tags) accumulate one image
|
||||
# per build and the Gitea Longhorn PVC keeps filling.
|
||||
#
|
||||
# HOW GITEA STORES CONTAINER IMAGES (load-bearing — drives the reclaim design)
|
||||
# A pushed tag is an OCI image *index* (a ~850 B pointer). The real bytes live
|
||||
# in untagged "sha256:..." manifest *versions* (config + layer blobs, ~9-25 MB
|
||||
# each), whose layer blobs are content-addressed and SHARED across tags.
|
||||
# Deleting a tag removes only the tiny index pointer — the manifest versions it
|
||||
# referenced linger as untagged versions, and their blobs stay referenced.
|
||||
#
|
||||
# PROVEN RECLAIM MECHANISM (verified live on this instance, 2026-06-17 — see §3
|
||||
# of the task / REPORT.md). It is THREE steps, not two:
|
||||
# 1. DELETE the tag(s) -> frees ~nothing (only the index ptr)
|
||||
# 2. DELETE the now-ORPHANED "sha256:" manifest versions (referenced by no
|
||||
# surviving tag) -> still frees nothing on its own, BUT
|
||||
# makes their unique blobs unreferenced. (Default cleanup_packages does
|
||||
# NOT remove untagged manifests — only a cleanup *rule* would — so the
|
||||
# script must delete the orphaned manifests itself. write:package scope.)
|
||||
# 3. cleanup_packages cron runs -> GCs unreferenced blobs created
|
||||
# >OLDER_THAN (24h default) ago -> THIS frees disk. Shared base layers
|
||||
# still referenced by surviving tags are correctly retained.
|
||||
# Live proof: deleting one 9.4 MB-apparent tag + its 2 manifests + GC freed
|
||||
# 5.1 MiB (the rest was shared base layers, correctly kept).
|
||||
# => "prune" deletes tags; "reclaim" deletes the orphaned manifests and then
|
||||
# triggers (or defers to) the cleanup_packages cron.
|
||||
#
|
||||
# TRIGGERING THE GC CRON: POST /api/v1/admin/cron/cleanup_packages needs
|
||||
# write:admin. If the token lacks it, the orphaned manifests are still deleted
|
||||
# and their blobs are freed by the daily "@midnight" run (or on the next Gitea
|
||||
# restart — RUN_AT_START was enabled in app.ini on 2026-06-17).
|
||||
#
|
||||
# REQUIRED TOKEN (env GITEA_TOKEN, or --token-file). Must belong to a Gitea
|
||||
# site-admin user. Minimal fine-grained scopes (Gitea 1.26):
|
||||
# read:package — list packages / versions / files (list, prune planning)
|
||||
# write:package — delete a version (prune --apply)
|
||||
# read:admin — list cron tasks
|
||||
# write:admin — run the cleanup_packages cron (reclaim)
|
||||
# The project read-only token is insufficient (no package scope).
|
||||
# The token is NEVER echoed, logged, or committed.
|
||||
#
|
||||
# USAGE
|
||||
# GITEA_TOKEN=... ./gitea-image-prune.sh # interactive menu
|
||||
# ./gitea-image-prune.sh --repo felhom-hub list # list one package
|
||||
# ./gitea-image-prune.sh --all list # list all packages
|
||||
# ./gitea-image-prune.sh --repo felhom-hub --keep 10 # dry-run prune (default)
|
||||
# ./gitea-image-prune.sh --repo felhom-hub --keep 10 --apply # really delete
|
||||
# ./gitea-image-prune.sh --repo felhom-controller --older-than 90 --apply --reclaim
|
||||
# ./gitea-image-prune.sh --all --keep 15 --apply --yes --reclaim --measure # cron-friendly
|
||||
# ./gitea-image-prune.sh --repo felhom-hub reclaim # run cleanup cron only
|
||||
#
|
||||
# SET-AND-FORGET COMPLEMENT: configure a native Gitea cleanup rule per owner
|
||||
# (package settings -> Cleanup Rules: keep most-recent N, exclude ^latest$);
|
||||
# the daily cleanup_packages cron then enforces it. See README. This script
|
||||
# does NOT auto-create rules — it is the on-demand tool.
|
||||
# =============================================================================
|
||||
set -euo pipefail
|
||||
|
||||
# --- Configuration --------------------------------------------------------
|
||||
GITEA_URL="${GITEA_URL:-https://gitea.dooplex.hu}"
|
||||
OWNER="${GITEA_OWNER:-admin}"
|
||||
CLEANUP_CRON="cleanup_packages"
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
|
||||
# --- Colors / log helpers (match build-felhom-hub.sh) ---------------------
|
||||
RED='\033[0;31m'; GREEN='\033[0;32m'; YELLOW='\033[1;33m'; CYAN='\033[0;36m'; BOLD='\033[1m'; NC='\033[0m'
|
||||
info() { echo -e "${GREEN}[INFO]${NC} $*"; }
|
||||
warn() { echo -e "${YELLOW}[WARN]${NC} $*" >&2; }
|
||||
error() { echo -e "${RED}[ERROR]${NC} $*" >&2; }
|
||||
step() { echo -e "${CYAN}[STEP]${NC} $*"; }
|
||||
note() { echo -e "${CYAN} $*${NC}"; }
|
||||
|
||||
# --- Defaults / arg state -------------------------------------------------
|
||||
declare -a REPOS=()
|
||||
declare -a PROTECT=('^latest$') # always-protected tags; --protect appends
|
||||
ALL=false
|
||||
ACTION="" # list | prune | reclaim (positional or inferred)
|
||||
KEEP=""
|
||||
OLDER_THAN=""
|
||||
APPLY=false # dry-run is the default
|
||||
ASSUME_YES=false
|
||||
DO_RECLAIM=false # run cleanup cron after an --apply prune
|
||||
MEASURE=false
|
||||
NO_SIZES=false
|
||||
TOKEN_FILE=""
|
||||
LOG_FILE=""
|
||||
GITEA_TOKEN="${GITEA_TOKEN:-}"
|
||||
|
||||
# --- Temp workspace -------------------------------------------------------
|
||||
TMP="$(mktemp -d)"
|
||||
cleanup() { rm -rf "$TMP"; }
|
||||
trap cleanup EXIT
|
||||
|
||||
usage() {
|
||||
sed -n '2,/^set -euo/p' "$0" | sed 's/^# \{0,1\}//; s/^#//' | sed '$d'
|
||||
exit "${1:-0}"
|
||||
}
|
||||
|
||||
# =============================================================================
|
||||
# Argument parsing
|
||||
# =============================================================================
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
list|prune|reclaim) ACTION="$1"; shift ;;
|
||||
--repo) REPOS+=("$2"); shift 2 ;;
|
||||
--all) ALL=true; shift ;;
|
||||
--keep) KEEP="$2"; shift 2 ;;
|
||||
--older-than) OLDER_THAN="$2"; shift 2 ;;
|
||||
--dry-run) APPLY=false; shift ;;
|
||||
--apply) APPLY=true; shift ;;
|
||||
--yes|-y) ASSUME_YES=true; shift ;;
|
||||
--reclaim) DO_RECLAIM=true; shift ;;
|
||||
--measure) MEASURE=true; shift ;;
|
||||
--no-sizes) NO_SIZES=true; shift ;;
|
||||
--protect) PROTECT+=("$2"); shift 2 ;;
|
||||
--token-file) TOKEN_FILE="$2"; shift 2 ;;
|
||||
--owner) OWNER="$2"; shift 2 ;;
|
||||
--log) LOG_FILE="$2"; shift 2 ;;
|
||||
-h|--help) usage 0 ;;
|
||||
*) error "Unknown argument: $1"; usage 1 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
# --- Validate flag combinations -------------------------------------------
|
||||
if [[ -n "$KEEP" && -n "$OLDER_THAN" ]]; then
|
||||
error "--keep and --older-than are mutually exclusive (pick one prune mode)."
|
||||
exit 2
|
||||
fi
|
||||
if [[ -n "$KEEP" && ! "$KEEP" =~ ^[0-9]+$ ]]; then error "--keep must be a non-negative integer"; exit 2; fi
|
||||
if [[ -n "$OLDER_THAN" && ! "$OLDER_THAN" =~ ^[0-9]+$ ]]; then error "--older-than must be a non-negative integer (days)"; exit 2; fi
|
||||
|
||||
# =============================================================================
|
||||
# Preflight
|
||||
# =============================================================================
|
||||
for bin in curl jq; do
|
||||
command -v "$bin" &>/dev/null || { error "Required tool not found: $bin"; exit 1; }
|
||||
done
|
||||
|
||||
# Token: --token-file beats env. Never printed.
|
||||
if [[ -n "$TOKEN_FILE" ]]; then
|
||||
[[ -r "$TOKEN_FILE" ]] || { error "--token-file not readable: $TOKEN_FILE"; exit 1; }
|
||||
GITEA_TOKEN="$(tr -d ' \t\r\n' < "$TOKEN_FILE")"
|
||||
fi
|
||||
if [[ -z "$GITEA_TOKEN" ]]; then
|
||||
error "No token. Set GITEA_TOKEN env or pass --token-file <path>."
|
||||
error "Needs a site-admin token with read/write:package + read/write:admin."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Default audit log
|
||||
if [[ -z "$LOG_FILE" ]]; then
|
||||
LOG_FILE="${SCRIPT_DIR}/logs/gitea-prune-$(date +%Y-%m-%d).log"
|
||||
fi
|
||||
mkdir -p "$(dirname "$LOG_FILE")"
|
||||
|
||||
# Redact the token from anything we print (belt-and-suspenders vs set -x etc.)
|
||||
redact() { sed "s|${GITEA_TOKEN}|***TOKEN***|g"; }
|
||||
|
||||
audit() { # free-form line -> audit log (token-free by construction)
|
||||
printf '%s %s\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)" "$*" >> "$LOG_FILE"
|
||||
}
|
||||
|
||||
# =============================================================================
|
||||
# HTTP helpers — every call checks status; non-2xx is never silent.
|
||||
# =============================================================================
|
||||
# api_get <path> -> body on stdout; dies on non-2xx
|
||||
# api_delete <path> -> echoes HTTP code; returns non-zero on non-204
|
||||
# api_post <path> -> echoes HTTP code; dies on non-2xx
|
||||
# oci_get <name> <ref> -> OCI manifest JSON (Basic auth); empty + rc1 on fail
|
||||
api_get() {
|
||||
local path="$1" out code
|
||||
out="$(mktemp)"
|
||||
code="$(curl -sS -H "Authorization: token ${GITEA_TOKEN}" \
|
||||
-o "$out" -w '%{http_code}' "${GITEA_URL}${path}" || true)"
|
||||
if [[ ! "$code" =~ ^2[0-9][0-9]$ ]]; then
|
||||
error "GET ${path} -> HTTP ${code}"
|
||||
head -c 400 "$out" | redact >&2; echo >&2
|
||||
rm -f "$out"; return 1
|
||||
fi
|
||||
cat "$out"; rm -f "$out"
|
||||
}
|
||||
|
||||
api_delete() {
|
||||
local path="$1" code
|
||||
code="$(curl -sS -X DELETE -H "Authorization: token ${GITEA_TOKEN}" \
|
||||
-o "$TMP/del.body" -w '%{http_code}' "${GITEA_URL}${path}" || true)"
|
||||
echo "$code"
|
||||
[[ "$code" == "204" ]]
|
||||
}
|
||||
|
||||
api_post() {
|
||||
local path="$1" code
|
||||
code="$(curl -sS -X POST -H "Authorization: token ${GITEA_TOKEN}" \
|
||||
-o "$TMP/post.body" -w '%{http_code}' "${GITEA_URL}${path}" || true)"
|
||||
if [[ ! "$code" =~ ^2[0-9][0-9]$ ]]; then
|
||||
error "POST ${path} -> HTTP ${code}"
|
||||
head -c 400 "$TMP/post.body" | redact >&2; echo >&2
|
||||
return 1
|
||||
fi
|
||||
echo "$code"
|
||||
}
|
||||
|
||||
oci_get() {
|
||||
local name="$1" ref="$2" out code
|
||||
out="$(mktemp)"
|
||||
code="$(curl -sS -u "${OWNER}:${GITEA_TOKEN}" \
|
||||
-H 'Accept: application/vnd.oci.image.index.v1+json,application/vnd.docker.distribution.manifest.list.v2+json,application/vnd.oci.image.manifest.v1+json,application/vnd.docker.distribution.manifest.v2+json' \
|
||||
-o "$out" -w '%{http_code}' "${GITEA_URL}/v2/${OWNER}/${name}/manifests/${ref}" || true)"
|
||||
if [[ ! "$code" =~ ^2[0-9][0-9]$ ]]; then rm -f "$out"; return 1; fi
|
||||
cat "$out"; rm -f "$out"
|
||||
}
|
||||
|
||||
# =============================================================================
|
||||
# Data load — fetch ALL container versions for OWNER once (paginated), cache it.
|
||||
# =============================================================================
|
||||
VERSIONS_JSON="$TMP/versions.json" # JSON-lines, one version object per line
|
||||
|
||||
load_versions() {
|
||||
step "Fetching container packages for owner '${OWNER}' from ${GITEA_URL} ..."
|
||||
: > "$VERSIONS_JSON"
|
||||
local page=1 limit=50 body n total=0
|
||||
while :; do
|
||||
body="$(api_get "/api/v1/packages/${OWNER}?type=container&page=${page}&limit=${limit}")" || exit 1
|
||||
n="$(echo "$body" | jq 'length')"
|
||||
echo "$body" | jq -c '.[]' >> "$VERSIONS_JSON"
|
||||
total=$((total + n))
|
||||
[[ "$n" -lt "$limit" ]] && break
|
||||
page=$((page + 1))
|
||||
done
|
||||
info "Loaded ${total} version records across $(jq -rs '[.[].name]|unique|length' "$VERSIONS_JSON") package(s)."
|
||||
}
|
||||
|
||||
# Distinct package names (sorted)
|
||||
package_names() { jq -rs '[.[].name]|unique|.[]' "$VERSIONS_JSON"; }
|
||||
|
||||
# Tagged versions of a package, newest first: "tag<TAB>created_at"
|
||||
tagged_versions() {
|
||||
local name="$1"
|
||||
jq -rs --arg n "$name" '
|
||||
[ .[] | select(.name==$n and (.version|startswith("sha256:")|not)) ]
|
||||
| sort_by(.created_at) | reverse
|
||||
| .[] | "\(.version)\t\(.created_at)"' "$VERSIONS_JSON"
|
||||
}
|
||||
|
||||
# Count of digest (sha256:) versions of a package
|
||||
digest_count() {
|
||||
local name="$1"
|
||||
jq -rs --arg n "$name" '[ .[] | select(.name==$n and (.version|startswith("sha256:"))) ] | length' "$VERSIONS_JSON"
|
||||
}
|
||||
|
||||
# =============================================================================
|
||||
# Size resolution (best-effort, OCI). Per-tag "apparent" image size in bytes.
|
||||
# tag -> OCI index -> referenced manifest digests -> files-API byte sums.
|
||||
# Caches digest->bytes so repeated/shared digests aren't refetched.
|
||||
# =============================================================================
|
||||
declare -A DIGEST_BYTES_CACHE=()
|
||||
|
||||
digest_files_bytes() { # <name> <sha256:...> -> bytes (files-API sum)
|
||||
local name="$1" digest="$2"
|
||||
local key="${name}@${digest}"
|
||||
if [[ -n "${DIGEST_BYTES_CACHE[$key]:-}" ]]; then echo "${DIGEST_BYTES_CACHE[$key]}"; return; fi
|
||||
local body sum
|
||||
body="$(api_get "/api/v1/packages/${OWNER}/container/${name}/${digest}/files" 2>/dev/null || echo '[]')"
|
||||
sum="$(echo "$body" | jq '[.[].size] | add // 0' 2>/dev/null || echo 0)"
|
||||
DIGEST_BYTES_CACHE[$key]="$sum"
|
||||
echo "$sum"
|
||||
}
|
||||
|
||||
resolve_tag_bytes() { # <name> <tag> -> apparent image bytes (0 if unresolved)
|
||||
local name="$1" tag="$2" man total=0 d
|
||||
man="$(oci_get "$name" "$tag")" || { echo 0; return; }
|
||||
if echo "$man" | jq -e '.manifests' >/dev/null 2>&1; then
|
||||
# OCI index / manifest list: sum referenced manifest digest versions
|
||||
while IFS= read -r d; do
|
||||
[[ -z "$d" ]] && continue
|
||||
total=$(( total + $(digest_files_bytes "$name" "$d") ))
|
||||
done < <(echo "$man" | jq -r '.manifests[].digest')
|
||||
else
|
||||
# Single image manifest: config + layers from the manifest itself
|
||||
total="$(echo "$man" | jq '((.config.size // 0) + ([.layers[].size] | add // 0))')"
|
||||
fi
|
||||
echo "$total"
|
||||
}
|
||||
|
||||
human() { # bytes -> human readable
|
||||
local b="${1:-0}"
|
||||
if command -v numfmt &>/dev/null; then numfmt --to=iec --suffix=B "$b" 2>/dev/null || echo "${b}B";
|
||||
else echo "${b}B"; fi
|
||||
}
|
||||
|
||||
is_protected() { # <tag> -> 0 if protected
|
||||
local tag="$1" rx
|
||||
for rx in "${PROTECT[@]}"; do [[ "$tag" =~ $rx ]] && return 0; done
|
||||
return 1
|
||||
}
|
||||
|
||||
# =============================================================================
|
||||
# Orphan-manifest detection (drives reclaim).
|
||||
# A "sha256:" manifest version is an ORPHAN if no SURVIVING tag's index
|
||||
# references it. Deleting orphans makes their unique blobs GC-able.
|
||||
# FAIL-CLOSED: if any tag fails to resolve (HTTP/format), we refuse to compute
|
||||
# orphans for that package — never risk deleting a still-referenced manifest.
|
||||
# =============================================================================
|
||||
# referenced_digests <name> <outfile> -> rc0 (outfile = digests referenced by
|
||||
# surviving tags), rc1 if unsafe to proceed
|
||||
referenced_digests() {
|
||||
local name="$1" out="$2" tag created man
|
||||
: > "$out"
|
||||
while IFS=$'\t' read -r tag created; do
|
||||
[[ -z "$tag" ]] && continue
|
||||
man="$(oci_get "$name" "$tag")" || {
|
||||
warn " cannot resolve tag '${tag}' of ${name} (HTTP/format) — skipping orphan cleanup (safety)"; return 1; }
|
||||
if echo "$man" | jq -e 'has("manifests")' >/dev/null 2>&1; then
|
||||
echo "$man" | jq -r '.manifests[].digest' >> "$out"
|
||||
else
|
||||
# Single-arch image manifest (no index). Our registry uses indexes;
|
||||
# if we hit this, bail rather than guess which sha256 version is safe.
|
||||
warn " tag '${tag}' of ${name} is not a manifest index — skipping orphan cleanup (safety)"; return 1
|
||||
fi
|
||||
done < <(tagged_versions "$name")
|
||||
return 0
|
||||
}
|
||||
|
||||
# find_orphans <name> -> prints orphan sha256: versions (one per line); rc1 if unsafe
|
||||
find_orphans() {
|
||||
local name="$1" ref="$TMP/ref.${name//\//_}" sha="$TMP/sha.${name//\//_}"
|
||||
referenced_digests "$name" "$ref" || return 1
|
||||
LC_ALL=C sort -u "$ref" -o "$ref"
|
||||
jq -rs --arg n "$name" \
|
||||
'.[] | select(.name==$n and (.version|startswith("sha256:"))) | .version' \
|
||||
"$VERSIONS_JSON" | LC_ALL=C sort -u > "$sha"
|
||||
LC_ALL=C comm -23 "$sha" "$ref"
|
||||
}
|
||||
|
||||
# Quiet cron trigger (separate from api_post so we can message a 403 nicely).
|
||||
# echoes "ok <code>" on 2xx, "denied" on 403, "err <code>" otherwise; rc reflects.
|
||||
trigger_cleanup_cron() {
|
||||
local code
|
||||
code="$(curl -sS -X POST -H "Authorization: token ${GITEA_TOKEN}" \
|
||||
-o "$TMP/cron.body" -w '%{http_code}' \
|
||||
"${GITEA_URL}/api/v1/admin/cron/${CLEANUP_CRON}" || true)"
|
||||
if [[ "$code" =~ ^2[0-9][0-9]$ ]]; then echo "ok ${code}"; return 0; fi
|
||||
if [[ "$code" == "403" ]]; then echo "denied"; return 1; fi
|
||||
echo "err ${code}"; return 1
|
||||
}
|
||||
|
||||
# =============================================================================
|
||||
# Disk measurement (best-effort; kubectl optional)
|
||||
# =============================================================================
|
||||
GITEA_NS="${GITEA_NS:-gitea-system}"
|
||||
GITEA_PKG_PATH="/data/gitea/packages"
|
||||
KUBECTL=""
|
||||
detect_kubectl() {
|
||||
if command -v kubectl &>/dev/null; then KUBECTL="kubectl";
|
||||
elif sudo -n kubectl version --client &>/dev/null 2>&1; then KUBECTL="sudo kubectl";
|
||||
elif command -v sudo &>/dev/null; then KUBECTL="sudo kubectl"; fi
|
||||
}
|
||||
gitea_pod() {
|
||||
[[ -n "$KUBECTL" ]] || return 1
|
||||
$KUBECTL get pods -n "$GITEA_NS" -o name 2>/dev/null | grep -i gitea | head -1 | sed 's|pod/||'
|
||||
}
|
||||
measure_bytes() { # echoes byte count of packages dir, or empty if unavailable
|
||||
local pod
|
||||
pod="$(gitea_pod)" || return 1
|
||||
[[ -n "$pod" ]] || return 1
|
||||
$KUBECTL exec -n "$GITEA_NS" "$pod" -c gitea -- du -sb "$GITEA_PKG_PATH" 2>/dev/null | awk '{print $1}'
|
||||
}
|
||||
|
||||
# =============================================================================
|
||||
# Actions
|
||||
# =============================================================================
|
||||
|
||||
# ---- LIST ----------------------------------------------------------------
|
||||
do_list() {
|
||||
local name="$1" line tag created bytes total_bytes=0 count=0 prot
|
||||
echo ""
|
||||
echo -e "${BOLD}== ${name} ==${NC} (owner ${OWNER}; $(digest_count "$name") digest manifests)"
|
||||
printf ' %-28s %-22s %-12s %s\n' "TAG" "UPLOADED" "APPARENT" "FLAG"
|
||||
printf ' %-28s %-22s %-12s %s\n' "---" "--------" "--------" "----"
|
||||
while IFS=$'\t' read -r tag created; do
|
||||
[[ -z "$tag" ]] && continue
|
||||
count=$((count + 1))
|
||||
if $NO_SIZES; then bytes=0; else bytes="$(resolve_tag_bytes "$name" "$tag")"; fi
|
||||
total_bytes=$((total_bytes + bytes))
|
||||
prot=""; is_protected "$tag" && prot="PROTECTED"
|
||||
printf ' %-28s %-22s %-12s %s\n' "$tag" "${created:0:19}" "$( $NO_SIZES && echo '-' || human "$bytes")" "$prot"
|
||||
done < <(tagged_versions "$name")
|
||||
echo " ----"
|
||||
if $NO_SIZES; then
|
||||
info " ${count} tagged version(s). (sizes skipped: --no-sizes)"
|
||||
else
|
||||
info " ${count} tagged version(s); apparent total ≈ $(human "$total_bytes")"
|
||||
note " CAVEAT: apparent sizes count shared base layers once PER TAG, so they"
|
||||
note " overlap heavily. Real reclaimed space is much less — measure with --measure."
|
||||
fi
|
||||
}
|
||||
|
||||
# ---- Compute prune plan: prints "DELETE\t<tag>\t<created>" / "KEEP..." ----
|
||||
# Sets globals: PLAN_DELETE (array of tags), PLAN_KEEP_N, PLAN_PROT_N
|
||||
declare -a PLAN_DELETE=()
|
||||
PLAN_KEEP_N=0; PLAN_PROT_N=0; PLAN_TOTAL=0
|
||||
compute_plan() {
|
||||
local name="$1" tag created epoch now cutoff idx=0
|
||||
PLAN_DELETE=(); PLAN_KEEP_N=0; PLAN_PROT_N=0; PLAN_TOTAL=0
|
||||
now="$(date +%s)"
|
||||
[[ -n "$OLDER_THAN" ]] && cutoff=$(( now - OLDER_THAN * 86400 ))
|
||||
while IFS=$'\t' read -r tag created; do
|
||||
[[ -z "$tag" ]] && continue
|
||||
PLAN_TOTAL=$((PLAN_TOTAL + 1))
|
||||
# Protected always wins.
|
||||
if is_protected "$tag"; then PLAN_PROT_N=$((PLAN_PROT_N + 1)); continue; fi
|
||||
if [[ -n "$KEEP" ]]; then
|
||||
# tags arrive newest-first; keep the first KEEP non-protected... but
|
||||
# protected tags don't consume a keep slot — count index over all tags.
|
||||
idx=$((idx + 1))
|
||||
if [[ "$idx" -le "$KEEP" ]]; then PLAN_KEEP_N=$((PLAN_KEEP_N + 1)); else PLAN_DELETE+=("$tag"); fi
|
||||
elif [[ -n "$OLDER_THAN" ]]; then
|
||||
epoch="$(date -d "$created" +%s 2>/dev/null || echo "$now")"
|
||||
if [[ "$epoch" -lt "$cutoff" ]]; then PLAN_DELETE+=("$tag"); else PLAN_KEEP_N=$((PLAN_KEEP_N + 1)); fi
|
||||
fi
|
||||
done < <(tagged_versions "$name")
|
||||
}
|
||||
|
||||
# ---- PRUNE ---------------------------------------------------------------
|
||||
PRUNE_DELETED=0; PRUNE_FAILED=0
|
||||
declare -a PRUNE_TOUCHED_REPOS=()
|
||||
do_prune_repo() {
|
||||
local name="$1" tag bytes code
|
||||
compute_plan "$name"
|
||||
|
||||
echo ""
|
||||
echo -e "${BOLD}== prune ${name} ==${NC} mode: $( [[ -n "$KEEP" ]] && echo "keep-last ${KEEP}" || echo "older-than ${OLDER_THAN}d" )"
|
||||
if [[ "${#PLAN_DELETE[@]}" -eq 0 ]]; then
|
||||
info " Nothing to prune (${PLAN_TOTAL} tags: ${PLAN_KEEP_N} kept, ${PLAN_PROT_N} protected)."
|
||||
return
|
||||
fi
|
||||
echo " Would delete ${#PLAN_DELETE[@]} tag(s); keep ${PLAN_KEEP_N}; protect ${PLAN_PROT_N}:"
|
||||
local plan_bytes=0 szlabel
|
||||
for tag in "${PLAN_DELETE[@]}"; do
|
||||
if $NO_SIZES; then bytes=0; szlabel=""; else bytes="$(resolve_tag_bytes "$name" "$tag")"; szlabel="≈ $(human "$bytes")"; fi
|
||||
plan_bytes=$((plan_bytes + bytes))
|
||||
printf ' %-28s %s\n' "$tag" "$szlabel"
|
||||
done
|
||||
$NO_SIZES || note " apparent total to delete ≈ $(human "$plan_bytes") (overlaps shared layers — real reclaim less)"
|
||||
|
||||
if ! $APPLY; then
|
||||
warn " DRY-RUN — nothing deleted. Re-run with --apply to delete."
|
||||
for tag in "${PLAN_DELETE[@]}"; do audit "DRY-RUN would-delete ${OWNER}/${name}:${tag}"; done
|
||||
return
|
||||
fi
|
||||
|
||||
# --- confirmation gate ---
|
||||
if ! $ASSUME_YES; then
|
||||
echo ""
|
||||
warn " About to DELETE ${#PLAN_DELETE[@]} tag(s) from ${name}. This is destructive."
|
||||
read -r -p " Type the package name '${name}' to confirm: " reply
|
||||
if [[ "$reply" != "$name" ]]; then warn " Skipped ${name} (confirmation mismatch)."; return; fi
|
||||
fi
|
||||
|
||||
for tag in "${PLAN_DELETE[@]}"; do
|
||||
if $NO_SIZES; then bytes=0; else bytes="$(resolve_tag_bytes "$name" "$tag")"; fi
|
||||
code="$(api_delete "/api/v1/packages/${OWNER}/container/${name}/${tag}")" && {
|
||||
info " deleted ${name}:${tag} (HTTP ${code})"
|
||||
audit "APPLIED deleted ${OWNER}/${name}:${tag} apparent=${bytes}B http=${code}"
|
||||
PRUNE_DELETED=$((PRUNE_DELETED + 1))
|
||||
} || {
|
||||
error " FAILED ${name}:${tag} (HTTP ${code}) — continuing"
|
||||
audit "FAILED delete ${OWNER}/${name}:${tag} http=${code}"
|
||||
PRUNE_FAILED=$((PRUNE_FAILED + 1))
|
||||
}
|
||||
done
|
||||
PRUNE_TOUCHED_REPOS+=("$name")
|
||||
}
|
||||
|
||||
# ---- RECLAIM -------------------------------------------------------------
|
||||
# Step 2 + 3 of the proven mechanism: delete orphaned manifest versions, then
|
||||
# trigger (or defer to) the cleanup_packages GC cron. Honors --dry-run/--apply.
|
||||
do_reclaim() {
|
||||
step "Reclaim: find & delete orphaned (untagged, unreferenced) manifests, then GC blobs."
|
||||
# Reload state — a preceding prune deleted tags, so the cached listing is stale.
|
||||
load_versions
|
||||
|
||||
local before; before=""
|
||||
if $MEASURE; then before="$(measure_bytes || true)"; [[ -n "$before" ]] && info " packages dir now: $(human "$before")"; fi
|
||||
|
||||
local name orphans norph d code total_orphans=0 deleted=0 failed=0 unsafe=0
|
||||
for name in "${TARGETS[@]}"; do
|
||||
if ! orphans="$(find_orphans "$name")"; then
|
||||
unsafe=$((unsafe + 1)); continue # warning already emitted; skip this package
|
||||
fi
|
||||
norph="$(printf '%s\n' "$orphans" | grep -c . || true)"
|
||||
if [[ "$norph" -eq 0 ]]; then info " ${name}: no orphaned manifests."; continue; fi
|
||||
total_orphans=$((total_orphans + norph))
|
||||
if ! $APPLY; then
|
||||
info " ${name}: ${norph} orphaned manifest version(s) WOULD be deleted (dry-run):"
|
||||
printf '%s\n' "$orphans" | sed 's/^/ /'
|
||||
continue
|
||||
fi
|
||||
info " ${name}: deleting ${norph} orphaned manifest version(s)..."
|
||||
while IFS= read -r d; do
|
||||
[[ -z "$d" ]] && continue
|
||||
code="$(api_delete "/api/v1/packages/${OWNER}/container/${name}/${d}")" && {
|
||||
deleted=$((deleted + 1)); audit "RECLAIM deleted-orphan ${OWNER}/${name}/${d} http=${code}"
|
||||
} || {
|
||||
failed=$((failed + 1)); error " FAILED orphan ${name}/${d} (HTTP ${code})"
|
||||
audit "RECLAIM FAILED-orphan ${OWNER}/${name}/${d} http=${code}"
|
||||
}
|
||||
done <<< "$orphans"
|
||||
done
|
||||
|
||||
if ! $APPLY; then
|
||||
echo ""
|
||||
warn " DRY-RUN — ${total_orphans} orphaned manifest(s) shown, none deleted, cron not triggered."
|
||||
warn " Re-run reclaim with --apply to delete them and free disk."
|
||||
return 0
|
||||
fi
|
||||
|
||||
info " Orphaned manifests: deleted=${deleted} failed=${failed}$( [[ $unsafe -gt 0 ]] && echo " (skipped ${unsafe} pkg for safety)")"
|
||||
|
||||
# --- Step 3: GC the now-unreferenced blobs ---
|
||||
local cron_ran=false res
|
||||
step " Triggering '${CLEANUP_CRON}' cron to GC now-unreferenced blobs..."
|
||||
res="$(trigger_cleanup_cron)" && cron_ran=true || true
|
||||
case "$res" in
|
||||
ok\ *) info " cron '${CLEANUP_CRON}' triggered (HTTP ${res#ok }) — GC running."
|
||||
audit "RECLAIM triggered cron ${CLEANUP_CRON} ${res}" ;;
|
||||
denied) warn " Token lacks write:admin — cannot trigger the GC cron directly."
|
||||
note " The orphaned manifests are deleted; their unique blobs are now unreferenced and"
|
||||
note " WILL be freed by the daily '@midnight' ${CLEANUP_CRON} run, or immediately on the"
|
||||
note " next Gitea restart (RUN_AT_START is enabled). No data is lost by waiting."
|
||||
audit "RECLAIM cron-trigger-denied (no write:admin); GC deferred to schedule/restart" ;;
|
||||
*) warn " cron trigger returned: ${res}"; audit "RECLAIM cron-trigger ${res}" ;;
|
||||
esac
|
||||
|
||||
# Only a real GC run changes disk; measuring otherwise is misleading.
|
||||
if $MEASURE && $cron_ran && [[ -n "$before" ]]; then
|
||||
sleep 6
|
||||
local after; after="$(measure_bytes || true)"
|
||||
if [[ -n "$after" ]]; then
|
||||
info " packages dir after: $(human "$after")"
|
||||
info " real reclaimed ≈ $(human "$(( before - after ))") (du; Longhorn 'actual size' lags)"
|
||||
audit "RECLAIM measured before=${before}B after=${after}B freed=$(( before - after ))B"
|
||||
else
|
||||
note " (could not re-measure — re-run reclaim --measure later)"
|
||||
fi
|
||||
elif $MEASURE && ! $cron_ran; then
|
||||
note " (skipping after-measurement: GC cron did not run; measure after the scheduled run/restart)"
|
||||
fi
|
||||
}
|
||||
|
||||
# =============================================================================
|
||||
# Interactive menu (when no --repo/--all and a TTY is present)
|
||||
# =============================================================================
|
||||
interactive_menu() {
|
||||
load_versions
|
||||
local -a names=(); local n
|
||||
while IFS= read -r n; do names+=("$n"); done < <(package_names)
|
||||
if [[ "${#names[@]}" -eq 0 ]]; then warn "No container packages found for owner '${OWNER}'."; exit 0; fi
|
||||
|
||||
echo ""
|
||||
echo -e "${BOLD}Container packages under '${OWNER}':${NC}"
|
||||
local i
|
||||
for i in "${!names[@]}"; do
|
||||
printf ' %2d) %-22s %s tags, %s digest manifests\n' \
|
||||
"$((i+1))" "${names[$i]}" \
|
||||
"$(tagged_versions "${names[$i]}" | grep -c . || true)" \
|
||||
"$(digest_count "${names[$i]}")"
|
||||
done
|
||||
echo " a) all packages"
|
||||
echo ""
|
||||
read -r -p "Select package(s) [numbers/space-sep, or 'a' for all]: " sel
|
||||
local -a chosen=()
|
||||
if [[ "$sel" == "a" || "$sel" == "all" ]]; then chosen=("${names[@]}");
|
||||
else
|
||||
for tok in $sel; do
|
||||
[[ "$tok" =~ ^[0-9]+$ ]] && [[ "$tok" -ge 1 ]] && [[ "$tok" -le "${#names[@]}" ]] && chosen+=("${names[$((tok-1))]}")
|
||||
done
|
||||
fi
|
||||
[[ "${#chosen[@]}" -eq 0 ]] && { warn "Nothing selected."; exit 0; }
|
||||
REPOS=("${chosen[@]}")
|
||||
|
||||
echo ""
|
||||
echo "Action: 1) list 2) prune 3) reclaim (cleanup cron)"
|
||||
read -r -p "Choose [1-3]: " act
|
||||
case "$act" in
|
||||
1) ACTION="list" ;;
|
||||
2) ACTION="prune"
|
||||
read -r -p "Keep how many most-recent tags? [10]: " KEEP; KEEP="${KEEP:-10}"
|
||||
[[ "$KEEP" =~ ^[0-9]+$ ]] || { error "invalid number"; exit 2; }
|
||||
read -r -p "Apply for real now? (dry-run otherwise) [y/N]: " ap
|
||||
[[ "$ap" =~ ^[Yy]$ ]] && APPLY=true
|
||||
read -r -p "Also reclaim disk (run cleanup cron) after? [y/N]: " rc
|
||||
[[ "$rc" =~ ^[Yy]$ ]] && DO_RECLAIM=true
|
||||
;;
|
||||
3) ACTION="reclaim" ;;
|
||||
*) error "invalid action"; exit 2 ;;
|
||||
esac
|
||||
}
|
||||
|
||||
# =============================================================================
|
||||
# Main
|
||||
# =============================================================================
|
||||
echo ""
|
||||
info "╔══════════════════════════════════════════╗"
|
||||
info "║ Gitea container-image prune ║"
|
||||
info "╚══════════════════════════════════════════╝"
|
||||
info "Server: ${GITEA_URL} Owner: ${OWNER} Log: ${LOG_FILE}"
|
||||
|
||||
# Sanity: Gitea version (public, no auth)
|
||||
GV="$(curl -fsS "${GITEA_URL}/api/v1/version" 2>/dev/null | jq -r '.version' 2>/dev/null || echo '?')"
|
||||
info "Gitea version: ${GV}"
|
||||
[[ "$GV" == 1.26.* ]] || warn "Tested against Gitea 1.26.2 — server reports '${GV}'. Verify API shapes."
|
||||
|
||||
if $MEASURE; then detect_kubectl; [[ -n "$KUBECTL" ]] || warn "kubectl not usable — --measure will be skipped."; fi
|
||||
|
||||
# --- Selection / mode -----------------------------------------------------
|
||||
if ! $ALL && [[ "${#REPOS[@]}" -eq 0 ]]; then
|
||||
if [[ -t 0 ]]; then
|
||||
interactive_menu
|
||||
else
|
||||
error "No --repo/--all given and not a TTY (cannot show interactive menu)."
|
||||
usage 1
|
||||
fi
|
||||
else
|
||||
load_versions
|
||||
fi
|
||||
|
||||
# Resolve repo set
|
||||
declare -a TARGETS=()
|
||||
if $ALL; then
|
||||
while IFS= read -r n; do TARGETS+=("$n"); done < <(package_names)
|
||||
else
|
||||
for r in "${REPOS[@]}"; do
|
||||
if jq -rs --arg n "$r" 'any(.[]; .name==$n)' "$VERSIONS_JSON" | grep -q true; then
|
||||
TARGETS+=("$r")
|
||||
else
|
||||
warn "Package not found for owner '${OWNER}': ${r} (skipping)"
|
||||
fi
|
||||
done
|
||||
fi
|
||||
[[ "${#TARGETS[@]}" -eq 0 ]] && { error "No valid target packages."; exit 1; }
|
||||
|
||||
# Infer action: keep/older-than => prune; else list. reclaim is always explicit.
|
||||
if [[ -z "$ACTION" ]]; then
|
||||
if [[ -n "$KEEP" || -n "$OLDER_THAN" ]]; then ACTION="prune"; else ACTION="list"; fi
|
||||
fi
|
||||
# prune needs a mode
|
||||
if [[ "$ACTION" == "prune" && -z "$KEEP" && -z "$OLDER_THAN" ]]; then
|
||||
error "prune needs --keep N or --older-than DAYS."; exit 2
|
||||
fi
|
||||
|
||||
info "Action: ${ACTION} Targets: ${TARGETS[*]}"
|
||||
audit "RUN action=${ACTION} owner=${OWNER} targets='${TARGETS[*]}' apply=${APPLY} keep='${KEEP}' older_than='${OLDER_THAN}' protect='${PROTECT[*]}'"
|
||||
|
||||
case "$ACTION" in
|
||||
list)
|
||||
for t in "${TARGETS[@]}"; do do_list "$t"; done
|
||||
;;
|
||||
prune)
|
||||
# Strict confirmation for --all --apply
|
||||
if $ALL && $APPLY && ! $ASSUME_YES; then
|
||||
echo ""
|
||||
warn "You are about to PRUNE ALL ${#TARGETS[@]} packages: ${TARGETS[*]}"
|
||||
read -r -p "Type 'DELETE ALL' to confirm: " reply
|
||||
[[ "$reply" == "DELETE ALL" ]] || { error "Aborted."; exit 1; }
|
||||
ASSUME_YES=true # per-repo prompts already covered by this gate
|
||||
fi
|
||||
for t in "${TARGETS[@]}"; do do_prune_repo "$t"; done
|
||||
echo ""
|
||||
info "Prune summary: deleted=${PRUNE_DELETED} failed=${PRUNE_FAILED} (dry-run=$( $APPLY && echo no || echo yes ))"
|
||||
audit "PRUNE-SUMMARY deleted=${PRUNE_DELETED} failed=${PRUNE_FAILED} apply=${APPLY}"
|
||||
if $APPLY && $DO_RECLAIM && [[ "$PRUNE_DELETED" -gt 0 ]]; then
|
||||
echo ""; do_reclaim || true
|
||||
elif $APPLY && [[ "$PRUNE_DELETED" -gt 0 ]]; then
|
||||
note "Tags deleted, but bytes remain until reclaim. Run: $0 reclaim --measure (or pass --reclaim)"
|
||||
fi
|
||||
;;
|
||||
reclaim)
|
||||
do_reclaim || exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
echo ""
|
||||
info "Done. Audit log: ${LOG_FILE}"
|
||||
Reference in New Issue
Block a user