restructured files, and updated for build outside

This commit is contained in:
2026-02-13 19:10:35 +01:00
parent f72bb57eda
commit e9dde0a340
14 changed files with 347 additions and 151 deletions
+5
View File
@@ -30,3 +30,8 @@ vendor/
# Local config (don't commit real customer configs)
controller.yaml
restic-password
# App assets (synced from felhom.eu website at build time)
assets/*.svg
assets/*.png
assets/*.webp
+120 -151
View File
@@ -1,160 +1,121 @@
# Building & Publishing felhom-controller
## Prerequisites
## Repository layout
- Docker installed on your build machine
- Docker Buildx plugin (for multi-arch builds — included with Docker Desktop, may need install on Linux)
- Access to Gitea at gitea.dooplex.hu
Source lives in the `controller/` subfolder of the deploy-felhom-compose repo:
```
https://gitea.dooplex.hu/admin/deploy-felhom-compose
└── controller/
├── cmd/controller/main.go
├── internal/
│ ├── config/config.go
│ ├── stacks/{manager,metadata,deploy}.go
│ ├── api/router.go
│ └── web/{server,templates}.go
├── configs/
├── scripts/
├── assets/ ← logos + screenshots (gitignored, synced at build)
├── Dockerfile
├── go.mod
└── Makefile
```
**Important:** Go files must be in their package subdirectories. If all `.go` files
are flat in `controller/`, run the restructure script first (see Step 0).
## Build setup
Git repo: `/home/kisfenyo/git/deploy-felhom-compose`
Build dir: `/home/kisfenyo/build/felhom-controller/` (outside the repo — keeps it clean)
```bash
# One-time: create build directory and copy the build script
mkdir -p ~/build/felhom-controller
cp build.sh ~/build/felhom-controller/
chmod +x ~/build/felhom-controller/build.sh
```
## Step 0: Fix directory structure (one-time, if files are flat)
If all `.go` files are in `controller/` without subdirectories:
```bash
cd ~/git/deploy-felhom-compose/controller
bash restructure.sh
# Verify it compiles
go mod tidy && go build ./cmd/controller/
# Commit the restructured layout
git add -A
git commit -m "Restructure controller into Go package directories"
git push
# Clean up
rm restructure.sh
```
## Step 1: Enable Gitea Container Registry
Gitea has a built-in container registry. Check if it's enabled:
Check if Gitea's package registry is enabled:
```bash
# SSH into your k3s node or wherever Gitea runs
# Check Gitea config (app.ini)
# Look for [packages] section in Gitea config
kubectl exec -it -n <namespace> deploy/gitea -- cat /data/gitea/conf/app.ini | grep -A5 '\[packages\]'
```
If the `[packages]` section is missing or `ENABLED=false`, add/update:
If missing or `ENABLED=false`, add to `app.ini`:
```ini
[packages]
ENABLED = true
```
Then restart Gitea:
Restart Gitea, then verify:
```bash
kubectl rollout restart deploy/gitea -n <namespace>
```
**Verify it works:**
```bash
# Login to the registry (use your Gitea username + password or access token)
docker login gitea.dooplex.hu
# Username: admin
# Password: <your-gitea-password-or-token>
```
If login succeeds, the registry is working. The image URL pattern is:
`gitea.dooplex.hu/<owner>/<image>:<tag>`
## Step 2: Sync app assets before building
The container image includes app logos and screenshots. Sync them from
the felhom.eu website repo before building:
## Step 2: Build the image
```bash
# If the website repo is checked out alongside this repo:
make sync-assets
cd ~/build/felhom-controller
# Or specify the path explicitly:
make sync-assets WEBSITE_ASSETS_DIR=/home/admin/repos/felhom.eu/website/assets
# Local build (quick test, current arch only)
./build.sh 0.1.0
# Verify
ls assets/*.webp
# Build + push to Gitea registry
./build.sh 0.1.0 --push
# Build for both amd64 (N100) + arm64 (Pi) and push
./build.sh 0.1.0 --multiarch
```
If you skip this step, the dashboard will work but show no app logos/screenshots
(the `onerror` handler hides broken images gracefully).
### What the build script does
## Step 3: Build for single architecture (quick test)
1. Copies `controller/` from the git repo into `workspace/` (temp directory)
2. Verifies Go package directory structure
3. Syncs app assets (logos, screenshots) from `felhom.eu/website/assets/`
4. Runs `go mod tidy` if Go is installed locally
5. Runs `docker build` (or `docker buildx build` for multi-arch)
Build artifacts live in `~/build/felhom-controller/workspace/` — your git repo stays clean.
### Configuration
Edit the top of `build.sh` if your paths differ:
```bash
cd ~/repos/deploy-felhom-compose
# Build for your current architecture
docker build \
--build-arg VERSION=$(git describe --tags --always 2>/dev/null || echo "0.1.0") \
--build-arg GIT_COMMIT=$(git rev-parse --short HEAD 2>/dev/null || echo "unknown") \
-t gitea.dooplex.hu/admin/felhom-controller:latest \
-t gitea.dooplex.hu/admin/felhom-controller:0.1.0 \
.
# Push
docker push gitea.dooplex.hu/admin/felhom-controller:latest
docker push gitea.dooplex.hu/admin/felhom-controller:0.1.0
REPO_DIR="/home/kisfenyo/git/deploy-felhom-compose"
CONTROLLER_SRC="${REPO_DIR}/controller"
WEBSITE_ASSETS_DIR="/home/kisfenyo/git/felhom.eu/website/assets"
```
## Step 4: Build for both architectures (production)
You need both amd64 (N100 mini PCs) and arm64 (Raspberry Pi). Use Docker Buildx:
## Step 3: Deploy on a customer node
```bash
# One-time: Create a buildx builder that supports multi-arch
docker buildx create --name felhom-builder --use --bootstrap
# Verify it supports the architectures we need
docker buildx inspect felhom-builder
# Should show: linux/amd64, linux/arm64 in Platforms
# Build + push multi-arch image in one step
docker buildx build \
--platform linux/amd64,linux/arm64 \
--build-arg VERSION=0.1.0 \
--build-arg GIT_COMMIT=$(git rev-parse --short HEAD) \
-t gitea.dooplex.hu/admin/felhom-controller:latest \
-t gitea.dooplex.hu/admin/felhom-controller:0.1.0 \
--push \
.
```
**Note:** `--push` is required with multi-arch builds because buildx doesn't store
multi-platform images in the local Docker cache. It pushes directly to the registry.
### If buildx multi-arch doesn't work (missing QEMU)
On Linux you might need QEMU for cross-compilation:
```bash
# Install QEMU user-mode emulation
sudo apt-get install -y qemu-user-static binfmt-support
# Register QEMU with Docker
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
# Verify
docker buildx ls
```
### Alternative: Build natively on each architecture
If you don't want to cross-compile, build on each machine:
```bash
# On the N100 (amd64):
docker build -t gitea.dooplex.hu/admin/felhom-controller:latest-amd64 .
docker push gitea.dooplex.hu/admin/felhom-controller:latest-amd64
# On the Pi (arm64):
docker build -t gitea.dooplex.hu/admin/felhom-controller:latest-arm64 .
docker push gitea.dooplex.hu/admin/felhom-controller:latest-arm64
# Then create a manifest list to combine them:
docker manifest create gitea.dooplex.hu/admin/felhom-controller:latest \
gitea.dooplex.hu/admin/felhom-controller:latest-amd64 \
gitea.dooplex.hu/admin/felhom-controller:latest-arm64
docker manifest push gitea.dooplex.hu/admin/felhom-controller:latest
```
## Step 5: Deploy on a customer node
On the customer's machine, the docker-compose.yml for the controller references
the image from the registry:
```yaml
services:
felhom-controller:
image: gitea.dooplex.hu/admin/felhom-controller:latest
# ...
```
Pull and start:
```bash
# Login to registry on the customer node (one-time)
# Login to registry (one-time)
docker login gitea.dooplex.hu
# Start the controller
@@ -167,60 +128,68 @@ docker compose logs -f
curl -s http://localhost:8080/api/health
```
## Step 6: Updating the controller
When you release a new version:
## Step 4: Updating the controller
```bash
# On your build machine:
docker buildx build \
--platform linux/amd64,linux/arm64 \
--build-arg VERSION=0.2.0 \
-t gitea.dooplex.hu/admin/felhom-controller:latest \
-t gitea.dooplex.hu/admin/felhom-controller:0.2.0 \
--push .
# On build machine: build new version
cd ~/build/felhom-controller
./build.sh 0.2.0 --push # or --multiarch
# On the customer node (manually for now; auto-update comes in Phase 5):
# On customer node: pull and restart
cd /opt/docker/felhom-controller
docker compose pull
docker compose up -d
```
## Makefile shortcuts
## Multi-arch notes
The Makefile has convenience targets:
### QEMU setup (if cross-compiling on amd64)
```bash
make docker-build # Build for current platform
make docker-buildx # Build multi-arch + push
make docker-push # Push current platform image
sudo apt-get install -y qemu-user-static binfmt-support
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
docker buildx ls # should show linux/amd64, linux/arm64
```
### Alternative: build natively on each architecture
```bash
# On N100 (amd64):
./build.sh 0.1.0 --push
# On Pi (arm64):
./build.sh 0.1.0 --push
# Then create manifest:
docker manifest create gitea.dooplex.hu/admin/felhom-controller:latest \
gitea.dooplex.hu/admin/felhom-controller:latest-amd64 \
gitea.dooplex.hu/admin/felhom-controller:latest-arm64
docker manifest push gitea.dooplex.hu/admin/felhom-controller:latest
```
## Troubleshooting
### "unauthorized" when pushing
```bash
docker logout gitea.dooplex.hu
docker login gitea.dooplex.hu
docker logout gitea.dooplex.hu && docker login gitea.dooplex.hu
```
### Gitea registry not accessible
Check if Gitea's HTTPS is working and the domain resolves:
### Gitea registry not reachable
```bash
curl -v https://gitea.dooplex.hu/v2/
# Should return: {"errors":[{"code":"UNAUTHORIZED",...}]}
```
### Build fails on arm64 via QEMU (too slow or errors)
Cross-compiling Go via QEMU can be slow. Since the Go binary itself is cross-compiled
(CGO_ENABLED=0), only the Debian packages in the runtime stage need QEMU.
Alternative: build the Go binary natively, then build only the runtime Docker layer
via buildx.
### Structure check fails
```bash
# The build script verifies package dirs exist. If it fails:
cd ~/git/deploy-felhom-compose/controller
ls -la cmd/controller/ internal/config/ internal/stacks/ internal/api/ internal/web/
# If these don't exist, run restructure.sh first
```
### Image too large
Expected sizes:
### Expected image size
- Go binary: ~15-20 MB
- Runtime image (debian-slim + docker-cli + restic + pg_dump): ~250-350 MB
To reduce: consider Alpine instead of Debian slim, but test pg_dump/mysqldump
compatibility first.
- Runtime (debian-slim + docker-cli + restic + pg_dump): ~250-350 MB
- Assets (logos + screenshots): ~20-30 MB
- **Total: ~300-400 MB**
+222
View File
@@ -0,0 +1,222 @@
#!/usr/bin/env bash
# =============================================================================
# felhom-controller — Docker image build script
# =============================================================================
# Location: /home/kisfenyo/build/felhom-controller/build.sh
#
# Copies source from the git repo, syncs app assets, and builds the image.
# Build artifacts stay here — the git repo stays clean.
#
# Usage:
# ./build.sh # Build for current platform, tag as :dev
# ./build.sh 0.1.0 # Build with version tag
# ./build.sh 0.1.0 --push # Build + push to Gitea registry
# ./build.sh 0.1.0 --multiarch # Build amd64+arm64 + push
# =============================================================================
set -euo pipefail
# --- Configuration (edit these if your paths differ) ---
REPO_DIR="/home/kisfenyo/git/deploy-felhom-compose"
CONTROLLER_SRC="${REPO_DIR}/controller"
WEBSITE_ASSETS_DIR="/home/kisfenyo/git/felhom.eu/website/assets"
REGISTRY="gitea.dooplex.hu/admin"
IMAGE="${REGISTRY}/felhom-controller"
# Build workspace — a temp directory next to this script, NOT in the git repo
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
BUILD_DIR="${SCRIPT_DIR}/workspace"
# --- Parse arguments ---
VERSION="${1:-dev}"
ACTION="${2:-}" # --push or --multiarch
# --- Colors ---
RED='\033[0;31m'; GREEN='\033[0;32m'; YELLOW='\033[1;33m'; CYAN='\033[0;36m'; NC='\033[0m'
info() { echo -e "${GREEN}[INFO]${NC} $*"; }
warn() { echo -e "${YELLOW}[WARN]${NC} $*"; }
error() { echo -e "${RED}[ERROR]${NC} $*"; }
step() { echo -e "${CYAN}[STEP]${NC} $*"; }
# --- Pre-flight checks ---
if [[ ! -d "${CONTROLLER_SRC}" ]]; then
error "Controller source not found: ${CONTROLLER_SRC}"
error "Clone the repo first: git clone https://gitea.dooplex.hu/admin/deploy-felhom-compose.git ${REPO_DIR}"
exit 1
fi
if ! command -v docker &>/dev/null; then
error "Docker not found."
exit 1
fi
# Git metadata from the repo
GIT_COMMIT="unknown"
if [[ -d "${REPO_DIR}/.git" ]]; then
GIT_COMMIT=$(cd "${REPO_DIR}" && git rev-parse --short HEAD 2>/dev/null || echo "unknown")
fi
echo ""
info "╔══════════════════════════════════════╗"
info "║ felhom-controller image builder ║"
info "╚══════════════════════════════════════╝"
info "Version: ${VERSION}"
info "Commit: ${GIT_COMMIT}"
info "Source: ${CONTROLLER_SRC}"
info "Build dir: ${BUILD_DIR}"
info "Image: ${IMAGE}:${VERSION}"
echo ""
# =========================================================================
# Step 1: Prepare clean build workspace
# =========================================================================
step "1/4 — Preparing build workspace..."
rm -rf "${BUILD_DIR}"
mkdir -p "${BUILD_DIR}"
# Copy the entire controller directory (preserving subdirectory structure)
cp -a "${CONTROLLER_SRC}/." "${BUILD_DIR}/"
# Verify the expected Go package structure exists
MISSING=()
for required in cmd/controller/main.go internal/config/config.go internal/stacks/manager.go internal/api/router.go internal/web/server.go; do
if [[ ! -f "${BUILD_DIR}/${required}" ]]; then
MISSING+=("${required}")
fi
done
if [[ ${#MISSING[@]} -gt 0 ]]; then
error "Directory structure check FAILED. Missing files:"
for f in "${MISSING[@]}"; do
error "${f}"
done
error ""
error "The Go source must be in package subdirectories, not flat."
error "Run the restructure script first:"
error " cd ${CONTROLLER_SRC} && bash restructure.sh"
error " git add -A && git commit -m 'Restructure into Go package dirs'"
rm -rf "${BUILD_DIR}"
exit 1
fi
GO_COUNT=$(find "${BUILD_DIR}" -name '*.go' | wc -l)
info "Source copied: ${GO_COUNT} Go files"
# =========================================================================
# Step 2: Sync app assets (logos + screenshots)
# =========================================================================
step "2/4 — Syncing app assets..."
mkdir -p "${BUILD_DIR}/assets"
if [[ -d "${WEBSITE_ASSETS_DIR}" ]]; then
# Count before copy
SVG_N=$(find "${WEBSITE_ASSETS_DIR}" -maxdepth 1 -name '*-logo.svg' 2>/dev/null | wc -l)
PNG_N=$(find "${WEBSITE_ASSETS_DIR}" -maxdepth 1 -name '*-logo.png' 2>/dev/null | wc -l)
SS_N=$(find "${WEBSITE_ASSETS_DIR}" -maxdepth 1 -name '*-screenshot-*.webp' 2>/dev/null | wc -l)
cp "${WEBSITE_ASSETS_DIR}"/*-logo.svg "${BUILD_DIR}/assets/" 2>/dev/null || true
cp "${WEBSITE_ASSETS_DIR}"/*-logo.png "${BUILD_DIR}/assets/" 2>/dev/null || true
cp "${WEBSITE_ASSETS_DIR}"/*-screenshot-*.webp "${BUILD_DIR}/assets/" 2>/dev/null || true
TOTAL=$(find "${BUILD_DIR}/assets" -type f ! -name 'README.md' 2>/dev/null | wc -l)
info "Synced ${TOTAL} assets (${SVG_N} SVG + ${PNG_N} PNG logos, ${SS_N} screenshots)"
else
warn "Website assets not found: ${WEBSITE_ASSETS_DIR}"
warn "Building without logos/screenshots. Set WEBSITE_ASSETS_DIR if needed."
fi
# =========================================================================
# Step 3: Run go mod tidy (to generate go.sum if missing)
# =========================================================================
step "3/4 — Preparing Go modules..."
cd "${BUILD_DIR}"
# The Dockerfile handles go mod download internally, but verify go.mod exists
if [[ ! -f "go.mod" ]]; then
error "go.mod not found in build workspace!"
exit 1
fi
# If go is installed locally, run tidy to catch issues early
if command -v go &>/dev/null; then
info "Running go mod tidy..."
go mod tidy 2>&1 || warn "go mod tidy had issues (Docker build may still work)"
else
info "Go not installed locally — Docker build stage will handle dependencies."
fi
# =========================================================================
# Step 4: Docker build
# =========================================================================
step "4/4 — Building Docker image..."
BUILD_ARGS=(
--build-arg "VERSION=${VERSION}"
--build-arg "GIT_COMMIT=${GIT_COMMIT}"
)
case "${ACTION}" in
--push)
info "Building for current platform + pushing..."
docker build "${BUILD_ARGS[@]}" \
-t "${IMAGE}:${VERSION}" \
-t "${IMAGE}:latest" \
.
info "Pushing..."
docker push "${IMAGE}:${VERSION}"
docker push "${IMAGE}:latest"
;;
--multiarch)
info "Building multi-arch (amd64 + arm64) + pushing..."
# Ensure buildx builder exists
if ! docker buildx inspect felhom-builder &>/dev/null; then
info "Creating buildx builder (one-time setup)..."
docker buildx create --name felhom-builder --use --bootstrap
else
docker buildx use felhom-builder
fi
docker buildx build "${BUILD_ARGS[@]}" \
--platform linux/amd64,linux/arm64 \
-t "${IMAGE}:${VERSION}" \
-t "${IMAGE}:latest" \
--push \
.
;;
*)
info "Building for current platform (local only)..."
docker build "${BUILD_ARGS[@]}" \
-t "${IMAGE}:${VERSION}" \
-t "${IMAGE}:latest" \
.
;;
esac
# =========================================================================
# Summary
# =========================================================================
echo ""
info "╔══════════════════════════════════════╗"
info "║ Build complete ✓ ║"
info "╚══════════════════════════════════════╝"
info "Image: ${IMAGE}:${VERSION}"
# Show image size if available locally
SIZE=$(docker image inspect "${IMAGE}:${VERSION}" --format='{{.Size}}' 2>/dev/null || echo "")
if [[ -n "${SIZE}" ]]; then
SIZE_HUMAN=$(numfmt --to=iec "${SIZE}" 2>/dev/null || echo "${SIZE} bytes")
info "Size: ${SIZE_HUMAN}"
fi
echo ""
if [[ "${ACTION}" == "" ]]; then
info "Image is local only. To push:"
info " ./build.sh ${VERSION} --push # current arch"
info " ./build.sh ${VERSION} --multiarch # amd64 + arm64"
fi