kubesolo-os/docs/ci-runners.md

CI Runners

KubeSolo OS is built and tested on Gitea Actions runners. This document records the runners currently in service and how to register a new one if a host is wiped.

Active runners

Name	Host	Arch	OS	Labels	Notes
odroid-arm64	odroid.local	aarch64	Ubuntu 22.04 LTS	arm64-linux, ubuntu-latest, ubuntu-24.04, ubuntu-22.04	Native ARM64 builder; 6 cores, 1.8 GB RAM + 4 GB swap; runs as systemd service act_runner

Workflow targeting

ARM64-specific jobs target the Odroid via the arm64-linux label:

jobs:
  build-arm64:
    runs-on: arm64-linux
    steps:
      - uses: actions/checkout@v4
      - run: make rootfs-arm64

Generic ubuntu jobs that don't care about arch fall through to whichever runner picks them up first; on the Odroid they run in Docker via the ubuntu-latest / ubuntu-22.04 / ubuntu-24.04 labels.

Workflows in this repo

Workflow file	Trigger	Where it runs	What it produces
.gitea/workflows/ci.yaml	push / PR to main	ubuntu-latest	Go tests, cross-arch binary build, shellcheck
.gitea/workflows/build-arm64.yaml	push to main, tags v*, manual	arm64-linux (Odroid)	ARM64 kernel + rootfs + disk image; uploads as workflow artifact only
.gitea/workflows/release.yaml	tags v*	mix: ubuntu-latest + arm64-linux	Full release: x86 ISO + disk, ARM64 disk, Go binaries, SHA256SUMS — posted to Gitea Releases via API

Release workflow specifics

release.yaml is what fires when you git push origin vX.Y.Z. The pipeline:

1. test — go test cloud-init + update modules (ubuntu-latest).
2. build-binaries — cross-compiles kubesolo-cloudinit and kubesolo-update for linux-amd64 + linux-arm64 with the version baked in via -X main.version=….
3. build-iso-amd64 — runs make iso disk-image on ubuntu-latest; produces the x86_64 ISO and a .img.xz compressed disk image.
4. build-disk-arm64 — runs the same flow on the Odroid (arm64-linux label); produces .arm64.img.xz.
5. release — downloads everything, computes SHA256SUMS, calls Gitea's POST /api/v1/repos/<owner>/<repo>/releases to create the release, then POST .../releases/<id>/assets?name=… once per asset.

Authentication uses Gitea's built-in ${{ secrets.GITHUB_TOKEN }} — the runner auto-populates that secret with repo-write scope. If your runner is configured without that automatic token (e.g. an older act_runner), generate a personal access token with repo:write scope, add it as an org secret named GITEA_TOKEN, and swap the TOKEN: ${{ secrets.GITHUB_TOKEN }} line in release.yaml for TOKEN: ${{ secrets.GITEA_TOKEN }}.

Why not the GitHub Marketplace release actions?

release.yaml used to call softprops/action-gh-release@v2. That action hard-codes calls to api.github.com instead of using ${{ github.api_url }} (which Gitea sets to its own API). On Gitea's act_runner the action fails silently — the job reports green but no release is created. We replaced it with a direct curl so the behaviour is explicit and debuggable.

Similarly, actions/upload-artifact@v4 and @download-artifact@v4 are not fully implemented by act_runner v1.0.x. Pin to @v3 until upstream support catches up.

Manually re-running a release

Releases are immutable once published, but you can:

• Delete and recreate the release through the Gitea UI on the releases/tag/vX.Y.Z page, then push the tag again (Gitea reuses the existing tag), and re-trigger the workflow via the Actions UI.
• Trigger the build-arm64 workflow manually for a one-off arm64 artifact: Gitea UI → Actions → ARM64 Build → Run workflow.

Don't force-update a published tag — anyone who already fetched it (or downloaded an asset) sees a checksum mismatch. Prefer cutting a new patch release (vX.Y.Z+1) over rewriting a published one.

Registering a new runner

Prerequisites

• Linux host (Ubuntu / Debian preferred; the install instructions below use Ubuntu 22.04+ paths).
• Outbound HTTPS to the Gitea instance.
• Root access on the runner host (the runner needs to create loop devices and run mkfs.ext4 for disk-image builds).
• A Gitea Actions runner registration token. Get it from:
  • Repo-scoped: <repo>/settings/actions/runners → "Create new Runner"
  • Org-scoped (preferred for this project): <org>/-/settings/actions/runners → "Create new Runner"
  • Site-scoped: /-/admin/actions/runners → "Create new Runner"

Step 1 — Add swap if the host has <4 GB RAM

Kernel builds in later phases need ~2 GB resident; tight hosts will OOM-kill cc1 without swap.

sudo fallocate -l 4G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab

Step 2 — Install the gitea-runner binary

Pinned to a known-good version. Check https://gitea.com/gitea/runner/releases for the current stable tag before bumping.

sudo -i
mkdir -p /opt/act_runner && cd /opt/act_runner

# Bump VERSION to the current stable release as needed
VERSION=1.0.3
ARCH=$(uname -m | sed 's/aarch64/arm64/; s/x86_64/amd64/')

curl -fL "https://gitea.com/gitea/runner/releases/download/v${VERSION}/gitea-runner-${VERSION}-linux-${ARCH}" \
  -o act_runner
chmod +x act_runner
./act_runner --version

The upstream project was renamed act_runner → gitea-runner at the v1.0.0 release. The release asset filenames use gitea-runner-* even though we keep the local binary named act_runner to match this systemd unit. The CLI surface (register, daemon, generate-config) is unchanged.

Step 3 — Register against Gitea

./act_runner register --no-interactive \
  --instance https://git.oe74.net \
  --token PASTE_TOKEN_HERE \
  --name <hostname> \
  --labels arm64-linux        # adjust label for amd64 hosts

This creates a .runner file with the registration credentials.

Step 4 — Generate and tune config

./act_runner generate-config > config.yaml

In config.yaml, confirm the runner.labels: block includes the labels you want. The :host suffix routes jobs directly to the host (no Docker wrapper) — required for disk-image builds that need loop devices and mkfs.

Example labels for an arm64 host:

runner:
  labels:
    - "arm64-linux:host"
    - "ubuntu-latest:docker://docker.gitea.com/runner-images:ubuntu-latest"
    - "ubuntu-24.04:docker://docker.gitea.com/runner-images:ubuntu-24.04"
    - "ubuntu-22.04:docker://docker.gitea.com/runner-images:ubuntu-22.04"

Step 5 — Install as a systemd service

cat > /etc/systemd/system/act_runner.service << 'EOF'
[Unit]
Description=Gitea Actions runner
After=network-online.target
Wants=network-online.target

[Service]
ExecStart=/opt/act_runner/act_runner daemon --config /opt/act_runner/config.yaml
WorkingDirectory=/opt/act_runner
User=root
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl enable --now act_runner
systemctl status act_runner --no-pager

Step 6 — Verify in Gitea UI

Visit the runners page at the scope you registered against. The runner should appear as Idle with the labels you configured.

Removing a runner

On the host:

systemctl disable --now act_runner
rm -rf /opt/act_runner /etc/systemd/system/act_runner.service
systemctl daemon-reload

Then delete the runner entry from the Gitea Actions UI so Gitea stops trying to schedule against it.

Operational notes

• The runner stores in-progress job working directories under /tmp/act_runner by default. Large disk-image builds may need that path moved to a larger volume — edit host.workdir_parent: in config.yaml.
• Logs are visible via journalctl -u act_runner -f.
• If a job is interrupted (e.g. host reboot mid-build), the Gitea UI will mark it as failed/cancelled. Re-run from the Actions UI.