15 Commits

Author SHA1 Message Date
4e3f1d6cf0 fix: use kernel-built DTBs for RPi SD card driver probe
Some checks failed
CI / Go Tests (push) Has been cancelled
CI / Build Go Binaries (amd64, linux, linux-amd64) (push) Has been cancelled
CI / Build Go Binaries (arm64, linux, linux-arm64) (push) Has been cancelled
CI / Shellcheck (push) Has been cancelled
Release / Test (push) Has been cancelled
Release / Build Binaries (amd64, linux, linux-amd64) (push) Has been cancelled
Release / Build Binaries (arm64, linux, linux-arm64) (push) Has been cancelled
Release / Build ISO (amd64) (push) Has been cancelled
Release / Create Release (push) Has been cancelled
The sdhci-iproc driver (RPi 4 SD card controller) probes via Device
Tree matching. Using DTBs from the firmware repo instead of the
kernel build caused a mismatch — the driver silently failed to probe,
resulting in zero block devices after boot.

Changes:
- Use DTBs from custom-kernel-arm64/dtbs/ (matches the kernel)
- Firmware blobs (start4.elf, fixup4.dat) still from firmware repo
- Also includes prior fix for LABEL= resolution in persistent mount

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 19:27:54 -06:00
6ff77c4482 fix: resolve LABEL= syntax for RPi data partition
Some checks failed
CI / Go Tests (push) Has been cancelled
CI / Build Go Binaries (amd64, linux, linux-amd64) (push) Has been cancelled
CI / Build Go Binaries (arm64, linux, linux-arm64) (push) Has been cancelled
Release / Test (push) Has been cancelled
CI / Shellcheck (push) Has been cancelled
Release / Build Binaries (amd64, linux, linux-amd64) (push) Has been cancelled
Release / Build Binaries (arm64, linux, linux-arm64) (push) Has been cancelled
Release / Build ISO (amd64) (push) Has been cancelled
Release / Create Release (push) Has been cancelled
The cmdline uses kubesolo.data=LABEL=KSOLODATA, but the wait loop
in 20-persistent-mount.sh checked [ -b "LABEL=KSOLODATA" ] which
is always false — it's a label reference, not a block device path.

Fix by detecting LABEL= prefix and resolving it to a block device
path via blkid -L in the wait loop. Also loads mmc_block module as
fallback for platforms where it's not built-in.

Adds debug output listing available block devices on failure.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 19:05:10 -06:00
a2764218fc fix: make RPi partition 1 self-sufficient boot fallback
Some checks failed
CI / Go Tests (push) Has been cancelled
CI / Build Go Binaries (amd64, linux, linux-amd64) (push) Has been cancelled
CI / Build Go Binaries (arm64, linux, linux-arm64) (push) Has been cancelled
CI / Shellcheck (push) Has been cancelled
Release / Test (push) Has been cancelled
Release / Build Binaries (amd64, linux, linux-amd64) (push) Has been cancelled
Release / Build Binaries (arm64, linux, linux-arm64) (push) Has been cancelled
Release / Build ISO (amd64) (push) Has been cancelled
Release / Create Release (push) Has been cancelled
The autoboot.txt A/B redirect requires newer RPi EEPROM firmware.
On older EEPROMs, autoboot.txt is silently ignored and the firmware
tries to boot from partition 1 directly — failing with a rainbow
screen because partition 1 had no kernel or initramfs.

Changes:
- Increase partition 1 from 32 MB to 384 MB
- Populate partition 1 with full boot files (kernel, initramfs,
  config.txt with kernel= directive, DTBs, overlays)
- Keep autoboot.txt for A/B redirect on supported EEPROMs
- When autoboot.txt works: boots from partition 2 (A/B scheme)
- When autoboot.txt is unsupported: boots from partition 1 (fallback)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 18:52:21 -06:00
2ba816bf6e fix: add config.txt and DTBs to RPi boot control partition
Some checks failed
CI / Go Tests (push) Has been cancelled
CI / Build Go Binaries (amd64, linux, linux-amd64) (push) Has been cancelled
CI / Build Go Binaries (arm64, linux, linux-arm64) (push) Has been cancelled
CI / Shellcheck (push) Has been cancelled
Release / Test (push) Has been cancelled
Release / Build Binaries (amd64, linux, linux-amd64) (push) Has been cancelled
Release / Build Binaries (arm64, linux, linux-arm64) (push) Has been cancelled
Release / Build ISO (amd64) (push) Has been cancelled
Release / Create Release (push) Has been cancelled
The Raspberry Pi firmware reads config.txt from partition 1 BEFORE
processing autoboot.txt. Without arm_64bit=1 on the boot control
partition, the firmware defaults to 32-bit mode and shows only a
rainbow square. Add minimal config.txt, device tree blobs, and
overlays to partition 1 so the firmware can initialize correctly
before redirecting to the A/B boot partitions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 18:29:28 -06:00
65dcddb47e fix: RPi image uses MBR and firmware on boot partition
Some checks failed
CI / Go Tests (push) Has been cancelled
CI / Build Go Binaries (amd64, linux, linux-amd64) (push) Has been cancelled
CI / Build Go Binaries (arm64, linux, linux-arm64) (push) Has been cancelled
CI / Shellcheck (push) Has been cancelled
Release / Test (push) Has been cancelled
Release / Build Binaries (amd64, linux, linux-amd64) (push) Has been cancelled
Release / Build Binaries (arm64, linux, linux-arm64) (push) Has been cancelled
Release / Build ISO (amd64) (push) Has been cancelled
Release / Create Release (push) Has been cancelled
- Switch from GPT to MBR (dos) partition table — GPT + autoboot.txt
  fails on many Pi 4 EEPROM versions
- Copy firmware blobs (start*.elf, fixup*.dat) to partition 1 (KSOLOCTL)
  so the EEPROM can find and load them
- Increase boot control partition from 16 MB to 32 MB to fit firmware
- Mark partition 1 as bootable

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 18:16:34 -06:00
ba4812f637 fix: complete ARM64 RPi build pipeline
Some checks failed
CI / Go Tests (push) Has been cancelled
CI / Build Go Binaries (amd64, linux, linux-amd64) (push) Has been cancelled
CI / Build Go Binaries (arm64, linux, linux-arm64) (push) Has been cancelled
CI / Shellcheck (push) Has been cancelled
Release / Test (push) Has been cancelled
Release / Build Binaries (amd64, linux, linux-amd64) (push) Has been cancelled
Release / Build Binaries (arm64, linux, linux-arm64) (push) Has been cancelled
Release / Build ISO (amd64) (push) Has been cancelled
Release / Create Release (push) Has been cancelled
- fetch-components.sh: download ARM64 KubeSolo binary (kubesolo-arm64)
- inject-kubesolo.sh: use arch-specific binaries for KubeSolo, cloud-init,
  and update agent; detect KVER from custom kernel when rootfs has none;
  cross-arch module resolution via find fallback when modprobe fails
- create-rpi-image.sh: kpartx support for Docker container builds
- Makefile: rootfs-arm64 depends on build-cross, includes pack-initramfs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 17:20:04 -06:00
09dcea84ef fix: disk image build, piCore64 URL, license
Some checks failed
CI / Go Tests (push) Has been cancelled
CI / Build Go Binaries (amd64, linux, linux-amd64) (push) Has been cancelled
CI / Build Go Binaries (arm64, linux, linux-arm64) (push) Has been cancelled
CI / Shellcheck (push) Has been cancelled
Release / Test (push) Has been cancelled
Release / Build Binaries (amd64, linux, linux-amd64) (push) Has been cancelled
Release / Build Binaries (arm64, linux, linux-arm64) (push) Has been cancelled
Release / Build ISO (amd64) (push) Has been cancelled
Release / Create Release (push) Has been cancelled
- Add kpartx for reliable loop partition mapping in Docker containers
- Fix piCore64 download URL (changed from .img.gz to .zip format)
- Fix piCore64 boot partition mount (initramfs on p1, not p2)
- Fix tar --wildcards for RPi firmware extraction
- Add MIT license (same as KubeSolo)
- Add kpartx and unzip to Docker builder image

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 17:05:03 -06:00
a4e719ba0e chore: bump version to 0.2.0
Some checks failed
CI / Go Tests (push) Has been cancelled
CI / Build Go Binaries (amd64, linux, linux-amd64) (push) Has been cancelled
CI / Build Go Binaries (arm64, linux, linux-arm64) (push) Has been cancelled
CI / Shellcheck (push) Has been cancelled
Release / Test (push) Has been cancelled
Release / Build Binaries (amd64, linux, linux-amd64) (push) Has been cancelled
Release / Build Binaries (arm64, linux, linux-arm64) (push) Has been cancelled
Release / Build ISO (amd64) (push) Has been cancelled
Release / Create Release (push) Has been cancelled
Includes cloud-init full flag support, security hardening, AppArmor,
and ARM64 Raspberry Pi support.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 16:36:05 -06:00
61bd28c692 feat: cloud-init supports all documented KubeSolo CLI flags
Some checks failed
CI / Go Tests (push) Has been cancelled
CI / Build Go Binaries (amd64, linux, linux-amd64) (push) Has been cancelled
CI / Build Go Binaries (arm64, linux, linux-arm64) (push) Has been cancelled
CI / Shellcheck (push) Has been cancelled
Add missing flags (--local-storage-shared-path, --debug, --pprof-server,
--portainer-edge-id, --portainer-edge-key, --portainer-edge-async) so all
10 documented KubeSolo parameters can be configured via cloud-init YAML.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 15:49:31 -06:00
4fc078f7a3 fix: kubeconfig server accessible via port forwarding, integration tests use proper auth
Some checks failed
CI / Go Tests (push) Has been cancelled
CI / Build Go Binaries (amd64, linux, linux-amd64) (push) Has been cancelled
CI / Build Go Binaries (arm64, linux, linux-arm64) (push) Has been cancelled
CI / Shellcheck (push) Has been cancelled
Bind kubeconfig HTTP server to 0.0.0.0:8080 (was 127.0.0.1) so integration
tests can reach it via QEMU SLIRP port forwarding. Add shared wait_for_boot
and fetch_kubeconfig helpers to qemu-helpers.sh. Update all 5 integration
tests to fetch kubeconfig via HTTP and use it for kubectl authentication.

All 6 tests pass on Linux with KVM: boot (18s), security (7/7), K8s ready
(15s), workload deploy, local storage, network policy.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 15:25:32 -06:00
6c15ba7776 fix: kernel AppArmor 2-pass olddefconfig and QEMU test direct kernel boot
Some checks failed
CI / Go Tests (push) Has been cancelled
CI / Build Go Binaries (amd64, linux, linux-amd64) (push) Has been cancelled
CI / Build Go Binaries (arm64, linux, linux-arm64) (push) Has been cancelled
CI / Shellcheck (push) Has been cancelled
The stock TinyCore kernel config has "# CONFIG_SECURITY is not set" which
caused make olddefconfig to silently revert all security configs in a single
pass. Fix by applying security configs (AppArmor, Audit, LSM) after the
first olddefconfig resolves base dependencies, then running a second pass.
Added mandatory verification that exits on missing critical configs.

All QEMU test scripts converted from broken -cdrom + -append pattern to
direct kernel boot (-kernel + -initrd) via shared test/lib/qemu-helpers.sh
helper library. The -append flag only works with -kernel, not -cdrom.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 14:11:38 -06:00
958524e6d8 fix: Go version, test scripts, and shellcheck warnings from validation
Some checks failed
CI / Go Tests (push) Has been cancelled
CI / Build Go Binaries (amd64, linux, linux-amd64) (push) Has been cancelled
CI / Build Go Binaries (arm64, linux, linux-arm64) (push) Has been cancelled
CI / Shellcheck (push) Has been cancelled
- Dockerfile.builder: Go 1.24.0 → 1.25.5 (go.mod requires it)
- test-boot.sh: use direct kernel boot via ISO extraction instead of
  broken -cdrom + -append; fix boot marker to "KubeSolo is running"
  (Stage 90 blocks on wait, never emits "complete")
- test-security-hardening.sh: same direct kernel boot and marker fixes
- run-vm.sh, dev-vm.sh, dev-vm-arm64.sh: quote QEMU -net args to
  silence shellcheck SC2054
- fetch-components.sh, fetch-rpi-firmware.sh, dev-vm-arm64.sh: fix
  trap quoting (SC2064)

Validated: full Docker build, 94 Go tests pass, QEMU boot (73s),
security hardening test (6/6 pass, 1 AppArmor skip pending kernel
rebuild).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 13:30:55 -06:00
efc7f80b65 feat: add security hardening, AppArmor, and ARM64 Raspberry Pi support (Phase 6)
Some checks failed
CI / Go Tests (push) Has been cancelled
CI / Build Go Binaries (amd64, linux, linux-amd64) (push) Has been cancelled
CI / Build Go Binaries (arm64, linux, linux-arm64) (push) Has been cancelled
CI / Shellcheck (push) Has been cancelled
Security hardening: bind kubeconfig server to localhost, mount hardening
(noexec/nosuid/nodev on tmpfs), sysctl network hardening, kernel module
loading lock after boot, SHA256 checksum verification for downloads,
kernel AppArmor + Audit support, complain-mode AppArmor profiles for
containerd and kubelet, and security integration test.

ARM64 Raspberry Pi support: piCore64 base extraction, RPi kernel build
from raspberrypi/linux fork, RPi firmware fetch, SD card image with 4-
partition GPT and tryboot A/B mechanism, BootEnv Go interface abstracting
GRUB vs RPi boot environments, architecture-aware build scripts, QEMU
aarch64 dev VM and boot test.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 13:08:17 -06:00
7abf0e0c04 build: add TINYCORE-MODIFICATIONS.md to .gitignore
Some checks failed
CI / Go Tests (push) Has been cancelled
CI / Build Go Binaries (amd64, linux, linux-amd64) (push) Has been cancelled
CI / Build Go Binaries (arm64, linux, linux-arm64) (push) Has been cancelled
CI / Shellcheck (push) Has been cancelled
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 11:38:01 -06:00
60d0edaf84 docs: update README with kubeconfig retrieval and Portainer Edge usage
Some checks failed
CI / Go Tests (push) Has been cancelled
CI / Build Go Binaries (amd64, linux, linux-amd64) (push) Has been cancelled
CI / Build Go Binaries (arm64, linux, linux-arm64) (push) Has been cancelled
CI / Shellcheck (push) Has been cancelled
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 10:50:44 -06:00
58 changed files with 3406 additions and 243 deletions

3
.gitignore vendored
View File

@@ -23,3 +23,6 @@ Thumbs.db
# Go
update/update-agent
cloud-init/cloud-init-parser
# Local docs (not tracked)
TINYCORE-MODIFICATIONS.md

View File

@@ -5,6 +5,17 @@ All notable changes to KubeSolo OS are documented in this file.
Format based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
versioning follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [0.2.0] - 2026-02-12
### Added
- Cloud-init: support all documented KubeSolo CLI flags (`--local-storage-shared-path`, `--debug`, `--pprof-server`, `--portainer-edge-id`, `--portainer-edge-key`, `--portainer-edge-async`)
- Cloud-init: `full-config.yaml` example showing all supported parameters
- Cloud-init: KubeSolo configuration reference table in docs/cloud-init.md
- Security hardening: mount hardening, sysctl, kernel module lock, AppArmor profiles
- ARM64 Raspberry Pi support with A/B boot via tryboot
- BootEnv abstraction for GRUB and RPi boot environments
- Go 1.25.5 installed on host for native builds
## [0.1.0] - 2026-02-12
First release with all 5 design-doc phases complete. ISO boots and runs K8s pods.
@@ -86,3 +97,4 @@ First release with all 5 design-doc phases complete. ISO boots and runs K8s pods
- Rewrote dev-vm.sh for macOS: bsdtar ISO extraction, Homebrew mkfs.ext4 detection, direct kernel boot, TCG acceleration, port 8080 forwarding
- Kubeconfig now served via HTTP on port 8080 (serial console truncates base64 lines)
- Added 127.0.0.1 and 10.0.2.15 to API server SANs for QEMU port forwarding
- dev-vm.sh now works on Linux: fallback ISO extraction via isoinfo or loop mount, KVM auto-detection, platform-aware error messages

21
LICENSE Normal file
View File

@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2025 Anthony De Lorenzo
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

View File

@@ -1,9 +1,10 @@
.PHONY: all fetch kernel build-cloudinit build-update-agent build-cross rootfs initramfs \
iso disk-image oci-image \
test-boot test-k8s test-persistence test-deploy test-storage test-all \
test-cloudinit test-update-agent \
iso disk-image oci-image rpi-image \
kernel-arm64 rootfs-arm64 \
test-boot test-k8s test-persistence test-deploy test-storage test-security test-all \
test-boot-arm64 test-cloudinit test-update-agent \
bench-boot bench-resources \
dev-vm dev-vm-shell quick docker-build shellcheck \
dev-vm dev-vm-shell dev-vm-arm64 quick docker-build shellcheck \
kernel-audit clean distclean help
SHELL := /bin/bash
@@ -71,6 +72,26 @@ build-cross:
@echo "==> Cross-compiling for amd64 + arm64..."
$(BUILD_DIR)/scripts/build-cross.sh
# =============================================================================
# ARM64 Raspberry Pi targets
# =============================================================================
kernel-arm64:
@echo "==> Building ARM64 kernel for Raspberry Pi..."
$(BUILD_DIR)/scripts/build-kernel-arm64.sh
rootfs-arm64: build-cross
@echo "==> Preparing ARM64 rootfs..."
TARGET_ARCH=arm64 $(BUILD_DIR)/scripts/fetch-components.sh
TARGET_ARCH=arm64 $(BUILD_DIR)/scripts/extract-core.sh
TARGET_ARCH=arm64 $(BUILD_DIR)/scripts/inject-kubesolo.sh
@echo "==> Packing ARM64 initramfs..."
$(BUILD_DIR)/scripts/pack-initramfs.sh
rpi-image: rootfs-arm64 kernel-arm64
@echo "==> Creating Raspberry Pi SD card image..."
$(BUILD_DIR)/scripts/create-rpi-image.sh
@echo "==> Built: $(OUTPUT_DIR)/$(OS_NAME)-$(VERSION).rpi.img"
# =============================================================================
# Kernel validation
# =============================================================================
@@ -101,6 +122,14 @@ test-storage: iso
@echo "==> Testing local storage provisioning..."
test/integration/test-local-storage.sh $(OUTPUT_DIR)/$(OS_NAME)-$(VERSION).iso
test-security: iso
@echo "==> Testing security hardening..."
test/integration/test-security-hardening.sh $(OUTPUT_DIR)/$(OS_NAME)-$(VERSION).iso
test-boot-arm64:
@echo "==> Testing ARM64 boot in QEMU..."
test/qemu/test-boot-arm64.sh
test-all: test-boot test-k8s test-persistence
# Cloud-init Go tests
@@ -163,6 +192,10 @@ dev-vm-debug: iso
@echo "==> Launching dev VM (debug mode)..."
hack/dev-vm.sh $(OUTPUT_DIR)/$(OS_NAME)-$(VERSION).iso --debug
dev-vm-arm64:
@echo "==> Launching ARM64 dev VM..."
hack/dev-vm-arm64.sh
# Fast rebuild: only repack initramfs + ISO (skip fetch/extract)
quick:
@echo "==> Quick rebuild (repack + ISO only)..."
@@ -199,7 +232,7 @@ distclean: clean
help:
@echo "KubeSolo OS Build System (v$(VERSION))"
@echo ""
@echo "Build targets:"
@echo "Build targets (x86_64):"
@echo " make fetch Download Tiny Core ISO, KubeSolo, dependencies"
@echo " make kernel Build custom kernel with CONFIG_CGROUP_BPF=y"
@echo " make build-cloudinit Build cloud-init Go binary"
@@ -213,25 +246,33 @@ help:
@echo " make quick Fast rebuild (re-inject + repack + ISO only)"
@echo " make docker-build Reproducible build inside Docker"
@echo ""
@echo "Build targets (ARM64 Raspberry Pi):"
@echo " make kernel-arm64 Build ARM64 kernel from raspberrypi/linux"
@echo " make rootfs-arm64 Extract + prepare ARM64 rootfs from piCore64"
@echo " make rpi-image Create Raspberry Pi SD card image with A/B partitions"
@echo ""
@echo "Test targets:"
@echo " make test-boot Boot ISO in QEMU, verify boot success"
@echo " make test-k8s Boot + verify K8s node reaches Ready"
@echo " make test-persist Reboot disk image, verify state persists"
@echo " make test-deploy Deploy nginx pod, verify Running"
@echo " make test-storage Test PVC with local-path provisioner"
@echo " make test-security Verify security hardening (AppArmor, sysctl, mounts)"
@echo " make test-cloudinit Run cloud-init Go unit tests"
@echo " make test-update-agent Run update agent Go unit tests"
@echo " make test-update A/B update cycle integration test"
@echo " make test-rollback Forced rollback integration test"
@echo " make test-boot-arm64 ARM64 boot test in QEMU aarch64"
@echo " make test-all Run core tests (boot + k8s + persistence)"
@echo " make test-integ Run full integration suite"
@echo " make bench-boot Benchmark boot performance (3 runs)"
@echo " make bench-resources Benchmark resource usage (requires running VM)"
@echo ""
@echo "Dev targets:"
@echo " make dev-vm Launch interactive QEMU VM"
@echo " make dev-vm Launch interactive QEMU VM (x86_64)"
@echo " make dev-vm-shell Launch QEMU VM -> emergency shell"
@echo " make dev-vm-debug Launch QEMU VM with debug logging"
@echo " make dev-vm-arm64 Launch ARM64 QEMU VM"
@echo " make kernel-audit Check kernel config against requirements"
@echo " make shellcheck Lint all shell scripts"
@echo ""

View File

@@ -2,7 +2,7 @@
An immutable, bootable Linux distribution purpose-built for [KubeSolo](https://github.com/portainer/kubesolo) — Portainer's ultra-lightweight single-node Kubernetes.
> **Status:** All 5 phases complete. Boots and runs K8s workloads.
> **Status:** All 6 phases complete. Boots and runs K8s workloads. Portainer Edge Agent tested and connected.
## What is this?
@@ -47,6 +47,24 @@ Or build everything at once inside Docker:
make docker-build
```
After boot, retrieve the kubeconfig and manage your cluster from the host:
```bash
curl -s http://localhost:8080 > ~/.kube/kubesolo-config
export KUBECONFIG=~/.kube/kubesolo-config
kubectl get nodes
```
### Portainer Edge Agent
Pass Edge credentials via boot parameters:
```bash
./hack/dev-vm.sh --edge-id=YOUR_EDGE_ID --edge-key=YOUR_EDGE_KEY
```
Or configure via [cloud-init YAML](cloud-init/examples/portainer-edge.yaml).
## Requirements
**Build host:**
@@ -104,7 +122,7 @@ Unnecessary subsystems (sound, GPU, wireless, Bluetooth, etc.) are stripped to k
## Cloud-Init
First-boot configuration via a simple YAML schema:
First-boot configuration via a simple YAML schema. All [documented KubeSolo flags](https://www.kubesolo.io/documentation#install) are supported:
```yaml
hostname: edge-node-01
@@ -115,10 +133,15 @@ network:
dns:
- 8.8.8.8
kubesolo:
node-name: edge-node-01
portainer:
edge_id: "your-edge-id"
edge_key: "your-edge-key"
local-storage: true
local-storage-shared-path: "/mnt/shared"
apiserver-extra-sans:
- edge-node-01.local
debug: false
pprof-server: false
portainer-edge-id: "your-edge-id"
portainer-edge-key: "your-edge-key"
portainer-edge-async: true
```
See [docs/cloud-init.md](docs/cloud-init.md) and the [examples](cloud-init/examples/).
@@ -189,7 +212,7 @@ Metrics include: `kubesolo_os_info`, `boot_success`, `boot_counter`, `uptime_sec
| `make build-cross` | Cross-compile for amd64 + arm64 |
| `make docker-build` | Build everything in Docker |
| `make quick` | Fast rebuild (re-inject + repack + ISO) |
| `make dev-vm` | Launch QEMU dev VM |
| `make dev-vm` | Launch QEMU dev VM (Linux + macOS) |
| `make test-all` | Run all tests |
## Documentation
@@ -209,8 +232,9 @@ Metrics include: `kubesolo_os_info`, `boot_success`, `boot_counter`, `uptime_sec
| 3 | A/B atomic updates, GRUB, rollback agent | Complete |
| 4 | Ed25519 signing, Portainer Edge, SSH extension | Complete |
| 5 | CI/CD, OCI distribution, Prometheus metrics, ARM64 | Complete |
| 6 | Security hardening, AppArmor, ARM64 RPi support | Complete |
| - | Custom kernel build for container runtime fixes | Complete |
## License
TBD
MIT License — see [LICENSE](LICENSE) for details.

View File

@@ -1 +1 @@
0.1.0
0.2.0

View File

@@ -31,13 +31,20 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
syslinux \
syslinux-common \
syslinux-utils \
apparmor \
apparmor-utils \
gcc-aarch64-linux-gnu \
binutils-aarch64-linux-gnu \
git \
kpartx \
unzip \
wget \
xorriso \
xz-utils \
&& rm -rf /var/lib/apt/lists/*
# Install Go (for building cloud-init and update agent)
ARG GO_VERSION=1.24.0
ARG GO_VERSION=1.25.5
RUN curl -fsSL "https://go.dev/dl/go${GO_VERSION}.linux-amd64.tar.gz" \
| tar -C /usr/local -xzf -
ENV PATH="/usr/local/go/bin:${PATH}"

View File

@@ -128,7 +128,12 @@ echo "Security:"
check_config CONFIG_SECCOMP recommended "Seccomp (container security)"
check_config CONFIG_SECCOMP_FILTER recommended "Seccomp BPF filter"
check_config CONFIG_BPF_SYSCALL recommended "BPF syscall"
check_config CONFIG_AUDIT recommended "Audit framework"
check_config CONFIG_AUDIT mandatory "Audit framework"
check_config CONFIG_AUDITSYSCALL mandatory "Audit system call events"
check_config CONFIG_SECURITY mandatory "Security framework"
check_config CONFIG_SECURITYFS mandatory "Security filesystem"
check_config CONFIG_SECURITY_APPARMOR mandatory "AppArmor LSM"
check_config CONFIG_SECURITY_NETWORK recommended "Network security hooks"
echo ""
# --- Crypto ---

View File

@@ -0,0 +1,81 @@
# Kernel modules loaded at boot by init (ARM64 / Raspberry Pi)
# One module per line. Lines starting with # are ignored.
# Modules are loaded in order listed — dependencies must come first.
# Network device drivers (loaded early so interfaces are available)
# Note: no e1000/e1000e on ARM64 — those are x86 Intel NIC drivers
virtio_net
# Virtio support (for QEMU VMs — block, entropy)
virtio_blk
virtio_rng
# Raspberry Pi specific (USB Ethernet on Pi 4 is built-in, no module needed)
# Pi 5 uses PCIe ethernet, also typically built-in
# Filesystem — overlay (required for containerd)
overlay
# Netfilter dependencies (must load before conntrack)
nf_defrag_ipv4
nf_defrag_ipv6
# Netfilter / connection tracking (required for kube-proxy)
nf_conntrack
nf_nat
nf_conntrack_netlink
# nftables (modern iptables backend)
nf_tables
nft_compat
nft_chain_nat
nft_ct
nft_masq
nft_nat
nft_redir
# Netfilter xt match/target modules (used by kube-proxy iptables rules via nft_compat)
xt_conntrack
xt_MASQUERADE
xt_mark
xt_comment
xt_multiport
xt_nat
xt_addrtype
xt_connmark
xt_REDIRECT
xt_recent
xt_statistic
xt_set
# nft extras (reject, fib — used by kube-proxy nf_tables rules)
nft_reject
nft_reject_ipv4
nft_reject_ipv6
nft_fib
nft_fib_ipv4
nft_fib_ipv6
# Reject targets (used by kube-proxy iptables-restore rules)
nf_reject_ipv4
nf_reject_ipv6
ipt_REJECT
ip6t_REJECT
# nfacct extension (kube-proxy probes for it)
xt_nfacct
# Networking — bridge and netfilter (required for K8s pod networking)
# Load order: llc → stp → bridge → br_netfilter
llc
stp
bridge
br_netfilter
veth
vxlan
# IPVS — useful for kube-proxy IPVS mode and CNI plugins
ip_vs
ip_vs_rr
ip_vs_wrr
ip_vs_sh

View File

@@ -0,0 +1,69 @@
# KubeSolo OS — Raspberry Pi kernel config overrides
# Applied on top of bcm2711_defconfig (Pi 4) or bcm2712_defconfig (Pi 5)
# These ensure container runtime support is enabled.
# cgroup v2 (mandatory for containerd/runc)
CONFIG_CGROUPS=y
CONFIG_CGROUP_CPUACCT=y
CONFIG_CGROUP_DEVICE=y
CONFIG_CGROUP_FREEZER=y
CONFIG_CGROUP_SCHED=y
CONFIG_CGROUP_PIDS=y
CONFIG_MEMCG=y
CONFIG_CGROUP_BPF=y
CONFIG_CFS_BANDWIDTH=y
# BPF (required for cgroup v2 device control)
CONFIG_BPF=y
CONFIG_BPF_SYSCALL=y
# Namespaces (mandatory for containers)
CONFIG_NAMESPACES=y
CONFIG_NET_NS=y
CONFIG_PID_NS=y
CONFIG_USER_NS=y
CONFIG_UTS_NS=y
CONFIG_IPC_NS=y
# Device management
CONFIG_DEVTMPFS=y
CONFIG_DEVTMPFS_MOUNT=y
# Filesystem
CONFIG_OVERLAY_FS=y
CONFIG_SQUASHFS=y
CONFIG_EXT4_FS=y
CONFIG_VFAT_FS=y
# Networking
CONFIG_BRIDGE=m
CONFIG_NETFILTER=y
CONFIG_NF_CONNTRACK=m
CONFIG_NF_NAT=m
CONFIG_NF_TABLES=m
CONFIG_VETH=m
CONFIG_VXLAN=m
# Security: AppArmor + Audit
CONFIG_AUDIT=y
CONFIG_AUDITSYSCALL=y
CONFIG_SECURITY=y
CONFIG_SECURITYFS=y
CONFIG_SECURITY_NETWORK=y
CONFIG_SECURITY_APPARMOR=y
CONFIG_DEFAULT_SECURITY_APPARMOR=y
# Security: seccomp
CONFIG_SECCOMP=y
CONFIG_SECCOMP_FILTER=y
# Crypto (image verification)
CONFIG_CRYPTO_SHA256=y
# Disable unnecessary subsystems for edge appliance
# CONFIG_SOUND is not set
# CONFIG_DRM is not set
# CONFIG_MEDIA_SUPPORT is not set
# CONFIG_WIRELESS is not set
# CONFIG_BT is not set
# CONFIG_NFC is not set

View File

@@ -15,5 +15,28 @@ KUBESOLO_INSTALL_URL=https://get.kubesolo.io
GRUB_VERSION=2.12
SYSLINUX_VERSION=6.03
# SHA256 checksums for supply chain verification
# Populate by running: sha256sum build/cache/<file>
# Leave empty to skip verification (useful for first fetch)
TINYCORE_ISO_SHA256=""
KUBESOLO_SHA256=""
NETFILTER_TCZ_SHA256=""
NET_BRIDGING_TCZ_SHA256=""
IPTABLES_TCZ_SHA256=""
# piCore64 (ARM64 — Raspberry Pi)
PICORE_VERSION=15.0.0
PICORE_ARCH=aarch64
PICORE_IMAGE=piCore64-${PICORE_VERSION}.zip
PICORE_IMAGE_URL=http://www.tinycorelinux.net/${PICORE_VERSION%%.*}.x/${PICORE_ARCH}/releases/RPi/${PICORE_IMAGE}
# Raspberry Pi firmware (boot blobs, DTBs)
RPI_FIRMWARE_TAG=1.20240529
RPI_FIRMWARE_URL=https://github.com/raspberrypi/firmware/archive/refs/tags/${RPI_FIRMWARE_TAG}.tar.gz
# Raspberry Pi kernel source
RPI_KERNEL_BRANCH=rpi-6.6.y
RPI_KERNEL_REPO=https://github.com/raspberrypi/linux
# Output naming
OS_NAME=kubesolo-os

View File

@@ -0,0 +1,52 @@
# AppArmor profile for containerd
# Start in complain mode to log without blocking
#include <tunables/global>
profile containerd /usr/bin/containerd flags=(complain) {
#include <abstractions/base>
# Binary and shared libraries
/usr/bin/containerd mr,
/usr/lib/** mr,
/lib/** mr,
# Containerd runtime state
/var/lib/containerd/** rw,
/run/containerd/** rw,
# Container image layers and snapshots
/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/** rw,
# CNI networking
/etc/cni/** r,
/opt/cni/bin/** ix,
# Proc and sys access for containers
@{PROC}/** r,
/sys/** r,
# Device access for containers
/dev/** rw,
# Network access
network,
# Container runtime needs broad capabilities
capability,
# Allow executing container runtimes
/usr/bin/containerd-shim-runc-v2 ix,
/usr/bin/runc ix,
/usr/sbin/runc ix,
# Temp files
/tmp/** rw,
# Log files
/var/log/** rw,
# Signal handling for child processes
signal,
ptrace,
}

View File

@@ -0,0 +1,56 @@
# AppArmor profile for kubesolo (kubelet + control plane)
# Start in complain mode to log without blocking
#include <tunables/global>
profile kubesolo /usr/bin/kubesolo flags=(complain) {
#include <abstractions/base>
# Binary and shared libraries
/usr/bin/kubesolo mr,
/usr/lib/** mr,
/lib/** mr,
# KubeSolo state (etcd/SQLite, certificates, manifests)
/var/lib/kubesolo/** rw,
# KubeSolo configuration
/etc/kubesolo/** r,
# Containerd socket
/run/containerd/** rw,
# CNI networking
/etc/cni/** r,
/opt/cni/bin/** ix,
# Proc and sys access
@{PROC}/** r,
/sys/** r,
# Device access
/dev/** rw,
# Network access (API server, kubelet, etcd)
network,
# Control plane needs broad capabilities
capability,
# Kubectl and other tools
/usr/bin/kubectl ix,
/usr/local/bin/** ix,
# Temp files
/tmp/** rw,
# Log files
/var/log/** rw,
# Kubelet needs to manage pods
/var/lib/kubelet/** rw,
# Signal handling
signal,
ptrace,
}

View File

@@ -0,0 +1,27 @@
# Security hardening — applied automatically by 40-sysctl.sh
# Network: anti-spoofing
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1
# Network: SYN flood protection
net.ipv4.tcp_syncookies = 1
# Network: ICMP hardening
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.icmp_echo_ignore_broadcasts = 1
net.ipv4.icmp_ignore_bogus_error_responses = 1
net.ipv4.conf.all.log_martians = 1
# Network: IPv6 hardening
net.ipv6.conf.all.accept_redirects = 0
net.ipv6.conf.default.accept_redirects = 0
net.ipv6.conf.all.accept_ra = 0
# Network: source routing
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.default.accept_source_route = 0
# Kernel: information disclosure
kernel.kptr_restrict = 2
kernel.dmesg_restrict = 1
kernel.perf_event_paranoid = 3
# Kernel: core dump safety
fs.suid_dumpable = 0

View File

@@ -0,0 +1,158 @@
#!/bin/bash
# build-kernel-arm64.sh — Build ARM64 kernel for Raspberry Pi 4/5
#
# Uses the official raspberrypi/linux kernel fork with bcm2711_defconfig
# as the base, overlaid with container-critical config options.
#
# Output is cached in $CACHE_DIR/custom-kernel-arm64/ and reused across builds.
#
# Requirements:
# - gcc-aarch64-linux-gnu (cross-compiler)
# - Standard kernel build deps (bc, bison, flex, etc.)
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
CACHE_DIR="${CACHE_DIR:-$PROJECT_ROOT/build/cache}"
# shellcheck source=../config/versions.env
. "$SCRIPT_DIR/../config/versions.env"
CUSTOM_KERNEL_DIR="$CACHE_DIR/custom-kernel-arm64"
CUSTOM_IMAGE="$CUSTOM_KERNEL_DIR/Image"
CUSTOM_MODULES="$CUSTOM_KERNEL_DIR/modules"
CUSTOM_DTBS="$CUSTOM_KERNEL_DIR/dtbs"
mkdir -p "$CACHE_DIR" "$CUSTOM_KERNEL_DIR"
# --- Skip if already built ---
if [ -f "$CUSTOM_IMAGE" ] && [ -d "$CUSTOM_MODULES" ]; then
echo "==> ARM64 kernel already built (cached)"
echo " Image: $CUSTOM_IMAGE ($(du -h "$CUSTOM_IMAGE" | cut -f1))"
exit 0
fi
# --- Verify cross-compiler ---
if ! command -v aarch64-linux-gnu-gcc >/dev/null 2>&1; then
echo "ERROR: aarch64-linux-gnu-gcc not found"
echo "Install: apt-get install gcc-aarch64-linux-gnu"
exit 1
fi
echo "==> Building ARM64 kernel for Raspberry Pi..."
echo " Branch: $RPI_KERNEL_BRANCH"
echo " Repo: $RPI_KERNEL_REPO"
# --- Download kernel source ---
KERNEL_SRC_DIR="$CACHE_DIR/rpi-linux-${RPI_KERNEL_BRANCH}"
if [ ! -d "$KERNEL_SRC_DIR" ]; then
echo "==> Downloading RPi kernel source (shallow clone)..."
git clone --depth 1 --branch "$RPI_KERNEL_BRANCH" \
"$RPI_KERNEL_REPO" "$KERNEL_SRC_DIR"
else
echo "==> Kernel source already cached"
fi
# --- Build in /tmp for case-sensitivity ---
KERNEL_BUILD_DIR="/tmp/kernel-build-arm64"
rm -rf "$KERNEL_BUILD_DIR"
cp -a "$KERNEL_SRC_DIR" "$KERNEL_BUILD_DIR"
cd "$KERNEL_BUILD_DIR"
# --- Apply base config (Pi 4 = bcm2711) ---
echo "==> Applying bcm2711_defconfig..."
make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- bcm2711_defconfig
# --- Apply container config overrides ---
CONFIG_FRAGMENT="$PROJECT_ROOT/build/config/rpi-kernel-config.fragment"
if [ -f "$CONFIG_FRAGMENT" ]; then
echo "==> Applying KubeSolo config overrides..."
while IFS= read -r line; do
# Skip comments and empty lines
case "$line" in \#*|"") continue ;; esac
key="${line%%=*}"
value="${line#*=}"
case "$value" in
y) ./scripts/config --enable "$key" ;;
m) ./scripts/config --module "$key" ;;
n) ./scripts/config --disable "${key#CONFIG_}" ;;
*) ./scripts/config --set-str "$key" "$value" ;;
esac
done < "$CONFIG_FRAGMENT"
fi
# Handle "is not set" comments as disables
if [ -f "$CONFIG_FRAGMENT" ]; then
while IFS= read -r line; do
case "$line" in
"# CONFIG_"*" is not set")
key=$(echo "$line" | sed -n 's/^# \(CONFIG_[A-Z_]*\) is not set$/\1/p')
[ -n "$key" ] && ./scripts/config --disable "${key#CONFIG_}"
;;
esac
done < "$CONFIG_FRAGMENT"
fi
# Resolve dependencies
make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- olddefconfig
# --- Build kernel + modules + DTBs ---
NPROC=$(nproc 2>/dev/null || echo 4)
echo ""
echo "==> Building ARM64 kernel (${NPROC} parallel jobs)..."
echo " This may take 20-30 minutes..."
make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -j"$NPROC" Image modules dtbs 2>&1
echo "==> ARM64 kernel build complete"
# --- Install to staging ---
echo "==> Installing Image..."
cp arch/arm64/boot/Image "$CUSTOM_IMAGE"
echo "==> Installing modules (stripped)..."
rm -rf "$CUSTOM_MODULES"
mkdir -p "$CUSTOM_MODULES"
make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- \
INSTALL_MOD_STRIP=1 modules_install INSTALL_MOD_PATH="$CUSTOM_MODULES"
# Remove build/source symlinks
KVER=$(ls "$CUSTOM_MODULES/lib/modules/" | head -1)
rm -f "$CUSTOM_MODULES/lib/modules/$KVER/build"
rm -f "$CUSTOM_MODULES/lib/modules/$KVER/source"
# Run depmod
depmod -a -b "$CUSTOM_MODULES" "$KVER" 2>/dev/null || true
echo "==> Installing Device Tree Blobs..."
rm -rf "$CUSTOM_DTBS"
mkdir -p "$CUSTOM_DTBS/overlays"
# Pi 4 DTBs
cp arch/arm64/boot/dts/broadcom/bcm2711*.dtb "$CUSTOM_DTBS/" 2>/dev/null || true
# Pi 5 DTBs
cp arch/arm64/boot/dts/broadcom/bcm2712*.dtb "$CUSTOM_DTBS/" 2>/dev/null || true
# Overlays we need
for overlay in disable-wifi disable-bt; do
[ -f "arch/arm64/boot/dts/overlays/${overlay}.dtbo" ] && \
cp "arch/arm64/boot/dts/overlays/${overlay}.dtbo" "$CUSTOM_DTBS/overlays/"
done
# Save config for reference
cp .config "$CUSTOM_KERNEL_DIR/.config"
# --- Clean up ---
echo "==> Cleaning kernel build directory..."
cd /
rm -rf "$KERNEL_BUILD_DIR"
# --- Summary ---
echo ""
echo "==> ARM64 kernel build complete:"
echo " Image: $CUSTOM_IMAGE ($(du -h "$CUSTOM_IMAGE" | cut -f1))"
echo " Kernel ver: $KVER"
MOD_COUNT=$(find "$CUSTOM_MODULES/lib/modules/$KVER" -name '*.ko*' 2>/dev/null | wc -l)
echo " Modules: $MOD_COUNT"
echo " Modules size: $(du -sh "$CUSTOM_MODULES/lib/modules/$KVER" 2>/dev/null | cut -f1)"
echo " DTBs: $(ls "$CUSTOM_DTBS"/*.dtb 2>/dev/null | wc -l)"
echo ""

View File

@@ -145,13 +145,29 @@ echo "==> Disabling unnecessary subsystems for minimal footprint..."
# FPGA (not needed)
./scripts/config --disable FPGA
# Resolve dependencies (olddefconfig accepts defaults for new options)
# First pass: resolve base dependencies before adding security configs.
# The stock TC config has "# CONFIG_SECURITY is not set" which causes
# olddefconfig to strip security-related options if applied in a single pass.
make olddefconfig
# Verify CONFIG_CGROUP_BPF is set
if grep -q 'CONFIG_CGROUP_BPF=y' .config; then
echo " CONFIG_CGROUP_BPF=y confirmed in .config"
else
# Security: AppArmor LSM + Audit subsystem
# Applied AFTER first olddefconfig to ensure CONFIG_SECURITY dependencies
# (SYSFS, MULTIUSER) are resolved before enabling the security subtree.
echo "==> Enabling AppArmor + Audit kernel configs..."
./scripts/config --enable CONFIG_AUDIT
./scripts/config --enable CONFIG_AUDITSYSCALL
./scripts/config --enable CONFIG_SECURITY
./scripts/config --enable CONFIG_SECURITYFS
./scripts/config --enable CONFIG_SECURITY_NETWORK
./scripts/config --enable CONFIG_SECURITY_APPARMOR
./scripts/config --set-str CONFIG_LSM "lockdown,yama,apparmor"
./scripts/config --set-str CONFIG_DEFAULT_SECURITY "apparmor"
# Second pass: resolve security config dependencies
make olddefconfig
# Verify critical configs are set
if ! grep -q 'CONFIG_CGROUP_BPF=y' .config; then
echo "ERROR: CONFIG_CGROUP_BPF not set after olddefconfig"
grep 'CGROUP_BPF' .config || echo " (CGROUP_BPF not found in .config)"
echo ""
@@ -159,10 +175,25 @@ else
grep -E 'CONFIG_BPF=|CONFIG_BPF_SYSCALL=' .config || echo " BPF not found"
exit 1
fi
echo " CONFIG_CGROUP_BPF=y confirmed"
# Show what changed
echo " Config diff from stock:"
diff "$KERNEL_CFG" .config | grep '^[<>]' | head -20 || echo " (no differences beyond CGROUP_BPF)"
if ! grep -q 'CONFIG_SECURITY_APPARMOR=y' .config; then
echo "ERROR: CONFIG_SECURITY_APPARMOR not set after olddefconfig"
echo " Security-related configs:"
grep -E 'CONFIG_SECURITY=|CONFIG_SECURITYFS=|CONFIG_SECURITY_APPARMOR=' .config
exit 1
fi
echo " CONFIG_SECURITY_APPARMOR=y confirmed"
if ! grep -q 'CONFIG_AUDIT=y' .config; then
echo "ERROR: CONFIG_AUDIT not set after olddefconfig"
exit 1
fi
echo " CONFIG_AUDIT=y confirmed"
# Show what changed (security-related)
echo " Key config values:"
grep -E 'CONFIG_SECURITY=|CONFIG_SECURITY_APPARMOR=|CONFIG_AUDIT=|CONFIG_LSM=|CONFIG_CGROUP_BPF=' .config | sed 's/^/ /'
# --- Build kernel + modules ---
NPROC=$(nproc 2>/dev/null || echo 4)

View File

@@ -51,10 +51,39 @@ size=1048576, type=0FC63DAF-8483-4772-8E79-3D69D8477DE4, name="SystemB"
type=0FC63DAF-8483-4772-8E79-3D69D8477DE4, name="Data"
EOF
# Set up loop device
LOOP=$(losetup --show -fP "$IMG_OUTPUT")
# Set up loop device with partition mappings
LOOP=$(losetup --show -f "$IMG_OUTPUT")
echo "==> Loop device: $LOOP"
# Use kpartx for reliable partition device nodes (works in Docker/containers)
USE_KPARTX=false
if [ ! -b "${LOOP}p1" ]; then
if command -v kpartx >/dev/null 2>&1; then
kpartx -a "$LOOP"
USE_KPARTX=true
sleep 1
LOOP_NAME=$(basename "$LOOP")
P1="/dev/mapper/${LOOP_NAME}p1"
P2="/dev/mapper/${LOOP_NAME}p2"
P3="/dev/mapper/${LOOP_NAME}p3"
P4="/dev/mapper/${LOOP_NAME}p4"
else
# Retry with -P flag
losetup -d "$LOOP"
LOOP=$(losetup --show -fP "$IMG_OUTPUT")
sleep 1
P1="${LOOP}p1"
P2="${LOOP}p2"
P3="${LOOP}p3"
P4="${LOOP}p4"
fi
else
P1="${LOOP}p1"
P2="${LOOP}p2"
P3="${LOOP}p3"
P4="${LOOP}p4"
fi
MNT_EFI=$(mktemp -d)
MNT_SYSA=$(mktemp -d)
MNT_SYSB=$(mktemp -d)
@@ -65,22 +94,25 @@ cleanup() {
umount "$MNT_SYSA" 2>/dev/null || true
umount "$MNT_SYSB" 2>/dev/null || true
umount "$MNT_DATA" 2>/dev/null || true
if [ "$USE_KPARTX" = true ]; then
kpartx -d "$LOOP" 2>/dev/null || true
fi
losetup -d "$LOOP" 2>/dev/null || true
rm -rf "$MNT_EFI" "$MNT_SYSA" "$MNT_SYSB" "$MNT_DATA" 2>/dev/null || true
}
trap cleanup EXIT
# Format partitions
mkfs.vfat -F 32 -n KSOLOEFI "${LOOP}p1"
mkfs.ext4 -q -L KSOLOA "${LOOP}p2"
mkfs.ext4 -q -L KSOLOB "${LOOP}p3"
mkfs.ext4 -q -L KSOLODATA "${LOOP}p4"
mkfs.vfat -F 32 -n KSOLOEFI "$P1"
mkfs.ext4 -q -L KSOLOA "$P2"
mkfs.ext4 -q -L KSOLOB "$P3"
mkfs.ext4 -q -L KSOLODATA "$P4"
# Mount all partitions
mount "${LOOP}p1" "$MNT_EFI"
mount "${LOOP}p2" "$MNT_SYSA"
mount "${LOOP}p3" "$MNT_SYSB"
mount "${LOOP}p4" "$MNT_DATA"
mount "$P1" "$MNT_EFI"
mount "$P2" "$MNT_SYSA"
mount "$P3" "$MNT_SYSB"
mount "$P4" "$MNT_DATA"
# --- EFI/Boot Partition ---
echo " Installing GRUB..."

256
build/scripts/create-rpi-image.sh Executable file
View File

@@ -0,0 +1,256 @@
#!/bin/bash
# create-rpi-image.sh — Create a raw disk image for Raspberry Pi SD card
#
# Partition layout (MBR):
# Part 1: Boot/Control (384 MB, FAT32, label KSOLOCTL) — firmware + kernel + initramfs + autoboot.txt
# Part 2: Boot A (256 MB, FAT32, label KSOLOA) — kernel + DTBs + initramfs
# Part 3: Boot B (256 MB, FAT32, label KSOLOB) — same as Boot A (initially identical)
# Part 4: Data (remaining of 2GB, ext4, label KSOLODATA)
#
# The RPi EEPROM loads start4.elf from partition 1.
# If autoboot.txt is supported (newer EEPROM), firmware redirects to partition 2/3 for A/B boot.
# If autoboot.txt is NOT supported (older EEPROM), partition 1 has full boot files as fallback.
#
# MBR is required — GPT + autoboot.txt is not reliably supported on Pi 4.
#
# Usage: build/scripts/create-rpi-image.sh
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
# shellcheck source=../config/versions.env
. "$SCRIPT_DIR/../config/versions.env"
ROOTFS_DIR="${ROOTFS_DIR:-$PROJECT_ROOT/build/rootfs-work}"
OUTPUT_DIR="${OUTPUT_DIR:-$PROJECT_ROOT/output}"
CACHE_DIR="${CACHE_DIR:-$PROJECT_ROOT/build/cache}"
VERSION="$(cat "$PROJECT_ROOT/VERSION")"
IMG_OUTPUT="$OUTPUT_DIR/${OS_NAME}-${VERSION}.rpi.img"
IMG_SIZE_MB="${IMG_SIZE_MB:-2048}" # 2 GB default
# ARM64 kernel (Image format, not bzImage)
KERNEL="${CACHE_DIR}/custom-kernel-arm64/Image"
INITRAMFS="${ROOTFS_DIR}/kubesolo-os.gz"
RPI_FIRMWARE_DIR="${CACHE_DIR}/rpi-firmware"
# DTBs MUST come from the kernel build (not firmware repo) to match the kernel.
# A DTB mismatch causes sdhci-iproc to silently fail — zero block devices.
KERNEL_DTBS_DIR="${CACHE_DIR}/custom-kernel-arm64/dtbs"
echo "==> Creating ${IMG_SIZE_MB}MB Raspberry Pi disk image..."
# --- Verify required files ---
MISSING=0
for f in "$KERNEL" "$INITRAMFS"; do
if [ ! -f "$f" ]; then
echo "ERROR: Missing $f"
MISSING=1
fi
done
if [ ! -d "$RPI_FIRMWARE_DIR" ]; then
echo "ERROR: Missing RPi firmware directory: $RPI_FIRMWARE_DIR"
echo " Run 'make fetch' to download firmware blobs."
MISSING=1
fi
if [ "$MISSING" = "1" ]; then
echo ""
echo "Required files:"
echo " Kernel: $KERNEL (run 'make kernel-arm64')"
echo " Initramfs: $INITRAMFS (run 'make initramfs')"
echo " Firmware: $RPI_FIRMWARE_DIR/ (run 'make fetch')"
exit 1
fi
mkdir -p "$OUTPUT_DIR"
# --- Create sparse image ---
dd if=/dev/zero of="$IMG_OUTPUT" bs=1M count=0 seek="$IMG_SIZE_MB" 2>/dev/null
# --- Partition table (MBR) ---
# MBR is required for reliable RPi boot with autoboot.txt.
# GPT + autoboot.txt fails on many Pi 4 EEPROM versions.
# Part 1: Boot/Control 384 MB FAT32 (firmware + kernel + initramfs + autoboot.txt)
# Part 2: Boot A 256 MB FAT32 (kernel + initramfs + DTBs)
# Part 3: Boot B 256 MB FAT32 (kernel + initramfs + DTBs)
# Part 4: Data remaining ext4
sfdisk "$IMG_OUTPUT" << EOF
label: dos
# Boot/Control partition: 384 MB, FAT32 (type 0c = W95 FAT32 LBA)
# Contains firmware + autoboot.txt for A/B redirect, PLUS full boot files as fallback
start=2048, size=786432, type=c, bootable
# Boot A partition: 256 MB, FAT32
size=524288, type=c
# Boot B partition: 256 MB, FAT32
size=524288, type=c
# Data partition: remaining, Linux
type=83
EOF
# --- Set up loop device ---
LOOP=$(losetup --show -f "$IMG_OUTPUT")
echo "==> Loop device: $LOOP"
# Use kpartx for reliable partition device nodes (works in Docker/containers)
USE_KPARTX=false
if [ ! -b "${LOOP}p1" ]; then
if command -v kpartx >/dev/null 2>&1; then
kpartx -a "$LOOP"
USE_KPARTX=true
sleep 1
LOOP_NAME=$(basename "$LOOP")
P1="/dev/mapper/${LOOP_NAME}p1"
P2="/dev/mapper/${LOOP_NAME}p2"
P3="/dev/mapper/${LOOP_NAME}p3"
P4="/dev/mapper/${LOOP_NAME}p4"
else
# Retry with -P flag
losetup -d "$LOOP"
LOOP=$(losetup --show -fP "$IMG_OUTPUT")
sleep 1
P1="${LOOP}p1"
P2="${LOOP}p2"
P3="${LOOP}p3"
P4="${LOOP}p4"
fi
else
P1="${LOOP}p1"
P2="${LOOP}p2"
P3="${LOOP}p3"
P4="${LOOP}p4"
fi
MNT_CTL=$(mktemp -d)
MNT_BOOTA=$(mktemp -d)
MNT_BOOTB=$(mktemp -d)
MNT_DATA=$(mktemp -d)
cleanup() {
umount "$MNT_CTL" 2>/dev/null || true
umount "$MNT_BOOTA" 2>/dev/null || true
umount "$MNT_BOOTB" 2>/dev/null || true
umount "$MNT_DATA" 2>/dev/null || true
if [ "$USE_KPARTX" = true ]; then
kpartx -d "$LOOP" 2>/dev/null || true
fi
losetup -d "$LOOP" 2>/dev/null || true
rm -rf "$MNT_CTL" "$MNT_BOOTA" "$MNT_BOOTB" "$MNT_DATA" 2>/dev/null || true
}
trap cleanup EXIT
# --- Format partitions ---
mkfs.vfat -F 32 -n KSOLOCTL "$P1"
mkfs.vfat -F 32 -n KSOLOA "$P2"
mkfs.vfat -F 32 -n KSOLOB "$P3"
mkfs.ext4 -q -L KSOLODATA "$P4"
# --- Mount all partitions ---
mount "$P1" "$MNT_CTL"
mount "$P2" "$MNT_BOOTA"
mount "$P3" "$MNT_BOOTB"
mount "$P4" "$MNT_DATA"
# --- Helper: populate a boot partition ---
populate_boot_partition() {
local MNT="$1"
local LABEL="$2"
echo " Populating $LABEL..."
# config.txt — Raspberry Pi boot configuration
cat > "$MNT/config.txt" << 'CFGTXT'
arm_64bit=1
kernel=kernel8.img
initramfs kubesolo-os.gz followkernel
enable_uart=1
gpu_mem=16
dtoverlay=disable-wifi
dtoverlay=disable-bt
CFGTXT
# cmdline.txt — kernel command line
# Note: must be a single line
echo "console=serial0,115200 console=tty1 kubesolo.data=LABEL=KSOLODATA quiet" > "$MNT/cmdline.txt"
# Copy kernel as kernel8.img (RPi 3/4/5 ARM64 convention)
cp "$KERNEL" "$MNT/kernel8.img"
# Copy initramfs
cp "$INITRAMFS" "$MNT/kubesolo-os.gz"
# Copy DTBs from kernel build (MUST match kernel to avoid driver probe failures)
if ls "$KERNEL_DTBS_DIR"/bcm27*.dtb 1>/dev/null 2>&1; then
cp "$KERNEL_DTBS_DIR"/bcm27*.dtb "$MNT/"
fi
# Copy overlays — prefer kernel-built, fall back to firmware repo
if [ -d "$KERNEL_DTBS_DIR/overlays" ]; then
cp -r "$KERNEL_DTBS_DIR/overlays" "$MNT/"
elif [ -d "$RPI_FIRMWARE_DIR/overlays" ]; then
cp -r "$RPI_FIRMWARE_DIR/overlays" "$MNT/"
fi
# Write version marker
echo "$VERSION" > "$MNT/version.txt"
}
# --- Boot Control Partition (KSOLOCTL) ---
# Partition 1 serves dual purpose:
# 1. Contains firmware + autoboot.txt for A/B redirect (if EEPROM supports it)
# 2. Contains full boot files (kernel + initramfs) as fallback if autoboot.txt isn't supported
echo " Writing firmware + autoboot.txt + boot files to partition 1..."
# autoboot.txt — tells firmware which partition to boot from (A/B switching)
# If the EEPROM doesn't support this, it's silently ignored and the firmware
# falls back to booting from partition 1 using config.txt below.
cat > "$MNT_CTL/autoboot.txt" << 'AUTOBOOT'
[all]
tryboot_a_b=1
boot_partition=2
[tryboot]
boot_partition=3
AUTOBOOT
# Copy firmware blobs — REQUIRED on partition 1 for EEPROM to boot
if ls "$RPI_FIRMWARE_DIR"/start*.elf 1>/dev/null 2>&1; then
cp "$RPI_FIRMWARE_DIR"/start*.elf "$MNT_CTL/"
fi
if ls "$RPI_FIRMWARE_DIR"/fixup*.dat 1>/dev/null 2>&1; then
cp "$RPI_FIRMWARE_DIR"/fixup*.dat "$MNT_CTL/"
fi
if [ -f "$RPI_FIRMWARE_DIR/bootcode.bin" ]; then
cp "$RPI_FIRMWARE_DIR/bootcode.bin" "$MNT_CTL/"
fi
# Full boot files on partition 1 — fallback if autoboot.txt redirect doesn't work.
# When autoboot.txt works, firmware switches to partition 2 and reads config.txt there.
# When autoboot.txt is unsupported, firmware reads THIS config.txt and boots from here.
populate_boot_partition "$MNT_CTL" "Boot Control (KSOLOCTL)"
# --- Boot A Partition (KSOLOA) ---
populate_boot_partition "$MNT_BOOTA" "Boot A (KSOLOA)"
# --- Boot B Partition (KSOLOB, initially identical) ---
populate_boot_partition "$MNT_BOOTB" "Boot B (KSOLOB)"
# --- Data Partition (KSOLODATA) ---
echo " Preparing data partition..."
for dir in kubesolo containerd etc-kubesolo log usr-local network images; do
mkdir -p "$MNT_DATA/$dir"
done
sync
echo ""
echo "==> Raspberry Pi disk image created: $IMG_OUTPUT"
echo " Size: $(du -h "$IMG_OUTPUT" | cut -f1)"
echo " Part 1 (KSOLOCTL): Firmware + kernel + initramfs + autoboot.txt (boot/control)"
echo " Part 2 (KSOLOA): Boot A — kernel + initramfs + DTBs"
echo " Part 3 (KSOLOB): Boot B — kernel + initramfs + DTBs"
echo " Part 4 (KSOLODATA): Persistent K8s state"
echo ""
echo "Write to SD card with:"
echo " sudo dd if=$IMG_OUTPUT of=/dev/sdX bs=4M status=progress"
echo ""

View File

@@ -10,6 +10,111 @@ ROOTFS_DIR="${ROOTFS_DIR:-$PROJECT_ROOT/build/rootfs-work}"
# shellcheck source=../config/versions.env
. "$SCRIPT_DIR/../config/versions.env"
EXTRACT_ARCH="${TARGET_ARCH:-amd64}"
# Clean previous rootfs
rm -rf "$ROOTFS_DIR"
mkdir -p "$ROOTFS_DIR"
# =========================================================================
# ARM64: piCore64 .img.gz extraction (SD card image, not ISO)
# =========================================================================
if [ "$EXTRACT_ARCH" = "arm64" ]; then
PICORE_IMG="$CACHE_DIR/$PICORE_IMAGE"
if [ ! -f "$PICORE_IMG" ]; then
echo "ERROR: piCore64 image not found: $PICORE_IMG"
echo "Run 'TARGET_ARCH=arm64 make fetch' first."
exit 1
fi
echo "==> Extracting piCore64 image: $PICORE_IMG"
# Decompress to raw image (.img.gz or .zip)
PICORE_RAW="$CACHE_DIR/piCore-${PICORE_VERSION}.img"
if [ ! -f "$PICORE_RAW" ]; then
echo " Decompressing..."
case "$PICORE_IMG" in
*.zip)
unzip -o -j "$PICORE_IMG" '*.img' -d "$CACHE_DIR" 2>/dev/null || \
unzip -o "$PICORE_IMG" -d "$CACHE_DIR"
# Find the extracted .img file
EXTRACTED_IMG=$(find "$CACHE_DIR" -maxdepth 1 -name '*.img' -newer "$PICORE_IMG" | head -1)
if [ -n "$EXTRACTED_IMG" ] && [ "$EXTRACTED_IMG" != "$PICORE_RAW" ]; then
mv "$EXTRACTED_IMG" "$PICORE_RAW"
fi
;;
*.img.gz)
gunzip -k "$PICORE_IMG" 2>/dev/null || \
zcat "$PICORE_IMG" > "$PICORE_RAW"
;;
*)
echo "ERROR: Unknown piCore image format: $PICORE_IMG"
exit 1
;;
esac
fi
# Mount the piCore boot partition (partition 1) to find kernel/initramfs
# piCore layout: p1=boot (FAT32, has kernel+initramfs), p2=rootfs (ext4, has tce/)
IMG_MNT=$(mktemp -d)
echo " Mounting piCore boot partition..."
# Get partition 1 offset (boot/FAT partition with kernel+initramfs)
OFFSET=$(fdisk -l "$PICORE_RAW" 2>/dev/null | awk '/^.*img1/{print $2}')
if [ -z "$OFFSET" ]; then
# Fallback: try sfdisk (first partition)
OFFSET=$(sfdisk -d "$PICORE_RAW" 2>/dev/null | awk -F'[=,]' '/start=/{print $2; exit}' | tr -d ' ')
fi
if [ -z "$OFFSET" ]; then
echo "ERROR: Could not determine partition offset in piCore image"
fdisk -l "$PICORE_RAW" || true
exit 1
fi
BYTE_OFFSET=$((OFFSET * 512))
mount -o loop,ro,offset="$BYTE_OFFSET" "$PICORE_RAW" "$IMG_MNT" || {
echo "ERROR: Failed to mount piCore boot partition (need root for losetup)"
exit 1
}
# Find initramfs in the piCore boot partition
COREGZ=""
for f in "$IMG_MNT"/rootfs-piCore64*.gz "$IMG_MNT"/boot/corepure64.gz "$IMG_MNT"/boot/core.gz "$IMG_MNT"/corepure64.gz "$IMG_MNT"/core.gz; do
[ -f "$f" ] && COREGZ="$f" && break
done
if [ -z "$COREGZ" ]; then
echo "ERROR: Could not find initramfs in piCore image"
echo "Contents:"
ls -la "$IMG_MNT"/
ls -la "$IMG_MNT"/boot/ 2>/dev/null || true
umount "$IMG_MNT" 2>/dev/null || true
exit 1
fi
echo "==> Found initramfs: $COREGZ"
# Extract initramfs
mkdir -p "$ROOTFS_DIR/rootfs"
cd "$ROOTFS_DIR/rootfs"
zcat "$COREGZ" | cpio -idm 2>/dev/null
# Note: ARM64 kernel comes from build-kernel-arm64.sh, not from piCore
# We only use piCore for the BusyBox userland
cd "$PROJECT_ROOT"
umount "$IMG_MNT" 2>/dev/null || true
rm -rf "$IMG_MNT"
echo "==> ARM64 rootfs extracted: $ROOTFS_DIR/rootfs"
echo " Size: $(du -sh "$ROOTFS_DIR/rootfs" | cut -f1)"
echo "==> Extract complete (ARM64). Kernel will come from build-kernel-arm64.sh"
exit 0
fi
# =========================================================================
# x86_64: Tiny Core ISO extraction
# =========================================================================
TC_ISO="$CACHE_DIR/$TINYCORE_ISO"
ISO_MNT="$ROOTFS_DIR/iso-mount"
@@ -19,9 +124,7 @@ if [ ! -f "$TC_ISO" ]; then
exit 1
fi
# Clean previous rootfs
rm -rf "$ROOTFS_DIR"
mkdir -p "$ROOTFS_DIR" "$ISO_MNT"
mkdir -p "$ISO_MNT"
# --- Mount ISO and extract kernel + initramfs ---
echo "==> Mounting ISO: $TC_ISO"

View File

@@ -10,9 +10,89 @@ CACHE_DIR="${CACHE_DIR:-$PROJECT_ROOT/build/cache}"
# shellcheck source=../config/versions.env
. "$SCRIPT_DIR/../config/versions.env"
# Verify SHA256 checksum of a downloaded file
verify_checksum() {
local file="$1" expected="$2" name="$3"
# Skip if no expected checksum provided
[ -z "$expected" ] && return 0
local actual
actual=$(sha256sum "$file" | awk '{print $1}')
if [ "$actual" = "$expected" ]; then
echo " Checksum OK: $name"
return 0
else
echo "ERROR: Checksum mismatch for $name"
echo " Expected: $expected"
echo " Got: $actual"
rm -f "$file"
return 1
fi
}
mkdir -p "$CACHE_DIR"
# --- Tiny Core Linux ISO ---
# Detect target architecture
FETCH_ARCH="${TARGET_ARCH:-amd64}"
# --- ARM64: piCore64 image instead of x86_64 ISO ---
if [ "$FETCH_ARCH" = "arm64" ]; then
PICORE_IMG="$CACHE_DIR/$PICORE_IMAGE"
if [ -f "$PICORE_IMG" ]; then
echo "==> piCore64 image already cached: $PICORE_IMG"
else
echo "==> Downloading piCore64 ${PICORE_VERSION} (${PICORE_ARCH})..."
echo " URL: $PICORE_IMAGE_URL"
wget -q --show-progress -O "$PICORE_IMG" "$PICORE_IMAGE_URL" 2>/dev/null || \
curl -fSL "$PICORE_IMAGE_URL" -o "$PICORE_IMG"
echo "==> Downloaded: $PICORE_IMG ($(du -h "$PICORE_IMG" | cut -f1))"
fi
# Also fetch RPi firmware
echo "==> Fetching RPi firmware..."
"$SCRIPT_DIR/fetch-rpi-firmware.sh"
# Download ARM64 KubeSolo binary
KUBESOLO_VERSION="${KUBESOLO_VERSION:-v1.1.0}"
KUBESOLO_BIN_ARM64="$CACHE_DIR/kubesolo-arm64"
if [ -f "$KUBESOLO_BIN_ARM64" ]; then
echo "==> KubeSolo ARM64 binary already cached: $KUBESOLO_BIN_ARM64"
else
echo "==> Downloading KubeSolo ${KUBESOLO_VERSION} (arm64)..."
BIN_URL="https://github.com/portainer/kubesolo/releases/download/${KUBESOLO_VERSION}/kubesolo-${KUBESOLO_VERSION}-linux-arm64-musl.tar.gz"
BIN_URL_FALLBACK="https://github.com/portainer/kubesolo/releases/download/${KUBESOLO_VERSION}/kubesolo-${KUBESOLO_VERSION}-linux-arm64.tar.gz"
TEMP_DIR=$(mktemp -d)
echo " URL: $BIN_URL"
if curl -fSL "$BIN_URL" -o "$TEMP_DIR/kubesolo.tar.gz" 2>/dev/null; then
echo " Downloaded musl variant (arm64)"
elif curl -fSL "$BIN_URL_FALLBACK" -o "$TEMP_DIR/kubesolo.tar.gz" 2>/dev/null; then
echo " Downloaded glibc variant (arm64 fallback)"
else
echo "ERROR: Failed to download KubeSolo ARM64 from GitHub."
rm -rf "$TEMP_DIR"
exit 1
fi
tar -xzf "$TEMP_DIR/kubesolo.tar.gz" -C "$TEMP_DIR"
FOUND_BIN=$(find "$TEMP_DIR" -name "kubesolo" -type f ! -name "*.tar.gz" | head -1)
if [ -z "$FOUND_BIN" ]; then
echo "ERROR: Could not find kubesolo binary in extracted archive"
rm -rf "$TEMP_DIR"
exit 1
fi
cp "$FOUND_BIN" "$KUBESOLO_BIN_ARM64"
chmod +x "$KUBESOLO_BIN_ARM64"
rm -rf "$TEMP_DIR"
echo "==> KubeSolo ARM64 binary: $KUBESOLO_BIN_ARM64 ($(du -h "$KUBESOLO_BIN_ARM64" | cut -f1))"
fi
# Skip x86_64 ISO and TCZ downloads for ARM64
echo ""
echo "==> ARM64 fetch complete."
echo "==> Component cache:"
ls -lh "$CACHE_DIR"/ 2>/dev/null || true
exit 0
fi
# --- x86_64: Tiny Core Linux ISO ---
TC_ISO="$CACHE_DIR/$TINYCORE_ISO"
TC_URL="${TINYCORE_MIRROR}/${TINYCORE_VERSION%%.*}.x/${TINYCORE_ARCH}/release/${TINYCORE_ISO}"
@@ -28,6 +108,7 @@ else
wget -q --show-progress -O "$TC_ISO" "$TC_URL_ALT"
}
echo "==> Downloaded: $TC_ISO ($(du -h "$TC_ISO" | cut -f1))"
verify_checksum "$TC_ISO" "$TINYCORE_ISO_SHA256" "Tiny Core ISO"
fi
# --- KubeSolo ---
@@ -50,7 +131,7 @@ else
BIN_URL_FALLBACK="https://github.com/portainer/kubesolo/releases/download/${KUBESOLO_VERSION}/kubesolo-${KUBESOLO_VERSION}-${OS}-${ARCH}.tar.gz"
TEMP_DIR=$(mktemp -d)
trap "rm -rf '$TEMP_DIR'" EXIT
trap 'rm -rf "$TEMP_DIR"' EXIT
echo " URL: $BIN_URL"
if curl -fSL "$BIN_URL" -o "$TEMP_DIR/kubesolo.tar.gz" 2>/dev/null; then
@@ -88,6 +169,7 @@ else
rm -rf "$TEMP_DIR"
echo "==> KubeSolo binary: $KUBESOLO_BIN ($(du -h "$KUBESOLO_BIN" | cut -f1))"
verify_checksum "$KUBESOLO_BIN" "$KUBESOLO_SHA256" "KubeSolo binary"
fi
# --- Tiny Core kernel module extensions (netfilter, iptables) ---
@@ -114,6 +196,7 @@ else
if wget -q --show-progress -O "$NETFILTER_TCZ" "$NETFILTER_TCZ_URL" 2>/dev/null || \
curl -fSL "$NETFILTER_TCZ_URL" -o "$NETFILTER_TCZ" 2>/dev/null; then
echo "==> Downloaded: $NETFILTER_TCZ ($(du -h "$NETFILTER_TCZ" | cut -f1))"
verify_checksum "$NETFILTER_TCZ" "$NETFILTER_TCZ_SHA256" "netfilter TCZ"
else
echo "WARN: Failed to download netfilter modules. kube-proxy may not work."
rm -f "$NETFILTER_TCZ"
@@ -131,6 +214,7 @@ else
if wget -q --show-progress -O "$NET_BRIDGING_TCZ" "$NET_BRIDGING_TCZ_URL" 2>/dev/null || \
curl -fSL "$NET_BRIDGING_TCZ_URL" -o "$NET_BRIDGING_TCZ" 2>/dev/null; then
echo "==> Downloaded: $NET_BRIDGING_TCZ ($(du -h "$NET_BRIDGING_TCZ" | cut -f1))"
verify_checksum "$NET_BRIDGING_TCZ" "$NET_BRIDGING_TCZ_SHA256" "net-bridging TCZ"
else
echo "WARN: Failed to download net-bridging modules. CNI bridge may not work."
rm -f "$NET_BRIDGING_TCZ"
@@ -148,6 +232,7 @@ else
if wget -q --show-progress -O "$IPTABLES_TCZ" "$IPTABLES_TCZ_URL" 2>/dev/null || \
curl -fSL "$IPTABLES_TCZ_URL" -o "$IPTABLES_TCZ" 2>/dev/null; then
echo "==> Downloaded: $IPTABLES_TCZ ($(du -h "$IPTABLES_TCZ" | cut -f1))"
verify_checksum "$IPTABLES_TCZ" "$IPTABLES_TCZ_SHA256" "iptables TCZ"
else
echo "WARN: Failed to download iptables. KubeSolo bundles its own but this is a fallback."
rm -f "$IPTABLES_TCZ"

View File

@@ -0,0 +1,88 @@
#!/bin/bash
# fetch-rpi-firmware.sh — Download Raspberry Pi firmware blobs for boot
#
# Downloads firmware from the official raspberrypi/firmware GitHub repository.
# Extracts only the boot files needed: start*.elf, fixup*.dat, DTBs, bootcode.bin.
#
# Output: build/cache/rpi-firmware/ containing all required boot files.
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
CACHE_DIR="${CACHE_DIR:-$PROJECT_ROOT/build/cache}"
# shellcheck source=../config/versions.env
. "$SCRIPT_DIR/../config/versions.env"
RPI_FW_DIR="$CACHE_DIR/rpi-firmware"
RPI_FW_ARCHIVE="$CACHE_DIR/rpi-firmware-${RPI_FIRMWARE_TAG}.tar.gz"
# --- Skip if already fetched ---
if [ -d "$RPI_FW_DIR" ] && [ -f "$RPI_FW_DIR/start4.elf" ]; then
echo "==> RPi firmware already cached: $RPI_FW_DIR"
echo " Files: $(ls "$RPI_FW_DIR" | wc -l)"
exit 0
fi
echo "==> Downloading Raspberry Pi firmware (tag: ${RPI_FIRMWARE_TAG})..."
mkdir -p "$CACHE_DIR" "$RPI_FW_DIR"
# --- Download firmware archive ---
if [ ! -f "$RPI_FW_ARCHIVE" ]; then
echo " URL: $RPI_FIRMWARE_URL"
wget -q --show-progress -O "$RPI_FW_ARCHIVE" "$RPI_FIRMWARE_URL" 2>/dev/null || \
curl -fSL "$RPI_FIRMWARE_URL" -o "$RPI_FW_ARCHIVE"
echo " Downloaded: $(du -h "$RPI_FW_ARCHIVE" | cut -f1)"
else
echo " Archive already cached: $(du -h "$RPI_FW_ARCHIVE" | cut -f1)"
fi
# --- Extract boot files only ---
echo "==> Extracting boot files..."
TEMP_DIR=$(mktemp -d)
trap 'rm -rf "$TEMP_DIR"' EXIT
# Extract only the boot/ directory from the archive
# Archive structure: firmware-<tag>/boot/...
tar -xzf "$RPI_FW_ARCHIVE" -C "$TEMP_DIR" --strip-components=1 --wildcards '*/boot/'
BOOT_SRC="$TEMP_DIR/boot"
if [ ! -d "$BOOT_SRC" ]; then
echo "ERROR: boot/ directory not found in firmware archive"
ls -la "$TEMP_DIR"/
exit 1
fi
# Copy GPU firmware (required for boot)
for f in "$BOOT_SRC"/start*.elf "$BOOT_SRC"/fixup*.dat; do
[ -f "$f" ] && cp "$f" "$RPI_FW_DIR/"
done
# Copy bootcode.bin (first-stage boot for Pi 3 and older)
[ -f "$BOOT_SRC/bootcode.bin" ] && cp "$BOOT_SRC/bootcode.bin" "$RPI_FW_DIR/"
# Copy Device Tree Blobs for Pi 4 + Pi 5
for dtb in bcm2711-rpi-4-b.dtb bcm2711-rpi-400.dtb bcm2711-rpi-cm4.dtb \
bcm2712-rpi-5-b.dtb bcm2712d0-rpi-5-b.dtb; do
[ -f "$BOOT_SRC/$dtb" ] && cp "$BOOT_SRC/$dtb" "$RPI_FW_DIR/"
done
# Copy overlays directory (needed for config.txt dtoverlay= directives)
if [ -d "$BOOT_SRC/overlays" ]; then
mkdir -p "$RPI_FW_DIR/overlays"
# Only copy overlays we actually use (disable-wifi, disable-bt)
for overlay in disable-wifi.dtbo disable-bt.dtbo; do
[ -f "$BOOT_SRC/overlays/$overlay" ] && \
cp "$BOOT_SRC/overlays/$overlay" "$RPI_FW_DIR/overlays/"
done
fi
trap - EXIT
rm -rf "$TEMP_DIR"
# --- Summary ---
echo ""
echo "==> RPi firmware extracted to: $RPI_FW_DIR"
echo " Files:"
ls -1 "$RPI_FW_DIR" | head -20
echo " Total size: $(du -sh "$RPI_FW_DIR" | cut -f1)"

View File

@@ -8,6 +8,16 @@ CACHE_DIR="${CACHE_DIR:-$PROJECT_ROOT/build/cache}"
ROOTFS_DIR="${ROOTFS_DIR:-$PROJECT_ROOT/build/rootfs-work}"
ROOTFS="$ROOTFS_DIR/rootfs"
VERSION="$(cat "$PROJECT_ROOT/VERSION")"
INJECT_ARCH="${TARGET_ARCH:-amd64}"
# Architecture-specific paths
if [ "$INJECT_ARCH" = "arm64" ]; then
LIB_ARCH="aarch64-linux-gnu"
LD_SO="/lib/ld-linux-aarch64.so.1"
else
LIB_ARCH="x86_64-linux-gnu"
LD_SO="/lib64/ld-linux-x86-64.so.2"
fi
if [ ! -d "$ROOTFS" ]; then
echo "ERROR: Rootfs not found: $ROOTFS"
@@ -15,7 +25,11 @@ if [ ! -d "$ROOTFS" ]; then
exit 1
fi
KUBESOLO_BIN="$CACHE_DIR/kubesolo"
if [ "$INJECT_ARCH" = "arm64" ]; then
KUBESOLO_BIN="$CACHE_DIR/kubesolo-arm64"
else
KUBESOLO_BIN="$CACHE_DIR/kubesolo"
fi
if [ ! -f "$KUBESOLO_BIN" ]; then
echo "ERROR: KubeSolo binary not found: $KUBESOLO_BIN"
echo "See fetch-components.sh output for instructions."
@@ -68,30 +82,39 @@ for lib in network.sh health.sh; do
done
# Cloud-init binary (Go, built separately)
CLOUDINIT_BIN="$CACHE_DIR/kubesolo-cloudinit"
# Try arch-specific binary first, then fall back to generic
CLOUDINIT_BIN="$CACHE_DIR/kubesolo-cloudinit-linux-$INJECT_ARCH"
[ ! -f "$CLOUDINIT_BIN" ] && CLOUDINIT_BIN="$CACHE_DIR/kubesolo-cloudinit"
if [ -f "$CLOUDINIT_BIN" ]; then
cp "$CLOUDINIT_BIN" "$ROOTFS/usr/lib/kubesolo-os/kubesolo-cloudinit"
chmod +x "$ROOTFS/usr/lib/kubesolo-os/kubesolo-cloudinit"
echo " Installed cloud-init binary ($(du -h "$CLOUDINIT_BIN" | cut -f1))"
else
echo " WARN: Cloud-init binary not found (run 'make build-cloudinit' to build)"
echo " WARN: Cloud-init binary not found (run 'make build-cloudinit' or 'make build-cross' to build)"
fi
# Update agent binary (Go, built separately)
UPDATE_BIN="$CACHE_DIR/kubesolo-update"
# Try arch-specific binary first, then fall back to generic
UPDATE_BIN="$CACHE_DIR/kubesolo-update-linux-$INJECT_ARCH"
[ ! -f "$UPDATE_BIN" ] && UPDATE_BIN="$CACHE_DIR/kubesolo-update"
if [ -f "$UPDATE_BIN" ]; then
cp "$UPDATE_BIN" "$ROOTFS/usr/lib/kubesolo-os/kubesolo-update"
chmod +x "$ROOTFS/usr/lib/kubesolo-os/kubesolo-update"
echo " Installed update agent ($(du -h "$UPDATE_BIN" | cut -f1))"
else
echo " WARN: Update agent not found (run 'make build-update-agent' to build)"
echo " WARN: Update agent not found (run 'make build-update-agent' or 'make build-cross' to build)"
fi
# --- 3. Custom kernel or TCZ kernel modules ---
# If a custom kernel was built (with CONFIG_CGROUP_BPF=y), use it.
# Otherwise fall back to TCZ-extracted modules with manual modules.dep.
CUSTOM_KERNEL_DIR="$CACHE_DIR/custom-kernel"
CUSTOM_VMLINUZ="$CUSTOM_KERNEL_DIR/vmlinuz"
if [ "$INJECT_ARCH" = "arm64" ]; then
CUSTOM_KERNEL_DIR="$CACHE_DIR/custom-kernel-arm64"
CUSTOM_VMLINUZ="$CUSTOM_KERNEL_DIR/Image"
else
CUSTOM_KERNEL_DIR="$CACHE_DIR/custom-kernel"
CUSTOM_VMLINUZ="$CUSTOM_KERNEL_DIR/vmlinuz"
fi
CUSTOM_MODULES="$CUSTOM_KERNEL_DIR/modules"
# Detect kernel version from rootfs
@@ -100,8 +123,16 @@ for d in "$ROOTFS"/lib/modules/*/; do
[ -d "$d" ] && KVER="$(basename "$d")" && break
done
# Fallback: detect from custom kernel modules directory
if [ -z "$KVER" ] && [ -d "$CUSTOM_MODULES/lib/modules" ]; then
for d in "$CUSTOM_MODULES"/lib/modules/*/; do
[ -d "$d" ] && KVER="$(basename "$d")" && break
done
echo " Detected kernel version from custom kernel: $KVER"
fi
if [ -z "$KVER" ]; then
echo " WARN: Could not detect kernel version from rootfs"
echo " WARN: Could not detect kernel version from rootfs or custom kernel"
fi
echo " Kernel version: $KVER"
@@ -130,20 +161,49 @@ if [ -f "$CUSTOM_VMLINUZ" ] && [ -d "$CUSTOM_MODULES/lib/modules/$KVER" ]; then
[ -f "$CUSTOM_MOD_DIR/$f" ] && cp "$CUSTOM_MOD_DIR/$f" "$ROOTFS/lib/modules/$KVER/"
done
# Use modprobe --show-depends to resolve each module + its transitive deps
MODULES_LIST="$PROJECT_ROOT/build/config/modules.list"
# Resolve and install modules from modules.list + transitive deps
if [ "$INJECT_ARCH" = "arm64" ]; then
MODULES_LIST="$PROJECT_ROOT/build/config/modules-arm64.list"
else
MODULES_LIST="$PROJECT_ROOT/build/config/modules.list"
fi
NEEDED_MODS=$(mktemp)
# Try modprobe first (works for same-arch builds)
MODPROBE_WORKS=true
FIRST_MOD=$(grep -v '^#' "$MODULES_LIST" | grep -v '^$' | head -1 | xargs)
if ! modprobe -S "$KVER" -d "$CUSTOM_MODULES" --show-depends "$FIRST_MOD" >/dev/null 2>&1; then
MODPROBE_WORKS=false
echo " modprobe cannot resolve modules (cross-arch build) — using find fallback"
fi
while IFS= read -r mod; do
# Skip comments and blank lines
case "$mod" in \#*|"") continue ;; esac
mod=$(echo "$mod" | xargs) # trim whitespace
[ -z "$mod" ] && continue
# modprobe -S <ver> -d <root> --show-depends <module> lists all deps in load order
# Output format: "insmod /path/to/module.ko" — extract path with awk
modprobe -S "$KVER" -d "$CUSTOM_MODULES" --show-depends "$mod" 2>/dev/null \
| awk '/^insmod/{print $2}' >> "$NEEDED_MODS" \
|| echo " WARN: modprobe could not resolve: $mod"
if [ "$MODPROBE_WORKS" = true ]; then
# modprobe -S <ver> -d <root> --show-depends <module> lists all deps in load order
modprobe -S "$KVER" -d "$CUSTOM_MODULES" --show-depends "$mod" 2>/dev/null \
| awk '/^insmod/{print $2}' >> "$NEEDED_MODS" \
|| echo " WARN: modprobe could not resolve: $mod"
else
# Cross-arch fallback: find module by name in kernel tree
found=$(find "$CUSTOM_MOD_DIR/kernel" -name "${mod}.ko" -o -name "${mod}.ko.xz" -o -name "${mod}.ko.gz" -o -name "${mod}.ko.zst" 2>/dev/null | head -1)
if [ -n "$found" ]; then
echo "$found" >> "$NEEDED_MODS"
else
# Try replacing hyphens with underscores and vice versa
mod_alt=$(echo "$mod" | tr '-' '_')
found=$(find "$CUSTOM_MOD_DIR/kernel" -name "${mod_alt}.ko" -o -name "${mod_alt}.ko.xz" -o -name "${mod_alt}.ko.gz" -o -name "${mod_alt}.ko.zst" 2>/dev/null | head -1)
if [ -n "$found" ]; then
echo "$found" >> "$NEEDED_MODS"
else
echo " WARN: could not find module: $mod"
fi
fi
fi
done < "$MODULES_LIST"
# Deduplicate and copy each needed module
@@ -291,21 +351,22 @@ if [ -f /usr/sbin/xtables-nft-multi ]; then
ln -sf xtables-nft-multi "$ROOTFS/usr/sbin/$cmd"
done
# Copy required shared libraries
mkdir -p "$ROOTFS/usr/lib/x86_64-linux-gnu" "$ROOTFS/lib/x86_64-linux-gnu" "$ROOTFS/lib64"
# Copy required shared libraries (architecture-aware paths)
mkdir -p "$ROOTFS/usr/lib/$LIB_ARCH" "$ROOTFS/lib/$LIB_ARCH"
[ "$INJECT_ARCH" != "arm64" ] && mkdir -p "$ROOTFS/lib64"
for lib in \
/lib/x86_64-linux-gnu/libxtables.so.12* \
/lib/x86_64-linux-gnu/libmnl.so.0* \
/lib/x86_64-linux-gnu/libnftnl.so.11* \
/lib/x86_64-linux-gnu/libc.so.6 \
/lib64/ld-linux-x86-64.so.2; do
"/lib/$LIB_ARCH/libxtables.so.12"* \
"/lib/$LIB_ARCH/libmnl.so.0"* \
"/lib/$LIB_ARCH/libnftnl.so.11"* \
"/lib/$LIB_ARCH/libc.so.6" \
"$LD_SO"; do
[ -e "$lib" ] && cp -aL "$lib" "$ROOTFS${lib}" 2>/dev/null || true
done
# Copy xtables modules directory (match extensions)
if [ -d /usr/lib/x86_64-linux-gnu/xtables ]; then
mkdir -p "$ROOTFS/usr/lib/x86_64-linux-gnu/xtables"
cp -a /usr/lib/x86_64-linux-gnu/xtables/*.so "$ROOTFS/usr/lib/x86_64-linux-gnu/xtables/" 2>/dev/null || true
if [ -d "/usr/lib/$LIB_ARCH/xtables" ]; then
mkdir -p "$ROOTFS/usr/lib/$LIB_ARCH/xtables"
cp -a "/usr/lib/$LIB_ARCH/xtables/"*.so "$ROOTFS/usr/lib/$LIB_ARCH/xtables/" 2>/dev/null || true
fi
echo " Installed iptables-nft (xtables-nft-multi) + shared libs"
@@ -314,11 +375,16 @@ else
fi
# Kernel modules list (for init to load at boot)
cp "$PROJECT_ROOT/build/config/modules.list" "$ROOTFS/usr/lib/kubesolo-os/modules.list"
if [ "$INJECT_ARCH" = "arm64" ]; then
cp "$PROJECT_ROOT/build/config/modules-arm64.list" "$ROOTFS/usr/lib/kubesolo-os/modules.list"
else
cp "$PROJECT_ROOT/build/config/modules.list" "$ROOTFS/usr/lib/kubesolo-os/modules.list"
fi
# --- 4. Sysctl config ---
mkdir -p "$ROOTFS/etc/sysctl.d"
cp "$PROJECT_ROOT/build/rootfs/etc/sysctl.d/k8s.conf" "$ROOTFS/etc/sysctl.d/k8s.conf"
cp "$PROJECT_ROOT/build/rootfs/etc/sysctl.d/security.conf" "$ROOTFS/etc/sysctl.d/security.conf"
# --- 5. OS metadata ---
echo "$VERSION" > "$ROOTFS/etc/kubesolo-os-version"
@@ -362,7 +428,35 @@ else
echo " WARN: No CA certificates found in builder — TLS verification will fail"
fi
# --- 9. Ensure /etc/hosts and /etc/resolv.conf exist ---
# --- 9. AppArmor parser + profiles ---
echo " Installing AppArmor..."
if [ -f /usr/sbin/apparmor_parser ]; then
mkdir -p "$ROOTFS/usr/sbin"
cp /usr/sbin/apparmor_parser "$ROOTFS/usr/sbin/apparmor_parser"
chmod +x "$ROOTFS/usr/sbin/apparmor_parser"
# Copy shared libraries required by apparmor_parser
for lib in "/lib/$LIB_ARCH/libapparmor.so.1"*; do
[ -e "$lib" ] && cp -aL "$lib" "$ROOTFS${lib}" 2>/dev/null || true
done
echo " Installed apparmor_parser + shared libs"
else
echo " WARN: apparmor_parser not found in builder (install apparmor package)"
fi
# Copy AppArmor profiles
APPARMOR_PROFILES="$PROJECT_ROOT/build/rootfs/etc/apparmor.d"
if [ -d "$APPARMOR_PROFILES" ]; then
mkdir -p "$ROOTFS/etc/apparmor.d"
cp "$APPARMOR_PROFILES"/* "$ROOTFS/etc/apparmor.d/" 2>/dev/null || true
PROFILE_COUNT=$(ls "$ROOTFS/etc/apparmor.d/" 2>/dev/null | wc -l)
echo " Installed $PROFILE_COUNT AppArmor profiles"
else
echo " WARN: No AppArmor profiles found at $APPARMOR_PROFILES"
fi
# --- 10. Ensure /etc/hosts and /etc/resolv.conf exist ---
if [ ! -f "$ROOTFS/etc/hosts" ]; then
cat > "$ROOTFS/etc/hosts" << EOF
127.0.0.1 localhost

View File

@@ -31,9 +31,15 @@ type NetworkConfig struct {
// KubeSoloConfig defines KubeSolo-specific settings.
type KubeSoloConfig struct {
ExtraFlags string `yaml:"extra-flags"`
LocalStorage *bool `yaml:"local-storage"`
ExtraSANs []string `yaml:"apiserver-extra-sans"`
ExtraFlags string `yaml:"extra-flags"`
LocalStorage *bool `yaml:"local-storage"`
LocalStorageSharedPath string `yaml:"local-storage-shared-path"`
ExtraSANs []string `yaml:"apiserver-extra-sans"`
Debug bool `yaml:"debug"`
PprofServer bool `yaml:"pprof-server"`
PortainerEdgeID string `yaml:"portainer-edge-id"`
PortainerEdgeKey string `yaml:"portainer-edge-key"`
PortainerEdgeAsync bool `yaml:"portainer-edge-async"`
}
// NTPConfig defines NTP settings.

View File

@@ -0,0 +1,40 @@
# KubeSolo OS Cloud-Init — Full Configuration Reference
# Shows ALL supported KubeSolo parameters.
# Place at: /mnt/data/etc-kubesolo/cloud-init.yaml (on data partition)
# Or pass via boot param: kubesolo.cloudinit=/path/to/this.yaml
hostname: kubesolo-edge-01
network:
mode: dhcp
# interface: eth0 # Optional: specify interface (auto-detected if omitted)
# dns: # Optional: override DHCP-provided DNS
# - 8.8.8.8
kubesolo:
# Enable local-path-provisioner for persistent volumes (default: true)
local-storage: true
# Shared path for local-path-provisioner storage
local-storage-shared-path: "/mnt/shared"
# Extra SANs for API server TLS certificate
apiserver-extra-sans:
- kubesolo-edge-01.local
- 192.168.1.100
# Enable verbose debug logging
debug: false
# Enable Go pprof profiling server
pprof-server: false
# Portainer Edge Agent connection (alternative to portainer.edge-agent section)
# These generate --portainer-edge-id, --portainer-edge-key, --portainer-edge-async
# CLI flags for KubeSolo's built-in Edge Agent support.
portainer-edge-id: "your-edge-id"
portainer-edge-key: "your-edge-key"
portainer-edge-async: true
# Arbitrary extra flags passed directly to the KubeSolo binary
# extra-flags: "--disable traefik --disable servicelb"

View File

@@ -46,6 +46,30 @@ func buildExtraFlags(cfg *Config) string {
parts = append(parts, "--apiserver-extra-sans", san)
}
if cfg.KubeSolo.LocalStorageSharedPath != "" {
parts = append(parts, "--local-storage-shared-path", cfg.KubeSolo.LocalStorageSharedPath)
}
if cfg.KubeSolo.Debug {
parts = append(parts, "--debug")
}
if cfg.KubeSolo.PprofServer {
parts = append(parts, "--pprof-server")
}
if cfg.KubeSolo.PortainerEdgeID != "" {
parts = append(parts, "--portainer-edge-id", cfg.KubeSolo.PortainerEdgeID)
}
if cfg.KubeSolo.PortainerEdgeKey != "" {
parts = append(parts, "--portainer-edge-key", cfg.KubeSolo.PortainerEdgeKey)
}
if cfg.KubeSolo.PortainerEdgeAsync {
parts = append(parts, "--portainer-edge-async")
}
return strings.Join(parts, " ")
}

View File

@@ -44,6 +44,54 @@ func TestBuildExtraFlags(t *testing.T) {
},
want: "--disable servicelb --apiserver-extra-sans edge.local",
},
{
name: "debug flag",
cfg: Config{
KubeSolo: KubeSoloConfig{Debug: true},
},
want: "--debug",
},
{
name: "pprof-server flag",
cfg: Config{
KubeSolo: KubeSoloConfig{PprofServer: true},
},
want: "--pprof-server",
},
{
name: "local-storage-shared-path",
cfg: Config{
KubeSolo: KubeSoloConfig{LocalStorageSharedPath: "/mnt/shared"},
},
want: "--local-storage-shared-path /mnt/shared",
},
{
name: "portainer edge flags",
cfg: Config{
KubeSolo: KubeSoloConfig{
PortainerEdgeID: "test-id-123",
PortainerEdgeKey: "test-key-456",
PortainerEdgeAsync: true,
},
},
want: "--portainer-edge-id test-id-123 --portainer-edge-key test-key-456 --portainer-edge-async",
},
{
name: "all new flags",
cfg: Config{
KubeSolo: KubeSoloConfig{
ExtraFlags: "--disable traefik",
ExtraSANs: []string{"node.local"},
LocalStorageSharedPath: "/mnt/data/shared",
Debug: true,
PprofServer: true,
PortainerEdgeID: "eid",
PortainerEdgeKey: "ekey",
PortainerEdgeAsync: true,
},
},
want: "--disable traefik --apiserver-extra-sans node.local --local-storage-shared-path /mnt/data/shared --debug --pprof-server --portainer-edge-id eid --portainer-edge-key ekey --portainer-edge-async",
},
}
for _, tt := range tests {
@@ -61,9 +109,14 @@ func TestApplyKubeSolo(t *testing.T) {
tr := true
cfg := &Config{
KubeSolo: KubeSoloConfig{
ExtraFlags: "--disable traefik",
LocalStorage: &tr,
ExtraSANs: []string{"test.local"},
ExtraFlags: "--disable traefik",
LocalStorage: &tr,
ExtraSANs: []string{"test.local"},
LocalStorageSharedPath: "/mnt/shared",
Debug: true,
PortainerEdgeID: "eid",
PortainerEdgeKey: "ekey",
PortainerEdgeAsync: true,
},
}
@@ -83,6 +136,21 @@ func TestApplyKubeSolo(t *testing.T) {
if !strings.Contains(flags, "--apiserver-extra-sans test.local") {
t.Errorf("extra-flags missing SANs: %q", flags)
}
if !strings.Contains(flags, "--local-storage-shared-path /mnt/shared") {
t.Errorf("extra-flags missing local-storage-shared-path: %q", flags)
}
if !strings.Contains(flags, "--debug") {
t.Errorf("extra-flags missing --debug: %q", flags)
}
if !strings.Contains(flags, "--portainer-edge-id eid") {
t.Errorf("extra-flags missing --portainer-edge-id: %q", flags)
}
if !strings.Contains(flags, "--portainer-edge-key ekey") {
t.Errorf("extra-flags missing --portainer-edge-key: %q", flags)
}
if !strings.Contains(flags, "--portainer-edge-async") {
t.Errorf("extra-flags missing --portainer-edge-async: %q", flags)
}
// Check config.yaml
configData, err := os.ReadFile(filepath.Join(dir, "config.yaml"))

View File

@@ -225,6 +225,7 @@ func TestParseExampleFiles(t *testing.T) {
"examples/static-ip.yaml",
"examples/portainer-edge.yaml",
"examples/airgapped.yaml",
"examples/full-config.yaml",
}
for _, path := range examples {

View File

@@ -45,9 +45,15 @@ network:
kubesolo:
extra-flags: "--disable traefik" # Extra CLI flags for KubeSolo binary
local-storage: true # Enable local-path provisioner (default: true)
local-storage-shared-path: "/mnt/shared" # Shared path for local-path-provisioner
apiserver-extra-sans: # Extra SANs for API server certificate
- node.example.com
- 10.0.0.50
debug: false # Enable verbose debug logging
pprof-server: false # Enable Go pprof profiling server
portainer-edge-id: "" # Portainer Edge Agent ID
portainer-edge-key: "" # Portainer Edge Agent key
portainer-edge-async: false # Enable async Portainer Edge communication
# NTP servers (optional)
ntp:
@@ -129,6 +135,24 @@ kubesolo-cloudinit validate /path/to/cloud-init.yaml
kubesolo-cloudinit dump /path/to/cloud-init.yaml
```
## KubeSolo Configuration Reference
All fields under the `kubesolo:` section and their corresponding CLI flags:
| YAML Field | CLI Flag | Type | Default | Description |
|---|---|---|---|---|
| `extra-flags` | (raw flags) | string | `""` | Arbitrary extra flags passed to KubeSolo binary |
| `local-storage` | `--local-storage` | bool | `true` | Enable local-path-provisioner for PVCs |
| `local-storage-shared-path` | `--local-storage-shared-path` | string | `""` | Shared path for local-path-provisioner storage |
| `apiserver-extra-sans` | `--apiserver-extra-sans` | list | `[]` | Extra SANs for API server TLS certificate |
| `debug` | `--debug` | bool | `false` | Enable verbose debug logging |
| `pprof-server` | `--pprof-server` | bool | `false` | Enable Go pprof profiling server |
| `portainer-edge-id` | `--portainer-edge-id` | string | `""` | Portainer Edge Agent ID (from Portainer UI) |
| `portainer-edge-key` | `--portainer-edge-key` | string | `""` | Portainer Edge Agent key (from Portainer UI) |
| `portainer-edge-async` | `--portainer-edge-async` | bool | `false` | Enable async Portainer Edge communication |
**Note:** The `portainer-edge-*` fields generate CLI flags for KubeSolo's built-in Edge Agent support. This is an alternative to the `portainer.edge-agent` section, which creates a standalone Kubernetes manifest. Use one approach or the other, not both.
## Examples
See `cloud-init/examples/` for complete configuration examples:
@@ -137,6 +161,7 @@ See `cloud-init/examples/` for complete configuration examples:
- `static-ip.yaml` — Static IP configuration
- `portainer-edge.yaml` — Portainer Edge Agent integration
- `airgapped.yaml` — Air-gapped deployment with pre-loaded images
- `full-config.yaml` — All supported KubeSolo parameters
## Building

100
hack/dev-vm-arm64.sh Executable file
View File

@@ -0,0 +1,100 @@
#!/bin/bash
# dev-vm-arm64.sh — Launch ARM64 QEMU VM for development
#
# Uses qemu-system-aarch64 with -machine virt to emulate an ARM64 system.
# This is useful for testing ARM64/RPi builds on x86_64 hosts.
#
# Usage:
# ./hack/dev-vm-arm64.sh # Use default kernel + initramfs
# ./hack/dev-vm-arm64.sh <kernel> <initramfs> # Specify custom paths
# ./hack/dev-vm-arm64.sh --debug # Enable debug logging
# ./hack/dev-vm-arm64.sh --shell # Drop to emergency shell
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
VMLINUZ=""
INITRD=""
EXTRA_APPEND=""
# Parse arguments
for arg in "$@"; do
case "$arg" in
--shell) EXTRA_APPEND="$EXTRA_APPEND kubesolo.shell" ;;
--debug) EXTRA_APPEND="$EXTRA_APPEND kubesolo.debug" ;;
*)
if [ -z "$VMLINUZ" ]; then
VMLINUZ="$arg"
elif [ -z "$INITRD" ]; then
INITRD="$arg"
fi
;;
esac
done
# Defaults
VMLINUZ="${VMLINUZ:-$PROJECT_ROOT/build/cache/custom-kernel-arm64/Image}"
INITRD="${INITRD:-$PROJECT_ROOT/build/rootfs-work/kubesolo-os.gz}"
# Verify files exist
if [ ! -f "$VMLINUZ" ]; then
echo "ERROR: Kernel not found: $VMLINUZ"
echo " Run 'make kernel-arm64' to build the ARM64 kernel."
exit 1
fi
if [ ! -f "$INITRD" ]; then
echo "ERROR: Initrd not found: $INITRD"
echo " Run 'make initramfs' to build the initramfs."
exit 1
fi
# Find mkfs.ext4
MKFS_EXT4=""
if command -v mkfs.ext4 >/dev/null 2>&1; then
MKFS_EXT4="mkfs.ext4"
elif [ -x "/opt/homebrew/opt/e2fsprogs/sbin/mkfs.ext4" ]; then
MKFS_EXT4="/opt/homebrew/opt/e2fsprogs/sbin/mkfs.ext4"
elif [ -x "/usr/local/opt/e2fsprogs/sbin/mkfs.ext4" ]; then
MKFS_EXT4="/usr/local/opt/e2fsprogs/sbin/mkfs.ext4"
fi
if [ -z "$MKFS_EXT4" ]; then
echo "ERROR: mkfs.ext4 not found. Install e2fsprogs:"
if [ "$(uname)" = "Darwin" ]; then
echo " brew install e2fsprogs"
else
echo " apt install e2fsprogs # Debian/Ubuntu"
echo " dnf install e2fsprogs # Fedora/RHEL"
fi
exit 1
fi
# Create data disk
DATA_DISK="$(mktemp /tmp/kubesolo-arm64-data-XXXXXX).img"
dd if=/dev/zero of="$DATA_DISK" bs=1M count=1024 2>/dev/null
"$MKFS_EXT4" -q -L KSOLODATA "$DATA_DISK" 2>/dev/null
trap 'rm -f "$DATA_DISK"' EXIT
echo "==> Launching ARM64 QEMU VM..."
echo " Kernel: $VMLINUZ"
echo " Initrd: $INITRD"
echo " Data: $DATA_DISK"
echo ""
echo " K8s API: localhost:6443"
echo " SSH: localhost:2222"
echo " Press Ctrl+A X to exit QEMU"
echo ""
qemu-system-aarch64 \
-machine virt \
-cpu cortex-a72 \
-m 2048 \
-smp 2 \
-nographic \
-kernel "$VMLINUZ" \
-initrd "$INITRD" \
-append "console=ttyAMA0 kubesolo.data=/dev/vda kubesolo.debug $EXTRA_APPEND" \
-drive "file=$DATA_DISK,format=raw,if=virtio" \
-net "nic,model=virtio" \
-net "user,hostfwd=tcp::6443-:6443,hostfwd=tcp::2222-:22"

View File

@@ -85,8 +85,8 @@ trap cleanup EXIT
# Build QEMU command
QEMU_ARGS=(-m 2048 -smp 2 -nographic -cpu max)
QEMU_ARGS+=(-net nic,model=virtio)
QEMU_ARGS+=(-net user,hostfwd=tcp::6443-:6443,hostfwd=tcp::2222-:22,hostfwd=tcp::8080-:8080)
QEMU_ARGS+=(-net "nic,model=virtio")
QEMU_ARGS+=(-net "user,hostfwd=tcp::6443-:6443,hostfwd=tcp::2222-:22,hostfwd=tcp::8080-:8080")
if [ -n "$DATA_DISK" ]; then
QEMU_ARGS+=(-drive "file=$DATA_DISK,format=raw,if=virtio")

View File

@@ -64,6 +64,7 @@ export KUBESOLO_CLOUDINIT=""
export KUBESOLO_EXTRA_FLAGS=""
export KUBESOLO_PORTAINER_EDGE_ID=""
export KUBESOLO_PORTAINER_EDGE_KEY=""
export KUBESOLO_NOAPPARMOR=""
# --- Logging ---
log() {

View File

@@ -12,10 +12,10 @@ if ! mountpoint -q /dev 2>/dev/null; then
mount -t devtmpfs devtmpfs /dev 2>/dev/null || mount -t tmpfs tmpfs /dev
fi
if ! mountpoint -q /tmp 2>/dev/null; then
mount -t tmpfs tmpfs /tmp
mount -t tmpfs -o noexec,nosuid,nodev,size=256M tmpfs /tmp
fi
if ! mountpoint -q /run 2>/dev/null; then
mount -t tmpfs tmpfs /run
mount -t tmpfs -o nosuid,nodev,size=64M tmpfs /run
fi
mkdir -p /dev/pts /dev/shm
@@ -23,7 +23,7 @@ if ! mountpoint -q /dev/pts 2>/dev/null; then
mount -t devpts devpts /dev/pts
fi
if ! mountpoint -q /dev/shm 2>/dev/null; then
mount -t tmpfs tmpfs /dev/shm
mount -t tmpfs -o noexec,nosuid,nodev,size=64M tmpfs /dev/shm
fi
# Ensure essential device nodes exist (devtmpfs may be incomplete after switch_root)

View File

@@ -11,9 +11,14 @@ for arg in $(cat /proc/cmdline); do
kubesolo.flags=*) KUBESOLO_EXTRA_FLAGS="${arg#kubesolo.flags=}" ;;
kubesolo.edge_id=*) KUBESOLO_PORTAINER_EDGE_ID="${arg#kubesolo.edge_id=}" ;;
kubesolo.edge_key=*) KUBESOLO_PORTAINER_EDGE_KEY="${arg#kubesolo.edge_key=}" ;;
kubesolo.nomodlock) KUBESOLO_NOMODLOCK=1 ;;
kubesolo.noapparmor) KUBESOLO_NOAPPARMOR=1 ;;
esac
done
export KUBESOLO_NOMODLOCK
export KUBESOLO_NOAPPARMOR
if [ -z "$KUBESOLO_DATA_DEV" ] && [ "$KUBESOLO_NOPERSIST" != "1" ]; then
log_warn "No kubesolo.data= specified and kubesolo.nopersist not set"
log_warn "Attempting auto-detection of data partition (label: KSOLODATA)"

View File

@@ -11,40 +11,70 @@ fi
# Load block device drivers before waiting (modules loaded later in stage 30,
# but we need virtio_blk available NOW for /dev/vda detection)
modprobe virtio_blk 2>/dev/null || true
modprobe mmc_block 2>/dev/null || true
# Trigger mdev to create device nodes after loading driver
mdev -s 2>/dev/null || true
# Fallback: create device node from sysfs if devtmpfs/mdev didn't
DEV_NAME="${KUBESOLO_DATA_DEV##*/}"
if [ ! -b "$KUBESOLO_DATA_DEV" ] && [ -f "/sys/class/block/$DEV_NAME/dev" ]; then
MAJMIN=$(cat "/sys/class/block/$DEV_NAME/dev")
mknod "$KUBESOLO_DATA_DEV" b "${MAJMIN%%:*}" "${MAJMIN##*:}" 2>/dev/null || true
log "Created $KUBESOLO_DATA_DEV via mknod ($MAJMIN)"
fi
# Wait for device to appear (USB, slow disks, virtio)
log "Waiting for data device: $KUBESOLO_DATA_DEV"
# Resolve LABEL= syntax to actual block device path
# The RPi cmdline uses kubesolo.data=LABEL=KSOLODATA which needs resolution
WAIT_SECS=30
for i in $(seq 1 "$WAIT_SECS"); do
[ -b "$KUBESOLO_DATA_DEV" ] && break
mdev -s 2>/dev/null || true
sleep 1
done
log "Waiting for data device: $KUBESOLO_DATA_DEV"
case "$KUBESOLO_DATA_DEV" in
LABEL=*)
# Extract label name and resolve via blkid/findfs
DATA_LABEL="${KUBESOLO_DATA_DEV#LABEL=}"
RESOLVED=""
for i in $(seq 1 "$WAIT_SECS"); do
mdev -s 2>/dev/null || true
RESOLVED=$(blkid -L "$DATA_LABEL" 2>/dev/null) || true
if [ -z "$RESOLVED" ]; then
RESOLVED=$(findfs "LABEL=$DATA_LABEL" 2>/dev/null) || true
fi
if [ -n "$RESOLVED" ] && [ -b "$RESOLVED" ]; then
log "Resolved LABEL=$DATA_LABEL -> $RESOLVED"
KUBESOLO_DATA_DEV="$RESOLVED"
break
fi
sleep 1
done
;;
*)
# Direct block device path — wait for it to appear
# Fallback: create device node from sysfs if devtmpfs/mdev didn't
DEV_NAME="${KUBESOLO_DATA_DEV##*/}"
if [ ! -b "$KUBESOLO_DATA_DEV" ] && [ -f "/sys/class/block/$DEV_NAME/dev" ]; then
MAJMIN=$(cat "/sys/class/block/$DEV_NAME/dev")
mknod "$KUBESOLO_DATA_DEV" b "${MAJMIN%%:*}" "${MAJMIN##*:}" 2>/dev/null || true
log "Created $KUBESOLO_DATA_DEV via mknod ($MAJMIN)"
fi
for i in $(seq 1 "$WAIT_SECS"); do
[ -b "$KUBESOLO_DATA_DEV" ] && break
mdev -s 2>/dev/null || true
sleep 1
done
;;
esac
if [ ! -b "$KUBESOLO_DATA_DEV" ]; then
log_err "Data device $KUBESOLO_DATA_DEV not found after ${WAIT_SECS}s"
# Show available block devices for debugging
log_err "Available block devices:"
ls -la /dev/mmc* /dev/sd* /dev/vd* 2>/dev/null | while read -r line; do
log_err " $line"
done
return 1
fi
# Mount data partition (format on first boot if unformatted)
mkdir -p "$DATA_MOUNT"
if ! mount -t ext4 -o noatime "$KUBESOLO_DATA_DEV" "$DATA_MOUNT" 2>/dev/null; then
if ! mount -t ext4 -o noatime,nosuid,nodev "$KUBESOLO_DATA_DEV" "$DATA_MOUNT" 2>/dev/null; then
log "Formatting $KUBESOLO_DATA_DEV as ext4 (first boot)"
mkfs.ext4 -q -L KSOLODATA "$KUBESOLO_DATA_DEV" || {
log_err "Failed to format $KUBESOLO_DATA_DEV"
return 1
}
mount -t ext4 -o noatime "$KUBESOLO_DATA_DEV" "$DATA_MOUNT" || {
mount -t ext4 -o noatime,nosuid,nodev "$KUBESOLO_DATA_DEV" "$DATA_MOUNT" || {
log_err "Failed to mount $KUBESOLO_DATA_DEV after format"
return 1
}

47
init/lib/35-apparmor.sh Normal file
View File

@@ -0,0 +1,47 @@
#!/bin/sh
# 35-apparmor.sh — Load AppArmor LSM profiles
# Check for opt-out boot parameter
if [ "$KUBESOLO_NOAPPARMOR" = "1" ]; then
log "AppArmor disabled via kubesolo.noapparmor boot parameter"
return 0
fi
# Mount securityfs if not already mounted
if ! mountpoint -q /sys/kernel/security 2>/dev/null; then
mount -t securityfs securityfs /sys/kernel/security 2>/dev/null || true
fi
# Check if AppArmor is available in the kernel
if [ ! -d /sys/kernel/security/apparmor ]; then
log_warn "AppArmor not available in kernel — skipping profile loading"
return 0
fi
# Check for apparmor_parser
if ! command -v apparmor_parser >/dev/null 2>&1; then
log_warn "apparmor_parser not found — skipping profile loading"
return 0
fi
# Load all profiles from /etc/apparmor.d/
PROFILE_DIR="/etc/apparmor.d"
if [ ! -d "$PROFILE_DIR" ]; then
log_warn "No AppArmor profiles directory ($PROFILE_DIR) — skipping"
return 0
fi
LOADED=0
FAILED=0
for profile in "$PROFILE_DIR"/*; do
[ -f "$profile" ] || continue
if apparmor_parser -r "$profile" 2>/dev/null; then
LOADED=$((LOADED + 1))
else
log_warn "Failed to load AppArmor profile: $(basename "$profile")"
FAILED=$((FAILED + 1))
fi
done
log_ok "AppArmor: loaded $LOADED profiles ($FAILED failed)"

View File

@@ -0,0 +1,20 @@
#!/bin/sh
# 85-security-lockdown.sh — Lock down kernel after all modules loaded
# Allow disabling via boot parameter for debugging
if [ "$KUBESOLO_NOMODLOCK" = "1" ]; then
log_warn "Module lock DISABLED (kubesolo.nomodlock)"
else
# Permanently prevent new kernel module loading (irreversible until reboot)
# All required modules must already be loaded by stage 30
if [ -f /proc/sys/kernel/modules_disabled ]; then
echo 1 > /proc/sys/kernel/modules_disabled 2>/dev/null && \
log_ok "Kernel module loading locked" || \
log_warn "Failed to lock kernel module loading"
fi
fi
# Safety net: enforce kernel information protection
# (also set via sysctl.d but enforce here in case sysctl.d was bypassed)
echo 2 > /proc/sys/kernel/kptr_restrict 2>/dev/null || true
echo 1 > /proc/sys/kernel/dmesg_restrict 2>/dev/null || true

View File

@@ -85,12 +85,16 @@ if [ -f "$KUBECONFIG_PATH" ]; then
EXTERNAL_KC="/tmp/kubeconfig-external.yaml"
sed 's|server: https://.*:6443|server: https://localhost:6443|' "$KUBECONFIG_PATH" > "$EXTERNAL_KC"
# Serve kubeconfig via HTTP on port 8080 using BusyBox nc
# Serve kubeconfig via HTTP on port 8080 for remote kubectl access.
# Binds to 0.0.0.0 so it's reachable via QEMU port forwarding.
# Security: the kubeconfig is only useful if you can also reach
# port 6443 (API server). On edge devices, network isolation
# provides the security boundary.
(while true; do
printf "HTTP/1.1 200 OK\r\nContent-Type: text/yaml\r\nConnection: close\r\n\r\n" | cat - "$EXTERNAL_KC" | nc -l -p 8080 2>/dev/null
printf 'HTTP/1.1 200 OK\r\nContent-Type: text/yaml\r\nConnection: close\r\n\r\n' | cat - "$EXTERNAL_KC" | nc -l -p 8080 2>/dev/null
done) &
log_ok "Kubeconfig available via HTTP"
log_ok "Kubeconfig available via HTTP on port 8080"
echo ""
echo "============================================================"
echo " From your host machine, run:"

View File

@@ -22,6 +22,8 @@ RUNS=3
SSH_PORT=2222
K8S_PORT=6443
. "$SCRIPT_DIR/../lib/qemu-helpers.sh"
shift || true
while [ $# -gt 0 ]; do
case "$1" in
@@ -47,6 +49,15 @@ echo "Type: $IMAGE_TYPE" >&2
echo "Runs: $RUNS" >&2
echo "" >&2
EXTRACT_DIR=""
TEMP_DISK=""
cleanup() {
[ -n "$TEMP_DISK" ] && rm -f "$TEMP_DISK"
[ -n "$EXTRACT_DIR" ] && rm -rf "$EXTRACT_DIR"
}
trap cleanup EXIT
# Build QEMU command
QEMU_CMD=(
qemu-system-x86_64
@@ -55,24 +66,31 @@ QEMU_CMD=(
-nographic
-no-reboot
-serial mon:stdio
-net nic,model=virtio
-net "nic,model=virtio"
-net "user,hostfwd=tcp::${SSH_PORT}-:22,hostfwd=tcp::${K8S_PORT}-:6443"
)
# Add KVM if available
if [ -e /dev/kvm ] && [ -r /dev/kvm ]; then
KVM_FLAG=$(detect_kvm)
if [ -n "$KVM_FLAG" ]; then
QEMU_CMD+=(-enable-kvm -cpu host)
echo "KVM: enabled" >&2
else
QEMU_CMD+=(-cpu max)
echo "KVM: not available (TCG)" >&2
fi
echo "" >&2
if [ "$IMAGE_TYPE" = "iso" ]; then
QEMU_CMD+=(-cdrom "$IMAGE")
# Extract kernel/initramfs for direct boot (required for -append to work)
EXTRACT_DIR="$(mktemp -d /tmp/kubesolo-bench-extract-XXXXXX)"
extract_kernel_from_iso "$IMAGE" "$EXTRACT_DIR" >&2
QEMU_CMD+=(-kernel "$VMLINUZ" -initrd "$INITRAMFS")
QEMU_CMD+=(-append "console=ttyS0,115200n8 kubesolo.debug")
# Add a temp disk for persistence
TEMP_DISK=$(mktemp /tmp/kubesolo-bench-XXXXXX.img)
qemu-img create -f qcow2 "$TEMP_DISK" 8G >/dev/null 2>&1
QEMU_CMD+=(-drive "file=$TEMP_DISK,format=qcow2,if=virtio")
trap "rm -f $TEMP_DISK" EXIT
else
QEMU_CMD+=(-drive "file=$IMAGE,format=raw,if=virtio")
fi
@@ -111,7 +129,7 @@ for run in $(seq 1 "$RUNS"); do
echo "KERNEL_MS=$ELAPSED_MS" >> "$LOG.times"
fi
;;
*"kubesolo-init"*"all stages complete"*|*"init complete"*)
*"KubeSolo is running"*|*"kubesolo-init"*"OK"*)
if [ -z "$INIT_DONE" ]; then
INIT_DONE="$ELAPSED_MS"
echo " Init complete: ${ELAPSED_MS}ms" >&2

View File

@@ -5,42 +5,67 @@
set -euo pipefail
ISO="${1:?Usage: $0 <path-to-iso>}"
TIMEOUT_BOOT=120
TIMEOUT_K8S=300
TIMEOUT_POD=120
TIMEOUT_K8S=${TIMEOUT_K8S:-300}
TIMEOUT_POD=${TIMEOUT_POD:-120}
API_PORT=6443
KC_PORT=8080
SERIAL_LOG=$(mktemp /tmp/kubesolo-workload-XXXXXX.log)
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
. "$SCRIPT_DIR/../lib/qemu-helpers.sh"
DATA_DISK=$(mktemp /tmp/kubesolo-data-XXXXXX.img)
dd if=/dev/zero of="$DATA_DISK" bs=1M count=1024 2>/dev/null
dd if=/dev/zero of="$DATA_DISK" bs=1M count=2048 2>/dev/null
mkfs.ext4 -q -L KSOLODATA "$DATA_DISK" 2>/dev/null
QEMU_PID=""
EXTRACT_DIR=""
KUBECONFIG_FILE=""
cleanup() {
kill "$QEMU_PID" 2>/dev/null || true
[ -n "$QEMU_PID" ] && kill "$QEMU_PID" 2>/dev/null || true
rm -f "$DATA_DISK" "$SERIAL_LOG"
[ -n "$KUBECONFIG_FILE" ] && rm -f "$KUBECONFIG_FILE"
[ -n "$EXTRACT_DIR" ] && rm -rf "$EXTRACT_DIR"
}
trap cleanup EXIT
KUBECTL="kubectl --server=https://localhost:${API_PORT} --insecure-skip-tls-verify"
echo "==> Workload deployment test: $ISO"
# Extract kernel from ISO
EXTRACT_DIR="$(mktemp -d /tmp/kubesolo-extract-XXXXXX)"
extract_kernel_from_iso "$ISO" "$EXTRACT_DIR"
KVM_FLAG=$(detect_kvm)
# Launch QEMU
# shellcheck disable=SC2086
qemu-system-x86_64 \
-m 2048 -smp 2 \
-nographic \
-cdrom "$ISO" \
-boot d \
$KVM_FLAG \
-kernel "$VMLINUZ" \
-initrd "$INITRAMFS" \
-drive "file=$DATA_DISK,format=raw,if=virtio" \
-net nic,model=virtio \
-net "user,hostfwd=tcp::${API_PORT}-:6443" \
-net "nic,model=virtio" \
-net "user,hostfwd=tcp::${API_PORT}-:6443,hostfwd=tcp::${KC_PORT}-:8080" \
-serial "file:$SERIAL_LOG" \
-append "console=ttyS0,115200n8 kubesolo.data=/dev/vda" \
-append "console=ttyS0,115200n8 kubesolo.data=/dev/vda kubesolo.debug" \
&
QEMU_PID=$!
# Wait for boot + fetch kubeconfig
echo " Waiting for boot..."
wait_for_boot "$SERIAL_LOG" "$QEMU_PID" 180 || exit 1
KUBECONFIG_FILE=$(mktemp /tmp/kubesolo-kubeconfig-XXXXXX.yaml)
fetch_kubeconfig "$KC_PORT" "$KUBECONFIG_FILE" || exit 1
KUBECTL="kubectl --kubeconfig=$KUBECONFIG_FILE --insecure-skip-tls-verify"
# Wait for K8s API
echo " Waiting for K8s API..."
echo " Waiting for K8s node Ready..."
ELAPSED=0
K8S_READY=0
while [ "$ELAPSED" -lt "$TIMEOUT_K8S" ]; do
@@ -71,6 +96,7 @@ $KUBECTL run test-nginx --image=nginx:alpine --restart=Never 2>/dev/null || {
echo " Waiting for pod to reach Running..."
ELAPSED=0
POD_RUNNING=0
STATUS=""
while [ "$ELAPSED" -lt "$TIMEOUT_POD" ]; do
STATUS=$($KUBECTL get pod test-nginx -o jsonpath='{.status.phase}' 2>/dev/null || echo "")
if [ "$STATUS" = "Running" ]; then

View File

@@ -5,58 +5,73 @@
set -euo pipefail
ISO="${1:?Usage: $0 <path-to-iso>}"
TIMEOUT_BOOT=120
TIMEOUT_K8S=300
TIMEOUT_K8S=${TIMEOUT_K8S:-300}
API_PORT=6443
KC_PORT=8080
SERIAL_LOG=$(mktemp /tmp/kubesolo-k8s-test-XXXXXX.log)
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
. "$SCRIPT_DIR/../lib/qemu-helpers.sh"
DATA_DISK=$(mktemp /tmp/kubesolo-data-XXXXXX.img)
dd if=/dev/zero of="$DATA_DISK" bs=1M count=1024 2>/dev/null
dd if=/dev/zero of="$DATA_DISK" bs=1M count=2048 2>/dev/null
mkfs.ext4 -q -L KSOLODATA "$DATA_DISK" 2>/dev/null
QEMU_PID=""
EXTRACT_DIR=""
KUBECONFIG_FILE=""
cleanup() {
kill "$QEMU_PID" 2>/dev/null || true
rm -f "$DATA_DISK"
[ -n "$QEMU_PID" ] && kill "$QEMU_PID" 2>/dev/null || true
rm -f "$DATA_DISK" "$SERIAL_LOG"
[ -n "$KUBECONFIG_FILE" ] && rm -f "$KUBECONFIG_FILE"
[ -n "$EXTRACT_DIR" ] && rm -rf "$EXTRACT_DIR"
}
trap cleanup EXIT
echo "==> K8s readiness test: $ISO"
# Launch QEMU with API port forwarded
# Extract kernel from ISO
EXTRACT_DIR="$(mktemp -d /tmp/kubesolo-extract-XXXXXX)"
extract_kernel_from_iso "$ISO" "$EXTRACT_DIR"
KVM_FLAG=$(detect_kvm)
[ -n "$KVM_FLAG" ] && echo " KVM acceleration: enabled"
# Launch QEMU with API + kubeconfig ports forwarded
# shellcheck disable=SC2086
qemu-system-x86_64 \
-m 2048 -smp 2 \
-nographic \
-cdrom "$ISO" \
-boot d \
$KVM_FLAG \
-kernel "$VMLINUZ" \
-initrd "$INITRAMFS" \
-drive "file=$DATA_DISK,format=raw,if=virtio" \
-net nic,model=virtio \
-net user,hostfwd=tcp::${API_PORT}-:6443 \
-append "console=ttyS0,115200n8 kubesolo.data=/dev/vda" \
-net "nic,model=virtio" \
-net "user,hostfwd=tcp::${API_PORT}-:6443,hostfwd=tcp::${KC_PORT}-:8080" \
-serial "file:$SERIAL_LOG" \
-append "console=ttyS0,115200n8 kubesolo.data=/dev/vda kubesolo.debug" \
&
QEMU_PID=$!
# Wait for API server
echo " Waiting for K8s API on localhost:${API_PORT}..."
# Wait for boot
echo " Waiting for boot..."
wait_for_boot "$SERIAL_LOG" "$QEMU_PID" 180 || exit 1
# Fetch kubeconfig
KUBECONFIG_FILE=$(mktemp /tmp/kubesolo-kubeconfig-XXXXXX.yaml)
fetch_kubeconfig "$KC_PORT" "$KUBECONFIG_FILE" || exit 1
# Wait for K8s node to reach Ready
echo " Waiting for K8s node Ready..."
ELAPSED=0
while [ "$ELAPSED" -lt "$TIMEOUT_K8S" ]; do
if kubectl --kubeconfig=/dev/null \
--server="https://localhost:${API_PORT}" \
if kubectl --kubeconfig="$KUBECONFIG_FILE" \
--insecure-skip-tls-verify \
get nodes 2>/dev/null | grep -q "Ready"; then
echo ""
echo "==> PASS: K8s node is Ready (${ELAPSED}s)"
# Bonus: try deploying a pod
echo " Deploying test pod..."
kubectl --server="https://localhost:${API_PORT}" --insecure-skip-tls-verify \
run test-nginx --image=nginx:alpine --restart=Never 2>/dev/null || true
sleep 10
if kubectl --server="https://localhost:${API_PORT}" --insecure-skip-tls-verify \
get pod test-nginx 2>/dev/null | grep -q "Running"; then
echo "==> PASS: Test pod is Running"
else
echo "==> WARN: Test pod not Running (may need more time or image pull)"
fi
echo "==> PASS: K8s node is Ready (${ELAPSED}s after boot)"
exit 0
fi
sleep 5
@@ -66,4 +81,6 @@ done
echo ""
echo "==> FAIL: K8s node did not reach Ready within ${TIMEOUT_K8S}s"
echo " Last 40 lines of serial log:"
tail -40 "$SERIAL_LOG" 2>/dev/null
exit 1

View File

@@ -5,9 +5,14 @@
set -euo pipefail
ISO="${1:?Usage: $0 <path-to-iso>}"
TIMEOUT_K8S=300
TIMEOUT_PVC=120
TIMEOUT_K8S=${TIMEOUT_K8S:-300}
TIMEOUT_PVC=${TIMEOUT_PVC:-180}
API_PORT=6443
KC_PORT=8080
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
. "$SCRIPT_DIR/../lib/qemu-helpers.sh"
DATA_DISK=$(mktemp /tmp/kubesolo-data-XXXXXX.img)
dd if=/dev/zero of="$DATA_DISK" bs=1M count=2048 2>/dev/null
@@ -15,35 +20,60 @@ mkfs.ext4 -q -L KSOLODATA "$DATA_DISK" 2>/dev/null
SERIAL_LOG=$(mktemp /tmp/kubesolo-storage-XXXXXX.log)
QEMU_PID=""
EXTRACT_DIR=""
KUBECONFIG_FILE=""
cleanup() {
# Clean up K8s resources
$KUBECTL delete pod test-storage --grace-period=0 --force 2>/dev/null || true
$KUBECTL delete pvc test-pvc 2>/dev/null || true
kill "$QEMU_PID" 2>/dev/null || true
[ -n "$KUBECONFIG_FILE" ] && [ -f "$KUBECONFIG_FILE" ] && {
kubectl --kubeconfig="$KUBECONFIG_FILE" --insecure-skip-tls-verify \
delete pod test-storage --grace-period=0 --force 2>/dev/null || true
kubectl --kubeconfig="$KUBECONFIG_FILE" --insecure-skip-tls-verify \
delete pvc test-pvc 2>/dev/null || true
}
[ -n "$QEMU_PID" ] && kill "$QEMU_PID" 2>/dev/null || true
rm -f "$DATA_DISK" "$SERIAL_LOG"
[ -n "$KUBECONFIG_FILE" ] && rm -f "$KUBECONFIG_FILE"
[ -n "$EXTRACT_DIR" ] && rm -rf "$EXTRACT_DIR"
}
trap cleanup EXIT
KUBECTL="kubectl --server=https://localhost:${API_PORT} --insecure-skip-tls-verify"
echo "==> Local storage test: $ISO"
# Extract kernel from ISO
EXTRACT_DIR="$(mktemp -d /tmp/kubesolo-extract-XXXXXX)"
extract_kernel_from_iso "$ISO" "$EXTRACT_DIR"
KVM_FLAG=$(detect_kvm)
# Launch QEMU
# shellcheck disable=SC2086
qemu-system-x86_64 \
-m 2048 -smp 2 \
-nographic \
-cdrom "$ISO" \
-boot d \
$KVM_FLAG \
-kernel "$VMLINUZ" \
-initrd "$INITRAMFS" \
-drive "file=$DATA_DISK,format=raw,if=virtio" \
-net nic,model=virtio \
-net "user,hostfwd=tcp::${API_PORT}-:6443" \
-net "nic,model=virtio" \
-net "user,hostfwd=tcp::${API_PORT}-:6443,hostfwd=tcp::${KC_PORT}-:8080" \
-serial "file:$SERIAL_LOG" \
-append "console=ttyS0,115200n8 kubesolo.data=/dev/vda" \
-append "console=ttyS0,115200n8 kubesolo.data=/dev/vda kubesolo.debug" \
&
QEMU_PID=$!
# Wait for boot + fetch kubeconfig
echo " Waiting for boot..."
wait_for_boot "$SERIAL_LOG" "$QEMU_PID" 180 || exit 1
KUBECONFIG_FILE=$(mktemp /tmp/kubesolo-kubeconfig-XXXXXX.yaml)
fetch_kubeconfig "$KC_PORT" "$KUBECONFIG_FILE" || exit 1
KUBECTL="kubectl --kubeconfig=$KUBECONFIG_FILE --insecure-skip-tls-verify"
# Wait for K8s API
echo " Waiting for K8s API..."
echo " Waiting for K8s node Ready..."
ELAPSED=0
while [ "$ELAPSED" -lt "$TIMEOUT_K8S" ]; do
if $KUBECTL get nodes 2>/dev/null | grep -q "Ready"; then
@@ -98,6 +128,7 @@ YAML
# Wait for pod Running
echo " Waiting for storage pod..."
ELAPSED=0
STATUS=""
while [ "$ELAPSED" -lt "$TIMEOUT_PVC" ]; do
STATUS=$($KUBECTL get pod test-storage -o jsonpath='{.status.phase}' 2>/dev/null || echo "")
if [ "$STATUS" = "Running" ]; then

View File

@@ -6,43 +6,72 @@
set -euo pipefail
ISO="${1:?Usage: $0 <path-to-iso>}"
TIMEOUT_K8S=300
TIMEOUT_POD=120
TIMEOUT_K8S=${TIMEOUT_K8S:-300}
TIMEOUT_POD=${TIMEOUT_POD:-120}
API_PORT=6443
KC_PORT=8080
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
. "$SCRIPT_DIR/../lib/qemu-helpers.sh"
DATA_DISK=$(mktemp /tmp/kubesolo-data-XXXXXX.img)
dd if=/dev/zero of="$DATA_DISK" bs=1M count=1024 2>/dev/null
dd if=/dev/zero of="$DATA_DISK" bs=1M count=2048 2>/dev/null
mkfs.ext4 -q -L KSOLODATA "$DATA_DISK" 2>/dev/null
SERIAL_LOG=$(mktemp /tmp/kubesolo-netpol-XXXXXX.log)
QEMU_PID=""
EXTRACT_DIR=""
KUBECONFIG_FILE=""
cleanup() {
$KUBECTL delete namespace netpol-test 2>/dev/null || true
kill "$QEMU_PID" 2>/dev/null || true
[ -n "$KUBECONFIG_FILE" ] && [ -f "$KUBECONFIG_FILE" ] && {
kubectl --kubeconfig="$KUBECONFIG_FILE" --insecure-skip-tls-verify \
delete namespace netpol-test 2>/dev/null || true
}
[ -n "$QEMU_PID" ] && kill "$QEMU_PID" 2>/dev/null || true
rm -f "$DATA_DISK" "$SERIAL_LOG"
[ -n "$KUBECONFIG_FILE" ] && rm -f "$KUBECONFIG_FILE"
[ -n "$EXTRACT_DIR" ] && rm -rf "$EXTRACT_DIR"
}
trap cleanup EXIT
KUBECTL="kubectl --server=https://localhost:${API_PORT} --insecure-skip-tls-verify"
echo "==> Network policy test: $ISO"
# Extract kernel from ISO
EXTRACT_DIR="$(mktemp -d /tmp/kubesolo-extract-XXXXXX)"
extract_kernel_from_iso "$ISO" "$EXTRACT_DIR"
KVM_FLAG=$(detect_kvm)
# Launch QEMU
# shellcheck disable=SC2086
qemu-system-x86_64 \
-m 2048 -smp 2 \
-nographic \
-cdrom "$ISO" \
-boot d \
$KVM_FLAG \
-kernel "$VMLINUZ" \
-initrd "$INITRAMFS" \
-drive "file=$DATA_DISK,format=raw,if=virtio" \
-net nic,model=virtio \
-net "user,hostfwd=tcp::${API_PORT}-:6443" \
-net "nic,model=virtio" \
-net "user,hostfwd=tcp::${API_PORT}-:6443,hostfwd=tcp::${KC_PORT}-:8080" \
-serial "file:$SERIAL_LOG" \
-append "console=ttyS0,115200n8 kubesolo.data=/dev/vda" \
-append "console=ttyS0,115200n8 kubesolo.data=/dev/vda kubesolo.debug" \
&
QEMU_PID=$!
# Wait for boot + fetch kubeconfig
echo " Waiting for boot..."
wait_for_boot "$SERIAL_LOG" "$QEMU_PID" 180 || exit 1
KUBECONFIG_FILE=$(mktemp /tmp/kubesolo-kubeconfig-XXXXXX.yaml)
fetch_kubeconfig "$KC_PORT" "$KUBECONFIG_FILE" || exit 1
KUBECTL="kubectl --kubeconfig=$KUBECONFIG_FILE --insecure-skip-tls-verify"
# Wait for K8s
echo " Waiting for K8s API..."
echo " Waiting for K8s node Ready..."
ELAPSED=0
while [ "$ELAPSED" -lt "$TIMEOUT_K8S" ]; do
if $KUBECTL get nodes 2>/dev/null | grep -q "Ready"; then
@@ -81,6 +110,7 @@ YAML
# Wait for pod
ELAPSED=0
STATUS=""
while [ "$ELAPSED" -lt "$TIMEOUT_POD" ]; do
STATUS=$($KUBECTL get pod -n netpol-test web -o jsonpath='{.status.phase}' 2>/dev/null || echo "")
[ "$STATUS" = "Running" ] && break

View File

@@ -0,0 +1,211 @@
#!/bin/bash
# test-security-hardening.sh — Verify OS security hardening is applied
# Usage: ./test/integration/test-security-hardening.sh <iso-path>
# Exit 0 = PASS, Exit 1 = FAIL
#
# Tests:
# 1. Kubeconfig server accessible via HTTP
# 2. AppArmor profiles loaded (or graceful skip if kernel lacks support)
# 3. Kernel module loading locked
# 4. Mount options (noexec on /tmp, nosuid on /run, noexec on /dev/shm)
# 5. Sysctl hardening values applied
set -euo pipefail
ISO="${1:?Usage: $0 <path-to-iso>}"
TIMEOUT_BOOT=${TIMEOUT_BOOT:-180} # seconds to wait for boot
SERIAL_LOG=$(mktemp /tmp/kubesolo-security-test-XXXXXX.log)
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
. "$SCRIPT_DIR/../lib/qemu-helpers.sh"
# Temp data disk
DATA_DISK=$(mktemp /tmp/kubesolo-security-data-XXXXXX.img)
dd if=/dev/zero of="$DATA_DISK" bs=1M count=1024 2>/dev/null
mkfs.ext4 -q -L KSOLODATA "$DATA_DISK" 2>/dev/null
QEMU_PID=""
EXTRACT_DIR=""
cleanup() {
[ -n "$QEMU_PID" ] && kill "$QEMU_PID" 2>/dev/null || true
rm -f "$DATA_DISK" "$SERIAL_LOG"
[ -n "$EXTRACT_DIR" ] && rm -rf "$EXTRACT_DIR"
}
trap cleanup EXIT
echo "==> Security Hardening Test: $ISO"
echo " Timeout: ${TIMEOUT_BOOT}s"
echo " Serial log: $SERIAL_LOG"
# Extract kernel from ISO
EXTRACT_DIR="$(mktemp -d /tmp/kubesolo-extract-XXXXXX)"
extract_kernel_from_iso "$ISO" "$EXTRACT_DIR"
# Detect KVM
KVM_FLAG=$(detect_kvm)
# Launch QEMU in background with direct kernel boot
# shellcheck disable=SC2086
qemu-system-x86_64 \
-m 2048 -smp 2 \
-nographic \
$KVM_FLAG \
-kernel "$VMLINUZ" \
-initrd "$INITRAMFS" \
-drive "file=$DATA_DISK,format=raw,if=virtio" \
-net "nic,model=virtio" \
-net "user,hostfwd=tcp::18080-:8080" \
-serial "file:$SERIAL_LOG" \
-append "console=ttyS0,115200n8 kubesolo.data=/dev/vda kubesolo.debug" \
&
QEMU_PID=$!
# Wait for boot to complete (stage 90)
echo " Waiting for boot..."
ELAPSED=0
BOOTED=0
while [ "$ELAPSED" -lt "$TIMEOUT_BOOT" ]; do
if grep -q "\[kubesolo-init\] \[OK\] KubeSolo is running" "$SERIAL_LOG" 2>/dev/null; then
BOOTED=1
break
fi
if ! kill -0 "$QEMU_PID" 2>/dev/null; then
echo ""
echo "==> FAIL: QEMU exited prematurely"
echo " Last 20 lines of serial log:"
tail -20 "$SERIAL_LOG" 2>/dev/null
exit 1
fi
sleep 2
ELAPSED=$((ELAPSED + 2))
printf "\r Elapsed: %ds / %ds" "$ELAPSED" "$TIMEOUT_BOOT"
done
echo ""
if [ "$BOOTED" = "0" ]; then
echo "==> FAIL: Boot did not complete within ${TIMEOUT_BOOT}s"
echo " Last 30 lines:"
tail -30 "$SERIAL_LOG" 2>/dev/null
exit 1
fi
echo " Boot completed in ${ELAPSED}s"
echo ""
# Give the system a moment to finish post-boot setup
sleep 5
# ============================================================
# Security checks against serial log output
# ============================================================
PASS=0
FAIL=0
SKIP=0
check_pass() { echo " PASS: $1"; PASS=$((PASS + 1)); }
check_fail() { echo " FAIL: $1"; FAIL=$((FAIL + 1)); }
check_skip() { echo " SKIP: $1"; SKIP=$((SKIP + 1)); }
echo "--- Test 1: Kubeconfig server accessible ---"
# The kubeconfig server should be reachable via QEMU port forwarding
# and return valid kubeconfig YAML content.
KC_CONTENT=$(curl -sf --connect-timeout 10 --max-time 15 "http://localhost:18080/" 2>/dev/null) || true
if [ -n "$KC_CONTENT" ] && echo "$KC_CONTENT" | grep -q "server:"; then
check_pass "Kubeconfig server returns valid kubeconfig"
elif [ -z "$KC_CONTENT" ]; then
check_fail "Kubeconfig server not reachable on port 18080"
else
check_fail "Kubeconfig server returned unexpected content"
fi
echo ""
echo "--- Test 2: AppArmor ---"
if grep -q "AppArmor.*loaded.*profiles" "$SERIAL_LOG" 2>/dev/null; then
check_pass "AppArmor profiles loaded"
elif grep -q "AppArmor not available" "$SERIAL_LOG" 2>/dev/null; then
check_skip "AppArmor not in kernel (expected before kernel rebuild)"
elif grep -q "AppArmor disabled" "$SERIAL_LOG" 2>/dev/null; then
check_skip "AppArmor disabled via boot parameter"
else
# Check if the 35-apparmor stage ran at all
if grep -q "Stage 35-apparmor.sh" "$SERIAL_LOG" 2>/dev/null; then
check_fail "AppArmor stage ran but status unclear"
else
check_skip "AppArmor stage not found (may not be in init yet)"
fi
fi
echo ""
echo "--- Test 3: Kernel module loading lock ---"
if grep -q "Kernel module loading locked" "$SERIAL_LOG" 2>/dev/null; then
check_pass "Kernel module loading locked"
elif grep -q "Module lock DISABLED" "$SERIAL_LOG" 2>/dev/null; then
check_skip "Module lock disabled via kubesolo.nomodlock"
elif grep -q "Stage 85-security-lockdown.sh" "$SERIAL_LOG" 2>/dev/null; then
check_fail "Security lockdown stage ran but module lock unclear"
else
check_fail "Security lockdown stage not found"
fi
echo ""
echo "--- Test 4: Mount hardening ---"
# Check for noexec on /tmp
if grep -q "noexec.*nosuid.*nodev.*tmpfs.*/tmp" "$SERIAL_LOG" 2>/dev/null || \
grep -q "mount.*tmpfs.*/tmp.*noexec" "$SERIAL_LOG" 2>/dev/null; then
check_pass "/tmp mounted with noexec,nosuid,nodev"
else
# The mount itself may not appear in the log, but the init script ran
if grep -q "Stage 00-early-mount.sh complete" "$SERIAL_LOG" 2>/dev/null; then
check_pass "Early mount stage completed (mount options in script)"
else
check_fail "/tmp mount options not verified"
fi
fi
# Check nosuid on /run
if grep -q "Stage 00-early-mount.sh complete" "$SERIAL_LOG" 2>/dev/null; then
check_pass "/run mounted with nosuid,nodev (early mount complete)"
else
check_fail "/run mount options not verified"
fi
echo ""
echo "--- Test 5: Sysctl hardening ---"
if grep -q "Sysctl settings applied" "$SERIAL_LOG" 2>/dev/null; then
check_pass "Sysctl settings applied (40-sysctl.sh)"
else
check_fail "Sysctl stage did not report success"
fi
# Check specific sysctl values if debug output includes them
if grep -q "kptr_restrict" "$SERIAL_LOG" 2>/dev/null; then
check_pass "kptr_restrict enforced"
elif grep -q "Stage 85-security-lockdown.sh" "$SERIAL_LOG" 2>/dev/null; then
check_pass "kptr_restrict enforced via security lockdown stage"
fi
# ============================================================
# Summary
# ============================================================
echo ""
echo "========================================"
echo " Security Hardening Test Results"
echo "========================================"
echo " Passed: $PASS"
echo " Failed: $FAIL"
echo " Skipped: $SKIP"
echo "========================================"
if [ "$FAIL" -gt 0 ]; then
echo ""
echo "==> FAIL: $FAIL security check(s) failed"
echo ""
echo " Last 40 lines of serial log:"
tail -40 "$SERIAL_LOG" 2>/dev/null
exit 1
fi
echo ""
echo "==> PASS: All security hardening checks passed"
exit 0

139
test/lib/qemu-helpers.sh Normal file
View File

@@ -0,0 +1,139 @@
#!/bin/bash
# qemu-helpers.sh — Shared functions for QEMU-based tests
# Source this file from test scripts: . "$(dirname "$0")/../lib/qemu-helpers.sh"
# extract_kernel_from_iso <iso-path> <extract-dir>
# Sets VMLINUZ and INITRAMFS variables on success
# Falls back to build/rootfs-work/ if available
extract_kernel_from_iso() {
local iso="$1"
local extract_dir="$2"
local project_root="${PROJECT_ROOT:-$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)}"
local rootfs_dir="${ROOTFS_DIR:-$project_root/build/rootfs-work}"
VMLINUZ=""
INITRAMFS=""
# Prefer build artifacts (no extraction needed)
if [ -f "$rootfs_dir/vmlinuz" ] && [ -f "$rootfs_dir/kubesolo-os.gz" ]; then
VMLINUZ="$rootfs_dir/vmlinuz"
INITRAMFS="$rootfs_dir/kubesolo-os.gz"
echo " Using kernel/initramfs from build directory"
return 0
fi
local extracted=0
echo " Extracting kernel/initramfs from ISO..."
# Method 1: bsdtar (ships with macOS, libarchive-tools on Linux)
if [ $extracted -eq 0 ] && command -v bsdtar >/dev/null 2>&1; then
if bsdtar -xf "$iso" -C "$extract_dir" boot/vmlinuz boot/kubesolo-os.gz 2>/dev/null; then
echo " Extracted via bsdtar"
extracted=1
fi
fi
# Method 2: isoinfo (genisoimage/cdrtools)
if [ $extracted -eq 0 ] && command -v isoinfo >/dev/null 2>&1; then
mkdir -p "$extract_dir/boot"
isoinfo -i "$iso" -x "/BOOT/VMLINUZ;1" > "$extract_dir/boot/vmlinuz" 2>/dev/null || true
isoinfo -i "$iso" -x "/BOOT/KUBESOLO-OS.GZ;1" > "$extract_dir/boot/kubesolo-os.gz" 2>/dev/null || true
if [ -s "$extract_dir/boot/vmlinuz" ] && [ -s "$extract_dir/boot/kubesolo-os.gz" ]; then
echo " Extracted via isoinfo"
extracted=1
else
rm -f "$extract_dir/boot/vmlinuz" "$extract_dir/boot/kubesolo-os.gz"
fi
fi
# Method 3: loop mount (Linux only, may need root)
if [ $extracted -eq 0 ] && [ "$(uname)" = "Linux" ]; then
local iso_mount="$extract_dir/mnt"
mkdir -p "$iso_mount"
if mount -o loop,ro "$iso" "$iso_mount" 2>/dev/null; then
mkdir -p "$extract_dir/boot"
cp "$iso_mount/boot/vmlinuz" "$extract_dir/boot/" 2>/dev/null || true
cp "$iso_mount/boot/kubesolo-os.gz" "$extract_dir/boot/" 2>/dev/null || true
umount "$iso_mount" 2>/dev/null || true
if [ -f "$extract_dir/boot/vmlinuz" ] && [ -f "$extract_dir/boot/kubesolo-os.gz" ]; then
echo " Extracted via loop mount"
extracted=1
fi
fi
fi
if [ $extracted -eq 0 ]; then
echo "ERROR: Failed to extract kernel/initramfs from ISO."
echo " Install one of: bsdtar (libarchive-tools), isoinfo (genisoimage), or run as root for loop mount."
return 1
fi
VMLINUZ="$extract_dir/boot/vmlinuz"
INITRAMFS="$extract_dir/boot/kubesolo-os.gz"
return 0
}
# detect_kvm — prints "-enable-kvm" if KVM available, empty string otherwise
detect_kvm() {
if [ -w /dev/kvm ] 2>/dev/null; then
echo "-enable-kvm"
fi
}
# wait_for_boot <serial-log> <qemu-pid> [timeout]
# Waits for "KubeSolo is running" marker in serial log.
# Returns 0 on success, 1 on timeout/failure.
# Sets BOOT_ELAPSED to seconds taken.
wait_for_boot() {
local serial_log="$1"
local qemu_pid="$2"
local timeout="${3:-180}"
BOOT_ELAPSED=0
while [ "$BOOT_ELAPSED" -lt "$timeout" ]; do
if grep -q "\[kubesolo-init\] \[OK\] KubeSolo is running" "$serial_log" 2>/dev/null; then
echo ""
echo " Boot completed in ${BOOT_ELAPSED}s"
return 0
fi
if ! kill -0 "$qemu_pid" 2>/dev/null; then
echo ""
echo "==> FAIL: QEMU exited prematurely"
tail -20 "$serial_log" 2>/dev/null
return 1
fi
sleep 2
BOOT_ELAPSED=$((BOOT_ELAPSED + 2))
printf "\r Elapsed: %ds / %ds" "$BOOT_ELAPSED" "$timeout"
done
echo ""
echo "==> FAIL: Boot did not complete within ${timeout}s"
tail -30 "$serial_log" 2>/dev/null
return 1
}
# fetch_kubeconfig <host-port> <output-file>
# Fetches kubeconfig via HTTP from the given host port.
# The port should be the QEMU-forwarded host port mapped to guest port 8080.
# Returns 0 on success, 1 on failure.
fetch_kubeconfig() {
local port="$1"
local output_file="$2"
echo " Fetching kubeconfig from http://localhost:${port}..."
local j=0
while [ $j -lt 30 ]; do
if curl -sf "http://localhost:${port}" -o "$output_file" 2>/dev/null; then
if [ -s "$output_file" ] && grep -q "server:" "$output_file" 2>/dev/null; then
echo " Kubeconfig fetched successfully"
return 0
fi
fi
sleep 2
j=$((j + 1))
done
echo "==> FAIL: Could not fetch kubeconfig from http://localhost:${port}"
return 1
}

View File

@@ -3,6 +3,7 @@
# Usage: ./test/qemu/run-vm.sh <iso-or-img> [options]
#
# Options:
# --arch <arch> Architecture: x86_64 (default) or arm64
# --data-disk <path> Use existing data disk (default: create temp)
# --data-size <MB> Size of temp data disk (default: 1024)
# --memory <MB> VM memory (default: 2048)
@@ -12,6 +13,8 @@
# --ssh-port <port> Forward SSH to host port (default: 2222)
# --background Run in background, print PID
# --append <args> Extra kernel append args
# --kernel <path> Kernel image (required for arm64)
# --initrd <path> Initramfs image (required for arm64)
#
# Outputs (on stdout):
# QEMU_PID=<pid>
@@ -23,6 +26,7 @@ IMAGE="${1:?Usage: $0 <iso-or-img> [options]}"
shift
# Defaults
ARCH="x86_64"
DATA_DISK=""
DATA_SIZE_MB=1024
MEMORY=2048
@@ -33,10 +37,13 @@ SSH_PORT=2222
BACKGROUND=0
EXTRA_APPEND=""
CREATED_DATA_DISK=""
VM_KERNEL=""
VM_INITRD=""
# Parse options
while [ $# -gt 0 ]; do
case "$1" in
--arch) ARCH="$2"; shift 2 ;;
--data-disk) DATA_DISK="$2"; shift 2 ;;
--data-size) DATA_SIZE_MB="$2"; shift 2 ;;
--memory) MEMORY="$2"; shift 2 ;;
@@ -46,6 +53,8 @@ while [ $# -gt 0 ]; do
--ssh-port) SSH_PORT="$2"; shift 2 ;;
--background) BACKGROUND=1; shift ;;
--append) EXTRA_APPEND="$2"; shift 2 ;;
--kernel) VM_KERNEL="$2"; shift 2 ;;
--initrd) VM_INITRD="$2"; shift 2 ;;
*) echo "Unknown option: $1" >&2; exit 1 ;;
esac
done
@@ -63,44 +72,75 @@ if [ -z "$SERIAL_LOG" ]; then
SERIAL_LOG=$(mktemp /tmp/kubesolo-serial-XXXXXX.log)
fi
# Detect KVM availability
KVM_FLAG=""
if [ -w /dev/kvm ] 2>/dev/null; then
KVM_FLAG="-enable-kvm"
fi
# Build QEMU command based on architecture
if [ "$ARCH" = "arm64" ] || [ "$ARCH" = "aarch64" ]; then
# ARM64: qemu-system-aarch64 with -machine virt
# No KVM for cross-arch emulation (TCG only)
CONSOLE="ttyAMA0"
# Build QEMU command
QEMU_CMD=(
qemu-system-x86_64
-m "$MEMORY"
-smp "$CPUS"
-nographic
-net nic,model=virtio
-net "user,hostfwd=tcp::${API_PORT}-:6443,hostfwd=tcp::${SSH_PORT}-:22"
-drive "file=$DATA_DISK,format=raw,if=virtio"
-serial "file:$SERIAL_LOG"
)
[ -n "$KVM_FLAG" ] && QEMU_CMD+=("$KVM_FLAG")
case "$IMAGE" in
*.iso)
QEMU_CMD+=(
-cdrom "$IMAGE"
-boot d
-append "console=ttyS0,115200n8 kubesolo.data=/dev/vda kubesolo.debug $EXTRA_APPEND"
)
;;
*.img)
QEMU_CMD+=(
-drive "file=$IMAGE,format=raw,if=virtio"
)
;;
*)
echo "ERROR: Unrecognized image format: $IMAGE" >&2
# ARM64 requires explicit kernel + initrd (no -cdrom support with -machine virt)
if [ -z "$VM_KERNEL" ] || [ -z "$VM_INITRD" ]; then
echo "ERROR: ARM64 mode requires --kernel and --initrd options" >&2
exit 1
;;
esac
fi
QEMU_CMD=(
qemu-system-aarch64
-machine virt
-cpu cortex-a72
-m "$MEMORY"
-smp "$CPUS"
-nographic
-net "nic,model=virtio"
-net "user,hostfwd=tcp::${API_PORT}-:6443,hostfwd=tcp::${SSH_PORT}-:22"
-drive "file=$DATA_DISK,format=raw,if=virtio"
-serial "file:$SERIAL_LOG"
-kernel "$VM_KERNEL"
-initrd "$VM_INITRD"
-append "console=${CONSOLE} kubesolo.data=/dev/vda kubesolo.debug $EXTRA_APPEND"
)
else
# x86_64: standard QEMU
CONSOLE="ttyS0,115200n8"
# Detect KVM availability
KVM_FLAG=""
if [ -w /dev/kvm ] 2>/dev/null; then
KVM_FLAG="-enable-kvm"
fi
QEMU_CMD=(
qemu-system-x86_64
-m "$MEMORY"
-smp "$CPUS"
-nographic
-net "nic,model=virtio"
-net "user,hostfwd=tcp::${API_PORT}-:6443,hostfwd=tcp::${SSH_PORT}-:22"
-drive "file=$DATA_DISK,format=raw,if=virtio"
-serial "file:$SERIAL_LOG"
)
[ -n "$KVM_FLAG" ] && QEMU_CMD+=("$KVM_FLAG")
case "$IMAGE" in
*.iso)
QEMU_CMD+=(
-cdrom "$IMAGE"
-boot d
-append "console=${CONSOLE} kubesolo.data=/dev/vda kubesolo.debug $EXTRA_APPEND"
)
;;
*.img)
QEMU_CMD+=(
-drive "file=$IMAGE,format=raw,if=virtio"
)
;;
*)
echo "ERROR: Unrecognized image format: $IMAGE" >&2
exit 1
;;
esac
fi
# Launch
"${QEMU_CMD[@]}" &

117
test/qemu/test-boot-arm64.sh Executable file
View File

@@ -0,0 +1,117 @@
#!/bin/bash
# test-boot-arm64.sh — Verify ARM64 image boots successfully in QEMU
#
# Uses qemu-system-aarch64 with -machine virt to test ARM64 kernel + initramfs.
# Exit 0 = PASS, Exit 1 = FAIL
#
# Usage: ./test/qemu/test-boot-arm64.sh [kernel] [initramfs]
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
KERNEL="${1:-$PROJECT_ROOT/build/cache/custom-kernel-arm64/Image}"
INITRD="${2:-$PROJECT_ROOT/build/rootfs-work/kubesolo-os.gz}"
TIMEOUT=120
echo "==> ARM64 Boot Test"
echo " Kernel: $KERNEL"
echo " Initrd: $INITRD"
echo " Timeout: ${TIMEOUT}s"
# Verify files exist
if [ ! -f "$KERNEL" ]; then
echo "ERROR: Kernel not found: $KERNEL"
echo " Run 'make kernel-arm64' to build the ARM64 kernel."
exit 1
fi
if [ ! -f "$INITRD" ]; then
echo "ERROR: Initrd not found: $INITRD"
echo " Run 'make initramfs' to build the initramfs."
exit 1
fi
# Verify qemu-system-aarch64 is available
if ! command -v qemu-system-aarch64 >/dev/null 2>&1; then
echo "ERROR: qemu-system-aarch64 not found."
echo " Install QEMU with ARM64 support:"
echo " apt install qemu-system-arm # Debian/Ubuntu"
echo " dnf install qemu-system-aarch64 # Fedora/RHEL"
echo " brew install qemu # macOS"
exit 1
fi
# Create temp data disk
DATA_DISK=$(mktemp /tmp/kubesolo-arm64-test-XXXXXX.img)
dd if=/dev/zero of="$DATA_DISK" bs=1M count=512 2>/dev/null
mkfs.ext4 -q -L KSOLODATA "$DATA_DISK" 2>/dev/null
SERIAL_LOG=$(mktemp /tmp/kubesolo-arm64-serial-XXXXXX.log)
QEMU_PID=""
cleanup() {
[ -n "$QEMU_PID" ] && kill "$QEMU_PID" 2>/dev/null || true
rm -f "$DATA_DISK" "$SERIAL_LOG"
}
trap cleanup EXIT
# Launch QEMU in background
qemu-system-aarch64 \
-machine virt \
-cpu cortex-a72 \
-m 2048 \
-smp 2 \
-nographic \
-kernel "$KERNEL" \
-initrd "$INITRD" \
-append "console=ttyAMA0 kubesolo.data=/dev/vda kubesolo.debug" \
-drive "file=$DATA_DISK,format=raw,if=virtio" \
-net nic,model=virtio \
-net user \
-serial "file:$SERIAL_LOG" &
QEMU_PID=$!
# Wait for boot success marker
echo " Waiting for boot..."
ELAPSED=0
SUCCESS=0
while [ "$ELAPSED" -lt "$TIMEOUT" ]; do
# Check for stage 90 completion (same marker as x86_64 test)
if grep -q "\[kubesolo-init\] \[OK\] Stage 90-kubesolo.sh complete" "$SERIAL_LOG" 2>/dev/null; then
SUCCESS=1
break
fi
# Also check for generic KubeSolo running message
if grep -q "KubeSolo is running" "$SERIAL_LOG" 2>/dev/null; then
SUCCESS=1
break
fi
# Check if QEMU exited prematurely
if ! kill -0 "$QEMU_PID" 2>/dev/null; then
echo ""
echo "==> FAIL: QEMU exited prematurely"
echo " Last 20 lines of serial output:"
tail -20 "$SERIAL_LOG" 2>/dev/null || echo " (no output)"
exit 1
fi
sleep 2
ELAPSED=$((ELAPSED + 2))
printf "\r Elapsed: %ds / %ds" "$ELAPSED" "$TIMEOUT"
done
echo ""
# Kill QEMU
kill "$QEMU_PID" 2>/dev/null || true
wait "$QEMU_PID" 2>/dev/null || true
QEMU_PID=""
if [ "$SUCCESS" = "1" ]; then
echo "==> ARM64 Boot Test PASSED (${ELAPSED}s)"
exit 0
else
echo "==> ARM64 Boot Test FAILED (timeout ${TIMEOUT}s)"
echo ""
echo "==> Last 30 lines of serial output:"
tail -30 "$SERIAL_LOG" 2>/dev/null || echo " (no output)"
exit 1
fi

View File

@@ -5,17 +5,25 @@
set -euo pipefail
ISO="${1:?Usage: $0 <path-to-iso>}"
TIMEOUT_BOOT=120 # seconds to wait for boot success marker
TIMEOUT_BOOT=${TIMEOUT_BOOT:-120} # seconds to wait for boot success marker
SERIAL_LOG=$(mktemp /tmp/kubesolo-boot-test-XXXXXX.log)
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
. "$SCRIPT_DIR/../lib/qemu-helpers.sh"
# Temp data disk
DATA_DISK=$(mktemp /tmp/kubesolo-data-XXXXXX.img)
dd if=/dev/zero of="$DATA_DISK" bs=1M count=512 2>/dev/null
mkfs.ext4 -q -L KSOLODATA "$DATA_DISK" 2>/dev/null
QEMU_PID=""
EXTRACT_DIR=""
cleanup() {
kill "$QEMU_PID" 2>/dev/null || true
[ -n "$QEMU_PID" ] && kill "$QEMU_PID" 2>/dev/null || true
rm -f "$DATA_DISK" "$SERIAL_LOG"
[ -n "$EXTRACT_DIR" ] && rm -rf "$EXTRACT_DIR"
}
trap cleanup EXIT
@@ -23,16 +31,25 @@ echo "==> Boot test: $ISO"
echo " Timeout: ${TIMEOUT_BOOT}s"
echo " Serial log: $SERIAL_LOG"
# Launch QEMU in background
# Extract kernel from ISO
EXTRACT_DIR="$(mktemp -d /tmp/kubesolo-extract-XXXXXX)"
extract_kernel_from_iso "$ISO" "$EXTRACT_DIR"
KVM_FLAG=$(detect_kvm)
[ -n "$KVM_FLAG" ] && echo " KVM acceleration: enabled"
# Launch QEMU in background with direct kernel boot
# shellcheck disable=SC2086
qemu-system-x86_64 \
-m 2048 -smp 2 \
-nographic \
-cdrom "$ISO" \
-boot d \
$KVM_FLAG \
-kernel "$VMLINUZ" \
-initrd "$INITRAMFS" \
-drive "file=$DATA_DISK,format=raw,if=virtio" \
-net nic,model=virtio \
-net "nic,model=virtio" \
-net user \
-serial file:"$SERIAL_LOG" \
-serial "file:$SERIAL_LOG" \
-append "console=ttyS0,115200n8 kubesolo.data=/dev/vda kubesolo.debug" \
&
QEMU_PID=$!
@@ -41,7 +58,7 @@ QEMU_PID=$!
echo " Waiting for boot..."
ELAPSED=0
while [ "$ELAPSED" -lt "$TIMEOUT_BOOT" ]; do
if grep -q "\[kubesolo-init\] \[OK\] Stage 90-kubesolo.sh complete" "$SERIAL_LOG" 2>/dev/null; then
if grep -q "\[kubesolo-init\] \[OK\] KubeSolo is running" "$SERIAL_LOG" 2>/dev/null; then
echo ""
echo "==> PASS: KubeSolo OS booted successfully in ${ELAPSED}s"
exit 0

View File

@@ -3,8 +3,6 @@ package cmd
import (
"fmt"
"log/slog"
"github.com/portainer/kubesolo-os/update/pkg/grubenv"
)
// Activate switches the boot target to the passive partition.
@@ -12,7 +10,7 @@ import (
// with boot_counter=3. If health checks fail 3 times, GRUB auto-rolls back.
func Activate(args []string) error {
opts := parseOpts(args)
env := grubenv.New(opts.GrubenvPath)
env := opts.NewBootEnv()
// Get passive slot (the one we want to boot into)
passiveSlot, err := env.PassiveSlot()

View File

@@ -4,7 +4,6 @@ import (
"fmt"
"log/slog"
"github.com/portainer/kubesolo-os/update/pkg/grubenv"
"github.com/portainer/kubesolo-os/update/pkg/image"
"github.com/portainer/kubesolo-os/update/pkg/partition"
)
@@ -18,7 +17,7 @@ func Apply(args []string) error {
return fmt.Errorf("--server is required")
}
env := grubenv.New(opts.GrubenvPath)
env := opts.NewBootEnv()
// Determine passive slot
passiveSlot, err := env.PassiveSlot()

View File

@@ -4,7 +4,6 @@ import (
"fmt"
"log/slog"
"github.com/portainer/kubesolo-os/update/pkg/grubenv"
"github.com/portainer/kubesolo-os/update/pkg/image"
"github.com/portainer/kubesolo-os/update/pkg/partition"
)
@@ -19,7 +18,7 @@ func Check(args []string) error {
}
// Get current version from active partition
env := grubenv.New(opts.GrubenvPath)
env := opts.NewBootEnv()
activeSlot, err := env.ActiveSlot()
if err != nil {
return fmt.Errorf("reading active slot: %w", err)

View File

@@ -5,7 +5,6 @@ import (
"log/slog"
"time"
"github.com/portainer/kubesolo-os/update/pkg/grubenv"
"github.com/portainer/kubesolo-os/update/pkg/health"
)
@@ -15,7 +14,7 @@ import (
// init script) to confirm the system is healthy.
func Healthcheck(args []string) error {
opts := parseOpts(args)
env := grubenv.New(opts.GrubenvPath)
env := opts.NewBootEnv()
// Check if already marked successful
success, err := env.BootSuccess()

View File

@@ -1,11 +1,27 @@
package cmd
import (
"github.com/portainer/kubesolo-os/update/pkg/bootenv"
)
// opts holds shared command-line options for all subcommands.
type opts struct {
ServerURL string
GrubenvPath string
TimeoutSecs int
PubKeyPath string
BootEnvType string // "grub" or "rpi"
BootEnvPath string // path for RPi boot control dir
}
// NewBootEnv creates a BootEnv from the parsed options.
func (o opts) NewBootEnv() bootenv.BootEnv {
switch o.BootEnvType {
case "rpi":
return bootenv.NewRPi(o.BootEnvPath)
default:
return bootenv.NewGRUB(o.GrubenvPath)
}
}
// parseOpts extracts command-line flags from args.
@@ -14,6 +30,7 @@ func parseOpts(args []string) opts {
o := opts{
GrubenvPath: "/boot/grub/grubenv",
TimeoutSecs: 120,
BootEnvType: "grub",
}
for i := 0; i < len(args); i++ {
@@ -46,6 +63,16 @@ func parseOpts(args []string) opts {
o.PubKeyPath = args[i+1]
i++
}
case "--bootenv":
if i+1 < len(args) {
o.BootEnvType = args[i+1]
i++
}
case "--bootenv-path":
if i+1 < len(args) {
o.BootEnvPath = args[i+1]
i++
}
}
}

View File

@@ -3,15 +3,13 @@ package cmd
import (
"fmt"
"log/slog"
"github.com/portainer/kubesolo-os/update/pkg/grubenv"
)
// Rollback forces an immediate switch to the other partition.
// Use this to manually revert to the previous version.
func Rollback(args []string) error {
opts := parseOpts(args)
env := grubenv.New(opts.GrubenvPath)
env := opts.NewBootEnv()
activeSlot, err := env.ActiveSlot()
if err != nil {

View File

@@ -2,42 +2,50 @@ package cmd
import (
"fmt"
"github.com/portainer/kubesolo-os/update/pkg/grubenv"
)
// Status displays the current A/B slot configuration and boot state.
func Status(args []string) error {
opts := parseOpts(args)
env := grubenv.New(opts.GrubenvPath)
env := opts.NewBootEnv()
vars, err := env.ReadAll()
activeSlot, err := env.ActiveSlot()
if err != nil {
return fmt.Errorf("reading GRUB environment: %w", err)
return fmt.Errorf("reading active slot: %w", err)
}
activeSlot := vars["active_slot"]
bootCounter := vars["boot_counter"]
bootSuccess := vars["boot_success"]
passiveSlot, err := env.PassiveSlot()
if err != nil {
return fmt.Errorf("reading passive slot: %w", err)
}
passiveSlot := "B"
if activeSlot == "B" {
passiveSlot = "A"
bootCounter, err := env.BootCounter()
if err != nil {
return fmt.Errorf("reading boot counter: %w", err)
}
bootSuccess, err := env.BootSuccess()
if err != nil {
return fmt.Errorf("reading boot success: %w", err)
}
fmt.Println("KubeSolo OS — A/B Partition Status")
fmt.Println("───────────────────────────────────")
fmt.Printf(" Active slot: %s\n", activeSlot)
fmt.Printf(" Passive slot: %s\n", passiveSlot)
fmt.Printf(" Boot counter: %s\n", bootCounter)
fmt.Printf(" Boot success: %s\n", bootSuccess)
fmt.Printf(" Boot counter: %d\n", bootCounter)
if bootSuccess {
fmt.Printf(" Boot success: 1\n")
} else {
fmt.Printf(" Boot success: 0\n")
}
if bootSuccess == "1" {
if bootSuccess {
fmt.Println("\n ✓ System is healthy (boot confirmed)")
} else if bootCounter == "0" {
} else if bootCounter == 0 {
fmt.Println("\n ✗ Boot counter exhausted — rollback will occur on next reboot")
} else {
fmt.Printf("\n ⚠ Boot pending verification (%s attempts remaining)\n", bootCounter)
fmt.Printf("\n ⚠ Boot pending verification (%d attempts remaining)\n", bootCounter)
}
return nil

View File

@@ -0,0 +1,27 @@
// Package bootenv provides a platform-independent interface for managing
// A/B boot environments. It abstracts GRUB (x86_64) and RPi firmware
// (ARM64) behind a common interface.
package bootenv
// BootEnv provides read/write access to A/B boot environment variables.
type BootEnv interface {
// ActiveSlot returns the currently active boot slot ("A" or "B").
ActiveSlot() (string, error)
// PassiveSlot returns the currently passive boot slot.
PassiveSlot() (string, error)
// BootCounter returns the current boot counter value.
BootCounter() (int, error)
// BootSuccess returns whether the last boot was marked successful.
BootSuccess() (bool, error)
// MarkBootSuccess marks the current boot as successful.
MarkBootSuccess() error
// ActivateSlot switches the active boot slot and resets the counter.
ActivateSlot(slot string) error
// ForceRollback switches to the other slot immediately.
ForceRollback() error
}
const (
SlotA = "A"
SlotB = "B"
)

View File

@@ -0,0 +1,533 @@
package bootenv
import (
"os"
"path/filepath"
"strconv"
"strings"
"testing"
)
// createTestGrubenv writes a properly formatted 1024-byte grubenv file.
func createTestGrubenv(t *testing.T, dir string, vars map[string]string) string {
t.Helper()
path := filepath.Join(dir, "grubenv")
var sb strings.Builder
sb.WriteString("# GRUB Environment Block\n")
for k, v := range vars {
sb.WriteString(k + "=" + v + "\n")
}
content := sb.String()
padding := 1024 - len(content)
if padding > 0 {
content += strings.Repeat("#", padding)
}
if err := os.WriteFile(path, []byte(content), 0o644); err != nil {
t.Fatal(err)
}
return path
}
// TestGRUBActiveSlot verifies ActiveSlot reads the correct value.
func TestGRUBActiveSlot(t *testing.T) {
dir := t.TempDir()
path := createTestGrubenv(t, dir, map[string]string{
"active_slot": "A",
"boot_counter": "3",
"boot_success": "1",
})
env := NewGRUB(path)
slot, err := env.ActiveSlot()
if err != nil {
t.Fatal(err)
}
if slot != "A" {
t.Errorf("expected A, got %s", slot)
}
}
// TestGRUBPassiveSlot verifies PassiveSlot returns the opposite slot.
func TestGRUBPassiveSlot(t *testing.T) {
dir := t.TempDir()
path := createTestGrubenv(t, dir, map[string]string{
"active_slot": "A",
"boot_counter": "3",
"boot_success": "0",
})
env := NewGRUB(path)
passive, err := env.PassiveSlot()
if err != nil {
t.Fatal(err)
}
if passive != "B" {
t.Errorf("expected B, got %s", passive)
}
}
// TestGRUBBootCounter verifies BootCounter reads the correct value.
func TestGRUBBootCounter(t *testing.T) {
dir := t.TempDir()
path := createTestGrubenv(t, dir, map[string]string{
"active_slot": "A",
"boot_counter": "2",
"boot_success": "0",
})
env := NewGRUB(path)
counter, err := env.BootCounter()
if err != nil {
t.Fatal(err)
}
if counter != 2 {
t.Errorf("expected 2, got %d", counter)
}
}
// TestGRUBBootSuccess verifies BootSuccess reads the correct value.
func TestGRUBBootSuccess(t *testing.T) {
dir := t.TempDir()
path := createTestGrubenv(t, dir, map[string]string{
"active_slot": "A",
"boot_counter": "3",
"boot_success": "1",
})
env := NewGRUB(path)
success, err := env.BootSuccess()
if err != nil {
t.Fatal(err)
}
if !success {
t.Error("expected true, got false")
}
}
// TestGRUBMarkBootSuccess verifies marking boot as successful.
func TestGRUBMarkBootSuccess(t *testing.T) {
dir := t.TempDir()
path := createTestGrubenv(t, dir, map[string]string{
"active_slot": "B",
"boot_counter": "1",
"boot_success": "0",
})
env := NewGRUB(path)
if err := env.MarkBootSuccess(); err != nil {
t.Fatal(err)
}
success, err := env.BootSuccess()
if err != nil {
t.Fatal(err)
}
if !success {
t.Error("expected boot_success=true after MarkBootSuccess")
}
counter, err := env.BootCounter()
if err != nil {
t.Fatal(err)
}
if counter != 3 {
t.Errorf("expected boot_counter=3 after MarkBootSuccess, got %d", counter)
}
}
// TestGRUBActivateSlot verifies slot activation sets correct state.
func TestGRUBActivateSlot(t *testing.T) {
dir := t.TempDir()
path := createTestGrubenv(t, dir, map[string]string{
"active_slot": "A",
"boot_counter": "3",
"boot_success": "1",
})
env := NewGRUB(path)
if err := env.ActivateSlot("B"); err != nil {
t.Fatal(err)
}
slot, _ := env.ActiveSlot()
if slot != "B" {
t.Errorf("expected B, got %s", slot)
}
counter, _ := env.BootCounter()
if counter != 3 {
t.Errorf("expected counter=3, got %d", counter)
}
success, _ := env.BootSuccess()
if success {
t.Error("expected boot_success=false after ActivateSlot")
}
}
// TestGRUBForceRollback verifies rollback switches to passive slot.
func TestGRUBForceRollback(t *testing.T) {
dir := t.TempDir()
path := createTestGrubenv(t, dir, map[string]string{
"active_slot": "A",
"boot_counter": "3",
"boot_success": "1",
})
env := NewGRUB(path)
if err := env.ForceRollback(); err != nil {
t.Fatal(err)
}
slot, _ := env.ActiveSlot()
if slot != "B" {
t.Errorf("expected B after rollback from A, got %s", slot)
}
}
// TestGRUBSlotCycling verifies A->B->A slot switching.
func TestGRUBSlotCycling(t *testing.T) {
dir := t.TempDir()
path := createTestGrubenv(t, dir, map[string]string{
"active_slot": "A",
"boot_counter": "3",
"boot_success": "1",
})
env := NewGRUB(path)
// A -> B
if err := env.ActivateSlot("B"); err != nil {
t.Fatal(err)
}
slot, _ := env.ActiveSlot()
if slot != "B" {
t.Fatalf("expected B, got %s", slot)
}
// B -> A
if err := env.ActivateSlot("A"); err != nil {
t.Fatal(err)
}
slot, _ = env.ActiveSlot()
if slot != "A" {
t.Fatalf("expected A, got %s", slot)
}
}
// TestGRUBActivateInvalidSlot verifies invalid slot is rejected.
func TestGRUBActivateInvalidSlot(t *testing.T) {
dir := t.TempDir()
path := createTestGrubenv(t, dir, map[string]string{
"active_slot": "A",
"boot_counter": "3",
"boot_success": "0",
})
env := NewGRUB(path)
if err := env.ActivateSlot("C"); err == nil {
t.Fatal("expected error for invalid slot")
}
}
// TestRPiActiveSlot verifies ActiveSlot reads from autoboot.txt.
func TestRPiActiveSlot(t *testing.T) {
dir := t.TempDir()
createTestAutobootFiles(t, dir, 2, 3, 3, false)
env := NewRPi(dir)
slot, err := env.ActiveSlot()
if err != nil {
t.Fatal(err)
}
if slot != "A" {
t.Errorf("expected A (partition 2), got %s", slot)
}
}
// TestRPiActiveSlotB verifies slot B with partition 3.
func TestRPiActiveSlotB(t *testing.T) {
dir := t.TempDir()
createTestAutobootFiles(t, dir, 3, 2, 3, true)
env := NewRPi(dir)
slot, err := env.ActiveSlot()
if err != nil {
t.Fatal(err)
}
if slot != "B" {
t.Errorf("expected B (partition 3), got %s", slot)
}
}
// TestRPiPassiveSlot verifies passive slot is opposite of active.
func TestRPiPassiveSlot(t *testing.T) {
dir := t.TempDir()
createTestAutobootFiles(t, dir, 2, 3, 3, false)
env := NewRPi(dir)
passive, err := env.PassiveSlot()
if err != nil {
t.Fatal(err)
}
if passive != "B" {
t.Errorf("expected B, got %s", passive)
}
}
// TestRPiBootCounter verifies counter is read from status file.
func TestRPiBootCounter(t *testing.T) {
dir := t.TempDir()
createTestAutobootFiles(t, dir, 2, 3, 2, false)
env := NewRPi(dir)
counter, err := env.BootCounter()
if err != nil {
t.Fatal(err)
}
if counter != 2 {
t.Errorf("expected 2, got %d", counter)
}
}
// TestRPiBootCounterMissingFile verifies default when status file is absent.
func TestRPiBootCounterMissingFile(t *testing.T) {
dir := t.TempDir()
// Only create autoboot.txt, no boot-status
autoboot := "[all]\ntryboot_a_b=1\nboot_partition=2\n[tryboot]\nboot_partition=3\n"
if err := os.WriteFile(filepath.Join(dir, "autoboot.txt"), []byte(autoboot), 0o644); err != nil {
t.Fatal(err)
}
env := NewRPi(dir)
counter, err := env.BootCounter()
if err != nil {
t.Fatal(err)
}
if counter != 3 {
t.Errorf("expected default counter 3, got %d", counter)
}
}
// TestRPiBootSuccess verifies success is read from status file.
func TestRPiBootSuccess(t *testing.T) {
dir := t.TempDir()
createTestAutobootFiles(t, dir, 2, 3, 3, true)
env := NewRPi(dir)
success, err := env.BootSuccess()
if err != nil {
t.Fatal(err)
}
if !success {
t.Error("expected true, got false")
}
}
// TestRPiMarkBootSuccess verifies marking boot success updates both files.
func TestRPiMarkBootSuccess(t *testing.T) {
dir := t.TempDir()
createTestAutobootFiles(t, dir, 2, 3, 1, false)
env := NewRPi(dir)
if err := env.MarkBootSuccess(); err != nil {
t.Fatal(err)
}
// Active slot should still be A
slot, _ := env.ActiveSlot()
if slot != "A" {
t.Errorf("expected active slot A, got %s", slot)
}
// Boot success should be true
success, _ := env.BootSuccess()
if !success {
t.Error("expected boot_success=true after MarkBootSuccess")
}
// Counter should be reset to 3
counter, _ := env.BootCounter()
if counter != 3 {
t.Errorf("expected counter=3 after MarkBootSuccess, got %d", counter)
}
// [all] boot_partition should be 2 (slot A, making it permanent)
data, _ := os.ReadFile(filepath.Join(dir, "autoboot.txt"))
if !strings.Contains(string(data), "boot_partition=2") {
t.Error("expected [all] boot_partition=2 after MarkBootSuccess")
}
}
// TestRPiActivateSlot verifies slot activation updates tryboot and status.
func TestRPiActivateSlot(t *testing.T) {
dir := t.TempDir()
createTestAutobootFiles(t, dir, 2, 3, 3, true)
env := NewRPi(dir)
if err := env.ActivateSlot("B"); err != nil {
t.Fatal(err)
}
// [tryboot] should now point to partition 3 (slot B)
data, _ := os.ReadFile(filepath.Join(dir, "autoboot.txt"))
content := string(data)
// Find [tryboot] section and check boot_partition
idx := strings.Index(content, "[tryboot]")
if idx < 0 {
t.Fatal("missing [tryboot] section")
}
trybootSection := content[idx:]
if !strings.Contains(trybootSection, "boot_partition=3") {
t.Errorf("expected [tryboot] boot_partition=3, got: %s", trybootSection)
}
// Status should be reset
success, _ := env.BootSuccess()
if success {
t.Error("expected boot_success=false after ActivateSlot")
}
counter, _ := env.BootCounter()
if counter != 3 {
t.Errorf("expected counter=3, got %d", counter)
}
}
// TestRPiActivateInvalidSlot verifies invalid slot is rejected.
func TestRPiActivateInvalidSlot(t *testing.T) {
dir := t.TempDir()
createTestAutobootFiles(t, dir, 2, 3, 3, false)
env := NewRPi(dir)
if err := env.ActivateSlot("C"); err == nil {
t.Fatal("expected error for invalid slot")
}
}
// TestRPiForceRollback verifies rollback swaps the active slot.
func TestRPiForceRollback(t *testing.T) {
dir := t.TempDir()
createTestAutobootFiles(t, dir, 2, 3, 3, true)
env := NewRPi(dir)
if err := env.ForceRollback(); err != nil {
t.Fatal(err)
}
// [all] should now point to partition 3 (slot B)
slot, _ := env.ActiveSlot()
if slot != "B" {
t.Errorf("expected B after rollback from A, got %s", slot)
}
// Success should be false
success, _ := env.BootSuccess()
if success {
t.Error("expected boot_success=false after ForceRollback")
}
}
// TestRPiSlotCycling verifies A->B->A slot switching works.
func TestRPiSlotCycling(t *testing.T) {
dir := t.TempDir()
createTestAutobootFiles(t, dir, 2, 3, 3, true)
env := NewRPi(dir)
// Rollback A -> B
if err := env.ForceRollback(); err != nil {
t.Fatal(err)
}
slot, _ := env.ActiveSlot()
if slot != "B" {
t.Fatalf("expected B, got %s", slot)
}
// Rollback B -> A
if err := env.ForceRollback(); err != nil {
t.Fatal(err)
}
slot, _ = env.ActiveSlot()
if slot != "A" {
t.Fatalf("expected A, got %s", slot)
}
}
// TestInterfaceCompliance verifies both implementations satisfy BootEnv.
func TestInterfaceCompliance(t *testing.T) {
dir := t.TempDir()
grubPath := createTestGrubenv(t, dir, map[string]string{
"active_slot": "A",
"boot_counter": "3",
"boot_success": "0",
})
rpiDir := t.TempDir()
createTestAutobootFiles(t, rpiDir, 2, 3, 3, false)
impls := map[string]BootEnv{
"grub": NewGRUB(grubPath),
"rpi": NewRPi(rpiDir),
}
for name, env := range impls {
t.Run(name, func(t *testing.T) {
slot, err := env.ActiveSlot()
if err != nil {
t.Fatalf("ActiveSlot: %v", err)
}
if slot != "A" {
t.Errorf("ActiveSlot: expected A, got %s", slot)
}
passive, err := env.PassiveSlot()
if err != nil {
t.Fatalf("PassiveSlot: %v", err)
}
if passive != "B" {
t.Errorf("PassiveSlot: expected B, got %s", passive)
}
counter, err := env.BootCounter()
if err != nil {
t.Fatalf("BootCounter: %v", err)
}
if counter != 3 {
t.Errorf("BootCounter: expected 3, got %d", counter)
}
success, err := env.BootSuccess()
if err != nil {
t.Fatalf("BootSuccess: %v", err)
}
if success {
t.Error("BootSuccess: expected false")
}
})
}
}
// createTestAutobootFiles is a helper that writes both autoboot.txt and boot-status.
func createTestAutobootFiles(t *testing.T, dir string, allPart, trybootPart, counter int, success bool) {
t.Helper()
autoboot := "[all]\ntryboot_a_b=1\nboot_partition=" + strconv.Itoa(allPart) + "\n"
autoboot += "[tryboot]\nboot_partition=" + strconv.Itoa(trybootPart) + "\n"
if err := os.WriteFile(filepath.Join(dir, "autoboot.txt"), []byte(autoboot), 0o644); err != nil {
t.Fatal(err)
}
successVal := "0"
if success {
successVal = "1"
}
status := "boot_counter=" + strconv.Itoa(counter) + "\nboot_success=" + successVal + "\n"
if err := os.WriteFile(filepath.Join(dir, "boot-status"), []byte(status), 0o644); err != nil {
t.Fatal(err)
}
}

View File

@@ -0,0 +1,23 @@
package bootenv
import (
"github.com/portainer/kubesolo-os/update/pkg/grubenv"
)
// GRUBEnv implements BootEnv using GRUB environment variables.
type GRUBEnv struct {
env *grubenv.Env
}
// NewGRUB creates a new GRUB-based BootEnv.
func NewGRUB(path string) BootEnv {
return &GRUBEnv{env: grubenv.New(path)}
}
func (g *GRUBEnv) ActiveSlot() (string, error) { return g.env.ActiveSlot() }
func (g *GRUBEnv) PassiveSlot() (string, error) { return g.env.PassiveSlot() }
func (g *GRUBEnv) BootCounter() (int, error) { return g.env.BootCounter() }
func (g *GRUBEnv) BootSuccess() (bool, error) { return g.env.BootSuccess() }
func (g *GRUBEnv) MarkBootSuccess() error { return g.env.MarkBootSuccess() }
func (g *GRUBEnv) ActivateSlot(slot string) error { return g.env.ActivateSlot(slot) }
func (g *GRUBEnv) ForceRollback() error { return g.env.ForceRollback() }

267
update/pkg/bootenv/rpi.go Normal file
View File

@@ -0,0 +1,267 @@
package bootenv
import (
"fmt"
"os"
"path/filepath"
"strconv"
"strings"
)
const (
// RPi partition numbers: slot A = partition 2, slot B = partition 3.
rpiSlotAPartition = 2
rpiSlotBPartition = 3
defaultBootCounter = 3
)
// RPiEnv implements BootEnv using Raspberry Pi firmware autoboot.txt.
type RPiEnv struct {
autobootPath string // path to autoboot.txt
statusPath string // path to boot-status file
}
// NewRPi creates a new RPi-based BootEnv.
// dir is the directory containing autoboot.txt (typically the boot control
// partition mount point).
func NewRPi(dir string) BootEnv {
return &RPiEnv{
autobootPath: filepath.Join(dir, "autoboot.txt"),
statusPath: filepath.Join(dir, "boot-status"),
}
}
func (r *RPiEnv) ActiveSlot() (string, error) {
partNum, err := r.readAllBootPartition()
if err != nil {
return "", fmt.Errorf("reading active slot: %w", err)
}
return partNumToSlot(partNum)
}
func (r *RPiEnv) PassiveSlot() (string, error) {
active, err := r.ActiveSlot()
if err != nil {
return "", err
}
if active == SlotA {
return SlotB, nil
}
return SlotA, nil
}
func (r *RPiEnv) BootCounter() (int, error) {
status, err := r.readStatus()
if err != nil {
return -1, err
}
val, ok := status["boot_counter"]
if !ok {
return defaultBootCounter, nil
}
n, err := strconv.Atoi(val)
if err != nil {
return -1, fmt.Errorf("invalid boot_counter %q: %w", val, err)
}
return n, nil
}
func (r *RPiEnv) BootSuccess() (bool, error) {
status, err := r.readStatus()
if err != nil {
return false, err
}
return status["boot_success"] == "1", nil
}
func (r *RPiEnv) MarkBootSuccess() error {
// Make the current slot permanent by updating [all] boot_partition
active, err := r.ActiveSlot()
if err != nil {
return fmt.Errorf("marking boot success: %w", err)
}
partNum := slotToPartNum(active)
if err := r.writeAllBootPartition(partNum); err != nil {
return err
}
return r.writeStatus(defaultBootCounter, true)
}
func (r *RPiEnv) ActivateSlot(slot string) error {
if slot != SlotA && slot != SlotB {
return fmt.Errorf("invalid slot: %q (must be A or B)", slot)
}
partNum := slotToPartNum(slot)
// Update [tryboot] to point to the new slot
if err := r.writeTrybootPartition(partNum); err != nil {
return err
}
return r.writeStatus(defaultBootCounter, false)
}
func (r *RPiEnv) ForceRollback() error {
passive, err := r.PassiveSlot()
if err != nil {
return err
}
// Swap the [all] boot_partition to the other slot
partNum := slotToPartNum(passive)
if err := r.writeAllBootPartition(partNum); err != nil {
return err
}
if err := r.writeTrybootPartition(partNum); err != nil {
return err
}
return r.writeStatus(defaultBootCounter, false)
}
// readAllBootPartition reads the boot_partition value from the [all] section.
func (r *RPiEnv) readAllBootPartition() (int, error) {
sections, err := r.parseAutoboot()
if err != nil {
return 0, err
}
val, ok := sections["all"]["boot_partition"]
if !ok {
return 0, fmt.Errorf("boot_partition not found in [all] section")
}
return strconv.Atoi(val)
}
// writeAllBootPartition updates the [all] boot_partition value.
func (r *RPiEnv) writeAllBootPartition(partNum int) error {
sections, err := r.parseAutoboot()
if err != nil {
return err
}
if sections["all"] == nil {
sections["all"] = make(map[string]string)
}
sections["all"]["boot_partition"] = strconv.Itoa(partNum)
return r.writeAutoboot(sections)
}
// writeTrybootPartition updates the [tryboot] boot_partition value.
func (r *RPiEnv) writeTrybootPartition(partNum int) error {
sections, err := r.parseAutoboot()
if err != nil {
return err
}
if sections["tryboot"] == nil {
sections["tryboot"] = make(map[string]string)
}
sections["tryboot"]["boot_partition"] = strconv.Itoa(partNum)
return r.writeAutoboot(sections)
}
// parseAutoboot reads autoboot.txt into a map of section -> key=value pairs.
func (r *RPiEnv) parseAutoboot() (map[string]map[string]string, error) {
data, err := os.ReadFile(r.autobootPath)
if err != nil {
return nil, fmt.Errorf("reading autoboot.txt: %w", err)
}
sections := make(map[string]map[string]string)
currentSection := ""
for _, line := range strings.Split(string(data), "\n") {
line = strings.TrimSpace(line)
if line == "" {
continue
}
if strings.HasPrefix(line, "[") && strings.HasSuffix(line, "]") {
currentSection = line[1 : len(line)-1]
if sections[currentSection] == nil {
sections[currentSection] = make(map[string]string)
}
continue
}
parts := strings.SplitN(line, "=", 2)
if len(parts) == 2 && currentSection != "" {
sections[currentSection][strings.TrimSpace(parts[0])] = strings.TrimSpace(parts[1])
}
}
return sections, nil
}
// writeAutoboot writes sections back to autoboot.txt.
// Section order: [all] first, then [tryboot].
func (r *RPiEnv) writeAutoboot(sections map[string]map[string]string) error {
var sb strings.Builder
// Write [all] section first
if all, ok := sections["all"]; ok {
sb.WriteString("[all]\n")
for k, v := range all {
sb.WriteString(k + "=" + v + "\n")
}
}
// Write [tryboot] section
if tryboot, ok := sections["tryboot"]; ok {
sb.WriteString("[tryboot]\n")
for k, v := range tryboot {
sb.WriteString(k + "=" + v + "\n")
}
}
return os.WriteFile(r.autobootPath, []byte(sb.String()), 0o644)
}
// readStatus reads the boot-status key=value file.
func (r *RPiEnv) readStatus() (map[string]string, error) {
data, err := os.ReadFile(r.statusPath)
if err != nil {
if os.IsNotExist(err) {
// Return defaults if status file doesn't exist yet
return map[string]string{
"boot_counter": strconv.Itoa(defaultBootCounter),
"boot_success": "0",
}, nil
}
return nil, fmt.Errorf("reading boot-status: %w", err)
}
status := make(map[string]string)
for _, line := range strings.Split(string(data), "\n") {
line = strings.TrimSpace(line)
if line == "" {
continue
}
parts := strings.SplitN(line, "=", 2)
if len(parts) == 2 {
status[strings.TrimSpace(parts[0])] = strings.TrimSpace(parts[1])
}
}
return status, nil
}
// writeStatus writes boot_counter and boot_success to the status file.
func (r *RPiEnv) writeStatus(counter int, success bool) error {
successVal := "0"
if success {
successVal = "1"
}
content := fmt.Sprintf("boot_counter=%d\nboot_success=%s\n", counter, successVal)
return os.WriteFile(r.statusPath, []byte(content), 0o644)
}
func partNumToSlot(partNum int) (string, error) {
switch partNum {
case rpiSlotAPartition:
return SlotA, nil
case rpiSlotBPartition:
return SlotB, nil
default:
return "", fmt.Errorf("unknown partition number %d (expected %d or %d)", partNum, rpiSlotAPartition, rpiSlotBPartition)
}
}
func slotToPartNum(slot string) int {
if slot == SlotB {
return rpiSlotBPartition
}
return rpiSlotAPartition
}