kubesolo-os/docs/arm64-architecture.md

ARM64 Build Architecture

KubeSolo OS supports ARM64 via two distinct build tracks. This document defines the split, lists which files belong to each track, and identifies the shared substrate.

The two tracks

Generic ARM64 (UEFI / virtio / GRUB)

Target: Any UEFI-compliant ARM64 host — Ampere/Graviton VMs, generic ARM64 servers, qemu-system-aarch64 -machine virt, future SBCs that boot via UEFI.

Boot path: UEFI firmware → GRUB-EFI → kernel + initramfs → KubeSolo init.

Kernel: Mainline Linux (kernel.org LTS), built from defconfig + shared container-config fragment.

Storage: virtio-blk / NVMe / SATA — detected and probed by mainline drivers.

Disk image format: GPT, identical 4-partition layout to x86_64 (EFI + System A

• System B + Data).

Raspberry Pi ARM64

Target: Raspberry Pi 4 and 5 specifically.

Boot path: RPi EEPROM → VideoCore firmware (start4.elf) → config.txt → kernel + DTB + initramfs → KubeSolo init. (No UEFI, no GRUB — autoboot.txt provides the A/B selection.)

Kernel: Built from raspberrypi/linux fork with bcm2711_defconfig (Pi 4) or bcm2712_defconfig (Pi 5). RPi-patched, includes BCM-specific drivers (sdhci-iproc, bcm2835-mmc, GPIO, mailbox).

Storage: SD card via sdhci-iproc driver — requires kernel-built DTBs to match the kernel binary.

Disk image format: MBR with autoboot.txt A/B redirect:

• Part 1: Boot/Control (FAT32, firmware + fallback kernel)
• Part 2: Boot A (FAT32, kernel + DTBs + initramfs)
• Part 3: Boot B (FAT32, same as A initially)
• Part 4: Data (ext4)

File-by-file ownership

Shared substrate (used by both tracks)

Path	Why shared
init/ (all of it)	Boot is identical post-kernel — same staged init, same persistent mount, same KubeSolo launch
cloud-init/	Arch-agnostic Go binary
update/	Arch-agnostic Go binary; bootenv abstraction handles GRUB vs RPi-autoboot variants
build/scripts/inject-kubesolo.sh	Single script; switches LIB_ARCH / LD_SO based on TARGET_ARCH
build/scripts/extract-core.sh	Single script; arm64 branch uses piCore64 userland (arch-agnostic BusyBox)
build/config/modules-arm64.list	Already generic — no BCM-specific modules; works in QEMU virt, AWS Graviton, and RPi
build/config/rpi-kernel-config.fragment	Misnamed. Contents (cgroup, namespaces, netfilter, AppArmor) are arch-agnostic. Will be renamed kernel-container.fragment in Phase 2 and applied to x86, generic-ARM64, and RPi kernels alike.
hack/dev-vm-arm64.sh	Uses -machine virt + virtio — generic, not RPi-specific
test/qemu/test-boot-arm64.sh	Same as above

Generic ARM64 only (to be created in Phases 2–3)

Path	Purpose
build/scripts/build-kernel-arm64.sh (rewritten in Phase 2)	Build mainline kernel.org LTS from defconfig + shared fragment + arm64-virt enables (VIRTIO_BLK, EFI_STUB). Replaces the existing RPi-flavoured script of the same name.
build/scripts/create-disk-image-arm64.sh (new in Phase 3)	Build UEFI-bootable raw disk image (GPT + System A/B + Data) using grub-efi-arm64. Or fold into existing create-disk-image.sh with an arch parameter.
build/cache/kernel-arm64-generic/	Build output for mainline ARM64 kernel — keep separate from RPi-kernel cache.

Raspberry Pi only (to be renamed/reorganised in Phase 2)

Path	Purpose
build/scripts/build-kernel-rpi.sh (renamed from build-kernel-arm64.sh)	Build kernel from raspberrypi/linux with bcm2711_defconfig + shared fragment + RPi-specific overrides.
build/scripts/create-rpi-image.sh	Build SD card image (MBR + autoboot.txt + firmware blobs + DTBs). Already correctly scoped.
build/scripts/fetch-rpi-firmware.sh	Download VideoCore firmware blobs from raspberrypi/firmware. Already correctly scoped.
build/config/rpi-kernel-overrides.fragment (new, Phase 2)	Pi-specific kernel config knobs (DMA, audio off, etc.) layered on top of the shared container fragment.
build/cache/custom-kernel-rpi/ (renamed from custom-kernel-arm64/)	Build output for RPi kernel — DTBs, modules, Image.
versions.env keys: RPI_KERNEL_BRANCH, RPI_KERNEL_REPO, RPI_FIRMWARE_TAG, RPI_FIRMWARE_URL, PICORE_*	Already correctly named.

Make targets

Target	Track
make iso	x86_64
make disk-image	x86_64
make kernel	x86_64
make kernel-arm64 (Phase 2: now builds mainline)	Generic ARM64
make rootfs-arm64	Generic ARM64 (and reusable for RPi rootfs)
make disk-image-arm64 (Phase 3: new)	Generic ARM64
make kernel-rpi (Phase 2: renamed from former kernel-arm64)	RPi
make rpi-image	RPi

Why two tracks, not one

The RPi boot path is fundamentally different from generic ARM64:

• No UEFI. RPi boots through a multi-stage firmware chain that ends with config.txt parsing and direct kernel load. UEFI/GRUB is not an option without third-party firmware (which has its own bugs).
• DTB required. RPi kernel needs a device tree blob matching the kernel binary; generic ARM64 under UEFI uses ACPI or self-describing virtio.
• Custom drivers. SD card (sdhci-iproc), GPIO, mailbox interfaces require RPi-patched kernel sources. Mainline support exists but lags behind the raspberrypi/linux fork for new boards.
• A/B selection mechanism. RPi uses autoboot.txt + EEPROM cooperation; generic ARM64 uses GRUB's boot_default/boot_counter envvars (same as x86_64).

Trying to unify into a single track would force compromises in both. Two tracks sharing the post-kernel substrate (init, cloud-init, update agent) gives us the best of both: code reuse where it makes sense, divergence only where the hardware demands it.

Migration plan

This document is descriptive of the target v0.3.0 layout. The current code (as of v0.2.0) has:

• build/scripts/build-kernel-arm64.sh building the RPi kernel (will be renamed in Phase 2).
• build/config/rpi-kernel-config.fragment containing generic configs (will be renamed in Phase 2).
• No generic ARM64 kernel script (will be created in Phase 2).
• No generic ARM64 disk image script (will be created in Phase 3).

Phases 2 and 3 of the v0.3.0 plan execute the migration.