adelorenzo/kubesolo-os
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases 4 Wiki Activity
Files
65938d6d04aa2d14194d485cafaaaa12f508855a
kubesolo-os/docs/arm64-architecture.md
Adolfo Delorenzo 19b99cf101 docs: define generic ARM64 vs RPi build-track architecture 
Phase 1 audit finding: existing ARM64 build code is mostly already generic.
Only build-kernel-arm64.sh and rpi-kernel-config.fragment are misnamed (the
former is RPi-only, the latter is actually arch-agnostic). The QEMU virt
harness, modules-arm64.list, extract-core arm64 branch, and inject-kubesolo
arm64 branch are all generic.

This document records the target two-track layout for v0.3.0:
- Generic ARM64: mainline kernel, UEFI, GRUB, virtio, GPT 4-part image
- Raspberry Pi: raspberrypi/linux fork, autoboot.txt, MBR 4-part image
- Shared: init, cloud-init, update agent, modules list, kernel-container fragment

Phases 2 and 3 will execute the migration (rename build-kernel-arm64.sh ->
build-kernel-rpi.sh, write a new mainline build-kernel-arm64.sh, etc.).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-14 10:02:29 -06:00

125 lines
6.3 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# ARM64 Build Architecture
KubeSolo OS supports ARM64 via two distinct build tracks. This document defines the
split, lists which files belong to each track, and identifies the shared substrate.
## The two tracks
### Generic ARM64 (UEFI / virtio / GRUB)
**Target:** Any UEFI-compliant ARM64 host — Ampere/Graviton VMs, generic ARM64
servers, `qemu-system-aarch64 -machine virt`, future SBCs that boot via UEFI.
**Boot path:** UEFI firmware → GRUB-EFI → kernel + initramfs → KubeSolo init.
**Kernel:** Mainline Linux (kernel.org LTS), built from `defconfig` + shared
container-config fragment.
**Storage:** virtio-blk / NVMe / SATA — detected and probed by mainline drivers.
**Disk image format:** GPT, identical 4-partition layout to x86_64 (EFI + System A
+ System B + Data).
### Raspberry Pi ARM64
**Target:** Raspberry Pi 4 and 5 specifically.
**Boot path:** RPi EEPROM → VideoCore firmware (`start4.elf`) → `config.txt` →
kernel + DTB + initramfs → KubeSolo init. (No UEFI, no GRUB — `autoboot.txt`
provides the A/B selection.)
**Kernel:** Built from `raspberrypi/linux` fork with `bcm2711_defconfig`
(Pi 4) or `bcm2712_defconfig` (Pi 5). RPi-patched, includes BCM-specific drivers
(sdhci-iproc, bcm2835-mmc, GPIO, mailbox).
**Storage:** SD card via `sdhci-iproc` driver — requires kernel-built DTBs to match
the kernel binary.
**Disk image format:** MBR with `autoboot.txt` A/B redirect:
- Part 1: Boot/Control (FAT32, firmware + fallback kernel)
- Part 2: Boot A (FAT32, kernel + DTBs + initramfs)
- Part 3: Boot B (FAT32, same as A initially)
- Part 4: Data (ext4)
## File-by-file ownership
### Shared substrate (used by both tracks)
| Path | Why shared |
|------|------------|
| `init/` (all of it) | Boot is identical post-kernel — same staged init, same persistent mount, same KubeSolo launch |
| `cloud-init/` | Arch-agnostic Go binary |
| `update/` | Arch-agnostic Go binary; bootenv abstraction handles GRUB vs RPi-autoboot variants |
| `build/scripts/inject-kubesolo.sh` | Single script; switches `LIB_ARCH` / `LD_SO` based on `TARGET_ARCH` |
| `build/scripts/extract-core.sh` | Single script; arm64 branch uses piCore64 userland (arch-agnostic BusyBox) |
| `build/config/modules-arm64.list` | Already generic — no BCM-specific modules; works in QEMU virt, AWS Graviton, and RPi |
| `build/config/rpi-kernel-config.fragment` | **Misnamed.** Contents (cgroup, namespaces, netfilter, AppArmor) are arch-agnostic. Will be renamed `kernel-container.fragment` in Phase 2 and applied to x86, generic-ARM64, and RPi kernels alike. |
| `hack/dev-vm-arm64.sh` | Uses `-machine virt` + virtio — generic, not RPi-specific |
| `test/qemu/test-boot-arm64.sh` | Same as above |
### Generic ARM64 only (to be created in Phases 2–3)
| Path | Purpose |
|------|---------|
| `build/scripts/build-kernel-arm64.sh` *(rewritten in Phase 2)* | Build mainline kernel.org LTS from `defconfig` + shared fragment + arm64-virt enables (`VIRTIO_BLK`, `EFI_STUB`). Replaces the existing RPi-flavoured script of the same name. |
| `build/scripts/create-disk-image-arm64.sh` *(new in Phase 3)* | Build UEFI-bootable raw disk image (GPT + System A/B + Data) using `grub-efi-arm64`. Or fold into existing `create-disk-image.sh` with an arch parameter. |
| `build/cache/kernel-arm64-generic/` | Build output for mainline ARM64 kernel — keep separate from RPi-kernel cache. |
### Raspberry Pi only (to be renamed/reorganised in Phase 2)
| Path | Purpose |
|------|---------|
| `build/scripts/build-kernel-rpi.sh` *(renamed from `build-kernel-arm64.sh`)* | Build kernel from `raspberrypi/linux` with `bcm2711_defconfig` + shared fragment + RPi-specific overrides. |
| `build/scripts/create-rpi-image.sh` | Build SD card image (MBR + autoboot.txt + firmware blobs + DTBs). Already correctly scoped. |
| `build/scripts/fetch-rpi-firmware.sh` | Download VideoCore firmware blobs from `raspberrypi/firmware`. Already correctly scoped. |
| `build/config/rpi-kernel-overrides.fragment` *(new, Phase 2)* | Pi-specific kernel config knobs (DMA, audio off, etc.) layered on top of the shared container fragment. |
| `build/cache/custom-kernel-rpi/` *(renamed from `custom-kernel-arm64/`)* | Build output for RPi kernel — DTBs, modules, Image. |
| `versions.env` keys: `RPI_KERNEL_BRANCH`, `RPI_KERNEL_REPO`, `RPI_FIRMWARE_TAG`, `RPI_FIRMWARE_URL`, `PICORE_*` | Already correctly named. |
## Make targets
| Target | Track |
|--------|-------|
| `make iso` | x86_64 |
| `make disk-image` | x86_64 |
| `make kernel` | x86_64 |
| `make kernel-arm64` *(Phase 2: now builds mainline)* | Generic ARM64 |
| `make rootfs-arm64` | Generic ARM64 (and reusable for RPi rootfs) |
| `make disk-image-arm64` *(Phase 3: new)* | Generic ARM64 |
| `make kernel-rpi` *(Phase 2: renamed from former kernel-arm64)* | RPi |
| `make rpi-image` | RPi |
## Why two tracks, not one
The RPi boot path is fundamentally different from generic ARM64:
- **No UEFI.** RPi boots through a multi-stage firmware chain that ends with
`config.txt` parsing and direct kernel load. UEFI/GRUB is not an option without
third-party firmware (which has its own bugs).
- **DTB required.** RPi kernel needs a device tree blob matching the kernel binary;
generic ARM64 under UEFI uses ACPI or self-describing virtio.
- **Custom drivers.** SD card (sdhci-iproc), GPIO, mailbox interfaces require
RPi-patched kernel sources. Mainline support exists but lags behind the
raspberrypi/linux fork for new boards.
- **A/B selection mechanism.** RPi uses `autoboot.txt` + EEPROM cooperation; generic
ARM64 uses GRUB's `boot_default`/`boot_counter` envvars (same as x86_64).
Trying to unify into a single track would force compromises in both. Two tracks
sharing the post-kernel substrate (init, cloud-init, update agent) gives us the best
of both: code reuse where it makes sense, divergence only where the hardware demands
it.
## Migration plan
This document is descriptive of the **target** v0.3.0 layout. The current code
(as of v0.2.0) has:
- `build/scripts/build-kernel-arm64.sh` building the RPi kernel (will be renamed in
Phase 2).
- `build/config/rpi-kernel-config.fragment` containing generic configs (will be
renamed in Phase 2).
- No generic ARM64 kernel script (will be created in Phase 2).
- No generic ARM64 disk image script (will be created in Phase 3).
Phases 2 and 3 of the v0.3.0 plan execute the migration.
Reference in New Issue View Git Blame Copy Permalink