From 19b99cf101a23be2cd8eba8b3801eb8c87e978ff Mon Sep 17 00:00:00 2001 From: Adolfo Delorenzo Date: Thu, 14 May 2026 10:02:29 -0600 Subject: [PATCH] docs: define generic ARM64 vs RPi build-track architecture Phase 1 audit finding: existing ARM64 build code is mostly already generic. Only build-kernel-arm64.sh and rpi-kernel-config.fragment are misnamed (the former is RPi-only, the latter is actually arch-agnostic). The QEMU virt harness, modules-arm64.list, extract-core arm64 branch, and inject-kubesolo arm64 branch are all generic. This document records the target two-track layout for v0.3.0: - Generic ARM64: mainline kernel, UEFI, GRUB, virtio, GPT 4-part image - Raspberry Pi: raspberrypi/linux fork, autoboot.txt, MBR 4-part image - Shared: init, cloud-init, update agent, modules list, kernel-container fragment Phases 2 and 3 will execute the migration (rename build-kernel-arm64.sh -> build-kernel-rpi.sh, write a new mainline build-kernel-arm64.sh, etc.). Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/arm64-architecture.md | 124 +++++++++++++++++++++++++++++++++++++ 1 file changed, 124 insertions(+) create mode 100644 docs/arm64-architecture.md diff --git a/docs/arm64-architecture.md b/docs/arm64-architecture.md new file mode 100644 index 0000000..3b773d8 --- /dev/null +++ b/docs/arm64-architecture.md @@ -0,0 +1,124 @@ +# ARM64 Build Architecture + +KubeSolo OS supports ARM64 via two distinct build tracks. This document defines the +split, lists which files belong to each track, and identifies the shared substrate. + +## The two tracks + +### Generic ARM64 (UEFI / virtio / GRUB) + +**Target:** Any UEFI-compliant ARM64 host — Ampere/Graviton VMs, generic ARM64 +servers, `qemu-system-aarch64 -machine virt`, future SBCs that boot via UEFI. + +**Boot path:** UEFI firmware → GRUB-EFI → kernel + initramfs → KubeSolo init. + +**Kernel:** Mainline Linux (kernel.org LTS), built from `defconfig` + shared +container-config fragment. + +**Storage:** virtio-blk / NVMe / SATA — detected and probed by mainline drivers. + +**Disk image format:** GPT, identical 4-partition layout to x86_64 (EFI + System A ++ System B + Data). + +### Raspberry Pi ARM64 + +**Target:** Raspberry Pi 4 and 5 specifically. + +**Boot path:** RPi EEPROM → VideoCore firmware (`start4.elf`) → `config.txt` → +kernel + DTB + initramfs → KubeSolo init. (No UEFI, no GRUB — `autoboot.txt` +provides the A/B selection.) + +**Kernel:** Built from `raspberrypi/linux` fork with `bcm2711_defconfig` +(Pi 4) or `bcm2712_defconfig` (Pi 5). RPi-patched, includes BCM-specific drivers +(sdhci-iproc, bcm2835-mmc, GPIO, mailbox). + +**Storage:** SD card via `sdhci-iproc` driver — requires kernel-built DTBs to match +the kernel binary. + +**Disk image format:** MBR with `autoboot.txt` A/B redirect: +- Part 1: Boot/Control (FAT32, firmware + fallback kernel) +- Part 2: Boot A (FAT32, kernel + DTBs + initramfs) +- Part 3: Boot B (FAT32, same as A initially) +- Part 4: Data (ext4) + +## File-by-file ownership + +### Shared substrate (used by both tracks) + +| Path | Why shared | +|------|------------| +| `init/` (all of it) | Boot is identical post-kernel — same staged init, same persistent mount, same KubeSolo launch | +| `cloud-init/` | Arch-agnostic Go binary | +| `update/` | Arch-agnostic Go binary; bootenv abstraction handles GRUB vs RPi-autoboot variants | +| `build/scripts/inject-kubesolo.sh` | Single script; switches `LIB_ARCH` / `LD_SO` based on `TARGET_ARCH` | +| `build/scripts/extract-core.sh` | Single script; arm64 branch uses piCore64 userland (arch-agnostic BusyBox) | +| `build/config/modules-arm64.list` | Already generic — no BCM-specific modules; works in QEMU virt, AWS Graviton, and RPi | +| `build/config/rpi-kernel-config.fragment` | **Misnamed.** Contents (cgroup, namespaces, netfilter, AppArmor) are arch-agnostic. Will be renamed `kernel-container.fragment` in Phase 2 and applied to x86, generic-ARM64, and RPi kernels alike. | +| `hack/dev-vm-arm64.sh` | Uses `-machine virt` + virtio — generic, not RPi-specific | +| `test/qemu/test-boot-arm64.sh` | Same as above | + +### Generic ARM64 only (to be created in Phases 2–3) + +| Path | Purpose | +|------|---------| +| `build/scripts/build-kernel-arm64.sh` *(rewritten in Phase 2)* | Build mainline kernel.org LTS from `defconfig` + shared fragment + arm64-virt enables (`VIRTIO_BLK`, `EFI_STUB`). Replaces the existing RPi-flavoured script of the same name. | +| `build/scripts/create-disk-image-arm64.sh` *(new in Phase 3)* | Build UEFI-bootable raw disk image (GPT + System A/B + Data) using `grub-efi-arm64`. Or fold into existing `create-disk-image.sh` with an arch parameter. | +| `build/cache/kernel-arm64-generic/` | Build output for mainline ARM64 kernel — keep separate from RPi-kernel cache. | + +### Raspberry Pi only (to be renamed/reorganised in Phase 2) + +| Path | Purpose | +|------|---------| +| `build/scripts/build-kernel-rpi.sh` *(renamed from `build-kernel-arm64.sh`)* | Build kernel from `raspberrypi/linux` with `bcm2711_defconfig` + shared fragment + RPi-specific overrides. | +| `build/scripts/create-rpi-image.sh` | Build SD card image (MBR + autoboot.txt + firmware blobs + DTBs). Already correctly scoped. | +| `build/scripts/fetch-rpi-firmware.sh` | Download VideoCore firmware blobs from `raspberrypi/firmware`. Already correctly scoped. | +| `build/config/rpi-kernel-overrides.fragment` *(new, Phase 2)* | Pi-specific kernel config knobs (DMA, audio off, etc.) layered on top of the shared container fragment. | +| `build/cache/custom-kernel-rpi/` *(renamed from `custom-kernel-arm64/`)* | Build output for RPi kernel — DTBs, modules, Image. | +| `versions.env` keys: `RPI_KERNEL_BRANCH`, `RPI_KERNEL_REPO`, `RPI_FIRMWARE_TAG`, `RPI_FIRMWARE_URL`, `PICORE_*` | Already correctly named. | + +## Make targets + +| Target | Track | +|--------|-------| +| `make iso` | x86_64 | +| `make disk-image` | x86_64 | +| `make kernel` | x86_64 | +| `make kernel-arm64` *(Phase 2: now builds mainline)* | Generic ARM64 | +| `make rootfs-arm64` | Generic ARM64 (and reusable for RPi rootfs) | +| `make disk-image-arm64` *(Phase 3: new)* | Generic ARM64 | +| `make kernel-rpi` *(Phase 2: renamed from former kernel-arm64)* | RPi | +| `make rpi-image` | RPi | + +## Why two tracks, not one + +The RPi boot path is fundamentally different from generic ARM64: + +- **No UEFI.** RPi boots through a multi-stage firmware chain that ends with + `config.txt` parsing and direct kernel load. UEFI/GRUB is not an option without + third-party firmware (which has its own bugs). +- **DTB required.** RPi kernel needs a device tree blob matching the kernel binary; + generic ARM64 under UEFI uses ACPI or self-describing virtio. +- **Custom drivers.** SD card (sdhci-iproc), GPIO, mailbox interfaces require + RPi-patched kernel sources. Mainline support exists but lags behind the + raspberrypi/linux fork for new boards. +- **A/B selection mechanism.** RPi uses `autoboot.txt` + EEPROM cooperation; generic + ARM64 uses GRUB's `boot_default`/`boot_counter` envvars (same as x86_64). + +Trying to unify into a single track would force compromises in both. Two tracks +sharing the post-kernel substrate (init, cloud-init, update agent) gives us the best +of both: code reuse where it makes sense, divergence only where the hardware demands +it. + +## Migration plan + +This document is descriptive of the **target** v0.3.0 layout. The current code +(as of v0.2.0) has: + +- `build/scripts/build-kernel-arm64.sh` building the RPi kernel (will be renamed in + Phase 2). +- `build/config/rpi-kernel-config.fragment` containing generic configs (will be + renamed in Phase 2). +- No generic ARM64 kernel script (will be created in Phase 2). +- No generic ARM64 disk image script (will be created in Phase 3). + +Phases 2 and 3 of the v0.3.0 plan execute the migration.