3 Commits

Author	SHA1	Message	Date
Adolfo Delorenzo	31eee77397	fix(kernel): enable nftables NUMGEN + HASH + helper expressions ... Fourth round of the v0.3 nftables-on-arm64 debug saga. After the NF_TABLES_IPV4 family fix from 7e46f8f, KubeSolo + containerd + a CoreDNS pod all reach Running state, but kube-proxy fails to install Service rules: add rule ip kube-proxy service-2QRHZV4L-default/kubernetes/tcp/https numgen random mod 1 vmap { 0 : goto ... } ^^^^^^^^^^^^^^^^^^^ Error: Could not process rule: No such file or directory The caret points at `numgen random mod 1`. That's the nftables NUMGEN expression — kube-proxy's nftables backend uses it for random endpoint load-balancing across Service endpoints. Without CONFIG_NFT_NUMGEN compiled into the kernel, every Service sync fails and kube-dns / any ClusterIP is unreachable. Cascade: kube-proxy sync fail -> kube-dns Service has no DNAT -> CoreDNS readiness probe never goes Ready -> KubeSolo's coredns deploy step times out after 15 attempts -> FTL -> kernel panic. Fix: add NFT_NUMGEN to kernel-container.fragment, plus the small family of expression modules kube-proxy and CNI plugins commonly use so we don't repeat this debug loop for the next missing one: CONFIG_NFT_NUMGEN=m random / inc LB CONFIG_NFT_HASH=m consistent-hash LB (sessionAffinity=ClientIP) CONFIG_NFT_OBJREF=m named objects (counters, quotas) refs in rules CONFIG_NFT_LIMIT=m rate-limit expression CONFIG_NFT_LOG=m log expression (used by some CNI debug rules) All =m so init's stage-30 loads them from modules.list / modules-arm64.list alongside the existing nft_nat / nft_masq / nft_compat. This needs another kernel rebuild (rm -rf build/cache/kernel-arm64-generic, sudo make kernel-arm64) on the Odroid. After that we should have a fully working KubeSolo OS v0.3 on ARM64 generic — at which point the only thing left is to tag v0.3.1 and verify the rewritten release.yaml workflow publishes both arches automatically. Note on runc-PATH log noise: containerd-shim-runc-v2 -info probes for runc in $PATH and fails because KubeSolo's runc lives at /var/lib/kubesolo/containerd/runc. This is cosmetic — actual container creation uses an absolute path from the containerd config and works fine (CoreDNS container did start successfully). Will polish in v0.3.2. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-15 11:48:43 -06:00
Adolfo Delorenzo	7e46f8fdc2	fix(kernel): enable nftables address-family handlers ... Third KubeSolo crash from the QEMU validation loop: nft add table ip kubesolo-masq: exit status 1 Error: Could not process rule: Operation not supported That's EOPNOTSUPP from netlink. nf_tables core is loaded (the binary even runs cleanly now after the previous dual-glibc fix), but no address families are registered with it — so any `nft add table ip ...`, `add table inet ...`, etc. is rejected. In modern Linux (5.x / 6.x) the nftables address families are gated by separate BOOL Kconfigs: CONFIG_NF_TABLES_IPV4 "ip" family CONFIG_NF_TABLES_IPV6 "ip6" family CONFIG_NF_TABLES_INET "inet" family (both) CONFIG_NF_TABLES_NETDEV "netdev" family These are bool (not tristate) — they must be built into the kernel; no module to load at runtime. Our shared kernel-container.fragment had CONFIG_NF_TABLES=m (the core) but none of the family Kconfigs, and the arm64 defconfig leaves them off. Fix: enable all four families as =y in kernel-container.fragment. Also pin the NFT expression modules KubeSolo v1.1.4+'s masquerade ruleset depends on (NFT_NAT, NFT_MASQ, NFT_CT, NFT_REDIR, NFT_REJECT, NFT_REJECT_INET, NFT_COMPAT, NFT_FIB + FIB_IPV4/6) as =m — they're already in modules-arm64.list / modules.list and get modprobed at boot, this just makes sure olddefconfig doesn't strip them when applied on top of a minimal defconfig. NF_NAT_MASQUERADE pinned =y because NFT_MASQ select-depends on it; on some kernels it would get auto-selected, on others it gets dropped by olddefconfig if not pinned. This change requires a kernel rebuild — the configs are bool / module defs, not runtime knobs. On the Odroid: rm -rf build/cache/kernel-arm64-generic sudo make kernel-arm64 # ~30-60 min from scratch sudo make rootfs-arm64 disk-image-arm64 x86 needs the same treatment when we cut v0.3.1. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-15 08:55:41 -06:00
Adolfo Delorenzo	d51618badb	build: separate generic ARM64 from Raspberry Pi kernel builds ... Splits the ARM64 build into two tracks per docs/arm64-architecture.md: Generic ARM64 (mainline kernel.org, UEFI, virtio, GRUB): - New build/scripts/build-kernel-arm64.sh builds mainline LTS (6.12.x by default) from arm64 defconfig + shared container fragment + arm64-virt enables (VIRTIO_*, EFI_STUB, NVMe). Output: build/cache/kernel-arm64-generic/. - New Makefile targets: kernel-arm64, rootfs-arm64 (now consumes the mainline kernel modules via TARGET_VARIANT=generic). - versions.env: pin MAINLINE_KERNEL_VERSION=6.12.10, declare cdn.kernel.org URL and SHA256 placeholder. Raspberry Pi (raspberrypi/linux fork, custom DTBs, autoboot.txt): - build-kernel-arm64.sh (RPi-flavoured) renamed to build-kernel-rpi.sh; cache dir renamed from custom-kernel-arm64 to custom-kernel-rpi. - New Makefile targets: kernel-rpi, rootfs-arm64-rpi (uses TARGET_VARIANT=rpi). - rpi-image now depends on rootfs-arm64-rpi + kernel-rpi instead of the generic rootfs-arm64. - create-rpi-image.sh + inject-kubesolo.sh updated to reference the new cache path. inject-kubesolo.sh now takes a TARGET_VARIANT env var (rpi|generic) to select which ARM64 kernel modules to consume. Shared substrate: - rpi-kernel-config.fragment renamed to kernel-container.fragment. The contents were never RPi-specific (cgroup, namespaces, AppArmor, netfilter) — just misnamed. Extended with extra subsystem disables (KVM, WLAN, CFG80211, INFINIBAND, PCMCIA, HAMRADIO, ISDN, ATM, INPUT_JOYSTICK, INPUT_TABLET, FPGA) and CONFIG_LSM=lockdown,yama,apparmor. - build-kernel.sh (x86) refactored to apply the shared fragment via a generic apply_fragment function (two-pass for the TC stock config security dance), killing ~50 lines of inline config duplication. Note: rename detection shows build-kernel-arm64.sh as 'modified' because the new file at that path is the mainline build, while the old RPi-flavoured content lives in build-kernel-rpi.sh (which appears as a new file). The git log for build-kernel-rpi.sh is empty; the RPi history is preserved at the original path until this commit. No actual kernel build runs in this commit — that's Phase 3 work. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-14 10:30:11 -06:00