Adolfo Delorenzo
3bcf2e115f
Some checks failed
ARM64 Build / Build generic ARM64 disk image (push) Failing after 6s
CI / Go Tests (push) Successful in 2m12s
CI / Shellcheck (push) Successful in 55s
CI / Build Go Binaries (amd64, linux, linux-amd64) (push) Successful in 1m48s
CI / Build Go Binaries (arm64, linux, linux-arm64) (push) Successful in 1m35s
After 31eee77 added CONFIG_NFT_NUMGEN=m and friends to the kernel
fragment, the rebuilt kernel does include nft_numgen.ko on disk in
build/cache/kernel-arm64-generic/modules/. But the runtime kernel
doesn't load it, and kube-proxy keeps failing with the same
"No such file or directory" pointing at `numgen` as before the
kernel rebuild.
Root cause is the boot-stage-vs-lockdown ordering combined with
inject-kubesolo.sh's selective module copy:
1. inject-kubesolo.sh ships modules listed in modules.list /
modules-arm64.list plus their transitive deps. nft_numgen wasn't
in either list, so its .ko is in the kernel build cache but
never makes it into the initramfs.
2. Stage 30 (kernel-modules) only modprobes from the same list, so
it wouldn't load nft_numgen even if the .ko were present.
3. Stage 85 (security-lockdown) writes 1 to
/proc/sys/kernel/modules_disabled, blocking any further module
loads — including the lazy request_module() that nftables would
otherwise do when kube-proxy first uses the `numgen` expression.
The kernel-side fix (=m in the fragment) is necessary but not
sufficient: we have to ship + load these in stage 30, before lockdown.
Add nft_numgen, nft_hash, nft_limit, nft_log to BOTH modules.list
(x86) and modules-arm64.list. Same justification on x86 — KubeSolo's
nftables kube-proxy backend uses numgen regardless of arch, we just
haven't exercised it on x86 since v0.2 deployments stuck with the
older iptables-restore backend.
After this lands on the Odroid:
sudo make rootfs-arm64 disk-image-arm64 # kernel cached, rootfs only
# no kernel rebuild needed; this is a rootfs-only change
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
91 lines
2.1 KiB
Plaintext
91 lines
2.1 KiB
Plaintext
# Kernel modules loaded at boot by init (ARM64 / Raspberry Pi)
|
|
# One module per line. Lines starting with # are ignored.
|
|
# Modules are loaded in order listed — dependencies must come first.
|
|
|
|
# Network device drivers (loaded early so interfaces are available)
|
|
# Note: no e1000/e1000e on ARM64 — those are x86 Intel NIC drivers
|
|
virtio_net
|
|
|
|
# Virtio support (for QEMU VMs — block, entropy)
|
|
virtio_blk
|
|
virtio_rng
|
|
|
|
# Raspberry Pi specific (USB Ethernet on Pi 4 is built-in, no module needed)
|
|
# Pi 5 uses PCIe ethernet, also typically built-in
|
|
|
|
# Filesystem — overlay (required for containerd)
|
|
overlay
|
|
|
|
# Netfilter dependencies (must load before conntrack)
|
|
nf_defrag_ipv4
|
|
nf_defrag_ipv6
|
|
|
|
# Netfilter / connection tracking (required for kube-proxy)
|
|
nf_conntrack
|
|
nf_nat
|
|
nf_conntrack_netlink
|
|
|
|
# nftables (modern iptables backend)
|
|
nf_tables
|
|
nft_compat
|
|
nft_chain_nat
|
|
nft_ct
|
|
nft_masq
|
|
nft_nat
|
|
nft_redir
|
|
|
|
# Netfilter xt match/target modules (used by kube-proxy iptables rules via nft_compat)
|
|
xt_conntrack
|
|
xt_MASQUERADE
|
|
xt_mark
|
|
xt_comment
|
|
xt_multiport
|
|
xt_nat
|
|
xt_addrtype
|
|
xt_connmark
|
|
xt_REDIRECT
|
|
xt_recent
|
|
xt_statistic
|
|
xt_set
|
|
|
|
# nft extras (reject, fib — used by kube-proxy nf_tables rules)
|
|
nft_reject
|
|
nft_reject_ipv4
|
|
nft_reject_ipv6
|
|
nft_fib
|
|
nft_fib_ipv4
|
|
nft_fib_ipv6
|
|
|
|
# nft expressions used by the Kubernetes 1.34+ nftables kube-proxy backend.
|
|
# Loading these at boot (stage 30) is mandatory because stage 85 sets
|
|
# kernel.modules_disabled=1, which would otherwise block kube-proxy from
|
|
# auto-loading them on first rule install.
|
|
nft_numgen # numgen random/inc mod N vmap — Service endpoint LB
|
|
nft_hash # hash — consistent-hash LB for sessionAffinity=ClientIP
|
|
nft_limit # rate-limit expression
|
|
nft_log # log expression
|
|
|
|
# Reject targets (used by kube-proxy iptables-restore rules)
|
|
nf_reject_ipv4
|
|
nf_reject_ipv6
|
|
ipt_REJECT
|
|
ip6t_REJECT
|
|
|
|
# nfacct extension (kube-proxy probes for it)
|
|
xt_nfacct
|
|
|
|
# Networking — bridge and netfilter (required for K8s pod networking)
|
|
# Load order: llc → stp → bridge → br_netfilter
|
|
llc
|
|
stp
|
|
bridge
|
|
br_netfilter
|
|
veth
|
|
vxlan
|
|
|
|
# IPVS — useful for kube-proxy IPVS mode and CNI plugins
|
|
ip_vs
|
|
ip_vs_rr
|
|
ip_vs_wrr
|
|
ip_vs_sh
|