From 3bcf2e115f164975b0dde39e99c955ab9fa55e54 Mon Sep 17 00:00:00 2001 From: Adolfo Delorenzo Date: Fri, 15 May 2026 14:25:11 -0600 Subject: [PATCH] fix(modules): ship and load nft_numgen/hash/limit/log at boot MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit After 31eee77 added CONFIG_NFT_NUMGEN=m and friends to the kernel fragment, the rebuilt kernel does include nft_numgen.ko on disk in build/cache/kernel-arm64-generic/modules/. But the runtime kernel doesn't load it, and kube-proxy keeps failing with the same "No such file or directory" pointing at `numgen` as before the kernel rebuild. Root cause is the boot-stage-vs-lockdown ordering combined with inject-kubesolo.sh's selective module copy: 1. inject-kubesolo.sh ships modules listed in modules.list / modules-arm64.list plus their transitive deps. nft_numgen wasn't in either list, so its .ko is in the kernel build cache but never makes it into the initramfs. 2. Stage 30 (kernel-modules) only modprobes from the same list, so it wouldn't load nft_numgen even if the .ko were present. 3. Stage 85 (security-lockdown) writes 1 to /proc/sys/kernel/modules_disabled, blocking any further module loads — including the lazy request_module() that nftables would otherwise do when kube-proxy first uses the `numgen` expression. The kernel-side fix (=m in the fragment) is necessary but not sufficient: we have to ship + load these in stage 30, before lockdown. Add nft_numgen, nft_hash, nft_limit, nft_log to BOTH modules.list (x86) and modules-arm64.list. Same justification on x86 — KubeSolo's nftables kube-proxy backend uses numgen regardless of arch, we just haven't exercised it on x86 since v0.2 deployments stuck with the older iptables-restore backend. After this lands on the Odroid: sudo make rootfs-arm64 disk-image-arm64 # kernel cached, rootfs only # no kernel rebuild needed; this is a rootfs-only change Co-Authored-By: Claude Opus 4.7 (1M context) --- build/config/modules-arm64.list | 9 +++++++++ build/config/modules.list | 7 +++++++ 2 files changed, 16 insertions(+) diff --git a/build/config/modules-arm64.list b/build/config/modules-arm64.list index cc280e2..92ae51f 100644 --- a/build/config/modules-arm64.list +++ b/build/config/modules-arm64.list @@ -56,6 +56,15 @@ nft_fib nft_fib_ipv4 nft_fib_ipv6 +# nft expressions used by the Kubernetes 1.34+ nftables kube-proxy backend. +# Loading these at boot (stage 30) is mandatory because stage 85 sets +# kernel.modules_disabled=1, which would otherwise block kube-proxy from +# auto-loading them on first rule install. +nft_numgen # numgen random/inc mod N vmap — Service endpoint LB +nft_hash # hash — consistent-hash LB for sessionAffinity=ClientIP +nft_limit # rate-limit expression +nft_log # log expression + # Reject targets (used by kube-proxy iptables-restore rules) nf_reject_ipv4 nf_reject_ipv6 diff --git a/build/config/modules.list b/build/config/modules.list index 86a4608..983728b 100644 --- a/build/config/modules.list +++ b/build/config/modules.list @@ -54,6 +54,13 @@ nft_fib nft_fib_ipv4 nft_fib_ipv6 +# nft expressions used by the Kubernetes 1.34+ nftables kube-proxy backend. +# Must be loaded at stage 30 because stage 85 sets modules_disabled=1. +nft_numgen # numgen random/inc mod N vmap — Service endpoint LB +nft_hash # hash — consistent-hash LB for sessionAffinity=ClientIP +nft_limit # rate-limit expression +nft_log # log expression + # Reject targets (used by kube-proxy iptables-restore rules) nf_reject_ipv4 nf_reject_ipv6