3bcf2e1 added nft_numgen / nft_hash / nft_limit / nft_log to both module
lists but in a format the inject parser doesn't handle:
nft_numgen # numgen random/inc mod N vmap — Service endpoint LB
The parser's only comment skip is `case "$mod" in \#*|"") continue ;;`
which matches lines STARTING with #, not lines with inline #-comments.
So each new line was passed to modprobe verbatim as a single (invalid)
module name, modprobe returned nonzero, and the .ko never made it into
the initramfs. ls'ing the rootfs after the rootfs rebuild confirmed:
ls .../lib/modules/*/kernel/net/netfilter/ | grep nft_numgen
<empty>
Two changes:
1. Strip inline comments from the new entries in modules.list and
modules-arm64.list. Each module name on its own line, matching the
convention the rest of the file uses.
2. Harden the parser in inject-kubesolo.sh to handle "name # comment"
regardless. Single-line tweak: `mod="${mod%%#*}"` before the
continue check. Prevents a future contributor's inline doc from
silently dropping a module the same way.
After rebuilding the rootfs on the Odroid (no kernel rebuild needed —
this is a rootfs-only change), the four .ko files should appear at
build/rootfs-work/rootfs/lib/modules/*/kernel/net/netfilter/.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
After 31eee77 added CONFIG_NFT_NUMGEN=m and friends to the kernel
fragment, the rebuilt kernel does include nft_numgen.ko on disk in
build/cache/kernel-arm64-generic/modules/. But the runtime kernel
doesn't load it, and kube-proxy keeps failing with the same
"No such file or directory" pointing at `numgen` as before the
kernel rebuild.
Root cause is the boot-stage-vs-lockdown ordering combined with
inject-kubesolo.sh's selective module copy:
1. inject-kubesolo.sh ships modules listed in modules.list /
modules-arm64.list plus their transitive deps. nft_numgen wasn't
in either list, so its .ko is in the kernel build cache but
never makes it into the initramfs.
2. Stage 30 (kernel-modules) only modprobes from the same list, so
it wouldn't load nft_numgen even if the .ko were present.
3. Stage 85 (security-lockdown) writes 1 to
/proc/sys/kernel/modules_disabled, blocking any further module
loads — including the lazy request_module() that nftables would
otherwise do when kube-proxy first uses the `numgen` expression.
The kernel-side fix (=m in the fragment) is necessary but not
sufficient: we have to ship + load these in stage 30, before lockdown.
Add nft_numgen, nft_hash, nft_limit, nft_log to BOTH modules.list
(x86) and modules-arm64.list. Same justification on x86 — KubeSolo's
nftables kube-proxy backend uses numgen regardless of arch, we just
haven't exercised it on x86 since v0.2 deployments stuck with the
older iptables-restore backend.
After this lands on the Odroid:
sudo make rootfs-arm64 disk-image-arm64 # kernel cached, rootfs only
# no kernel rebuild needed; this is a rootfs-only change
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Security hardening: bind kubeconfig server to localhost, mount hardening
(noexec/nosuid/nodev on tmpfs), sysctl network hardening, kernel module
loading lock after boot, SHA256 checksum verification for downloads,
kernel AppArmor + Audit support, complain-mode AppArmor profiles for
containerd and kubelet, and security integration test.
ARM64 Raspberry Pi support: piCore64 base extraction, RPi kernel build
from raspberrypi/linux fork, RPi firmware fetch, SD card image with 4-
partition GPT and tryboot A/B mechanism, BootEnv Go interface abstracting
GRUB vs RPi boot environments, architecture-aware build scripts, QEMU
aarch64 dev VM and boot test.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
