Adolfo Delorenzo
7e46f8fdc2
Some checks failed
ARM64 Build / Build generic ARM64 disk image (push) Failing after 6s
CI / Go Tests (push) Successful in 2m40s
CI / Shellcheck (push) Successful in 1m39s
CI / Build Go Binaries (amd64, linux, linux-amd64) (push) Failing after 10s
CI / Build Go Binaries (arm64, linux, linux-arm64) (push) Failing after 7s
Third KubeSolo crash from the QEMU validation loop:
nft add table ip kubesolo-masq: exit status 1
Error: Could not process rule: Operation not supported
That's EOPNOTSUPP from netlink. nf_tables core is loaded (the binary
even runs cleanly now after the previous dual-glibc fix), but no address
families are registered with it — so any `nft add table ip ...`,
`add table inet ...`, etc. is rejected.
In modern Linux (5.x / 6.x) the nftables address families are gated by
separate BOOL Kconfigs:
CONFIG_NF_TABLES_IPV4 "ip" family
CONFIG_NF_TABLES_IPV6 "ip6" family
CONFIG_NF_TABLES_INET "inet" family (both)
CONFIG_NF_TABLES_NETDEV "netdev" family
These are bool (not tristate) — they must be built into the kernel; no
module to load at runtime. Our shared kernel-container.fragment had
CONFIG_NF_TABLES=m (the core) but none of the family Kconfigs, and the
arm64 defconfig leaves them off.
Fix: enable all four families as =y in kernel-container.fragment.
Also pin the NFT expression modules KubeSolo v1.1.4+'s masquerade
ruleset depends on (NFT_NAT, NFT_MASQ, NFT_CT, NFT_REDIR, NFT_REJECT,
NFT_REJECT_INET, NFT_COMPAT, NFT_FIB + FIB_IPV4/6) as =m — they're
already in modules-arm64.list / modules.list and get modprobed at boot,
this just makes sure olddefconfig doesn't strip them when applied on
top of a minimal defconfig.
NF_NAT_MASQUERADE pinned =y because NFT_MASQ select-depends on it; on
some kernels it would get auto-selected, on others it gets dropped by
olddefconfig if not pinned.
This change requires a kernel rebuild — the configs are bool / module
defs, not runtime knobs. On the Odroid:
rm -rf build/cache/kernel-arm64-generic
sudo make kernel-arm64 # ~30-60 min from scratch
sudo make rootfs-arm64 disk-image-arm64
x86 needs the same treatment when we cut v0.3.1.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
119 lines
3.1 KiB
Plaintext
119 lines
3.1 KiB
Plaintext
# KubeSolo OS — Shared kernel config fragment for container workloads
|
|
#
|
|
# Applied on top of:
|
|
# - Tiny Core stock config (x86_64) via build-kernel.sh
|
|
# - mainline kernel.org arm64 defconfig via build-kernel-arm64.sh
|
|
# - bcm2711_defconfig / bcm2712_defconfig via build-kernel-rpi.sh
|
|
#
|
|
# All entries here are architecture-agnostic.
|
|
# Apply this fragment twice with `make olddefconfig` between passes — TC's stock
|
|
# config has CONFIG_SECURITY disabled, which causes a single-pass olddefconfig
|
|
# to strip the security subtree before its dependencies (SYSFS, MULTIUSER) are
|
|
# resolved.
|
|
|
|
# cgroup v2 (mandatory for containerd/runc)
|
|
CONFIG_CGROUPS=y
|
|
CONFIG_CGROUP_CPUACCT=y
|
|
CONFIG_CGROUP_DEVICE=y
|
|
CONFIG_CGROUP_FREEZER=y
|
|
CONFIG_CGROUP_SCHED=y
|
|
CONFIG_CGROUP_PIDS=y
|
|
CONFIG_MEMCG=y
|
|
CONFIG_CGROUP_BPF=y
|
|
CONFIG_CFS_BANDWIDTH=y
|
|
|
|
# BPF (required for cgroup v2 device control)
|
|
CONFIG_BPF=y
|
|
CONFIG_BPF_SYSCALL=y
|
|
|
|
# Namespaces (mandatory for containers)
|
|
CONFIG_NAMESPACES=y
|
|
CONFIG_NET_NS=y
|
|
CONFIG_PID_NS=y
|
|
CONFIG_USER_NS=y
|
|
CONFIG_UTS_NS=y
|
|
CONFIG_IPC_NS=y
|
|
|
|
# Device management
|
|
CONFIG_DEVTMPFS=y
|
|
CONFIG_DEVTMPFS_MOUNT=y
|
|
|
|
# Filesystem
|
|
CONFIG_OVERLAY_FS=y
|
|
CONFIG_SQUASHFS=y
|
|
CONFIG_EXT4_FS=y
|
|
CONFIG_VFAT_FS=y
|
|
|
|
# Networking
|
|
CONFIG_BRIDGE=m
|
|
CONFIG_NETFILTER=y
|
|
CONFIG_NF_CONNTRACK=m
|
|
CONFIG_NF_NAT=m
|
|
CONFIG_NF_TABLES=m
|
|
CONFIG_VETH=m
|
|
CONFIG_VXLAN=m
|
|
|
|
# nftables address-family handlers. These are BOOL Kconfigs (not tristate)
|
|
# so they have to be built into the kernel — there's no module to modprobe
|
|
# at runtime. Without them, `nft add table ip ...` returns EOPNOTSUPP and
|
|
# KubeSolo v1.1.4+'s pod-masquerade setup fails at boot.
|
|
CONFIG_NF_TABLES_IPV4=y
|
|
CONFIG_NF_TABLES_IPV6=y
|
|
CONFIG_NF_TABLES_INET=y
|
|
CONFIG_NF_TABLES_NETDEV=y
|
|
|
|
# nftables expression modules used by KubeSolo's masquerade ruleset and
|
|
# kube-proxy's nft-compat path. Listed in modules.list / modules-arm64.list
|
|
# so init loads them at boot.
|
|
CONFIG_NFT_NAT=m
|
|
CONFIG_NFT_MASQ=m
|
|
CONFIG_NFT_CT=m
|
|
CONFIG_NFT_REDIR=m
|
|
CONFIG_NFT_REJECT=m
|
|
CONFIG_NFT_REJECT_INET=m
|
|
CONFIG_NFT_COMPAT=m
|
|
CONFIG_NFT_FIB=m
|
|
CONFIG_NFT_FIB_IPV4=m
|
|
CONFIG_NFT_FIB_IPV6=m
|
|
|
|
# IPv4 NAT bits NFT_MASQ depends on. Auto-selected on most kernels but we
|
|
# pin them explicitly so olddefconfig doesn't strip them when the fragment
|
|
# is applied on top of a minimal defconfig.
|
|
CONFIG_NF_NAT_MASQUERADE=y
|
|
|
|
# Security: AppArmor + Audit
|
|
CONFIG_AUDIT=y
|
|
CONFIG_AUDITSYSCALL=y
|
|
CONFIG_SECURITY=y
|
|
CONFIG_SECURITYFS=y
|
|
CONFIG_SECURITY_NETWORK=y
|
|
CONFIG_SECURITY_APPARMOR=y
|
|
CONFIG_DEFAULT_SECURITY_APPARMOR=y
|
|
CONFIG_LSM=lockdown,yama,apparmor
|
|
|
|
# Security: seccomp
|
|
CONFIG_SECCOMP=y
|
|
CONFIG_SECCOMP_FILTER=y
|
|
|
|
# Crypto (image verification)
|
|
CONFIG_CRYPTO_SHA256=y
|
|
|
|
# Disable unnecessary subsystems for headless edge appliance
|
|
# CONFIG_SOUND is not set
|
|
# CONFIG_DRM is not set
|
|
# CONFIG_KVM is not set
|
|
# CONFIG_MEDIA_SUPPORT is not set
|
|
# CONFIG_WIRELESS is not set
|
|
# CONFIG_WLAN is not set
|
|
# CONFIG_CFG80211 is not set
|
|
# CONFIG_BT is not set
|
|
# CONFIG_NFC is not set
|
|
# CONFIG_INFINIBAND is not set
|
|
# CONFIG_PCMCIA is not set
|
|
# CONFIG_HAMRADIO is not set
|
|
# CONFIG_ISDN is not set
|
|
# CONFIG_ATM is not set
|
|
# CONFIG_INPUT_JOYSTICK is not set
|
|
# CONFIG_INPUT_TABLET is not set
|
|
# CONFIG_FPGA is not set
|