diff --git a/README.md b/README.md index e3b60e6..c28e665 100644 --- a/README.md +++ b/README.md @@ -130,6 +130,9 @@ sudo wg setconf wg0 peer.ini [See the VPN docs for more details](./docs/vpn.md). +In case you want to only run Kilo as VPN server, [see the VPN-only walkthrough](./docs/vpn-only.md) +for details. + ## Multi-cluster Services A logical application of Kilo's VPN is to connect two different Kubernetes clusters. diff --git a/docs/vpn-only.md b/docs/vpn-only.md new file mode 100644 index 0000000..060f0c7 --- /dev/null +++ b/docs/vpn-only.md @@ -0,0 +1,125 @@ +# Kilo as VPN Server to connect into the cluster + +Use Case: Connect into the Kubernetes Cluster via WireGuard VPN, so that a WireGuard client has direct +access to Pods and Services. + +## Prerequisites + +- Ensure Wireguard is installed on the host system +- UDP port 51820 must be externally reachable to the cluster + +## Deployment + +- For this case, it is enough to deploy a single instance of Kilo into the cluster. Kilo should be + **pinned** to a single Node, because the WireGuard private key is host-specific. +- CNI has to be disabled, because you are keeping the existing CNI plugin. +- You can use in-cluster Kubernetes configuration, and do not need to mount the host Kubernetes config. + (The latter is only needed to extract in the CNI case to detect the outside-visible Hostname for + each node). +- We only tested this with Flannel or Flannel+Calico (=Canal) so far. + +You can still access all Kubernetes Services and Pods, no matter where they run, in this configuration. + +The full configuration can be found at [manifests/kilo-vpn-only-example.yaml](../manifests/kilo-vpn-only-example.yaml). + +**Make sure to adjust the `nodeSelector` in the `DaemonSet`**. + +## Registering a new VPN client + +1. Create a new WireGuard keypair on the VPN Client. Remember the public key. + +2. For the client, create a Kubernetes `peer` resource: + + - pick a new, unique, IP address for the client from the `10.5.0.*` IP range. + Based on this IP, we can lateron decide what the client can access. + + - add the public key from the VPN client (see step 1) to the `peer` resource. + + Example: + + ``` + apiVersion: kilo.squat.ai/v1alpha1 + kind: Peer + metadata: + name: squat + spec: + allowedIPs: + # desired IP address of the client's interface. + - 10.5.0.1/32 + # Public Key of the client + publicKey: A......................................= + persistentKeepalive: 10 + ``` + +3. Configure your local VPN client in the following way: + + ```ini + [Interface] + PrivateKey = (already filled) + # IP address of VPN client; from the "peer" kubernetes resource as configured above + Address = 10.5.0.1/32 + + [Peer] + # from within the "kilo" Pod, run "wg" - that outputs the persistent, public key for + # the server + PublicKey = B......................................= + + # Add the Pod and Service networks, e.g. if 10.42.* is the Pod Network; and 10.43.* + # is the Service network: + AllowedIPs = 10.42.0.0/16, 10.43.0.0/16 + + # public IP address of the Kilo node + Wireguard/Kilo UDP Port + Endpoint = 138.201.76.122:51820 + + # the server is always reachable, so we do not need PersistentKeepalive in + # this direction. + PersistentKeepalive = 0 + ``` + +## Optional: Using Calico / Canal to restrict the NetworkPolicy in the Cluster + +If you are running Calico + Flannel (=Canal), you can use `HostEndpoint` combined +with a `GlobalNetworkPolicy` to restrict what each VPN client can access in the cluster: + +```yaml +--- +apiVersion: crd.projectcalico.org/v1 +kind: HostEndpoint +metadata: + name: wireguard-kilo + labels: + interface: wireguard-kilo +spec: + node: THE_KILO_NODE_HERE + expectedIPs: + # in the "kilo" container, we run "ip address show kilo0", + # and this is the IP of the "inner" side of the wireguard interface. + # thus, we need to match on this IP here. + - 10.4.0.1 + +--- +apiVersion: crd.projectcalico.org/v1 +kind: GlobalNetworkPolicy +metadata: + name: wireguard-kilo +spec: + selector: interface == 'wireguard-kilo' + # we want to apply the policy as it enters our cluster (exits the wireguard + # interface). On the application pods, we could not apply it anymore, because + # the IP address gets rewritten to the Flannel interface IP. + applyOnForward: true + types: + - Ingress + ingress: + - action: Allow + source: + nets: + - 10.5.0.1/32 # a certain VPN client ... + destination: + # ... can access a certain app + namespaceSelector: network-policy-namespace == "cattle-prometheus" + selector: app == "grafana" + + # anything which is not whitelisted explicitly is forbidden + - action: Deny +``` \ No newline at end of file diff --git a/docs/vpn.md b/docs/vpn.md index fef876f..2bd1f51 100644 --- a/docs/vpn.md +++ b/docs/vpn.md @@ -60,6 +60,9 @@ for ip in $(kgctl showconf peer $PEER | grep AllowedIPs | cut -f 3- -d ' ' | tr done ``` +When using the official Mac OS WireGuard client, the routes from `AllowedIPs` will be automatically +routed to the VPN tunnel. You do not need to manually register routes there. + Once the routes are in place, the connection to the cluster can be tested. For example, try connecting to the API server: @@ -105,3 +108,22 @@ EOF ``` [See the multi-cluster services docs for more details on connecting clusters to external services](./multi-cluster-services.md). + +## Accessing Service IPs via the VPN + +Service IPs are usually assigned to a separate IP address range compared to the Pod IPs. Kilo will only +output the Pod IP range in the WireGuard Client configuration when running `kgctl showconf peer`. This is +because Service IPs can be sent to any Kubernetes node, and then routing happens internally towards +the pods. + +To access service IPs via the VPN client, simply add them in your WireGuard client configuration +to the `AllowedIPs` list, f.e. like `10.43.0.0/15` (if your services are allocated from the `10.43` IP +range). + +## Using Kilo only as VPN server + +You can also use Kilo only for accessing your cluster pods and services via VPN client; and not as +CNI Plugin. + +This is documented [in the docs for vpn-only](./vpn-only.md), because this is easier to configure +and deploy. \ No newline at end of file diff --git a/manifests/kilo-vpn-only-example.yaml b/manifests/kilo-vpn-only-example.yaml new file mode 100644 index 0000000..3f940fb --- /dev/null +++ b/manifests/kilo-vpn-only-example.yaml @@ -0,0 +1,100 @@ +apiVersion: v1 +kind: ServiceAccount +metadata: + name: kilo + namespace: kilo +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: kilo +rules: +- apiGroups: + - "" + resources: + - nodes + verbs: + - list + - patch + - watch +- apiGroups: + - kilo.squat.ai + resources: + - peers + verbs: + - list + - update + - watch +- apiGroups: + - apiextensions.k8s.io + resources: + - customresourcedefinitions + verbs: + - create +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: kilo +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: kilo +subjects: + - kind: ServiceAccount + name: kilo + namespace: kilo +--- +apiVersion: apps/v1 +kind: DaemonSet +metadata: + name: kilo + namespace: kilo + labels: + app.kubernetes.io/name: kilo +spec: + selector: + matchLabels: + app.kubernetes.io/name: kilo + template: + metadata: + labels: + app.kubernetes.io/name: kilo + spec: + nodeSelector: + # !!! Decide where you want to run your Kilo ingress. + kubernetes.io/hostname: TODO-ADD-YOUR-HOST-HERE + serviceAccountName: kilo + # we need to be part of the host network; otherwise, we cannot configure wireguard. + hostNetwork: true + containers: + - name: kilo + image: squat/kilo + args: + - --hostname=$(NODE_NAME) + # we only want to use Kilo as VPN; and not as CNI interface. + - --cni=false + - --encapsulate=never + # we want to work together with Flannel. + - --compatibility=flannel + - --local=false + env: + - name: NODE_NAME + valueFrom: + fieldRef: + fieldPath: spec.nodeName + # we need to be root to configure wireguard + securityContext: + privileged: true + volumeMounts: + - name: kilo-dir + mountPath: /var/lib/kilo + tolerations: + - effect: NoSchedule + operator: Exists + - effect: NoExecute + operator: Exists + volumes: + - name: kilo-dir + hostPath: + path: /var/lib/kilo