adelorenzo
/
kilo
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

docs,README: document multi-cluster services

This commit is contained in:
Lucas Servén Marín 2019-05-08 17:10:09 +02:00
parent 90e68c7735
commit 545bc4186f
No known key found for this signature in database
GPG Key ID: 586FEAF680DA74AD
4 changed files with 166 additions and 10 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

44
README.md
View File

@ -12,10 +12,11 @@ Kilo is a multi-cloud network overlay built on WireGuard and designed for Kubern
Kilo connects nodes in a cluster by providing an encrypted layer 3 network that can span across data centers and public clouds.
By allowing pools of nodes in different locations to communicate securely, Kilo enables the operation of multi-cloud clusters.
Kilo's design allows clients to VPN to a cluster in order to securely access services running on the cluster.
In addition to creating multi-cloud clusters, Kilo enables the creation of multi-cluster services, i.e. services that span across different Kubernetes clusters.
## How it works
Kilo uses [WireGuard](https://www.wireguard.com/), a performant and secure VPN, to create a mesh between the different logical locations in a cluster.
Kilo uses [WireGuard](https://www.wireguard.com/), a performant and secure VPN, to create a mesh between the different nodes in a cluster.
The Kilo agent, `kg`, runs on every node in the cluster, setting up the public and private keys for the VPN as well as the necessary rules to route packets between locations.
Kilo can operate both as a complete, independent networking provider as well as an add-on complimenting the cluster-networking solution currently installed on a cluster.
@ -94,7 +95,7 @@ metadata:
name: squat
spec:
allowedIPs:
- 10.4.1.1/32
- 10.5.0.1/32
publicKey: GY5aT1N9dTR/nJnT1N2f4ClZWVj0jOAld0r8ysWLyjg=
persistentKeepalive: 10
EOF
@ -109,9 +110,46 @@ sudo wg setconf wg0 peer.ini
[See the VPN docs for more details](./docs/vpn.md).
## Multi-cluster Services
A logical application of Kilo's VPN is to connect two different Kubernetes clusters.
This allows workloads running in one cluster to access services running in another.
For example, if `cluster1` is running a Kubernetes Service that we need to access from Pods running in `cluster2`, we could do the following:
```shell
# Register cluster1 as a peer of cluster2.
kgctl --kubeconfig $KUBECONFIG1 showconf node $NODE1 --as-peer -o yaml --allowed-ips $PODCIDR1,$SERVICECIDR1 | kubectl --kubeconfig KUBECONFIG2 apply -f -
# Register cluster2 as a peer of cluster1.
kgctl --kubeconfig $KUBECONFIG2 showconf node $NODE2 --as-peer -o yaml --allowed-ips $PODCIDR2,$SERVICECIDR2 | kubectl --kubeconfig KUBECONFIG1 apply -f -
# Create a Service in cluster2 to mirror the Service in cluster1.
cat <<'EOF' | kubectl --kubeconfig $KUBECONFIG2 apply -f -
apiVersion: v1
kind: Service
metadata:
name: important-service
spec:
ports:
- port: 80
---
apiVersion: v1
kind: Endpoints
metadata:
name: important-service
subsets:
- addresses:
- ip: $CLUSTERIP # The cluster IP of the important service on cluster1.
ports:
- port: 80
EOF
```
Now, `important-service` can be used on `cluster2` just like any other Kubernetes Service.
[See the multi-cluster services docs for more details](./docs/multi-cluster-services.md).
## Analysis
The topology of a Kilo network can be analyzed using the `kgctl` binary.
The topology and configuration of a Kilo network can be analyzed using the `kgctl` binary.
For example, the `graph` command can be used to generate a graph of the network in Graphviz format:
```shell

34
cmd/kgctl/showconf.go
View File

@ -17,6 +17,7 @@ package main
import (
"errors"
"fmt"
"net"
"os"
"strings"
@ -43,6 +44,8 @@ var (
outputFormatWireGuard,
outputFormatYAML,
}, ", ")
allowedIPs []string
aips []*net.IPNet
asPeer bool
output string
serializer *json.Serializer
@ -63,6 +66,7 @@ func showConf() *cobra.Command {
}
cmd.PersistentFlags().BoolVar(&asPeer, "as-peer", false, "Should the resource be shown as a peer? Useful to configure this resource as a peer of another WireGuard interface.")
cmd.PersistentFlags().StringVarP(&output, "output", "o", "wireguard", fmt.Sprintf("The output format of the resource. Only valid when combined with 'as-peer'. Possible values: %s", availableOutputFormats))
cmd.PersistentFlags().StringSliceVar(&allowedIPs, "allowed-ips", []string{}, "Override the allowed IPs of the configuration. Only valid when combined with 'as-peer'.")
return cmd
}
@ -77,6 +81,13 @@ func runShowConf(c *cobra.Command, args []string) error {
default:
return fmt.Errorf("output format %v unknown; posible values are: %s", output, availableOutputFormats)
}
for i := range allowedIPs {
_, aip, err := net.ParseCIDR(allowedIPs[i])
if err != nil {
return fmt.Errorf("allowed-ips must contain only valid CIDRs; got %q", allowedIPs[i])
}
aips = append(aips, aip)
}
return runRoot(c, args)
}
@ -148,12 +159,17 @@ func runShowConfNode(_ *cobra.Command, args []string) error {
case outputFormatYAML:
p := translatePeer(t.AsPeer())
p.Name = hostname
if len(aips) != 0 {
p.Spec.AllowedIPs = allowedIPs
}
return serializer.Encode(p, os.Stdout)
case outputFormatWireGuard:
p := t.AsPeer()
if len(aips) != 0 {
p.AllowedIPs = aips
}
c, err := (&wireguard.Conf{
Peers: []*wireguard.Peer{
t.AsPeer(),
},
Peers: []*wireguard.Peer{p},
}).Bytes()
if err != nil {
return fmt.Errorf("failed to generate configuration: %v", err)
@ -215,12 +231,18 @@ func runShowConfPeer(_ *cobra.Command, args []string) error {
case outputFormatYAML:
p := translatePeer(t.AsPeer())
p.Name = peer
p.Name = hostname
if len(aips) != 0 {
p.Spec.AllowedIPs = allowedIPs
}
return serializer.Encode(p, os.Stdout)
case outputFormatWireGuard:
p := &peers[peer].Peer
if len(aips) != 0 {
p.AllowedIPs = aips
}
c, err := (&wireguard.Conf{
Peers: []*wireguard.Peer{
&peers[peer].Peer,
},
Peers: []*wireguard.Peer{p},
}).Bytes()
if err != nil {
return fmt.Errorf("failed to generate configuration: %v", err)

57
docs/multi-cluster-services.md Normal file
View File

@ -0,0 +1,57 @@
# Multi-cluster Services
Just as Kilo can connect a Kubernetes cluster to external services over WireGuard, it can connect multiple independent Kubernetes clusters.
This enables clusters to provide services to other clusters over a secure connection.
For example, a cluster on AWS with access to GPUs could run a machine learning service that could be consumed by workloads running in a another location, e.g. an on-prem cluster without GPUs.
Unlike services exposed via Ingresses or NodePort Services, multi-cluster services can remain private and internal to the clusters.
*Note*: clusters connected with Kilo must have non-overlapping pod and service CIDRs.
Consider two clusters, `cluster1` with:
* kubeconfig: `KUBECONFIG1`
* pod CIDR: $PODCIDR1`
* service CIDR: $SERVICECIDR1`
* a node named: `$NODE1`
and `cluster2` with:
* kubeconfig: `KUBECONFIG2`
* pod CIDR: $PODCIDR2`
* service CIDR: $SERVICECIDR2`
* a node named: `$NODE2`
In order to give `cluster2` access to a service running on `cluster1`, start by peering the nodes:
```shell
# Register cluster1 as a peer of cluster2.
kgctl --kubeconfig $KUBECONFIG1 showconf node $NODE1 --as-peer -o yaml --allowed-ips $PODCIDR1,$SERVICECIDR1 | kubectl --kubeconfig KUBECONFIG2 apply -f -
# Register cluster2 as a peer of cluster1.
kgctl --kubeconfig $KUBECONFIG2 showconf node $NODE2 --as-peer -o yaml --allowed-ips $PODCIDR2,$SERVICECIDR2 | kubectl --kubeconfig KUBECONFIG1 apply -f -
```
Now, `cluster2` has access to Pods and Services on `cluster1`, and vice-versa.
However, as it stands the external Services can only be accessed by using their clusterIPs directly; in other words, they are not Kubernetes-native.
We can change that by creating a Kubernetes Service in `cluster2` to mirror the Service in `cluster1`:
```
cat <<'EOF' | kubectl --kubeconfig $KUBECONFIG2 apply -f -
apiVersion: v1
kind: Service
metadata:
name: important-service
spec:
ports:
- port: 80
---
apiVersion: v1
kind: Endpoints
metadata:
name: important-service
subsets:
- addresses:
- ip: $CLUSTERIP # The cluster IP of the important service on cluster1.
ports:
- port: 80
EOF
```
Now, `important-service` can be used on `cluster2` just like any other Kubernetes Service.

41
docs/vpn.md
View File

@ -16,7 +16,7 @@ metadata:
name: squat
spec:
allowedIPs:
- 10.4.1.1/32
- 10.5.0.1/32 # Example IP address on the peer's interface.
publicKey: GY5aT1N9dTR/nJnT1N2f4ClZWVj0jOAld0r8ysWLyjg=
persistentKeepalive: 10
```
@ -66,3 +66,42 @@ For example, try connecting to the API server:
```shell
curl -k https://10.0.27.179:6443
```
Likewise, the cluster now also has layer 3 access to the newly added peer.
From any node or Pod on the cluster, one can now ping the peer:
```shell
ping 10.5.0.1
```
If the peer exposes a layer 4 service, for example an HTTP service, then one could also make requests against that endpoint from the cluster:
```shell
curl http://10.5.0.1
```
Kubernetes Services can be created to provide better discoverability to cluster workloads for services exposed by peers, for example:
```shell
cat <<'EOF' | kubectl apply -f -
apiVersion: v1
kind: Service
metadata:
name: important-service
spec:
ports:
- port: 80
---
apiVersion: v1
kind: Endpoints
metadata:
name: important-service
subsets:
- addresses:
- ip: 10.5.0.1
ports:
- port: 80
EOF
```
[See the multi-cluster services docs for more details on connecting clusters to external services](./docs/multi-cluster-services.md).