Single Node K8s Cluster with Kubespray

This article explains how you can set up a single node Kubernetes cluster installation process. Also, includes MetalLB for our use case.

What we need

• A machine that your K8s cluster will run on
• Have a ssh access with key pair our don’t have it — we have two option for installation
  - remote installation
  - local installation

Here, we are going to use a virtual machine on OpenStack.

Kubespray uses Ansbile playbook to set-up K8s cluster. For remote installation, If you didn’t set-up a key pair already, you can follow the steps below to create and install a key pair. If you are going to use local installation just skip this part.

# Generate the key pair on the machine that Ansible will run
$ ssh-keygen -t rsa -b 2048 -f $HOME/.ssh/id_rsa -C "my k8s cluster key"
# Install the public key to the remote machine
$ ssh-copy-id -i $HOME/.ssh/id_rsa.pub <username>@<host>

Now, let’s get Kubespray from repository and create a virtual environment to install the dependencies for Ansible which is a python application

# Clone the repository
$ git clone https://github.com/kubernetes-sigs/kubespray.git
$ cd kubespray
# We don't know what the future will change. This is why we are going to use a specific version
$ git checkout release-2.16
# Create virtual environment. If it's new machine you'll probably missing a package. Use `apt-get install python3-venv` to install it.
$ python3 -m venv venv
# Activate the environment
$ source venv/bin/activate
# Install the dependencies
$ pip install -r requirements.txt

Basically, we have all the tools and equipment to create K8s cluster.

Time to work on Ansible inventory.

The Ansible inventory file defines the hosts and groups of hosts upon which commands, modules, and tasks in a playbook operate.
Source: https://www.juniper.net/documentation/us/en/software/junos-ansible/ansible/topics/concept/junos-ansible-inventory-file-overview.html

We’ll use inventory generator to create an Ansible inventory which is provided by Kubespray community.

We will copy a sample inventory from the repository we just cloned.

# Define a folder name
$ declare -r CLUSTER_FOLDER='my-cluster'
# This for remote installation
cp -rfp inventory/sample inventory/$CLUSTER_FOLDER
# This for local installation
cp -rfp inventory/local inventory/$CLUSTER_FOLDER

Update Ansible inventory with inventory generator

# Create a list of ip since we are working on a single node we'll just provide single ip
# For local installation, skip this part. We do'nt need to provide a ip address.
# For remote installation provide the ip address
$ declare -a IPS=(<host>)
$ CONFIG_FILE=inventory/$CLUSTER_FOLDER/hosts.yaml python3 contrib/inventory_builder/inventory.py ${IPS[@]}

Since we are building single node K8s cluster, we don’t need to adjust the host.yaml file which is just generated by inventory generator. You may want to adjust this file if you have more nodes.

kube_network_plugin is set to calico by default. Let’s leave it this way.

Moreoever, Kubespray provide us easy installation for popular kubernetes add-ons. For our cluster we’ll enable metallb in inventory/$CLUSTER_FOLDER/group_vars/k8s_cluster/addons.yml

Enabling MetalLB:

MetalLB is a load-balancer implementation for bare metal Kubernetes clusters, using standard routing protocols.
Source: https://metallb.universe.tf/

MetalLB has layer 2 and BGP mode. Since we are using a single node cluster, we’ll be using layer 2 mode. Because layer 2 mode can work on very small deployments and all traffic is going to a single node. With BGP, we can distribute the traffic across multiple nodes and it requires a fundamental understanding of networks.

You can read more about MetalLB and it’s modes here.

Layer 2 mode comes with a limitation of using ARP. To overcome this limitation Kubespray provides us kube_proxy_strict_arp parameter that must be set to true for MetalLB to work. This setting is located in inventory/$CLUSTER_FOLDER/group_vars/k8s_cluster/k8s-cluster.yml file. It will configure arp_ignore and arp_announce to avoid answering ARP queries from kube-ipvs0 interface.

What is ARP ?

ARP stands for Address Resolution Protocol. ARP is the protocol to find destination machine physical address(layer 2 address) in the same network. Briefly, a machine broadcast a request for the physical address of a machine in the network. And, every other machines in the same network receives the request and the machine who has the the destination machine will response to the request with its physical address.

A great video about ARP:

You can read more about arp_ignore and arp_announce here.

Let’s get back to enabling MetalLB. You will find following lines as comment lines in addons.yml file. Activate following lines and make sure you change metallb_ip_range with your host ip.

metallb_enabled: true  # Set this `true`
metallb_speaker_enabled: true  # This should be already set to `true`
metallb_ip_range:
  - "<host>-<host>"  # Since we are creating single node kubernetes cluster, provide the single host ip. Ex: 10.0.0.10-10.0.0.10
metallb_controller_tolerations:  # uncomment line
  - key: "node-role.kubernetes.io/master"  # uncomment line
    operator: "Equal"  # uncomment line
    value: ""  # uncomment line
    effect: "NoSchedule"  # uncomment line
  - key: "node-role.kubernetes.io/control-plane"  # uncomment line
    operator: "Equal"  # uncomment line
    value: ""  # uncomment line
    effect: "NoSchedule"  # uncomment line

MetalLB is enabled for you cluster. Also, you can enable other add-ons that you want in the addons.yml.

The default installation is using Docker as container runtime. If you want to change this to something else go here to find the name of the runtime and follow the documentations. In this article, we are going to use containerd. If you want to keep Docker as runtime, you can skip this part.

To use containerd container runtime set the following parameters:

inventory/$CLUSTER_FOLDER/group_vars/k8s_cluster/k8s-cluster.yml :

container_manager: containerd

$CLUSTER_FOLDER/group_vars/etcd.yml :

etcd_deployment_type: host

Define registry mirror for Docker Hub

$CLUSER_FOLDER/group_vars/all/containerd.yml :

containerd_registries:
  "docker.io":
    - "https://mirror.gcr.io"
    - "https://registry-1.docker.io"

Let’s fire it up and watch the magic happen.

Note: If you are root user, you can omit the -b flag in the command.

$ USERNAME=$(whoami)
# Local installation
$ ansible-playbook -i inventory/$CLUSTER_FOLDER/hosts.ini --connection=local -b -v cluster.yml
# Remote installation
$ ansible-playbook -i inventory/$CLUSTER_FOLDER/hosts.yaml -u $USERNAME -b -v --private-key=~/.ssh/id_rsa cluster.yml

To use the kubectl with other users, apply the followings

# Copy the .kube folder to $HOME
$ sudo cp -r /root/.kube $HOME
# Fix the permission
$ sudo chown -R $USER $HOME/.kube

Check with the following command that your single node K8s cluster is runnnig and heathy.

$ kubectl get all -A

References:

https://kubespray.io/#/

https://github.com/kubernetes-sigs/kubespray/blob/master/docs/setting-up-your-first-cluster.md

https://github.com/kubernetes-sigs/kubespray/issues/6368

https://github.com/metallb/metallb/issues/153#issuecomment-518651132

https://github.com/kubernetes-sigs/kubespray/issues/5932

https://github.com/kubernetes-sigs/kubespray/blob/master/docs/containerd.md

https://www.cyberciti.biz/faq/how-to-set-up-ssh-keys-on-linux-unix/