December 16, 2021

Day 16 - Setting up k3s in your home lab

By: Joe Block (@curiousbiped)
Edited by: Jennifer Davis (@sigje)

Background

Compute, even at home with consumer-grade hardware, has gotten ridiculously cheap. You can get a quad-core ARM machine with 4GB like a Raspberry Pi 4 for under $150, including power supply and SD card for booting - and it'll idle at less than 5 watts of power draw and be completely silent because it is fanless.

What we're going to do

In this post, I'll show you how to set up a Kubernetes cluster on a cheap ARM board (or an x86 box if you prefer) using k3s and k3sup so you can learn Kubernetes without breaking an environment in use.

These instructions will also work on x86 machines, so you can repurpose that old hardware instead of buying a new Raspberry Pi.

Why k3s?

k3s was created by Rancher as a lightweight, easy to install, and secure Kubernetes option.

It's packaged as a single ~40MB binary that reduces the dependencies needed to get a cluster up and running. It even includes an embedded containerd, so you don't need to install that or docker. The ARM64 and ARM7 architectures are fully supported, so it's perfect for running on a Raspberry Pi in a home lab environment.

Why k3sup?

Alex Ellis wrote k3sup, a great tool for bringing up k3s clusters and we're going to use it in this post to simplify setting up a brand new cluster. With k3sup, we'll have a running kubernetes cluster in less than ten minutes.

Lets get started!

Pre-requisites.

  • A spare linux box. I'll be using a Raspberry Pi for my examples, but you can follow along on an x86 linux box or VM if you prefer.
  • k3sup - download the latest release from k3sup/releases into a directory in your $PATH.

Set up your cluster.

In the following example, I'm assuming you've created a user (you can use the pi user on rPi if you prefer) for configuring the cluster (I used borg below), you've added your ssh public key to ~pi/.ssh/authorized_keys and that the user has sudo privileges. I'm also assuming you've downloaded k3sup and put it into /usr/local/bin, and that /usr/local/bin is in your $PATH.

Create the leader node

The first step is to create the leader node with the k3sup utility: 


k3sup install --host $HOSTNAME --user pi

Below is the output when I ran it against my scratch rPi. In the scrollback you'll see that I'm using my borg account instead of the pi user. After setting up the rPi, the first step I took was to disable the known pi account. I also specify the path to an SSH key that is in the borg account's authorized_keys, and configure the borg account to allow passwordless sudo.

Notice that I don't have to specify an architecture - k3sup automagically determines the architecture of the host and installs the correct binaries when it connects to the machine. All I have to do is tell it what host to connect to, what user to use, what ssh key, and whether I want to use the stable or latest k3s channels or a specific version. 


❯ k3sup install --host cephalopod.example.com --user borg --ssh-key demo-key
--k3s-channel stable


k3sup install --host cephalopod.example.com --user borg --ssh-key demo-key --k3s-channel stable
Running: k3sup install
2021/12/13 16:30:49 cephalopod.example.com
Public IP: cephalopod.example.com
[INFO]  Finding release for channel stable
[INFO]  Using v1.21.7+k3s1 as release
[INFO]  Downloading hash https://github.com/k3s-io/k3s/releases/download/v1.21.7+k3s1/sha256sum-arm64.txt
[INFO]  Downloading binary https://github.com/k3s-io/k3s/releases/download/v1.21.7+k3s1/k3s-arm64
[INFO]  Verifying binary download
[INFO]  Installing k3s to /usr/local/bin/k3s
[INFO]  Skipping installation of SELinux RPM
[INFO]  Creating /usr/local/bin/kubectl symlink to k3s
[INFO]  Creating /usr/local/bin/crictl symlink to k3s
[INFO]  Skipping /usr/local/bin/ctr symlink to k3s, command exists in PATH at /usr/bin/ctr
[INFO]  Creating killall script /usr/local/bin/k3s-killall.sh
[INFO]  Creating uninstall script /usr/local/bin/k3s-uninstall.sh
[INFO]  env: Creating environment file /etc/systemd/system/k3s.service.env
[INFO]  systemd: Creating service file /etc/systemd/system/k3s.service
[INFO]  systemd: Enabling k3s unit
Created symlink /etc/systemd/system/multi-user.target.wants/k3s.service → /etc/systemd/system/k3s.service.
[INFO]  systemd: Starting k3s
Result: [INFO]  Finding release for channel stable
[INFO]  Using v1.21.7+k3s1 as release
[INFO]  Downloading hash https://github.com/k3s-io/k3s/releases/download/v1.21.7+k3s1/sha256sum-arm64.txt
[INFO]  Downloading binary https://github.com/k3s-io/k3s/releases/download/v1.21.7+k3s1/k3s-arm64
[INFO]  Verifying binary download
[INFO]  Installing k3s to /usr/local/bin/k3s
[INFO]  Skipping installation of SELinux RPM
[INFO]  Creating /usr/local/bin/kubectl symlink to k3s
[INFO]  Creating /usr/local/bin/crictl symlink to k3s
[INFO]  Skipping /usr/local/bin/ctr symlink to k3s, command exists in PATH at /usr/bin/ctr
[INFO]  Creating killall script /usr/local/bin/k3s-killall.sh
[INFO]  Creating uninstall script /usr/local/bin/k3s-uninstall.sh
[INFO]  env: Creating environment file /etc/systemd/system/k3s.service.env
[INFO]  systemd: Creating service file /etc/systemd/system/k3s.service
[INFO]  systemd: Enabling k3s unit
[INFO]  systemd: Starting k3s
 Created symlink /etc/systemd/system/multi-user.target.wants/k3s.service → /etc/systemd/system/k3s.service.

Saving file to: /Users/jpb/democluster/kubeconfig

# Test your cluster with:
export KUBECONFIG=/Users/jpb/democluster/kubeconfig
kubectl config set-context default
kubectl get node -o wide

Test it out

Per the directions output by k3sup, you can now test your brand new cluster by setting the environment variable KUBECONFIG, and then run kubectl to work with your new cluster.

My steps to verify my new cluster is up and running:

  1. export KUBECONFIG=/Users/jpb/democluster/kubeconfig
  2. kubectl config set-context default
  3. kubectl get node -o wide

And I see nice healthy output where the status shows Ready - 



NAME         STATUS   ROLES                  AGE     VERSION        INTERNAL-IP
 EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION   CONTAINER-RUNTIME

cephalopod   Ready    control-plane,master   2m53s   v1.21.7+k3s1   10.1.2.3
<none>        Ubuntu 18.04.3 LTS   4.9.196-63       containerd://1.4.12-k3s1

And I can also look at pods in the cluster 



❯ kubectl get pods -A
Alias tip: kc get pods -A
NAMESPACE     NAME                                      READY   STATUS
RESTARTS   AGE
kube-system   coredns-7448499f4d-b2rdp                  1/1     Running     0
      9m29s
kube-system   local-path-provisioner-5ff76fc89d-d9rrc   1/1     Running     0
      9m29s
kube-system   metrics-server-86cbb8457f-cqk6q           1/1     Running     0
      9m29s
kube-system   helm-install-traefik-crd-jgk2x            0/1     Completed   0
      9m29s
kube-system   helm-install-traefik-l2j96                0/1     Completed   2
      9m29s
kube-system   svclb-traefik-7tzzs                       2/2     Running     0
      8m38s
kube-system   traefik-6b84f7cbc-92kkp                   1/1     Running     0
      8m38s

Clean Up

k3s is tidy and easy to uninstall, so you can stand up a cluster on a machine, do some experimentation, then dispose of the cluster and have a clean slate for your next experiment. This makes it great for continuous integration! 


sudo /usr/local/bin/k3s-uninstall.sh to shut down the node and delete
/var/lib/rancher and the data stored there.

Next Steps

Learn kubernetes! Some interesting tutorials that I recommend -

Finally, now that you've set up a cluster the easy way, if you want to understand everything k3sup did behind the scenes to get your Kubernetes cluster up and running, Kubernetes the Hard Way by Kelsey Hightower is a must-read.

No comments:

Post a Comment