Ubuntu 20.04 Raspberry Pi Cluster

I thought I’d have some fun in December, setting up a Raspberry Pi cluster for running Kubernetes.

My goal was to automate as much of the process of setting up each node of the cluster as possible, as I’d read several blog posts on the subject and most require many manual steps having to be repeated on each node, and since I value repeatable processes I enjoyed the challenge of figuring it out.

I’m assuming a certain level of knowledge for readers of this post, so I’m not spelling out every step needed.

There are many posts on the internet which do a much better job of explaining things; for example, checkout this post Make Ubuntu server 20.04 boot from an SSD on Raspberry Pi 4 by Zakaria Smahi.


First off, the inventory of components I am using:


Since the Raspberry Pi 4 supports booting off an external drive via USB, I only purchased one SD Card, which will be needed to boot each Pi in order to enable booting from USB.

I’m planning to run Kubernetes on my cluster, so it is recommended to run off Solid State Drives as Kubernetes is disk heavy and the performance and lifetime of SSD’s is considerably better than that of an SD card.

EDIT: I had to update the firmware of the SSD drive enclosures to solve a slow boot issue. The updated firmware can be found on the Sabrent website and seaching for the EC-UASP model. I had to use a Windows computer to perform the firmware update.

Disk Images


I followed these steps to setup my cluster from my Ubuntu laptop.

Enabling USB Boot

Booting the Raspberry Pi off USB isn’t enabled by default; enabling requires changing the boot order to first attempt USB followed by the SD card.

Flash the SD card with the Raspberry Pi OS Lite operating system. This article explains how to install the Raspberry Pi operating system image on an SD card.

SSH needs to be enabled so headless installation is possible. After flashing the SD card, mount it and create an empty file called ssh in the “boot” partition. Unmount and eject the SD card when done.

Insert the SD card into the first Raspberry Pi node and switch it on.

Figure out it’s IP address and connect via SSH from your PC. I used nmap to figure out the IP address; given my home network is in the 192.168.0.x range, I limited the search to addresses between 1 and 254.

$ nmap

Starting Nmap 7.80 ( https://nmap.org ) at 2020-12-22 17:21 GMT


Nmap scan report for raspberrypi (
Host is up (0.0094s latency).
Not shown: 997 closed ports
22/tcp  open  ssh


In my case, the IP was and the default username is pi and password is raspberry.

$ ssh pi@
The authenticity of host ' (' can't be established.
ECDSA key fingerprint is SHA256:XxXxXXXxxx/ZZzyyyzZZZxxxXXxyyYYYZZzZzZZxxYy.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '' (ECDSA) to the list of known hosts.
pi@'s password:
Linux raspberrypi 5.10.0-v7l+ #1382 SMP Tue Dec 15 18:23:34 GMT 2020 armv7l

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Tue Dec 22 18:57:21 2020

SSH is enabled and the default password for the 'pi' user has not been changed.
This is a security risk - please login as the 'pi' user and type 'passwd' to set a new password.

Wi-Fi is currently blocked by rfkill.
Use raspi-config to set the country before use.

pi@raspberrypi:~ $

Once you have an SSH terminal onto to first Raspberry Pi node you can configure the boot order.

See Raspberry Pi 4 bootloader configuration for how it is done.

Or use the raspi-config utility to configure via the console user-interface.

sudo raspi-config

Power off the node, running sudo poweroff, remove the SD card and repeat these steps for each of the nodes.

Provisioning Ubuntu

The following configuration files and scripts are required to provision the SSD drive for each Raspberry Pi node.

Create a directory on your PC and create each file with the following content, or clone this gist

  • usercfg.txt

    initramfs initrd.img followkernel
  • user-data

    hostname: HOSTNAME
    manage_etc_hosts: true
      - k8s
      - name: k8s
        gecos: Kubernetes
        passwd: "PASSWORD_HASH"
        lock_passwd: false
        primary_group: k8s
        groups: [adm, audio, cdrom, dialout, dip, floppy, lxd, netdev, plugdev, sudo, videom, k8s]
        sudo: ["ALL=(ALL) NOPASSWD:ALL"]
        shell: /bin/bash
          - "SSH_AUTHORIZED_KEY"
    ssh_pwauth: false
    package_update: true
    package_upgrade: true
  • network-config

    My home network is on 192.168.0.*, but you may need to change the range for your network.

    version: 2
        dhcp4: false
        optional: true
        addresses: [192.168.0.IPSUFFIX/24]
          search: [home]
          addresses: []
  • auto_decompress_kernel

    #!/bin/bash -e
    # set variables
    # check if compression needs to be done
    if [ -e $BTPATH/check.md5 ]; then
      if md5sum --status --ignore-missing -c $BTPATH/check.md5; then
        echo -e "\e[32mFiles have not changed, decompression not needed\e[0m"
        exit 0
        echo -e "\e[31mHash failed, kernel will be decompressed\e[0m"
    # backup the old decompressed kernel
    mv $DKPATH $DKPATH.bak
    if [ ! $? == 0 ]; then
      echo -e "\e[31mDECOMPRESSED KERNEL BACKUP FAILED!\e[0m"
      exit 1
      echo -e "\e[32mDecompressed kernel backup was successful\e[0m"
    # decompress the new kernel
    echo "Decompressing kernel: "$CKPATH"..."
    zcat $CKPATH > $DKPATH
    if [ ! $? == 0 ]; then
      echo -e "\e[31mKERNEL FAILED TO DECOMPRESS!\e[0m"
      exit 1
      echo -e "\e[32mKernel decompressed succesfully\e[0m"
    # hash the new kernel for checking
    md5sum $CKPATH $DKPATH > $BTPATH/check.md5
    if [ ! $? == 0 ]; then
      echo -e "\e[31mMD5 GENERATION FAILED!\e[0m"
      echo -e "\e[32mMD5 generated succesfully\e[0m"
    exit 0
  • 999_decompress_rpi_kernel

    DPkg::Post-Invoke {"/bin/bash /boot/firmware/auto_decompress_kernel"; };
  • provision

    NB: Make this file executable. I.e. chmod +x provision.

    #!/bin/bash -e
    if [[ $EUID -ne 0 ]]; then
       echo "Error: This script must be run as root"
       exit 1
    if [ -z $1 ] || [ -z $2 ] || [ -z $3 ]; then
      echo "Usage: provision <device> <hostname> <ip-suffix>"
      exit 1
    PASSWORD_HASH=$(mkpasswd --method=SHA-512 --rounds=4096 "$PASSWORD")
    if [ -z "$SKIP_FLASH" ];
      echo "Writing image to /dev/$DEV"
      time xzcat $IMAGE | dd of=/dev/$DEV bs=1M
    echo "Mounting /dev/$DEV"
    mkdir -p $BOOT_VOL $ROOT_VOL
    mount /dev/${DEV}1 $BOOT_VOL
    mount /dev/${DEV}2 $ROOT_VOL
    # copy user boot configuration
    cp usercfg.txt $BOOT_VOL/
    # decompress kernel
    zcat $BOOT_VOL/vmlinuz > $BOOT_VOL/vmlinux
    # add script to automatically decompress kernel on apt install/upgrade
    cp auto_decompress_kernel $BOOT_VOL/
    cp 999_decompress_rpi_kernel $ROOT_VOL/etc/apt/apt.conf.d/
    # insert cgroup configurations
    sed -i "s/rootwait/rootwait cgroup_enable=cpuset cgroup_enable=memory cgroup_memory=1/g" $BOOT_VOL/cmdline.txt
    # user-data for cloud init
    cp user-data $BOOT_VOL/
    sed -i "s/HOSTNAME/${HOSTNAME}/g" $BOOT_VOL/user-data
    sed -i "s|PASSWORD_HASH|${PASSWORD_HASH}|g" $BOOT_VOL/user-data
    # networking
    cp network-config $BOOT_VOL/
    sed -i "s/IPSUFFIX/${IPSUFFIX}/g" $BOOT_VOL/network-config
    sleep 2
    echo "Unmounting /dev/$DEV"
    umount $BOOT_VOL
    umount $ROOT_VOL
    exit 0

Make sure you have downloaded the Ubuntu disk image file from requirements above, and placed it in the same directory as the provision script, so your file layout should look like:

$ tree
├── 999_decompress_rpi_kernel
├── auto_decompress_kernel
├── network-config
├── provision
├── ubuntu-20.04.1-preinstalled-server-arm64+raspi.img.xz
├── user-data
└── usercfg.txt

0 directories, 7 files


After inserting the SSD drive you will need to figure out it’s block device name using lsblk, since the provision script will flash the SSD with the Ubuntu image, so you must be sure you have the correct device name.

For example, on my computer it is sdb but it might be different for you!

You can use lsblk to figure out which device name is correct.

$ lsblk

loop0                    7:0    0 146.6M  1 loop  /snap/code/51
loop1                    7:1    0 143.8M  1 loop  /snap/code/52
sda                      8:0    0 238.5G  0 disk
├─sda1                   8:1    0   487M  0 part  /boot
├─sda2                   8:2    0     1K  0 part
└─sda5                   8:5    0   238G  0 part
  └─sda5_crypt         253:0    0   238G  0 crypt
    ├─ubuntu--vg-root  253:1    0   230G  0 lvm   /
sdb                     11:0    0 128.0G  0 disk

The provision script requires providing the block device name, hostname for the node and the IP suffix.

I am going to have a 4 node Raspberry Pi cluster running Kubernetes, so I settled on host naming convention of rpi-k8s-<role>-<number> for my nodes, where <role> is the role of the node; either “server” or “agent”; and <number> is the instance number; starting from 001 to 999.

I decided to have my server node have the IP, followed by, and for each agent node.

Note: The Ubuntu image filename and password can be overridden by setting the IMAGE and PASSWORD environment variables before running the provision script.


export IMAGE=ubuntu-20.04.1-preinstalled-server-arm64+raspi.img.xz
export PASSWORD=p@ssw0rD

Run the provision script with the following arguments for the server node:

./provision sbd rpi-k8s-server-001 100

I intend to run an HA server in future, so having the number 001 for the first server node makes it consistent for when I add the second server with 002.

And then for each agent node (inserting and ejecting each respective SSD in between):

./provision sbd rpi-k8s-agent-001 101
./provision sbd rpi-k8s-agent-002 102
./provision sbd rpi-k8s-agent-003 103

Finally, connect the SSD drives to each Raspberry Pi node and power them on.

Each node will automatically provision itself and after some time, you will be able to SSH onto them using the k8s user.


ssh k8s@
ssh k8s@
ssh k8s@
ssh k8s@

Next, I’ll write a post on installing Kubernetes on the cluster.

Go Projects



A developer tool for providing DNS for Docker containers running on a local development host.



A prototype project to try out NATS Streaming via a gRPC service, using streaming request/response for publishing and subscribing to messages in golang.

NATS Streaming Demo


Demonstrates running a 6 node NATS and STAN (streaming) cluster using docker-compose.

ASDF Version Manager Plugins

Installing and managing the latest version of a command line tool can be a pain and very time consuming.

As developers we typically want to use the latest and greatest version of a tool before it becomes more widely available via official deployment channels, such as deb or snap on Linux and homebrew on Mac.

Luckily, there is a simple version manager, asdf, which makes life very easy with it’s plugin capabilities.

There are many plugins provided by the community, including for:

  • DotNet Core
  • Crystal
  • GitHub CLI
  • Helm
  • kubectx

And, creating plugins is easy. You need 3 shell scripts, namely:

  • bin/list-all - lists all installable versions
  • bin/download - download source code or binary for the specified version
  • bin/install - installs the specified version

See the official documentation for more details.

I spend a fair amount of time doing technical research, so often times there isn’t a distribution of the command line tool I’m testing out, so I usually resort to spending 10 to 15 minutes whipping up an asdf plugin for the given tool.

This involves creating a new repository on GitHub, from the asdf-plugin-template and then editing the lib/utils.bash file, providing the plugin name and GitHub repo URL of the tool in question.

Here are some of the plugins I’ve created:

  • asdf-aws-vault - AWS Vault is a tool to securely store and access AWS credentials in a development environment.
  • asdf-devspace - DevSpace is the next-generation tool for fast cloud-native software development.
  • asdf-docker-compose - Compose is a tool for defining and running multi-container Docker applications.
  • asdf-docker-machine - Machine lets you create Docker hosts on your computer, on cloud providers, and inside your own data center.
  • asdf-glooctl - Gloo is a feature-rich, Kubernetes-native ingress controller, and next-generation API gateway.
  • asdf-grv - GRV is a terminal interface for viewing git repositories.
  • asdf-helm - Helm is the best way to find, share, and use software built for Kubernetes.
  • asdf-istioctl - Istio configuration command line utility for service operators to debug and diagnose their Istio mesh.
  • asdf-k9s - Kubernetes CLI to manage your clusters in style.
  • asdf-kubebuilder - SDK for building Kubernetes APIs using CRDs.
  • asdf-kubectx - Faster way to switch between clusters and namespaces in kubectl.
  • asdf-skaffold - For easy and repeatable Kubernetes development.
  • asdf-sonarscanner - SonarScanner is the scanner to use when there is no specific scanner for your build system.
  • asdf-tilt - A multi-service dev environment for teams on Kubernetes.

Ruby Projects

Ruby on Rails



A simple orchestration library for running complex processes or workflows in Ruby. Processes are defined using a simple DSL, where the sequences and tasks are defined. Processes can then be queued for execution. Sequences can be synchronous or asynchronous, and the overall process can be monitored for completion or failure.



Provides several useful patterns, packaged in a gem, for use in Rails and other web based applications, including generators to help improve consistency in your applications.

Rails Simple Config


A simple YAML based configuration for Ruby on Rails 3+, which supports shared settings, ERB and more.

Easy Enumerated Type


Provides a simple abstraction for defining enumerations in a Ruby module. Supports listing members and localization of constant names.

Heroku Projects

The following projects are for helping developers deploy their R applications on Heroku. Each include support for the Shiny and Plumber web frameworks and [packrat][packrat] and renv package managers.

Heroku Buildpack for R


A traditional buildpack for deploying R applications on Heroku for the heroku-16 and heroku-18 stacks.

Some example R applications:

Heroku Docker Images for R


Prebuilt Docker images for deploying R applications on Heroku for the container stack.

Some example R applications: