Building a Kubernetes Cluster on Red Hat Enterprise Linux...

Introduction

In this post, I'll walk you through deploying a production-ready Kubernetes cluster on Red Hat Enterprise Linux 10 using kubeadm. This lab was inspired by Anthony E. Nocentino's excellent Certified Kubernetes Administrator (CKA): Using kubadm to Install a Basic Cluster training course, which is part of the official Certified Kubernetes Administrator (CKA) path on Pluralsight.

⭐ Shout-out: Anthony is a fantastic trainer! His course uses Ubuntu 22.04 as the base OS. I adapted his approach to work on RHEL 10, adding some additional considerations specific to Red Hat's ecosystem.

One intentional decision in this setup: I deployed Kubernetes v1.35 and CRI-O v1.35, which wasn't the latest version available at installation time.

This was purposeful. Anthony's course includes a dedicated section on upgrading clusters, and using a slightly older baseline makes that learning path clearer.

The upgrade procedures (not covered here) are what really solidify your understanding of cluster lifecycle management.

Lab Infrastructure Overview

Nodes Configuration

Node	Role	RAM	vCPUs	IP Address
rh-cp1	Control Plane	12 GiB	2	192.168.110.120
rh-node1	Worker	6 GiB	2	192.168.110.121
rh-node2	Worker	6 GiB	2	192.168.110.122
rh-node3	Worker	6 GiB	2	192.168.110.123

Note: The IP address schema is just an example and what was more convenient for me.

Supporting Infrastructure

A dedicated utilities VM (also RHEL 10) provides essential services:

DNS (BIND/named)
NTP (chrony)
HTTP (Apache/httpd)
DHCP (Kea)

This centralized infrastructure simplifies name resolution across all cluster nodes. But this is not essential for this project. You can, instead, ensure the nodes are able to reach each other updating the file /etc/hosts on all nodes.

Prerequisites & OS Preparation

Before diving into Kubernetes, we need consistent node preparation across all machines.

1. System Registration and Updates

$ sudo subscription-manager register --username <username> --password <password>
$ sudo dnf update redhat-release
$ sudo dnf upgrade
$ sudo reboot

2. Disable Swap (Required by Kubernetes)

Edit /etc/fstab to comment out swap entries:

UUID=xxxxxxxx-xxx-xxxx-xxxx-xxxxxxxxxxxx none swap defaults 0 0
# ^ Comment this line out

Verify:

$ sudo swapoff -a
$ free

3. Disable Firewalld

Disable firewalld, as indicated in the Calico System requirements for Kubernetes:

$ sudo systemctl stop firewalld
$ sudo systemctl disable firewalld
$ sudo systemctl mask firewalld

⚠️ Production Note: Use Calico to maintaining security and enforce network policies later.

4. Load Kernel Modules and Enable IP Forwarding

$ cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF

$ sudo modprobe overlay
$ sudo modprobe br_netfilter

Configure sysctl parameters:

$ cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
EOF

$ sudo sysctl --system

Verify the modules loaded correctly:

$ lsmod | grep overlay
$ lsmod | grep br_netfilter

Installing Kubernetes Components

Setting Version Variables

$ KUBERNETES_VERSION=v1.35
$ CRIO_VERSION=v1.35

Adding Repositories

Create the Kubernetes repo:

$ cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://pkgs.k8s.io/core:/stable:/$KUBERNETES_VERSION/rpm/
enabled=1
gpgcheck=1
gpgkey=https://pkgs.k8s.io/core:/stable:/$KUBERNETES_VERSION/rpm/repodata/repomd.xml.key
EOF

Create the CRI-O repo:

$ cat <<EOF | sudo tee /etc/yum.repos.d/cri-o.repo
[cri-o]
name=CRI-O
baseurl=https://download.opensuse.org/repositories/isv:/cri-o:/stable:/$CRIO_VERSION/rpm/
enabled=1
gpgcheck=1
gpgkey=https://download.opensuse.org/repositories/isv:/cri-o:/stable:/$CRIO_VERSION/rpm/repodata/repomd.xml.key
EOF

Installing Packages

$ sudo dnf install -y kubelet kubeadm kubectl cri-o container-selinux

Configuring CRI-O

Create a cgroup manager configuration file:

$ cat <<EOF | sudo tee /etc/crio/crio.conf.d/02-cgroup-manager.conf
[crio.runtime]
conmon_cgroup = "pod"
cgroup_manager = "cgroupfs"
EOF

Enable and start services:

$ sudo systemctl enable --now crio kubelet
$ sudo systemctl restart crio

Version Locking

To prevent accidental upgrades:

$ sudo dnf install 'dnf-command(versionlock)'
$ sudo dnf versionlock add kubeadm-1.35.4 kubelet-1.35.4 kubectl-1.35.4 cri-o-1.35.2

Note: Review the output from the installation of the packages kubeadm, kubelet, kubectl and cri-o, and update the versions to lock in the command above.

Initializing the Control Plane

On rh-cp1, download and configure Calico networking. To know the current lastest version, check Tigera documentation, in the Manifest tab:

$ wget https://raw.githubusercontent.com/projectcalico/calico/v3.32.0/manifests/calico.yaml

Edit the CALICO_IPV4POOL_CIDR value to match your pod network plan:

- name: CALICO_IPV4POOL_CIDR
  value: "10.244.0.0/16"

Initialize the cluster, to use the same subnet:

$ sudo kubeadm init \
  --kubernetes-version v1.35.4 \
  --pod-network-cidr=10.244.0.0/16 \
  --cri-socket unix:///var/run/crio/crio.sock \
  --upload-certs

Once successful, save the join commands that appear at the end of the output—you'll need these for worker nodes!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.110.120:6443 --token xxxxxx.xxxxxxxxxxxxxxxx \
 --discovery-token-ca-cert-hash sha256:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx  
$

Your Kubernetes control-plane has initialized successfully!

Configuring kubectl

$ mkdir -p $HOME/.kube
$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
$ sudo chown $(id -u):$(id -g) $HOME/.kube/config

Deploy Calico Network

$ kubectl apply -f calico.yaml

Wait a few minutes and verify pods are running:

$ kubectl get pods --all-namespaces
$ kubectl get nodes

Expected output:

$ kubectl get pods --all-namespaces
NAMESPACE     NAME                                       READY   STATUS    RESTARTS   AGE
kube-system   calico-kube-controllers-6b4b6457d5-p98c2   1/1     Running   0          2m47s
kube-system   calico-node-5vjrr                          1/1     Running   0          2m47s
kube-system   coredns-7d764666f9-r4vqp                   1/1     Running   0          4m9s
kube-system   coredns-7d764666f9-vh7df                   1/1     Running   0          4m9s
kube-system   etcd-rh-cp1                                1/1     Running   0          4m28s
kube-system   kube-apiserver-rh-cp1                      1/1     Running   0          4m28s
kube-system   kube-controller-manager-rh-cp1             1/1     Running   0          4m27s
kube-system   kube-proxy-4r6h8                           1/1     Running   0          4m10s
kube-system   kube-scheduler-rh-cp1                      1/1     Running   0          4m28s
$ kubectl get nodes
NAME     STATUS   ROLES           AGE     VERSION
rh-cp1   Ready    control-plane   4m40s   v1.35.4
$

Joining Worker Nodes

On each worker node (rh-node1, rh-node2, rh-node3), run the join command saved during kubeadm init:

$ sudo kubeadm join 192.168.110.120:6443 \
  --token <token> \
  --discovery-token-ca-cert-hash sha256:<hash>

Verify the cluster health from the control plane:

$ kubectl get nodes

All nodes should show Ready status.

$ kubectl get nodes
NAME       STATUS   ROLES           AGE     VERSION
rh-cp1     Ready    control-plane   10m     v1.35.4
rh-node1   Ready    <none>          2m11s   v1.35.4
rh-node2   Ready    <none>          113s    v1.35.4
rh-node3   Ready    <none>          102s    v1.35.4
$

Install bash completion for kubectl:

$ sudo dnf install bash-completion
$ echo "source <(kubectl completion bash)" >> ~/.bashrc
$ source ~/.bashrc

Testing the Deployment

Deploy a test application:

$ kubectl create deployment hello-world --image=psk8s.azurecr.io/hello-app:1.0
$ kubectl get pods -o wide

Expose it via a service:

$ kubectl expose deployment hello-world --port=80 --target-port=8080
$ kubectl get service hello-world

For example:

$ kubectl get service hello-world
NAME          TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
hello-world   ClusterIP   10.99.168.218   <none>        80/TCP    8s
$

Then, test it. You should see a response from your running container!

$ curl http://10.99.168.218:80
Hello, world!
Version: 1.0.0
hello-world-b5b7f67cc-d26dt
$

Clean up after testing:

$ kubectl delete service hello-world
$ kubectl delete deployment hello-world

Key Considerations When Moving from Ubuntu to RHEL

Here are the main differences I encountered adapting Anthony's tutorial:

Aspect	Ubuntu Approach	RHEL 10 Adaptation
Package Manager	apt/dpkg	dnf/rpm
Firewall Management	ufw/firewalld optional	firewalld disabled (use Calico policies)
Subscription	N/A	subscription-manager required
SELinux	Permissive mode default	Need to handle SELinux context
CRI Runtime	containerd	CRI-O

What's Next?

This setup provides a solid foundation for learning Kubernetes administration. From here, you could explore:

Cluster upgrades (covered extensively in Anthony's course)
Network policy enforcement with Calico
High availability with multiple control plane nodes
Storage classes and persistent volumes
Monitoring stack with Prometheus/Grafana

If you found this walkthrough helpful, I'd highly recommend checking out the original Pluralsight course. Anthony's explanations are crystal clear, and adapting them to different distributions is an excellent way to deepen your understanding of what happens under the hood.

Resources

Pluralsight: Certified Kubernetes Administrator Certification Path
Official Kubernetes Documentation
Calico Project GitHub

Thanks for reading! Feel free to share your own experiences with Kubernetes on RHEL in the comments below.

推荐订阅源

DEV Community