Deploying On-prem Kubernetes

Deploying On-prem Kubernetes

Base OS - CentOS 7 running on Microsoft Hyper V

To run Kubernetes your host must have at least 2 x CPUs

Master Node

Once you have deployed your base operating system, get the OS updated:

yum update -y

Disable selinux

setenforce 0
sed -i --follow-symlinks 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux

Although you might be usign external DNS, it's best to have the cluster nodes set in your host file on each node, replace these values as required:

cat <<EOF > /etc/hosts
x.x.x.x k8s-master
y.y.y.y k8s-worker-node1
z.z.z.z k8s-worker-node2
EOF

Set hostname

hostnamectl set-hostname k8s-master
exec bash

Disable selinux

setenforce 0
sed -i --follow-symlinks 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux

Add firewall rules

firewall-cmd --permanent --add-port=6443/tcp
firewall-cmd --permanent --add-port=2379-2380/tcp
firewall-cmd --permanent --add-port=10250/tcp
firewall-cmd --permanent --add-port=10251/tcp
firewall-cmd --permanent --add-port=10252/tcp
firewall-cmd --permanent --add-port=10255/tcp
firewall-cmd --reload

Networking Bridge

modprobe br_netfilter
echo '1' > /proc/sys/net/bridge/bridge-nf-call-iptables

Disable Swap

swapoff -a

Comment out swap partitions or files from fstab

 nano /etc/fstab

Configure the Kubernetes Repo

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg
        https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
EOF

Install Kubeadm and Docker

yum install kubeadm docker -y

Start the services

systemctl restart docker && systemctl enable docker
systemctl  restart kubelet && systemctl enable kubelet

Initialise the Kubernetes Master

kubeadm init

You should see that the master has been initialized successfully and be given a token to join worker nodes, make a note of this and keep it safe!

Configure kubectl

mkdir -p $HOME/.kube

cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

chown $(id -u):$(id -g) $HOME/.kube/config

You should now be able to run:

kubectl get nodes

and

kubectl get pods --all-namespaces

You will notice that you have pods stuck pending. We need to deploy the overlay network.

Deploying overlay network

export kubever=$(kubectl version | base64 | tr -d '\n')
kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$kubever"

By issuing

kubectl get pods

You should see all pods are now running.

Worker Nodes

Once you have deployed your base operating system, get the OS updated:

yum update -y

Disable selinux

setenforce 0
sed -i --follow-symlinks 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux

Although you might be usign external DNS, it's best to have the cluster nodes set in your host file on each node, replace these values as required:

cat <<EOF > /etc/hosts
x.x.x.x k8s-master
y.y.y.y k8s-worker-node1
z.z.z.z k8s-worker-node2
EOF

Set hostname

hostnamectl set-hostname k8s-worker-node1
exec bash

Disable selinux

setenforce 0
sed -i --follow-symlinks 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux

Firewall Rules

firewall-cmd --permanent --add-port=10250/tcp
firewall-cmd --permanent --add-port=10255/tcp
firewall-cmd --permanent --add-port=30000-32767/tcp
firewall-cmd --permanent --add-port=6783/tcp
firewall-cmd  --reload

Networking

modprobe br_netfilter
echo '1' > /proc/sys/net/bridge/bridge-nf-call-iptables

Configure the Kubernetes Repo

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg
        https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
EOF

Install Kubeadm and Docker

yum install kubeadm docker -y

Start the services

systemctl restart docker && systemctl enable docker
systemctl  restart kubelet && systemctl enable kubelet

Disable Swap

swapoff -a

Comment out swap partitions or files from fstab

 nano /etc/fstab

Joining the worker node to the master

Using the token you noted earlier, with the master IP, run the following

kubeadm join --token a3bd48.1bc42347c3b35851 mas.t.er.ip:6443

All being well the machine will join the cluster and you will see it's status by running:

kubectl get nodes
NAME               STATUS   ROLES    AGE   VERSION
k8s-master         Ready    master   31m   v1.17.3
k8s-worker-node1   Ready    <none>   82s   v1.17.3
Show Comments