Rancher cluster installation on Rocky Linux 8.5 on Dell Wyse 5070

Sun, Feb 6, 2022 4-minute read

Introduction

After having struggled with getting docker+rancher to work properly - I finally have a range of docker machines running - and a single rancher cluster controller node running.

Now comes the cluster installation which basically means joining all my wyse machines into the cluster, so they can be managed in a single place.

Right now I just have a bunch of machines with a working docker installation, which is nice, but quite a management issue.

So they need to be joined to docker.

Being lazy - it needs to be automated.

Automation

I googled a bit and thought it might be possible to add nodes via the UI - but no dice.

But luckily its possible to do via the command like with a bunch of json requests.

I found this page

That I have copied my commands from.

This I have made into a script that I can run on any of my nodes and simply execute it to get the required information.

It all boila down to calling a bunch of http requests and end up with a single docker run command with a lot of arguments that starts the container image docker-agent - which starts and join the node to the cluster and installs a lot of other docker containers that make up the moving part of the cluster.

My script is as follows:

#!/bin/sh

# install jq - since that is not installed per default
sudo dnf install jq -y

PASSWORD="<ADMIN PASSWORD in RANCHER>"

# Log the user on and get a token, so we dont have to pass the password in every single call
LOGINRESPONSE=`curl -s 'https://rancher.root.dom/v3-public/localProviders/local?action=login' -H 'content-type: application/json' --data-binary '{"username":"admin","password":"'$PASSWORD'"}' --insecure`

LOGINTOKEN=`echo $LOGINRESPONSE | jq -r .token`

# Create API key, which will be used in subsequent requests to create the cluster
APIRESPONSE=`curl -s 'https://rancher.root.dom/v3/token' -H 'content-type: application/json' -H "Authorization: Bearer $LOGINTOKEN" --data-binary '{"type":"token","description":"automation"}' --insecure`


# Grab the token from the response and store it in APITOKEN
APITOKEN=`echo $APIRESPONSE | jq -r .token`

# Now we have to make sure that the rancher server is configured correctly with a server url - so it can generate correct docker commands for us

RANCHER_SERVER='https://rancher.root.dom'

curl -s 'https://rancher.root.dom/v3/settings/server-url' -H 'content-type: application/json' -H "Authorization: Bearer $APITOKEN" -X PUT --data-binary '{"name":"server-url","value":"'$RANCHER_SERVER'"}' --insecure

# Create a new cluster where the wyse nodes will be joined - lets call it 'wyse'
CLUSTERRESPONSE=`curl -s 'https://rancher.root.dom/v3/cluster' -H 'content-type: application/json' -H "Authorization: Bearer $APITOKEN" --data-binary '{"dockerRootDir":"/var/lib/docker","enableNetworkPolicy":false,"type":"cluster","rancherKubernetesEngineConfig":{"addonJobTimeout":30,"ignoreDockerVersion":true,"sshAgentAuth":false,"type":"rancherKubernetesEngineConfig","authentication":{"type":"authnConfig","strategy":"x509"},"network":{"type":"networkConfig","plugin":"canal"},"ingress":{"type":"ingressConfig","provider":"nginx"},"monitoring":{"type":"monitoringConfig","provider":"metrics-server"},"services":{"type":"rkeConfigServices","kubeApi":{"podSecurityPolicy":false,"type":"kubeAPIService"},"etcd":{"snapshot":false,"type":"etcdService","extraArgs":{"heartbeat-interval":500,"election-timeout":5000}}}},"name":"wyse"}' --insecure`

# Extract clusterid to use for generating the docker run command
CLUSTERID=`echo $CLUSTERRESPONSE | jq -r .id`

# Create token
curl -s 'https://rancher.root.dom/v3/clusterregistrationtoken' -H 'content-type: application/json' -H "Authorization: Bearer $APITOKEN" --data-binary '{"type":"clusterRegistrationToken","clusterId":"'$CLUSTERID'"}' --insecure > /dev/null 

# Set role flags for our nodes - for now we will just give all nodes the same roles.
ROLEFLAGS="--etcd --controlplane --worker" 

# Generate nodecommand
AGENTCMD=`curl -s 'https://rancher.root.dom/v3/clusterregistrationtoken?id="'$CLUSTERID'"' -H 'content-type: application/json' -H "Authorization: Bearer $APITOKEN" --insecure | jq -r '.data[].nodeCommand' | head -1`

# Concat commands
DOCKERRUNCMD="$AGENTCMD $ROLEFLAGS" 

# Echo command
echo $DOCKERRUNCMD

Should give output similar to:

sudo docker run -d --privileged --name rancher-agent --restart=unless-stopped --net=host -v /etc/kubernetes:/etc/kubernetes -v /var/run:/var/run rancher/rancher-agent:v2.6.3 --server https://rancher.root.dom --token nfffwxfk75h9xfbmslxm2pb84wrj9l576xcl5x49hnjtrw4tkw2sj7 --ca-checksum ada9148a39a7df3398c016bde55fabedc4a55c8b3ce181b59e3f5e011cfad1df --etcd --controlplane --worker

Then its simply as simple and ssh’ing to each of my nodes and execute the above command.

of course if you have more nodes than I do - its probably easier to just do a for loop over all the nodes in your cluster where you ssh to each and run the above command.

Something similar to

#!/bin/sh

command="sudo docker run -d --privileged --name rancher-agent --restart=unless-stopped --net=host -v /etc/kubernetes:/etc/kubernetes -v /var/run:/var/run rancher/rancher-agent:v2.6.3 --server https://rancher.root.dom --token nfffwxfk75h9xfbmslxm2pb84wrj9l576xcl5x49hnjtrw4tkw2sj7 --ca-checksum ada9148a39a7df3398c016bde55fabedc4a55c8b3ce181b59e3f5e011cfad1df --etcd --controlplane --worker"

nodes="wyse1 wyse2 wyse3 wyse4 wyse ...."
for node in $nodes
do
  ssh username@$node "$command"
done

Result

After a couple of minutes depending on the speed of the nodes they should be joined to the cluster and you should have a nice view like this:

Cluster Nodes Joined