Tanzu Kubernetes Grid, version 1.5.1 introduced support for bring your own host. The feature is in the alpha stage and shouldn’t be used for production deployments as of this post. This capability allows customers to run TKG on bare metal, or a VM they control.
The use case for this could be something like a cell site where a customer doesn’t want to manage a hypervisor or they need all the physical resources to be available to the pods.
The latest information on this feature is available at: https://github.com/vmware-tanzu/cluster-api-provider-bringyourownhost
Today only Ubuntu 20.04 is supported, this will expand in the future. This could be bare metal or VMs, in my lab I chose to go the VM route for this testing.
First thing you need is a TKG management cluster. Create this as you normally would. Once it’s up we need to install the BYOH provider. To do this download the clusterctl package from the cluster api repository. Create ~/.cluster-api/clusterctl.yaml open it in your editor of choice and paste this:
providers:
- name: byoh
url: https://github.com/vmware-tanzu/cluster-api-provider-bringyourownhost/releases/latest/infrastructure-components.yaml
type: InfrastructureProvider
Now install the BYOH provider into the management cluster by running:
clusterctl init --infrastructure byoh Fetching providers Skipping installing cert-manager as it is already installed Installing Provider="infrastructure-byoh" Version="v0.1.1" TargetNamespace="byoh-system" k get po -n byoh-system NAME READY STATUS RESTARTS AGE byoh-controller-manager-554fd6c65f-lxvtl 2/2 Running 0 27s
Next we need at least two hosts to use. Again this must be Ubuntu 20.04. Once you have the two hosts up ensure the socat, ebtables, ethtool, and conntrack packages are installed. Download the BYOH agent https://github.com/vmware-tanzu/cluster-api-provider-bringyourownhost/releases/download/v0.1.0/byoh-hostagent-linux-amd64 and mark it executable with chmod +x. Next you need the Kubernetes config file for the management cluster, copy it to the same location as the agent and name it management.conf.
Now run the agent as root (this is required in this release).
sudo ./byoh-hostagent-linux-amd64 -kubeconfig management.conf > byoh-agent.log 2>&1 &
On the management cluster run
kubectl get byoh NAME AGE byoh01 65m byoh02 65m
You should see all the hosts you’ve run the agent on.
Next we need to generate the cluster config for our BYOH cluster.
Export BUNDLE_LOOKUP_TAG=v0.1.0_vmware.3 CONTROL_PLANE_ENDPOINT_IP=172.16.108.248 clusterctl generate cluster byoh01 —infrastructure byoh --kubernetes-version v1.22.3 --control-plane-machine-count 1 --worker-machine-count 1 > byoh01.yaml
Now apply the config file to the management cluster.
kubectl apply -f byoh01.yaml
You can watch the progress by tailing the byoh-agent.log file. Once you see bootstrap complete on both nodes the process is complete. If you looked at the byoh01.yaml config file you noticed there is no CNI so we need to install one. Download the config file from the management cluster as you normally would. Now change your context to the BYOH cluster and install a CNI such as antrea.
kubectl apply -f https://raw.githubusercontent.com/antrea-io/antrea/main/build/yamls/antrea.yml kubectl get nodes NAME STATUS ROLES AGE VERSION byoh01 Ready control-plane,master 57m v1.22.3+vmware.1 byoh02 Ready <none> 56m v1.22.3+vmware.1
