Getting Started with TKG BYOH

Tanzu Kubernetes Grid, version 1.5.1 introduced support for bring your own host. The feature is in the alpha stage and shouldn’t be used for production deployments as of this post. This capability allows customers to run TKG on bare metal, or a VM they control.

The use case for this could be something like a cell site where a customer doesn’t want to manage a hypervisor or they need all the physical resources to be available to the pods.

The latest information on this feature is available at:

Today only Ubuntu 20.04 is supported, this will expand in the future. This could be bare metal or VMs, in my lab I chose to go the VM route for this testing.

First thing you need is a TKG management cluster. Create this as you normally would. Once it’s up we need to install the BYOH provider. To do this download the clusterctl package from the cluster api repository. Create ~/.cluster-api/clusterctl.yaml open it in your editor of choice and paste this:

  - name: byoh
    type: InfrastructureProvider

Now install the BYOH provider into the management cluster by running:

clusterctl init --infrastructure byoh

Fetching providers
Skipping installing cert-manager as it is already installed
Installing Provider="infrastructure-byoh" Version="v0.1.1" TargetNamespace="byoh-system"

k get po -n byoh-system
NAME                                       READY   STATUS    RESTARTS   AGE
byoh-controller-manager-554fd6c65f-lxvtl   2/2     Running   0          27s

Next we need at least two hosts to use. Again this must be Ubuntu 20.04. Once you have the two hosts up ensure the socat, ebtables, ethtool, and conntrack packages are installed. Download the BYOH agent and mark it executable with chmod +x. Next you need the Kubernetes config file for the management cluster, copy it to the same location as the agent and name it management.conf.

Now run the agent as root (this is required in this release).

sudo ./byoh-hostagent-linux-amd64  -kubeconfig management.conf > byoh-agent.log 2>&1 &

On the management cluster run

 kubectl get byoh

byoh01   65m
byoh02   65m

You should see all the hosts you’ve run the agent on.

Next we need to generate the cluster config for our BYOH cluster.

Export BUNDLE_LOOKUP_TAG=v0.1.0_vmware.3

CONTROL_PLANE_ENDPOINT_IP= clusterctl generate cluster byoh01  —infrastructure byoh --kubernetes-version v1.22.3 --control-plane-machine-count 1 --worker-machine-count 1 > byoh01.yaml

Now apply the config file to the management cluster.

kubectl apply -f byoh01.yaml

You can watch the progress by tailing the byoh-agent.log file. Once you see bootstrap complete on both nodes the process is complete. If you looked at the byoh01.yaml config file you noticed there is no CNI so we need to install one. Download the config file from the management cluster as you normally would. Now change your context to the BYOH cluster and install a CNI such as antrea.

kubectl apply -f

kubectl get nodes

NAME     STATUS   ROLES                  AGE   VERSION
byoh01   Ready    control-plane,master   57m   v1.22.3+vmware.1
byoh02   Ready    <none>                 56m   v1.22.3+vmware.1

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.