There is no fog, only Clouds

10 August 2023

Running Kubeflow on Akamai/Linode LKE!

by Stefan Krantz

Kubeflow is an open-source platform designed to make it easier to develop, deploy, and manage machine learning (ML) and artificial intelligence (AI) workflows on Kubernetes. By combining Kubernetes and ML tools, Kubeflow simplifies the process of building, training, and deploying ML models at scale. In this blog post, we’ll walk you through the steps to set up Kubeflow on Linode Kubernetes Engine (LKE), an Akamai Connected Cloud product, offering a seamless and powerful Kubernetes experience.

 
 

Introduction to Linode Kubernetes Engine (LKE)

Linode Kubernetes Engine, an Akamai Connected Cloud product, is a managed Kubernetes service that provides a hassle-free way to create, deploy, and manage Kubernetes clusters. It is designed to simplify the complexities of Kubernetes, making it accessible to developers and businesses looking for a robust container orchestration solution. With LKE, you can harness the power of Kubernetes without the overhead of infrastructure management.

More information on the official website.

 
 

Prerequisites

Before diving into the installation process, ensure you have the following prerequisites in place:

  1. Linode Account: You should have an active Linode account. If you don’t have one, sign up at Linode’s official website.

  2. Kubernetes Cluster: Create a Linode Kubernetes Engine cluster by following Linode’s documentation on how to set up LKE. The integration of LKE with Akamai Connected Cloud ensures fast and reliable connectivity. If you are interrested in IaC (terraform/opentofu) you can also follow this guide.

  3. kubectl: Make sure you have kubectl installed, and it’s properly configured to connect to your Linode Kubernetes cluster. Again, some pointers in this guide on how to do that.

  4. Kustomize: Install Kustomize, a tool for managing Kubernetes manifests. You can find installation instructions on the official website.

  5. kfctl: Download the kfctl CLI tool from the Kubeflow releases page. This tool will help you install and manage Kubeflow deployments.

 
 

Step 1: Install Kubeflow

1.1: Download Kubeflow

First, let’s download the Kubeflow manifests. Replace KF_NAME with the name of your choice.

export KF_NAME=my-kubeflow
export BASE_DIR=.
export KF_DIR=${BASE_DIR}/${KF_NAME}

mkdir -p ${KF_DIR}
cd ${KF_DIR}

1.2: Generate the Configuration

Use the kfctl tool to generate a configuration file. Replace KF_NAME with your chosen name.

kfctl build -V -f https://github.com/kubeflow/manifests/archive/master.tar.gz

1.3: Deploy Kubeflow

Deploy Kubeflow using the generated configuration.

kfctl apply -V -f . -n kubeflow

This process might take a few minutes. Once completed, you’ll have Kubeflow running in your Linode Kubernetes cluster, taking advantage of the Akamai Connected Cloud infrastructure for optimized network performance.

 
 

Step 2: Access Kubeflow

2.1: Port Forward the UI

To access the Kubeflow UI from your local machine, run the following command:

kubectl port-forward svc/istio-ingressgateway -n istio-system 8080:80

You can now access the Kubeflow dashboard at http://localhost:8080.

2.2: Authenticate

To log in, you’ll need to use one of the available authentication methods. You can choose from OIDC, Dex, or other options. Set up the authentication method that best suits your needs.

 
 

Step 3: Use Kubeflow

Now that Kubeflow is up and running, you can start using it to build, train, and deploy machine learning models. You can create Jupyter notebooks, set up pipelines, and manage your ML workflows through the Kubeflow UI.

 
 

Conclusion

Running Kubeflow in Linode Kubernetes Engine (LKE), an Akamai Connected Cloud product, offers you a powerful and reliable environment for your machine learning and artificial intelligence projects. Linode Kubernetes Engine simplifies the management of Kubernetes, and when combined with Kubeflow, it provides a robust platform for scalable and efficient ML workloads. By following the steps in this guide, you’ll have Kubeflow set up and ready to use in your Linode Kubernetes cluster, connected to Akamai’s high-performance infrastructure. So, get started and explore the possibilities of Kubeflow in your ML projects on Linode Kubernetes Engine!

 
 

All The best! Stefan

tags: