Kubernetes is the de facto container orchestration platform used to deploy, orchestrate, and manage cloud-native containerized applications. As a complex system with many moving parts, components and variables, Kubernetes must be correctly configured to orchestrate and scale containerized applications as efficiently as possible.  

Load balancing is one of Kubernetes' moving parts. It is a vital cog in the Kubernetes ecosystem and deserves an in-depth look into its inner workings and its role in making containerized applications robust, reliable, and easily accessible.

However, before we dive into this topic, let’s take a brief look at what containerization is and why it is needed.

Containerization: What and Why?

The authors of the academic paper, Machine Learning-based Orchestration of Containers: A Taxonomy and Future Directions, describe containerization as follows:

“Containerization is a lightweight application virtualization technology, providing high environmental consistency, operating system distribution portability, and resource isolation.”

Moreover, the authors of this paper (Zhiheng Zhong et al.) note that organizations (and public cloud service providers like Amazon and Google) use container technologies and container orchestration platforms like Kubernetes to manage their distributed systems for automated cloud-native application management.

In practice, a cloud-native application is modular, consisting of multiple containerized microservices, which must be coordinated to ensure that the application executes as expected without falling over because the microservices are spun up in the wrong order or cannot communicate with each other.

Kubernetes & Load Balancing: Distributing Traffic Among Microservices

Orchestrating containerized microservices in a cloud-native application is only one of Kubernetes’ functions. Another is making sure that network traffic is distributed evenly between microservices in use, ensuring high availability and efficient resource utilization.

Before we describe how Kubernetes implements load balancing, let’s look at what load balancing is.

What is Load Balancing?

Wikipedia.com describes load balancing as:
“The process of distributing a set of tasks over a set of resources (computing units), with the aim of making their overall processing more efficient.”

In other words, load balancing looks at the available resources and the total number of tasks, such as users interrogating a database through a web application’s UI (user interface), and decides how best to distribute the tasks among the available resources.

The principles and practice of load balancing are fundamental to networking and infrastructure management, designed to efficiently distribute tasks or requests among multiple resources, such as servers or containers, guaranteeing optimal resource utilization, responsiveness, and availability.

In summary, load balancing is possible at different levels within a technology stack, including:

• Network Load Balancing: At the network level, load balancers distribute incoming network traffic (like HTTP requests) to multiple servers or pods. This is often used in web applications to distribute user requests across multiple servers.
• Application Load Balancing: Load balancing can also occur at the application layer, where a load balancer routes requests based on the type of request (like routing API requests to the relevant backend services).
• Database Load Balancing: For database applications, load balancing can distribute database queries or CRUD (create, read, update, delete) operations across multiple database servers or replicas to optimize database performance, ensuring constant data availability.

Kubernetes and Load Balancing

One of the levels where load balancing occurs that is not described above, is at a container orchestration level. In container orchestration platforms, like Kubernetes, the load balancer ensures that incoming traffic is distributed evenly among pods or containerized microservices, maintaining high availability and scalability.

Consequently, Kubernetes and load balancing are commonly used together to address a variety of use cases, particularly in modern, cloud-native application deployments. Here are several use cases highlighting how Kubernetes and load balancing work together:

1. High-Traffic Web Applications

High-traffic web applications such as big eCommerce sites (like Amazon, Netflix, and Spotify) during the annual Black Friday sales must be able to handle extremely high volumes of user traffic at any given point in time.

Kubernetes scales multiple replicas of the application across multiple nodes. However, there is no guarantee that the incoming traffic will be evenly distributed across the replicas without using a load balancer to distribute incoming web traffic evenly among these replica containers, ensuring a robust, highly available application.

2. Containerized Microservices Architectures

Cloud-native applications are typically developed using containerized microservices architectures.

As with the high-traffic web application use case highlighted above, Kubernetes orchestrates, and auto-scales containers based on demand in relation to the available resources (specified in Kubernetes’ configurations). Kubernetes’ load balancer also routes traffic to the necessary microservices based on the functionality required, making sure that each microservice remains responsive and scales independently.

For instance, imagine you have a FinTech web application, a payment gateway for eCommerce shopping carts. This app is divided up into the following microservices, each in a container:

• Payment gateway microservice
• User authentication service
• Transaction processing service
• Notification microservice
• Fraud detection microservice

Each microservice is spun up and scaled in the order of demand and the number of users accessing a particular microservice at a given moment. Suppose a payment gateway container can handle 10 concurrent user requests, and 30 requests are coming through simultaneously.

Kubernetes will scale the application by spinning up two additional containers (of the payment gateway microservice). The Kubernetes load balancer will then direct the network traffic to each container, making sure a single container handles no more than 10 user requests.

3. Containerized APIs and Backend Services

Containerized APIs and backend services that support mobile or front-end web applications play a critical role in modern software development. This facilitates communication between the client (mobile or web app) and the server, allowing data retrieval, storage, and execution of business logic.

Kubernetes deploys, orchestrates, and scales these APIs and backend services in containers. Its load balancer distributes traffic to these services, ensuring responsive API endpoints and high availability of backend services.

For instance, a sports watch mobile app relies heavily on API endpoints and backend services to provide users with accurate and comprehensive sports tracking and performance analysis, including:

• Activity tracking service
• GPS and location service
• User profile and account service
• Wearable device integration
• User authentication service

When a user completes an activity such as a 6-mile run, the watch syncs with the smartphone app. Once the watch data has been uploaded to the phone app, the app connects to these backend services and APIs, uploading the data to the user’s profile in the cloud.

Kubernetes scales up the different containerized microservices as required while the data is uploaded from the mobile app to the cloud. At the same time, the load balancer distributes the traffic to multiple instances of the same services, ensuring that the APIs and the backend services of this watch brand’s web application remain highly available and do not crash because one or more services are overloaded and fall over.

Final Thoughts

Lastly, utilizing load balancing in theory and practice is a mandatory part of successfully orchestrating cloud-native containerized applications, playing a pivotal role in ensuring accessible, robust, scalable, highly available applications.

In practice, Kubernetes (and other cloud-native container orchestration platforms) include native (integrated) load-balancing capabilities. For example, Kubernetes uses its service objects to enact load balancing for the containers in its cluster. It simplifies deploying, scaling, and managing containerized applications while ensuring the cloud-native application is highly available and responsive throughout its lifecycle.

In summary, load balancing is a theoretical concept and a practical necessity when orchestrating cloud-native containerized applications. It is fundamental to designing and deploying resilient and scalable applications in a modern cloud-native environment.