In distributed environments, the network is part of the application. Native container networking constructs available in Docker and Kubernetes enable organizations to start their containerization journey with relative ease. However, organizations can easily fail to realize the value-add of a container networking solution and only use primitives for setting up the pipes.
Using basic networking capabilities means the network will eventually become a bottleneck without enterprise-grade mechanisms for scaling up. The good news is that developers and network engineers are not locked into the native networking constructs that come with Docker and Kubernetes.
Container networking innately solves challenges that go beyond connectivity.
- First, it is a foundation for container security by handling segmentation, filtering, access controls, intrusion detection and others.
- Second, for distributed applications, container networking provides a basis for application performance by offering load balancing, observability, diagnostics, and troubleshooting.
- Third, it supports application development by enabling multi-cluster, multi-cloud, and edge connectivity.
In this article, we explore currently available container networking solutions. These can be broadly classified as open source, open source with an enterprise plan, and commercial solutions. To understand the similarities and differences between these three categories, we need to understand some core technical features.
Container Networking Interfaces and Ingress Controllers
While Kubernetes natively provides pod networking and DNS, it does not provide a network interface system by default; this functionality is provided by network plugins. These plugins are Container Network Interfaces (CNIs) and Ingress Controllers. A CNI provides essential layer 2-3 constructs, plus additional low-level features such as network policy enforcement, load balancing, network encryption, and integration with network infrastructure for multi-host and multi-cluster networking. Ingress controllers are responsible for fulfilling incoming requests (north-south traffic), usually with a load balancer, though they may also configure edge routers or additional front-ends to help handle the traffic.
CNIs are a good point of reference for understanding the core capabilities of a container networking solution. Most CNIs are open-source, and most enterprise-grade solutions leverage open-source CNIs to build more advanced capabilities. As such, we note the following:
- Enterprise versions of open source container networking solutions are maintained by the original developers of the open source software.
- Commercial solutions also leverage open source software to build their solutions.
- Commercial solutions can also develop close-sourced CNIs and additional services.
Open source solutions
Open source networking solutions for container-based systems like Kubernetes provide different features and implementations of the CNI, which allow containers to connect with each other and the broader network. These tools handle various aspects of networking, including but not limited to IP addressing, routing, load balancing, network policy enforcement, and service discovery.
Some of the most popular open source solutions available today include:
- Cilium: an open-source project to provide networking, security, and observability for cloud-native environments such as Kubernetes clusters and other container orchestration platforms. At the foundation of Cilium is a new Linux kernel technology called eBPF, which enables the dynamic insertion of powerful security, visibility, and networking control logic into the Linux kernel.
- Project Calico: Calico Open Source is a networking and security solution for containers, virtual machines, and native host-based workloads. It supports a broad range of platforms, including Kubernetes, OpenShift, Docker EE, OpenStack, and bare metal services. Calico can use both an eBPF data plane and the Windows data plane.
- Weave Net: a cloud-native networking toolkit that creates a virtual network for connecting Docker containers across multiple hosts and enables their automatic discovery.
- Antrea: a Kubernetes-native project that implements the CNI and Kubernetes NetworkPolicy, for network connectivity and security of pod workloads. Antrea extends the benefit of programmable networks from Open vSwitch (OVS) to Kubernetes.
As with all open source software, these are free to use – in terms of upfront investment, the cheapest option available. However, additional development and upskilling employees can rapidly dilute the zero upfront costs.
Enterprise versions of open source
Some creators of the open source software solutions – notably Isovalent for Cilium and Tigera for Project Calico – also offer enterprise-grade versions of their solutions.
- Isovalent Enterprise for Cilium – offers additional capabilities such as zero-trust network policies, load balancing, multi-cluster connectivity and automation, segment routing, and automatic and policy creation based on network traffic. Isovalent Enterprise for Cilium is extensively tested, fully backported, and covered by 24×7 support from the builders of eBPF and Cilium.
- Calico Enterprise – the commercial product and extension of Calico open source. It provides the same secure application connectivity across multi-cloud and legacy environments as Calico but adds enterprise control and compliance capabilities for mission-critical deployments. It offers the Calico CNI network plugin, Calico CNI IP address management plugin, overlay network modes, non-overlay network modes, and network policy enforcement.
Opting for an enterprise version means getting support directly from the people who know the software best. They’re more likely to understand the nuances and edge cases that might arise, leading to quicker and more effective problem-solving. Updates to the enterprise features and the open source version are typically synchronized, so any advancements in the open source quickly find their way into the enterprise version as well.
Commercial solutions
Network engineers will see familiar names in the container networking space. It’s worth noting that some of these vendors have container networking capabilities available within a wider solution.
- Arista CloudEOS and CloudVision software provide a consistent operational model for container networking CNIs, private on-premise cloud, public cloud infrastructures, and bare metal environments. Some benefits of CloudEOS for Kubernetes include network operator visibility into what is happening with the container networking environment, real-time analytics for the container network infrastructure, and correlation between the physical network infrastructure, virtual machine hosts, and containerized workloads.
- Juniper’s Contrail Networking is supported as a CNI in Kubernetes environments. Contrail integrated with Kubernetes adds additional networking functionality, including multi-tenancy, network isolation, micro-segmentation with network policies, load-balancing, and more.
- Cisco Intersight Kubernetes Service (IKS) is a lightweight container management platform for delivering multi-cloud production-grade upstream Kubernetes. It simplifies the process of provisioning, securing, scaling, and managing virtualized Kubernetes clusters by providing end-to-end automation, including the integration of networking, load balancers, native dashboards, and storage provider interfaces.
- Cisco Application Centric Infrastructure (ACI) CNI Plugin provides IP Address Management for Pods and Services, Distributed Routing and Switching, and Distributed Firewall for implementing Network Policies.
- VMware Container Networking with Antrea offers users signed images, binaries, and full support for Project Antrea. Container Networking with Antrea has been designed into Tanzu Kubernetes Cluster (TKG) on vSphere and clouds, and Tanzu Kubernetes Cluster Service for running on vSphere with Tanzu. Any customer with a valid license of VMware NSX-T Advanced and above can automatically get support for VMware Container Networking with Antrea for no additional charge.
- F5 BIG-IP Container Ingress Services (CIS) integrates with container orchestration environments to dynamically create L4/L7 services on F5 BIG-IP systems and load balance network traffic across the services. By monitoring the orchestration API server, CIS can modify the BIG-IP system configuration based on changes made to containerized applications.
Compared to the enterprise versions offered by the creators of the open-source software, commercial solutions present a number of benefits, such as vendor incumbency, standardized management, and broader product portfolios. If an organization already has an existing deployment from one of the vendors described above, leveraging their container networking solutions may entail a flick of a switch.
Closing thoughts
There’s a wide range of solutions available on the market. But to truly realize the benefits of the solution, it’s important to reframe the strategy for container networking from a necessary set of pain points to an enabler of secure and robust containerized applications.