Tutorial: Creating a Cluster With a Custom CNI Configuration

OKS clusters are deployed with a default Container Network Interface (CNI) to provide network communication between Kubernetes pods.

By default, OKS installs Cilium as the cluster CNI. Cilium is deployed with a full kube-proxy replacement, which means that kube-proxy is not deployed on worker nodes.

Most cluster components are managed by OKS and are not customizable. However, the CNI configuration can be customized during cluster creation. As a result, you can either:

  • Choose the Cilium version to install,

  • Disable the default CNI installation to install and configure your own CNI.

  • Disabling the default CNI installation makes the cluster non-functional until you manually install and configure your own CNI.

  • This option is intended for advanced users only. It is not recommended for novice users, as you must repair and configure the cluster network yourself before workloads can communicate properly.

Understanding CNI Configuration

A Container Network Interface (CNI) is a network plugin used by Kubernetes to provide network connectivity between pods and other cluster components.

Kubernetes requires a CNI plugin so that:

  • Pods can communicate with each other.

  • Pods can communicate with services.

  • Network policies can be applied.

  • Cluster networking can function across worker nodes.

Cilium is installed by default as the cluster CNI. Cilium is a Kubernetes networking plugin that provides network connectivity, service routing, and network policy enforcement for cluster workloads. In OKS, Cilium also replaces kube-proxy, meaning that service routing is handled directly by Cilium instead of by a separate kube-proxy component.

Supported Cilium Versions

You can choose a specific Cilium version supported by OKS during cluster creation.

Supported values are:

  • cilium-1.16

  • cilium-1.17

  • cilium-1.18

  • cilium-1.19

OKS deploys the selected Cilium version and keeps managing it.

Configuring a CNI on Your Cluster

Creating a Cluster With a Specific Cilium Version Using OKS CLI

Before you begin:

To create a cluster with a specific Cilium version, use the cluster create command with the set cni option:

Request sample
$ oks-cli cluster create \
    --profile NAME_OF_PROFILE \
    --cluster-name NAME_OF_CLUSTER \
    --project-name NAME_OF_PROJECT \
    --set cni=CILIUM_VERSION

For more information, see Creating a Cluster and Supporter Cilium Versions.

Installing a CNI Manually on a Cluster

Disabling the Default CNI On Your Cluster

To install and configure your own CNI, first disable the default CNI installation by creating a cluster with the set cni option set to none:

Disabling the default CNI installation is an advanced configuration option:

  • OKS will not install Cilium during cluster creation. The cluster network will thus not be functional until you manually install and configure your own CNI.

  • Pods will not be able to communicate properly, and some cluster components may remain unavailable or unhealthy.

  • You will be responsible for configuring, updating, upgrading, and troubleshooting your CNI. OKS does not manage custom CNI installations.
Request sample
$ oks-cli cluster create \
    --profile NAME_OF_PROFILE \
    --cluster-name NAME_OF_CLUSTER \
    --project-name NAME_OF_PROJECT \
    --set cni=none

For more information, see Creating a Cluster.

Installing Cilium Manually On Your Cluster

Before you begin:

  • Install the Cilium CLI.

  • Make sure your KUBECONFIG points to the cluster created with cni=none:

    Request sample
    $ export KUBECONFIG=$(oks-cli cluster kubeconfig \
  --project-name NAME_OF_PROJECT \
  --cluster-name NAME_OF_CLUSTER \
  --print-path)

Once you have created your cluster with the cni option set to none, you can manually install a Cilium version on it. In this case, you will be responsible for deploying, configuring, updating, and supporting Cilium.

  • 3DS OUTSCALE will not update, upgrade, or reconfigure this CNI installation for you.

  • This section provides an example using Cilium. If you wish to install another CNI, follow the official documentation of the selected network plugin and make sure it provides the networking features required by your cluster.

  1. Install the desired Cilium version:

    Request sample
    $ cilium install --version CILIUM_VERSION

    Cilium automatically detects that kube-proxy is not installed and fully replaces its functionalities.

    For more information, see Supported Cilium Versions.

  2. Apply a node pool manifest to create worker nodes for the cluster:

    Request sample
    $ kubectl apply -f NODEPOOL_MANIFEST.yaml

    • For more information on how to write a node pool manifest, see Node Pool Manifest Reference.

    • Cilium pods remain in Pending state until worker nodes are available in the cluster.

  3. Verify that the worker nodes are in Ready state:

    Request sample
    $ kubectl get nodes

  4. Verify that Cilium is running properly:

    Request sample
    $ cilium status

    Cilium is correctly installed when the Cilium, Operator, and Envoy DaemonSet components are OK.

  5. Verify the network connectivity:

    Request sample
    $ cilium connectivity test

Verifying Your CNI Configuration Using kubectl

  1. After setting it on your cluster one way or the other, you can check the CNI configuration by retrieving your kubeconfig file:

    Request sample
    $ export KUBECONFIG=`oks-cli cluster kubeconfig \
    --project-name NAME_OF_PROJECT \
    --cluster-name NAME_OF_CLUSTER \
    --print-path`

  2. Check the resources deployed in the kube-system namespace:

    Request sample
    $ kubectl get deployment -n kube-system

If the default CNI is installed, Cilium resources will be present in the cluster.

You can also check its status with the following command:

Request sample
$ cilium status

Related Pages

Export to PDF