Integration Guide

This document explains the components necessary to install Calico on Kubernetes for integrating with custom configuration management.

The manifests we provide in Installing Calico for policy and networking, Installing Calico for policy and flannel for networking, and Installing Calico for policy will perform these steps automatically for you and are strongly recommended for most users. These instructions should only be followed by users who have a specific need that cannot be met by using manifests.

Before you begin

Ensure that your cluster meets the Calico system requirements.

About the Calico components

There are three components of a Calico / Kubernetes integration.

  • The Calico per-node Docker container calico/node.
  • The cni-plugin network plugin binaries. This is the combination of two binary executables and a configuration file.
  • The Calico Kubernetes controllers, which run in a single-instance pod. These components monitor the Kubernetes API to keep Calico in sync.

The calico/node docker container must be run on the Kubernetes master and each Kubernetes node in your cluster. It contains the BGP agent necessary for Calico routing to occur, and the Felix agent which programs network policy rules.

The cni-plugin plugin integrates directly with the Kubernetes kubelet process on each node to discover which pods have been created, and adds them to Calico networking.

The calico/kube-controllers container runs as a pod on top of Kubernetes and keeps Calico in-sync with Kubernetes.

Installing calico/node

Run calico/node and configure the node.

The Kubernetes master and each Kubernetes node require the calico/node container. Each node must also be recorded in the Calico datastore.

The calico/node container can be run directly through Docker, or it can be done using the calicoctl utility.

# Download and install calicoctl
wget 
sudo chmod +x calicoctl

# Run the calico/node container
sudo ETCD_ENDPOINTS=http://<ETCD_IP>:<ETCD_PORT> ./calicoctl node run --node-image=quay.io/calico/node:v3.3.1

See the calicoctl node run documentation for more information.

Example systemd unit file (calico-node.service)

If you’re using systemd as your init system then the following service file can be used.

[Unit]
Description=calico-node
After=docker.service
Requires=docker.service

[Service]
User=root
Environment=ETCD_ENDPOINTS=http://<ETCD_IP>:<ETCD_PORT>
PermissionsStartOnly=true
ExecStart=/usr/bin/docker run --net=host --privileged --name=calico-node \
  -e ETCD_ENDPOINTS=${ETCD_ENDPOINTS} \
  -e NODENAME=${HOSTNAME} \
  -e IP= \
  -e IP6= \
  -e AS= \
  -e NO_DEFAULT_POOLS= \
  -e CALICO_LIBNETWORK_ENABLED=false \
  -e CALICO_NETWORKING_BACKEND=bird \
  -e FELIX_DEFAULTENDPOINTTOHOSTACTION=ACCEPT \
  -v /lib/modules:/lib/modules \
  -v /run/docker/plugins:/run/docker/plugins \
  -v /var/run/calico:/var/run/calico \
  -v /var/log/calico:/var/log/calico \
  -v /var/lib/calico:/var/lib/calico \
  quay.io/calico/node:v3.3.1
ExecStop=/usr/bin/docker rm -f calico-node
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target

Replace <ETCD_IP>:<ETCD_PORT> with your etcd configuration.

Note: To ensure reasonable dataplane programming latency on a system under load, calico/node requires a CPU reservation of at least 0.25 cores with additional benefits up to 0.5 cores.

Installing the Calico CNI plugins

The Kubernetes kubelet should be configured to use the calico and calico-ipam plugins.

Install the Calico plugins

Download the binaries and make sure they’re executable.

wget -N -P /opt/cni/bin https://github.com/projectcalico/cni-plugin/releases/download/v3.3.1/calico
wget -N -P /opt/cni/bin https://github.com/projectcalico/cni-plugin/releases/download/v3.3.1/calico-ipam
chmod +x /opt/cni/bin/calico /opt/cni/bin/calico-ipam

The Calico CNI plugins require a standard CNI config file. The policy section is only required when running the calico/kube-controllers container .

mkdir -p /etc/cni/net.d
cat >/etc/cni/net.d/10-calico.conf <<EOF
{
    "name": "calico-k8s-network",
    "cniVersion": "0.6.0",
    "type": "calico",
    "etcd_endpoints": "http://<ETCD_IP>:<ETCD_PORT>",
    "log_level": "info",
    "ipam": {
        "type": "calico-ipam"
    },
    "policy": {
        "type": "k8s"
    },
    "kubernetes": {
        "kubeconfig": "</PATH/TO/KUBECONFIG>"
    }
}
EOF

Replace <ETCD_IP>:<ETCD_PORT> with your etcd configuration. Replace </PATH/TO/KUBECONFIG> with your kubeconfig file. See Kubernetes kubeconfig for more information about kubeconfig.

For more information on configuring the Calico CNI plugins, see the configuration guide

Install standard CNI loopback plugin

In addition to the CNI plugin specified by the CNI config file, Kubernetes requires the standard CNI loopback plugin.

Download the file loopback and copy it to the CNI binary directory.

wget https://github.com/containernetworking/cni/releases/download/v0.6.0/cni-v0.6.0.tgz
tar -zxvf cni-v0.6.0.tgz
sudo cp loopback /opt/cni/bin/

Installing the Calico Kubernetes controllers

The calico/kube-controllers container keeps Calico’s datastore in-sync with Kubernetes. It runs as a single pod managed by a Deployment.

Note: The calico/kube-controllers container is required even if policy is not in use.

To install the controllers:

$ kubectl create -f calico-kube-controllers.yaml

After a few moments, you should see the controllers enter Running state:

$ kubectl get pods --namespace=kube-system
NAME                                     READY     STATUS    RESTARTS   AGE
calico-kube-controllers                  1/1       Running   0          1m

For more information on how to configure the controllers, see the configuration guide.

Role-based access control (RBAC)

When installing Calico on Kubernetes clusters with RBAC enabled, it is necessary to provide Calico access to some Kubernetes APIs. To do this, subjects and roles must be configured in the Kubernetes API and Calico components must be provided with the appropriate tokens or certificates to present which identify it as the configured API user.

Detailed instructions for configuring Kubernetes RBAC are outside the scope of this document. For more information, please see the upstream Kubernetes documentation on the topic.

The following YAML file defines the necessary API permissions required by Calico when using the etcd datastore.

kubectl apply -f https://docs.projectcalico.org/v3.3/getting-started/kubernetes/installation/rbac.yaml

Click here to view the above yaml directly.