Manual Installation of Calico with rkt
This tutorial describes how to manually configure a working environment for a cluster of Calico-rkt enabled nodes.
- System requirements
- About the Calico Components
- Installing
calico/node
- Installing calicoctl
- Installing Calico as a CNI plugin
- Next steps
System requirements
- A cluster of bare metal servers or VMs (nodes) with a modern 64-bit Linux OS and IP connectivity between them.
- A recent version of
rkt
installed on each node in the cluster. We recommend version > 1.20.0 as this is contains important fixes that prevent leaking IP addresses when containers are deleted. - An
etcd
cluster accessible by all nodes in the cluster
About the Calico Components
There are three components of a Calico / rkt integration.
- The Calico per-node rkt container, calico/node
- The
calicoctl
command line tool. - The calico-cni network plugin binaries.
- This is the combination of two binary executables and a configuration file.
The calico/node
docker container must be run on each node in your cluster. It contains
the BGP agent which provides Calico routing, and the Felix agent which programs network policy
rules.
The calicoctl
binary is a command line utility that can be used to manage network policy
for your rkt containers, and can be used to monitor the status of your Calico services.
The calico-cni
network plugin binaries are a combination of two binary executables.
These binaries are invoked from the rkt container lifecycle hooks on each node to configure
the container interfaces, manage IP addresses and enable Calico policy on the containers.
Installing calico/node
Prepare host directory structure
The calicoctl
binary uses certain known directories for service diagnostics and
status discovery. In addition, this tutorial assumes binaries and CNI network
configuration will be placed in a particular location.
Ensure the following directories are created by running the following commands on each node:
mkdir -p /var/run/calico
mkdir -p /var/log/calico
mkdir -p /opt/bin
mkdir -p /etc/rkt/net.d
Run calico/node
and configure the node.
Each Calico-rkt enabled node requires the calico/node
container to be running.
The calico/node container can be run directly through rkt and needs to be run as as fly stage-1 container.
sudo rkt run --stage1-path=/usr/share/rkt/stage1-fly.aci \
--set-env=ETCD_ENDPOINTS=http://<ETCD_IP>:<ETCD_PORT> \
--set-env=IP=autodetect \
--insecure-options=image \
--volume=birdctl,kind=host,source=/var/run/calico,readOnly=false \
--mount=volume=birdctl,target=/var/run/calico \
--volume=mods,kind=host,source=/lib/modules,readOnly=false \
--mount=volume=mods,target=/lib/modules \
--volume=logs,kind=host,source=/var/log/calico,readOnly=false \
--mount=volume=logs,target=/var/log/calico \
--net=host \
quay.io/calico/node:v1.0.2 &
Replace
<ETCD_IP>:<ETCD_PORT>
with your etcd configuration. TheETCD_ENDPOINTS
environment may contain a comma separated list of endpoints of your etcd cluster. If the environment is omitted, Calico defaults to a single etcd endpoint at http://127.0.0.1:2379.
You can check that it’s running using sudo rkt list
.
$ sudo rkt list
UUID APP IMAGE NAME STATE CREATED STARTED NETWORKS
b52bba11 node quay.io/calico/node:v1.0.2 running 10 seconds ago 10 seconds ago
Installing calicoctl
Download the calicoctl binary:
sudo wget -O /usr/local/bin/calicoctl https://www.projectcalico.org/builds/calicoctl
sudo chmod +x calicoctl
The calicoctl
documentation has commandline options and configuration.
Running calicoctl as a container
Alternatively to downloading the calicoctl
binary it can be run as a container.
sudo rkt run quay.io/calico/ctl --exec /calicoctl -- version
To also specify the ETCD_ENDPOINTS use:
sudo rkt run --set-env=ETCD_ENDPOINTS="http://etcd1:2379" quay.io/calico/ctl --exec /calicoctl -- version
Installing Calico as a CNI plugin
To install Calico as a CNI plugin used by rkt, we need to first install the actual plugin binaries, and then once installed create any CNI networks that you require with the appropriate Calico CNI plugin references.
Install the Calico plugin binaries
Download the binaries and make sure they’re executable. We download to the
/etc/rkt/net.d
directory since it is one of the default locations that rkt uses
for config discovery. You may change the location and override the rkt configuration
if desired.
wget -N -P /etc/rkt/net.d https://github.com/projectcalico/calico-cni/releases/download/v1.5.6/calico
wget -N -P /etc/rkt/net.d https://github.com/projectcalico/calico-cni/releases/download/v1.5.6/calico-ipam
chmod +x /etc/rkt/net.d/calico /etc/rkt/net.d/calico-ipam
The Calico CNI plugins require a standard CNI config file.
Create a Calico network
To define a rkt network for Calico, create a configuration file in /etc/rkt/net.d/
.
- Each network should be given a unique “name”.
- To use Calico networking, specify “type”: “calico”
- To use Calico IPAM, specify “type”: “calico-ipam” in the “ipam” section.
Calico will create an identically named profile for each Calico-rkt network which, by default, contains policy to allow full communication between containers within the same network (i.e. using the same profile) but prohibit ingress traffic from containers on other networks.
The same network configuration needs to be added to each node for the network to be discoverable on that node. Mutliple networks may be created using unique names.
For example, run the following to create a network called “mynet”
cat >/etc/rkt/net.d/10-calico-mynet.conf <<EOF
{
"name": "mynet",
"etcd_endpoints": "http://<ETCD_IP>:<ETCD_PORT>",
"type": "calico",
"ipam": {
"type": "calico-ipam"
}
}
EOF
Replace
<ETCD_IP>:<ETCD_PORT>
with your etcd configuration. Theetcd_endpoints
paramater may contain a comma separated list of endpoints of your etcd cluster. If the parameter is omitted from the config file, Calico defaults to a single etcd endpoint at http://127.0.0.1:2379.
Next steps
Now that you have your cluster setup, see the Basic Network Isolation guide for an example of managing Calico policy with your rkt containers.