Using Calico to Secure Host Interfaces
This guide describes how to use Calico to secure the network interfaces of the host itself (as opposed to those of any container/VM workloads that are present on the host). We call such interfaces “host endpoints”, to distinguish them from “workload endpoints” (such as containers or VMs).
Calico supports the same rich security policy model for host endpoints that it supports for workload endpoints. Host endpoints can have labels, and their labels are in the same “namespace” as those of workload endpoints. This allows security rules for either type of endpoint to refer to the other type (or a mix of the two) using labels and selectors.
Calico does not support setting IPs or policing MAC addresses for host interfaces, it assumes that the interfaces are configured by the underlying network fabric.
Calico distinguishes workload endpoints from host endpoints by a
configurable prefix controlled by the InterfacePrefix
configuration
value, (see: Calico Configuration). Interfaces that start with the value of
InterfacePrefix
are assumed to be workload interfaces. Others are
treated as host interfaces.
Calico blocks all traffic to/from workload interfaces by default; allowing traffic only if the interface is known and policy is in place. However, for host endpoints, Calico is more lenient; it only polices traffic to/from interfaces that it’s been explicitly told about. Traffic to/from other interfaces is left alone.
NOTE
If you have a host with workloads on it then traffic that is forwarded to workloads bypasses the policy applied to host endpoints. If that weren’t the case, the host endpoint policy would need to be very broad to allow all traffic destined for any possible workload.
Overview
To make use of Calico’s host endpoint support, you will need to follow these steps, described in more detail below:
- download the calicoctl binary
- create an etcd cluster, if you haven’t already
- install Calico’s Felix daemon on each host
- initialize the etcd database
- add policy to allow basic connectivity and Calico function
- create host endpoint objects in etcd for each interface you want Calico to police (in a later release, we plan to support interface templates to remove the need to explicitly configure every interface)
- insert policy into etcd for Calico to apply
- decide whether to disable “failsafe SSH/etcd” access.
Download the calicoctl binary
Download the calicoctl binary onto your host.
wget https://github.com/projectcalico/calicoctl/releases/download/v1.0.2/calicoctl
chmod +x calicoctl
This binary should be placed in your $PATH
so it can be run from any
directory.
Creating an etcd cluster
If you haven’t already created an etcd cluster for your Calico deployment, you’ll need to create one.
To create a single-node etcd cluster for testing, download an etcd v3.x release from the etcd releases archive; we recommend using the most recent bugfix release. Then follow the instructions on that page to unpack and run the etcd binary.
To create a production cluster, you should follow the guidance in the etcd manual. In particular, the clustering guide.
Installing Felix
There are several ways to install Felix.
-
if you are running Ubuntu 14.04, then you can install a version from our PPA:
sudo add-apt-repository ppa:project-calico/calico-<version> sudo apt-get update sudo apt-get upgrade sudo apt-get install calico-felix
As of writing, <version> should be 2.0.
-
if you are running a RedHat 7-derived distribution, you can install from our RPM repository:
cat > /etc/yum.repos.d/calico.repo <<EOF [calico] name=Calico Repository baseurl=http://binaries.projectcalico.org/rpm_stable/ enabled=1 skip_if_unavailable=0 gpgcheck=1 gpgkey=http://binaries.projectcalico.org/rpm/key priority=97 EOF yum install calico-felix
-
if you are running another distribution, follow the instructions in this document to use our installer bundle.
-
if you want to run under docker, you can use
calicoctl node run
to start the calico/node container image. This container packages up the core Calico components to provide both Calico networking and network policy. Running the container automatically pre-initializes the etcd database (which the other installations methods do not). See thecalicoctl node run
guide for details.
Until you initialize the database, Felix will make a regular log that it
is in state “wait-for-ready”. The default location for the log file is
/var/log/calico/felix.log
.
Initialising the etcd database
If you are using the container-based installation (using the calico/node container image), the database is initialized as soon as you start the first node instance.
If you are self-installed you should configure a node
resource for each
host running Felix. In this case, the database is initialized after
creating the first node
resource. For a deployment that does not include
the Calico/BGP integration, the specification of a node resource just requires
the name of the node; for most deployments this will be the same as the
hostname.
cat << EOF | calicoctl create -f -
- apiVersion: v1
kind: node
metadata:
name: <node name or hostname>
EOF
If you check the felix logfile after this step, the logs should transition from periodic notifications that felix is in state “wait-for-ready” to a stream of initialisation messages.
Creating basic connectivity and Calico policy
When a host endpoint is added, if there is no security policy for that endpoint, Calico will default to denying traffic to/from that endpoint, except for traffic that is allowed by the failsafe rules.
While the failsafe rules provide protection against removing all connectivity to a host,
- they are overly broad in allowing inbound SSH on any interface and allowing traffic out to etcd’s ports on any interface
- depending on your network, they may not cover all the ports that are required; for example, your network may reply on allowing ICMP, or DHCP.
Therefore, we recommend creating a failsafe Calico security policy that
is tailored to your environment. The example command below shows one
example of how you might do that; the command uses calicoctl
to:
- Create a single policy resource, which
- applies to all known endpoints
- allows inbound ssh access from a defined “management” subnet
- allows outbound connectivity to etcd on a particular IP; if you have multiple etcd servers you should duplicate the rule for each destination
- allows inbound ICMP
- allows outbound UDP on port 67, for DHCP.
When running this command, replace the placeholders in angle brackets with appropriate values for your deployment.
cat << EOF | calicoctl create -f -
- apiVersion: v1
kind: policy
metadata:
name: failsafe
spec:
selector: "all()"
order: 0
ingress:
- action: allow
protocol: tcp
source:
net: "<your management CIDR>"
destination:
ports: [22]
- action: allow
protocol: icmp
egress:
- action: allow
protocol: tcp
destination:
net: "<your etcd IP>/32"
ports: [<your etcd ports>]
- action: allow
protocol: udp
destination:
ports: [67]
EOF
Once you have such a policy in place, you may want to disable the failsafe rules.
NOTE
Packets that reach the end of the list of rules fall-through to the next policy (sorted by the order field).
The selector in the policy,
all()
, will match all endpoints, including any workload endpoints. If you have workload endpoints as well as host endpoints then you may wish to use a more restrictive selector. For example, you could label management interfaces with labelendpoint_type = management
and then use selectorendpoint_type == "management"
If you are using Calico for networking workloads, you should add inbound and outbound rules to allow BGP: add an ingress and egress rule to allow TCP traffic to destination port 179.
Creating host endpoint objects
For each host endpoint that you want Calico to secure, you’ll need to
create a host endpoint object in etcd. Use the calicoctl create
command
to create a host endpoint resource (hostEndpoint).
There are two ways to specify the interface that a host endpoint should refer to. You can either specify the name of the interface or its expected IP address. In either case, you’ll also need to know the name given to the Calico node running on the host that owns the interface; in most cases this will be the same as the hostname of the host.
For example, to secure the interface named eth0
with IP 10.0.0.1 on
host my-host
, run the command below. The name of the endpoint is an
arbitrary name required for endpoint identification.
When running this command, replace the placeholders in angle brackets with appropriate values for your deployment.
cat << EOF | calicoctl create -f -
- apiVersion: v1
kind: hostEndpoint
metadata:
name: <name of endpoint>
node: <node name or hostname>
labels:
role: webserver
environment: production
spec:
interfaceName: eth0
profiles: [<list of profile IDs>]
expectedIPs: ["10.0.0.1"]
EOF
NOTE
Felix tries to detect the correct hostname for a system. It logs out the value it has determined at start-of-day in the following format:
2015-10-20 17:42:09,813 \[INFO\]\[30149/5\] calico.felix.config 285: Parameter FelixHostname (Felix compute host hostname) has value 'my-hostname' read from None
The value (in this case “my-hostname”) needs to match the hostname used in etcd. Ideally, the host’s system hostname should be set correctly but if that’s not possible, the Felix value can be overridden with the FelixHostname configuration setting. See configuration for more details.
Where <list of profile IDs>
is an optional list of security profiles
to apply to the endpoint and labels contains a set of arbitrary
key/value pairs that can be used in selector expressions.
Warning
When rendering security rules on other hosts, Calico uses the
expectedIPs
field to resolve label selectors to IP addresses. If theexpectedIPs
field is omitted then security rules that use labels will fail to match this endpoint.
Or, if you knew that the IP address should be 10.0.0.1, but not the name of the interface:
cat << EOF | calicoctl create -f -
- apiVersion: v1
kind: hostEndpoint
metadata:
name: <name of endpoint>
node: <node name or hostname>
labels:
role: webserver
environment: production
spec:
profiles: [<list of profile IDs>]
expectedIPs: ["10.0.0.1"]
EOF
After you create host endpoint objects, Felix will start policing traffic to/from that interface. If you have no policy or profiles in place, then you should see traffic being dropped on the interface.
NOTE
By default, Calico has a failsafe in place that whitelists certain traffic such as ssh. See below for more details on disabling/configuring the failsafe rules.
If you don’t see traffic being dropped, check the hostname, IP address
and (if used) the interface name in the configuration. If there was
something wrong with the endpoint data, Felix will log a validation
error at WARNING
level and it will ignore the endpoint:
$ grep "Validation failed" /var/log/calico/felix.log
2016-05-31 12:16:21,651 [WARNING][8657/3] calico.felix.fetcd 1017:
Validation failed for host endpoint HostEndpointId<eth0>, treating as
missing: 'name' or 'expected_ipvX_addrs' must be present.;
'{ "labels": {"foo": "bar"}, "profile_ids": ["prof1"]}'
The error can be quite long but it should log the precise cause of the rejection; in this case “‘name’ or ‘expected_ipvX_addrs’ must be present” tells us that either the interface’s name or its expected IP address must be specified.
Creating more security policy
We recommend using selector-based security policy with bare-metal workloads. This allows ordered policy to be applied to endpoints that match particular label selectors.
+For example, you could add a second policy for webserver access:
cat << EOF | dist/calicoctl create -f -
- apiVersion: v1
kind: policy
metadata:
name: webserver
spec:
selector: "role==\"webserver\""
order: 100
ingress:
- action: allow
protocol: tcp
destination:
ports: [80]
egress:
- action: allow
EOF
Failsafe rules
To avoid completely cutting off a host via incorrect or malformed policy, Calico has a failsafe mechanism that keeps various pinholes open in the firewall.
By default, Calico keeps port 22 inbound open on all host endpoints, which allows access to ssh; as well as outbound communication to ports 2379, 2380, 4001 and 7001, which allows access to etcd’s default ports.
The lists of failsafe ports can be configured via the configuration parameters described in Calico Configuration. They can be disabled by setting each configuration value to “none”.
WARNING
Removing the inbound failsafe rules can leave a host inaccessible.
Removing the outbound failsafe rules can leave Felix unable to connect to etcd.
Before disabling the failsafe rules, we recommend creating a policy to replace it with more-specific rules for your environment: see above.