How to Run Containers Securely with gVisor on EKS

This article was originally posted to Medium on Dec 23, 2021.

Here at Blink, we needed a solution for full isolation of running Pods on EKS cluster, not just for the network, but for system calls as well.

Blink is building a DevOps-oriented platform where users can create and run applications or playbooks for any internal tasks, like troubleshooting, security and etc. It can run on our managed cloud, their public/private cloud, or even their own personal desktop. Since it can be basically anything, when running on our cloud, we have to isolate these running applications to give a fully secured solution.

Our platform running on EKS, so we considered several options that can work there, and gVisor looked like the best solution.

gVisor is a userspace re-implementation of the Linux kernel API that does not need elevated privileges. In conjunction with a container runtime such as containerd, the userspace kernel re-implements the majority of system calls and services them on behalf of the host kernel. Direct access to the host kernel is limited. This way it provides an additional layer of isolation between running applications and the host operating system.
From the container’s point of view, gVisor is nearly transparent and does not require any changes to the containerized application.

On GKE you can use GKE Sandbox , but if your cluster is running on EKS, there are a few things you need to do to make it work:

• build an AMI with gVisor runtime for your nodes
• Use this AMI for your nodes pools
• Set a flag for kubelet to run with containerd

Create AMI with gVisor runtime

Create a VM from the latest AMI for eks node (from amazon-eks-node-1.21-* or newer). Note that your cluster has to be in version 1.21 or newer, older amazon-eks-node AMIs do not have containerd, and you will have to install and configure it manually.

Connect to your new VM and run and few steps:

1. Install gVisor runsc:

wget https://storage.googleapis.com/gvisor/releases/nightly/latest/runsc
 sudo mv runsc /usr/local/bin
 sudo chown root:root /usr/local/bin/runsc
 sudo chmod 0755 /usr/local/bin/runsc

2. Install gvisor-containerd-shim:

ARCH=$(uname -m)
        URL=https://storage.googleapis.com/gvisor/releases/release/latest/${ARCH}
wget ${URL}/runsc ${URL}/runsc.sha512 \
   ${URL}/containerd-shim-runsc-v1 ${URL}/containerd-shim-runsc-v1.sha512
sha512sum -c runsc.sha512 \
   -c containerd-shim-runsc-v1.sha512
rm -f *.sha512
chmod a+rx runsc containerd-shim-runsc-v1

3. Configure containerd:

cat <<EOF | sudo tee /etc/containerd/config.toml
version = 2
[plugins."io.containerd.runtime.v1.linux"]
  shim_debug = true
        [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
  runtime_type = "io.containerd.runc.v2"
        [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runsc]
  runtime_type = "io.containerd.runsc.v1"
EOF

That’s it, now just create an AMI from your VM and use it for your EKS nodes.
Don’t forget to add a flag for your nodes kubelet to use containerd:

--container-runtime containerd