When shipping applications using containers, one often is confronted with overly large final images. Multi-stage builds are a common way to circumvent this issue, especially for compiled languages like Go or Java. In our latest blog post we show how to utilise multi-stage builds for python images to bring down image sizes and thereby improving security.

Using docker multi-stage builds to reduce image size

When writing container images, it is always preferable to have smaller images and to only include what is really needed. This has two main advantages:

  • images take up less storage space
  • security vulnerabilities of software that is not installed cannot be exploited

Example dockerfile

Let's take a very simple image that starts off with an official python-docker image, using a slim version of the current stable debian release, bullseye. We are going to use python 3.8, but this is not important. The image is pulled from docker hub.

The dockerfiles are provided in this repository in the docker directory and can be built using './bin/build.sh' on Linux.

Reducing the image size

The all-in-one solution

We are simply going to `COPY` over a `requirements.txt` and install it using `pip`, as this is a fairly common use case. I have chosen to install pyodbc since it has some system
dependencies that have to be installed.

Note: The dockerfiles are a minimal reproducible example and do not follow some common best practices!

```dockerfile
FROM python:3.8-slim-bullseye

RUN apt-get update && \
    apt-get install -y gcc g++ unixodbc-dev

COPY requirements.txt /tmp/requirements.txt

RUN pip3 install --no-cache-dir -r /tmp/requirements.txt
```

After building the image, we get an image size of 422MB:

```bash
$ docker images original
REPOSITORY   TAG       IMAGE ID       CREATED              SIZE
original     latest    09ec99a1bafa   About a minute ago   422MB
```

Introducing: multi-stage

Multi-stage builds are a neat way to keep build dependencies from blowing up the size of your final image. I am not going to go into detail on how they work.

I am going to create wheels. From the pip documentation:

"Wheel is a built-package format, and offers the advantage of not recompiling your software during every install."

And this is exactly what we want. Compile it and then reuse it without compilation.

The original dockerfile is split in 2: the builder image will install the build dependencies and create the wheels. The final image will install the packages from the wheels without the need to install any build tools:

the wheels. The final image will install the packages from the wheels without the need to install any 
```dockerfile
ARG WHEEL_DIST="/tmp/wheels"

FROM python:3.8-slim-bullseye as builder

ARG WHEEL_DIST

RUN apt-get update && \
    apt-get install -y gcc g++ unixodbc-dev

COPY requirements.txt /tmp/requirements.txt

RUN python3 -m pip wheel -w "${WHEEL_DIST}" -r /tmp/requirements.txt


FROM python:3.8-slim-bullseye

ARG WHEEL_DIST

COPY --from=builder "${WHEEL_DIST}" "${WHEEL_DIST}"

WORKDIR "${WHEEL_DIST}"

RUN pip3 --no-cache-dir install *.whl
``` 

The size of the image has gone down significantly:

```bash
$ docker images multi-stage
REPOSITORY    TAG       IMAGE ID       CREATED          SIZE
multi-stage   latest    b90d97997b3b   44 seconds ago   128MB
```

The difference is starker when considering the size of the base image:

```bash
docker images python:3.8-slim-bullseye
REPOSITORY   TAG                 IMAGE ID       CREATED      SIZE
python       3.8-slim-bullseye   caf584a25606   5 days ago   122MB
```

Increasing security

Smaller images can contribute to more security.
To illustrate this point, we are going to use trivy to scan our images for known security vulnerabilities.

The base image has 85 vulnerabilities:

  • LOW: 12
  • MEDIUM: 35
  • HIGH: 30
  • CRITICAL: 8

The original image that includes the build tools has 331 total vulnerabilities:

  • UNKNOWN: 2
  • LOW: 24
  • MEDIUM: 175
  • HIGH: 109
  • CRITICAL: 21

The multi-stage image again has 85 images that it "inherits" from the base image, but does not introduce any new ones.

Summary

When using python, do utilise wheels and multi-stage builds to decrease the image size and increase the security of your deployements.