/ Tech

Docker multi-stage builds are awesome.

I have a new found love of Docker; despite my early frustrations with CoreOS (as it was) in 2015, I have come to love the simplicity and encapsulation that containers provide. Run a single command and whoosh - you now have a completely functional unit that runs whichever package you require, and it's nice and isolated too: awesome!

In contrast though, I have no love for build processes: you're always one step away from dependency hell, small environmental issues can prove to be show-stoppers, and you end up polluting your development or build environment with various tools and libraries. There must be a better way, perhaps one that leverages our friend the container?

Multistage Builds

Introduced in Docker 17.05, Multistage Builds are an interesting feature that allow you to chain images via a Dockerfile; allowing later images to access resources from earlier ones. Simply put, you can use multiple FROM directives - and re-use resources in multiple images.

So an average use case would look something like: (a) run a build container, compiling any source code and building an executable, and (b) copy that executable in to a lightweight image, and set it as the entrypoint.

Here's an example of what this could look like:

FROM golang:alpine

WORKDIR /usr/src/app
COPY . .

RUN go build -v -o MultiStageExample


FROM alpine
MAINTAINER Fergus In London <fergus@fergus.london>

WORKDIR /var/app/example
COPY --from=0 /usr/src/app/MultiStageExample .

RUN chmod +x MultiStageExample
ENTRYPOINT ["/var/app/example/MultiStageExample"]

This example should underline just how simplistic multistage builds can be, and also just how concise they can be: this example is an incredibly short 10 lines! Although if you're still a bit puzzled, for a fully annotated version check-out my accompanying Docco documentation.

Dependency Management

One of the most frustrating aspects of a multistage build is dependency management though; every build requires you to download the entire dependency tree, and if you have a complex application, then this process can be quite time consuming and painful.

One way around this is to also COPY any vendor directory; but this isn't ideal in itself because ideally you want the dependencies to be downloaded in the environment that compilation is occuring in.

Unfortunately when building an image via a Dockerfile you can't mount a volume - i.e like the docker -v flag - so the ephemeral nature of Docker can really sting you here. Although if your multistage build is limited to a CI environment, or another pre-deployment phase, then this isn't a huge issue as it's not a blocker for the developer.

In short, multi-stage builds can be quite time consuming for the actual development process. They rock for production though, and enable some nice CI workflows that can sandbox and contain the entirety of the build process though.

Example

If you've got this far you'll have already seen the above example, but if you're new to Docker - or otherwise confused by the example above - then I'd definitely suggest you checkout the accompanying documentation that I wrote, as well as the Github Repository for a downloadable/runnable example.


Fergus

Contract Software Developer and DevSecOps Consultant, based out of London in England. Interests include information security, current affairs, and photography.