A word on ‘docker buildx’

What is Docker Buildx ?

Docker Buildx can be defined as a CLI plugin that provides us with the extra features supported by the Moby BuildKit builder toolkit along with the features of the normal docker commands. Furthermore, it also provides us with the same user experience as building with the docker build command. Also, along with some new extra features added over it like building against multiple nodes with multi-arch images concurrently, builder instances, etc. In addition to that, buildx also supports new features that are not yet available for regular docker build like building manifest lists, distributed caching, etc.

Working of Buildx

And how is “docker buildx” different from “docker build” ?

Buildx allows us to build an image for the native architecture, similar to a docker build but it also supports and allows for emulation.

Here, emulation means that from a specific machine (for example, say Intel machine) we can build an image targeted for a different architecture-supported machine (for example, a raspberry pi machine).

So, what we can do with buildx is, in simplest terms, we can build an image on Mac, targeted for Raspberry Pi, push it to dockerhub and then on our Raspberry Pi machine, we can pull that image and run it.

Advantage of this is :

1. It is quicker to do on the emulator rather than the actual device.
2. It provides us with – building and testing on the device, in a loop, without being actually connected to the device being targeted, i.e. for which the image is being built.
3. The other advantage here is, we can build a genuine Build farm.

So, we have all the machines and without connecting to each of them independently and then adding a Dockerfile to each of them, and at last creating a manifest file for each of them and pushing it to dockerhub, what we can do is :
We can use docker buildx to make a build farm, so when everything has been setup, we can perform a :

docker buildx build .

We pass this a single Dockerfile and it parallelizes the build for all the machines specified in the build farm. After the build is over, it pushes the result to the Dockerhub and tells the machines to perform a build immediately.

This would result in a native build from each builder in the farm :

• A windows builder will build a windows-specific image.
• A mac builder will build a mac-specific image.
• A raspberry pi builder will build a raspberry pi-specific image.

And the docker buildx will build the manifest for me at the output of this command but using the 2 extra parameters –platform and –push :

–platform : This parameter is used for listing all of the platforms which are targeted, i.e. for which the images are to be built (for example, linux/amd64, linux/arm64, darwin/amd64, etc). And docker buildx will reach out to each of these elements in the farm and give them the right platform to build the image. And when we use this image in docker run or docker service, docker picks up the correct image based on the node’s platform.

–push : This parameter is used when the images have been built for all targeted platforms and now the images have to be pushed to the registry ( the public registry, by default : dockerhub ).

So, now we can say that we get a multi-arch image with buildx that we can use anywhere with just a single command. And docker buildx will reach out to each of these elements in the farm and give them the right platform to build.

We can build multi-platform images using three different strategies that are supported by Buildx and Dockerfiles:

1. Using the QEMU emulation support in the kernel.
   In particular, this approach is considered to be the most suitable approach when one is working on a Docker Desktop ( usually Mac or Windows ).
2. Building on multiple native nodes using the same builder instance.
   This approach is suitable for covering all the use-cases that are not handled efficiently by QEMU approach, and as a result we achieve better performance.
3. Using a stage in Dockerfile to cross-compile to different architectures.
   In this case, multi-stage builds in Dockerfiles can be effectively used to build binaries for the platform specified with --platform using the native architecture of the build node. A list of build arguments like BUILDPLATFORM and TARGETPLATFORM is available automatically inside your Dockerfile.

Docker Buildx – An Experimental feature.

Buildx is in it’s experimental stage as it is based on BuildKit.

To enable the experimental features in the Docker CLI, edit the config.json file and set experimental to enabled.

Few Important Commands

• Build from a file – docker buildx bake

BuildKit provides us with parallelization, i.e. it can handle multiple concurrent build requests efficiently , but with the usual docker build command, it performs the invocation of builds in sequence.
For this reason, we have a command called, docker buildx bake.
The bake command supports building images from compose files, similar to a compose build, but allowing all the services to be built concurrently as part of a single request.

Few options with the bake command are :

• Starts a new build – docker buildx build
• Create a new builder instance – docker buildx create

This creates a new builder instance with a single node based on your current configuration.
To use a remote node you can specify the DOCKER_HOST or the remote context name while creating the new builder.

• Inspect current builder instance – docker buildx inspect
• List out all the builder instances – docker buildx ls
• Commands to work on images in registry – docker buildx imagetools
• Remove a builder instance – docker buildx rm
• Stop builder instance – docker buildx stop
• Set the current builder instance – docker buildx use

To switch between different builders use docker buildx use <name> command. After running this command, the build commands will now automatically use the specified builder.

• docker buildx version – Show buildx version information

To set buildx as the default builder :

 docker buildx install 

To remove buildx as the default builder :

docker buildx uninstall

This is all for now, for more information, one can always refer to the official documentation of Docker.