What is Docker Buildx ?
Docker Buildx can be defined as a CLI plugin that provides us with the extra features supported by the Moby BuildKit builder toolkit along with the features of the normal docker commands. Furthermore, it also provides us with the same user experience as building with the
docker build command. Also, along with some new extra features added over it like building against multiple nodes with multi-arch images concurrently, builder instances, etc. In addition to that, buildx also supports new features that are not yet available for regular
docker build like building manifest lists, distributed caching, etc.
Working of Buildx
And how is “docker buildx” different from “docker build” ?
Buildx allows us to build an image for the native architecture, similar to a docker build but it also supports and allows for emulation.
Here, emulation means that from a specific machine (for example, say Intel machine) we can build an image targeted for a different architecture-supported machine (for example, a raspberry pi machine).
So, what we can do with buildx is, in simplest terms, we can build an image on Mac, targeted for Raspberry Pi, push it to dockerhub and then on our Raspberry Pi machine, we can pull that image and run it.
Advantage of this is :
- It is quicker to do on the emulator rather than the actual device.
- It provides us with – building and testing on the device, in a loop, without being actually connected to the device being targeted, i.e. for which the image is being built.
- The other advantage here is, we can build a genuine Build farm.
So, we have all the machines and without connecting to each of them independently and then adding a Dockerfile to each of them, and at last creating a manifest file for each of them and pushing it to dockerhub, what we can do is :
We can use docker buildx to make a build farm, so when everything has been setup, we can perform a :
docker buildx build .
We pass this a single Dockerfile and it parallelizes the build for all the machines specified in the build farm. After the build is over, it pushes the result to the Dockerhub and tells the machines to perform a build immediately.
This would result in a native build from each builder in the farm :
- A windows builder will build a windows-specific image.
- A mac builder will build a mac-specific image.
- A raspberry pi builder will build a raspberry pi-specific image.
And the docker buildx will build the manifest for me at the output of this command but using the 2 extra parameters –platform and –push :
–platform : This parameter is used for listing all of the platforms which are targeted, i.e. for which the images are to be built (for example, linux/amd64, linux/arm64, darwin/amd64, etc). And docker buildx will reach out to each of these elements in the farm and give them the right platform to build the image. And when we use this image in
docker run or
docker service, docker picks up the correct image based on the node’s platform.
–push : This parameter is used when the images have been built for all targeted platforms and now the images have to be pushed to the registry ( the public registry, by default : dockerhub ).
So, now we can say that we get a multi-arch image with buildx that we can use anywhere with just a single command. And docker buildx will reach out to each of these elements in the farm and give them the right platform to build.
We can build multi-platform images using three different strategies that are supported by Buildx and Dockerfiles:
- Using the QEMU emulation support in the kernel.
In particular, this approach is considered to be the most suitable approach when one is working on a Docker Desktop ( usually Mac or Windows ).
- Building on multiple native nodes using the same builder instance.
This approach is suitable for covering all the use-cases that are not handled efficiently by QEMU approach, and as a result we achieve better performance.
- Using a stage in Dockerfile to cross-compile to different architectures.
In this case, multi-stage builds in Dockerfiles can be effectively used to build binaries for the platform specified with
--platformusing the native architecture of the build node. A list of build arguments like
TARGETPLATFORMis available automatically inside your Dockerfile.
Docker Buildx – An Experimental feature.
Buildx is in it’s experimental stage as it is based on BuildKit.
To enable the experimental features in the Docker CLI, edit the
config.json file and set
experimental to enabled.
Few Important Commands
- Build from a file – docker buildx bake
BuildKit provides us with parallelization, i.e. it can handle multiple concurrent build requests efficiently , but with the usual
docker buildcommand, it performs the invocation of builds in sequence.
For this reason, we have a command called,
docker buildx bake.
bakecommand supports building images from compose files, similar to a compose build, but allowing all the services to be built concurrently as part of a single request.
Few options with the bake command are :
|Build definition file|
|Print the options without building|
|No use of cache when building the image|
|Set type of progress output (auto, plain, tty).|
|Always attempt to pull a newer version of the image|
|Override target value|
- Starts a new build – docker buildx build
- Create a new builder instance – docker buildx create
This creates a new builder instance with a single node based on your current configuration.
To use a remote node you can specify the
DOCKER_HOST or the remote context name while creating the new builder.
- Inspect current builder instance – docker buildx inspect
- List out all the builder instances – docker buildx ls
- Commands to work on images in registry – docker buildx imagetools
- Remove a builder instance – docker buildx rm
- Stop builder instance – docker buildx stop
- Set the current builder instance – docker buildx use
To switch between different builders use
docker buildx use <name> command. After running this command, the build commands will now automatically use the specified builder.
|Set builder as default for current context|
|Builder persists context changes|
- docker buildx version – Show buildx version information
To set buildx as the default builder :
docker buildx install
To remove buildx as the default builder :
docker buildx uninstall
This is all for now, for more information, one can always refer to the official documentation of Docker.