COPY --link there is a new buildkit Convenience that can significantly speed up your Docker image creation. It works by copying files into independent image layers that do not depend on the presence of their predecessors. You can add new content to images even without the original image being on your system.
In this article, we will show what
--link does and explains how it works. We will also look at some of the situations in which this should not used.
What is “-link”?
--link There is a new optional argument for the existing Dockerfile
COPY Instructions. It changes the way copy works by creating a new snapshot layer each time you use it.
COPY Statements add files to the layer that precedes them in the Dockerfile. The contents of that layer must be present on your disk in order for the new content to be merged into:
FROM alpine COPY my-file /my-file COPY another-file /another-file
Copies of the Dockerfile above
my-file In the layer created by the previous command. After
FROM Instructions, the content of Alpine in the image is included:
bin/ dev/ etc/ ...
COPY The instruction produces an image that includes everything from Alpine, as well as
my-file bin/ dev/ etc/ ...
COPY adds instructions
another-file At the top of this image:
another-file my-file bin/ dev/ etc/ ...
The layer produced by each instruction includes everything that came before it, plus anything new that was added. At the end of the build, Docker uses a different process to complete the changes within each layer. The final image blob only contains the files that were added at each snapshot step but this is not reflected in the assembly process during build.
COPY To create a new standalone file system each time it is used. Instead of copying new files on top of the previous layer, they are sent to a completely different location to become an independent layer. The layers are then joined together to form the final image.
Let’s change the example Dockerfile to use
FROM alpine COPY --link my-file /my-file COPY --link another-file /another-file
FROM The instruction is unchanged – it generates the alpine layer with all the contents of that image:
bin/ dev/ etc/ ...
COPY The instruction has a different effect. This time another independent layer is created. This is a new file system with only
COPY The instruction creates another new snapshot only with
When the build is complete, Docker stores these independent snapshots as new layer archives (tarballs). The tarballs are linked back in a series of previous layers, forming the final image. In this all three snapshots are merged together, resulting in a filesystem that matches the original when the container was created:
my-file another-file bin/ dev/ etc/ ...
This image from the BuildKit project shows the differences between the two approaches.
Adding “Copy-Link” to Your Build
COPY --link available only when You are using BuildKit to build your images. either run your build
docker buildx --create or use
docker build with
DOCKER_BUILDKIT=1 Environment variable set.
You must also opt-in to Dockerfile v1.4 syntax by using a comment at the top of your file:
# syntax=docker/dockerfile:1.4 FROM alpine:latest COPY --link my-file /my-file COPY --link another-file /another-file
Now you can create your own image with the support of linked copies:
DOCKER_BUILDKIT=1 docker build -t my-image:latest .
Images created using Dockerfiles
COPY --link Can be used like any other. You can start a container with
docker run And push them directly to the registries.
--link The flag only affects how content is added to the image layers during construction.
Why linked copies matter
--link The flag allows the build cache to be reused, even when you build content.
COPY in changes. In addition, builds may be able to complete without their original image being present on your machine.
Returning to the example from above, the standard
COPY behavior is required
alpine The image will be present on your Docker host before new content is added. The image will be downloaded automatically during creation if you haven’t drawn it before.
With linked copies, Docker doesn’t need
alpine image content. it pulls
alpine manifest creates new independent layers for the copied files, then creates a modified manifest that links the layers to the ones they provided
alpine, contents of
alpine The image – its layer blob – will only download if you start a container from your new image or export it to a tar archive. When you push the image to the registry, that registry will store its new layers and get the remote
alpine The ones.
This functionality also facilitates efficient image rebase. Perhaps you are currently distributing a Docker image using the latest Ubuntu 20.04 LTS release:
FROM golang AS build ... RUN go build -o /app . FROM ubuntu:20.04 COPY --link --from=build /app /bin/app ENTRYPOINT ["/bin/app"]
You can use BuildKit’s . You can create the image with caching enabled by using
inline Cache stores build cache data inside the output image, where it can be reused in subsequent builds:
docker buildx build --cache-to type=inline -t example-image:20.04 .
Now let’s say you want to provide an image that will be based on the next LTS after its release, Ubuntu 22.04:
FROM golang AS build ... RUN go build -o /app . FROM ubuntu:22.04 COPY --link --from=build /app /bin/app ENTRYPOINT ["/bin/app"]
Rebuild the image using the cache data embedded in the original version:
docker buildx build --cache-from example-image:20.04 -t example-image:22.04 .
The construction will be completed almost immediately. Using cached data from an existing image, Docker can verify the files needed to build
/app hasn’t changed. It means the cache for the independent layer created by
COPY The instruction remains valid. Since this layer does not depend on any other,
ubuntu:22.04 The image won’t even be drawn. Docker only adds the snapshot layer in which
/bin/app in a new manifest within
ubuntu:22.04 layer series. The snapshot layer is effectively “rebased” on a new original image, without any filesystem operations.
The model also optimizes multi-stage builds where changes can occur between any of the stages:
FROM golang AS build RUN go build -o /app . FROM config-builder AS config RUN generate-config --out /config.yaml FROM ubuntu:latest COPY --link --from=config /config.yaml build.conf COPY --link --from=build /app /bin/app
--linkany change in the generated
ubuntu:latest to be dragged and the file to be copied. The binary has to be recompiled because filesystem changes have invalidated its cache. With linked copies, changes to
config.yaml Allows construction to continue without stretching
ubuntu:latest Or recompiling the binary. with snapshot layer
build.conf The inside is simply replaced by a new version that is independent of all other layers.
when not to use it
There are some situations where
--link The flag will not work properly. Because this copies the files to a new layer, instead of appending them on top of the previous one, you cannot use ambiguous references as your destination path:
COPY --link my-file /data
my-file will be copied into
/data The image already exists as a directory. with
--linkThe target layer’s filesystem will always be empty, so
my-file is written to
The same idea applies to symlink resolution. standard
COPY Automatically resolves destination paths that are symlinked to the image. when you are using
--linkThis behavior is not supported because the symlink will not exist in the independent layer of the copy.
It is recommended that you start using
--link where these limits do not apply. Adopting this feature will speed up your builds and make caching more powerful. If you cannot remove the ambiguous or associated destination paths immediately, you can continue to use the existing
COPY Instructions. It is because of these backward incompatible changes that
--link New is an optional flag instead of the default.
COPY --link There is a new Dockerfile feature that can make builds faster and more efficient. Images that use linked copies do not need to drag past layers, so that files can be copied into them. Docker creates a new independent layer for each
COPY Instead, it then adds those layers back into the chain.
If you’re building images with the latest version of BuildKit and Buildx or Docker CLI, you can start using the linked copies now. Adopting “-link” is a new best practice Docker build phase, provided you are not affected by necessary changes to the destination path solution.
#speed #Docker #builds #optimize #caching #copylink