Operational Docker Removing the extra layers when you build containers

Building containers makes lots of different layers. When you make a change to an element in the build all following layers have to be rebuilt because they cannot be taken from cache. A simple example I have used before that has way too many layers:

docker build -t test .
Sending build context to Docker daemon  5.632kB
Step 1/7 : FROM ubuntu:latest
 ---> 7698f282e524
Step 2/7 : RUN echo "Test"
 ---> Using cache
 ---> 75ac3bfbeaba
Step 3/7 : COPY 1 .
 ---> Using cache
 ---> d457a7492d2c
Step 4/7 : ADD 2 .
 ---> Using cache
 ---> 1c3284c1e6a0
Step 5/7 : ADD 3 .
 ---> Using cache
 ---> 96ab91bcf3df
Step 6/7 : ADD 4 .
 ---> Using cache
 ---> 2889643b631b
Step 7/7 : ADD 5 .
 ---> Using cache
 ---> adb6797fe48a
Successfully built adb6797fe48a
Successfully tagged test:latest

If you examine the number of dangling layers (layers not connected to an active image) for my build there are none:

docker images -f dangling=true
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE

Now I am going to modify one of the files that are being added at step 3/7 so it triggers a rebuild:

docker build -t test .
Sending build context to Docker daemon   5.12kB
Step 1/7 : FROM ubuntu:latest
 ---> 7698f282e524
Step 2/7 : RUN echo "Test"
 ---> Using cache
 ---> 75ac3bfbeaba
Step 3/7 : COPY 1 .
 ---> Using cache
 ---> d457a7492d2c
Step 4/7 : ADD 2 .
 ---> a93a35a37ebe
Step 5/7 : ADD 3 .
 ---> b90d01ee0806
Step 6/7 : ADD 4 .
 ---> 43af309b28d8
Step 7/7 : ADD 5 .
 ---> 540007e7c833
Successfully built 540007e7c833
Successfully tagged test:latest

Now if we check for dangling layers we now have layer that was replaced 3/7:

docker images -f dangling=true
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
<none>              <none>              adb6797fe48a        10 minutes ago      69.9MB

You can identify where these dangling layers are stored by doing docker image inspect:

docker image inspect test
[
    {
        "Id": "sha256:540007e7c833d09da0edaad933d4126075bffa32badb1170da363e1e1f220c4c",
        "RepoTags": [
            "test:latest"
        ],
        "RepoDigests": [],
        "Parent": "sha256:43af309b28d81faa983b87d2db2f64b27b7658f93639f10b07d53b50dded7c45",
        "Comment": "",
        "Created": "2019-06-19T02:46:38.5065201Z",
        "Container": "",
        "ContainerConfig": {
            "Hostname": "",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
            ],
            "Cmd": [
                "/bin/sh",
                "-c",
                "#(nop) ADD file:f5b9e73db3de1fef2d430837d0d31af0af9c9405c64349def90530b2fc8ca6d2 in . "
            ],
            "ArgsEscaped": true,
            "Image": "sha256:43af309b28d81faa983b87d2db2f64b27b7658f93639f10b07d53b50dded7c45",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": null
        },
        "DockerVersion": "18.09.2",
        "Author": "",
        "Config": {
            "Hostname": "",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
            ],
            "Cmd": [
                "/bin/bash"
            ],
            "ArgsEscaped": true,
            "Image": "sha256:43af309b28d81faa983b87d2db2f64b27b7658f93639f10b07d53b50dded7c45",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": null
        },
        "Architecture": "amd64",
        "Os": "linux",
        "Size": 69859108,
        "VirtualSize": 69859108,
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/77fcf68e04407dd297d93203e242e52cff942b2d8508b3b6e4db2f62d53f38bc/diff:/var/lib/docker/overlay2/dd3f244fb32b847a5d8b2b18e219142bbe3b2e61fe107e290c144154b4513d5e/diff:/var/lib/docker/overlay2/839011eb0edb75c6fedb5e4a9155de2e4bd6305d233ed085d40a4e5166328736/diff:/var/lib/docker/overlay2/3dc3a1f4d37b525f672196b55033c6b82e4ddfc20de16aa803c36fe2358bcb32/diff:/var/lib/docker/overlay2/6e11b07d20377b78ee134a037fc6e661364d273d861419eb77126d0d228abbf0/diff:/var/lib/docker/overlay2/f8c5f20e6ebd1ec759101d926a5101a36ef2378af828ef57a0f8e4a8a467f76f/diff:/var/lib/docker/overlay2/77a101af01c69427ced57be20f01d4a6a688ff2b13d50260be7a7fda1bd7fbf5/diff",
                "MergedDir": "/var/lib/docker/overlay2/28f3e078ee98a327274c59c653858559ad81b866a40e62a70cca989ee403f2a6/merged",
                "UpperDir": "/var/lib/docker/overlay2/28f3e078ee98a327274c59c653858559ad81b866a40e62a70cca989ee403f2a6/diff",
                "WorkDir": "/var/lib/docker/overlay2/28f3e078ee98a327274c59c653858559ad81b866a40e62a70cca989ee403f2a6/work"
            },
            "Name": "overlay2"
        },
        "RootFS": {
            "Type": "layers",
            "Layers": [
                "sha256:02571d034293cb241c078d7ecbf7a84b83a5df2508f11a91de26ec38eb6122f1",
                "sha256:270f934787edf0135132b6780cead0f12ca11690c5d6a5d395e44d290912100a",
                "sha256:8d267010480fed7e616b9b7861854042aad4ef5e55f8771f2c738061640d2cb0",
                "sha256:ea9703e9d50c6fdd693103fee05c65e8cc25be44c6e6587dd89c6559d8df2de7",
                "sha256:69d3f4708a57a9355cf65a99274e6b79788a052564c4fb0fd90f5283c109946a",
                "sha256:d18953dc7e1eef0e19b52db05c2ff34089e9f1166766c8f57b8475db5a3c79b8",
                "sha256:f1ce2d9ca96cc9cd13caab945986580eae2404e87d81b1b485b12ee242c37889",
                "sha256:aeb58c1f315c5baacbe4c2db1745dec548753197e2b251a958704addfd33a8c2"
            ]
        },
        "Metadata": {
            "LastTagTime": "2019-06-19T02:46:38.5547836Z"
        }
    }
]

You can see all the layers are stored in /var/lib/docker/overlay2 you can use the dangling layer id to locate how much space is now wasted on your hard drive. You can remove these dangling layers with:

docker rmi $(docker images -f dangling=true)

These dangling images can eat up tons of space on your build machine. So you need to automate the cleanup process to avoid wasting space.

One Reply to “Operational Docker Removing the extra layers when you build containers”

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.