This post includes a grab-bag of Docker commands that I always find myself having to Google. Now I can just bookmark my own post and have them all at my finger tips! I use Docker locally inside a Linux VM rather than using Docker for Windows, so they all apply to that setup. Your mileage may vary - I imagine they work in Docker for Windows but I haven't checked.
I've split the list over 2 posts, but this is what you can expect:
- Don't require
sudoto execute Docker commands - Examining the file system of a failed Docker build
- Examining the file system of an image with an ENTRYPOINT
- Copying files from a Docker container to the host
- Copying files from a Docker image to the host
- View the space used by Docker
- Remove old docker images and containers
- Speeding up builds by minimising the Docker context
- Viewing (and minimising) the Docker context
- Bonus: Making a file executable in git
- Bonus 2: Forcing script files to keep Unix Line endings in git
Don't require sudo to execute Docker commands
By default, when you first install Docker on a Linux machine, you might find that you get permission errors when you try and run any Docker commands:
To get round this, you need to run all your commands with sudo, e.g. sudo docker images. There's good security reasons for requiring sudo, but when I'm just running it locally in a VM, I'm not too concerned about them. To get round it, and avoid having to type sudo every time, you can add your current user to the docker group, which effectively gives it root access.
sudo usermod -aG docker $USER
After running the command, just log out of your VM (or SSH session) and log back in, and you should be able to run your docker commands without needing to type sudo for everything.
Examining the file system of a failed Docker build
When you're initially writing a Dockerfile to build your app, you may find that it fails to build for some reason. That could happen for lots of reasons - it could be an error in your code,it could be an error in your Dockerfile invoking the wrong commands, or you might not be copying all the required files into the image for example.
Sometimes you can get obscure errors that leave you unsure what went wrong. When that happens, you might want to inspect the filesystem when the build failed, to see what went wrong. You can do this by running one of the previous image layers for your Dockerfile.
When Docker builds an image using a Dockerfile, it does so in layers. Each command in the Dockerfile creates a new layer when it executes. When a command fails, the layer is not created. To view the filesystem of the image when the command failed, we can just run the image that contains all the preceding layers.
Luckily, when you build a Dockerfile, Docker shows you the unique reference for each layer it creates - it's the random strings of numbers and letters like b1f30360c452 in the following example:
Step 1/13 : FROM microsoft/aspnetcore-build:2.0.3 AS builder
---> b1f30360c452
Step 2/13 : WORKDIR /sln
---> Using cache
---> 4dee75249988
Step 3/13 : COPY ./build.sh ./build.cake ./NuGet.config ./aspnetcore-in-docker.sln ./
---> fee6f958bf9f
Step 4/13 : RUN ./build.sh -Target=Clean
---> Running in ab207cd28503
/usr/bin/env: 'bash\r': No such file or directory
This build failed executing the ./build.sh -Target=Clean command. To view the filesystem we can create a container from the image created by the previous layer, fee6f958bf9f by calling docker run. We can execute a bash shell inside the container, and inspect the contents of the filesystem (or do anything else we like).
docker run --rm -it fee6f958bf9f /bin/bash
The arguments are as follows:
--rm- when we exit the container, it will be deleted. this prevents the build up of exited transient images like this.-it- create an "interactive" session. When the docker container starts up, you will be connected to it, rather than it running in the background.fee6f958bf9f- the image layer to run in, taken from our failed output./bin/bash- The command to execute in the container when it starts. Using/bin/bashcreates a shell, so you can execute commands and generally inspect the filesystem.
Note that
--rmdeletes the container, not the image. A container is basically a running image. Think of the image as a hard drive, it contains all the details on how to run a process, but you need a computer or a VM to actually do anything with the hard drive.
Once you've looked around and figured out the problem, you can quit the bash shell by running exit, which will also kill the container.
This docker run command works if you want to inspect an image during a failed build, but what if your build succeeded, and now you want to check the filesystem rather than running the app it contains? For that you'll need the command in the next section.
Examining the file system of an image with an ENTRYPOINT
Lets say your Docker build succeeded, but for some reason your app isn't starting correctly when you call docker run. You suspect there may be a missing file, and so you want to inspect the filesystem. Unfortunately, the command in the previous section won't work for you. If you try it, you'll probably be greeted with an error that looks like the following:
$ docker run -it --rm myImage /bin/bash
Unhandled Exception: System.FormatException: Value for switch '/bin/bash' is missing.
at Microsoft.Extensions.Configuration.CommandLine.CommandLineConfigurationProvider.Load(
The problem, is that the Docker image you're running (myImage) in this case, already includes an ENTRYPOINT command in the Dockerfile used to build it. The ENTRYPOINT defines the command to run when the container is started, which for ASP.NET Core apps, typically looks something like the following:
ENTRYPOINT ["dotnet", "MyApp.dll"]
With the previous example, the /bin/bash argument is actually passed as an extra command line argument to the previously-defined entrypoint, so you're actually running dotnet MyApp.dll /bin/bash in the container when it starts up.
In order to override the entrypoint, and to just run the shell directly, you need to use the --entrypoint argument instead. Note that the argument order is different in this case - the image name goes at the end in this command, whereas the shell was at the end in the previous example
$ docker run --rm -it --entrypoint /bin/bash myImage
You'll now have a shell inside the container that you can use to inspect the contents or run other commands.
Copying files from a Docker container to the host
If you're not particularly au fait with Linux (🙋) then trying to diagnose a problem from inside a shell can be tricky. Sometimes, I'd rather copy a file back from a container and inspect it on the Windows side, using tools I'm familiar with. I typically have folders mapped between my Windows machine and my Linux VM, so I just need to get the file out of the Docker container and into the Linux host.
If you have a running (or stopped) container, you can copy a file from the container to the host with the docker cp command. For example, if you created a container called my-image-app from the myImage container using:
docker run -d --name my-image-app myImage
then you could copy a file from the container to the host using something like the following:
docker cp my-image-app:/app/MyApp.dll /path/on/the/host/MyApp.dll
Copying files from a Docker image to the host
The previous example shows how to copy files from a container, but you can't use this to copy files directly from an image. Unfortunately, to do that you have to create a container from the image and run it.
There's a number of ways you can do this, depending on exactly what state your image is in (e.g. does it have a defined ENTRYPOINT), but I tend to just use the following:
docker run --rm -it --entrypoint cat myImage /app/MyApp.dll > /path/on/the/host/MyApp.dll
This gives me a single command I can run to grab the /app/MyApp.dll file from the image, and write it out to /path/on/the/host/MyApp.dll. It relies on the cat command being available, which it is for the ASP.NET Core base image. If that's not available, you'll essentially have to manually create a container from your image, copy the file across, and then kill the container. For example:
id=$(docker create myImage)
docker cp $id:/app/MyApp.dll /path/on/the/host/MyApp.dll
docker rm -v $id
If the image has a defined ENTRYPOINT you may need to override it in the docker create call:
id=$(docker create --entrypoint / andrewlock/aspnetcore-in-docker)
docker cp $id:/app/MyApp.dll /path/on/the/host/MyApp.dll
docker rm -v $id
That gives you three ways to copy files out of your containers - hopefully at least one of them works for you!
Summary
That's it for this first post of Docker commands. Hope you find them useful!