Moving Docker's files to a custom location

By: (plus.google.com) +David Herron; Date: January 30, 2018

Tags: Docker

Docker is a wonderful tool that abstracts away all kinds of details about configuring and maintaining Linux Containers. The power to simply type "docker run image-name" and have a bunch of complexity automatically handled is great. But you may want to change Docker's defaults, and just how do you do so? In my case "/var/lib/docker" would be on an SSD drive, and to lengthen its lifetime I want to minimize the number of writes to that drive. Moving this directory to the SSD should help with that goal.

I have an Ubuntu system that's newly setup. Therefore I had to first set up Docker, using the instructions here: (docs.docker.com) https://docs.docker.com/install/linux/docker-ce/ubuntu/ Followed by post-install instructions: https://docs.docker.com/install/linux/linux-postinstall/

Transcript:

45  apt-get remove docker docker-engine docker.io
46  apt-get update
47  apt-get install apt-transport-https ca-certificates curl software-properties-common
48  curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
49  apt-key fingerprint 0EBFCD88
50  sudo add-apt-repository    "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) \
stable"
51  apt-get update
52  sudo add-apt-repository    "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) \
stable"
53  docker run hello-world
54  sudo groupadd docker
55  usermod -aG docker david
56  systemctl
57  systemctl enable docker

At this point Docker has its out-of-the-box configuration. It does successfully download and setup containers. What's more important is the file-system image for each container is on the SSD. As containers and images are created, destroyed, and executed, that probably means lots of write operations on the SSD.

The docker info command tells us a bunch of information about the system. One detail is the directory, /var/lib/Docker to keep the images and everything.

Because I used systemctl to enable Docker, we configure Docker using these instructions: (docs.docker.com) https://docs.docker.com/engine/admin/systemd/

That page says we can put a JSON file at /etc/docker/daemon.json

The dockerd documentation lists what can be put in that file: (docs.docker.com) https://docs.docker.com/engine/reference/commandline/dockerd//

{
  "data-root": "/home/docker/root",
  "storage-driver": "overlay2"
}

There's a file at /lib/systemd/system/docker.service containing some configuration settings. It was unclear if anything could be put in that file to affect the location of the data-root directory.

It worked to install the daemon.json file as shown.

Then this sequence of commands are run:

75  ls -d /var/lib/docker
76  ls -ld /var/lib/docker
77  mkdir -p /home/docker/root
78  chmod 711 /home/docker/root
79  ls -ld /home/docker/root
81  systemctl stop docker
82  (cd /var/lib/docker/; tar cf - .) | (cd /home/docker/root/; tar xvf -)
84  mv /var/lib/docker/ /var/lib/docker-old
85  systemctl start docker

We start by inspecting the current /var/lib/docker so the replacement, /home/docker/root, can have the same settings.

Next we stop docker so that the directory contents are quiescent.

Next we copy the current contents in /var/lib/docker to /home/docker/root using a tar pipeline.

Next we, for good measure, we move /var/lib/docker out of the way so that if we'll see whether Docker successfully used the new location for its files. If the declared docker-root direectory is missing, Docker will autocreate it for you.

Afterwards run docker info to check the docker-root location was changed.

Then you can run a couple Docker images, and verify if those images get created in the new docker-root location. This is sufficient to cause a Docker image to download and its container execute: docker run -it python bash

To verify:

103  docker ps -a
105  docker images -a
106  du /home/docker/root/ | less
107  du /var/lib/docker* | less

This lets us view the current set of containers/images and validate that the newly created container/image occurred in the new location.

Finally once you're satisfied Docker is using the new docker-root location:

108  rm -rf /var/lib/docker*
109  ln -s /home/docker/root /var/lib/docker
110  ls -l /var/lib

A symbolic link is left in the old location pointing to the new for good measure -- just in case some piece of software hard-coded that pathname.