Menu

Change Docker storage driver to avoid Overlay issue

Symptom

QASymphony's Postgres container (qasymphony/postgres:9.4) can't be started with the below error during startup

Digest: sha256:ef0264c1500f62d0acd4d0d...3ae6ad184fcce61
Status: Image is up to date for qasymphony/postgres:9.4
container 1fab31b723b55b30d41e873f24e47...68 not running
postgres cannot be started
Error response from daemon: Container 1fab31b723b55b30d41e...72bd4e0ec4f8fb4b1e6768 is not running
Reconfigure postgres max_connections setting...
Error response from daemon: Container 1fab31b723b55b30d41e873f...e0ec4f8fb4b1e6768 is not running
Restarting postgres...
container 3355cab010b2de0dc5701fcfceaac858141e1f46de...6484a not running
postgres cannot be started
... output truncated ...

 

Cause

Docker Engine is using "overlay" as its storage driver and most version of RHEL/CentOS kernel 3.10 is lacking the support or having issue supporting xfs, thus the overlay storage driver Docker is using.

$ sudo docker info
Containers: 0
Images: 2
Server Version: 17.03.1-ce
Storage Driver: overlay
 Backing Filesystem: xfs
Logging Driver: json-file
Kernel Version: 3.10.0-514.el7.x86_64
Operating System: Red Hat Enterprise Linux Server 7.3 (Maipo)
... output truncated ...

 

Docker will not run correctly if your kernel is older than version 3.10 or if it is missing some modules. To check kernel compatibility, you can download and run the check-compatibility.sh script.

-- https://docs.docker.com

Solution

By changing the storage driver, all your current Docker resources (containers, images, volumes) will be unavailable for accessing by the new storage driver.

Make sure you have backup all the data in your containers before proceeding

Abstraction

Change Docker Engine to use a different storage driver rather than "overlay". Suggest using "devicemapper"

HOW-TO

Identify the configuration file for Docker daemon, the default value should be "/usr/lib/systemd/system/docker.service"

$ sudo systemctl status docker
 docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; disabled; vendor preset: disabled)
   Active: active (running) since Tue 2017-04-04 01:13:46 EDT; 6min ago
     Docs: https://docs.docker.com
 Main PID: 17003 (dockerd)
   Memory: 315.3M
   CGroup: /system.slice/docker.service
           17003 /usr/bin/dockerd
           17006 docker-containerd -l unix:///var/run/docker/libcontainerd/docker-containerd.sock --metrics-interval=0 --start-timeout 2m --state-dir /var/run/doc...
           17500 docker-containerd-shim 908d301529b81953c3c6a2e8c170d95f5196d0d555ec14197455612bf47f7949 /var/run/docker/libcontainerd/908d301529b81953c3c6a2e8c17...

Use a text editor to edit the configuration file

Under the "[Service]" section, append "--storage-driver=devicemapper" at the end of the ExecStart configuration. Then save and close the file

ExecStart=/usr/bin/dockerd --storage-driver=devicemapper

Reload the daemon configuration

$ sudo systemctl daemon-reload

Restart the Docker daemon

$ sudo systemctl restart docker

Double check to make sure the storage driver has been changed

$ sudo docker info
Containers: 0
Images: 2
Server Version: 17.03.1-ce
Storage Driver: devicemapper
 Backing Filesystem: xfs
Logging Driver: json-file
Kernel Version: 3.10.0-514.el7.x86_64
Operating System: Red Hat Enterprise Linux Server 7.3 (Maipo)
... output truncated ...


References

Docker - Select a storage driver: https://docs.docker.com/engine/userguide/storagedriver/selectadriver/

Docker - Change engine/daemon configuration: https://docs.docker.com/engine/admin/

Docker - Control and configure Docker with systemd: https://docs.docker.com/engine/admin/systemd/

Docker - Kernel compatibility: https://docs.docker.com/engine/installation/linux/linux-postinstall/#kernel-compatibility

Subscribe To Our Blog
Powered by Zendesk