Answers: <p>There are options in docker that will retry the startup of a container, <a href="https://docs.docker.com/compose/compose-file/#restart" rel="nofollow noreferrer">restart in version 2</a> and <a href="https://docs.docker.com/compose/compose-file/#restart_policy" rel="nofollow noreferrer">restart-policy in version 3</a> (you'd need compatibility mode enabled for the version 3 syntax to work). However, I believe they only work when the issue is from the application inside the container fails, not when there is an issue creating the container like you see with a volume mount failing (or would also happen if the image couldn't be retrieved from a registry).</p>
  
  <p>To handle the failing volume mount, I believe swarm mode is your best option despite your objections. You can create a single node cluster with <code>docker swarm init</code> and deploy your compose file with <code>docker stack deploy -c docker-compose.yml stack_name</code>, making an easy transition from docker-compose. Swarm mode looks at the overall state of the service and continuously tries to make the current state match your target state (as defined in the compose file), which will handle a failing volume mount that eventually corrects itself. I don't have an NFS server to test on right now, but here's a scenario with a bind mount to a missing folder:</p>
  
  <pre><code>$ cat docker-compose.vol-bind.yml
  version: '3'
  volumes:
    bind-test:
      driver: local
      driver_opts:
        type: none
        o: bind
        device: /home/bmitch/data/docker/test/missing
  services:
    bind-test:
      image: busybox
      command: tail -f /dev/null
      volumes:
        - bind-test:/bind-test
  
  $ docker stack deploy -c docker-compose.vol-bind.yml voltest   
  Creating network voltest_default                              
  Creating service voltest_bind-test   
  
  $ docker service ls
  ID                  NAME                   MODE                REPLICAS            IMAGE                                             PORTS
  omzaeo7mrour        voltest_bind-test      replicated          0/1                 busybox:latest
  
  $ docker service ps omzaeo7mrour
  ID                  NAME                      IMAGE               NODE                DESIRED STATE       CURRENT STATE           ERROR                              POR
  TS
  kpz0l79eucaw        voltest_bind-test.1       busybox:latest      bmitch-asusr556l    Ready               Ready 2 seconds ago
  j6fylzhvcv60         \_ voltest_bind-test.1   busybox:latest      bmitch-asusr556l    Shutdown            Failed 5 seconds ago    "starting container failed: er…"
  61o6raohp0xl         \_ voltest_bind-test.1   busybox:latest      bmitch-asusr556l    Shutdown            Failed 12 seconds ago   "starting container failed: er…"
  
  $ docker inspect kpz0l79eucaw
  [
      {
          "ID": "kpz0l79eucaw1856obmwvcak1",
          "Version": {
              "Index": 445
          },
          "CreatedAt": "2019-04-29T12:57:25.925788528Z",
          "UpdatedAt": "2019-04-29T12:57:34.3467203Z",
          "Labels": {},
          "Spec": {
              "ContainerSpec": {
                  "Image": "busybox:latest",
                  "Labels": {
                      "com.docker.stack.namespace": "voltest"
                  },
  ...
          "Status": {
              "Timestamp": "2019-04-29T12:57:33.936048295Z",
              "State": "failed",
              "Message": "starting",
              "Err": "starting container failed: error while mounting volume '/home/var-docker/volumes/voltest_bind-test/_data': failed to mount local volume: mount /home
  /bmitch/data/docker/test/missing:/home/var-docker/volumes/voltest_bind-test/_data, flags: 0x1000: no such file or directory",
              "ContainerStatus": {
                  "ContainerID": "4c75c851bd43b5ef57b4785a4611f78501279933822eab90c693e1108f53ee82",
                  "PID": 0,
                  "ExitCode": 128
              },
              "PortStatus": {}
          },
  ...
  
  $ mkdir missing
  
  $ docker service ps omzaeo7mrour
  ID                  NAME                      IMAGE               NODE                DESIRED STATE       CURRENT STATE           ERROR                              POR
  TS
  okylli7smu1c        voltest_bind-test.1       busybox:latest      bmitch-asusr556l    Running             Running 4 seconds ago
  9tpm188ysu2k         \_ voltest_bind-test.1   busybox:latest      bmitch-asusr556l    Shutdown            Failed 14 seconds ago   "starting container failed: er…"
  kpz0l79eucaw         \_ voltest_bind-test.1   busybox:latest      bmitch-asusr556l    Shutdown            Failed 21 seconds ago   "starting container failed: er…"
  </code></pre>
  
  <p>From the above, you can see as soon as the missing directory was created, the bind mount succeeded, and the container that failed to be created as retried and started successfully.</p>
  
Answers: CC BY-SA 4.0
Answers: 2019-04-30T14:02:58.990Z
Answers: I cannot use Swarm since some services I host require access to devices on the host. And `devices` cannot be specified in YML for Docker Stacks.
Answers: void.pointer
Answers: CC BY-SA 4.0
Answers: 2019-04-30T19:15:26.483Z
Answers: @void.pointer you can use an ugly workaround for that: https://github.com/docker/swarmkit/issues/1244#issuecomment-394343097
Answers: BMitch
Answers: CC BY-SA 4.0
Answers: 2019-04-29T13:03:59.650Z
Answers: 2019-04-29T13:03:59.650Z
Answers: BMitch
Answers: <p>I know the question is old, but as I assume others also will run into this requirement...</p>
  <p>As OP is bound to this exactly host anyway (reason to use docker-copose over swarm), I suggest another route.</p>
  <p>NFS on Synology is a PITA. One has to manage UIDs to be the same on server and client. It's impossible to change UIDs on Synology, hence only client side UIDs can be adopted to fit server UIDs, which can cause ID clashes and is cumbersome. For this reasons I went the SMB/CIFS route.</p>
  <p>To be able to mounted a Synology share to a host directory vie the <code>/etc/fstab</code> file. Authentication is done by <code>credentials=/.smbcredentials_3</code> option. This subdirectory of this then is mounted to the container. This is an easy setup in the compose.yml.</p>
  <p>Regarding reboots of the NAS. I run this now since two years without having any issues. I can't remember how it's done, but AFAIR the remount is done autamaically by the host.</p>
  <p>My containers are short lived and created by a cron job. In case of a failure due to the NAS shutdown / reboot, the next run will find the mounted NAS share again.</p>
  
Answers: CC BY-SA 4.0
Answers: 2021-04-19T11:35:41.113Z
Answers: 2021-04-19T11:35:41.113Z
Answers: andreas
Body: <p>I host various docker containers on my Ubuntu 18 machine. A few of them require storing their data on my Synology NAS. At first, I was using the host machine's <code>/etc/fstab</code> to control NFS mounts, which I then mounted in the containers (as a mount, not a volume).</p>
  
  <p>However, I figured it would be a better idea to have the containers map NFS to mount points in the containers. The host really has no use for the mounts, so it didn't make sense to maintain them there.</p>
  
  <p>At the moment, I am configuring my NFS volumes like this (using Docker Compose v3 format):</p>
  
  <pre><code>volumes:
    data:
      driver_opts:
        type: nfs
        o: addr=192.168.1.51,nolock,soft,rw
        device: :/volume2/nextcloud
  </code></pre>
  
  <p>This works great when the NAS is booted and working normally. However, I had a power outage and noticed all sorts of problems. Also, the timing of which machines boot first (NAS vs Ubuntu box) affects reliability of my docker container volumes. In my last situation, the NAS was not powered on. So when the container was started, it failed:</p>
  
  <blockquote>
    <p>ERROR: for app  Cannot start service app: error while mounting volume '/var/lib/docker/volumes/nextcloud_data/_data': error while mounting volume with options: type='nfs' device=':/volume2/nextcloud' o='addr=192.168.1.51,nolock,soft,rw': no route to host</p>
  </blockquote>
  
  <p>What would be nice is if docker would keep retrying to mount the volume until the NAS was powered on again. That would make this hands-free and prevent any timing issues (on which devices boot first across the network) from causing permanent failures like this.</p>
  
  <p>I'm also not sure what happens if a volume is created, and the NAS is powered off at any point. Does the volume stay available? Does docker keep trying to reconnect the NFS mount? I feel like there is very little control here.</p>
  
  <p>Note I just use Docker Compose. I don't use Swarm for technical reasons I won't go into here. Can someone recommend a way to resolve these reliability issues? Are NFS volumes in Docker the way to go? Should I go back to mounting on the host? Is there an amazing 3rd option out there somewhere?</p>
  
Comments: CC BY-SA 4.0
Comments: 2021-04-19T20:25:14.617Z
Comments: Intuition is to suggest that mounting on the host is better, for the sake on controlling runorder where nfsd may be failing while NAS is unavailable. My sense of it is that asking containers to phone home where they won't have any telemetry into the state of the filesystem share is just architecturally a dogfight..
Comments: rhoyerboat
Comments: CC BY-SA 4.0
Comments: 2022-01-11T18:47:34.800Z
Comments: Did you resolve this issue finally?  I'm running into a similar situation where I am want to get my container launched even if my NFS server is down.
Comments: Eric B.
Comments: CC BY-SA 4.0
Comments: 2022-01-12T19:04:49.300Z
Comments: @EricB. No sadly. I haven't revisited this in quite a while (since I posted, actually). I am still using a NFS mount on my host OS and I just do a standard volume mount from that to my docker container.
Comments: void.pointer
Tags: <docker><docker-compose>
Title: How do I make NFS volumes reliable in Docker?
UserDisplayName: void.pointer
ThreadText: # How do I make NFS volumes reliable in Docker?
  
  (Asked by: username_0 on 2019-04-27)
  
  I host various docker containers on my Ubuntu 18 machine. A few of them require storing their data on my Synology NAS. At first, I was using the host machine's `/etc/fstab` to control NFS mounts, which I then mounted in the containers (as a mount, not a volume).
  
  However, I figured it would be a better idea to have the containers map NFS to mount points in the containers. The host really has no use for the mounts, so it didn't make sense to maintain them there.
  
  At the moment, I am configuring my NFS volumes like this (using Docker Compose v3 format):
  
  ```
  volumes:
    data:
      driver_opts:
        type: nfs
        o: addr=192.168.1.51,nolock,soft,rw
        device: :/volume2/nextcloud
  
  ```
  
  This works great when the NAS is booted and working normally. However, I had a power outage and noticed all sorts of problems. Also, the timing of which machines boot first (NAS vs Ubuntu box) affects reliability of my docker container volumes. In my last situation, the NAS was not powered on. So when the container was started, it failed:
  
  > ERROR: for app Cannot start service app: error while mounting volume '/var/lib/docker/volumes/nextcloud\_data/\_data': error while mounting volume with options: type='nfs' device=':/volume2/nextcloud' o='addr=192.168.1.51,nolock,soft,rw': no route to host
  
  What would be nice is if docker would keep retrying to mount the volume until the NAS was powered on again. That would make this hands-free and prevent any timing issues (on which devices boot first across the network) from causing permanent failures like this.
  
  I'm also not sure what happens if a volume is created, and the NAS is powered off at any point. Does the volume stay available? Does docker keep trying to reconnect the NFS mount? I feel like there is very little control here.
  
  Note I just use Docker Compose. I don't use Swarm for technical reasons I won't go into here. Can someone recommend a way to resolve these reliability issues? Are NFS volumes in Docker the way to go? Should I go back to mounting on the host? Is there an amazing 3rd option out there somewhere?
  
  **Tags:** docker, docker-compose
  
  > **Comments:**
  > **username_1**: Intuition is to suggest that mounting on the host is better, for the sake on controlling runorder where nfsd may be failing while NAS is unavailable. My sense of it is that asking containers to phone home where they won't have any telemetry into the state of the filesystem share is just architecturally a dogfight..  
  > **username_2**: Did you resolve this issue finally?  I'm running into a similar situation where I am want to get my container launched even if my NFS server is down.  
  > **username_0**: @EricB. No sadly. I haven't revisited this in quite a while (since I posted, actually). I am still using a NFS mount on my host OS and I just do a standard volume mount from that to my docker container.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_3 on 2019-04-29. Score: 2
  
  There are options in docker that will retry the startup of a container, [restart in version 2](https://docs.docker.com/compose/compose-file/#restart) and [restart-policy in version 3](https://docs.docker.com/compose/compose-file/#restart_policy) (you'd need compatibility mode enabled for the version 3 syntax to work). However, I believe they only work when the issue is from the application inside the container fails, not when there is an issue creating the container like you see with a volume mount failing (or would also happen if the image couldn't be retrieved from a registry).
  
  To handle the failing volume mount, I believe swarm mode is your best option despite your objections. You can create a single node cluster with `docker swarm init` and deploy your compose file with `docker stack deploy -c docker-compose.yml stack_name`, making an easy transition from docker-compose. Swarm mode looks at the overall state of the service and continuously tries to make the current state match your target state (as defined in the compose file), which will handle a failing volume mount that eventually corrects itself. I don't have an NFS server to test on right now, but here's a scenario with a bind mount to a missing folder:
  
  ```
  $ cat docker-compose.vol-bind.yml
  version: '3'
  volumes:
    bind-test:
      driver: local
      driver_opts:
        type: none
        o: bind
        device: /home/username_3/data/docker/test/missing
  services:
    bind-test:
      image: busybox
      command: tail -f /dev/null
      volumes:
        - bind-test:/bind-test
  
  $ docker stack deploy -c docker-compose.vol-bind.yml voltest   
  Creating network voltest_default                              
  Creating service voltest_bind-test   
  
  $ docker service ls
  ID                  NAME                   MODE                REPLICAS            IMAGE                                             PORTS
  omzaeo7mrour        voltest_bind-test      replicated          0/1                 busybox:latest
  
  $ docker service ps omzaeo7mrour
  ID                  NAME                      IMAGE               NODE                DESIRED STATE       CURRENT STATE           ERROR                              POR
  TS
  kpz0l79eucaw        voltest_bind-test.1       busybox:latest      username_3-asusr556l    Ready               Ready 2 seconds ago
  j6fylzhvcv60         \_ voltest_bind-test.1   busybox:latest      username_3-asusr556l    Shutdown            Failed 5 seconds ago    "starting container failed: er…"
  61o6raohp0xl         \_ voltest_bind-test.1   busybox:latest      username_3-asusr556l    Shutdown            Failed 12 seconds ago   "starting container failed: er…"
  
  $ docker inspect kpz0l79eucaw
  [
      {
          "ID": "kpz0l79eucaw1856obmwvcak1",
          "Version": {
              "Index": 445
          },
          "CreatedAt": "2019-04-29T12:57:25.925788528Z",
          "UpdatedAt": "2019-04-29T12:57:34.3467203Z",
          "Labels": {},
          "Spec": {
              "ContainerSpec": {
                  "Image": "busybox:latest",
                  "Labels": {
                      "com.docker.stack.namespace": "voltest"
                  },
  ...
          "Status": {
              "Timestamp": "2019-04-29T12:57:33.936048295Z",
              "State": "failed",
              "Message": "starting",
              "Err": "starting container failed: error while mounting volume '/home/var-docker/volumes/voltest_bind-test/_data': failed to mount local volume: mount /home
  /username_3/data/docker/test/missing:/home/var-docker/volumes/voltest_bind-test/_data, flags: 0x1000: no such file or directory",
              "ContainerStatus": {
                  "ContainerID": "4c75c851bd43b5ef57b4785a4611f78501279933822eab90c693e1108f53ee82",
                  "PID": 0,
                  "ExitCode": 128
              },
              "PortStatus": {}
          },
  ...
  
  $ mkdir missing
  
  $ docker service ps omzaeo7mrour
  ID                  NAME                      IMAGE               NODE                DESIRED STATE       CURRENT STATE           ERROR                              POR
  TS
  okylli7smu1c        voltest_bind-test.1       busybox:latest      username_3-asusr556l    Running             Running 4 seconds ago
  9tpm188ysu2k         \_ voltest_bind-test.1   busybox:latest      username_3-asusr556l    Shutdown            Failed 14 seconds ago   "starting container failed: er…"
  kpz0l79eucaw         \_ voltest_bind-test.1   busybox:latest      username_3-asusr556l    Shutdown            Failed 21 seconds ago   "starting container failed: er…"
  
  ```
  
  From the above, you can see as soon as the missing directory was created, the bind mount succeeded, and the container that failed to be created as retried and started successfully.
  
  > **Comments:**
  > **username_0**: I cannot use Swarm since some services I host require access to devices on the host. And `devices` cannot be specified in YML for Docker Stacks.  
  > **username_3**: @username_0 you can use an ugly workaround for that: https://github.com/docker/swarmkit/issues/1244#issuecomment-394343097  
  
  ---
  
  ### Answer by username_4 on 2021-04-19. Score: 0
  
  I know the question is old, but as I assume others also will run into this requirement...
  
  As OP is bound to this exactly host anyway (reason to use docker-copose over swarm), I suggest another route.
  
  NFS on Synology is a PITA. One has to manage UIDs to be the same on server and client. It's impossible to change UIDs on Synology, hence only client side UIDs can be adopted to fit server UIDs, which can cause ID clashes and is cumbersome. For this reasons I went the SMB/CIFS route.
  
  To be able to mounted a Synology share to a host directory vie the `/etc/fstab` file. Authentication is done by `credentials=/.smbcredentials_3` option. This subdirectory of this then is mounted to the container. This is an easy setup in the compose.yml.
  
  Regarding reboots of the NAS. I run this now since two years without having any issues. I can't remember how it's done, but AFAIR the remount is done autamaically by the host.
  
  My containers are short lived and created by a cron job. In case of a failure due to the NAS shutdown / reboot, the next run will find the mounted NAS share again.
  
  
  ---
  
  