Body: <p>I do R&amp;D work on spot GPU's on AWS and have been, so far, launching them from a 50GB AMI that I create every time I need to shut off the instance. This has been working pretty well, except that startup times/accessing data on the AMI takes a while at first access because the AMI has to be downloaded from S3 and unpacked (I think this is how it works). </p>
  
  <p>The reason I need to persist the image is that I want my configuration for python packages, as well as the packages themselves to be persisted, and for data to be saved locally in an uncompressed format. So, I can't just mount an NFS to solve this problem (although all of my data/code resides on an NFS in some format, and I just move the data over to the EBS once launched to decompress it. </p>
  
  <p>For some reason EFS is slow with dealing with lots of small files, but thats a separate issue). It seems that EC2 doesn't allow using an EBS volume as root volume for spot instances. </p>
  
  <p>Is there a way to achieve the above mentioned goals without purchasing the on-demand instance? </p>
  
Comments: CC BY-SA 3.0
Comments: 2017-09-19T07:38:06.703Z
Comments: What is the NFS client version of your AMI ?
Comments: Tensibai
Comments: CC BY-SA 3.0
Comments: 2017-09-19T08:29:11.690Z
Comments: Why does your data need to be stored in an uncompressed format? Why is EFS being slow a separate issue - if it was fast, would you use it?
Comments: user2640621
Comments: CC BY-SA 3.0
Comments: 2017-09-23T05:10:18.467Z
Comments: If EFS was fast, then I could store all of my uncompressed data on it, but I would still need a way to quickly access already installed/configured packages. The reason the data needs to stay uncompressed is that I don't want to have to copy it over to a readable format for image processing every time I start up the instance.
Comments: Austin Shin
Comments: CC BY-SA 3.0
Comments: 2017-09-23T18:48:27.103Z
Comments: NFS in general is subject to consistent latency, even under NFS 4.1 (though it is better than 4.0). A series of small files is actually not NFS' strong suit. What is your AMI based off of? Also, what Instance Type are you using?
Comments: Matt O.
Comments: CC BY-SA 3.0
Comments: 2017-09-23T19:10:13.103Z
Comments: I've used NFS successfully with small files before. This is a problem inherent to EFS, not NFS in general. To give you an idea, copying over 1000 small files (10KB each) from one folder on an EFS to another folder on the same EFS could take 5 minutes. My AMI is an Ubuntu 14.04 base image that I installed a bunch of python packages that I'd like to keep. I know that I could probably use docker, but the problem with that is I'd still have to run the docker image on first boot which would mean a 3-5 min install time.
Comments: Austin Shin
Tags: <amazon-ec2><amazon-ebs>
Title: What are some ways of persisting volumes when dealing with spot instances for R/D purposes?
UserDisplayName: Austin Shin
ThreadText: # What are some ways of persisting volumes when dealing with spot instances for R/D purposes?
  
  (Asked by: username_0 on 2017-09-18)
  
  I do R&D work on spot GPU's on AWS and have been, so far, launching them from a 50GB AMI that I create every time I need to shut off the instance. This has been working pretty well, except that startup times/accessing data on the AMI takes a while at first access because the AMI has to be downloaded from S3 and unpacked (I think this is how it works).
  
  The reason I need to persist the image is that I want my configuration for python packages, as well as the packages themselves to be persisted, and for data to be saved locally in an uncompressed format. So, I can't just mount an NFS to solve this problem (although all of my data/code resides on an NFS in some format, and I just move the data over to the EBS once launched to decompress it.
  
  For some reason EFS is slow with dealing with lots of small files, but thats a separate issue). It seems that EC2 doesn't allow using an EBS volume as root volume for spot instances.
  
  Is there a way to achieve the above mentioned goals without purchasing the on-demand instance?
  
  **Tags:** amazon-ec2, amazon-ebs
  
  > **Comments:**
  > **username_1**: What is the NFS client version of your AMI ?  
  > **username_2**: Why does your data need to be stored in an uncompressed format? Why is EFS being slow a separate issue - if it was fast, would you use it?  
  > **username_0**: If EFS was fast, then I could store all of my uncompressed data on it, but I would still need a way to quickly access already installed/configured packages. The reason the data needs to stay uncompressed is that I don't want to have to copy it over to a readable format for image processing every time I start up the instance.  
  > **username_3**: NFS in general is subject to consistent latency, even under NFS 4.1 (though it is better than 4.0). A series of small files is actually not NFS' strong suit. What is your AMI based off of? Also, what Instance Type are you using?  
  > **username_0**: I've used NFS successfully with small files before. This is a problem inherent to EFS, not NFS in general. To give you an idea, copying over 1000 small files (10KB each) from one folder on an EFS to another folder on the same EFS could take 5 minutes. My AMI is an Ubuntu 14.04 base image that I installed a bunch of python packages that I'd like to keep. I know that I could probably use docker, but the problem with that is I'd still have to run the docker image on first boot which would mean a 3-5 min install time.  
  
  ---
  
  ## 0 answers
  
  