title: How to minimize Google Cloud launch latency
body: <p>I have a persistent server that unpredictably receives new data from users, needing about 10 GPU instances to crank at the problem for about 5 minutes, and I send the answer back to the users.    The server itself is a cheap always-persistent single CPU Google Cloud instance. When a user request comes in, my code launches my 10 created but stopped Google Cloud GPU instances with </p>
  
  <pre><code>gcloud compute instances start (instance list)
  </code></pre>
  
  <p>In the rare case if the stopped instances don't exist (sometimes they get wiped) that's detected and they're recreated with</p>
  
  <pre><code>gcloud beta compute instances create (...)
  </code></pre>
  
  <p>This system all works fine.  My only complaint is that even with created but stopped instances, the launch time before my GPU code finally starts to run is about 5 minutes. Most of this is just the time for the instance itself to launch its Ubuntu host and call my code.. the delay once Ubuntu is running to start the GPU is only about 10 seconds.</p>
  
  <p>How can I reduce this 5 minute delay? I imagine most of it comes from Google having to copy over the 4GB of instance data to the target machine, but the startup time of (vanilla) Ubuntu adds probably 1 more minute.  I'm not even sure if I could quantify these two numbers independently, I only can measure the combined 3-7 minutes delay from the launch until my code starts responding. </p>
  
  <p>I don't think Ubuntu OS startup time is the major startup latency contributor since I timed an actual machine with the same Ubuntu and same GPU on my desk from poweron boot up and it began running my GPU code in 46 seconds.</p>
  
  <p>My goal is to get results back to my users as soon as possible, and that 5 minute startup delay is a bottleneck.</p>
  
  <p>Would making a smaller instance SIZE of say 2GB help?  What else can I do to reduce the latency?</p>