Answers.Body: <p>I do not know much about the Neural Compute Stick but I can tell you a little bit about the Coral Edge TPU, since I used it myself (which I think also applies to the Neural Compute Stick).</p>
  <p>The Edgte TPU is a specialized ASIC which is very efficient at the main calculations (convolutions, relu etc.) for neural network inferencing. That means it cannot be used for training* a neural network but for deploying a neural network in production after it has been trained and optimized/quantized (precision reduced from float32 to int8). But I think you already new that, as it seems from your question.</p>
  <p>Now to your actual question in terms of speed: You cannot really compare speed of such chip in terms of framerate alone, nor can you call it a visual processing unit or not. The google coral is a general ASIC that is very fast at doing convolutions/relus etc. For what you gonna use your neural network (like Image recognition, or maybe stock predicitons) is complitly up to you, and especially up to your neural network for which task it was trained for. The only limitation you have here is regarding the supported layer operations. E.g. it is not possible to do operations like 3D convolutions or some fancy new non-linear activation functions (There was an overview of supported operations in the docs that I cannot find right now).</p>
  <p>Furthermore the framerate also depends on your NN architecture, image resolution etc. so a comparison here is completly misleading. If you want to have a general indication of speed look at how many int8 operations it can handle per second (TOPS) also under what energy consuption (Watts), if you care.</p>
  <p>The main advantage of this unit compared to a GPU that is usually used for training+inferencing is much lower energy consumption and unit cost. With roughly 4 Watts on the Edge TPU (as far as I can remember) compared to e.g. 250 Watts GPU. Energy consumption also dictates the necessary cooling solution, which can just be a passive for the Edge Tpu in many cases. Regarding the unit costs, I guess you can easily sees that yourself, if you keep in mind that you can get &quot;nearly&quot; similar inferencing speeds than with a full workstation class GPU. Furthermore, such unit has a much smaller formfactor and weight, which makes it perfect for applications at the edge. (I should also add that the efficiency gain is also a lot due to the precision reduction to int8 (quantization). Quantization is also possible on GPUs.)</p>
  <blockquote>
  <p>And lastly, do i need to keep my neural network in a single computer board for training or can i have it at a cloud?</p>
  </blockquote>
  <p>I am not quite sure what you mean with this. The general workflow would look like this: you use a GPU for training your neural network, after training you optimize/quantize your network, and lastly you deploy your neural network to the &quot;computer board&quot; of your choice that just needs to have a CPU and the Edge TPU and do inferencing operations/predictions for your task.</p>
  <p>*Transfer learning is possible of e.g. the last layer. But no backpropagation for full NN training.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-14T23:47:32.630Z
Answers.LastActivityDate: 2020-11-15T00:04:57.227Z
Answers.LastEditDate: 2020-11-15T00:04:57.227Z
Answers.UserDisplayName: Marcel_marcel1991
Body: <p>I am trying understand machine learning inferece, and i would like to know what exactly is the difference between Google Coral USB and Movidius Intel Neural Compute Stick 2. From what i could gather the google coral USB speeds up the frame rate, but that doesn't look clear to me. My questions are:
  What exactly is the benefit from both of them in units? Like, is it frame rate? prediction speed? Are both visual processing units? And lastly, do i need to keep my neural network in a single computer board for training or can i have it at a cloud?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-11-15T00:59:05.553Z
Comments.Text: Hello. This question is off-topic here. Our site focuses on the theoretical aspects of artificial intelligence (which does not include questions about specific pieces of hardware). See https://ai.stackexchange.com/help/on-topic for more details.
Comments.UserDisplayName: nbro
Tags: <neural-networks><machine-learning><convolutional-neural-networks><inference>
Title: Difference between Neural Compute Stick 2 and Google Coral USB for edge computing
UserDisplayName: John Jones
ThreadText: # Difference between Neural Compute Stick 2 and Google Coral USB for edge computing
  
  (Asked by: username_0 on 2020-11-14)
  
  I am trying understand machine learning inferece, and i would like to know what exactly is the difference between Google Coral USB and Movidius Intel Neural Compute Stick 2. From what i could gather the google coral USB speeds up the frame rate, but that doesn't look clear to me. My questions are:
  What exactly is the benefit from both of them in units? Like, is it frame rate? prediction speed? Are both visual processing units? And lastly, do i need to keep my neural network in a single computer board for training or can i have it at a cloud?
  
  **Tags:** neural-networks, machine-learning, convolutional-neural-networks, inference
  
  > **Comments:**
  > **username_1**: Hello. This question is off-topic here. Our site focuses on the theoretical aspects of artificial intelligence (which does not include questions about specific pieces of hardware). See https://ai.stackexchange.com/help/on-topic for more details.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-11-14. Score: 0 | Accepted answer
  
  I do not know much about the Neural Compute Stick but I can tell you a little bit about the Coral Edge TPU, since I used it myself (which I think also applies to the Neural Compute Stick).
  
  The Edgte TPU is a specialized ASIC which is very efficient at the main calculations (convolutions, relu etc.) for neural network inferencing. That means it cannot be used for training\* a neural network but for deploying a neural network in production after it has been trained and optimized/quantized (precision reduced from float32 to int8). But I think you already new that, as it seems from your question.
  
  Now to your actual question in terms of speed: You cannot really compare speed of such chip in terms of framerate alone, nor can you call it a visual processing unit or not. The google coral is a general ASIC that is very fast at doing convolutions/relus etc. For what you gonna use your neural network (like Image recognition, or maybe stock predicitons) is complitly up to you, and especially up to your neural network for which task it was trained for. The only limitation you have here is regarding the supported layer operations. E.g. it is not possible to do operations like 3D convolutions or some fancy new non-linear activation functions (There was an overview of supported operations in the docs that I cannot find right now).
  
  Furthermore the framerate also depends on your NN architecture, image resolution etc. so a comparison here is completly misleading. If you want to have a general indication of speed look at how many int8 operations it can handle per second (TOPS) also under what energy consuption (Watts), if you care.
  
  The main advantage of this unit compared to a GPU that is usually used for training+inferencing is much lower energy consumption and unit cost. With roughly 4 Watts on the Edge TPU (as far as I can remember) compared to e.g. 250 Watts GPU. Energy consumption also dictates the necessary cooling solution, which can just be a passive for the Edge Tpu in many cases. Regarding the unit costs, I guess you can easily sees that yourself, if you keep in mind that you can get "nearly" similar inferencing speeds than with a full workstation class GPU. Furthermore, such unit has a much smaller formfactor and weight, which makes it perfect for applications at the edge. (I should also add that the efficiency gain is also a lot due to the precision reduction to int8 (quantization). Quantization is also possible on GPUs.)
  
  > And lastly, do i need to keep my neural network in a single computer board for training or can i have it at a cloud?
  
  I am not quite sure what you mean with this. The general workflow would look like this: you use a GPU for training your neural network, after training you optimize/quantize your network, and lastly you deploy your neural network to the "computer board" of your choice that just needs to have a CPU and the Edge TPU and do inferencing operations/predictions for your task.
  
  \*Transfer learning is possible of e.g. the last layer. But no backpropagation for full NN training.
  
  
  ---
  
  