Answers.Body: <p><a href="https://www.tensorflow.org/lite/performance/model_optimization#quantization" rel="nofollow noreferrer">Quantization</a> is a technique used to make deep learning models smaller and faster to run. </p>
  
  <p>Deep learning models are essentially collections of real-valued numbers. Because there are infinitely many real numbers, computers represent them using a format call 'floating point' numbers, which are not completely accurate. For example, a 32-bit floating point number can only represent at most <span class="math-container">$2^{32}$</span> distinct values. In contrast, a 64-bit floating point number can represent <span class="math-container">$2^{64}$</span> distinct values. </p>
  
  <p>Most CPUs and GPUs cannot operate directly on large floating point numbers. This means that to do something like multiply two floating point numbers, the CPU might have to work on half of each number at a time, and do some tricky work to combine the results. Some CPUs and GPUs can operate directly on large floating point numbers, but only by using more than one core to work on a single number.</p>
  
  <p>To get around this, you might chose to take a model you have that was trained with high-precision floating point weights, and reduce it to lower precision. The weights won't be exactly the same, but they'll be very close. Doing this will make the models run much faster, but you might lose some accuracy.</p>
  
  <p>So the advantage of using a tool that only supports Quantized models is that models will run faster, but might be slightly less accurate.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-05-21T03:26:00.770Z
Answers.LastActivityDate: 2020-05-21T03:26:00.770Z
Answers.UserDisplayName: John Doucette
Body: <p>I was reading about the possibility of using Google's Coral for deep learning-based object detection and image classification. I heard it has a good speed in terms of frames/sec.</p>
  
  <p>I also read that Google's Coral is only compatible with quantized models. What does this mean? How will this affect the performance of object detection or classification in terms of accuracy and speed?</p>
  
  <p>What is the advantage of using Google's Coral over Nvidia's Xavier?</p>
  
Tags: <deep-learning><convolutional-neural-networks>
Title: What is the advantage of using Google's Coral over Nvidia's Xavier?
UserDisplayName: nikki2
ThreadText: # What is the advantage of using Google's Coral over Nvidia's Xavier?
  
  (Asked by: username_0 on 2020-05-20)
  
  I was reading about the possibility of using Google's Coral for deep learning-based object detection and image classification. I heard it has a good speed in terms of frames/sec.
  
  I also read that Google's Coral is only compatible with quantized models. What does this mean? How will this affect the performance of object detection or classification in terms of accuracy and speed?
  
  What is the advantage of using Google's Coral over Nvidia's Xavier?
  
  **Tags:** deep-learning, convolutional-neural-networks
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-05-21. Score: 1
  
  [Quantization](https://www.tensorflow.org/lite/performance/model_optimization#quantization) is a technique used to make deep learning models smaller and faster to run.
  
  Deep learning models are essentially collections of real-valued numbers. Because there are infinitely many real numbers, computers represent them using a format call 'floating point' numbers, which are not completely accurate. For example, a 32-bit floating point number can only represent at most $2^{32}$ distinct values. In contrast, a 64-bit floating point number can represent $2^{64}$ distinct values.
  
  Most CPUs and GPUs cannot operate directly on large floating point numbers. This means that to do something like multiply two floating point numbers, the CPU might have to work on half of each number at a time, and do some tricky work to combine the results. Some CPUs and GPUs can operate directly on large floating point numbers, but only by using more than one core to work on a single number.
  
  To get around this, you might chose to take a model you have that was trained with high-precision floating point weights, and reduce it to lower precision. The weights won't be exactly the same, but they'll be very close. Doing this will make the models run much faster, but you might lose some accuracy.
  
  So the advantage of using a tool that only supports Quantized models is that models will run faster, but might be slightly less accurate.
  
  
  ---
  
  