Answers.Body: <p>I have explored the AI for edge devices. My findings for <code>tflite</code> model.</p>
  
  <ol>
  <li><code>TFLite</code> is just the tool suite to convert <code>TFModel</code> into <code>TFLite</code>.</li>
  <li>TFLite optimizes the model for edge or embedded devices using <a href="https://medium.com/analytics-vidhya/optimization-techniques-tflite-5f6d9ae676d5" rel="nofollow noreferrer">quantization techniques</a>.</li>
  </ol>
  
  <blockquote>
    <p>Quantization dramatically reduces both the memory requirement and
    computational cost of using neural networks.</p>
  </blockquote>
  
  <p><strong>Answer in brief:</strong></p>
  
  <ol>
  <li>When we optimize the model definitely they are faster to take 
  inference but it will impact the accuracy. Quantization leads to bit accuracy loss in smaller networks.</li>
  <li>TFLite is increasing the model performance but we have to pay a cost in terms of accuracy that's why we have both TFLite for edge computation and TFModel as well.</li>
  </ol>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-03T05:19:43.900Z
Answers.LastActivityDate: 2020-02-03T05:19:43.900Z
Answers.UserDisplayName: Maheshwar Ligade
Answers.Body: <p>This partly answer to question 1.
  There is no general rule concerning accuracy or size of the model. It depends on the training data and the processed data.
  The lightest is your model compared to the full accuracy model the less accurate it will be. I would run the lite model on test data and compare to the accuracy of the full model to get an exact measure of the difference.</p>
  
  <p>Tensor flow has different options to save the "lite" model (optimized in size, latency, none and default). </p>
  
  <p>The following mostly answer question 2. </p>
  
  <p>Tensor flow lite is intented to provide the ability to use the model to on line <strong>predict only</strong> and load the model not to train the model. </p>
  
  <p>On the other hand Tensor flow is used to build (train) the model off line.</p>
  
  <p>If your edge platform support any of the binding language provided for TensorFlow (javascript, java/kotlin, C++, python) you can use Tensorflow for prediction. The accuracy or speed options you might have selected to create the model will not be affected whether you use Tensor Flow or Tensor Flow Lite. Typically Tensor flow lite can be used on mobile devices (Ios, Android). There are other supported target, see this <a href="https://www.tensorflow.org/lite/microcontrollers" rel="nofollow noreferrer">link</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-12-18T13:50:35.103Z
Answers.LastActivityDate: 2020-01-09T13:38:20.123Z
Answers.LastEditDate: 2020-01-09T13:38:20.123Z
Answers.UserDisplayName: pascal sautot
Body: <p>I have been exploring edge computation for AI, and I came across multiple libraries or frameworks, which can help to convert the model into a lite format, which is suitable for edge devices.</p>
  <ul>
  <li><a href="https://www.tensorflow.org/lite" rel="nofollow noreferrer">TensorFlow Lite</a> will help us to convert the TensorFlow model into TensorFlow lite.</li>
  <li><a href="https://docs.openvinotoolkit.org/latest/index.html" rel="nofollow noreferrer">OpenVino</a> will optimise the model for edge devices.</li>
  </ul>
  <p><strong>Questions</strong></p>
  <ol>
  <li><p>If we have a library to optimise the model for edge devices (e.g. TensorFlow Lite), after conversion, could it make the accuracy decrease?</p>
  </li>
  <li><p>If not, then why do people prefer don't always use e.g. TensorFlow Lite?</p>
  </li>
  </ol>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-02-17T22:42:35.613Z
Comments.Text: Note that [the comparison of software is off-topic here now](https://ai.stackexchange.com/help/on-topic). However, I tried to reformulate your post so that it's more acceptable and that the question is more focused on the conceptual issue (although I'm still not fully sure it's on-topic).
Comments.UserDisplayName: nbro
Tags: <deep-learning><tensorflow><accuracy><efficiency>
Title: Why don't people always use TensorFlow Lite, if it doesn't decrease the accuracy of the models?
UserDisplayName: Maheshwar Ligade
ThreadText: # Why don't people always use TensorFlow Lite, if it doesn't decrease the accuracy of the models?
  
  (Asked by: username_0 on 2019-12-18)
  
  I have been exploring edge computation for AI, and I came across multiple libraries or frameworks, which can help to convert the model into a lite format, which is suitable for edge devices.
  
  * [TensorFlow Lite](https://www.tensorflow.org/lite) will help us to convert the TensorFlow model into TensorFlow lite.
  * [OpenVino](https://docs.openvinotoolkit.org/latest/index.html) will optimise the model for edge devices.
  
  **Questions**
  
  1. If we have a library to optimise the model for edge devices (e.g. TensorFlow Lite), after conversion, could it make the accuracy decrease?
  2. If not, then why do people prefer don't always use e.g. TensorFlow Lite?
  
  **Tags:** deep-learning, tensorflow, accuracy, efficiency
  
  > **Comments:**
  > **username_1**: Note that [the comparison of software is off-topic here now](https://ai.stackexchange.com/help/on-topic). However, I tried to reformulate your post so that it's more acceptable and that the question is more focused on the conceptual issue (although I'm still not fully sure it's on-topic).  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_0 on 2020-02-03. Score: 1 | Accepted answer
  
  I have explored the AI for edge devices. My findings for `tflite` model.
  
  1. `TFLite` is just the tool suite to convert `TFModel` into `TFLite`.
  2. TFLite optimizes the model for edge or embedded devices using [quantization techniques](https://medium.com/analytics-vidhya/optimization-techniques-tflite-5f6d9ae676d5).
  
  > Quantization dramatically reduces both the memory requirement and
  > computational cost of using neural networks.
  
  **Answer in brief:**
  
  1. When we optimize the model definitely they are faster to take
     inference but it will impact the accuracy. Quantization leads to bit accuracy loss in smaller networks.
  2. TFLite is increasing the model performance but we have to pay a cost in terms of accuracy that's why we have both TFLite for edge computation and TFModel as well.
  
  
  ---
  
  ### Answer by username_2 on 2019-12-18. Score: 3
  
  This partly answer to question 1.
  There is no general rule concerning accuracy or size of the model. It depends on the training data and the processed data.
  The lightest is your model compared to the full accuracy model the less accurate it will be. I would run the lite model on test data and compare to the accuracy of the full model to get an exact measure of the difference.
  
  Tensor flow has different options to save the "lite" model (optimized in size, latency, none and default).
  
  The following mostly answer question 2.
  
  Tensor flow lite is intented to provide the ability to use the model to on line **predict only** and load the model not to train the model.
  
  On the other hand Tensor flow is used to build (train) the model off line.
  
  If your edge platform support any of the binding language provided for TensorFlow (javascript, java/kotlin, C++, python) you can use Tensorflow for prediction. The accuracy or speed options you might have selected to create the model will not be affected whether you use Tensor Flow or Tensor Flow Lite. Typically Tensor flow lite can be used on mobile devices (Ios, Android). There are other supported target, see this [link](https://www.tensorflow.org/lite/microcontrollers)
  
  
  ---
  
  