<p>You could look into <a href="https://www.tensorflow.org/model_optimization/guide/pruning" rel="nofollow noreferrer">models pruning</a>. There are several <a href="https://towardsdatascience.com/model-compression-via-pruning-ac9b730a7c7b" rel="nofollow noreferrer">techniques</a> out here, and all of them aim to reduce the amount of parameters of a model without affecting its performance metrics. Of course less parameters means less calculation and therefore faster inference time.</p>

CC BY-SA 4.0
2022-02-12T22:54:23.817Z
Thank you. I haven't looked into pruning much because it's not as straightforward as the others I've done. I'll look more into it now.
theastronomist
CC BY-SA 4.0
2022-02-10T08:02:22.757Z
2022-02-10T08:02:22.757Z
Edoardo Guerriero
<p>I am aware of several ways to optimize a convolutional (or any) model after training to make inferencing quicker. I am currently implementing <a href="https://scortex.io/batch-norm-folding-an-easy-way-to-improve-your-network-speed/" rel="nofollow noreferrer">BatchNormalization Folding</a> and removing Dropout layers from the network. I am also aware of <a href="https://www.tensorflow.org/lite/performance/post_training_float16_quant" rel="nofollow noreferrer">post training quantization</a> (specifically 16-bit quantization for use on GPU).</p>
<p>Are there other layer optimization techniques that I can use other than quantization?</p>
<p>My current model uses, Conv2D, Activation(relu), BatchNormalization, Dropout, Dense layers.</p>
<p>Basic mnist Convnet metrics for 10K images, batch size of 1: (All have 98.92% accuracy)</p>
<ul>
<li>Original Network: 49.3s</li>
<li>Folded Network: 33.37s</li>
<li>Quantized Original: 7.449s</li>
</ul>

<convolutional-neural-networks><tensorflow><optimization><batch-normalization>
Additional Optimizations for Convolutional Models On Inferencing
theastronomist
# Additional Optimizations for Convolutional Models On Inferencing

(Asked by: username_0 on 2022-02-10)

I am aware of several ways to optimize a convolutional (or any) model after training to make inferencing quicker. I am currently implementing [BatchNormalization Folding](https://scortex.io/batch-norm-folding-an-easy-way-to-improve-your-network-speed/) and removing Dropout layers from the network. I am also aware of [post training quantization](https://www.tensorflow.org/lite/performance/post_training_float16_quant) (specifically 16-bit quantization for use on GPU).

Are there other layer optimization techniques that I can use other than quantization?

My current model uses, Conv2D, Activation(relu), BatchNormalization, Dropout, Dense layers.

Basic mnist Convnet metrics for 10K images, batch size of 1: (All have 98.92% accuracy)

* Original Network: 49.3s
* Folded Network: 33.37s
* Quantized Original: 7.449s

**Tags:** convolutional-neural-networks, tensorflow, optimization, batch-normalization


---

## 1 answers

### Answer by username_1 on 2022-02-10. Score: 2 | Accepted answer

You could look into [models pruning](https://www.tensorflow.org/model_optimization/guide/pruning). There are several [techniques](https://towardsdatascience.com/model-compression-via-pruning-ac9b730a7c7b) out here, and all of them aim to reduce the amount of parameters of a model without affecting its performance metrics. Of course less parameters means less calculation and therefore faster inference time.

> **Comments:**
> **username_0**: Thank you. I haven't looked into pruning much because it's not as straightforward as the others I've done. I'll look more into it now.  

---

