Deep Learning, Artificial Intelligence, NLP, Self Supervised Learning, Quantization.
the input tensor. 2.2. Dynamic Quantization Since the second Adam state is strictly positive, in this work, dynamic tree quantization is extended for non-signed input tensors by re-purposing the sign bit. The dynamic tree quantization is extended with a fixed bit for the fraction. This extension is