Hi thanks for your reply ! 
I admit using relu as a non linearity (or the quantization at that step) may not have been the best example thanks for pointing it out 
To be more precise on the problem at hand, I need to diagnose if the model will have a correct performance by only using few bits at each layer (ideally binarized). Thus only applying at the output will still allow the model to use 32/16FP at each hidden layer.
The goal would be to artificially constraint a layer’s output bitwidth, as if the model was operating in a binary setting.
See i.e. https://github.com/itayhubara/BinaryNet.pytorch 17 where they supercharge nn.Conv2d or nn.Linear. This is exactly the same use case but I’d like to avoid meddling with each implementation to replace nn.Conv2d by its binarized alter ego : making use of an external class to manage quantization seems much more “clean” and would allow to test rapidly a range of different quantization solutions.