madlag:

I have almost finished my work on an extension of @VictorSanh work, I tried to make it as generic as possible, to be able to patch any network with only minimal additional work, and to include it in your own training infrastructure.
As @VictorSanh mentionned, it won’t probably be part of transformers, but a standalone tool. I will be releasing it in the following weeks (hopefully before end of month), so I hope you can wait until that point !
To be able to patch a network “on the fly” you can use the approach I used in pytorch_block_sparse  , using inspection of pytorch modules.


Hi @madlag
Is this extension referring to the method of quantizing the layers other than nn.Linear and nn.LSTM, which are not supported by the quantization api of PyTorch by default?
I have been experiencing issues while quantizing a GPT2 based model, where most of the layers contain nn.Conv1D, using the Movement Pruning notebook in examples.
Thanks,
Mrigank