hi @skurzhanskyi, one other thing you could try is https://pytorch.org/blog/pytorch-1.6-released/#graph-mode-quantization 14 , which we just released today in v1.6.  It might be easier to make multiheadattention work in graph mode.
As far as first class quantization for nn.MultiheadAttention and nn.EmbeddingBag / nn.Embedding - we don’t have a specific timeline we can share, but it should be on the order of months (not weeks or years) - we have folks actively working on this.