@driazati  Yup, that issue is resolved but it fails now at this stage:
if hasattr(self, '_qkv_same_embed_dim') and self._qkv_same_embed_dim is False:
           ~~~~~~~ <--- HERE
            return F.multi_head_attention_forward(
                query, key, value, self.embed_dim, self.num_heads,
                self.in_proj_weight, self.in_proj_bias,
                self.bias_k, self.bias_v, self.add_zero_attn,
                self.dropout, self.out_proj.weight, self.out_proj.bias, 
                training=self.training,
                key_padding_mask=key_padding_mask, need_weights=need_weights, 
                attn_mask=attn_mask, use_separate_proj_weight=True,
                q_proj_weight=self.q_proj_weight, k_proj_weight=self.k_proj_weight,

Is it because hasattr  or and is not supported yet ?