Slow Decoding Speed when using BertForLMModel
I set the BertLMHeadModel as Decoder in my Seq2Seq model. It seems to work well in training. But when decoing, it decodes very slowly. I think there is no layer_past used in GPT2, XLNet in BertLMHeadModel and many attentions are computed repetitively? 

Yeah, exactly - we should / could add the layer cache for `BertLMHeadModel`. It's not trivial to do it, but feel free to give it a try. 