losses does not decrease when trainning  with TFTransfoXLLMHeadModel  . 
I inspect the source code , because  the weight  initializer in class TFAdaptiveSoftmaxMask  were all set to zeros.   This causes the gradient can't propagate back . please double check that .  

eg:   weight = self.add_weight(
                    shape=(r_idx - l_idx, d_emb_i,),
                    initializer='zeros',
                    trainable=True,
                    name="out_layers_._{}_._weight".format(i),
                )

initializer='zeros' should be some  random initializer like : TruncatedNormal, right?
Hey @gaiyongbo,

Thanks for the issue. Will ping our TransfoXL master - @TevenLeScao can you take look maybe? <|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
