Learning from scratch not working
I'm using simple_lm_learning as it was, except I didn't use the model from pretrained, but a new BertForPreTraining model with the same config as bert-base, how come it's not learning anything and predicting the same token 1996 ("the") for every output?
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
