Solved by vitouphy in post #8 
              
              
                Thank you @Samarth_Mishra for trying to help. After going through it line-by-line, I have identified the problem. And as weird as it sounds, it has nothing to do with the model. It was this line. 
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased', additional_special_tokens=['_eos', '_go'…