NLP, Transfer Learning, Language Model, Ulmfit, Text Classification.
of layers : 4 Embedding size : 768 Once your model is able to predict correctly on your training set, the next thing you want to avoid is overfitting. Here are some regularizations that I tried to make sure my model would not overfit. Add dropout Add weight decay Add gradient clipping Here are the