You might lose some of your training accuracy, but will most likely gain more validation accuracy, which is your goal. Unfortunately I’m not really familiar with text data, so I don’t know, if any data augmentations are available or what kind of regularization is state of the art now.