NLP, Transfer Learning, Language Model, Ulmfit, Text Classification.
Increase embedding size Increase number of hidden layers Changing optimizer functions Changing learning rate After lots of attempts, here is the structure and hyperparameters that allowed my model to start learning : Neural network architecture: QRNN Structure Number of hidden layers: 2500 Number