Hello there,
I am training a model of predicting the nationality from the name, so I have 95 countries as labels.
But out of those 95 the model is always giving only among 5 or 6 countries as the output.
I have read some posts for this, @ptrblck says that it can be due to high learning rate(this could be the reason as I have high learning rate), some say that it is due to “batch_normalization” (Although, I haven’t used it in my model).
But I have used dropout(I have 3 layers for unigrams, bigrams and trigrams) and gradient clipping, I am wondering that if these 2 factors are also contributing to this problem or not?
PS :- Also I am decaying the learning rate by a decay rate of 0.99 per epoch, but still a high learning rate is there for about 150 epochs.
Any help is appreciated
Thanks in advance.