LSTM returns nan after using the pretrained BERT embedding as input 
Hello ,
 i'm using the pretrained Bert model (from pytorch-transformers) to get the contextual embedding of a written text ,  i summed the last 4 hidden layers outputs (i red that the concatenation of the last four layers usually produce the best results )
than i use a LSTM layer with attention to get the paragraph level embedding from the word embedding produced by the Bert model
the output should be a score range [-1,1]
I tried the rmsprop , adam ... optimizers with MSEloss and always after just  few batches iterations the lstm layer produces nan values .
 any suggestions will be greatly appreciated
i found that the problem is related to the data
i will close this issue , or if anyone has the permission he could delete this :) 