Hi,
I am using the implementation for NCE training found in https://github.com/parthaca/examples/tree/master/word_language_model 82
After a few (6-9) backprops I start getting Nans. I suspect I generate very small numbers.
And the overall loss for the different batches results in Nan, any idea what may have gone wrong?
Should I normalize some of these outputs to make it work?
Would appreciate your advice,
Shiran