Thank you for your advice,  I was able to locate the overflow.
nans appear in the 22/24 layer of my BERT model.
Since I do not control where exactly mixed precision is applied, what can I do to overcome the issue?