You can use the anomaly detection tool 82 to check where the nans are produced. In my experience nans appear when you have high values in your gradients or if you are doing a mathematically undefined operation (e.g log(0)). In this case the latter is not likely, so it must be the former. Try and double check that your dataloader is working correctly and that it is providing you with the correct annotations.