I’m working on native amp and found that above issue is not occurred because torch.cdist is included in Ops that autocast to float32.
However, I faces another issue. The targets in the code which is a list of dict of tensors (line 2) and it passes to the criterion.
It seems autocast cannot handle this type of input.
What is the best way to make it work with the autocast?
Currently, I disable autocast at each loss and cast the outputs and targets to float following this 4.
It seems loss functions are mostly calculated with float32. Is it right?
I wonder if there is a float_function like decorator for the method (that you informed above) in native amp so that I can decorate the loss function such as this 5.