Solved by ptrblck in post #6 
              
              
                float16 can easily overflow if you are using values with a value close to the min. and max. values: 
torch.finfo(torch.float16).max
> 65504.0

E.g. this code snippet overflows in the second approach and yields Infs in the result after applying torch.mm: 
x = torch.randn(1024, 1024, device='cuda').ha…