Hello,
While browsing for half precision or mixed precision training in pytorch, I found these tools https://nvidia.github.io/apex/index.html 1 and particularly the apex.amp that should allow for a rather straight-forward switch between different FP for training.
However, when I run existing FP32 codes to mixed precision or FP16 I get the following:
RuntimeError: arange_out not supported on CPUType for Half
which seems to be caused by torch.stft that I use for computing spectral reconstruction losses.
… Any way to use stft with FP16 please ? (I use pytorch 1.1.0 and cuda 9.1.85)
Thanks ! And if anyone can recommend some other tools/ressources for mixed precision training in pytorch, I would be very thankful for more references as I did not get good results with apex.amp