Hi @Tim_Zhang – are you using torch.nn.DataParallel for training on multiple GPUs? If so, this could be some sort of initialization bug where cuFFT is initialized on the first device only and not others.