@Muhammad_Maaz Are you using DDP here for multiple GPUs? If so Distributed: RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [2048]] is at version 4; expected version 3 instead · Issue #62474 · pytorch/pytorch · GitHub 2 might be related.