That’s interesting (and terrifying that I didn’t notice this major change before).
So I can just get my input tensors from my data loader and just apply .requires_grad() on them? For example, in a training session, should we just do the following?
for i, x in enumerate(data_loader):
    out = model(x.requires_grad())

And in the case where we need to upload data on GPU, we just apply .cuda() after .requires_grad()?