I don’t think the problem come from the data loader, by the way, you don’t need to set requires_grad to the loaded data. In most case you only need requires_grad on model’s parameters, which have by default requieres_grad set to True.
Simple example to illustrate:
dummy_input = torch.randn(1, 1, 5)
print('input requires_grad ?', dummy_input.requires_grad)
simple_model = torch.nn.Conv1d(1, 1, 3)
print('model param requires_grad ?', simple_model.weight.requires_grad) # weight attribute of conv is a parameter
model_output = simple_model(dummy_input)
print('output requires_grad ?', model_output.requires_grad)
random_target = torch.randn(1, 1, 3)
loss = ((model_output - random_target)**2).sum()
print('loss requires_grad ?', loss.requires_grad)

# Return :
# > input requires_grad ? False
# > model param requires_grad ? True
# > output requires_grad ? True
# > loss requires_grad ? True

In fact, when runing an operation on two tensors, usually if at least one input needs gradients, the output will also requires the gradients. So If your model has parameters with requieres_grad = True, your output should automatically get requieres_grad = True (unless you explicitly froze the model or use torch.no_grad context).
I do not know what could have gone wrong if you changed your code. But you should check some points:

How do you initialize your optimizer? It should be something like optimizer = optim.Adam(net_to_train.parameters(), lr=0.0001)

Set your input data with image = torch.tensor(image) without the requires_grad argument, and then check if your model output has requires_grad = True.
Check that at every step from the model’s output to the loss if the attribute requires_grad goes to false. Few operations on tensors are not differentiable and you cannot backward on them (if an operation is not differentiable, it will be stated in the doc), for instance indexing is not differentiable with respect to the input indices.

If you do not find where the problem is, please show me your current training loop and loss computation, so I can get what could go wrong.