Solved by albanD in post #4 
              
              
                It won’t break anything, the things that can happen are: 

you will compute gradients for some Tensors and don’t use it. Slowing down your code slightly for no reason.
If your label is of a non-contiguous dtype (like long), it will raise an error if you try to set requires_grad=True. Because that is…