For some weird reason if I set requires_grad outside of the loss_function, the backward works while setting the requires_grad inside loss function fails completely and not even a single iteration happens! This however wont happen if I doloss1.backward()! at all!