Are you sure the backward operation on loss_temp isn’t raising an error?
Since you are currently using numpy operations and are only wrapping the results to tensors, no computation graph would be created.
Could you post an executable code snippet showing the backward call as well?