Hi,
The problem most likely lies with the torch.eye(rows, cols) that you use.
Keep in mind that the backward pass computes a vector jacobian product. So you need to make sure that the vector you backprop matches this identity matrix here.