I put this in my loss function and when I try to train my model with this, the weights become NaN after a few iterations. However, when I remove  torch.bmm(x, y_t) , the model is able to train. Does anyone know what in  torch.bmm()  can cause this issue to occur?
At first I thought maybe I had to normalize my inputs  x,y  but that did not make a difference. I also tried using a much lower learning rate but it does not make a difference.