Albert65:

If I understand it correctly, as soon as a loss function can be expressed as a chain of linear algebra operations, the automatic differentiation can be applied


Yes, that’s right. So, in that respect the code is correct.
So, there is no learnable parameters here, maybe that’s why the loss does not change. What is the model trying to optimize here? Does x comes from output of another network or it is just the input? I assume y is the target, right?