As far as my understanding goes, a loss function basically behaves just as a layer does. Therefore, a custom loss function should be implementable by following Peter’s guide on how to write a custom layer https://pytorch.org/tutorials/advanced/cpp_extension.html 62
It is necessary that you define the forward as well as the backward pass of that loss function, which means that you need to find the gradient of your custom loss function for the backward pass. However, I am not entirely sure if that is true. Can someone copy that?