If you can write that loss function in terms of already-implemented pytorch operators, then it will be automatically differentiable. If it’s the case, you can just write a nn.Module as if it were a new layer.
If you need to create a new operator you will have to code how the gradient is computed.
Anyway, several post about this topic has been written. Look for them.