The fastest thing to do is to write a new loss function in Pytorch. This can either be done through writing an autograd Function in python, or just defining some python function that does the math you want it to.
For example,
def mse_loss(input, target):
    return (input - target) ** 2

would give you some mse loss.