Actually,
def weighted_mse_loss(input, target, weight):
        return (weight * (input - target) ** 2).mean()

is slightly better, since the sum is some big. Taking the average is exactly the original way that nn.MSELoss does.