I intend during the train compute the loss using MSELoss but using max error as reference for stoping to train and to compute the error.
Note that I set reduce=False, so the loss_func will returns a loss per input/target element. How I can use an optimizer that I can use max error to compute?
Error:
/Envs/climaenv/lib/python3.6/site-packages/torch/optim/lbfgs.py in step(self, closure)
    102         # evaluate initial f(x) and df/dx
    103         orig_loss = closure()
--> 104         loss = float(orig_loss)
    105         current_evals = 1
    106         state['func_evals'] += 1

ValueError: only one element tensors can be converted to Python scalars

Here is the code:

    def fit(self, x=None, y=None, lr=0.001, epochs=1000):
        '''
         Training function
         Argurmets:
            x: features train set
            y: target train set
            lr: Learning rate
            epochs: number of epochs 
        '''
        optimizer = torch.optim.LBFGS([{'params': [self.hidden.weight,self.predict.weight]}], 
                                      lr=lr,max_iter=20)
        
        loss_func = torch.nn.MSELoss(reduce=False) 
    
        def closure():
            optimizer.zero_grad()                  # clear gradients for next train
            prediction = self(x)                   # input x and predict based on x
            loss = loss_func(prediction, y) 
            loss.max().backward()                  # backpropagation, compute gradients 
            return loss
        
        for t in range(epochs):  
            optimizer.step(closure)                # apply gradient
            
        return self.loss_epochs