Hello everyone.
I am writting a single layer perceptron for multidimensional function fitting.
The point is that during the training process I would like to change the optimizer once a certain error is reached.
In this moment my code looks like this:
optthres = 1e-4 #Optimization method threshold.
    # Define optimizer.
    opt1 = torch.optim.Rprop(model.parameters())
    opt2 = torch.optim.LBFGS(model.parameters())

    if (loss.data[0] > optthres):
        opt1.zero_grad()
        loss.backward()
        opt1.step()
    else:
        def closure():
            opt2.zero_grad()
            y_pred = model(x)
            loss = loss_fn(y_pred, y)
            loss.backward()
            return loss
        opt2.step(closure)

Is there any other way to define different optimizers according to the error of the training process?
Thank you very much!