LBFGS Optimizer:

  
      github.com
  
  
    pytorch/pytorch/blob/6be85603757a810be3d18b70279b104c0653e9b3/torch/optim/lbfgs.py#L183 2

        if abs(bracket[1] - bracket[0]) * d_norm < tolerance_change:
            break


    # return stuff
    t = bracket[low_pos]
    f_new = bracket_f[low_pos]
    g_new = bracket_g[low_pos]
    return f_new, g_new, t, ls_func_evals




class LBFGS(Optimizer):
    """Implements L-BFGS algorithm, heavily inspired by `minFunc
    <https://www.cs.ubc.ca/~schmidtm/Software/minFunc.html>`.


    .. warning::
        This optimizer doesn't support per-parameter options and parameter
        groups (there can be only one).


    .. warning::
        Right now all parameters have to be on a single device. This will be
        improved in the future.



  
  
    
    
  
  


I was wondering if this optimizer is supposed to produce non-deterministic results? Given that these results were produced under the same parameter initialization and optimization setting. Simply running the program multiple times without any changes would produce different results.
I have set the following to attempt disabling the non-deterministic outcome but no luck:
np.random.seed(0)
torch.manual_seed(0)
torch.backends.cudnn.deterministic = True
torch.backends.cudnn.benchmark = False