Hi,
The torch.optim has a wide an rich selection of both common and more exotique optimizers. I am wondering which optimizers to pick for an object segmentation project.
I was reading through this paper: S. Ruder - An overview of gradient descent optimization algorithms.

  
      
      arXiv.org
  
  
    

An overview of gradient descent optimization algorithms 13

Gradient descent optimization algorithms, while increasingly popular, are
often used as black-box optimizers, as practical explanations of their
strengths and weaknesses are hard to come by. This article aims to provide the
reader with intuitions...


  
  
    
    
  
  


And I was thinking if something similar has been published with updated guidelines on more modern optimizers like AdamW, SparseAdam, etc.
In other words, which optimizer should I take?
Thank you for your time