This new fancy optimizer requires no learning rate at all and seems to perform quite well compared to previous optimizers.
See the paper for details.
Paper: “Backprop without Learning Rates Through Coin Betting”, Orabona & Tommasi 2017
The original implementation:

  
      
      GitHub
  
  
    

bremen79/cocob 47

TensorFlow implementation of COCOB. Contribute to bremen79/cocob development by creating an account on GitHub.