Hi Katherine!



 crowsonkb:

Even better—if I use the Adam optimizer with its default learning rate of 1e-3, I get full convergence within 30 epochs:
Epoch: 29, Loss: 4.0959913412408344e-12
Final a: 0.5829995274543762, b: 0.11200009286403656



Thanks.  Very nice.


It looks like it really benefits from having per-parameter adaptive learning rates.


Especially in this case where the optimizations for the a and b
parameters separate entirely.
Best.
K. Frank