Hi All!
I wanted to use a CosineAnnealingWithRestarts schedulers on Google Cloud TPU cores, with distributed parallel training.
I couldn’t find a tutorial which shows how to do that.
Please help me out.