Hi! I want to transform the codes below implemented with TensorFlow into a PyTorch version:
lr = tf.train.exponential_decay(start_lr, global_step, 3000, 0.96, staircase=True)
optimizer = tf.train.AdamOptimizer(learning_rate=lr, epsilon=0.1)

But I don’t know what’s the counterpart of PyTorch of exponential learning rate decay. Anyone who can tell me? Thanks a lot!