Solved by Epoching in post #3 
              
              
                Ah it’s interesting how you make the learning rate scheduler first in TensorFlow, then pass it into your optimizer. 
In PyTorch, we first make the optimizer: 
my_model = torchvision.models.resnet50()

my_optim = torch.optim.Adam(params=my_model.params, lr=0.001, betas=(0.9, 0.999), eps=1e-08, weight…