Can the Max_LR value for CycleLR be less than the lr that is used for optim.Adam ?
I used the values below (lr 0.002 and max_lr 0.0005 (I did that by mistake) and that seemed to be giving better results than Max_LR of 0.005 )
Max_LR of 0.005 did not seem to give better result
optimizer = optim.Adam(list(net.parameters()),lr=0.002)
scheduler = torch.optim.lr_scheduler.OneCycleLR(optimizer, max_lr=0.0005, steps_per_epoch=len(dataloaders[‘train’]), epochs=total_epocs_var)