I am trying CIFAR10 classification using an Alexnet type network and a multistepLR schedule, which reduces lr at epochs = 100, 150 and wd=1e-2. The training runs smoothly when the initial lr = 1e-3, but when I am using initial lr= 4e-3, I get random accuracy (~10%) with pytorch seed = 0, and around the expected accuracy (~85%) when I set the seed = 2. Is there any possible explanation for this behaviour, or is this solely due to a bug in my code? I am not posting any code for now as it is distributed among many files. Also, I am using kaiming initialization for the weights.