Thank you. The reason I asked this question is because I am trying to train a model which is very demanding in terms of GPU space. Now if I create the optimizer after moving the model to GPU then it is taking more space in GPU and sometime my GPU is overflowing. But by creating the optimizer before moving model to GPU I am able to save some space in GPU.