Yeah. My RNN model uses RMS prop and my dqn model uses adamax.
Earlier I declared:
self.rnn_optimizer = optim.RMSprop(self.rnn_model.parameters())
self.optimizer = optim.Adamax(self.qnet.parameters())