Thank you!
I Have one more question:
As I understand these two blocks of code are the same.
critic = Critic()
critic_old = critic(Variable(torch.Tensor(observation).view(1,4)))
critic_new = critic(Variable(torch.Tensor(observation).view(1,4)))
critic_loss = ((reward+(gamma*critic_new.detach())) - critic_old)**2

.
critic = Critic()
critic_old = critic(Variable(torch.Tensor(observation).view(1,4))) critic_new = 
critic(Variable(torch.Tensor(observation).view(1,4), volatile = True))
critic_new.volatile = False
critic_loss = ((reward+(gamma*critic_new)) - critic_old)**2

Am I correct?