Try this:
self.value_optimizer.zero_grad()
preValue = self.value_net(state)
nextValue = self.value_net(next_state).detach() # don't backprop this way
expectedValue = (self.gamma * nextValue) + reward
loss = F.smooth_l1_loss(preValue, expectedValue)
loss.backward()
self.value_optimizer.step()