Hi,
In Deep-Q-Networks we compute the loss as:
loss = Q(s,a) - (r + gamma * maxQ'(s', a'))

So in order to calculate values of Q(s,a) and maxQ’(s’,a’), we need to do to a forward pass on the model two times. If I understood correctly this would create two different computation graphs for each forward pass.
So my question is whether I “have to” detach() the resulting value of maxQ’(s’,a’) before doing the backward pass. Does it lead to errors if I don’t, and why?