Finally! The solution is to put in the replay buffer only data from Tensors obs/action/etc. Because they participate in backward pass for value_loss and in the next iteration they are not in the graph any more.
In my case such a problem was caused by actions Tensor, which has required_grad=True and was missed on the next iteration of backward. Thanks to albanD!