I had a memory leak in a similar situation (to your first instance), making sarsd tensors from numpy arrays for rl. The solution for my instance was changing the dtype in a tensor constructor. I can’t offer a principled solution but you could try changing the constructor for terminal to:
 torch.as_tensor(terminal, dtype=torch.bool).unsqueeze(1).to(device=self.device)

and estimated q_value can be made like:
estimated_q_value = reward + self.gamma * (~terminal) * estimated_value

Boolean dtype should be preferred in this case right?