@J_Johnson Thanks for your reply. But the fact in my case is that the output tensors are too large to be kept in GPU. And I have to trade some outputs out of GPUs to preserve space for other batches’ computation. BTW, my case is that I should run all batches forward, and then run them backward, so GPU memory is key constraint.
Do you have any suggestions for this case?