I noticed that during testing the grad would still be true for the tensors we got from the models as outputs. Would using detach() reduce the memory usage, or give any benefit?
The code during testing can be like:
my_model.eval()
with torch.no_grad():
       for batch_idx, batch in enumerate(test_dataloader):            
           imgs =get_imgs(batch)                      
           output  =  my_model(imgs).detach()

Any ideas?