Can u try deleting both the inputs, outputs, and predictions, then use torch.cuda.empty_cache()?
A way to make sure that no useless tensors are stored is to declare the variables in the minimum scope possible and check all the global variables.