You could delete all tensors, parameters, models etc. and call empty_cache() afterwards to remove all allocations created by PyTorch. To also remove the CUDA context, you would have to shut down the Python session.