torch.cuda.empty_cache() will only clear the PyTorch memory cache on the device.
I meant you should check via nvidia-smi, if other processes are using the GPU.
Also, if a batch size of 1 doesn’t fit on the GPU, you might need to use torch.utils.checkpoint to trade compute for memory.
Although it would be surprising to see a FastAI lecture code would need a bigger GPU.