Internally empty_cache is called if an out of memory error was detected as an attempt to recover from it. Also, if you are using cudnn with benchmark=True, different algorithms will be profiled in the first iteration, which use a different amount of memory for their workspaces. To recover this memory and allow other processes to use it, empty_cache will also be used.