Note that this approach will free the GPU memory indeed, so that other applications can use it.
However, if you want to use the memory again in PyTorch, it would have to be reallocated, which might slow down your code. To avoid it, PyTorch uses a custom caching mechanism, so that the cached GPU memory can easily be reused without any allocations.