I got

RuntimeError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 10.76 GiB total capacity; 9.76 GiB already allocated; 21.12 MiB free; 9.88 GiB reserved in total by PyTorch)

I know that my GPU has a total memory of at least 10.76 GB, yet PyTorch is only reserving 9.88 GB.
In another instance, I got a CUDA out of memory error in middle of an epoch and it said “5.50 GiB reserved in total by PyTorch”
Why is it not making use of the available memory? How do I change this amount of memory reserved?
Using PyTorch 1.4.0 and cuda 10.1