I am facing a similar issue:
CUDA out of memory. Tried to allocate 196.50 MiB (GPU 0; 15.75 GiB total capacity; 7.09 GiB already allocated; 20.62 MiB free; 72.48 MiB cached)
It looks like there is enough memory left, yet I get an OOM error. Do we have any update on this ?
Thanks!