I just noticed that I had a zombie process taking up some memory, so this may have been the culprit. After killing the zombie process, my gpu memory is down to  4228MiB / 12189MiB. Hopefully this was the issue.