Thank you. The problem is that I am using a shared workstation and so other students can run their code and also take up any remaining memory. So I am worried that the small fluctuations may also cause an oom.
For example, the GPU is free at 18MiB / 11178MiB before I run my model. Then it goes up to 10873MiB / 11178MiB when I start. During this time any other student can run their code provided it does not exceed the remaining memory. So if the GPU is now fully utilized then any fluctutations from my code will result in an oom.