SingleShockControl:

Or always keep half of my GPU memory available for potential “error space” 


No, you don’t have to, because even though it looks used when looking from the OS point of view, it is actually available to allocate more Tensors.
You can consider this a bad case scenario for the allocator that doesn’t realize you’re doing the same thing and so is caching more memory (even though it has a lot of available memory already).