This looks like the exact scenario I outlined above. The (10.72-8.86)=1.86 can on be released back to CUDA (and potentially joined with the 0.633 in a new allocation to meet your requirement) if the blocks contained within that are not already partially allocated, i.e. they have not been split and some part is currently allocated to a tensor. If that has happened (which it looks like it has otherwise you wouldn’t see an error) then the only way to recover the 1.86 is to delete the tensor(s) which are allocated to the now fragmented block. Then when you run torch.cuda.empty_cache() the block will be released and rejoin the main CUDA pool, showing up as “free” and you should be able to allocate a new block to hold your new 1.93 tensor.