You are right, the non-releasable memory indeed can be reused by pytorch, but it is not shared with the released memory. When the new defined tensor is large, the non-releasable memory can not be applied with released memory simultaneously.