For efficiency reasons, tensors are not allocated individually but in blocks.
Non-releasable memory is the empty part of a memory block that also has some parts of it allocated.
Obviously frames_batches and pred are still on the GPU. Also, using gc.collect() before emtpy_cache helps Python not hold on to Tensors that are not needed anymore.
Best regards
Thomas