Unfortunately it’s not possible. When you allocate memory, you ask for a “block” of it of a given size. Then you can free this “block” but you cannot free only part of it. So it’s not possible to do this 
If this is not performance critical, you can use the cpu to do this: send the slice to the cpu, delete the big gpu tensor and then send back the slice into the gpu.