select, narrow and indexing operations (except when using a LongTensor index) return views onto the same memory. However, the transforms operate out-of-place, so afterwards, you’re going to get a copy. Again, cating views also needs to allocate a new tensor for the output. Of course the outputs of __getindex__ should be out of scope a moment later, so the memory pressure stays constant.
Is it the CPU memory that’s blowing up or the GPU? Are you caching anything inside the model? Did you try adding gc.collect() calls inside the loop (not recommended, but if that helps it means that you have reference cycles somewhere)? deling the Variables shouldn’t be necessary.